typos and errata (last updated 2021/05/18)

[xiaoouwang]

• chapter 1, should be text generation

Finally, a third class of text classification is unconditional text generation, where natural language text is generated stochastically from a model. You can train models so that they can generate some random academic papers, Linux source code, or even some poems and play scripts. For example, Andrej Karpathy trained an RNN model form all works of Shakespeare and succeeded in generation pieces of text that look exactly like his work (http://realworldnlpbook.com/ch1.html#karpathy15):

• 4.2.3 typo swtich in the pseudocode

def update_gru(state, word):
    new_state = update_hidden(state, word)
 
    switch = get_switch(state, word)
 
    state = swtich * new_state + (1 – switch) * state
 
    return state

• chapter 5: micro and macro should be switched.

original text:

If these metrics are computed while ignoring entity types, it’s called a micro average. For example, the micro-averaged precision is the total number of true positives of all types divided by the total number of retrieved named entities regardless of the type. On the other hand, if these metrics are computed per entity type and then get averaged, it’s called a macro average. For example, if the precision for PER and GPE is 80% and 90%, respectively, its macro average is 85%. What AllenNLP computes in the following is the micro average.

• 5.6.1

The language detection in a previous chapter used RNN and character as input.

original text:

In the first half of this section, we are going to build an English language model and train it using a generic English corpus. Before we start, we note that the RNN language model we build in this chapter operates on characters, not on words or tokens. All the RNN models we’ve seen so far operate on words, which means the input to the RNN was always sequences of words. On the other hand, the RNN we are going to use in this section takes sequences of characters as the input.