FileNotFoundError: data path not found: data/conll2003/resource/label.vocab

[ScottLiao920]

Cannot find this file. Also, what's the usage of run_embed.py?

[stevezheng23]

@ScottLiao920 this file and the train/dev/test data should be prepared for each NER task.

For CoNLL2003 NER task, you can use the followings,
<pad> O X <cls> <sep> B-LOC B-MISC B-ORG B-PER I-LOC I-MISC I-ORG I-PER

[stevezheng23]

For run_embed, it's used to export saved model for sent/token-level embedding generation

[ScottLiao920]

Does this mean that for CoNLL2003 NER task, each line of label.vocab corresponds to each line of another training data file?

[stevezheng23]

No, label.vocab is only for CoNLL2003 NER tag list, <pad> O X <cls> <sep> B-LOC B-MISC B-ORG B-PER I-LOC I-MISC I-ORG I-PER. It doesn't correspond to lines in training data file.

[stevezheng23]

Here are the screenshots for vocab file and data file,
label.vocab:

train-conll2003.json:

[stevezheng23]

The data file format is a little bit different from CoNLL format, you can easily convert CoNLL format data to json format data using the following command, where input file is in CoNLL format and output file is in json format
python prepro/prepro_conll.py --data_format json --input_file data/ner/conll2003/raw/eng.xxx --output_file data/ner/conll2003/xxx-conll2003/xxx-conll2003.json

[ScottLiao920]

solved already. Thanks!

[stevezheng23]

@ScottLiao920 Great!