fix import issue when running huggingface_lowresource.sh #4
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
RenShuhuai-Andy
requested changes
Nov 13, 2021
There was a problem hiding this comment.
Thanks very much for the PR, here is some feedback:
- There is no need to change the way of importing packages in taa/archive.py etc. since I have moved the
__main__function in thetaa/search.pyoutside (examples/reproduce_experiment.py). - The
__main__function intaa/search_augment_train.pymay also need to move to theexamplesfold, please check it. - If a dataset in
huggingface/datasetsdoesn't have the original val set, please split out 10% training samples for validation. - Other specific suggestions on revision are commented after each file, please check it.
Thanks again.
Comment on lines
+109
to
+113
| if C.get()['ir'] < 1 and C.get()['method'] != 'bt': | ||
| # rebalanced data | ||
| ir_index = np.where(labels == 0) | ||
| texts = np.append(texts, texts[ir_index].repeat(int(1 / C.get()['ir']) - 1)) | ||
| labels = np.append(labels, labels[ir_index].repeat(int(1 / C.get()['ir']) - 1)) |
Comment on lines
-48
to
+59
| transform_train.transforms.insert(0, Augmentation(default_policy())) | ||
| pass |
| [('tfidf_word_insert', 0.6572641245063084, 0.32120987775289295), | ||
| ('random_word_swap', 0.4009335761117499, 0.3015697007069029)]] | ||
|
|
||
|
|
||
| def default_policy(): |
There was a problem hiding this comment.
I have deleted the default policy since it is not been used anymore
| class_num = train_dataset.features['label'].num_classes | ||
| all_train_examples = get_examples(train_dataset, text_key) | ||
|
|
||
| train_examples, valid_examples = general_split(all_train_examples, test_size=test_size, train_size=1-test_size) |
There was a problem hiding this comment.
A judgment statement should be added: if the dataset originally has a validation set, there is no need to split the val set from the training set.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.