You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I had modified the preprocess_function.And the used data is the sft data of the model.
The pictures are the results of the training and testing in training:
but I use Spec-Bench project to test on Chineses dataset, the Mean accepted tokens is only 2.5994782086414654, not good enough.
Anything else to pay attention, Can you give some advice?Thanks!
The text was updated successfully, but these errors were encountered:
2024WY
changed the title
I use Chinese dataset and a another custom 7B model to train eagle,but the performance in Chinese is not good enough
I use Chinese and English mixed dataset and a another custom 7B model to train eagle,but the performance in Chinese is not good enough
Jun 17, 2024
I noticed that your top-3 accuracy on the training set is only about 0.8, which is relatively low. What is your training accuracy on the English dataset? If it is close to the accuracy on the Chinese dataset, it could be that the structure or size of the draft model is not suitable. If the English accuracy is significantly higher than the Chinese accuracy, it is possible that your base model is not sufficiently trained on Chinese, and its features cannot effectively capture the semantic information of Chinese.
I had modified the preprocess_function.And the used data is the sft data of the model.
![train](https://private-user-images.githubusercontent.com/173039829/340230783-a218955c-3037-4bed-a7e7-60ad6d64bd77.jpg?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjU3NTYxNjcsIm5iZiI6MTcyNTc1NTg2NywicGF0aCI6Ii8xNzMwMzk4MjkvMzQwMjMwNzgzLWEyMTg5NTVjLTMwMzctNGJlZC1hN2U3LTYwYWQ2ZDY0YmQ3Ny5qcGc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwOTA4JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDkwOFQwMDM3NDdaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT05MDEzNWJkZTA3ZTUzNGFhMmM3NTQwNzFlMjg2NGMxNTU4Y2M1NmMwZTExYzY2MjJhZDlhZTdlM2IwM2E2MTc1JlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.3-auPf-S6cy1MN5GKSOpqLcDLOBfviaSh69z8uerb6g)
![test](https://private-user-images.githubusercontent.com/173039829/340227773-3c58fbe0-c8a1-46e0-8705-bed0dd4e9bab.jpg?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjU3NTYxNjcsIm5iZiI6MTcyNTc1NTg2NywicGF0aCI6Ii8xNzMwMzk4MjkvMzQwMjI3NzczLTNjNThmYmUwLWM4YTEtNDZlMC04NzA1LWJlZDBkZDRlOWJhYi5qcGc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwOTA4JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDkwOFQwMDM3NDdaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1iNjkzYjM3NzRlZWQwOWE2ZDVlYWE2YTdhYjRjY2NjM2NhOGRlZGZiYzA5NzRhN2QwMjNiZWI2MzE5YjRjMjVmJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.i4R4DNgL8P0xkJV7TsNspR79aVzyOc9ynUxjebldV7k)
The pictures are the results of the training and testing in training:
but I use Spec-Bench project to test on Chineses dataset, the Mean accepted tokens is only 2.5994782086414654, not good enough.
Anything else to pay attention, Can you give some advice?Thanks!
The text was updated successfully, but these errors were encountered: