You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Thanks for your excellent work.I am confused about the target input_ids mentioned above /eagle/train/main.py .It seems that the target input_ids are sourced from dataset instead of the target LLM.
But I find that function getkacc() calculates the draft token acc with the hidden_states of target input_ids directly. For example ,
the dataset input_ids is "Deep learning is a powerful approach to machine learning" ,
the orginal LLM output is " Deep learning is a subfield of machine learning" .
so the target input_ids in dataset is calculated as follows:
"Deep learning is a powerful approach to machine learning" -> "learning is a subfield approach to machine learning"(target input_ids).
It should be noted that the next token of "Deep learning is a powerful" in original LLM is "approach", hypothetically.
the EAGLE draft prediction is as follows in getkacc() :
"Deep learning is a"-> "subfield"
"Deep learning is a subfield"-> "approach"
In fact , the real EAGLE inference in original LLM is as follows:
"Deep learning is a"-> "subfield"
"Deep learning is a subfield"-> "of"
I would like to know whether there's an issue with the getkacc() function calculation or if my understanding is flawed.
The text was updated successfully, but these errors were encountered:
Thanks for your excellent work.I am confused about the target input_ids mentioned above /eagle/train/main.py .It seems that the target input_ids are sourced from dataset instead of the target LLM.
But I find that function getkacc() calculates the draft token acc with the hidden_states of target input_ids directly. For example ,
the dataset input_ids is "Deep learning is a powerful approach to machine learning" ,
the orginal LLM output is " Deep learning is a subfield of machine learning" .
so the target input_ids in dataset is calculated as follows:
"Deep learning is a powerful approach to machine learning" -> "learning is a subfield approach to machine learning"(target input_ids).
It should be noted that the next token of "Deep learning is a powerful" in original LLM is "approach", hypothetically.
the EAGLE draft prediction is as follows in getkacc() :
![image](https://private-user-images.githubusercontent.com/86292192/328579790-59782803-0a70-4a51-b132-3261b52614e2.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjU3NTYyNzIsIm5iZiI6MTcyNTc1NTk3MiwicGF0aCI6Ii84NjI5MjE5Mi8zMjg1Nzk3OTAtNTk3ODI4MDMtMGE3MC00YTUxLWIxMzItMzI2MWI1MjYxNGUyLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA5MDglMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwOTA4VDAwMzkzMlomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTk0ODlmOTUxOWU3ZmY5NjQ1YzM2NjgzNzM0ZTlmM2IxMGM4OWI4OGI5ZTRhNTEwNTM4Y2M4ZjMyZTBmMzQyYzEmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.XGSMz4jgb9XJMsSDFeC9oiPFVI0hqpU7dhVKHCZqr7M)
"Deep learning is a"-> "subfield"
"Deep learning is a subfield"-> "approach"
In fact , the real EAGLE inference in original LLM is as follows:
"Deep learning is a"-> "subfield"
"Deep learning is a subfield"-> "of"
I would like to know whether there's an issue with the getkacc() function calculation or if my understanding is flawed.
The text was updated successfully, but these errors were encountered: