[LLM][Experiment] Tokenization model using lexical patterns #303
dishmint
started this conversation in
Applications
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Training a tokenizer on lexical patterns. Does/Can this do anything useful?
Beta Was this translation helpful? Give feedback.
All reactions