Skip to content

Issues: OpenGPTX/lm-evaluation-harness

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

29 Open 19 Closed
29 Open 19 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

MegatronLMClient expects cache usage
#106 opened Jan 22, 2024 by janEbert
Add MT-Bench-X to lm-evaluation-harness
#104 opened Dec 12, 2023 by lllAlexanderlll
@lllAlexanderlll
Inspect LAMBADA OpenAI fpr HF converted checkpoints
#103 opened Dec 12, 2023 by lllAlexanderlll
Add Batching to Megatron-LM server
#90 opened Aug 4, 2023 by KlaudiaTH
@KlaudiaTH
trust_remote_code is not passed to AutoTokenizer.from_pretrained
#87 opened Jun 20, 2023 by KlaudiaTH
Upstreaming contributions?
#81 opened Apr 30, 2023 by StellaAthena
1
pawsx_de, pawsx_en, x_stance, gnad10 error message for metrics
#76 opened Mar 27, 2023 by katrinklug
1
xquad, mlqa and mlsum tasks are not correctly implemented
#75 opened Mar 16, 2023 by KlaudiaTH
3
Add evaluations for anli_de
#71 opened Jan 26, 2023 by katrinklug
Add evaluations for miam
#67 opened Jan 16, 2023 by katrinklug
Add evaluations for germandpr
#65 opened Jan 16, 2023 by katrinklug
Add evaluation for fairlex
#64 opened Jan 16, 2023 by katrinklug
Add evaluations for lextreme
#63 opened Jan 16, 2023 by katrinklug
Add evaluation tasks with omp dataset
#62 opened Jan 16, 2023 by katrinklug
Add evaluation for QG from xglue benchmark
#60 opened Jan 16, 2023 by katrinklug
Add evaluation for germEval14
#46 opened Nov 10, 2022 by katrinklug
Evaluation for German LER task
#44 opened Nov 4, 2022 by malteos
@aishwaryaanegundi
XStannce Topic Classification task
#43 opened Nov 4, 2022 by malteos
@aishwaryaanegundi
1
Multi GPU Evaluation task
#42 opened Nov 3, 2022 by Lena-Jurkschat
Results for xnli_de,pawsx_de,xquad_de
#31 opened Aug 17, 2022 by malteos
Evaluation for XGLUE Task task
#27 opened Aug 16, 2022 by malteos
Evaluation for STSb Multi MT task task
#26 opened Aug 16, 2022 by malteos
GitHub actions: Fix out of disk space error
#25 opened Aug 16, 2022 by malteos
Evaluation for GermEval 2015: LexSub task
#21 opened Aug 16, 2022 by malteos
Evaluation for CoNLL-2003 task (NER) task
#20 opened Aug 16, 2022 by malteos
ProTip! Add no:assignee to see everything that’s not assigned.