Skip to content

Issues: NVIDIA/TensorRT-LLM

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

668 Open 1,031 Closed
668 Open 1,031 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Does TensorRT-LLM support passing input_embeds directly？
#2104 opened Aug 9, 2024 by Hukongtao
How to preprocess a int4 model?
#2103 opened Aug 9, 2024 by Septend-fun
How to add a new quantization method?
#2102 opened Aug 9, 2024 by Davids048
Finding protobuf files while benchmarking TensorRT-LLM bug Something isn't working
#2101 opened Aug 8, 2024 by KuntaiDu
2 of 4 tasks
1
Error trying to build the visual encoder for llava-v1.6-34b-hf using build_visual_engine.py bug Something isn't working
#2100 opened Aug 8, 2024 by alexemme
2 of 4 tasks
Add support for Nougat model bug Something isn't working
#2098 opened Aug 8, 2024 by sherlcok314159
2 of 4 tasks
Random generation when inferring llama 2 with and without LoRA in the same batch bug Something isn't working
#2096 opened Aug 7, 2024 by ilyaonoff
2 of 4 tasks
[0.11.0] Model Conversion OOMed in P4D.24xl bug Something isn't working
#2093 opened Aug 6, 2024 by lanking520
4 tasks
[Question] What is the difference between gptManagerBenchmark and gptSessionBenchmark in cpp benchmarks?
#2092 opened Aug 6, 2024 by hcnhcn012
questions on return-generation-logits when streaming
#2090 opened Aug 6, 2024 by binhtranmcs
Segmentation fault when building engine
#2089 opened Aug 6, 2024 by kevinsouthByteDance
return-generation-logits bug when fp8 enabled
#2088 opened Aug 6, 2024 by binhtranmcs
convert qwen2-0.5b-instruct failed when using smoothquant bug Something isn't working
#2087 opened Aug 5, 2024 by ReginaZh
4 tasks
InternVL2 Support ? bug Something isn't working
#2083 opened Aug 4, 2024 by Jeremy-J-J
4 tasks
When will cuda12.5 && tritonserver 24.07-py3 be supported?
#2082 opened Aug 3, 2024 by dafu-wu
performance: fp8 vs smoothquant int8
#2074 opened Aug 1, 2024 by enozhu
1
How to invoke the constructor interface of Executor"
#2073 opened Aug 1, 2024 by HongfengDu
-1 token id with Mixtral FP8 and tensorrt_llm 0.11.0
#2072 opened Jul 31, 2024 by v-dicicco
2
[Error] Static dimension mismatch while setting input shape. while running llama 3 8B bug Something isn't working
#2071 opened Jul 31, 2024 by manickavela29
2 of 4 tasks
[TensorRT-LLM][ERROR] Assertion failed: mModule != nullptr bug Something isn't working
#2069 opened Jul 31, 2024 by KeitaW
2 of 4 tasks
Which source file contains the code for loading the model? question Further information is requested
#2068 opened Jul 31, 2024 by HongfengDu
2
RuntimeError: Can't enable access between nodes 1 and 0
#2066 opened Jul 31, 2024 by EASTERNTIGER
1
Error while running any tensorrt-llm script, issue from cython bindings bug Something isn't working
#2062 opened Jul 31, 2024 by manickavela29
2 of 4 tasks
Support for Grace Hopper bug Something isn't working
#2060 opened Jul 31, 2024 by sraskar
4 tasks
Error: MPI worldSize is expected to be equal to tp*pp when participantIds are not specified bug Something isn't working
#2058 opened Jul 30, 2024 by ydm-amazon
1 of 4 tasks
@symphonylyh
4
ProTip! Adding no:label will show everything without a label.