ImportError: DLL load failed while importing tensorrt

[Plasma4004]

CPU- Intel i9-13900H
GPU-NVIDIA 4060Laptop
CUDA-12.4
CUDNN-8.9.7
SYSTEM-Win11

I thought it was a tensorrt installation error, so I used 'setup_build_env.ps1'. But it still reported an error, and installed it again according to the 'https://docs.nvidia.com/deeplearning/tensorrt/install-guide/index.html#installing-zip', and it was not solved.What should I do?

PS C:\Users\Plasm> python
Python 3.10.11 (tags/v3.10.11:7d4cc5a, Apr 5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.

import tensorrt
Traceback (most recent call last):
File "", line 1, in
File "C:\Users\Plasm\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\tensorrt_init_.py", line 71, in
from .tensorrt import *
ImportError: DLL load failed while importing tensorrt: 找不到指定的模块。（means ‘The specified module could not be found’）

[tp5uiuc]

Hi @Plasma4004

Thanks for reporting the bug. I can repro on python 3.11 with tensorrt==9.3.0.post12.dev1 wheel on windows. I have root-caused the issue to missing cuDNN libs. Here are my findings

1. import tensorrt_llm works OOTB because TRT-LLM first imports torch (which loads up cuDNN libs) and then imports tensorrt (which can find these cuDNN libs) loaded by torch.
2. import tensorrt as trt does NOT work because tensorrt 9.3.0 requires cuDNN libs. Now cuDNN does get installed along with TRT but these are nvidia-cublas-cu12 libs

PS C:\Users\tejaswinp\workspace\turtle_runs> python3 -m pip show tensorrt_libs
Name: tensorrt-libs
Version: 9.3.0.post12.dev1
Summary: TensorRT Libraries
Home-page: https://developer.nvidia.com/tensorrt
Author: NVIDIA Corporation
Author-email:
License: Proprietary
Location: c:\users\tejaswinp\.pyenv\pyenv-win\versions\3.10.4\lib\site-packages
Requires: nvidia-cublas-cu12, nvidia-cuda-runtime-cu12, nvidia-cudnn-cu12
Required-by:

I suspected that these libs were not getting loaded, so I tried installing cudnn-12 libs from another version using
python3 -m pip uninstall nvidia-cudnn-cu12 and python3 -m pip install nvidia-cudnn-cu12==8.9.4.25 and after that import TRT works

>>> import tensorrt as trt
>>> trt.__version__
'9.3.0.post12.dev1'

Also note that if you are getting strange error messages like #1062, you need to first import torch followed by tensorrt.

I don't know why this issue surfaced now, but it may be better to report this in the TRT package repo. Let me consult with packaging experts and get back to you. For now the workaround is to do

# Remove older version of TRT
pip3 uninstall -y tensorrt nvidia-cudnn-cu12
pip3 install nvidia-cudnn-cu12==8.9.4.25
pip3 install tensorrt==9.3.0.post12.dev1 --extra-index-url https://pypi.nvidia.com/

[tp5uiuc]

Seems like this is a known issue and we missed it in the windows packaging requirements. For now, please use the workaround mentioned above : we will fix this soon. Thanks!

[Plasma4004]

CUDA:12.4
CUDNN:9.1.0(I've updated)
(But there's a question I'm not sure about, what is the relationship between the cudnn manually added to CUDA and the cudnn downloaded by pip)(It's not clear to me if pip's cudnn is compatible with my cudnn 9.1.0)

I have followed the steps you gave.

PS C:\Users\Plasm> python -c "import tensorrt as trt; print(trt.version)"
9.3.0.post12.dev1

It seems that tensorrt has been recognized normally.But when checking tensorrt-llm,it's still an error.

PS C:\Users\Plasm> python -c "import tensorrt_llm; print(tensorrt_llm.utils.trt_version())"
Traceback (most recent call last):
File "", line 1, in
File "C:\Users\Plasm\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\tensorrt_llm_init.py", line 28, in
add_trt_llm_dll_directory()
File "C:\Users\Plasm\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\tensorrt_llm_init.py", line 24, in _add_trt_llm_dll_directory
os.add_dll_directory(
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\os.py", line 1118, in add_dll_directory
cookie = nt._add_dll_directory(path)
FileNotFoundError: [WinError 3] 系统找不到指定的路径。: 'C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\Lib\site-packages\tensorrt_llm\libs'

It doesn't seem to be able to find Tensorrt-llm.
However,

PS C:\Users\Plasm> pip install tensorrt_llm --extra-index-url https://pypi.nvidia.com --extra-index-url https://download.pytorch.org/whl/cu121
...
Successfully installed tensorrt_llm-0.9.0

[Plasma4004]

And after the operation

pip3 install tensorrt==9.3.0.post12.dev1 --extra-index-url https://pypi.nvidia.com/

There is a conflict in the pip package.

PS C:\Users\Plasm> pip check
torchdata 0.7.0 has requirement torch==2.1.0, but you have torch 2.2.0+cu121.
torchtext 0.16.0+cpu has requirement torch==2.1.0, but you have torch 2.2.0+cu121.
torchvision 0.16.0+cu121 has requirement torch==2.1.0+cu121, but you have torch 2.2.0+cu121.

How do I make sure that torch doesn't conflict with 3 packages? Update 3 packages to a certain version?

[tp5uiuc]

@Plasma4004 : Good to know that the import issue has been resolved. I am answering other questions inline.

PS C:\Users\Plasm> python -c "import tensorrt_llm; print(tensorrt_llm.utils.trt_version())"
Traceback (most recent call last):
File "", line 1, in
File "C:\Users\Plasm\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\tensorrt_llm_init.py", line 28, in
add_trt_llm_dll_directory()
File "C:\Users\Plasm\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\tensorrt_llm_init.py", line 24, in _add_trt_llm_dll_directory
os.add_dll_directory(
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\os.py", line 1118, in add_dll_directory
cookie = nt._add_dll_directory(path)
FileNotFoundError: [WinError 3] 系统找不到指定的路径。: 'C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\Lib\site-packages\tensorrt_llm\libs'

There seems to be two versions of python site-packages in your local env:

1. C:\Users\Plasm\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages
2. C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\Lib\site-packages
   and this leads to confusion. trt-llm looks for the site-packages\tensorrt_llm\libs directory to load DLLs. So while location (1) is where you have installed trt-llm looks like (2) is the "official" site-packages directory. Can you let me know

1. if you are installing tensorrt-llm as a user (using pip install -u flag)
2. how you installed python in the first place?

[Plasma4004]

@tp5uiuc I seem to understand that my python is downloaded from the Microsoft Store, not the official python website.The catalog downloaded by the store is different from that downloaded by the official one.This may be the cause of the error.

[tp5uiuc]

Great, thanks for letting me know @Plasma4004 . I am closing the issue, but feel free to re-open in case you have more questions.