[BUG] The batch_manage.a choice error in --cpp-only when torch's cxx_abi version is different with gcc

[thisjiang]

Bug Report

The cpp benchmark will compile error when

1. torch is compiled with cxx11 abi = 0: check by python -c "import torch; print(torch.compiled_with_cxx11_abi(),end='');"
2. gcc default cxx11 abi = 1 : check by echo '#include <string>' | g++ -x c++ -E -dM - | fgrep _GLIBCXX_USE_CXX11_ABI
3. trt-llm was compile with --cpp-only

Bug Reason

The bug was caused by:

1. In cpp/tensorrt_llm/CMakeLists.txt#L63, the BATCH_MANAGER_TARGET is only choice by check torch's cxx11 abi version.
2. But if set --cpp-only, the cmake option BUILD_PYT will be False, that means, the code of add_compile_options(${TORCH_CXX_FLAGS}) in cpp/CMakeLists.txt#L284 will not run, in other word, the trt-llm will compile with the gcc's default cxx11 abi version.

In a word, the trt-llm is compile by gcc's default cxx11 abi version, but the batch_manage.a is choiced by pytorch's cxx11 abi version when trt-llm was compiled by --cpp-only.

[juney-nvidia]

@thisjiang thanks for reporting this. But sorry for that I didn't get the full point here.
When the batch_manager.a is packaged, we do taken the C++11 compatibility into consideration. If you have any issue with it, can you share the concrete steps of re-producing the issue in your environment? Also pls share your GPU device, and CUDA/OS version.

Thanks
June

[Shixiaowei02]

I think @thisjiang is referring to the fact that we don't check the type of abi under certain compilation options. We will fix it.

[Shixiaowei02]

The bug has been fixed in the main branch, thanks for the support. I will close this issue. If you have any questions, please open it and continue the discussion. Thanks!