-
Notifications
You must be signed in to change notification settings - Fork 397
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Regression in using NVCC as a CUDA compiler: redzone_allocator_kernel_cuda compilation fails #13460
Comments
Do we even support nvcc as a CUDA compiler? |
also CC @ddunl |
FWIW, NVCC is the default cuda compiler in configure.py. |
Sorry for the delay - I was out. I will look into this. Interestingly we don't see this on the CI. Which version of CUDA are you using? |
12.1.0 |
It used to be a `gpu_kernel_library` which means it gets compiled for device using NVCC in OSS. But the sources files don't contain any CUDA code, so there is no need for that. #13460 reports that this behaviour even fails with some versions of NVCC because NVCC can't deal with C++ template magic being pulled in by Abseil. PiperOrigin-RevId: 644658241
It used to be a `gpu_kernel_library` which means it gets compiled for device using NVCC in OSS. But the sources files don't contain any CUDA code, so there is no need for that. openxla/xla#13460 reports that this behaviour even fails with some versions of NVCC because NVCC can't deal with C++ template magic being pulled in by Abseil. Reverts c0e79da PiperOrigin-RevId: 644658241
It used to be a `gpu_kernel_library` which means it gets compiled for device using NVCC in OSS. But the sources files don't contain any CUDA code, so there is no need for that. #13460 reports that this behaviour even fails with some versions of NVCC because NVCC can't deal with C++ template magic being pulled in by Abseil. PiperOrigin-RevId: 644658241
It used to be a `gpu_kernel_library` which means it gets compiled for device using NVCC in OSS. But the sources files don't contain any CUDA code, so there is no need for that. openxla/xla#13460 reports that this behaviour even fails with some versions of NVCC because NVCC can't deal with C++ template magic being pulled in by Abseil. PiperOrigin-RevId: 644658241
It used to be a `gpu_kernel_library` which means it gets compiled for device using NVCC in OSS. But the sources files don't contain any CUDA code, so there is no need for that. #13460 reports that this behaviour even fails with some versions of NVCC because NVCC can't deal with C++ template magic being pulled in by Abseil. PiperOrigin-RevId: 644658241
It used to be a `gpu_kernel_library` which means it gets compiled for device using NVCC in OSS. But the sources files don't contain any CUDA code, so there is no need for that. #13460 reports that this behaviour even fails with some versions of NVCC because NVCC can't deal with C++ template magic being pulled in by Abseil. PiperOrigin-RevId: 644658241
It used to be a `gpu_kernel_library` which means it gets compiled for device using NVCC in OSS. But the sources files don't contain any CUDA code, so there is no need for that. openxla/xla#13460 reports that this behaviour even fails with some versions of NVCC because NVCC can't deal with C++ template magic being pulled in by Abseil. PiperOrigin-RevId: 644658241
It used to be a `gpu_kernel_library` which means it gets compiled for device using NVCC in OSS. But the sources files don't contain any CUDA code, so there is no need for that. #13460 reports that this behaviour even fails with some versions of NVCC because NVCC can't deal with C++ template magic being pulled in by Abseil. PiperOrigin-RevId: 644685844
It used to be a `gpu_kernel_library` which means it gets compiled for device using NVCC in OSS. But the sources files don't contain any CUDA code, so there is no need for that. openxla/xla#13460 reports that this behaviour even fails with some versions of NVCC because NVCC can't deal with C++ template magic being pulled in by Abseil. PiperOrigin-RevId: 644685844
Hey @pearu, I've pushed a (potential) fix. Would you be able to conform whether it actually fixes your issue? |
IIUC, the (potential) fix is equivalent to applying a patch containing:
When I apply this to my local branch (I can try main later if needed), there is a progress:
Just for the sake of experiment, making
HTH |
Ok cool. That seems to be the same issue, but in a different file. I can fix that tomorrow. |
Hey, I won't be getting to this today. As a workaround you could use a later version of the CUDA toolkit. We use CUDA 12.3 in the CI, so I believe 12.3+ shouldn't have the bug anymore. Or - as you already discovered - you can use Clang as the CUDA compiler instead. |
Yes, it should be changed to Clang since that's the only thing we test in CI. |
I confirm that when using CUDA 12.3.2, this issue cannot be reproduced: bazel build is successful without applying the above mentioned patches. |
After commit d8f0c1a, building XLA fails for CUDA backend. Reproducer:
With
the XLA build is succesful.
Using:
CC: @beckerhe
The text was updated successfully, but these errors were encountered: