Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update ROOT and Cling to LLVM18 #15696

Open
wants to merge 41 commits into
base: master
Choose a base branch
from

Conversation

devajithvs
Copy link
Contributor

@devajithvs devajithvs commented May 31, 2024

This Pull request:

Rebase

A few relevant LLVM commits that were referenced for the rebase:

Notes

  • Cling and Root commits for the rebase are kept separate for now (can squash similar commits together later).
  • Also added a temporary commit root-project/llvm-project@4066693 in LLVM to prevent building of mlgo-utils, which will not compile when all the test folders are removed. We can either keep it or add an exception to this file in the Root GitHub workflow to not remove the test folders under mlgo-utils.

TO DISCUSS/PROPERLY FIX

  • The commit 64f7cf6 is memleaking; it is a temporary fix for now to prevent some failing tests, like gtest-roofit-histfactory-test-testHistFactory.
  • This commit: de5d141 is a temporarily fix reintroducing a memory leak.
  • This commit: d79963e fixes an issue with failing tests like tutorial-hist-cumulative when JITLink is turned on. This works for now, but will need to investigate further on why resource trackers were not destructed in the right order.
    • Not a critical issue: Can also look into this later after the merge.
  • Discussion on a proper solution for the macOS modulemap issue

Changes or fixes:

Checklist:

  • Fix mac13 build
  • tested changes locally
  • updated the docs (if necessary)

This PR fixes #

@devajithvs devajithvs self-assigned this May 31, 2024
@hahnjo hahnjo added in:Cling clean build Ask CI to do non-incremental build on PR labels May 31, 2024
Copy link

github-actions bot commented May 31, 2024

Test Results

    13 files      13 suites   3d 7h 50m 50s ⏱️
 3 028 tests  3 028 ✅ 0 💤 0 ❌
33 845 runs  33 845 ✅ 0 💤 0 ❌

Results for commit 2443702.

♻️ This comment has been updated with latest results.

@stephanlachnit
Copy link
Contributor

I know this is totally not relevant for you, but I am still wondering: does this allow me to use system clang or are there still downstream patches required?

@hahnjo
Copy link
Member

hahnjo commented Jul 19, 2024

I know this is totally not relevant for you, but I am still wondering: does this allow me to use system clang or are there still downstream patches required?

Hi Stephan, no we still require downstream patches. Their number gets reduced on every upgrade, you can track the status for LLVM 18 here: https://github.com/root-project/llvm-project/commits/ROOT-llvm18. More importantly, and not yet reflected in ROOT-llvm18, we will now also require a patch to the core LLVM JIT infrastructure (until upgrading past LLVM 19): llvm/llvm-project#95532

@stephanlachnit
Copy link
Contributor

Thanks for the update!

@bellenot
Copy link
Member

bellenot commented Aug 9, 2024

Here is the stack trace on Windows:

  Generating etc/allDict.cxx.pch

  Generating PCH for bindings\tpython core core\clingutils core\imt core\rint core\thread graf2d\asimage graf2d\fitsio graf2d\gpad graf2d\gpadv7 graf2d\graf graf2d\postscript graf2d\primitivesv7 graf2d\win32gdk gr
  af3d\g3d graf3d\gl gui\fitpanel gui\fitpanelv7 gui\gui hist\hist hist\histdrawv7 hist\histpainter hist\histv7 hist\spectrum hist\spectrumpainter hist\unfold io\io math\genetic math\genvector math\mathcore math\m
  athmore math\matrix math\minuit math\minuit2 math\physics math\smatrix math\vecops net\net roofit\RDataFrameHelpers roofit\histfactory roofit\hs3 roofit\jsoninterface roofit\roofit roofit\roofitcore roofit\roofi
  tmore roofit\roostats tmva\pymva tmva\sofie tmva\tmva tmva\tmvagui tree\dataframe tree\ntuple tree\ntupleutil tree\tree tree\treeplayer tree\treeviewer tree\webviewer

  missing serialization code for annotation token
  UNREACHABLE executed at C:\root-dev\git\dev.llvm18\interpreter\llvm-project\clang\lib\Serialization\ASTWriter.cpp:4514!
  PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
  Stack dump:
  0.    Program arguments: .\\bin\\rootcling -rootbuild -generate-pch -f allDict.cxx -noDictSelection -D__CLING__ -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DROOT_PCH -I.\\include -I.\\etc -I.\\etc\\dictpch -
  I.\\etc\\cling -IC:\\Python311\\Lib\\site-packages\\numpy\\core\\include -IC:\\Python311\\include -IC:\\libs\\x64\\GSL\\2.7\\include -IC:\\libs\\x64\\libxml2\\2.7.8\\include -IC:\\libs\\x64\\sqlite3\\3.45.2 -IC:
  /root-dev/git/dev.llvm18/graf2d/win32gdk/gdk/src etc\\dictpch\\allHeaders.h -IC:/root-dev/git/dev.llvm18/graf2d/win32gdk/gdk/src -IC:/root-dev/git/dev.llvm18/graf2d/win32gdk/gdk/src/gdk -IC:/root-dev/git/dev.llv
  m18/graf2d/win32gdk/gdk/src/glib -IC:/root-dev/build/x64/dev.llvm18/include -IC:/root-dev/git/dev.llvm18/core etc\\dictpch\\allLinkDefs.h
  Exception Code: 0x80000003
   #0 0x00007ffc2c2306eb HandleAbort C:\root-dev\git\dev.llvm18\interpreter\llvm-project\llvm\lib\Support\Windows\Signals.inc:424:0
   #1 0x00007ffcabe91881 (C:\windows\System32\ucrtbase.dll+0x71881)
   #2 0x00007ffcabe92851 (C:\windows\System32\ucrtbase.dll+0x72851)
   #3 0x00007ffc2c1dad4c llvm::llvm_unreachable_internal(char const *, char const *, unsigned int) C:\root-dev\git\dev.llvm18\interpreter\llvm-project\llvm\lib\Support\ErrorHandling.cpp:212:0
   #4 0x00007ffc25258963 clang::ASTWriter::AddToken(class clang::Token const &, class llvm::SmallVectorImpl<unsigned __int64> &) C:\root-dev\git\dev.llvm18\interpreter\llvm-project\clang\lib\Serialization\ASTWrite
  r.cpp:4514:0
   #5 0x00007ffc2525399f clang::ASTWriter::WriteLateParsedTemplates(class clang::Sema &) C:\root-dev\git\dev.llvm18\interpreter\llvm-project\clang\lib\Serialization\ASTWriter.cpp:4340:0
   #6 0x00007ffc252568d1 clang::ASTWriter::WriteASTCore(class clang::Sema &, class llvm::StringRef, class clang::Module *) C:\root-dev\git\dev.llvm18\interpreter\llvm-project\clang\lib\Serialization\ASTWriter.cpp:
  5117:0
   #7 0x00007ffc2525852a clang::ASTWriter::WriteAST(class clang::Sema &, class llvm::StringRef, class clang::Module *, class llvm::StringRef, bool) C:\root-dev\git\dev.llvm18\interpreter\llvm-project\clang\lib\Ser
  ialization\ASTWriter.cpp:4649:0
   #8 0x00007ffc237996f8 WriteAST C:\root-dev\git\dev.llvm18\core\dictgen\src\rootcling_impl.cxx:2021:0
   #9 0x00007ffc23799959 GenerateAllDict C:\root-dev\git\dev.llvm18\core\dictgen\src\rootcling_impl.cxx:2042:0
  #10 0x00007ffc237ac399 RootClingMain(int, char **, bool) C:\root-dev\git\dev.llvm18\core\dictgen\src\rootcling_impl.cxx:5010:0
  #11 0x00007ffc2378c455 ROOT_rootcling_Driver C:\root-dev\git\dev.llvm18\core\dictgen\src\rootcling_impl.cxx:6238:0
  #12 0x00007ff68e2f2605 main C:\root-dev\git\dev.llvm18\main\src\rootcling.cxx:44:0
  #13 0x00007ff68e2f2924 invoke_main D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:78:0
  #14 0x00007ff68e2f2924 __scrt_common_main_seh D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288:0
  #15 0x00007ffcacad7374 (C:\windows\System32\KERNEL32.DLL+0x17374)
  #16 0x00007ffcae65cc91 (C:\windows\SYSTEM32\ntdll.dll+0x4cc91)
C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(254,5): error MSB8066: Custom build for 'C:\root-dev\build\x64\dev.llvm18\CMakeFiles\6485b15e5a52a9a93f
294ac9c6ea6f8f\allDict.cxx.pch.rule;C:\root-dev\build\x64\dev.llvm18\CMakeFiles\c2921ee0b70b9f5a32ad0eb8246e0cd2\onepcm.rule;C:\root-dev\git\dev.llvm18\CMakeLists.txt' exited with code -1. [C:\root-dev\build\x64\d
ev.llvm18\onepcm.vcxproj]

@hahnjo
Copy link
Member

hahnjo commented Aug 20, 2024

FWIW the current state compiles fine on RISC-V and is at least able to run df103_NanoAODHiggsAnalysis.py 😉

@vgvassilev
Copy link
Member

@smuzaffar, @devajithvs and @hahnjo have everything working reasonably with the llvm18 upgrade. Can you check this PR against cmssw?

@vgvassilev
Copy link
Member

@smuzaffar, @devajithvs and @hahnjo have everything working reasonably with the llvm18 upgrade. Can you check this PR against cmssw?

@iarspider, could you give us a hand?

@dpiparo
Copy link
Member

dpiparo commented Aug 27, 2024

@smuzaffar, @devajithvs and @hahnjo have everything working reasonably with the llvm18 upgrade. Can you check this PR against cmssw?

@iarspider, could you give us a hand?

I allow myself to add @aandvalenzuela to perhaps get some help.

- Rename `CharacterLiteral` enum to `CharacterLiteralKind`
- Rename `CGFT_{}` enums to `CodeGenFileType::{}`
- Rename `ETK_{}` enums to `ElaboratedTypeKeyword::{}`
- Rename `TTK_{}` enums to `TagTypeKind::{}`
- Use `ArraySizeModifier` and `StringLiteralKind`
- Use `LinkageSpecLanguageIDs`
- Rename `llvm::CodeGenOpt::Level` to `llvm::CodeGenOptLevel`
We have to get the FAM via the MAM otherwise it won't invalidate function
analyses in FAM

See: https://reviews.llvm.org/D146160
- Include `EnterExpressionEvaluationContext.h`
This was moved to its own header in:
llvm/llvm-project@6d68805

- Include `StringExtras.h`
This is now needed due to:
llvm/llvm-project@3ff3af3
devajithvs and others added 16 commits August 28, 2024 18:46
In LLVM 18 with incremental extensions enabled, the Preprocessor will
never report tok::eof, but tok::annot_repl_input_end instead.
Prevents failures such as:

```
Processing roottest/root/meta/runnamespace.C...
Error in cling::MetaProcessor: cannot read from binary input:
'runnamespace.C'
```
This is needed for macros that start with `{` after the rebase
C++20 builds were failing due to the commit:
llvm-project/commit/574ee1c02ef73b66c5957cf93888234b0471695f

with an error `imported non C++20 importable modules`
This makes them run after the <Platform> library: It provides the
__cxa_atexit function, which must not be found in the process.
LLVM now defines multiple Dylibs, and __cxa_atexit is provided in
the <Platform> library.
We started seeing an assertion failure in ~FinalizedAlloc after
commit d2479f4. Likely changed the destruction order that caused
the assertion to trigger.
This fixes the Cling test Driver/E.C.
This applies the upstream changes in commit
llvm/llvm-project@9992b38
to a copy of the modulemap from MacOSX14.2.sdk.
This fixes the failing python enum tests like:
- `roottest-python-cpp-cpp`
- `roottest-python-cmdLineUtils-ROOT_8197`

Also remove `R__BYTESWAP` which should not be needed anymore.

The leak was fixed with:
llvm/llvm-project#78311

This caused our tests to fail as they relied on the
previous behavior.
@smuzaffar
Copy link
Contributor

ROOT failed to build for aarch64, the build error is

Processing hsimple.C...
cling JIT session error: Failed to materialize symbols: { (<Process Symbols>, { DW.ref.__gxx_personality_v0 }) }
ninja: build stopped: subcommand failed.
error: Bad exit status from /data/cmsbld/jenkins_a/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.3N7prC (%build)

See https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b2e919/41190/externals/root/6.33.01-1689b1f60f821828a7e2948d6cf5f908/log for detail root build log

@smuzaffar
Copy link
Contributor

For x86_64, this change passed all cmssw CI tests

@vgvassilev
Copy link
Member

For x86_64, this change passed all cmssw CI tests

Does both have the same version of gcc? Does one have libgcc_s.so and the other libgcc.a? And which one?

@smuzaffar
Copy link
Contributor

@vgvassilev , gcc version is same (note that we use our own gcc and not from system). Both archs has libgcc_s.so in its lib64 path and libgcc.a in lib/gcc/<arch>-redhat-linux-gnu/12.3.1/ directory [b]

[a]

  • x86_64
Singularity> gcc --version
gcc (GCC) 12.3.1 20230527
  • aarch64
Singularity> gcc --version
gcc (GCC) 12.3.1 20230527

[b]

/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02853/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/lib/gcc/x86_64-redhat-linux-gnu/12.3.1/libgcc.a
/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02853/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/lib/gcc/x86_64-redhat-linux-gnu/12.3.1/libgcc_eh.a
/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02853/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/lib64/libgcc_s.so
/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02853/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/lib64/libgcc_s.so.1
/cvmfs/cms-ib.cern.ch/sw/aarch64/nweek-02853/el8_aarch64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/lib/gcc/aarch64-redhat-linux-gnu/12.3.1/libgcc.a
/cvmfs/cms-ib.cern.ch/sw/aarch64/nweek-02853/el8_aarch64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/lib/gcc/aarch64-redhat-linux-gnu/12.3.1/libgcc_eh.a
/cvmfs/cms-ib.cern.ch/sw/aarch64/nweek-02853/el8_aarch64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/lib64/libgcc_s.so
/cvmfs/cms-ib.cern.ch/sw/aarch64/nweek-02853/el8_aarch64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/lib64/libgcc_s.so.1

Upstream LLVM now unconditionally uses indirect / PC relative
personality encodings since commit
llvm/llvm-project@ca65969

This leads to references of DW.ref.__gxx_personality_v0 that cannot
be resolved with RuntimeDyld, that we are still using for the moment.
As a fix, auto-claim symbols as we are doing for PPC, see upstream commit:
llvm/llvm-project@72ea1a7
@vgvassilev
Copy link
Member

@smuzaffar, can you rerun testing of this PR?

@smuzaffar
Copy link
Contributor

smuzaffar commented Sep 5, 2024

@vgvassilev , @aandvalenzuela has already started the tests for aarch64 and looks like latest change has fixed the root build (tests are now building cmssw)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
clean build Ask CI to do non-incremental build on PR in:Cling
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants