upgrade selected GAP packages at startup

[ThomasBreuer]

resolves #3219

[lgoettgens]

I'll try to explain the Aqua issue reported here: A packages __init__() function gets called during precompilation of its dependents. In OSCAR's case (with this PR), GAP.Packages.install is called which creates a Downloads.Downloader object. This keeps the connection open for an extended period of time, which hinders the precompilation of OSCAR to finish (but only if OSCAR is a dependency of something).
A fix for something similar in Polymake.jl is oscar-system/Polymake.jl#466 by @benlorenz. There also is a "real" fix in JuliaLang/Downloads.jl#234, but this will only be available in 1.11+.

[benlorenz]

That Downloads.jl fix will also be in 1.10.1.

Edit:
But I am not totally sure if this will fix this issue on 1.10 as there might be some other changes in the atexit code on nightly.

Also there might be something else involved here, since that Downloads.jl issue should be fixed and included on the latest julia nightly (via JuliaLang/julia#52885).
But it did error even on nightly.

[lgoettgens]

Hm, I don't see it in JuliaLang/julia#52755 or am I just blind?

[benlorenz]

The commit is there but it doesn't seem to appear in the list: JuliaLang/julia@aaaf5de (#52755)
Maybe because it was added to that branch via a separate PR: JuliaLang/julia#52886

[ThomasBreuer]

@lgoettgens @benlorenz Thanks for the information, but I do not understand what the proposal is.
The failing tests with Julia 1.10 are run with Julia 1.10.0, which does not know Downloads.default_downloader.
Thus the cited fix from Polymake.jl would not help there. Or what am I missing?
And if I understand @benlorenz' comment right then the failure in Julia nightly (also in the Aqua.jl tests) indicates another (?) problem.

[benlorenz]

1.10 should have that function, it has an exclamation mark at the end:

julia> Downloads.default_downloader!
default_downloader! (generic function with 2 methods)

I am indeed confused why this also happens on nightly which is supposed to clean up these downloaders automatically. Especially since the following seems to work, even though it is not really nice:

  GAP.Packages.Downloads.default_downloader!(GAP.Packages.Downloads.Downloader(grace=0.01))
  for (pkg, version) in [
     ("repsn", "3.1.1"),
     ]
    GAP.Packages.install(pkg, version, interactive = false, quiet = false)
  end
  GAP.Packages.Downloads.default_downloader!()

Note that this would need some extra if statement to skip these calls for 1.9 and older.

Another option would be to add a custom Downloader in the GAP packages.jl file like this:

   dlhelper = Downloads.Downloader(; grace=0.1)
...
        buffer = Downloads.download(String(url), IOBuffer(); downloader=dlhelper)

The dlhelper can probably be created in the init_packagemanager function and be used in both lambda functions.

[ThomasBreuer]

Thanks @benlorenz.

Logically, the right place to fix the problem is in GAP.jl, as you suggest.
I have created a pull request for that.

This does not solve the problem in Oscar until the next release of GAP.jl. I am going to add the "not really nice" solution sketched above to the current pull request.

[codecov]

Codecov Report

Merging #3222 (b83b4e6) into master (e5a2cdf) will increase coverage by 0.05%.
Report is 14 commits behind head on master.
The diff coverage is 100.00%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3222      +/-   ##
==========================================
+ Coverage   81.62%   81.68%   +0.05%     
==========================================
  Files         546      546              
  Lines       72877    73059     +182     
==========================================
+ Hits        59489    59677     +188     
+ Misses      13388    13382       -6     

Files	Coverage Δ
src/Oscar.jl	62.02% <100.00%> (+2.56%)	⬆️

... and 26 files with indirect coverage changes

[fingolfin]

I have just release GAP.jl 0.10.2 so you could remove this and update Project.toml instead.

[ThomasBreuer]

Great. Thanks.