Core dumped in sdk:6.0.100-bullseye-slim-arm32v7 and sdk:6.0.100-alpine3.14-arm32v7 without --privileged #3253
Comments
|
I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label. |
|
Thanks @pablofrommars! I can repro this. I'm just gathering some more data to post on this. |
|
It definitely seems to be distro version-related. I spent some time trying to determine the scope of the environments in which this can repro. Here's what I've got:
Here's the core dump to be investigated: @janvorli - Is this something you can investigate to determine the root cause of the crash? |
|
Could it be ICU related? $ docker run -it --rm --privileged -v /home/pi/dotnet:/repo arm32v7/debian:bullseye-slim bash
root@45622b4ee36b:/# cd repo
root@45622b4ee36b:/repo# ./dotnet --info
Process terminated. Couldn't find a valid ICU package installed on the system. Please install libicu using your package manager and try again. Alternatively you can set the configuration flag System.Globalization.Invariant to true if you want to run with no globalization support. Please see https://aka.ms/dotnet-missing-libicu for more information.
at System.Environment.FailFast(System.String)
at System.Globalization.GlobalizationMode+Settings..cctor()
at System.Globalization.CultureData.CreateCultureWithInvariantData()
at System.Globalization.CultureData.get_Invariant()
at System.Globalization.CultureInfo..cctor()
at System.Globalization.CultureInfo.get_CurrentUICulture()
at System.TimeZoneInfo.GetUtcStandardDisplayName()
at System.TimeZoneInfo.CreateUtcTimeZone()
at System.TimeZoneInfo..cctor()
at System.DateTime.get_Now()
at Microsoft.DotNet.Cli.Program.Main(System.String[])
Aborted (core dumped)Note that it's run with --privileged and still core dumped. Without --privileged, it fails a bit more silently (no exception stack trace). $ docker run -it --rm -v /home/pi/source/dotnet:/repo arm32v7/debian:bullseye-slim bash
root@c93e94cda139:/# cd repo/
root@c93e94cda139:/repo# ./dotnet --info
Aborted (core dumped) |
|
.NET requires a number of packages, including ICU, that aren't installed in the bullseye-slim image so it would certainly be expected for things not to work if they aren't installed. Here are the dependencies: |
|
Both the core.zip and coredump.zip shared above show that a call to QueryPerformanceCounter has failed. Here is the top of the stack trace: All the QueryPerformanceCounter does is this: BOOL retval = TRUE;
struct timespec ts;
int result = clock_gettime(CLOCK_MONOTONIC, &ts);
if (result != 0)
{
ASSERT("clock_gettime(CLOCK_MONOTONIC) failed: %d\n", result);
retval = FALSE;
}
else
{
lpPerformanceCount->QuadPart =
((LONGLONG)(ts.tv_sec) * (LONGLONG)(tccSecondsToNanoSeconds)) + (LONGLONG)(ts.tv_nsec);
}So the clock_gettime must have failed. If it was failing both with and without |
|
There have definitely been some changes in clock_gettime between Debian 10, where it works, and Debian 11, where it doesn't work. Specifically around 32-/64-bit.
|
|
Hmm, strange thing is that on my device running aarch64 distro and using docker with the 6.0.100-bullseye-slim-arm32v7 image, both dotnet and a simple test C app that calls clock_gettime work both with and without the --privileged as expected. So maybe the kernel is related too. |
|
Yes, exactly. This is why it hasn't been caught by any of our builds, because all of our Arm build machines are aarch64. It only repros on an Arm32 machine. |
|
It seems the issue might be the same one as the one mentioned here: debuerreotype/docker-debian-artifacts#106 |
|
Yes, it is the same issue. I was able to repro it on my RPI 4 with 32 bit Raspbian installed as a host OS for the docker. To install the updated libseccomp, I had to add testing repo to the package repositories in /etc/apt/sources.list: Then I've followed an advice to set the priority of the testing repo to low so that things get still installed from the main repo by default (https://tech.borpin.co.uk/2019/12/17/install-a-package-from-the-testing-repository/) Then I've ran |
|
Please note that this is not a .NET issue, other things in the bullseye image fail too (e.g. just |
|
I confirm @janvorli fix my build issue. Thank you so much. |
|
Thanks again! |
|
Thanks @janvorli, that solved the issue for me too. Date and time were incorrect in many recent images when run on arm32. Before: docker run --rm httpd date
Thu Jan 1 00:00:00 UTC 1970docker run --rm httpd:buster date
Fri Nov 26 16:59:12 UTC 2021And after updating libseccomp on host: docker run --rm httpd date
Fri Nov 26 17:01:29 UTC 2021 |
|
@janvorli We have the same issue for our .NET 6 IoT edge modules. Updating hundreds of devices and installing The devices are ARM32v7 and running Debian 9 and Debian 10 on the host with Docker 3.0.13+azure. |
|
So it seems that manually building .NET 6 (wget runtime) based on Debian Buster Slim works for us. Is there any way to get an official image for that? |
|
@florianbader unfortunately I am not aware of other alternatives (other than running docker with |
@florianbader - Our policy is to only provide images for the latest stable version of Debian when a major version of .NET is released. Depending on feedback from people being negatively impacted by this, it's possible that we could reconsider the set of official images that are provided. But I don't think we're there yet. You're likely familiar with this already, but we do have documentation on how to install .NET in containers for scenarios where we don't provide official images. |
|
I would love to see an official image because this affects all our edge devices. We cannot update from Debian 9 or 10 to the latest stable which means we cannot update the docker (moby) version. For anyone that finds this and wants to build their own .NET 6 image for Debian Buster this is what we did and worked for us: FROM arm32v7/debian:buster-slim
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
ca-certificates \
\
# .NET Core dependencies
libc6 \
libgcc1 \
libgssapi-krb5-2 \
libssl1.1 \
libstdc++6 \
zlib1g \
wget \
&& rm -rf /var/lib/apt/lists/*
ENV \
# Configure web servers to bind to port 80 when present
ASPNETCORE_URLS=http://+:80 \
# Enable detection of running in a container
DOTNET_RUNNING_IN_CONTAINER=true \
# Set the invariant mode since icu-libs isn't included (see https://github.com/dotnet/announcements/issues/20)
DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=true \
# ASP.NET Core version
ASPNET_VERSION=6.0.0 \
# Set the default console formatter to JSON
Logging__Console__FormatterName=Json
ENV DOTNET_VERSION=6.0.0
RUN wget -O aspnetcore.tar.gz https://dotnetcli.azureedge.net/dotnet/aspnetcore/Runtime/$DOTNET_VERSION/aspnetcore-runtime-$DOTNET_VERSION-linux-arm.tar.gz \
&& aspnetcore_sha512='36be738bb40a0cadacd4531c3597a25fd45deb7c48090ffb61c79a5db7742a5b8e13051b06556e15e7e162e4a044795c0ca5e6da4db26b63b05c37b39e74e301' \
&& echo "$aspnetcore_sha512 aspnetcore.tar.gz" | sha512sum -c - \
&& mkdir -p /usr/share/dotnet \
&& tar -C /usr/share/dotnet -oxzf aspnetcore.tar.gz \
&& ln -s /usr/share/dotnet/dotnet /usr/bin/dotnet \
&& rm aspnetcore.tar.gzIf you only need the .NET runtime instead of ASP.NET Core use the following wget: RUN wget -O dotnet.tar.gz https://dotnetcli.azureedge.net/dotnet/Runtime/$DOTNET_VERSION/dotnet-runtime-$DOTNET_VERSION-linux-arm.tar.gz \
&& dotnet_sha512='575037f2e164deaf3bcdd82f7b3f2b5a5784547c5bad4070375c00373722265401b88a81695b919f92ca176f21c1bdf1716f8fce16ab3d301ae666daa8cae750' \
&& echo "$dotnet_sha512 dotnet.tar.gz" | sha512sum -c - \
&& mkdir -p /usr/share/dotnet \
&& tar -C /usr/share/dotnet -oxzf dotnet.tar.gz \
&& ln -s /usr/share/dotnet/dotnet /usr/bin/dotnet \
&& rm dotnet.tar.gz |
|
As recommended in this issue, I've updated @mthalman How can I solve this? |
|
@mu88 have you followed all the steps from the comment including setting priority of the testing repo to low? |
|
@janvorli I've created the file Afterwards I called Now And 1083 seems incredibly high... 🤔 |
|
|
Aborted (core dumped) 1、宿主libseccomp 2.4.2 或更高版本 2、Docker 版本 19.03.9 或更高版本 |
Describe the Bug
In dotnet/nightly/sdk:6.0.100-bullseye-slim-arm32v7 and 6.0.100-alpine3.14-arm32v7, running dotnet commands inside the container started without --privileged result in core dumped. Docker host is running on raspberry pi OS (32 bits) on a pi v4. This is the minimal way to reproduce the issue. I noticed the issue after upgrading my Dockerfiles from 5.0 to 6.0 and attempting to build.
sdk:5.0.402-buster-slim-arm32v7 works as expected in both privileged and unprivileged.
sdk:5.0.402-bullseye-slim-arm32v7 leads to core dumped without --privileged but works fine when --privileged is supplied. That would indicate the issue has more to do with bullseye and alpine3.14 than the version of dotnet.
Thanks to anyone involved.
Steps to Reproduce
$ docker run -it mcr.microsoft.com/dotnet/nightly/sdk:6.0.100-bullseye-slim-arm32v7 bash root@679d83583dbd:/# dotnet --version Aborted (core dumped)While the following works fine:
$ docker run -it --privileged mcr.microsoft.com/dotnet/nightly/sdk:6.0.100-bullseye-slim-arm32v7 bash root@271ea1adfedd:/# dotnet --version 6.0.100Other Information
I am unaware of a way to build images in "privileged mode" and bypass the issue.
Output of
docker versionOutput of
docker infoThe text was updated successfully, but these errors were encountered: