-
Notifications
You must be signed in to change notification settings - Fork 118
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
M1 Mac "EOF" during http access to docker-hosted webserver "Connection reset by peer" #5407
Comments
Thanks for the report, @rfay. If you have a spare Big Sur Intel machine to test it on, I'd be interested to know whether it fails on that using the new virtualization framework. |
I went ahead and switched a Big Sur amd64 test runner to new virtualization, we'll see what happens. Note that my initial experience wasn't good - it just went off into spinning-circle land forever. However, when I closed the window after many minutes and came back it seemed to be set. Not satisfied, I reset to factory defaults and then changed it to new virt again. It didn't do the infinite spinning the second time. |
@stephen-turner Networking to container (host.docker.internal) is nonfunctional with new virtualization enabled, so that's a no-go for NFS or xdebug. Opened #5410 |
@stephen-turner @djs55 The EOF issue may have gone away in RC2, I'm not sure. However, and I assume related, I still get "Connection reset by peer" quite a lot. "read tcp 127.0.0.1:62795->127.0.0.1:80: read: connection reset by peer". It's rare that I don't have to restart an M1 test run due to this one. Doesn't happen on the same test each time. |
I see this related things quite a lot still "Get "http://127.0.0.1//README.txt": read tcp 127.0.0.1:60801->127.0.0.1:80: read: connection reset by peer" Sometimes I have to restart tests many times to get a clean run. Of course, while I'm doing that every other platform, Docker for Mac amd64, Docker for Windows, WSL2, Linux, have all completed without any of these errors on the same code. |
I do still see the EOF, it hasn't gone away:
|
We just started having the same issue on Apple Silicon macs with Docker 3.3.1. Reverting to Preview 7 solves the issue. Happy to provide more details. |
This problem is not solved in Docker Desktop 3.3.2. I had hopes that it might have been related to this in the 3.3.2 release notes:
|
Not sure how you're going to chase this @djs55 but it's a really significant problem. Casual users probably just click through it and try again, but I haven't had a successful M1 test run of the ddev test suite for more than a month, I'm starting to ignore the failures. And it's always the EOF |
@rfay thanks for trying with 3.3.2. The code path which handles A bit of a long shot but: Is the EOF from the first request to a container or does it happen after successful requests? I ask because running |
Thanks for the thinking and attention on this.
Again, thanks. And I know this is a hard one. I'll try to re-investigate a few of these and keep some notes to see if there's any kind of pattern. But it seems like... |
We had problem with HTTP clients complaining about mismatched content length (response got cut off) and now it seems to work in 3.3.2 so either this is not the same issue or there are more variables at play Sent with GitHawk |
@LeZuse I'm betting that your fix was recorded in the 3.3.2 release notes,
Sadly, this one seems to be different. |
This problem remains. I'll work on a recreation scenario. It's only one time in 10 that the ddev test suite completes without this problem on M1 (problem is ONLY on M1, same tests everywhere). |
I think I have a recreation scenario, and here's a diagnostic: 165EF212-DB99-4FE1-90A4-E516A309B23F/20210609215723 (Edit: Got another one: 165EF212-DB99-4FE1-90A4-E516A309B23F/20210609221754 ) Most of the time this is probably "Connection reset by peer" It appears that in this situation both ddev-router and ddev-webserver are completely ready (and have already served something, but internally in the container) |
Upgraded all test runners to 3.5.0, but this remains a failure somewhere in almost every test run. |
Oh, man... After 24 hours of extreme frustration, here I am. We have some piece of code that Just Don't Work™ – no errors, no exceptions, no log. Nothing. The request just have a sudden stop coming from nowhere, like being hit by a lightning on a beautiful summer day. And then I realized that I was the only one suffering from this rare phenomenon. My coleges –followers of Linus Torvalds– didn't even notice that something was odd – just me.
Well, it was. After accepting that M1 could be the cause I started looking for someone as unlucky as me and found this issue. I uninstalled every piece of Docker in my computer and downgraded from 3.5.1 to 3.3.1, the first to support Apple Silicon, and everything is back to normal – except me, that even now hadn't accepted that I spent 8+ hours of working hours looking for a bug in code. |
This remains an issue on 3.5.2. |
Same on 3.6.0 Although I can usually get a full ddev test suite to pass on mac amd64 and Windows amd64, it's very rare that I can get through a full suite on mac M1. Sometimes I retry several times. It's always some random test that has an EOF, connection reset by peer |
Issues go stale after 90 days of inactivity. Prevent issues from auto-closing with an If this issue is safe to close now please do so. Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows. |
/remove-lifecycle stale Just because issues don't get any attention doesn't mean they're stale. This is a consistent problem. |
Possibly related? #3448 |
This remains a consistent problem. As speculated here it may be related to the use of localhost. |
I have the same issue with a test. I have a mac mini and a studio that are running 2 different java apps that need to communicate with each other through TCP Sockets, and I get intermitent issues with socket disconnection quite often. This is a stress test, and I have never been able to run it with the 2 computers for more than 1 hour, or 1.5 hours. If I run everything in the mac studio using a docker internal network, then everything runs ok, and I am able to have the communication going for days on end. |
The only solution to this problem that I found was to reduce the allocated resources in docker-desktop down to 1 CPU (inspired by this post). My containers are now consistently passing, whereas with 4 CPUs, I would get the EOF error around 4/5 times. Docker-Desktop Version: 4.18.0 (104112) |
This still happens regularly, but it's not going to be addressed here. Closing. |
Hey @rfay, could you point us to the issue where this is going to be addressed so we can keep track of the issue? Thanks! 🙏 |
Unfortunately, if it hasn't gotten any attention in 2 1/2 years, I don't think it will get any. However, it's a poor issue and doesn't have a repro case, as it's intermittent. If you have a good repro case, something that the docker team can easily reproduce, perhaps they'll pay attention if you open a new issue with uploaded diagnostics and a repro case. Consider making a github test repo that demonstrates it. |
Expected behavior
Predictable behavior on docker supporting an nginx webserver.
Actual behavior
Intermittent EOF when hitting a webserver.
I now have Mac M1 CI going for ddev, currently using build 3.1.0 (60984)
Every day I see a few tests where an http request gets an EOF. This is different from many docker fails I've experienced before, and seems new to the M1 docker.
I know it's not much help, but
The thing is that EOF on M1 has happened intermittently on different tests, and it's the most likely failure on mac M1. It usually succeeds on a retry (there's an hour of testing, with hundreds or thousands of http requests). There is only one machine in the pool at this point, so it's not about different computers with different state.
I just thought you should know about this one because it's reasonably common.
Information
MacOS Big Sur 11.2.1
Docker Desktop for Mac 3.1.0 (60984)
Steps to reproduce the behavior
The text was updated successfully, but these errors were encountered: