-
-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Podman support #2088
Comments
Thanks for asking! Since this is not the first time somebody asks for Podman, I would like to post a (non complete, most probably) list of requirements we need to do the Testcontainers' magic:
I am not a Podman expert, and if somebody from the community can help clarifying these, I would be happy to learn, and, as long as every point is clarified (the list is not 100% complete and we may need to add more in future) we can consider adding the support for Podman. |
and BTW, "daemonless" here makes it harder for Testcontainers :) |
From my Podman experience:
I wish @rhatdan could chime in the discussion to clarify or correct my statements 😃 |
We're not using the CLI, but the REST API. It looks to me that we should rather focus on waiting for the rootless Docker (an ongoing effort upstream), Podman as it is (plus Buildah, plus whatever else is needed) cannot provide the same functionality we need to do what we're doing in Testcontainers. |
Dropping this here because I found it useful and relevant to the discussion :) |
Podman recently started offering a REST API: https://podman.io/blogs/2020/01/17/podman-new-api.html |
@ppalaga nice! Do you have a compatibility table or something, to see which Docker's API endpoints are supported or not? Thanks! |
@bsideup |
@crunchtime-ali I believe running Testcontainers' test suite is the best way to check whether all APIs are supported or not. Our tests should cover all types of Docker features we are using. You can also start a TCP proxy between Testcontainers and Podman to collect the list of endpoints that are used. |
Some more information that might be helpful: https://github.com/containers/libpod/blob/master/docs/source/markdown/podman-system-service.1.md https://github.com/containers/libpod/blob/master/API.md The Podman API supposedly has a docker API compatible endpoint. |
And it can also expose a |
Correct we are under heavy development and are looking at testcontainers to prove that we handle the Docker API correctly. We have an extended version to support advanced features of Podman as well. |
I'm also looking at doing the reverse: Running testcontainers tests in podman CI. Solving the challenges here, will help me get to there. (subscribed) |
how does one run the test suite? hints around how to specify the sock to connect to is a bonus. ill do the work. |
@baude I'm not a testcontainers dev, but I suspect you can get insight into both of your questions from: https://github.com/testcontainers/testcontainers-java/blob/master/.circleci/config.yml and |
Thanks for working on it! we read the same environment variables as the Docker CLI does (as one of the strategies we support), e.g. By default, we also search for Docker environment in well-known locations (like Our primary CI is Azure Pipelines: |
@baude @rhatdan If there is any way I can support you, feel free to get in touch or join our Slack, if you want to have more extensive discussions. |
Thanks @baude for taking this up, the java-stuff is out of my depth, but poke me on IRC if/when I can help with any automation-things. |
recent update here ... our compatibility layer requires the use of versioned paths (by api). the testcontainers suite does not appear to deal with that. i wrote a small java app using docker-java, which test containers also uses, and by default the versioned paths are not used there but it does provide the capability to do so with:
So we need some sort of way to configure that in testcontainers and perhaps things will progress. |
update! a contributor has fixed ^^ so now just a matter of understanding how to dissect the test failures. |
Defining DOCKER_HOST with |
Podman != Docker. |
Just I'd mention that I came to this ticket after installing Fedora 32, realising I couldn't get docker-ce to run without lots of kernel/firewal changes docker/for-linux#955 Looks like redhat are pushing podman So until/if the docker ticket is fixed, future fedora installations of docker are out of reach for layman users. |
It is so indeed, although Podman does not support all Docker's features and does not work with tools like Docker Compose, Fabric8 Docker Maven Plugin by Red Hat ( fabric8io/docker-maven-plugin#1330 ), Testcontainers and others. It is a bit sad to see a lot of push with statements like "just replace Docker CLI with Podman CLI and everything will be working" while, in reality, there is a ton of things missing.
Thank you! FYI @kiview is running Fedora 32 with Docker, he just had to apply a couple of modifications (cgroups v1, firewall settings) Also FYI, I did some experiments with Rootless Docker and apparently it is working! (in PoC mode) |
Could you please list which specific features Podman is missing to be able to serve as a replacement for Docker in Testcontainers? |
@ppalaga sorry, but I don't know (I am on Mac and can't use Podman to even test) perhaps ask @baude: Also, If Podman serves as a drop-in replacement for Docker (as claimed), it should be trivial to test it - just run Testcontainers' and |
The original goal was Podman as a drop in replacement for Docker Potential issues we have seen so far are:
|
Bottom line on this, is we need help from the community to help with tests, and fixes to get to the point where we can support the Docker.sock API. |
@msausu any success with this? wanting to do the same |
Just a status report on minimal testing (running JUnit with testcontainers):
Testing works fine - I use Files edited
|
Following @guss77 findings, another status report and workaround for the issue: I've managed to have a successful Testcontainers and Podman setup on Mac OS by setting up a ssh tunnel that redirects unix socket from Guest CoreOS VM to Host filesystem. Herewith recipe I have to run tests atm: # Bootstrap guest CoreOS VM
podman machine init -m 4096
podman machine start
# use this command to identify ssh port guest CoreOS VM is using:
podman system connection list
# Create ssh tunnel, this will create a unix socket in /tmp/podman.sock
ssh -i ~/.ssh/podman-machine-default -p <port> -L'/tmp/podman.sock:/run/user/1000/podman/podman.sock' -N core@localhost
in another shell session: export DOCKER_HOST=unix:///tmp/podman.sock
export TESTCONTAINERS_CHECKS_DISABLE=true
export TESTCONTAINERS_RYUK_DISABLED=true
mvn clean install # works for me |
The workaround proposed by @bsantanna works for some containers but not others. Even ones that work I get this a lot: ERRO[1205] accept tcp [::]:39779: use of closed network connection |
I understand this issue is closed, although just for the record, here is a workaround I found that worked in my case: My integration test was letting TestContainer launch docker-compose as follows:
I'm using Podman 3.4.2 and TestContainer 1.6.3 on Linux with the following env variables:
The
I solved the issue by telling
|
Here's my recipe. I was able to avoid the need to disable ryuk. Edit
Then run the following
If you're podman 4.1 or higher, you don't need the the If you're on mac,
Then (for all oses)
The rootful part makes ryuk behave ok, based on my testing (see containers/podman#14238). If ryuk is still causing problems or you don't want to run rootful then Finally, if you're on Mac M1, update podman to behave better with images for a different architecture (see https://edofic.com/posts/2021-09-12-podman-m1-amd64/ )
Once the virtual machine restarts, you should be good to run testcontainers. |
This is awesome @holly-cummins. One question - With podman 4.1.1 on apple silicon mac, did you have to downgrade qemu back to version 6.2.0? |
Ummm. I don't remember doing that, but I am indeed running 6.2.0:
I think I must have been lucky, and done my installation before v7 was released. Is 7 not so good on your machine? |
When I did a I had to manually back qemu down to 6.2.0_1 using this procedure:
I found that procedure out on some GitHub issue somewhere - I just can't seem to remember where exactly... |
@holly-cummins There's also this - containers/podman#14303 |
Oh, sorry, @edeandrea, I've got muddled between the qemu-user-static in the podman machine, and the qemu on my mac machine. My mac has podman 4.1.1 and qemu 7.0.0, but it seems happy. So either the issue is fixed, or some other environmental factor on my machine masks the issue. |
Good to know @holly-cummins . Maybe I'll give it a try. |
@holly-cummins thanks for the Just wanted to add that in case the |
Is there known issue with pulling image by testcontainers ? I keep getting below errors when image is not in local repo and testcontainers tries to pull it from remote:
|
@mario45211 I had the same issue initially. When I tried to pull the image manually, I realized that Podman did not have access to that repo yet where the image "repohost:8080/path/to/image:latest" was located. So I did It seems there is a bug in docker-java-api that triggers the NPE in such a case. At the code where the NPE occurrs in the class |
To follow up, I did not find a tracked issue regarding the NPE. As the problem lies in the docker-java-api code and not with testcontainerst itself, I created an issue in their issue tracker: docker-java/docker-java#1998 |
For those landing here hunting for the solution for NPE, here is one workaround until the docker-java bug is fixed: |
As far as I can tell, nowadays Podman works fine with Testcontainers, as long as you enable the Podman socket ( The reason why some need to run Ryuk as a privileged container (or disable it altogether), as far as I can tell, is not because of Podman, but rather because of SELinux. If SELinux is enabled, Ryuk will be unable to connect to the Docker/Podman UNIX socket which is bind-mounted inside the container. See this and this discussion for more detail. Running Ryuk as a privileged container works around this problem because privileged containers are not isolated through SELinux. Similarly, non-privileged Ryuk also works fine with Podman on systems without SELinux. And also, the same problem with Ryuk reproduces if you run the Docker daemon with SELinux enabled (#7177). I suspect the reason this issue comes up in discussions with Podman because (1) Podman has SELinux support enabled by default and most adopters are likely on Red Hat distros which ship with SELinux enabled and (2) Docker has SELinux support disabled by default, even on distributions with SELinux enabled. |
It would be really nice if TestContainers supported podman.
This is a real game changer to no longer depend on Docker daemons to run.
The text was updated successfully, but these errors were encountered: