Podman with Docker-compose not working

[ssrikanthc]

Trying to get testcontainers work with Podman and Docker-compose in Fedora 35 OS to run tests on a custom Kafka cluster. Tried the following approaches but all of them are failing:

1st Approach:
@container
private static final DockerComposeContainer kafka = new DockerComposeContainer(
new File("src/test/resources/compose-test.yml")).withOptions("--compatibility")
.withExposedService(KAFKA_SERVICE, KAFKA_PORT,
Wait.forListeningPort().withStartupTimeout(Duration.ofSeconds(360)))
.withLocalCompose(true);

This approach throws the below error "Link is not supported"

Seemingly related to the deprecated addLink method below:

2nd Approach:
@container
private static final DockerComposeContainer kafka = new DockerComposeContainer(
new File("src/test/resources/compose-test.yml")).withOptions("--compatibility")
.waitingFor(KAFKA_SERVICE, Wait.defaultWaitStrategy().withStartupTimeout(Duration.ofSeconds(360)))
.withLocalCompose(true);

This approach simply throws a null pointer error:

3rd Approach:
@container
private static final DockerComposeContainer kafka = new DockerComposeContainer(
new File("src/test/resources/compose-test.yml")).withOptions("--compatibility")
.waitingFor(KAFKA_SERVICE, Wait.defaultWaitStrategy().withStartupTimeout(Duration.ofSeconds(360)));

This approach throws the below error:


But, the file in the error has Read/Write/Execute permissions for all users.

[kiview]

Would you expect that Podman works together with Docker Compose outside of Testcontainers? Can you give a CLI example of how Podman can work with Docker Compose?

[ssrikanthc]

@kiview, thanks for your response. I am using the below CLI commands to make Podman work with Docker-compose
sudo systemctl enable podman.socket
sudo systemctl start podman.socket
sudo systemctl status podman.socket

##or##

systemctl --user enable podman.socket
systemctl --user start podman.socket
systemctl --user status podman.socket
export DOCKER_HOST=unix:///run/user/$UID/podman/podman.sock

Reference:
https://fedoramagazine.org/use-docker-compose-with-podman-to-orchestrate-containers-on-fedora/
https://www.redhat.com/sysadmin/podman-docker-compose

[bjconlan]

I'm seeing better behaviour than this on podman 4.3.1 (rootless).

But im seeing issue surrounding socat initialisation:

11:38:09.386 [main] INFO  🐳 [docker-compose] - Docker Compose has finished running
11:38:09.530 [main] INFO  🐳 [alpine/socat:1.7.4.3-r0] - Creating container for image: alpine/socat:1.7.4.3-r0
11:38:09.741 [main] ERROR 🐳 [alpine/socat:1.7.4.3-r0] - Could not start container
org.testcontainers.containers.ContainerLaunchException: Aborting attempt to link to container 2t1bdw4yyjmb_formio.mock_1 as it is not running

It looks like there is a similar reported side effect issue for this problem. But i suspect it also might be due to the slirp networking functionality.

Further investigation looks like this is actually a failure in starting the socat container due to using the 'link' functionality not matching names in GenericContainer::findLinksFromThisContainer due to snake vs kebab naming

There also looks like some other issues here with podman around naming (in regards to linking which is moot since podman linking fails independent of correctly named containers/links

Looks like this might stem from go vs python docker-compose implementations

This naming convention difference appears to be a difference between Docker Compose versions v1 (Python) and v2 (Go). The latest docker/compose repo that is packaged with Docker Desktop is the golang version in the docker/compose v2 branch. Looking at the source code here in this branch:

so perhaps a compat flag is required to allow kebab naming in https://github.com/testcontainers/testcontainers-java/blob/main/core/src/main/java/org/testcontainers/containers/DockerComposeContainer.java#L430)

[jfrantzius]

I do have docker-compose from CLI working with Podman Desktop on Mac, so that generally does work.

Any attempt at running Testcontainer's ComposeContainer, though, (with or without withLocalCompose(true) or DockerComposeContainer eventually result in

15:35:57.082 [main] ERROR tc.alpine/socat:1.7.4.3-r0 - Could not start container
com.github.dockerjava.api.exception.BadRequestException: Status 400: {"cause":"link is not supported","message":"bad parameter: link is not supported","response":400}

This seems to be because Testcontainer makes use of Legacy container links, which is deprecated also in Testcontainer's own API (as @ssrikanthc pointed out). With Testcontainers 1.19.3, this seems to be happening in org.testcontainers.containers.ComposeDelegate.withExposedService(String, int, WaitStrategy), line 315, which calls ambassadorContainer.addLink()

The workaround seems to be not to use (ComposeContainer|DockerComposeContainer).withExposedServices(), as that is what is triggering the call to ambassadorContainer.addLink().

There is #465, which lists as "Modify Docker Compose support; can we remove use of the ambassador container now?" as a completed task?

[chenlin2001]

Hi, Is there an update on this issue? We are facing similar problems after moving from docker desktop to podman.
Using DockerComposeContainer with testcontainers version 1.17.6 we get the following error:
[10] Failed to execute script docker-compose Traceback (most recent call last): File "urllib3/connectionpool.py", line 677, in urlopen File "urllib3/connectionpool.py", line 392, in _make_request File "http/client.py", line 1277, in request File "http/client.py", line 1323, in _send_request File "http/client.py", line 1272, in endheaders File "http/client.py", line 1032, in _send_output File "http/client.py", line 972, in send File "docker/transport/unixconn.py", line 43, in connect PermissionError: [Errno 13] Permission denied
Upgrading to version 1.19.0 using DockerComposecontainer fails on out of heap memory as described in: #7239
Tried using ComposeContainer with version 1.19.0 and localmode in order to avoid the heap issues and facing the issue of Link is not supported.

Podman compose without testcontainers work on that docker-compose.yaml

[jfrantzius]

Hi @chenlin2001 , I'm not sure I had the same error, but concerning Permission denied errors you might want to have a look at https://gist.github.com/jfrantzius/0996e224cbe81d885553ece6f8abf99a#status-500-lsetxattr-varrundockersock-operation-not-permitted

[ben-gineer]

We're still seeing this with latest TestContainers (1.20.4), local docker-compose, and ComposeContainer (for v2 support). Tried with and without withExposedServices.

Be great to know how people are managing this?

[MajerMartin]

We were using Testcontainers for Scala, which obviously shares the same issue. We have tried every suggested option on GitHub but unfortunately had to get rid of Testcontainers in the end. However we were not using any advanced features of Testcontainers, so we could easily switch to manual startup and teardown using docker compose commands.

override def beforeAll(): Unit = {
  system.log.info("Starting services...")
  "docker compose up -d".!
}

override def afterAll(): Unit = {
  system.log.info("Shutting down actor system...")
  Await.result(system.terminate(), DefaultTimeout)

  system.log.info("Stopping services...")
  "docker compose down".!
 }

[alersrt]

Any updates with deprecated link is not supported? Now I'm using random port mapping in compose and retrieve mapped port via docker compose -p xxx port xxx xxx but it has some limitations.

[frankjkelly]

Also seeing this after the switchover to podman.

I tried the instructions here to no avail
https://podman-desktop.io/tutorial/testcontainers-with-podman

Running podman in--rootful state

[frankjkelly]

FYI I tried switching to Orbstack and ALSO seeing this.
Also using ComposeContainer and withLocalCompose(true).

NOTE: We can use Podman and OrbStack fine with Golang Test containers and docker compose based tests so can't really blame this on the docker service provider software.

[frankjkelly]

FYI a suggestion from @eddumelendez here in slack

Docker compose in java use linked containers, which it has been deprecated in Docker but AFAIK Podman 
doesn't use it and that's why it fails. I guess is similar for Orbstack