Docker's Python SDK "container.start()" exit immediately

I want to create a docker container via the docker’s Python SDK.
The bash equivalence (which works on the cli) of what I am trying to achieve is as follows. (the image entry point is /bin/bash)

docker container create --name <container_name>-i --mount source=<src>,target=/project_data <image_name>

docker container start -i <container_name>

While this works on the Cli, after creating the container via the Python SDK and starting it, the container is exited shortly after, hence when I try to execute commands on the container I get an error which states that the container is not running.

container = client.containers.run("<image_name>", name="<container_name>", detach=True, volumes=volumes)

I wasn’t able to start the container in detached mode.
my goal eventually, after starting the container is to execute multiple cmd in the same container instance. i.e.

result = container.exec_run('ls')
result = container.exec_run('./run_scripts > out.txt')
result = container.exec_run('./run_scripts_2 > out.txt')

# some data processing
... <omitted> ...

# execute more scripts
result = container.exec_run('./run_scripts _3> out.txt')

When I try to run container.start(), the container is started but it is exited shortly after, hence when I try to execute container.exec_run function I get the following error:

---------------------------------------------------------------------------
HTTPError                                 Traceback (most recent call last)
File ~/miniconda3/envs/foldynamics/lib/python3.8/site-packages/docker/api/client.py:268, in APIClient._raise_for_status(self, response)
    267 try:
--> 268     response.raise_for_status()
    269 except requests.exceptions.HTTPError as e:

File ~/miniconda3/envs/foldynamics/lib/python3.8/site-packages/requests/models.py:1021, in Response.raise_for_status(self)
   1020 if http_error_msg:
-> 1021     raise HTTPError(http_error_msg, response=self)

HTTPError: 409 Client Error: Conflict for url: http+docker://localhost/v1.41/containers/43cb7fd5549053bf426ed3d39c1d4f5a982d329f7483e890604d0d5b2a38f7b1/exec

The above exception was the direct cause of the following exception:

APIError                                  Traceback (most recent call last)
Cell In[87], line 1
----> 1 result = container.exec_run("echo pwd", tty=True, detach=True)

File ~/miniconda3/envs/foldynamics/lib/python3.8/site-packages/docker/models/containers.py:193, in Container.exec_run(self, cmd, stdout, stderr, stdin, tty, privileged, user, detach, stream, socket, environment, workdir, demux)
    152 def exec_run(self, cmd, stdout=True, stderr=True, stdin=False, tty=False,
    153              privileged=False, user='', detach=False, stream=False,
    154              socket=False, environment=None, workdir=None, demux=False):
    155     """
    156     Run a command inside this container. Similar to
    157     ``docker exec``.
   (...)
    191             If the server returns an error.
    192     """
--> 193     resp = self.client.api.exec_create(
    194         self.id, cmd, stdout=stdout, stderr=stderr, stdin=stdin, tty=tty,
    195         privileged=privileged, user=user, environment=environment,
    196         workdir=workdir,
    197     )
    198     exec_output = self.client.api.exec_start(
    199         resp['Id'], detach=detach, tty=tty, stream=stream, socket=socket,
    200         demux=demux
    201     )
    202     if socket or stream:

File ~/miniconda3/envs/foldynamics/lib/python3.8/site-packages/docker/utils/decorators.py:19, in check_resource.<locals>.decorator.<locals>.wrapped(self, resource_id, *args, **kwargs)
     15 if not resource_id:
     16     raise errors.NullResource(
     17         'Resource ID was not provided'
     18     )
---> 19 return f(self, resource_id, *args, **kwargs)

File ~/miniconda3/envs/foldynamics/lib/python3.8/site-packages/docker/api/exec_api.py:78, in ExecApiMixin.exec_create(self, container, cmd, stdout, stderr, stdin, tty, privileged, user, environment, workdir, detach_keys)
     76 url = self._url('/containers/{0}/exec', container)
     77 res = self._post_json(url, data=data)
---> 78 return self._result(res, True)

File ~/miniconda3/envs/foldynamics/lib/python3.8/site-packages/docker/api/client.py:274, in APIClient._result(self, response, json, binary)
    272 def _result(self, response, json=False, binary=False):
    273     assert not (json and binary)
--> 274     self._raise_for_status(response)
    276     if json:
    277         return response.json()

File ~/miniconda3/envs/foldynamics/lib/python3.8/site-packages/docker/api/client.py:270, in APIClient._raise_for_status(self, response)
    268     response.raise_for_status()
    269 except requests.exceptions.HTTPError as e:
--> 270     raise create_api_error_from_http_exception(e) from e

File ~/miniconda3/envs/foldynamics/lib/python3.8/site-packages/docker/errors.py:39, in create_api_error_from_http_exception(e)
     37     else:
     38         cls = NotFound
---> 39 raise cls(e, response=response, explanation=explanation) from e

APIError: 409 Client Error for http+docker://localhost/v1.41/containers/43cb7fd5549053bf426ed3d39c1d4f5a982d329f7483e890604d0d5b2a38f7b1/exec: Conflict ("Container 43cb7fd5549053bf426ed3d39c1d4f5a982d329f7483e890604d0d5b2a38f7b1 is not running")

>Solution :

The docker-py repository seems to have had a issue with the same issue, this addresses this.

Basically, the sdk doesn’t implement interactive feature. The workaround is to pass stdin_open = True, tty = True to the container = client.containers.run command.

E.g. container = client.containers.run("<image_name>", name="<container_name>", stdin_open = True, tty = True detach=True, volumes=volumes)