I am trying to auto-reload celery that is part of a Django application when code changes are detected. I am starting celery with the following management command:
"""
https://avilpage.com/2017/05/how-to-auto-reload-celery-workers-in-development.html
"""
import shlex
import subprocess
from django.core.management.base import BaseCommand
from django.utils import autoreload
def restart_celery_worker():
cmd = "pkill -9 celery"
subprocess.call(shlex.split(cmd))
cmd = "celery -A my_app.celery_app worker -l info --concurrency=2"
subprocess.call(shlex.split(cmd))
class Command(BaseCommand):
def handle(self, *args, **options):
print("Starting celery worker with autoreload...")
autoreload.run_with_reloader(restart_celery_worker)
I’m seeing the following error:
Starting celery worker with autoreload...
Exception in thread django-main-thread:
Traceback (most recent call last):
File "/usr/local/lib/python3.10/threading.py", line 1009, in _bootstrap_inner
self.run()
File "/usr/local/lib/python3.10/threading.py", line 946, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.10/site-packages/django/utils/autoreload.py", line 64, in wrapper
fn(*args, **kwargs)
File "/code/core/management/commands/start_worker_local.py", line 14, in restart_celery_worker
subprocess.call(shlex.split(cmd))
File "/usr/local/lib/python3.10/subprocess.py", line 345, in call
with Popen(*popenargs, **kwargs) as p:
File "/usr/local/lib/python3.10/subprocess.py", line 966, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "/usr/local/lib/python3.10/subprocess.py", line 1842, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'pkill'
I started seeing this issue when trying to update to Python 3.10. Here’s the Dockerfile that I’m using:
FROM python:3.10.0-slim-buster
ENV LANG C.UTF-8
ENV DEBIAN_FRONTEND=noninteractive
RUN mkdir -m 775 /code
ENV PYTHONUNBUFFERED=1
ENV PYTHONPATH=/code
WORKDIR /code
RUN python3 -m pip install --upgrade pip setuptools
RUN pip install pipenv
COPY Pipfile ./
RUN pipenv lock --python /usr/local/bin/python
RUN pipenv install --system --deploy --dev
COPY app /code
>Solution :
pkill
is part of the procps
package which can be installed via:
apt-get update && apt-get install procps
You can confirm that the procps
package is not included in the python:3.10.0-slim-buster
image via:
$ docker run --rm python:3.10.0-slim-buster dpkg --get-selections | grep procps
This is because the slim
image variant does not include the procps
package. However, we can confirm that the procps
package is included in the buster
image:
$ docker run --rm python:3.10.0-buster dpkg --get-selections | grep procps
libprocps7:amd64 install
procps install