本文是为django项目配置celery时的总结,参考官方文档Celery 4.3.0 documentation.

一. 配置服务文件: celery.service

1. 创建文件并编辑: celery.service

sudo vim /etc/systemd/system/celery.service

2. 复制下方的内容到文件中

[Unit]
Description=Celery Service
After=network.target

[Service]
Type=forking
User=celery
Group=celery
EnvironmentFile=/etc/conf.d/celery
WorkingDirectory=/opt/celery
ExecStart=/bin/sh -c '${CELERY_BIN} multi start ${CELERYD_NODES} \
  -A ${CELERY_APP} --pidfile=${CELERYD_PID_FILE} \
  --logfile=${CELERYD_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL} ${CELERYD_OPTS}'
ExecStop=/bin/sh -c '${CELERY_BIN} multi stopwait ${CELERYD_NODES} \
  --pidfile=${CELERYD_PID_FILE}'
ExecReload=/bin/sh -c '${CELERY_BIN} multi restart ${CELERYD_NODES} \
  -A ${CELERY_APP} --pidfile=${CELERYD_PID_FILE} \
  --logfile=${CELERYD_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL} ${CELERYD_OPTS}'

[Install]
WantedBy=multi-user.target

3. 修改配置

User=someone # 启动celery的用户
Group=somegroup # 启动celery的用户组
WorkingDirectory=/path/my_project # 你的项目所在文件夹

4. 创建日志存放的文件夹

celery.serivce文件设置的,存放日志的文件夹celery不会自动创建.需要手动创建,否则启动服务时会报错.

这里用官方的方法, 使用systemd-tmpfiles指令创建服务需要的两个文件夹

创建文件并编辑: celery.conf

sudo vim /etc/tmpfiles.d/celery.conf

复制下面内容到配置文件中

d /var/run/celery 0755 celery celery -
d /var/log/celery 0755 celery celery -

修改配置文件内容

用户与用户组应与celery.service内配置的一致.

d /var/run/celery 0755 文件夹所属用户 文件夹所属用户组 -
d /var/log/celery 0755 文件夹所属用户 文件夹所属用户组 -

生成文件夹

sudo systemd-tmpfiles --create

配置项目文件: celery

1. 创建并编辑文件: celery

sudo vim /etc/conf.d/celery

2. 复制下面内容到文件中

# Name of nodes to start
# here we have a single node
CELERYD_NODES="w1"
# or we could have three nodes:
#CELERYD_NODES="w1 w2 w3"

# Absolute or relative path to the 'celery' command:
CELERY_BIN="/usr/local/bin/celery"
#CELERY_BIN="/virtualenvs/def/bin/celery"

# App instance to use
# comment out this line if you don't use an app
CELERY_APP="proj"
# or fully qualified:
#CELERY_APP="proj.tasks:app"

# How to call manage.py
CELERYD_MULTI="multi"

# Extra command-line arguments to the worker
CELERYD_OPTS="--time-limit=300 --concurrency=8"

# - %n will be replaced with the first part of the nodename.
# - %I will be replaced with the current child process index
#   and is important when using the prefork pool to avoid race conditions.
CELERYD_PID_FILE="/var/run/celery/%n.pid"
CELERYD_LOG_FILE="/var/log/celery/%n%I.log"
CELERYD_LOG_LEVEL="INFO"

# you may wish to add these options for Celery Beat
# CELERYBEAT_PID_FILE="/var/run/celery/beat.pid"
# CELERYBEAT_LOG_FILE="/var/log/celery/beat.log"

3. 修改配置文件

一般来说,celery是通过pip安装的.可以通过which pip指令获取pip的路径,再找到celery.

路径类似于/usr/local/bin/pip3/bin/celery

CELERY_BIN="你的celery执行文件所在位置"
CELERY_APP="你的项目名称"

重载配置文件

每次修改配置文件后,都要运行下面的指令使修改生效.

systemctl daemon-reload

通过指令管理服务

经过上面的配置后,就可以通过systemctl来管理celery服务啦.

systemctl start celery.service # 启动
systemctl stop celery.service # 停止
systemctl restart celery.service # 重启
systemctl status celery.service # 查看服务状态

使用systemd启动celery守护进程