dc8ed2d510
Combines celery confs under one process group for easy restarting. Updates docs to reflect new commands.
2.8 KiB
Supervisor
Supervisor is a client/server system that allows its users to control a number of processes on UNIX-like operating systems.
What that means is supervisor will take care of ensuring the celery workers are running (and mumble authenticator) and start the automatically on reboot. Handy, eh?
Installation
Most OSes have a supervisor package available in their distribution.
Ubuntu:
sudo apt-get install supervisor
CentOS:
sudo yum install supervisor
sudo systemctl enable supervisord.service
sudo systemctl start supervisord.service
Configuration
Auth provides example config files for the celery workers, the periodic task scheduler (celery beat), and the mumble authenticator. All of these are available in thirdparty/Supervisor.
For most users, all you have to do is copy the config files to /etc/supervisor/conf.d then restart the service. Copy auth.conf for the celery workers, and auth-mumble.conf for the mumble authenticator. For all three just use a wildcard:
sudo cp thirdparty/Supervisor/* /etc/supervisor/conf.d
Ubuntu:
sudo service supervisor restart
CentOS:
sudo systemctl restart supervisor.service
Checking Status
To ensure the processes are working, check their status:
sudo supervisorctl status
Processes will be STARTING, RUNNING, or ERROR. If an error has occurred, check their log files:
- celery workers:
log/worker.log - celery beat:
log/beat.log - authenticator:
log/authenticator.log
Restarting Processes
To restart the celery group:
sudo supervisorctl restart auth:*
To restart just celerybeat:
sudo supervisorctl restart auth:celerybeat
To restart just celeryd:
sudo supervisorctl restart auth:celeryd
To restart just mumble authenticator:
sudo supervisorctl restart auth-mumble
Customizing Config Files
The only real customization needed is if running in a virtual environment. The python path will have to be changed in order to start in the venv.
Edit the config files and find the line saying command. Replace python with /path/to/venv/bin/python. For Celery replace celery with /path/to/venv/bin/celery. This can be relative to the directory specified in the config file.
Note that for config changes to be loaded, the supervisor service must be restarted.
Troubleshooting
auth-celerybeat fails to start
Most often this is caused by a permissions issue on the allianceauth directory (the error will talk about celerybeat.pid). The easiest fix is to edit its config file and change the user from allianceserver to root.
Workers are using old settings
Every time the codebase is updated or settings file changed, workers will have to be restarted. Easiest way is to restart the supervisor service (see configuration above for commands)