document cron offsetting

This commit is contained in:
Joel Falknau 2024-12-30 18:57:09 +10:00
parent 6f2f39d7fa
commit 156e7c891e
No known key found for this signature in database

View File

@ -123,6 +123,7 @@ Example setting:
CELERYBEAT_SCHEDULE['structures_update_all_structures'] = {
'task': 'structures.tasks.update_all_structures',
'schedule': crontab(minute='*/30'),
'apply_offset': True,
}
```
@ -130,6 +131,7 @@ CELERYBEAT_SCHEDULE['structures_update_all_structures'] = {
- `'task'`: Name of your task (full path)
- `'schedule'`: Schedule definition (see Celery documentation on [Periodic Tasks](https://docs.celeryproject.org/en/latest/userguide/periodic-tasks.html) for details)
- `'apply_offset'`: Boolean, Apply a Delay unique to the install, in order to reduce impact on ESI. See [Apply Offset](#apply-offset)
## How can I use priorities for tasks?
@ -174,9 +176,54 @@ Large numbers of installs running the same crontab (ie. `0 * * * *`) can all sla
Consider Artificially smoothing out your tasks with a few methods
### Offset Crontabs
### Apply Offset
Avoid running your tasks on the hour or other nice neat human numbers, consider 23 minutes on the hour instead of at zero (`28 * * * *`)
`allianceauth.crontab` contains a series of Offsets stored in the DB that are both static for an install, but random across all AA installs.
This enables us to spread our load on ESI (or other external resources) across a greater window, making it unlikely that two installs will hit ESI at the same time.
Tasks defined in local.py, can have `'apply_offset': True` added to their Task Definition
```python
CELERYBEAT_SCHEDULE['taskname'] = {
'task': 'module.tasks.task',
'schedule': crontab(minute='*/30'),
'apply_offset': True,
}
```
Tasks added to directly to Django Celery Beat Models (Using a Management Task etc) can pass their Cron Schedule through offset_cron(crontab)
```{eval-rst}
.. automodule:: allianceauth.crontab.utils
:members:
:undoc-members:
```
```python
from django_celery_beat.models import CrontabSchedule, PeriodicTask
from celery.schedules import crontab
schedule = CrontabSchedule.from_schedule(offset_cron(crontab(minute='0', hour='0')))
schedule, created = CrontabSchedule.objects.get_or_create(
minute=schedule.minute,
hour=schedule.hour,
day_of_month=schedule.day_of_month,
month_of_year=schedule.month_of_year,
day_of_week=schedule.day_of_week,
timezone=schedule.timezone,
)
PeriodicTask.objects.update_or_create(
task='module.tasks.task',
defaults={
'crontab': schedule,
'name': 'task name',
'enabled': True
}
)
```
### Subset Tasks