Compare commits

...

52 Commits

Author SHA1 Message Date
Ariel Rin
86abc4f169 Update file __init__.py 2025-01-10 12:04:44 +00:00
Ariel Rin
fbafcac5b1 Merge branch 'master' into 'cronbuilder'
# Conflicts:
#   allianceauth/project_template/project_name/settings/base.py
2025-01-10 11:48:03 +00:00
Ariel Rin
a5971314f5 Merge branch 'structure-timer-updates' into 'master'
Timerboard Updates

See merge request allianceauth/allianceauth!1674
2025-01-10 11:22:24 +00:00
Peter Pfeufer
a03c766840 [FIX] Spelling mistakes
Thx @soratidus999
2025-01-10 11:41:31 +01:00
Ariel Rin
ad47ff2c54 Merge branch 'dockerbaremetalanalytics' into 'master'
Docker/Baremetal Analytics

See merge request allianceauth/allianceauth!1671
2025-01-10 08:29:02 +00:00
Ariel Rin
3efdb8f12b Docker/Baremetal Analytics 2025-01-10 08:29:02 +00:00
Ariel Rin
823fc82d19 Merge branch 'sri-hashes' into 'master'
[ADD] `django-sri` to provide integrity hashes for local static files

See merge request allianceauth/allianceauth!1673
2025-01-10 08:27:28 +00:00
Ariel Rin
a93e510895 Merge branch 'fix-moment.js-language-code' into 'master'
[FIX] Moment.JS Localisation

See merge request allianceauth/allianceauth!1675
2025-01-10 08:21:20 +00:00
Peter Pfeufer
d99f5858d8 [FIX] Moment.JS Localisation
Load the right JS when our language code has 2 parts
- fr_FR
- it_IT
- pl_PL
And so on
2025-01-08 23:04:29 +01:00
Peter Pfeufer
4578ecf21d [ADD] Missing migration 2025-01-06 18:17:21 +01:00
Peter Pfeufer
b737504d52 [REMOVE] Unused imports 2025-01-06 18:08:52 +01:00
Peter Pfeufer
c6b6443901 [CHANGE] Move structure label BG detection to Python instead of Django
Python is faster and needs less memory …
It's also much mire readable that way.
2025-01-06 18:08:20 +01:00
Peter Pfeufer
f51523dc07 [CHANGE] Use TextChoices displayed name in template 2025-01-06 17:03:00 +01:00
Peter Pfeufer
bd4dd60c98 [CHANGE] Rename I-Hub to Sovereignty Hub 2025-01-06 16:48:02 +01:00
Peter Pfeufer
a4ea48e14e [CHANGE] Group timer types by BG color BS class 2025-01-06 16:43:59 +01:00
Peter Pfeufer
646d3f5408 [ADD] Mercenary Dens to structure type selection 2025-01-06 16:24:30 +01:00
Peter Pfeufer
0f057ffa84 [ADD] django-sri to provide integrity hashes for local static files 2025-01-02 16:54:37 +01:00
Joel Falknau
156e7c891e document cron offsetting 2024-12-30 18:57:09 +10:00
Joel Falknau
6f2f39d7fa shift to custom Scheduler 2024-12-30 18:12:11 +10:00
Joel Falknau
a66aa6de80 rename submodule properly 2024-12-29 22:19:51 +10:00
Joel Falknau
d12d6e7cdb move to its own module 2024-12-29 22:04:52 +10:00
Joel Falknau
7022cb7050 not relevant comment 2024-12-29 15:14:29 +10:00
Joel Falknau
30a79362f4 remove development note 2024-12-29 15:13:13 +10:00
Joel Falknau
c3fa8acd8e Tweak migrations to use random function not preset 2024-12-29 15:05:53 +10:00
Joel Falknau
8fd1411f09 shift schedule to ready() in order to access DB 2024-12-29 14:51:57 +10:00
Joel Falknau
1aa90adac3 Cron Offset Module 2024-12-29 14:51:33 +10:00
Joel Falknau
7033406ba6 Version Bump 4.5.0 2024-12-10 13:07:03 +10:00
Ariel Rin
6b395ca1d4 Merge branch 'executableflag' into 'master'
Executableflag

See merge request allianceauth/allianceauth!1667
2024-12-09 23:57:02 +00:00
Ariel Rin
795a7e006f Merge branch 'randomdelay' into 'master'
Spread esi tasks over 10 minutes

See merge request allianceauth/allianceauth!1666
2024-12-09 23:56:13 +00:00
Ariel Rin
2a894cd62c Merge branch 'dockermariadbcnf' into 'master'
DockerMariaDB Config Template

See merge request allianceauth/allianceauth!1668
2024-12-09 23:54:33 +00:00
Ariel Rin
9ada26e849 DockerMariaDB Config Template 2024-12-09 23:54:33 +00:00
Ariel Rin
7120b3956c Merge branch 'group_display' into 'master'
fix group display for Groups that are Group Leaders

See merge request allianceauth/allianceauth!1670
2024-12-09 23:54:18 +00:00
root
4da67cfaf6 fix group display for Groups that are Group Leaders 2024-12-08 13:01:59 -06:00
Joel Falknau
0a940810bd dont need this now the flag is set correctly, more consistent 2024-12-05 11:49:54 +10:00
Joel Falknau
a868438492 force these flags on setup 2024-12-05 11:49:32 +10:00
Joel Falknau
dc1ed8c570 +x 2024-12-05 11:48:45 +10:00
Joel Falknau
8489f204dd fix test patch 2024-12-04 22:10:06 +10:00
Ariel Rin
1478588016 Merge branch 'add-js-type' into 'master'
[ADD] `js_type` parameter to allow JS modules to be loaded

See merge request allianceauth/allianceauth!1664
2024-12-04 11:24:21 +00:00
Ariel Rin
a16eb4b7f7 Merge branch 'image-overflow-fix' into 'master'
[FIX] Prevent images from overflowing their parent element

See merge request allianceauth/allianceauth!1665
2024-12-04 11:23:38 +00:00
Joel Falknau
292fb7b29d Add docs for smoothing out task execution 2024-12-04 18:35:07 +10:00
Joel Falknau
c6890dd2c6 Spread esi tasks over 10 minutes 2024-12-04 18:01:01 +10:00
Ariel Rin
702564d15e correct top level indentation to be able to be directly copy-pasteable 2024-12-03 09:50:55 +00:00
Peter Pfeufer
cef2e86ea1 [FIX] Prevent images from overflowing their parent element 2024-11-26 22:28:35 +01:00
Peter Pfeufer
50681b023b [CHANGE] Move the if inside the script tag
Makes the code better readable and maintainable.
2024-11-23 01:55:04 +01:00
Peter Pfeufer
2822775fb8 [ADD] js_type parameter to allow JS modules to be loaded 2024-11-22 15:01:32 +01:00
Ariel Rin
ef7c8be7b5 Merge branch 'fix/default-collation' into 'master'
Fix Different collations being used by docker and baremetal installations

See merge request allianceauth/allianceauth!1662
2024-11-17 01:57:59 +00:00
Ariel Rin
d639617eba Merge branch 'master' into 'master'
Celery documentation improvements

See merge request allianceauth/allianceauth!1663
2024-11-17 01:57:05 +00:00
r0kym
2125192f72 Fix typo in the long-running tasks example 2024-11-15 15:13:25 +01:00
r0kym
8d63801b00 Change logging imports and commands
Now uses `get_extension_logger` as is recommanded by https://allianceauth.readthedocs.io/en/v4.4.2/development/custom/logging.html
2024-11-15 15:12:03 +01:00
Matteo Ghia
e053fb7d96 also in baremetal docs 2024-11-09 12:45:59 +01:00
Matteo Ghia
ae7ed5c297 add default keyword required by mariadb 2024-11-09 12:45:31 +01:00
Matteo Ghia
d624ba4427 set default collation in startup script 2024-11-09 12:01:57 +01:00
51 changed files with 744 additions and 176 deletions

View File

@@ -5,7 +5,7 @@ manage online service access.
# This will make sure the app is always imported when
# Django starts so that shared_task will use this app.
__version__ = '4.4.2'
__version__ = '4.5.0'
__title__ = 'Alliance Auth'
__url__ = 'https://gitlab.com/allianceauth/allianceauth'
NAME = f'{__title__} v{__version__}'

View File

@@ -1,15 +1,16 @@
from django.contrib import admin
from .models import AnalyticsIdentifier, AnalyticsTokens
from solo.admin import SingletonModelAdmin
@admin.register(AnalyticsIdentifier)
class AnalyticsIdentifierAdmin(admin.ModelAdmin):
class AnalyticsIdentifierAdmin(SingletonModelAdmin):
search_fields = ['identifier', ]
list_display = ('identifier',)
list_display = ['identifier', ]
@admin.register(AnalyticsTokens)
class AnalyticsTokensAdmin(admin.ModelAdmin):
search_fields = ['name', ]
list_display = ('name', 'type',)
list_display = ['name', 'type', ]

View File

@@ -0,0 +1,17 @@
# Generated by Django 4.2.16 on 2024-12-11 02:17
from django.db import migrations
class Migration(migrations.Migration):
dependencies = [
('analytics', '0009_remove_analyticstokens_ignore_paths_and_more'),
]
operations = [
migrations.AlterModelOptions(
name='analyticsidentifier',
options={'verbose_name': 'Analytics Identifier'},
),
]

View File

@@ -1,23 +1,19 @@
from typing import Literal
from django.db import models
from django.core.exceptions import ValidationError
from django.utils.translation import gettext_lazy as _
from solo.models import SingletonModel
from uuid import uuid4
class AnalyticsIdentifier(models.Model):
class AnalyticsIdentifier(SingletonModel):
identifier = models.UUIDField(
default=uuid4,
editable=False)
identifier = models.UUIDField(default=uuid4, editable=False)
def save(self, *args, **kwargs):
if not self.pk and AnalyticsIdentifier.objects.exists():
# Force a single object
raise ValidationError('There is can be only one \
AnalyticsIdentifier instance')
self.pk = self.id = 1 # If this happens to be deleted and recreated, force it to be 1
return super().save(*args, **kwargs)
def __str__(self) -> Literal['Analytics Identifier']:
return "Analytics Identifier"
class Meta:
verbose_name = "Analytics Identifier"
class AnalyticsTokens(models.Model):

View File

@@ -5,6 +5,7 @@ from django.apps import apps
from celery import shared_task
from .models import AnalyticsTokens, AnalyticsIdentifier
from .utils import (
existence_baremetal_or_docker,
install_stat_addons,
install_stat_tokens,
install_stat_users)
@@ -67,8 +68,8 @@ def analytics_event(namespace: str,
value=value).apply_async(priority=9)
@shared_task()
def analytics_daily_stats():
@shared_task
def analytics_daily_stats() -> None:
"""Celery Task: Do not call directly
Gathers a series of daily statistics
@@ -77,6 +78,7 @@ def analytics_daily_stats():
users = install_stat_users()
tokens = install_stat_tokens()
addons = install_stat_addons()
existence_type = existence_baremetal_or_docker()
logger.debug("Running Daily Analytics Upload")
analytics_event(namespace='allianceauth.analytics',
@@ -84,6 +86,11 @@ def analytics_daily_stats():
label='existence',
value=1,
event_type='Stats')
analytics_event(namespace='allianceauth.analytics',
task='send_install_stats',
label=existence_type,
value=1,
event_type='Stats')
analytics_event(namespace='allianceauth.analytics',
task='send_install_stats',
label='users',
@@ -99,7 +106,6 @@ def analytics_daily_stats():
label='addons',
value=addons,
event_type='Stats')
for appconfig in apps.get_app_configs():
if appconfig.label in [
"django_celery_beat",
@@ -135,7 +141,7 @@ def analytics_daily_stats():
event_type='Stats')
@shared_task()
@shared_task
def send_ga_tracking_celery_event(
measurement_id: str,
secret: str,
@@ -165,7 +171,7 @@ def send_ga_tracking_celery_event(
}
payload = {
'client_id': AnalyticsIdentifier.objects.get(id=1).identifier.hex,
'client_id': AnalyticsIdentifier.get_solo().identifier.hex,
"user_properties": {
"allianceauth_version": {
"value": __version__

View File

@@ -1,9 +1,8 @@
from allianceauth.analytics.models import AnalyticsIdentifier
from django.core.exceptions import ValidationError
from django.test.testcases import TestCase
from uuid import UUID, uuid4
from uuid import uuid4
# Identifiers
@@ -14,14 +13,4 @@ uuid_2 = "7aa6bd70701f44729af5e3095ff4b55c"
class TestAnalyticsIdentifier(TestCase):
def test_identifier_random(self):
self.assertNotEqual(AnalyticsIdentifier.objects.get(), uuid4)
def test_identifier_singular(self):
AnalyticsIdentifier.objects.all().delete()
AnalyticsIdentifier.objects.create(identifier=uuid_1)
# Yeah i have multiple asserts here, they all do the same thing
with self.assertRaises(ValidationError):
AnalyticsIdentifier.objects.create(identifier=uuid_2)
self.assertEqual(AnalyticsIdentifier.objects.count(), 1)
self.assertEqual(AnalyticsIdentifier.objects.get(
pk=1).identifier, UUID(uuid_1))
self.assertNotEqual(AnalyticsIdentifier.get_solo(), uuid4)

View File

@@ -1,3 +1,4 @@
import os
from django.apps import apps
from allianceauth.authentication.models import User
from esi.models import Token
@@ -34,3 +35,16 @@ def install_stat_addons() -> int:
The Number of Installed Apps"""
addons = len(list(apps.get_app_configs()))
return addons
def existence_baremetal_or_docker() -> str:
"""Checks the Installation Type of an install
Returns
-------
str
existence_baremetal or existence_docker"""
docker_tag = os.getenv('AA_DOCKER_TAG')
if docker_tag:
return "existence_docker"
return "existence_baremetal"

View File

@@ -1,5 +1,4 @@
from django.apps import AppConfig
from django.core.checks import Warning, Error, register
class AllianceAuthConfig(AppConfig):

View File

@@ -0,0 +1,3 @@
"""
Alliance Auth Crontab Utilities
"""

View File

@@ -0,0 +1,14 @@
"""
Crontab App Config
"""
from django.apps import AppConfig
class CrontabConfig(AppConfig):
"""
Crontab App Config
"""
name = "allianceauth.crontab"
label = "crontab"

View File

@@ -0,0 +1,23 @@
from random import random
from django.db import models
from django.utils.translation import gettext_lazy as _
from solo.models import SingletonModel
def random_default() -> float:
return random()
class CronOffset(SingletonModel):
minute = models.FloatField(_("Minute Offset"), default=random_default)
hour = models.FloatField(_("Hour Offset"), default=random_default)
day_of_month = models.FloatField(_("Day of Month Offset"), default=random_default)
month_of_year = models.FloatField(_("Month of Year Offset"), default=random_default)
day_of_week = models.FloatField(_("Day of Week Offset"), default=random_default)
def __str__(self) -> str:
return "Cron Offsets"
class Meta:
verbose_name = "Cron Offsets"

View File

@@ -0,0 +1,63 @@
from django.core.exceptions import ObjectDoesNotExist
from django_celery_beat.schedulers import (
DatabaseScheduler
)
from django_celery_beat.models import CrontabSchedule
from django.db.utils import OperationalError, ProgrammingError
from celery import schedules
from celery.utils.log import get_logger
from allianceauth.crontab.models import CronOffset
from allianceauth.crontab.utils import offset_cron
logger = get_logger(__name__)
class OffsetDatabaseScheduler(DatabaseScheduler):
"""
Customization of Django Celery Beat, Database Scheduler
Takes the Celery Schedule from local.py and applies our AA Framework Cron Offset, if apply_offset is true
Otherwise it passes it through as normal
"""
def update_from_dict(self, mapping):
s = {}
try:
cron_offset = CronOffset.get_solo()
except (OperationalError, ProgrammingError, ObjectDoesNotExist) as exc:
# This is just incase we haven't migrated yet or something
logger.warning(
"OffsetDatabaseScheduler: Could not fetch CronOffset (%r). "
"Defering to DatabaseScheduler",
exc
)
return super().update_from_dict(mapping)
for name, entry_fields in mapping.items():
try:
apply_offset = entry_fields.pop("apply_offset", False)
entry = self.Entry.from_entry(name, app=self.app, **entry_fields)
if entry.model.enabled and apply_offset:
schedule_obj = entry.schedule
if isinstance(schedule_obj, schedules.crontab):
offset_cs = CrontabSchedule.from_schedule(offset_cron(schedule_obj))
offset_cs, created = CrontabSchedule.objects.get_or_create(
minute=offset_cs.minute,
hour=offset_cs.hour,
day_of_month=offset_cs.day_of_month,
month_of_year=offset_cs.month_of_year,
day_of_week=offset_cs.day_of_week,
timezone=offset_cs.timezone,
)
entry.model.crontab = offset_cs
entry.model.save()
logger.debug(f"Offset applied for '{name}' due to 'apply_offset' = True.")
s[name] = entry
except Exception as e:
logger.exception("Error updating schedule for %s: %r", name, e)
self.schedule.update(s)

View File

View File

@@ -0,0 +1,63 @@
from unittest.mock import patch
from django.test import TestCase
from allianceauth.crontab.models import CronOffset
class CronOffsetModelTest(TestCase):
def test_cron_offset_is_singleton(self):
"""
Test that CronOffset is indeed a singleton and that
multiple calls to get_solo() return the same instance.
"""
offset1 = CronOffset.get_solo()
offset2 = CronOffset.get_solo()
# They should be the exact same object in memory
self.assertEqual(offset1.pk, offset2.pk)
def test_default_values_random(self):
"""
Test that the default values are set via random_default() when
no explicit value is provided. We'll patch 'random.random' to
produce predictable output.
"""
with patch('allianceauth.crontab.models.random', return_value=0.1234):
# Force creation of a new CronOffset by clearing the existing one
CronOffset.objects.all().delete()
offset = CronOffset.get_solo() # This triggers creation
# All fields should be 0.1234, because we patched random()
self.assertAlmostEqual(offset.minute, 0.1234)
self.assertAlmostEqual(offset.hour, 0.1234)
self.assertAlmostEqual(offset.day_of_month, 0.1234)
self.assertAlmostEqual(offset.month_of_year, 0.1234)
self.assertAlmostEqual(offset.day_of_week, 0.1234)
def test_update_offset_values(self):
"""
Test that we can update the offsets and retrieve them.
"""
offset = CronOffset.get_solo()
offset.minute = 0.5
offset.hour = 0.25
offset.day_of_month = 0.75
offset.month_of_year = 0.99
offset.day_of_week = 0.33
offset.save()
# Retrieve again to ensure changes persist
saved_offset = CronOffset.get_solo()
self.assertEqual(saved_offset.minute, 0.5)
self.assertEqual(saved_offset.hour, 0.25)
self.assertEqual(saved_offset.day_of_month, 0.75)
self.assertEqual(saved_offset.month_of_year, 0.99)
self.assertEqual(saved_offset.day_of_week, 0.33)
def test_str_representation(self):
"""
Verify the __str__ method returns 'Cron Offsets'.
"""
offset = CronOffset.get_solo()
self.assertEqual(str(offset), "Cron Offsets")

View File

@@ -0,0 +1,80 @@
# myapp/tests/test_tasks.py
import logging
from unittest.mock import patch
from django.test import TestCase
from django.db import ProgrammingError
from celery.schedules import crontab
from allianceauth.crontab.utils import offset_cron
from allianceauth.crontab.models import CronOffset
logger = logging.getLogger(__name__)
class TestOffsetCron(TestCase):
def test_offset_cron_normal(self):
"""
Test that offset_cron modifies the minute/hour fields
based on the CronOffset values when everything is normal.
"""
# We'll create a mock CronOffset instance
mock_offset = CronOffset(minute=0.5, hour=0.5)
# Our initial crontab schedule
original_schedule = crontab(
minute=[0, 5, 55],
hour=[0, 3, 23],
day_of_month='*',
month_of_year='*',
day_of_week='*'
)
# Patch CronOffset.get_solo to return our mock offset
with patch('allianceauth.crontab.models.CronOffset.get_solo', return_value=mock_offset):
new_schedule = offset_cron(original_schedule)
# Check the new minute/hour
# minute 0 -> 0 + round(60 * 0.5) = 30 % 60 = 30
# minute 5 -> 5 + 30 = 35 % 60 = 35
# minute 55 -> 55 + 30 = 85 % 60 = 25 --> sorted => 25,30,35
self.assertEqual(new_schedule._orig_minute, '25,30,35')
# hour 0 -> 0 + round(24 * 0.5) = 12 % 24 = 12
# hour 3 -> 3 + 12 = 15 % 24 = 15
# hour 23 -> 23 + 12 = 35 % 24 = 11 --> sorted => 11,12,15
self.assertEqual(new_schedule._orig_hour, '11,12,15')
# Check that other fields are unchanged
self.assertEqual(new_schedule._orig_day_of_month, '*')
self.assertEqual(new_schedule._orig_month_of_year, '*')
self.assertEqual(new_schedule._orig_day_of_week, '*')
def test_offset_cron_programming_error(self):
"""
Test that if a ProgrammingError is raised (e.g. before migrations),
offset_cron just returns the original schedule.
"""
original_schedule = crontab(minute=[0, 15, 30], hour=[1, 2, 3])
# Force get_solo to raise ProgrammingError
with patch('allianceauth.crontab.models.CronOffset.get_solo', side_effect=ProgrammingError()):
new_schedule = offset_cron(original_schedule)
# Should return the original schedule unchanged
self.assertEqual(new_schedule, original_schedule)
def test_offset_cron_unexpected_exception(self):
"""
Test that if any other exception is raised, offset_cron
also returns the original schedule, and logs the error.
"""
original_schedule = crontab(minute='0', hour='0')
# Force get_solo to raise a generic Exception
with patch('allianceauth.crontab.models.CronOffset.get_solo', side_effect=Exception("Something bad")):
new_schedule = offset_cron(original_schedule)
# Should return the original schedule unchanged
self.assertEqual(new_schedule, original_schedule)

View File

@@ -0,0 +1,41 @@
from celery.schedules import crontab
import logging
from allianceauth.crontab.models import CronOffset
from django.db import ProgrammingError
logger = logging.getLogger(__name__)
def offset_cron(schedule: crontab) -> crontab:
"""Take a crontab and apply a series of precalculated offsets to spread out tasks execution on remote resources
Args:
schedule (crontab): celery.schedules.crontab()
Returns:
crontab: A crontab with offsetted Minute and Hour fields
"""
try:
cron_offset = CronOffset.get_solo()
new_minute = [(m + (round(60 * cron_offset.minute))) % 60 for m in schedule.minute]
new_hour = [(m + (round(24 * cron_offset.hour))) % 24 for m in schedule.hour]
return crontab(
minute=",".join(str(m) for m in sorted(new_minute)),
hour=",".join(str(h) for h in sorted(new_hour)),
day_of_month=schedule._orig_day_of_month,
month_of_year=schedule._orig_month_of_year,
day_of_week=schedule._orig_day_of_week)
except ProgrammingError as e:
# If this is called before migrations are run hand back the default schedule
# These offsets are stored in a Singleton Model,
logger.error(e)
return schedule
except Exception as e:
# We absolutely cant fail to hand back a schedule
logger.error(e)
return schedule

View File

@@ -1,4 +1,5 @@
import logging
from random import randint
from celery import shared_task
@@ -9,7 +10,8 @@ from . import providers
logger = logging.getLogger(__name__)
TASK_PRIORITY = 7
CHUNK_SIZE = 500
CHARACTER_AFFILIATION_CHUNK_SIZE = 500
EVEONLINE_TASK_JITTER = 600
def chunks(lst, n):
@@ -19,13 +21,13 @@ def chunks(lst, n):
@shared_task
def update_corp(corp_id):
def update_corp(corp_id: int) -> None:
"""Update given corporation from ESI"""
EveCorporationInfo.objects.update_corporation(corp_id)
@shared_task
def update_alliance(alliance_id):
def update_alliance(alliance_id: int) -> None:
"""Update given alliance from ESI"""
EveAllianceInfo.objects.update_alliance(alliance_id).populate_alliance()
@@ -37,23 +39,30 @@ def update_character(character_id: int) -> None:
@shared_task
def run_model_update():
def run_model_update() -> None:
"""Update all alliances, corporations and characters from ESI"""
#update existing corp models
# Queue update tasks for Known Corporation Models
for corp in EveCorporationInfo.objects.all().values('corporation_id'):
update_corp.apply_async(args=[corp['corporation_id']], priority=TASK_PRIORITY)
update_corp.apply_async(
args=[corp['corporation_id']],
priority=TASK_PRIORITY,
countdown=randint(1, EVEONLINE_TASK_JITTER))
# update existing alliance models
# Queue update tasks for Known Alliance Models
for alliance in EveAllianceInfo.objects.all().values('alliance_id'):
update_alliance.apply_async(args=[alliance['alliance_id']], priority=TASK_PRIORITY)
update_alliance.apply_async(
args=[alliance['alliance_id']],
priority=TASK_PRIORITY,
countdown=randint(1, EVEONLINE_TASK_JITTER))
# update existing character models
# Queue update tasks for Known Character Models
character_ids = EveCharacter.objects.all().values_list('character_id', flat=True)
for character_ids_chunk in chunks(character_ids, CHUNK_SIZE):
for character_ids_chunk in chunks(character_ids, CHARACTER_AFFILIATION_CHUNK_SIZE):
update_character_chunk.apply_async(
args=[character_ids_chunk], priority=TASK_PRIORITY
)
args=[character_ids_chunk],
priority=TASK_PRIORITY,
countdown=randint(1, EVEONLINE_TASK_JITTER))
@shared_task
@@ -68,8 +77,9 @@ def update_character_chunk(character_ids_chunk: list):
logger.info("Failed to bulk update characters. Attempting single updates")
for character_id in character_ids_chunk:
update_character.apply_async(
args=[character_id], priority=TASK_PRIORITY
)
args=[character_id],
priority=TASK_PRIORITY,
countdown=randint(1, EVEONLINE_TASK_JITTER))
return
affiliations = {
@@ -107,5 +117,5 @@ def update_character_chunk(character_ids_chunk: list):
if corp_changed or alliance_changed or name_changed:
update_character.apply_async(
args=[character.get('character_id')], priority=TASK_PRIORITY
)
args=[character.get('character_id')],
priority=TASK_PRIORITY)

View File

@@ -84,7 +84,7 @@ class TestUpdateTasks(TestCase):
@override_settings(CELERY_ALWAYS_EAGER=True)
@patch('allianceauth.eveonline.providers.esi_client_factory')
@patch('allianceauth.eveonline.tasks.providers')
@patch('allianceauth.eveonline.tasks.CHUNK_SIZE', 2)
@patch('allianceauth.eveonline.tasks.CHARACTER_AFFILIATION_CHUNK_SIZE', 2)
class TestRunModelUpdate(TransactionTestCase):
def test_should_run_updates(self, mock_providers, mock_esi_client_factory):
# given
@@ -139,7 +139,7 @@ class TestRunModelUpdate(TransactionTestCase):
@patch('allianceauth.eveonline.tasks.update_character', wraps=update_character)
@patch('allianceauth.eveonline.providers.esi_client_factory')
@patch('allianceauth.eveonline.tasks.providers')
@patch('allianceauth.eveonline.tasks.CHUNK_SIZE', 2)
@patch('allianceauth.eveonline.tasks.CHARACTER_AFFILIATION_CHUNK_SIZE', 2)
class TestUpdateCharacterChunk(TestCase):
@staticmethod
def _updated_character_ids(spy_update_character) -> set:

View File

@@ -13,6 +13,15 @@
}
}
/* Image overflow fix
------------------------------------------------------------------------------------- */
@media all {
img {
max-width: 100%;
height: auto;
}
}
/* Side Navigation
------------------------------------------------------------------------------------- */
@media all {

View File

@@ -56,7 +56,7 @@
{% endif %}
{% endfor %}
{% endif %}
{% if g.group.authgroup.group_leaders.all.count %}
{% if g.group.authgroup.group_leader_groups.all.count %}
{% for group in g.group.authgroup.group_leader_groups.all %}
<span class="my-1 me-1 badge bg-secondary">{{group.name}}</span>
{% endfor %}

View File

@@ -43,13 +43,16 @@ INSTALLED_APPS = [
'allianceauth.theme.flatly',
'allianceauth.theme.materia',
"allianceauth.custom_css",
'allianceauth.crontab',
'sri',
]
SRI_ALGORITHM = "sha512"
SECRET_KEY = "wow I'm a really bad default secret key"
# Celery configuration
BROKER_URL = 'redis://localhost:6379/0'
CELERYBEAT_SCHEDULER = "django_celery_beat.schedulers.DatabaseScheduler"
CELERYBEAT_SCHEDULER = "allianceauth.crontab.schedulers.OffsetDatabaseScheduler"
CELERYBEAT_SCHEDULE = {
'esi_cleanup_callbackredirect': {
'task': 'esi.tasks.cleanup_callbackredirect',
@@ -62,10 +65,12 @@ CELERYBEAT_SCHEDULE = {
'run_model_update': {
'task': 'allianceauth.eveonline.tasks.run_model_update',
'schedule': crontab(minute='0', hour="*/6"),
'apply_offset': True
},
'check_all_character_ownership': {
'task': 'allianceauth.authentication.tasks.check_all_character_ownership',
'schedule': crontab(minute='0', hour='*/4'),
'apply_offset': True
},
'analytics_daily_stats': {
'task': 'allianceauth.analytics.tasks.analytics_daily_stats',
@@ -73,6 +78,7 @@ CELERYBEAT_SCHEDULE = {
}
}
# Build paths inside the project like this: os.path.join(BASE_DIR, ...)
PROJECT_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
BASE_DIR = os.path.dirname(PROJECT_DIR)

View File

@@ -1,3 +1,3 @@
{% load static %}
{% load sri %}
<link href="{% static 'allianceauth/css/auth-base.css' %}" rel="stylesheet">
{% sri_static 'allianceauth/css/auth-base.css' %}

View File

@@ -1,3 +1,3 @@
{% load static %}
{% load sri %}
<link href="{% static 'allianceauth/framework/css/auth-framework.css' %}" rel="stylesheet">
{% sri_static 'allianceauth/framework/css/auth-framework.css' %}

View File

@@ -1,4 +1,6 @@
{% load static %}
{% load sri %}
<!-- Bootstrap CSS -->
{% if NIGHT_MODE %}
{% if debug %}
@@ -6,7 +8,7 @@
<link rel="stylesheet/less" href="{% static 'allianceauth/css/themes/darkly/darkly.less' %}">
<script src='https://cdnjs.cloudflare.com/ajax/libs/less.js/4.1.3/less.min.js' integrity='sha512-6gUGqd/zBCrEKbJqPI7iINc61jlOfH5A+SluY15IkNO1o4qP1DEYjQBewTB4l0U4ihXZdupg8Mb77VxqE+37dg==' crossorigin='anonymous' referrerpolicy="no-referrer"></script>
{% else %}
<link rel="stylesheet" href="{% static 'allianceauth/css/themes/darkly/darkly.min.css' %}">
{% sri_static 'allianceauth/css/themes/darkly/darkly.min.css' %}
{% endif %}
{% else %}
{% if debug %}
@@ -14,7 +16,7 @@
<link rel="stylesheet/less" href="{% static 'allianceauth/css/themes/flatly/flatly.less' %}">
<script src='https://cdnjs.cloudflare.com/ajax/libs/less.js/4.1.3/less.min.js' integrity='sha512-6gUGqd/zBCrEKbJqPI7iINc61jlOfH5A+SluY15IkNO1o4qP1DEYjQBewTB4l0U4ihXZdupg8Mb77VxqE+37dg==' crossorigin='anonymous' referrerpolicy="no-referrer"></script>
{% else %}
<link rel="stylesheet" href="{% static 'allianceauth/css/themes/flatly/flatly.min.css' %}">
{% sri_static 'allianceauth/css/themes/flatly/flatly.min.css' %}
{% endif %}
{% endif %}
<!-- End Bootstrap CSS -->

View File

@@ -1,3 +1,3 @@
{% load static %}
{% load sri %}
<link href="{% static 'allianceauth/css/checkbox.css' %}" rel="stylesheet">
{% sri_static 'allianceauth/css/checkbox.css' %}

View File

@@ -1,3 +1,3 @@
{% load static %}
{% load sri %}
<script src="{% static 'allianceauth/js/eve-time.js' %}"></script>
{% sri_static 'allianceauth/js/eve-time.js' %}

View File

@@ -1,3 +1,3 @@
{% load static %}
{% load sri %}
<script src="{% static 'allianceauth/js/filterDropDown/filterDropDown.min.js' %}"></script>
{% sri_static 'allianceauth/js/filterDropDown/filterDropDown.min.js' %}

View File

@@ -1,5 +1,6 @@
{% load static %}
<!-- Start jQuery UI CSS from Alliance Auth -->
<!-- CDNs all contain theme.css, which is not supposed to be in the base CSS, Which is why this is uniquely bundled in not using a CDN -->
<link rel="stylesheet" href="{% static 'allianceauth/js/jquery-ui/1.13.2/css/jquery-ui.min.css' %}" integrity="VEqAhOZvZrx/WaxlpMoLvZDSLeLNYhkL5LU2R4/ihPJb/+qkGoMrA15SqEGtI+PCLgKwCDiby7tgdvdiAZkJGg==" crossorigin="anonymous" referrerpolicy="no-referrer">
<!-- CDNs all contain theme.css, which is not supposed to be in the base CSS, which is why this is uniquely bundled in not using a CDN -->
{% load sri %}
{% sri_static 'allianceauth/js/jquery-ui/1.13.2/css/jquery-ui.min.css' %}
<!-- End jQuery UI CSS from aa-gdpr -->

View File

@@ -1,7 +1,19 @@
{% load i18n %}
<!-- Start Moment.js from cdnjs -->
<script src="https://cdnjs.cloudflare.com/ajax/libs/moment.js/2.29.4/moment.min.js" integrity="sha512-+H4iLjY3JsKiF2V6N366in5IQHj2uEsGV7Pp/GRcm0fn76aPAk5V8xB6n8fQhhSonTqTXs/klFz4D0GIn6Br9g==" crossorigin="anonymous" referrerpolicy="no-referrer"></script>
{% if locale and LANGUAGE_CODE != 'en' %}
<!-- Moment.JS Not EN-en -->
<script src="https://cdnjs.cloudflare.com/ajax/libs/moment.js/2.29.4/locale/{{ LANGUAGE_CODE }}.js" crossorigin="anonymous" referrerpolicy="no-referrer"></script>
{% get_current_language as LANGUAGE_CODE %}
{% get_language_info for LANGUAGE_CODE as lang %}
{% if lang.code == 'zh-hans' %}
<!-- Moment.JS Localisation ({{ lang.code }}) -->
<script src="https://cdnjs.cloudflare.com/ajax/libs/moment.js/2.29.4/locale/zh-cn.js" crossorigin="anonymous" referrerpolicy="no-referrer"></script>
{% else %}
<!-- Moment.JS Localisation ({{ lang.code }}) -->
<script src="https://cdnjs.cloudflare.com/ajax/libs/moment.js/2.29.4/locale/{{ lang.code }}.js" crossorigin="anonymous" referrerpolicy="no-referrer"></script>
{% endif %}
{% endif %}
<!-- End Moment JS from cdnjs -->

View File

@@ -1,3 +1,3 @@
{% load static %}
{% load sri %}
<script src="{% static 'allianceauth/js/refresh-notification-icon.js' %}"></script>
{% sri_static 'allianceauth/js/refresh-notification-icon.js' %}

View File

@@ -1,3 +1,3 @@
{% load static %}
{% load sri %}
<script src="{% static 'allianceauth/js/refresh_notifications.js' %}"></script>
{% sri_static 'allianceauth/js/refresh_notifications.js' %}

View File

@@ -1,3 +1,3 @@
{% load static %}
{% load sri %}
<script src="{% static 'allianceauth/js/timerboard.js' %}"></script>
{% sri_static 'allianceauth/js/timerboard.js' %}

View File

@@ -1,3 +1,3 @@
{% load static %}
{% load sri %}
<script src="{% static 'allianceauth/js/timers.js' %}"></script>
{% sri_static 'allianceauth/js/timers.js' %}

View File

@@ -8,14 +8,16 @@ class ThemeHook:
"""
def __init__(self,
name: str,
description: str,
css: List[dict],
js: List[dict],
css_template: Optional[str] = None,
js_template: Optional[str] = None,
html_tags: Optional[Union[dict, str]] = None,
header_padding: Optional[str] = "4em"):
name: str,
description: str,
css: List[dict],
js: List[dict],
css_template: Optional[str] = None,
js_template: Optional[str] = None,
js_type: Optional[str] = None,
html_tags: Optional[Union[dict, str]] = None,
header_padding: Optional[str] = "4em"
):
"""
:param name: Theme python name
:type name: str
@@ -29,11 +31,14 @@ class ThemeHook:
:type css_template: Optional[str], optional
:param js_template: _description_, defaults to None
:type js_template: Optional[str], optional
:param js_type: The type of the JS (e.g.: 'module'), defaults to None
:type js_type: Optional[str], optional
:param html_tags: Attributes added to the `<html>` tag, defaults to None
:type html_tags: Optional[dict|str], optional
:param header_padding: Top padding, defaults to "4em"
:type header_padding: Optional[str], optional
"""
self.name = name
self.description = description
@@ -45,11 +50,15 @@ class ThemeHook:
self.css_template = css_template
self.js_template = js_template
# Define the JS type (e.g.: 'module')
self.js_type = js_type
self.html_tags = (
" ".join([f"{key}={value}" for key, value in html_tags.items()])
if isinstance(html_tags, dict)
else html_tags
)
self.header_padding = header_padding
def get_name(self):
return f"{self.__class__.__module__}.{self.__class__.__name__}"

View File

@@ -4,7 +4,13 @@
{% include theme.js_template %}
{% else %}
{% for x in theme.js %}
<script src="{{ x.url }}" integrity="{{ x.integrity }}" crossorigin="anonymous" referrerpolicy="no-referrer"></script>
<script
{% if x.js_type %}type="{{ x.js_type }}"{% endif %}
src="{{ x.url }}"
integrity="{{ x.integrity }}"
crossorigin="anonymous"
referrerpolicy="no-referrer"
></script>
{% endfor %}
{% endif %}
<!-- allianceauth.theme.{{ theme.name }} JS Ends-->

View File

@@ -0,0 +1,45 @@
# Generated by Django 4.2.17 on 2025-01-06 17:16
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("timerboard", "0006_alter_timer_objective_alter_timer_structure_and_more"),
]
operations = [
migrations.AlterField(
model_name="timer",
name="structure",
field=models.CharField(
choices=[
("POCO", "POCO"),
("Orbital Skyhook", "Orbital Skyhook"),
("I-HUB", "Sovereignty Hub"),
("TCU", "TCU"),
("POS[S]", "POS [S]"),
("POS[M]", "POS [M]"),
("POS[L]", "POS [L]"),
("Astrahus", "Astrahus"),
("Fortizar", "Fortizar"),
("Keepstar", "Keepstar"),
("Raitaru", "Raitaru"),
("Azbel", "Azbel"),
("Sotiyo", "Sotiyo"),
("Athanor", "Athanor"),
("Tatara", "Tatara"),
("Pharolux Cyno Beacon", "Cyno Beacon"),
("Tenebrex Cyno Jammer", "Cyno Jammer"),
("Ansiblex Jump Gate", "Ansiblex Jump Gate"),
("Mercenary Den", "Mercenary Den"),
("Moon Mining Cycle", "Moon Mining Cycle"),
("Metenox Moon Drill", "Metenox Moon Drill"),
("Other", "Other"),
],
default="Other",
max_length=254,
),
),
]

View File

@@ -1,6 +1,6 @@
from django.contrib.auth.models import User
from django.db import models
from django.utils.translation import gettext as _
from django.utils.translation import gettext_lazy as _
from allianceauth.eveonline.models import EveCharacter
from allianceauth.eveonline.models import EveCorporationInfo
@@ -23,7 +23,7 @@ class Timer(models.Model):
POCO = "POCO", _("POCO")
ORBITALSKYHOOK = "Orbital Skyhook", _("Orbital Skyhook")
IHUB = "I-HUB", _("I-HUB")
IHUB = "I-HUB", _("Sovereignty Hub")
TCU = "TCU", _("TCU") # Pending Remval
POSS = "POS[S]", _("POS [S]")
POSM = "POS[M]", _("POS [M]")
@@ -36,9 +36,10 @@ class Timer(models.Model):
SOTIYO = "Sotiyo", _("Sotiyo")
ATHANOR = "Athanor", _("Athanor")
TATARA = "Tatara", _("Tatara")
PHAROLUX = "Pharolux Cyno Beacon", _("Pharolux Cyno Beacon")
TENEBREX = "Tenebrex Cyno Jammer", _("Tenebrex Cyno Jammer")
PHAROLUX = "Pharolux Cyno Beacon", _("Cyno Beacon")
TENEBREX = "Tenebrex Cyno Jammer", _("Cyno Jammer")
ANSIBLEX = "Ansiblex Jump Gate", _("Ansiblex Jump Gate")
MERCDEN = "Mercenary Den", _("Mercenary Den")
MOONPOP = "Moon Mining Cycle", _("Moon Mining Cycle")
METENOX = "Metenox Moon Drill", _("Metenox Moon Drill")
OTHER = "Other", _("Other")

View File

@@ -19,7 +19,6 @@
{% for timer in timers %}
<tr class="{% if timer.important == True %}bg-danger bg-opacity-25{% else %}bg-info bg-opacity-25{% endif %}">
<td style="width: 150px;" class="text-center">
{{ timer.details }}
@@ -30,13 +29,21 @@
</td>
<td class="text-center">
{% comment %} Objective: Hostile (BG: Danger) {% endcomment %}
{% if timer.objective == "Hostile" %}
<div class="badge bg-danger">{% translate "Hostile" %}</div>
<div class="badge bg-danger">
{% comment %} Objective: Friendly (BG: Primare) {% endcomment %}
{% elif timer.objective == "Friendly" %}
<div class="badge bg-primary">{% translate "Friendly" %}</div>
<div class="badge bg-primary">
{% comment %} Objective: Neutral (BG: Secondary) {% endcomment %}
{% elif timer.objective == "Neutral" %}
<div class="badge bg-secondary">{% translate "Neutral" %}</div>
<div class="badge bg-secondary">
{% endif %}
{{ timer.get_objective_display }}
</div>
</td>
<td class="text-center">
@@ -44,49 +51,9 @@
</td>
<td class="text-center">
{% if timer.structure == "POCO" %}
<div class="badge bg-info">{% translate "POCO" %}</div>
{% elif timer.structure == "Orbital Skyhook" %}
<div class="badge bg-warning">{% translate "Orbital Skyhook" %}</div>
{% elif timer.structure == "I-HUB" %}
<div class="badge bg-warning">{% translate "I-HUB" %}</div>
{% elif timer.structure == "TCU" %} {% comment %} Pending Removal {% endcomment %}
<div class="badge bg-secondary">{% translate "TCU" %}</div>
{% elif timer.structure == "POS[S]" %}
<div class="badge bg-info">{% translate "POS [S]" %}</div>
{% elif timer.structure == "POS[M]" %}
<div class="badge bg-info">{% translate "POS [M]" %}</div>
{% elif timer.structure == "POS[L]" %}
<div class="badge bg-info">{% translate "POS [L]" %}</div>
{% elif timer.structure == "Citadel[M]" or timer.structure == "Astrahus" %}
<div class="badge bg-danger">{% translate "Astrahus" %}</div>
{% elif timer.structure == "Citadel[L]" or timer.structure == "Fortizar" %}
<div class="badge bg-danger">{% translate "Fortizar" %}</div>
{% elif timer.structure == "Citadel[XL]" or timer.structure == "Keepstar" %}
<div class="badge bg-danger">{% translate "Keepstar" %}</div>
{% elif timer.structure == "Engineering Complex[M]" or timer.structure == "Raitaru" %}
<div class="badge bg-warning">{% translate "Raitaru" %}</div>
{% elif timer.structure == "Engineering Complex[L]" or timer.structure == "Azbel" %}
<div class="badge bg-warning">{% translate "Azbel" %}</div>
{% elif timer.structure == "Engineering Complex[XL]" or timer.structure == "Sotiyo" %}
<div class="badge bg-danger">{% translate "Sotiyo" %}</div>
{% elif timer.structure == "Refinery[M]" or timer.structure == "Athanor" %}
<div class="badge bg-warning">{% translate "Athanor" %}</div>
{% elif timer.structure == "Refinery[L]" or timer.structure == "Tatara" %}
<div class="badge bg-warning">{% translate "Tatara" %}</div>
{% elif timer.structure == "Cyno Beacon" or timer.structure == "Pharolux Cyno Beacon" %}
<div class="badge bg-warning">{% translate "Cyno Beacon" %}</div>
{% elif timer.structure == "Cyno Jammer" or timer.structure == "Tenebrex Cyno Jammer" %}
<div class="badge bg-warning">{% translate "Cyno Jammer" %}</div>
{% elif timer.structure == "Jump Gate" or timer.structure == "Ansiblex Jump Gate" %}
<div class="badge bg-warning">{% translate "Ansiblex Jump Gate" %}</div>
{% elif timer.structure == "Moon Mining Cycle" %}
<div class="badge bg-success">{% translate "Moon Mining Cycle" %}</div>
{% elif timer.structure == "Metenox Moon Drill" %}
<div class="badge bg-warning">{% translate "Metenox Moon Drill" %}</div>
{% elif timer.structure == "Other" %}
<div class="badge bg-secondary">{% translate "Other" %}</div>
{% endif %}
<div class="badge bg-{{ timer.bg_modifier }}">
{{ timer.get_structure_display }}
</div>
</td>
<td class="text-center" nowrap>{{ timer.eve_time | date:"Y-m-d H:i" }}</td>

View File

@@ -1,12 +1,12 @@
import datetime
import logging
from django.contrib import messages
from django.contrib.auth.mixins import (
LoginRequiredMixin, PermissionRequiredMixin,
LoginRequiredMixin,
PermissionRequiredMixin,
)
from django.db.models import Q
from django.shortcuts import get_object_or_404, redirect, render
from django.shortcuts import get_object_or_404, render
from django.template.loader import render_to_string
from django.urls import reverse_lazy
from django.utils import timezone
@@ -20,8 +20,8 @@ from allianceauth.timerboard.models import Timer
logger = logging.getLogger(__name__)
TIMER_VIEW_PERMISSION = 'auth.timer_view'
TIMER_MANAGE_PERMISSION = 'auth.timer_management'
TIMER_VIEW_PERMISSION = "auth.timer_view"
TIMER_MANAGE_PERMISSION = "auth.timer_management"
class BaseTimerView(LoginRequiredMixin, PermissionRequiredMixin, View):
@@ -29,22 +29,112 @@ class BaseTimerView(LoginRequiredMixin, PermissionRequiredMixin, View):
class TimerView(BaseTimerView):
template_name = 'timerboard/view.html'
template_name = "timerboard/view.html"
permission_required = TIMER_VIEW_PERMISSION
def get(self, request):
"""
Renders the timer view
:param request:
:type request:
:return:
:rtype:
"""
def get_bg_modifier(structure):
"""
Returns the bootstrap bg modifier for the given structure
:param structure:
:type structure:
:return:
:rtype:
"""
if structure in bg_info:
return "info"
elif structure in bg_warning:
return "warning"
elif structure in bg_danger:
return "danger"
elif structure in bg_secondary:
return "secondary"
return "primary"
logger.debug(f"timer_view called by user {request.user}")
char = request.user.profile.main_character
if char:
corp = char.corporation
else:
corp = None
base_query = Timer.objects.select_related('eve_character')
base_query = Timer.objects.select_related("eve_character")
timers = []
corp_timers = []
future_timers = []
past_timers = []
bg_info = [
Timer.Structure.POCO.value, # POCO
Timer.Structure.POSS.value, # POS[S]
Timer.Structure.POSM.value, # POS[M]
Timer.Structure.POSL.value, # POS[L]
]
bg_warning = [
Timer.Structure.ANSIBLEX.value, # Ansiblex Jump Gate
Timer.Structure.ATHANOR.value, # Athanor
Timer.Structure.AZBEL.value, # Azbel
Timer.Structure.MERCDEN.value, # Mercenary Den
Timer.Structure.METENOX.value, # Metenox Moon Drill
Timer.Structure.ORBITALSKYHOOK.value, # Orbital Skyhook
Timer.Structure.PHAROLUX.value, # Pharolux Cyno Beacon
Timer.Structure.RAITARU.value, # Raitaru
"Station", # Legacy structure, remove in future update
Timer.Structure.TATARA.value, # Tatara
Timer.Structure.TENEBREX.value, # Tenebrex Cyno Jammer
]
bg_danger = [
Timer.Structure.ASTRAHUS.value, # Astrahus
Timer.Structure.FORTIZAR.value, # Fortizar
Timer.Structure.IHUB.value, # I-HUB
Timer.Structure.KEEPSTAR.value, # Keepstar
Timer.Structure.SOTIYO.value, # Sotiyo
Timer.Structure.TCU.value, # TCU (Legacy structure, remove in future update)
]
bg_secondary = [
Timer.Structure.MOONPOP.value, # Moon Mining Cycle
Timer.Structure.OTHER.value, # Other
]
# Timers
for timer in base_query.filter(corp_timer=False):
timer.bg_modifier = get_bg_modifier(timer.structure)
timers.append(timer)
# Corp Timers
for timer in base_query.filter(corp_timer=True, eve_corp=corp):
timer.bg_modifier = get_bg_modifier(timer.structure)
corp_timers.append(timer)
# Future Timers
for timer in base_query.filter(corp_timer=False, eve_time__gte=timezone.now()):
timer.bg_modifier = get_bg_modifier(timer.structure)
future_timers.append(timer)
# Past Timers
for timer in base_query.filter(corp_timer=False, eve_time__lt=timezone.now()):
timer.bg_modifier = get_bg_modifier(timer.structure)
past_timers.append(timer)
render_items = {
'timers': base_query.filter(corp_timer=False),
'corp_timers': base_query.filter(corp_timer=True, eve_corp=corp),
'future_timers': base_query.filter(corp_timer=False, eve_time__gte=timezone.now()),
'past_timers': base_query.filter(corp_timer=False, eve_time__lt=timezone.now()),
"timers": timers,
"corp_timers": corp_timers,
"future_timers": future_timers,
"past_timers": past_timers,
}
return render(request, self.template_name, context=render_items)
@@ -52,7 +142,7 @@ class TimerView(BaseTimerView):
class TimerManagementView(BaseTimerView):
permission_required = TIMER_MANAGE_PERMISSION
index_redirect = 'timerboard:view'
index_redirect = "timerboard:view"
success_url = reverse_lazy(index_redirect)
model = Timer
@@ -66,12 +156,12 @@ class AddUpdateMixin:
Inject the request user into the kwargs passed to the form
"""
kwargs = super().get_form_kwargs()
kwargs.update({'user': self.request.user})
kwargs.update({"user": self.request.user})
return kwargs
class AddTimerView(TimerManagementView, AddUpdateMixin, CreateView):
template_name_suffix = '_create_form'
template_name_suffix = "_create_form"
form_class = TimerForm
def form_valid(self, form):
@@ -82,17 +172,18 @@ class AddTimerView(TimerManagementView, AddUpdateMixin, CreateView):
)
messages.success(
self.request,
_('Added new timer in %(system)s at %(time)s.') % {"system": timer.system, "time": timer.eve_time}
_("Added new timer in %(system)s at %(time)s.")
% {"system": timer.system, "time": timer.eve_time},
)
return result
class EditTimerView(TimerManagementView, AddUpdateMixin, UpdateView):
template_name_suffix = '_update_form'
template_name_suffix = "_update_form"
form_class = TimerForm
def form_valid(self, form):
messages.success(self.request, _('Saved changes to the timer.'))
messages.success(self.request, _("Saved changes to the timer."))
return super().form_valid(form)
@@ -107,21 +198,20 @@ def dashboard_timers(request):
except (EveCorporationInfo.DoesNotExist, AttributeError):
return ""
timers = Timer.objects.select_related(
'eve_character'
).filter(
timers = Timer.objects.select_related("eve_character").filter(
(Q(corp_timer=True) & Q(eve_corp=corp)) | Q(corp_timer=False),
eve_time__gte=timezone.now()
eve_time__gte=timezone.now(),
)[:5]
if timers.count():
context = {
'timers': timers,
"timers": timers,
}
return render_to_string(
template_name='timerboard/dashboard.timers.html',
context=context, request=request
template_name="timerboard/dashboard.timers.html",
context=context,
request=request,
)
else:
return ""

View File

@@ -1,7 +1,7 @@
PROTOCOL=https://
AUTH_SUBDOMAIN=%AUTH_SUBDOMAIN%
DOMAIN=%DOMAIN%
AA_DOCKER_TAG=registry.gitlab.com/allianceauth/allianceauth/auth:v4.4.2
AA_DOCKER_TAG=registry.gitlab.com/allianceauth/allianceauth/auth:v4.5.0
# Nginx Proxy Manager
PROXY_HTTP_PORT=80

View File

@@ -1,5 +1,5 @@
FROM python:3.11-slim
ARG AUTH_VERSION=v4.4.2
ARG AUTH_VERSION=v4.5.0
ARG AUTH_PACKAGE=allianceauth==${AUTH_VERSION}
ENV AUTH_USER=allianceauth
ENV AUTH_GROUP=allianceauth

View File

@@ -0,0 +1,6 @@
[mariadb]
# Provided as an Example
# AA Doesnt use Aria or MyISAM, So these are worth Considering
# aria_pagecache_buffer_size = 16M
# key_buffer_size = 16M

0
docker/conf/redis_healthcheck.sh Normal file → Executable file
View File

View File

@@ -49,6 +49,7 @@ services:
volumes:
- ./mysql-data:/var/lib/mysql
- ./setup.sql:/docker-entrypoint-initdb.d/setup.sql
- ./conf/aa_mariadb.cnf:/etc/mysql/conf.d/aa_mariadb.cnf
environment:
- MYSQL_ROOT_PASSWORD=${AA_DB_ROOT_PASSWORD?err}
- MARIADB_MYSQL_LOCALHOST_USER=1
@@ -83,7 +84,7 @@ services:
- "redis-data:/data"
- ./conf/redis_healthcheck.sh:/usr/local/bin/redis_healthcheck.sh
healthcheck:
test: ["CMD", "bash", "/usr/local/bin/redis_healthcheck.sh"]
test: ["CMD", "/usr/local/bin/redis_healthcheck.sh"]
logging:
driver: "json-file"
options:

View File

@@ -1,4 +1,6 @@
#!/bin/bash
git clone https://gitlab.com/allianceauth/allianceauth.git aa-git
cp -R aa-git/docker ./aa-docker
chmod +x aa-docker/conf/memory_check.sh
chmod +x aa-docker/conf/redis_healthcheck.sh
rm -rf aa-git

View File

@@ -1,6 +1,6 @@
CREATE USER 'aauth'@'%' IDENTIFIED BY 'authpass';
CREATE USER 'grafana'@'%' IDENTIFIED BY 'grafanapass';
CREATE DATABASE alliance_auth CHARACTER SET utf8mb4;
CREATE DATABASE alliance_auth CHARACTER SET utf8mb4 DEFAULT COLLATE utf8mb4_unicode_ci;
GRANT ALL PRIVILEGES ON alliance_auth.* TO 'aauth'@'%';
GRANT
SELECT,

View File

@@ -40,10 +40,10 @@ Please use the following approach to ensure your tasks are working properly with
Here is an example implementation of a task:
```python
import logging
from allianceauth.services.hooks import get_extension_logger
from celery import shared_task
logger = logging.getLogger(__name__)
logger = get_extension_logger(__name__)
@shared_task
@@ -80,10 +80,10 @@ However, many long-running tasks consist of several smaller processes that need
Example implementation for a celery chain:
```python
import logging
from allianceauth.services.hooks import get_extension_logger
from celery import shared_task, chain
logger = logging.getLogger(__name__)
logger = get_extension_logger(__name__)
@shared_task
@@ -96,7 +96,7 @@ def long_runner():
my_tasks = list()
for _ in range(10):
task_signature = example.si()
my_task.append(task_signature)
my_tasks.append(task_signature)
chain(my_tasks).delay()
```
@@ -123,6 +123,7 @@ Example setting:
CELERYBEAT_SCHEDULE['structures_update_all_structures'] = {
'task': 'structures.tasks.update_all_structures',
'schedule': crontab(minute='*/30'),
'apply_offset': True,
}
```
@@ -130,6 +131,7 @@ CELERYBEAT_SCHEDULE['structures_update_all_structures'] = {
- `'task'`: Name of your task (full path)
- `'schedule'`: Schedule definition (see Celery documentation on [Periodic Tasks](https://docs.celeryproject.org/en/latest/userguide/periodic-tasks.html) for details)
- `'apply_offset'`: Boolean, Apply a Delay unique to the install, in order to reduce impact on ESI. See [Apply Offset](#apply-offset)
## How can I use priorities for tasks?
@@ -168,6 +170,94 @@ example.apply_async(priority=3)
For defining a priority to tasks, you cannot use the convenient shortcut ``delay()``, but instead need to start a task with ``apply_async()``, which also requires you to pass parameters to your task function differently. Please check out the `official docs <https://docs.celeryproject.org/en/stable/reference/celery.app.task.html#celery.app.task.Task.apply_async>`_ for details.
:::
## Rate-Limiting and Smoothing of Task Execution
Large numbers of installs running the same crontab (ie. `0 * * * *`) can all slam an external service at the same time.
Consider Artificially smoothing out your tasks with a few methods
### Apply Offset
`allianceauth.crontab` contains a series of Offsets stored in the DB that are both static for an install, but random across all AA installs.
This enables us to spread our load on ESI (or other external resources) across a greater window, making it unlikely that two installs will hit ESI at the same time.
Tasks defined in local.py, can have `'apply_offset': True` added to their Task Definition
```python
CELERYBEAT_SCHEDULE['taskname'] = {
'task': 'module.tasks.task',
'schedule': crontab(minute='*/30'),
'apply_offset': True,
}
```
Tasks added to directly to Django Celery Beat Models (Using a Management Task etc) can pass their Cron Schedule through offset_cron(crontab)
```{eval-rst}
.. automodule:: allianceauth.crontab.utils
:members:
:undoc-members:
```
```python
from django_celery_beat.models import CrontabSchedule, PeriodicTask
from celery.schedules import crontab
schedule = CrontabSchedule.from_schedule(offset_cron(crontab(minute='0', hour='0')))
schedule, created = CrontabSchedule.objects.get_or_create(
minute=schedule.minute,
hour=schedule.hour,
day_of_month=schedule.day_of_month,
month_of_year=schedule.month_of_year,
day_of_week=schedule.day_of_week,
timezone=schedule.timezone,
)
PeriodicTask.objects.update_or_create(
task='module.tasks.task',
defaults={
'crontab': schedule,
'name': 'task name',
'enabled': True
}
)
```
### Subset Tasks
Slice your tasks needed up into more manageable chunks and run them more often. 1/10th of your tasks run 10x more often will return the same end result with less peak loads on external services and your task queue.
### Celery ETA/Countdown
Scatter your tasks across a larger window using <https://docs.celeryq.dev/en/latest/userguide/calling.html#eta-and-countdown>
This example will queue up tasks across the next 10 minutes, trickling them into your workers (and the external service)
```python
for corp in EveCorporationInfo.objects.all().values('corporation_id'):
update_corp.apply_async(args=[corp['corporation_id']], priority=TASK_PRIORITY)
update_corp.apply_async(
args=[corp['corporation_id']],
priority=TASK_PRIORITY,
countdown=randint(1, 600))
```
### Celery Rate Limits
Celery Rate Limits come with a small catch, its _per worker_, you may have to be either very conservative or have these configurable by the end user if they varied their worker count.
<https://docs.celeryq.dev/en/latest/userguide/tasks.html#Task.rate_limit>
This example of 10 Tasks per Minute will result in ~100 tasks per minute at 10 Workers
```python
@shared_task(rate_limit="10/m")
def update_charactercorporationhistory(character_id: int) -> None:
"""Update CharacterCorporationHistory models from ESI"""
```
## What special features should I be aware of?
Every Alliance Auth installation will come with a couple of special celery related features "out-of-the-box" that you can make use of in your apps.
@@ -192,6 +282,6 @@ You can use it like so:
Please see the [official documentation](https://pypi.org/project/celery_once/) of celery-once for details.
### task priorities
### Task Priorities
Alliance Auth is using task priorities to enable priority-based scheduling of task execution. Please see [How can I use priorities for tasks?](#how-can-i-use-priorities-for-tasks) for details.

View File

@@ -27,6 +27,7 @@ Analytics comes preloaded with our Google Analytics token, and the three types o
Our Daily Stats contain the following:
- A phone-in task to identify a server's existence
- A phone-in task to identify if a server is Bare-Metal or Dockerized
- A task to send the Number of User models
- A task to send the Number of Token Models
- A task to send the Number of Installed Apps

View File

@@ -65,7 +65,7 @@ Using a custom docker image is the preferred approach, as it gives you the stabi
* e.g.
```docker
x-allianceauth-base: &allianceauth-base
x-allianceauth-base: &allianceauth-base
# image: ${AA_DOCKER_TAG?err}
build:
context: .

View File

@@ -246,7 +246,7 @@ and create them as follows, replacing `PASSWORD` with an actual secure password:
```sql
CREATE USER 'allianceserver'@'localhost' IDENTIFIED BY 'PASSWORD';
CREATE DATABASE alliance_auth CHARACTER SET utf8mb4;
CREATE DATABASE alliance_auth CHARACTER SET utf8mb4 DEFAULT COLLATE utf8mb4_unicode_ci;
GRANT ALL PRIVILEGES ON alliance_auth . * TO 'allianceserver'@'localhost';
```

View File

@@ -52,6 +52,7 @@ dependencies = [
"django-registration<3.4,>=3.3",
"django-solo",
"django-sortedm2m",
"django-sri",
"dnspython",
"mysqlclient>=2.1",
"openfire-restapi",