Docs: Add python upgrade guide, remove old AA 1.15 upgrade guide, improve install guide

This commit is contained in:
Erik Kalkoken 2020-03-15 12:21:21 +00:00 committed by Ariel Rin
parent a1e8903128
commit 78e05b84e9
6 changed files with 295 additions and 102 deletions

View File

@ -1,6 +1,6 @@
# Overview
**Alliance Auth** (AA) is a web application that helps Eve Online organizations efficiently manage access to external services and web apps.
**Alliance Auth** (AA) is a web site that helps Eve Online organizations efficiently manage access to applications and external services.
It has the following key features:
@ -13,7 +13,3 @@ It has the following key features:
- Includes a set of web [apps](/features/apps/index) which add many useful functions, e.g.: fleet schedule, timer board, SRP request management, fleet activity tracker
- Can be easily extended with additional services and apps. Many are provided by the [community](/features/community/index).
Here is an example how the main page of the web site looks:
![dashboard](/_static/images/features/core/dashboard/dashboard.png)

View File

@ -3,7 +3,9 @@
Welcome to the official documentation for **Alliance Auth**!
**Alliance Auth** is a web application that helps Eve Online organizations efficiently manage access to external services and web apps.
![dashboard](/_static/images/features/core/dashboard/dashboard.png)
**Alliance Auth** is a web site that helps Eve Online organizations efficiently manage access to applications and external services.
```eval_rst
.. toctree::

View File

@ -9,7 +9,7 @@ This document describes how to install **Alliance Auth** from scratch.
```eval_rst
.. note::
There are additional installation steps for activating services and apps that come with **Alliance Auth**. Please see the page for the respective service or apps in chapter [Features](/features/index) for details.
There are additional installation steps for activating services and apps that come with **Alliance Auth**. Please see the page for the respective service or apps in chapter **Features** for details.
```
## Dependencies
@ -44,7 +44,14 @@ yum install python36u python36u-devel python36u-setuptools python36u-pip
### Database
It's recommended to use a database service instead of SQLite. Many options are available, but this guide will use MariaDB. Note that Alliance Auth requires Maria DB 10.2.x or higher.
It's recommended to use a database service instead of SQLite. Many options are available, but this guide will use MariaDB.
```eval_rst
.. warning::
Many Ubuntu distributions come with an older version of Maria DB, which is not compatible with **Alliance Auth**. You need Maria DB 10.3 or higher!
For instructions on how To install a newer version of Maria DB on Ubuntu visit this page: `MariaDB Repositories <https://downloads.mariadb.org/mariadb/repositories/#distro=Ubuntu&mirror=osuosl>`_.
```
Ubuntu:
@ -188,6 +195,12 @@ You can install **Alliance Auth** with the following command. This will install
pip install allianceauth
```
You should also install Gunicorn now unless you want to use another WSGI server (see [Gunicorn](#gunicorn) for details):
```bash
pip install gunicorn
```
Now you need to create the application that will run the **Alliance Auth** install. Ensure you are in the allianceserver home directory by issuing:
```bash
@ -237,16 +250,15 @@ To run the **Alliance Auth** website a [WSGI Server](https://www.fullstackpython
The default configuration is good enough for most installations. Additional information is available in the [gunicorn](gunicorn.md) doc.
Use this command to install Gunicorn:
```bash
pip install gunicorn
```
### Supervisor
[Supervisor](http://supervisord.org/) is a process watchdog service: it makes sure other processes are started automatically and kept running. It can be used to automatically start the WSGI server and Celery workers for background tasks. Installation varies by OS:
```eval_rst
.. note::
Many package managers will install Supervisor 3 by default, which requires Python 2.
```
Ubuntu:
```bash

View File

@ -12,5 +12,5 @@ In addition to main guide for installation Alliance Auth you also find guides fo
nginx
apache
gunicorn
upgradev1
upgrade_python
```

View File

@ -0,0 +1,270 @@
# Upgrading Python 3
This guide describes how to upgrade an existing Alliance Auth (AA) installation to a newer Python 3 version.
```eval_rst
.. hint::
In accordance with the installation guide we will assume you perform all actions as root. If you are not running as root you need to add ``sudo`` to some commands.
```
```eval_rst
.. note::
This guide will upgrade the software components only but not change any data or configuration.
```
## Install a new Python version
To run AA with a newer Python 3 version than your system's default you need to install it first. Technically it would be possible to upgrade your system's default Python 3, but since many of your system's tools have been tested to work with that specific version we would not recommend it. Instead we recommend to install an additional Python 3 version alongside your default version and use that for AA.
```eval_rst
.. note::
For stability and performance we currently recommend to run AA with Python 3.7. Since at the time of writing Python 3.7 was not available for CentOS through yum install this guide will upgrade to Python 3.6. For Ubuntu one can just replace "3.6" with "3.7" in the installation commands to get Python 3.7.
```
To install other Python versions than come with your distro you need to add a new installation repository. Then you can install the specific Python 3 to your system.
Ubuntu:
```bash
add-apt-repository ppa:deadsnakes/ppa
```
```bash
apt-get update
```
```bash
apt-get install python3.6 python3.6-dev python3.6-venv
```
CentOS:
```bash
yum install https://centos7.iuscommunity.org/ius-release.rpm
```
```bash
yum update
```
```bash
yum install python36u python36u-pip python36u-devel
```
## Preparing your venv
Before updating your venv it is important to make sure that your current installation is stable. Otherwise your new venv might not be consistent with your data, which might create problems.
Start by navigating to your main project folder (the one that has `manage.py` in it). If you followed the default installation the path is: `/home/allianceserver/myauth`
Activate your venv:
```bash
source /home/allianceserver/venv/auth/bin/activate
```
### Upgrade AA
Make sure to upgrade AA to the newest version:
```bash
pip install -U allianceauth
```
Run migrations and collectstatic.
```bash
python manage.py migrate
```
```bash
python manage.py collectstatic
```
Restart your AA supervisor:
```bash
supervisorctl restart myauth:
```
### Upgrade your apps
You also need to upgrade all additional apps to their newest version that you have installed. And you need to make sure that you can reinstall all your apps later, e.g. you know from which repo they came. We recommend to make a list of all your apps, so you can just go through them later when you rebuild your venv.
If you unsure which apps you have installed from repos check `INSTALLED_APPS` in your settings. Alternatively run this command to get a list all apps in your venv.
```bash
pip list
```
Some AA installations might still be running an older version of django_celery_beat. We would recommend to upgrade to the current version before doing the Python update:
```bash
pip install -U django_celery_beat
```
```bash
python manage.py migrate
```
Make sure to run migrations and collect static files for all upgraded apps.
### Restart and final check
Do a final restart of your AA supervisors and make sure your installation is still running normally.
For a final check that they are no issues - e.g. any outstanding migrations - run this command:
```bash
python manage.py check
```
If you get the following result you are good to go. Otherwise make sure to fix any issues first before proceeding.
```bash
System check identified no issues (0 silenced).
```
## Backup current venv
Make sure you are in your venv!
First we create a list of all installed packages in your venv. You can use this list later as reference to see what packages should be installed.
```bash
pip freeze > requirements.txt
```
At this point we recommend creating a list of the additional packages that you need to manually reinstall later on top of AA:
- Community AA apps (e.g. aa-structures)
- Additional tools you are using (e.g. flower, django-extensions)
```eval_rst
.. hint::
While `requirements.txt` will contain a complete list of your packages, it will also contain many packages that are automatically installed as dependencies and don't need be manually reinstalled.
```
```eval_rst
.. note::
Some guide on the Internet will suggest to use use the requirements.txt file to recreate a venv. This is indeed possible, but only works if all packages can be installed from PyPI. Since most community apps are installed directly from repos this guide will not follow that approach.
```
Leave the venv and shutdown all AA services:
```bash
deactivate
```
```bash
supervisorctl stop myauth:
```
Rename and keep your old venv so we have a fallback in case of some unforeseeable issues:
```bash
mv /home/allianceserver/venv/auth /home/allianceserver/venv/auth_old
```
## Create your new venv
Now let's create our new venv with Python 3.6 and activate it:
```bash
python3.6 -m venv /home/allianceserver/venv/auth
```
```bash
source /home/allianceserver/venv/auth/bin/activate
```
## Reinstall packages
Now we need to reinstall all packages into your new venv.
### Install basic packages
```bash
pip install --upgrade pip
```
```bash
pip install --upgrade setuptools
```
```bash
pip install wheel
```
### Installing AA & Gunicorn
```bash
pip install allianceauth
```
```bash
pip install gunicorn
```
### Install all other packages
Last, but not least you need to reinstall all other packages, e.g. for AA community apps or additional tools.
Use the list of packages you created earlier as a checklist. Alternatively you use the `requirements.txt` file we created earlier to see what you need. During the installation process you can run `pip list` to see what you already got installed.
To check whether you are missing any apps you can also run the check command:
```bash
python manage.py check
```
Note: In case you forget to install an app you will get this error
```bash
ModuleNotFoundError: No module named 'xyz'
```
Note that you should not need to run any migrations unless you forgot to upgrade one of your existing apps or you got the newer version of an app through a dependency. In that case you just migrations normally.
## Restart
After you have completed installing all packages just start your AA supervisor again.
```bash
supervisorctl start myauth:
```
We recommend to keep your old venv copy for a couple of days so you have a fallback just in case. After that you should be fine to remove it.
## Fallback
In case you run into any major issue you can always switch back to your initial venv.
Before you start double-check that you still have your old venv for auth:
```bash
ls /home/allianceserver/venv/auth /home/allianceserver/venv
```
If the output shows these two folders you should be safe to proceed:
- `auth`
- `auth_old`
Run these commands to remove your current venv and switch back to the old venv for auth:
```bash
supervisorctl stop myauth:
```
```bash
rm -rf /home/allianceserver/venv/auth
```
```bash
mv /home/allianceserver/venv/auth_old /home/allianceserver/venv/auth
```
```bash
supervisorctl start myauth:
```

View File

@ -1,87 +0,0 @@
# Upgrading from AA v1.15
It's possible to preserve a v1 install's database and migrate it to v2. This will retain all service accounts, user accounts with their main character, but will purge API keys and alts.
## Preparing to Upgrade
Ensure your auth install is at least version 1.15 - it's preferred to be on v1.15.8 but any v1.15.x should work. If not, update following normal procedures and ensure you run migrations.
If you will not be using any apps (or they have been removed in v2) clear their database tables before beginning the v2 install procedure. From within your v1 install, `python manage.py migrate APPNAME zero`, replacing `APPNAME` with the appropriate name.
It's strongly encouraged to perform the upgrade with a **copy** of the original v1 database. If something goes wrong this allows you to roll back and try again. This can be achieved with:
mysqldump -u root -p v1_database_name_here | mysql -u root -p v2_database_name_here
Note this command will prompt you for the root password twice.
Alliance Auth v2 requires Python 3.4 or newer. If this is not available on your system be sure to install the [dependencies listed.](allianceauth.md#python)
## Installation
Follow the [normal install procedures for Alliance Auth v2](allianceauth.md). If you're coming from v1 you likely already have the user account and dependencies.
When configuring settings, ensure to enter the database information relating to the copy you made of the v1 database, but **do not run migrations**. See below before continuing.
Do not start Celery yet.
## Migration
During the upgrade process a migration is run which ports data to the new system. Its behaviour can be configured through some optional settings.
### User Accounts
A database migration is included to port users to the new SSO-based login system. It will automatically assign states and main characters.
Password for non-staff accounts are destroyed so they cannot be used to log in - only SSO is available for regular members. Staff can login using username and password through the admin site or by SSO.
### EVE Characters
Character ownership is now tracked by periodically validating a refreshable SSO token. When migrating from v1 not all users may have such a refreshable token: to allow these users to retain their existing user account their main character is set to the one present in v1, bypassing this validation mechanism.
It is essential to get your users to log in via SSO soon after migration to ensure their accounts are managed properly. Any user who does not log in will not lose their main character under any circumstance, even character sale. During a sale characters are transferred to an NPC corp - get celery tasks running within 24 hours of migration so that during a sale the user loses their state (and hence permissions). This can be an issue if you've granted service access permissions to a user or their groups: it's strongly encouraged to grant service access by state from now on.
Because character ownership is tracked separately of main character it is not possible to preserve alts unless a refreshable SSO token is present for them.
### Members and Blues
The new [state system](../../features/core/states.md) allows configuring dynamic membership states through the admin page. Unfortunately if you make a change after migrating it will immediately assess user states and see that no one should be a member. You can add additional settings to your auth project's settings file to generate the member and blue states as you have them defined in v1:
- `ALLIANCE_IDS = []` a list of member alliance IDs
- `CORP_IDS = []` a list of member corporation IDs
- `BLUE_ALLIANCE_IDS = []` a list of blue alliance IDs
- `BLUE_CORP_IDS = []` a list of blue corporation IDs
Put comma-separated IDs into the brackets and the migration will create states with the members and blues you had before. This will prevent unexpected state purging when you edit states via the admin site.
### Default Groups
If you used member/blue group names other than the standard "Member" and "Blue" you can enter settings to have the member/blue states created through this migration take these names.
- `DEFAULT_AUTH_GROUP = ""` the desired name of the "Member" state
- `DEFAULT_BLUE_GROUP = ""` the desired name of the "Blue" state
Any permissions assigned to these groups will be copied to the state replacing them. Because these groups are no longer managed they pose a security risk and so are deleted at the end of the migration automatically.
### Run Migrations
Once you've configured any optional settings it is now safe to run the included migrations.
## Validating Upgrade
Before starting the Celery workers it's a good idea to validate the states were created and assigned correctly. Any mistakes now will trigger deletion of service accounts: if Celery workers aren't running these tasks aren't yet processed. States can be checked through the admin site.
The site (and not Celery) can be started with `supervisorctl start myauth:gunicorn`. Then navigate to the admin site and log in with your v1 username and password. States and User Profiles can be found under the Authentication app.
Once you have confirmed states are correctly configured (or adjusted them as needed) clear the Celery task queue: from within your auth project folder, `celery -A myauth purge`
It should now be safe to bring workers online (`supervisorctl start myauth:`) and invite users to use the site.
## Securing User Accounts
After migration users will have main characters without a method to check for character sales. After a brief period to allow your users to log in (and provide a refreshable token to use), it's a good idea to clear main characters of users who haven't yet logged in. This can be achieved with an included command: `python manage.py checkmains`
A similar process can be used to ensure users who may have lost service permissions during the migration do not have active service accounts. This is done using `python manage.py validate_service_accounts`.
## Help
If something goes wrong during the migration reach out for help on [Gitter](https://gitter.im/R4stl1n/allianceauth) or open an [issue](https://gitlab.com/allianceauth/allianceauth/issues).