Basraah 1066e6ac98 The Great Services Refactor (#594)
* Hooks registration, discovery and retrieval module

Will discover @hooks.register decorated functions inside
the auth_hooks module in any installed django app.

* Class to register modular service apps

* Register service modules URLs

* Example service module

* Refactor services into modules

Each service type has been split out into its own django app/module. A
hook mechanism is provided to register a subclass of the ServiceHook
class. The modules then overload functions defined in ServiceHook as
required to provide interoperability with alliance auth. Service modules
provide their own urls and views for user registration and account
management and a partial template to display on the services page. Where
possible, new modules should provide their own models for local data
storage.

* Added menu items hooks and template tags

* Added menu item hook for broadcasts

* Added str method to ServicesHook

* Added exception handling to hook iterators

* Refactor mumble migration and table name

Upgrading will require `migrate mumble --fake-initial` to be run first
and then `migrate mumble` to rename the table.

* Refactor teamspeak3 migration and rename table

Upgrading will require `migrate teamspeak3 --fake-initial`

* Added module models and migrations for refactoring AuthServicesInfo

* Migrate AuthServiceInfo fields to service modules models

* Added helper for getting a users main character

* Added new style celery instance

* Changed Discord from AuthServicesInfo to DiscordUser model

* Switch celery tasks to staticmethods

* Changed Discourse from AuthServicesInfo to DiscourseUser model

* Changed IPBoard from AuthServicesInfo to IpboardUser model

* Changed Ips4 from AuthServicesInfo to Ips4User model

Also added disable service task.

This service still needs some love though. Was always missing a
deactivate services hook (before refactoring) for reasons I'm unsure of
so I'm reluctant to add it without knowing why.

* Changed Market from AuthServicesInfo to MarketUser model

* Changed Mumble from AuthServicesInfo to MumbleUser model

Switched user foreign key to one to one relationship.
Removed implicit password change on user exists.
Combined regular and blue user creation.

* Changed Openfire from AuthServicesInfo to OpenfireUser model

* Changed SMF from AuthServicesInfo to SmfUser model

Added disable task

* Changed Phpbb3 from AuthServicesInfo to Phpbb3User model

* Changed XenForo from AuthServicesInfo to XenforoUser model

* Changed Teamspeak3 from AuthServicesInfo to Teamspeak3User model

* Remove obsolete manager functions

* Standardise URL format

This will break some callback URLs
Discord changes from /discord_callback/ to /discord/callback/

* Removed unnecessary imports

* Mirror upstream decorator change

* Setup for unit testing

* Unit tests for discord service

* Added add main character helper

* Added Discourse unit tests

* Added Ipboard unit tests

* Added Ips4 unit tests

* Fix naming of market manager, switch to use class methods

* Remove unused hook functions

* Added market service unit tests

* Added corp ticker to add main character helper

* Added mumble unit tests

* Fix url name and remove namespace

* Fix missing return and add missing URL

* Added openfire unit tests

* Added missing return

* Added phpbb3 unit tests

* Fix SmfManager naming inconsistency and switch to classmethods

* Added smf unit tests

* Remove unused functions, Added missing return

* Added xenforo unit tests

* Added missing return

* Fixed reference to old model

* Fixed error preventing groups from syncing on reset request

* Added teamspeak3 unit tests

* Added nose as test runner and some test settings

* Added package requirements for running tests

* Added unit tests for services signals and tasks

* Remove unused tests file

* Fix teamspeak3 service signals

* Added unit tests for teamspeak3 signals

Changed other unit tests setUp to inert signals

* Fix password gen and hashing python3 compatibility

Fixes #630

Adds unit tests to check the password functions run on both platforms.

* Fix unit test to not rely on checking url params

* Add Travis CI settings file

* Remove default blank values from services models

* Added dynamic user model admin actions for syncing service groups

* Remove unused search fields

* Add hook function for syncing nicknames

* Added discord hook for sync nickname

* Added user admin model menu actions for sync nickname hook

* Remove obsolete code

* Rename celery config app to avoid package name clash

* Added new style celerybeat schedule configuration

periodic_task decorator is depreciated

* Added string representations

* Added admin pages for services user models

* Removed legacy code

* Move link discord button to correct template

* Remove blank default fields from example model

* Disallow empty django setting

* Fix typos

* Added coverage configuration file

* Add coverage and coveralls to travis config

Should probably use nose's built in coverage, but this works for now.

* Replace AuthServicesInfo get_or_create instances with get

Reflects upstream changes to AuthServicesInfo behaviour.

* Update mumble user table name

* Split out mumble authenticator requirements

zeroc-ice seems to cause long build times on travis-ci and isn't
required for the core projects functionality or testing.
2017-01-25 12:50:16 +10:00

303 lines
13 KiB
Python
Executable File

from __future__ import unicode_literals
import random
import string
import calendar
import re
from datetime import datetime
from passlib.apps import phpbb3_context
from django.db import connections
import logging
from django.conf import settings
logger = logging.getLogger(__name__)
class Phpbb3Manager:
SQL_ADD_USER = r"INSERT INTO phpbb_users (username, username_clean, " \
r"user_password, user_email, group_id, user_regdate, user_permissions, " \
r"user_sig) VALUES (%s, %s, %s, %s, %s, %s, %s, %s)"
SQL_DEL_USER = r"DELETE FROM phpbb_users where username = %s"
SQL_DIS_USER = r"UPDATE phpbb_users SET user_email= %s, user_password=%s WHERE username = %s"
SQL_USER_ID_FROM_USERNAME = r"SELECT user_id from phpbb_users WHERE username = %s"
SQL_ADD_USER_GROUP = r"INSERT INTO phpbb_user_group (group_id, user_id, user_pending) VALUES (%s, %s, %s)"
SQL_GET_GROUP_ID = r"SELECT group_id from phpbb_groups WHERE group_name = %s"
SQL_ADD_GROUP = r"INSERT INTO phpbb_groups (group_name,group_desc,group_legend) VALUES (%s,%s,0)"
SQL_UPDATE_USER_PASSWORD = r"UPDATE phpbb_users SET user_password = %s WHERE username = %s"
SQL_REMOVE_USER_GROUP = r"DELETE FROM phpbb_user_group WHERE user_id=%s AND group_id=%s "
SQL_GET_ALL_GROUPS = r"SELECT group_id, group_name FROM phpbb_groups"
SQL_GET_USER_GROUPS = r"SELECT phpbb_groups.group_name FROM phpbb_groups , phpbb_user_group WHERE " \
r"phpbb_user_group.group_id = phpbb_groups.group_id AND user_id=%s"
SQL_ADD_USER_AVATAR = r"UPDATE phpbb_users SET user_avatar_type=2, user_avatar_width=64, user_avatar_height=64, " \
"user_avatar=%s WHERE user_id = %s"
SQL_CLEAR_USER_PERMISSIONS = r"UPDATE phpbb_users SET user_permissions = '' WHERE user_Id = %s"
SQL_DEL_SESSION = r"DELETE FROM phpbb_sessions where session_user_id = %s"
SQL_DEL_AUTOLOGIN = r"DELETE FROM phpbb_sessions_keys where user_id = %s"
def __init__(self):
pass
@staticmethod
def __add_avatar(username, characterid):
logger.debug("Adding EVE character id %s portrait as phpbb avater for user %s" % (characterid, username))
avatar_url = "https://image.eveonline.com/Character/" + characterid + "_64.jpg"
cursor = connections['phpbb3'].cursor()
userid = Phpbb3Manager.__get_user_id(username)
cursor.execute(Phpbb3Manager.SQL_ADD_USER_AVATAR, [avatar_url, userid])
@staticmethod
def __generate_random_pass():
return ''.join([random.choice(string.ascii_letters + string.digits) for n in range(16)])
@staticmethod
def __gen_hash(password):
return phpbb3_context.encrypt(password)
@staticmethod
def __santatize_username(username):
sanatized = username.replace(" ", "_")
sanatized = sanatized.replace("'", "_")
return sanatized.lower()
@staticmethod
def _sanitize_groupname(name):
name = name.strip(' _')
return re.sub('[^\w.-]', '', name)
@staticmethod
def __get_group_id(groupname):
logger.debug("Getting phpbb3 group id for groupname %s" % groupname)
cursor = connections['phpbb3'].cursor()
cursor.execute(Phpbb3Manager.SQL_GET_GROUP_ID, [groupname])
row = cursor.fetchone()
logger.debug("Got phpbb group id %s for groupname %s" % (row[0], groupname))
return row[0]
@staticmethod
def __get_user_id(username):
logger.debug("Getting phpbb3 user id for username %s" % username)
cursor = connections['phpbb3'].cursor()
cursor.execute(Phpbb3Manager.SQL_USER_ID_FROM_USERNAME, [username])
row = cursor.fetchone()
if row is not None:
logger.debug("Got phpbb user id %s for username %s" % (row[0], username))
return row[0]
else:
logger.error("Username %s not found on phpbb. Unable to determine user id." % username)
return None
@staticmethod
def __get_all_groups():
logger.debug("Getting all phpbb3 groups.")
cursor = connections['phpbb3'].cursor()
cursor.execute(Phpbb3Manager.SQL_GET_ALL_GROUPS)
rows = cursor.fetchall()
out = {}
for row in rows:
out[row[1]] = row[0]
logger.debug("Got phpbb groups %s" % out)
return out
@staticmethod
def __get_user_groups(userid):
logger.debug("Getting phpbb3 user id %s groups" % userid)
cursor = connections['phpbb3'].cursor()
cursor.execute(Phpbb3Manager.SQL_GET_USER_GROUPS, [userid])
out = [row[0] for row in cursor.fetchall()]
logger.debug("Got user %s phpbb groups %s" % (userid, out))
return out
@staticmethod
def __get_current_utc_date():
d = datetime.utcnow()
unixtime = calendar.timegm(d.utctimetuple())
return unixtime
@staticmethod
def __create_group(groupname):
logger.debug("Creating phpbb3 group %s" % groupname)
cursor = connections['phpbb3'].cursor()
cursor.execute(Phpbb3Manager.SQL_ADD_GROUP, [groupname, groupname])
logger.info("Created phpbb group %s" % groupname)
return Phpbb3Manager.__get_group_id(groupname)
@staticmethod
def __add_user_to_group(userid, groupid):
logger.debug("Adding phpbb3 user id %s to group id %s" % (userid, groupid))
try:
cursor = connections['phpbb3'].cursor()
cursor.execute(Phpbb3Manager.SQL_ADD_USER_GROUP, [groupid, userid, 0])
cursor.execute(Phpbb3Manager.SQL_CLEAR_USER_PERMISSIONS, [userid])
logger.info("Added phpbb user id %s to group id %s" % (userid, groupid))
except:
logger.exception("Unable to add phpbb user id %s to group id %s" % (userid, groupid))
pass
@staticmethod
def __remove_user_from_group(userid, groupid):
logger.debug("Removing phpbb3 user id %s from group id %s" % (userid, groupid))
try:
cursor = connections['phpbb3'].cursor()
cursor.execute(Phpbb3Manager.SQL_REMOVE_USER_GROUP, [userid, groupid])
cursor.execute(Phpbb3Manager.SQL_CLEAR_USER_PERMISSIONS, [userid])
logger.info("Removed phpbb user id %s from group id %s" % (userid, groupid))
except:
logger.exception("Unable to remove phpbb user id %s from group id %s" % (userid, groupid))
pass
@staticmethod
def add_user(username, email, groups, characterid):
logger.debug("Adding phpbb user with username %s, email %s, groups %s, characterid %s" % (
username, email, groups, characterid))
cursor = connections['phpbb3'].cursor()
username_clean = Phpbb3Manager.__santatize_username(username)
password = Phpbb3Manager.__generate_random_pass()
pwhash = Phpbb3Manager.__gen_hash(password)
logger.debug("Proceeding to add phpbb user %s and pwhash starting with %s" % (username_clean, pwhash[0:5]))
# check if the username was simply revoked
if Phpbb3Manager.check_user(username_clean):
logger.warn("Unable to add phpbb user with username %s - already exists. Updating user instead." % username)
Phpbb3Manager.__update_user_info(username_clean, email, pwhash)
else:
try:
cursor.execute(Phpbb3Manager.SQL_ADD_USER, [username_clean, username_clean, pwhash,
email, 2, Phpbb3Manager.__get_current_utc_date(),
"", ""])
Phpbb3Manager.update_groups(username_clean, groups)
Phpbb3Manager.__add_avatar(username_clean, characterid)
logger.info("Added phpbb user %s" % username_clean)
except:
logger.exception("Unable to add phpbb user %s" % username_clean)
pass
return username_clean, password
@staticmethod
def disable_user(username):
logger.debug("Disabling phpbb user %s" % username)
cursor = connections['phpbb3'].cursor()
password = Phpbb3Manager.__gen_hash(Phpbb3Manager.__generate_random_pass())
revoke_email = "revoked@" + settings.DOMAIN
try:
pwhash = Phpbb3Manager.__gen_hash(password)
cursor.execute(Phpbb3Manager.SQL_DIS_USER, [revoke_email, pwhash, username])
userid = Phpbb3Manager.__get_user_id(username)
cursor.execute(Phpbb3Manager.SQL_DEL_AUTOLOGIN, [userid])
cursor.execute(Phpbb3Manager.SQL_DEL_SESSION, [userid])
Phpbb3Manager.update_groups(username, [])
logger.info("Disabled phpbb user %s" % username)
return True
except TypeError:
logger.exception("TypeError occured while disabling user %s - failed to disable." % username)
return False
@staticmethod
def delete_user(username):
logger.debug("Deleting phpbb user %s" % username)
cursor = connections['phpbb3'].cursor()
if Phpbb3Manager.check_user(username):
cursor.execute(Phpbb3Manager.SQL_DEL_USER, [username])
logger.info("Deleted phpbb user %s" % username)
return True
logger.error("Unable to delete phpbb user %s - user not found on phpbb." % username)
return False
@staticmethod
def update_groups(username, groups):
userid = Phpbb3Manager.__get_user_id(username)
logger.debug("Updating phpbb user %s with id %s groups %s" % (username, userid, groups))
if userid is not None:
forum_groups = Phpbb3Manager.__get_all_groups()
user_groups = set(Phpbb3Manager.__get_user_groups(userid))
act_groups = set([Phpbb3Manager._sanitize_groupname(g) for g in groups])
addgroups = act_groups - user_groups
remgroups = user_groups - act_groups
logger.info("Updating phpbb user %s groups - adding %s, removing %s" % (username, addgroups, remgroups))
for g in addgroups:
if not g in forum_groups:
forum_groups[g] = Phpbb3Manager.__create_group(g)
Phpbb3Manager.__add_user_to_group(userid, forum_groups[g])
for g in remgroups:
Phpbb3Manager.__remove_user_from_group(userid, forum_groups[g])
@staticmethod
def remove_group(username, group):
logger.debug("Removing phpbb user %s from group %s" % (username, group))
cursor = connections['phpbb3'].cursor()
userid = Phpbb3Manager.__get_user_id(username)
if userid is not None:
groupid = Phpbb3Manager.__get_group_id(group)
if userid:
if groupid:
try:
cursor.execute(Phpbb3Manager.SQL_REMOVE_USER_GROUP, [userid, groupid])
logger.info("Removed phpbb user %s from group %s" % (username, group))
except:
logger.exception(
"Exception prevented removal of phpbb user %s with id %s from group %s with id %s" % (
username, userid, group, groupid))
pass
@staticmethod
def check_user(username):
logger.debug("Checking phpbb username %s" % username)
cursor = connections['phpbb3'].cursor()
cursor.execute(Phpbb3Manager.SQL_USER_ID_FROM_USERNAME, [Phpbb3Manager.__santatize_username(username)])
row = cursor.fetchone()
if row:
logger.debug("Found user %s on phpbb" % username)
return True
logger.debug("User %s not found on phpbb" % username)
return False
@staticmethod
def update_user_password(username, characterid, password=None):
logger.debug("Updating phpbb user %s password" % username)
cursor = connections['phpbb3'].cursor()
if not password:
password = Phpbb3Manager.__generate_random_pass()
if Phpbb3Manager.check_user(username):
pwhash = Phpbb3Manager.__gen_hash(password)
logger.debug(
"Proceeding to update phpbb user %s password with pwhash starting with %s" % (username, pwhash[0:5]))
cursor.execute(Phpbb3Manager.SQL_UPDATE_USER_PASSWORD, [pwhash, username])
Phpbb3Manager.__add_avatar(username, characterid)
logger.info("Updated phpbb user %s password." % username)
return password
logger.error("Unable to update phpbb user %s password - user not found on phpbb." % username)
return ""
@staticmethod
def __update_user_info(username, email, password):
logger.debug(
"Updating phpbb user %s info: username %s password of length %s" % (username, email, len(password)))
cursor = connections['phpbb3'].cursor()
try:
cursor.execute(Phpbb3Manager.SQL_DIS_USER, [email, password, username])
logger.info("Updated phpbb user %s info" % username)
except:
logger.exception("Unable to update phpbb user %s info." % username)
pass