4. Administration

The BI Explorer is a python web application that runs from a virtualenv environment against a PostgreSQL server. The database connection parameters are set in the explorer configuration file.

4.1. Explorer Configuration

The following sections describe the sections of explorer configuration file. On disk, this file appears as a single complete file consisting of all code sections in sequence.

4.1.1. Main application

Configuration start with the definition of our application called explorer and the fact that application code can be found in the local environment as a python egg.

[app:explorer]
use = egg:explorer

4.1.2. Pyramid

BI Explorer is built on top of pyramid. Pyramid can be made to emit debugging information and show a debugging toolbar. For production use these should all be turned off:

pyramid.reload_templates = false
pyramid.debug_authorization = false
pyramid.debug_notfound = false
pyramid.debug_routematch = false
pyramid.default_locale_name = en
#pyramid.includes = pyramid_debugtoolbar

4.1.3. Sessions

Server side caches are tied to users using Web sessions. Users receive a user specific http-cookie that facilitates quick continuation of a session without additional login. session.secret is an arbitrary secret that is used to encode the session data before committing it to disk. It should be set to a new value for every explorer installation. session.data_dir defines in which data directory the session data should be stored. This directory should only be readable by the explorer application. If the webapplication is stopped, the data directory can be deleted to purge all standing user sessions. It will be recreated and populated after the server starts.

The session times out after one day.

session.type = file
session.data_dir = %(here)s/data/sessions/data
session.lock_dir = %(here)s/data/sessions/lock
session.key = explorer
session.secret = FG87sjsAKdoqQweXC

4.1.4. Program caches

The explorer uses program caches to store information that is used often. These caches are tied to webserver sessions, and are not shared between users.

cache.regions = short_term
cache.type = memory
cache.second.expire = 1
cache.short_term.expire = 3600

4.1.5. User security

User passwords are salted, peppered and hashed using sha512 to the database. The salt is randomly generated and stored per user. The security.cookiesecret is used to encode data in the cookie before sending it to the user. secret.tablename finally, is the tablename where data for a particular user lives. The explorer will resolve this as userid_schema.tablename.

Make sure pepper, cookiesecret are set differently per explorer installation.

security.cookiesecret = FwBUd5VG65Ghjq1
security.cookiehashalg = sha512
security.pepper = testing
security.tablename = explorer

4.1.6. Database

All database interactions are through an in application database pooler. The configuration is straightforward:

dbpool.minconns = 1
dbpool.maxconns = 5
dbpool.host = 127.0.0.1
dbpool.user = mgrid
dbpool.database = dwh

4.1.7. Explorer

Branding, filenames for downloads, seperators used in pivot construction, canned report server configuration can be found here.

# Explorer settings
explorer.logo = mgrid_logo.png
explorer.title = Patient Explorer
explorer.tabledefaultwindowsize = 50
# Filenames; note that the current date, time and extension are appended automatically
explorer.filecsv = patients
explorer.filetab = patients
explorer.filesql = patients
# String used in pivot bucket calculations, must not occur in data columns
explorer.pivotsep = '@@~SEP~@@'
explorer.pivotnull = '@@~NULL~@@'

explorer.reportsurl = https://reportserver:9443
explorer.reportscacert = ../certs/reportsca.crt
explorer.certificate = ../certs/explorer.crt
explorer.privatekey = ../certs/explorer.pem

Explorer can also be configured in a debug mode that defines additional uris using this statement:

# Enable test-settings for view exports
explorer.webtests = true

4.1.8. WSGI

Python container configuration that exposes the application on the localhost using the waitress webservice. Note that nginx is used to expose the server to the outside world.

# pipeline configuration
[pipeline:main]
pipeline =
    paste_prefix
    explorer

[filter:paste_prefix]
use = egg:PasteDeploy#prefix

[server:main]
use = egg:waitress#main
host = 127.0.0.1
port = 6543

4.2. SSL configuration

nginx is used to expose the web application externally. By default the Python webcontainer is configured to only serve on 127.0.0.1. Additional configuration in /etc/nginx/nginx.conf:

user              nginx;
worker_processes  1;

error_log  /var/log/nginx/error.log;
pid        /var/run/nginx.pid;

events {
    worker_connections  1024;
}

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile        on;

    keepalive_timeout  65;

    include /etc/nginx/conf.d/*.conf;

    # disable SSLv3 (POODLE attack/CVE-2014-3566)
    ssl_protocols  TLSv1 TLSv1.1 TLSv1.2;

}

With /etc/nginx/conf.d/explorer.conf:

upstream explorer-site {
    server 127.0.0.1:6543;
}

server {
    listen 443 ssl;
    server_name  explorer.outsidename.nl;
    ssl_certificate /etc/ssl/explorer/server.crt;
    ssl_certificate_key /etc/ssl/explorer/server.key;

    location / {
        proxy_set_header        Host $http_host;
        proxy_set_header        X-Real-IP $remote_addr;
        proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header        X-Forwarded-Proto $scheme;

        client_max_body_size    10m;
        client_body_buffer_size 128k;
        proxy_connect_timeout   60s;
        proxy_send_timeout      90s;
        proxy_read_timeout      90s;
        proxy_buffering         off;
        proxy_temp_file_write_size 64k;
        proxy_pass http://explorer-site;
        proxy_redirect          off;
    }
}

4.3. Boot at startup

The explorer application requires a PostgreSQL database server. This should be available before the explorer application is started. A supervisord.conf [1] is supplied and can be used to ensure that the application is started at boottime.

[1]The user to be used for running both supervisord and explorer can be set here.

There are supervisord packages for most linux distributions. These typically define a supervisord service that can be enabled at boot time. The exact location of the supervisord.conf varies. Some distributions also define a /etc/supervisord/conf.d/ where only the [program:explorer] snippet from the example configuration is required.

For debugging use supervisord can also be started by hand:

supervisord -c $EXPLORER_INSTALL/Explorer/supervisord.conf

This will automatically start the explorer application and allow management via supervisorctl:

$ supervisorctl -c $EXPLORER_INSTALL/Explorer/supervisord.conf
explorer:explorer-0              RUNNING   pid 3789, uptime 0:05:37
supervisor> ?

default commands (type help <topic>):
=====================================
add    clear  fg        open  quit    remove  restart   start   stop  update
avail  exit   maintail  pid   reload  reread  shutdown  status  tail  version

supervisor> tail explorer:explorer-0
2015-02-02 17:51:24,516 INFO  [explorer][MainThread] Configuration loaded

supervisor> restart explorer:*
explorer:explorer-0: stopped
explorer:explorer-0: started

supervisor> tail explorer:explorer-0
2015-02-02 17:51:24,516 INFO  [explorer][MainThread] Configuration loaded
2015-02-02 17:52:07,994 INFO  [explorer][MainThread] Configuration loaded

Default configuration will:

  • Auto restart explorer at errors. Note that 5x quickly respawning explorers will make autorespawn stop.
  • Log explorer stdout log output to $EXPLORER_INSTALL/pyenv/explorer-0.log

4.4. User Management

User management is done using the command line tool useradmin.py. This tool determines database location using a supplied configuration file, and can be instructed to add, remove and list users. There is a facility to change a stored password for a user, and to reset the locked views for a user to the views as set in config/views.py.

$ python useradmin.py -h
usage: useradmin.py [-h] -c CONFIG
                    (-a user schema password | -r user | -p user password |
                     -l | -x schema)

Administation tool for Explorer user table

optional arguments:
  -h, --help            show this help message and exit
  -c CONFIG, --config CONFIG
                        explorer configuration file
  -a user schema password, --add user schema password
                        add a user
  -r user, --remove user
                        remove a user
  -p user password, --password user password
                        change a stored password
  -l, --list            show the known users
  -x schema, --resetpreset schema
                        reset user presets