In 2015 major release 3 was introduced for the infrastructure components HDL, HDM, MSG and XFM. The highlights of the changes are the following:

Healthcare Datatype Library (HDL)

See http://www.mgrid.net/static/doc/hdl/3.0/html/history.html for a detailed list of changes.

Change to JSONB disk format

The on-disk format for the JSON complex types was changed from an internal format to the PostgreSQL core JSONB format. This way there is benefit from upstream JSONB bugfixes, performance improvements en feature enhancements. All new JSONB functions added in PostgreSQL 9.5, such as jsonb_pretty(), concatenation operator || and new JSONB generator functions, as well as the JsQuery extension, are compatible with the complex healthcare datatypes.

Performance enhancements from the past couple of years have made standard PostgreSQL a good candidate to perform analytical workloads. In the future this is expected to improve even further as columnar storage for core PostgreSQL is in development. For large projects that require scale-out, Postgres-XL has recently merged the latest PostgreSQL 9.5 changes, and is actively being developed and supported.

Because of the new disk format, release 3 does not run on Greenplum database, and support is currently not planned.

Remove of hdl.pretty_print configuration setting

This feature was part of the original JSON format code base, and not present in the core PostgreSQL JSONB code. In PostgreSQL 9.5, the jsonb_pretty() function is available.

Healthcare Data Model (HDM)

There are no apparent changes in the supplied table-per-RIM-class database models and associated vocabulary between version 2 and version 3. All version 2 HDM databases can be exported with a logical export using pg_dump and imported in a version 3 database, using the commands described at http://www.mgrid.net/static/doc/hdl/3.0/html/administration.html#sql-dump-and-restore. Note that since the on-disk binary format is changed, a physical upgrade is not possible.

New NoSQL-like storage for JSONB documents

There is a new option available to configure databases, with the new conversion to whole JSONB documents in the MGRID Messaging SDK. This enables a NoSQL kind of approach to storing healthcare data, by storing the complete JSONB documents in a column in the database. This method can be used in conjunction with the ‘shredded’ table-per-RIM-class traditional models.

In the latest tests in the AXLE project, this ‘dual head’ setup was used. The patient, organization and practitioner update messages were routed to the table-per-RIM-class model, and all remaining ‘data’ CDA documents were routed to the JSONB document table. Since this setup had a constant translation time for the bulk of the messages (the data CDA documents), that does not slow down with respect of the size of the result database, this architecture proved extremely scalable. The per-message translation time was reduced and the amount of hits on RabbitMQ was reduced from three times to one publish/consume per message. On a single server the performance was improved from 350 msg/s to 1500 msg/s.

Messaging SDK

The release log for mgrid messaging can be found at http://www.mgrid.net/static/doc/msg/3.0/html/history.html.

Conversion to JSON

Until version 2, XML messages could only be translated to a so called ‘anonymous PL/pgSQL’ block, a set of instructions for interpretation by a MGRID HDM database with the hl7v3crud extension installed.

In version 3 there is a new target format for translation of a message: a complete JSON document. This is the original XML message in JSON form, where the message is augmented with information from the MIF, such as MIF names, clonenames and rim class names for each object. Object parts of the JSON documents that correspond to a single datatype value are JSON objects by themselves and are compatible with the JSON r1 and r2 healthcare datatypes.

The big benefit of this new format, adoption of NoSQL approaches to persistence of large amounts of data, is described in the previous section.

Message conversion factory

A new message converter factory was added, that allows switching of the parser by editing a configuration file, rather than changing code. Consider the following example:

#
# parse POCD_MT000040 aka CDA R2 messages and rewrite them to MGRID SQL
#
# Copyright (c) 2015, MGRID BV Netherlands
#
# -*- coding: utf-8 -*-

from mgridenv import log, files
from lib.messageconverterfactory import MessageConverterFactory
from config import conversion

mcf = MessageConverterFactory(log, conversion)

if len(files) < 1:
    raise ValueError("Please provide a source file")

for i in files:
    try:
        print mcf.convert('CDA_R2_NE2010', open(i).read())
    except IOError as e:
        log.error('Could not read file %s' % i)
        exit(2)
    except SyntaxError as e:
        log.error('Syntax error in file %s: %s' % (i, e.msg))
        exit(2)

And associated config.py:

conversion = {
    'output': 'sql',
    'rim': 'rim229R1',
    'interactions': {
        'CDA_R2_NE2010' : {
            'parser' : 'generated.parser.CDA_R2_NE2010.CDA_parser',
            'mt': ['generated.mif.CDA_R2_NE2010.POCD_MT000040'],
            'fixmt': [],
            'in' : []
            }
        }
    }

MGRID Messaging version 3 is not backwards compatible, since the MessageConverter class is renamed to Msg2SQL.

Message Transformation XFM

Main new features of XFM are orchestrating version 3 of the HDL, HDM and Messaging components, integration of the new MGRID package repository, support for Enterprise Linux 7, and packaged deployment configurations. This last feature allows creating packages which describe custom deployment configurations (e.g., specific broker configuration, message validators, message parsers, etc.). The XFM installation tools setup a running system based on the configuration.

There are no specific backwards incompatibility features added to XFM.