4. Message Conversion

_images/messageconversion.png

A message converter translates HL7v3 messages into SQL insert statements, or a JSON document.

In order to perform this function it requires:

  • A description of the database tables as defined by the RIM. Note that because these are not hardwired, you are free to choose any (normative) RIM edition.
  • A description of the datatypes defined in this RIM. This helps to determine if an incoming XML element should result in either a database record or one of its columns.
  • A description of the cloned classes used in the message. Cloned classes are RIM classes constrained for a specific message. These class maps allow us to map an incoming message fragment into its RIM representation.
  • A parser that reads the incoming xml message and exposes the message as a DOM tree.

These descriptions and parser are all generated from the model that describes the message. This model is called a RMIM and can be found in the a HL7 Normative Edition for standard models, or be specified using the HL7v3 RMIM Designer for local models.

Note

The first two descriptions stem from the RIM used, the third is from the message MIF and the fourth from the message XSD.

Two possible conversion targets are supported:

  • A conversion from the message into its RIM elements. The conversion will return a set of insert statements that together represent the entire message.
  • A conversion from the message into a single JSON statement, ready for database insertion. The incoming message will still be investigated for its RIM elements, and identified elements will be annotated in the JSON.

Each of the generators that provide the input necessary for conversion explored in the sections below. In typical projects a single Makefile will generate the descriptions and parser automatically. All these are then combined into a conversion program.

4.1. Translation to SQL

Here is an explicit example conversion program that translates CDA R2 messages:

from mgridenv import log
import sys

from lib.msg2sql import Msg2SQL

import generated.rim.rim210 as rim
from generated.rim.rim210_dt import xml2dt as datatypes
import generated.mif.CDA_R2_NE2010.POCD_MT000040
import generated.parser.CDA_R2_NE2010.CDA_parser as parser

knownmt = ['POCD_MT000040']

root = parser.parse(sys.argv[1])
converter = Msg2SQL(log = log, rim = rim, parser = parser, \
                             datatypes = datatypes, knownmt = knownmt)
print converter.convert(root)

The result of this conversion program on a CDA R2 message is single string transaction block, containing MGRID HDM insert function calls for all RIM objects in the message. Statements are ordered such that leaf objects get inserted first, and objects pointing to leafs are inserted last. The relations between objects is maintained using the id that is returned by the insert functions. To give a general idea of what the output looks like, here is an abbreviated example of a CDA R2 import:

DO
$$DECLARE
document0 bigint;
person1 bigint;
organization2 bigint;
role52 bigint;
participation100 bigint;
BEGIN
document0 := Document_insert(_mif := 'POCD_MT000040', _clonename := 'ClinicalDocument',
   "classCode" := 'DOCCLIN', "code" := ceinval(code := '18842-5',
      codeSystem := '2.16.840.1.113883.6.1', codeSystemName := 'LOINC',
      displayName := 'SUMMARIZATION OF EPISODE NOTE'),
   "confidentialityCode" := ceinval(code := 'N', codeSystem := '2.16.840.1.113883.5.25',
      codeSystemName := 'Confidentiality', displayName := 'Normal'));

person1 := Person_insert(_mif := 'POCD_MT000040', _clonename := 'Person',
   "classCode" := 'PSN', "determinerCode" := 'INSTANCE',
   "name" := ARRAY[pninval(family := ARRAY[en_familyinval(
      mediaType := 'text/plain', representation := 'TXT', value := 'Admin')])]);

organization2 := Organization_insert(_mif := 'POCD_MT000040',
   _clonename := 'Organization', "addr" := ARRAY[adinval(
      city := ARRAY[adxp_cityinval(mediaType := 'text/plain',
         representation := 'TXT', value := 'Amsterdam')])], "classCode" := 'ORG',
   "determinerCode" := 'INSTANCE');

role52 := Role_insert(_mif := 'POCD_MT000040', _clonename := 'AssignedAuthor',
   "classCode" := 'ASSIGNED', "player" := person1, "scoper" := organization2);

participation100 := Participation_insert(_mif := 'POCD_MT000040',
   _clonename := 'Author', "act" := document0, "contextControlCode" := 'OP',
   "role" := role52, "time" := tsinval(value := '20091028102753+0200'),
   "typeCode" := 'AUT');

END$$;

Import points to note are:

  • Original clone names are stored in the inserts. This helps identifying RIM fragments in the database, but also allows the DB Conversion to work back from RIM instances to Clone instances.
  • The INSERT statements return database ids for each RIM row inserted. The ids are used to tie classes that point to each other together. Note the ties between Participation, Role and Document in the second to last line above, for example.

4.2. Translation to JSON

The next example shows how to call a parser for conversion to JSON. In addition, this example also demonstrates how to use the MessageConverterFactory to select the right parser.

from mgridenv import log, files
from lib.messageconverterfactory import MessageConverterFactory

conversion = {
    'output': 'json',
    'rim': 'rim229R1',
    'interactions': {
        'CDA_R2_NE2010' : {
            'parser' : 'generated.parser.CDA_R2_NE2010.CDA_parser',
            'mt': ['generated.mif.CDA_R2_NE2010.POCD_MT000040'],
            'fixmt': [],
            'in' : []
            }
        }
    }

mcf = MessageConverterFactory(log, conversion)

if len(files) < 1:
    raise ValueError("Please provide a source file")

for i in files:
    print mcf.convert('CDA_R2_NE2010', open(i).read())

The result of this conversion is a JSON string, where individual datatypes conform to the MGRID HDL complex datatypes, and RIM classes are extended with a number of additional properties from the associated MIFs.

{"_mif":"POCD_MT000040","_clonename":"ClinicalDocument","_rimname":"Document","author":[{"_mif":"POCD_MT000040","_clonename":"Author","_rimname":"Participation","assignedAuthor":{"_mif":"POCD_MT000040","_clonename":"AssignedAuthor","_rimname":"Role","classCode":"ASSIGNED","id":[{"dataType":"II","extension":"-955258132","root":"2.16.840.1.113883.2.4.3.31.3.2"}],"representedOrganization":{"_mif":"POCD_MT000040","_clonename":"Organization","_rimname":"Organization","classCode":"ORG","determinerCode":"INSTANCE","id":[{"dataType":"II","extension":"347763604","root":"2.16.840.1.113883.2.4.3.31.3.2"}]}},"contextControlCode":"OP","time":{"dataType":"TS","value":"20141215062206"},"typeCode":"AUT"}],"classCode":"DOCCLIN","code":{"dataType":"CE","code":"68608-9","codeSystem":"2.16.840.1.113883.6.1","codeSystemName":"LOINC","displayName":"Summarization note"},"component":{"_mif":"POCD_MT000040","_clonename":"Component2","_rimname":"ActRelationship","structuredBody":{"_mif":"POCD_MT000040","_clonename":"StructuredBody","_rimname":"Act","classCode":"DOCBODY","component":[{"_mif":"POCD_MT000040","_clonename":"Component3","_rimname":"ActRelationship","section":{"_mif":"POCD_MT000040","_clonename":"Section","_rimname":"Act","classCode":"DOCSECT","entry":[{"_mif":"POCD_MT000040","_clonename":"Entry","_rimname":"ActRelationship","organizer":{"_mif":"POCD_MT000040","_clonename":"Organizer","_rimname":"Act","classCode":"BATTERY","code":{"dataType":"CD","code":"Portavita600","codeSystem":"2.16.840.1.113883.2.4.3.31.2.1","codeSystemName":"Portavita","displayName":"Physiotherapy"},"component":[{"_mif":"POCD_MT000040","_clonename":"Component4","_rimname":"ActRelationship","organizer":{"_mif":"POCD_MT000040","_clonename":"Organizer","_rimname":"Act","classCode":"BATTERY","code":{"dataType":"CD","code":"Portavita601","codeSystem":"2.16.840.1.113883.2.4.3.31.2.1","codeSystemName":"Portavita","displayName":"Treatment goals"},"component":[{"_mif":"POCD_MT000040","_clonename":"Component4","_rimname":"ActRelationship","organizer":{"_mif":"POCD_MT000040","_clonename":"Organizer","_rimname":"Act","classCode":"BATTERY","code":{"dataType":"CD","code":"Portavita602","codeSystem":"2.16.840.1.113883.2.4.3.31.2.1","codeSystemName":"Portavita","displayName":"Informing patient"},"component":[{"_mif":"POCD_MT000040","_clonename":"Component4","_rimname":"ActRelationship","observation":{"_mif":"POCD_MT000040","_clonename":"Observation","_rimname":"Observation","classCode":"OBS","code":{"dataType":"CD","code":"Portavita603","codeSystem":"2.16.840.1.113883.2.4.3.31.2.1","codeSystemName":"Portavita","displayName":"information"},"effectiveTime":{"dataType":"IVL_TS","low":{"dataType":"IVXB_TS","inclusive":true,"value":"20120925230218"},"operator":"I"},"id":[{"dataType":"II","extension":"1681665382","root":"2.16.840.1.113883.2.4.3.31.3.1"}],"moodCode":"EVN","statusCode":{"dataType":"CS","code":"completed"},"value":[{"dataType":"CD","code":"373066001","codeSystem":"2.16.840.1.113883.6.96","codeSystemName":"SNOMED-CT","displayName":"yes"}]},"typeCode":"COMP"},{"_mif":"POCD_MT000040","_clonename":"Component4","_rimname":"ActRelationship","observation":{"_mif":"POCD_MT000040","_clonename":"Observation","_rimname":"Observation","classCode":"OBS","code":{"dataType":"CD","code":"Portavita604","codeSystem":"2.16.840.1.113883.2.4.3.31.2.1","codeSystemName":"Portavita","displayName":"instruction"},"effectiveTime":{"dataType":"IVL_TS","low":{"dataType":"IVXB_TS","inclusive":true,"value":"20120925230218"},"operator":"I"},"id":[{"dataType":"II","extension":"1681665389","root":"2.16.840.1.113883.2.4.3.31.3.1"}],"moodCode":"EVN","statusCode":{"dataType":"CS","code":"completed"},"value":[{"dataType":"CD","code":"373066001","codeSystem":"2.16.840.1.113883.6.96","codeSystemName":"SNOMED-CT","displayName":"yes"}]},"typeCode":"COMP"},{"_mif":"POCD_MT000040","_clonename":"Component4","_rimname":"ActRelationship","observation":{"_mif":"POCD_MT000040","_clonename":"Observation","_rimname":"Observation","classCode":"OBS","code":{"dataType":"CD","code":"Portavita605","codeSystem":"2.16.840.1.113883.2.4.3.31.2.1","codeSystemName":"Portavita","displayName":"education"},"effectiveTime":{"dataType":"IVL_TS","low":{"dataType":"IVXB_TS","inclusive":true,"value":"20120925230218"},"operator":"I"},"id":[{"dataType":"II","extension":"1681665386","root":"2.16.840.1.113883.2.4.3.31.3.1"}],"moodCode":"EVN","statusCode":{"dataType":"CS","code":"completed"},"value":[{"dataType":"CD","code":"373066001","codeSystem":"2.16.840.1.113883.6.96","codeSystemName":"SNOMED-CT","displayName":"yes"}]},"typeCode":"COMP"}],"effectiveTime":{"dataType":"IVL_TS","low":{"dataType":"IVXB_TS","inclusive":true,"value":"20120925230218"},"operator":"I"},"id":[{"dataType":"II","extension":"1681665390","root":"2.16.840.1.113883.2.4.3.31.3.1"}],"moodCode":"EVN","statusCode":{"dataType":"CS","code":"completed"}},"typeCode":"COMP"},{"_mif":"POCD_MT000040","_clonename":"Component4","_rimname":"ActRelationship","organizer":{"_mif":"POCD_MT000040","_clonename":"Organizer","_rimname":"Act","classCode":"BATTERY","code":{"dataType":"CD","code":"Portavita606","codeSystem":"2.16.840.1.113883.2.4.3.31.2.1","codeSystemName":"Portavita","displayName":"Improve mucustransport"},"component":[{"_mif":"POCD_MT000040","_clonename":"Component4","_rimname":"ActRelationship","observation":{"_mif":"POCD_MT000040","_clonename":"Observation","_rimname":"Observation","classCode":"OBS","code":{"dataType":"CD","code":"Portavita607","codeSystem":"2.16.840.1.113883.2.4.3.31.2.1","codeSystemName":"Portavita","displayName":"improve cough techniques"},"effectiveTime":{"dataType":"IVL_TS","low":{"dataType":"IVXB_TS","inclusive":true,"value":"20120925230218"},"operator":"I"},"id":[{"dataType":"II","extension":"1681665387","root":"2.16.840.1.113883.2.4.3.31.3.1"}],"moodCode":"EVN","statusCode":{"dataType":"CS","code":"completed"},"value":[{"dataType":"CD","code":"373066001","codeSystem":"2.16.840.1.113883.6.96","codeSystemName":"SNOMED-CT","displayName":"yes"}]},"typeCode":"COMP"},{"_mif":"POCD_MT000040","_clonename":"Component4","_rimname":"ActRelationship","observation":{"_mif":"POCD_MT000040","_clonename":"Observation","_rimname":"Observation","classCode":"OBS","code":{"dataType":"CD","code":"Portavita608","codeSystem":"2.16.840.1.113883.2.4.3.31.2.1","codeSystemName":"Portavita","displayName":"strengthen abdominal muscles"},"effectiveTime":{"dataType":"IVL_TS","low":{"dataType":"IVXB_TS","inclusive":true,"value":"20120925230218"},"operator":"I"},"id":[{"dataType":"II","extension":"1681665381","root":"2.16.840.1.113883.2.4.3.31.3.1"}],"moodCode":"EVN","statusCode":{"dataType":"CS","code":"completed"},"value":[{"dataType":"CD","code":"373066001","codeSystem":"2.16.840.1.113883.6.96","codeSystemName":"SNOMED-CT","displayName":"yes"}]},"typeCode":"COMP"}],"effectiveTime":{"dataType":"IVL_TS","low":{"dataType":"IVXB_TS","inclusive":true,"value":"20120925230218"},"operator":"I"},"id":[{"dataType":"II","extension":"1681665392","root":"2.16.840.1.113883.2.4.3.31.3.1"}],"moodCode":"EVN","statusCode":{"dataType":"CS","code":"completed"}},"typeCode":"COMP"},{"_mif":"POCD_MT000040","_clonename":"Component4","_rimname":"ActRelationship","organizer":{"_mif":"POCD_MT000040","_clonename":"Organizer","_rimname":"Act","classCode":"BATTERY","code":{"dataType":"CD","code":"Portavita610","codeSystem":"2.16.840.1.113883.2.4.3.31.2.1","codeSystemName":"Portavita","displayName":"Reduce dyspnea"},"component":[{"_mif":"POCD_MT000040","_clonename":"Component4","_rimname":"ActRelationship","observation":{"_mif":"POCD_MT000040","_clonename":"Observation","_rimname":"Observation","classCode":"OBS","code":{"dataType":"CD","code":"Portavita611","codeSystem":"2.16.840.1.113883.2.4.3.31.2.1","codeSystemName":"Portavita","displayName":"optimize diaphragmatic function"},"effectiveTime":{"dataType":"IVL_TS","low":{"dataType":"IVXB_TS","inclusive":true,"value":"20120925230218"},"operator":"I"},"id":[{"dataType":"II","extension":"1681665384","root":"2.16.840.1.113883.2.4.3.31.3.1"}],"moodCode":"EVN","statusCode":{"dataType":"CS","code":"completed"},"value":[{"dataType":"CD","code":"373066001","codeSystem":"2.16.840.1.113883.6.96","codeSystemName":"SNOMED-CT","displayName":"yes"}]},"typeCode":"COMP"},{"_mif":"POCD_MT000040","_clonename":"Component4","_rimname":"ActRelationship","observation":{"_mif":"POCD_MT000040","_clonename":"Observation","_rimname":"Observation","classCode":"OBS","code":{"dataType":"CD","code":"Portavita612","codeSystem":"2.16.840.1.113883.2.4.3.31.2.1","codeSystemName":"Portavita","displayName":"relaxation"},"effectiveTime":{"dataType":"IVL_TS","low":{"dataType":"IVXB_TS","inclusive":true,"value":"20120925230218"},"operator":"I"},"id":[{"dataType":"II","extension":"1681665380","root":"2.16.840.1.113883.2.4.3.31.3.1"}],"moodCode":"EVN","statusCode":{"dataType":"CS","code":"completed"},"value":[{"dataType":"CD","code":"373066001","codeSystem":"2.16.840.1.113883.6.96","codeSystemName":"SNOMED-CT","displayName":"yes"}]},"typeCode":"COMP"},{"_mif":"POCD_MT000040","_clonename":"Component4","_rimname":"ActRelationship","observation":{"_mif":"POCD_MT000040","_clonename":"Observation","_rimname":"Observation","classCode":"OBS","code":{"dataType":"CD","code":"Portavita613","codeSystem":"2.16.840.1.113883.2.4.3.31.2.1","codeSystemName":"Portavita","displayName":"breathing exercises"},"effectiveTime":{"dataType":"IVL_TS","low":{"dataType":"IVXB_TS","inclusive":true,"value":"20120925230218"},"operator":"I"},"id":[{"dataType":"II","extension":"1681665378","root":"2.16.840.1.113883.2.4.3.31.3.1"}],"moodCode":"EVN","statusCode":{"dataType":"CS","code":"completed"},"value":[{"dataType":"CD","code":"373066001","codeSystem":"2.16.840.1.113883.6.96","codeSystemName":"SNOMED-CT","displayName":"yes"}]},"typeCode":"COMP"},{"_mif":"POCD_MT000040","_clonename":"Component4","_rimname":"ActRelationship","observation":{"_mif":"POCD_MT000040","_clonename":"Observation","_rimname":"Observation","classCode":"OBS","code":{"dataType":"CD","code":"Portavita614","codeSystem":"2.16.840.1.113883.2.4.3.31.2.1","codeSystemName":"Portavita","displayName":"improve strength breathing muscles"},"effectiveTime":{"dataType":"IVL_TS","low":{"dataType":"IVXB_TS","inclusive":true,"value":"20120925230218"},"operator":"I"},"id":[{"dataType":"II","extension":"1681665385","root":"2.16.840.1.113883.2.4.3.31.3.1"}],"moodCode":"EVN","statusCode":{"dataType":"CS","code":"completed"},"value":[{"dataType":"CD","code":"373066001","codeSystem":"2.16.840.1.113883.6.96","codeSystemName":"SNOMED-CT","displayName":"yes"}]},"typeCode":"COMP"},{"_mif":"POCD_MT000040","_clonename":"Component4","_rimname":"ActRelationship","observation":{"_mif":"POCD_MT000040","_clonename":"Observation","_rimname":"Observation","classCode":"OBS","code":{"dataType":"CD","code":"Portavita615","codeSystem":"2.16.840.1.113883.2.4.3.31.2.1","codeSystemName":"Portavita","displayName":"improve ergonomics"},"effectiveTime":{"dataType":"IVL_TS","low":{"dataType":"IVXB_TS","inclusive":true,"value":"20120925230218"},"operator":"I"},"id":[{"dataType":"II","extension":"1681665379","root":"2.16.840.1.113883.2.4.3.31.3.1"}],"moodCode":"EVN","statusCode":{"dataType":"CS","code":"completed"},"value":[{"dataType":"CD","code":"373066001","codeSystem":"2.16.840.1.113883.6.96","codeSystemName":"SNOMED-CT","displayName":"yes"}]},"typeCode":"COMP"}],"effectiveTime":{"dataType":"IVL_TS","low":{"dataType":"IVXB_TS","inclusive":true,"value":"20120925230218"},"operator":"I"},"id":[{"dataType":"II","extension":"1681665393","root":"2.16.840.1.113883.2.4.3.31.3.1"}],"moodCode":"EVN","statusCode":{"dataType":"CS","code":"completed"}},"typeCode":"COMP"},{"_mif":"POCD_MT000040","_clonename":"Component4","_rimname":"ActRelationship","organizer":{"_mif":"POCD_MT000040","_clonename":"Organizer","_rimname":"Act","classCode":"BATTERY","code":{"dataType":"CD","code":"Portavita617","codeSystem":"2.16.840.1.113883.2.4.3.31.2.1","codeSystemName":"Portavita","displayName":"Improve physical condition"},"component":[{"_mif":"POCD_MT000040","_clonename":"Component4","_rimname":"ActRelationship","observation":{"_mif":"POCD_MT000040","_clonename":"Observation","_rimname":"Observation","classCode":"OBS","code":{"dataType":"CD","code":"Portavita619","codeSystem":"2.16.840.1.113883.2.4.3.31.2.1","codeSystemName":"Portavita","displayName":"stamina"},"effectiveTime":{"dataType":"IVL_TS","low":{"dataType":"IVXB_TS","inclusive":true,"value":"20120925230218"},"operator":"I"},"id":[{"dataType":"II","extension":"1681665383","root":"2.16.840.1.113883.2.4.3.31.3.1"}],"moodCode":"EVN","statusCode":{"dataType":"CS","code":"completed"},"value":[{"dataType":"CD","code":"373066001","codeSystem":"2.16.840.1.113883.6.96","codeSystemName":"SNOMED-CT","displayName":"yes"}]},"typeCode":"COMP"},{"_mif":"POCD_MT000040","_clonename":"Component4","_rimname":"ActRelationship","observation":{"_mif":"POCD_MT000040","_clonename":"Observation","_rimname":"Observation","classCode":"OBS","code":{"dataType":"CD","code":"Portavita620","codeSystem":"2.16.840.1.113883.2.4.3.31.2.1","codeSystemName":"Portavita","displayName":"adl training"},"effectiveTime":{"dataType":"IVL_TS","low":{"dataType":"IVXB_TS","inclusive":true,"value":"20120925230218"},"operator":"I"},"id":[{"dataType":"II","extension":"1681665388","root":"2.16.840.1.113883.2.4.3.31.3.1"}],"moodCode":"EVN","statusCode":{"dataType":"CS","code":"completed"},"value":[{"dataType":"CD","code":"373066001","codeSystem":"2.16.840.1.113883.6.96","codeSystemName":"SNOMED-CT","displayName":"yes"}]},"typeCode":"COMP"}],"effectiveTime":{"dataType":"IVL_TS","low":{"dataType":"IVXB_TS","inclusive":true,"value":"20120925230218"},"operator":"I"},"id":[{"dataType":"II","extension":"1681665394","root":"2.16.840.1.113883.2.4.3.31.3.1"}],"moodCode":"EVN","statusCode":{"dataType":"CS","code":"completed"}},"typeCode":"COMP"}],"effectiveTime":{"dataType":"IVL_TS","low":{"dataType":"IVXB_TS","inclusive":true,"value":"20120925230218"},"operator":"I"},"id":[{"dataType":"II","extension":"1681665391","root":"2.16.840.1.113883.2.4.3.31.3.1"}],"moodCode":"EVN","statusCode":{"dataType":"CS","code":"completed"}},"typeCode":"COMP"}],"effectiveTime":{"dataType":"IVL_TS","low":{"dataType":"IVXB_TS","inclusive":true,"value":"20120925230218"},"operator":"I"},"id":[{"dataType":"II","extension":"1681665395","root":"2.16.840.1.113883.2.4.3.31.3.1"}],"moodCode":"EVN","performer":[{"_mif":"POCD_MT000040","_clonename":"Performer2","_rimname":"Participation","assignedEntity":{"_mif":"POCD_MT000040","_clonename":"AssignedEntity","_rimname":"Role","classCode":"ASSIGNED","id":[{"dataType":"II","extension":"-284483747","root":"2.16.840.1.113883.2.4.3.31.3.2"}],"representedOrganization":{"_mif":"POCD_MT000040","_clonename":"Organization","_rimname":"Organization","classCode":"ORG","determinerCode":"INSTANCE","id":[{"dataType":"II","extension":"347763604","root":"2.16.840.1.113883.2.4.3.31.3.2"}]}},"typeCode":"PRF"}],"statusCode":{"dataType":"CS","code":"completed"}},"typeCode":"COMP"}],"moodCode":"EVN"},"typeCode":"COMP"}],"moodCode":"EVN"},"typeCode":"COMP"},"confidentialityCode":{"dataType":"CE","code":"N","codeSystem":"2.16.840.1.113883.5.25","codeSystemName":"Confidentiality","displayName":"Normal"},"dataEnterer":{"_mif":"POCD_MT000040","_clonename":"DataEnterer","_rimname":"Participation","assignedEntity":{"_mif":"POCD_MT000040","_clonename":"AssignedEntity","_rimname":"Role","classCode":"ASSIGNED","id":[{"dataType":"II","extension":"-240083420","root":"2.16.840.1.113883.2.4.3.31.3.2"}],"representedOrganization":{"_mif":"POCD_MT000040","_clonename":"Organization","_rimname":"Organization","classCode":"ORG","determinerCode":"INSTANCE","id":[{"dataType":"II","extension":"347763604","root":"2.16.840.1.113883.2.4.3.31.3.2"}]}},"contextControlCode":"OP","time":{"dataType":"TS","value":"20141215062206"},"typeCode":"ENT"},"documentationOf":[{"_mif":"POCD_MT000040","_clonename":"DocumentationOf","_rimname":"ActRelationship","serviceEvent":{"_mif":"POCD_MT000040","_clonename":"ServiceEvent","_rimname":"Act","classCode":"ACT","code":{"dataType":"CE","code":"17074200","codeSystem":"2.16.840.1.113883.6.96","codeSystemName":"SNOMED-CT","displayName":"Diabetes treatment"},"id":[{"dataType":"II","extension":"-13661778","root":"2.16.840.1.113883.2.4.3.31.3.1"}],"moodCode":"EVN"},"typeCode":"DOC"}],"effectiveTime":{"dataType":"TS","value":"20141215062206"},"id":{"dataType":"II","extension":"90502a78-c491-4fc8-bacb-bfef79068a72","root":"2.16.840.1.113883.2.4.3.31.3.1"},"legalAuthenticator":{"_mif":"POCD_MT000040","_clonename":"LegalAuthenticator","_rimname":"Participation","assignedEntity":{"_mif":"POCD_MT000040","_clonename":"AssignedEntity","_rimname":"Role","classCode":"ASSIGNED","id":[{"dataType":"II","extension":"-1345161193","root":"2.16.840.1.113883.2.4.3.31.3.2"}],"representedOrganization":{"_mif":"POCD_MT000040","_clonename":"Organization","_rimname":"Organization","classCode":"ORG","determinerCode":"INSTANCE","id":[{"dataType":"II","extension":"347763604","root":"2.16.840.1.113883.2.4.3.31.3.2"}]}},"contextControlCode":"OP","signatureCode":{"dataType":"CS","code":"S"},"time":{"dataType":"TS","value":"20141215062206"},"typeCode":"LA"},"moodCode":"EVN","recordTarget":[{"_mif":"POCD_MT000040","_clonename":"RecordTarget","_rimname":"Participation","contextControlCode":"OP","patientRole":{"_mif":"POCD_MT000040","_clonename":"PatientRole","_rimname":"Patient","classCode":"PAT","id":[{"dataType":"II","nullFlavor":"UNK"}],"patient":{"_mif":"POCD_MT000040","_clonename":"Patient","_rimname":"Person","classCode":"PSN","determinerCode":"INSTANCE","id":{"dataType":"II","extension":"1198877214","root":"2.16.840.1.113883.2.4.3.31.3.2"}},"providerOrganization":{"_mif":"POCD_MT000040","_clonename":"Organization","_rimname":"Organization","classCode":"ORG","determinerCode":"INSTANCE","id":[{"dataType":"II","extension":"347763604","root":"2.16.840.1.113883.2.4.3.31.3.2"}]}},"typeCode":"RCT"}],"title":{"dataType":"ST","mediaType":"text/plain","representation":"TXT","content":"PHR Update"},"typeId":{"dataType":"II","extension":"POCD_HD000040","root":"2.16.840.1.113883.1.3"}}

The output is a single line of properly escaped JSON. Use PostgreSQL 9.5+ jsonb_pretty() or another tool to view a pretty printed json document. A part of a pretty printed document is shown below:

{
    "_mif": "POCD_MT000040",
    "_clonename": "ClinicalDocument",
    "_rimname": "Document",
    "author": [{
        "_mif": "POCD_MT000040",
        "_clonename": "Author",
        "_rimname": "Participation",
        "assignedAuthor": {
            "_mif": "POCD_MT000040",
            "_clonename": "AssignedAuthor",
            "_rimname": "Role",
            "classCode": "ASSIGNED",
            "id": [{
                "dataType": "II",
                "extension": "-955258132",
                "root": "2.16.840.1.113883.2.4.3.31.3.2"
            }],
            "representedOrganization": {
                "_mif": "POCD_MT000040",
                "_clonename": "Organization",
                "_rimname": "Organization",
                "classCode": "ORG",
                "determinerCode": "INSTANCE",
                "id": [{
                    "dataType": "II",
                    "extension": "347763604",
                    "root": "2.16.840.1.113883.2.4.3.31.3.2"
                }]
            }
        },
        "contextControlCode": "OP",
        "time": {
            "dataType": "TS",
            "value": "20141215062206"
        },
        "typeCode": "AUT"
    }],
    "classCode": "DOCCLIN",
    "code": {
        "dataType": "CE",
        "code": "68608-9",
        "codeSystem": "2.16.840.1.113883.6.1",
        "codeSystemName": "LOINC",
        "displayName": "Summarization note"
    },
    "component": {
        "_mif": "POCD_MT000040",
        "_clonename": "Component2",
        "_rimname": "ActRelationship",
        "structuredBody": {
            "_mif": "POCD_MT000040",
            "_clonename": "StructuredBody",
            "_rimname": "Act",
            "classCode": "DOCBODY",
            "component": [{
                "_mif": "POCD_MT000040",
                "_clonename": "Component3",
                "_rimname": "ActRelationship",
                "section": {
                    "_mif": "POCD_MT000040",
                    "_clonename": "Section",
                    "_rimname": "Act",
                    "classCode": "DOCSECT",
                    "entry": [{
                        "_mif": "POCD_MT000040",
                        "_clonename": "Entry",
                        "_rimname": "ActRelationship",
                        "organizer": {
                            "_mif": "POCD_MT000040",
                            "_clonename": "Organizer",
                            "_rimname": "Act",
                            "classCode": "BATTERY",
                            "code": {
                                "dataType": "CD",
                                "code": "Portavita600",
                                "codeSystem": "2.16.840.1.113883.2.4.3.31.2.1",
                                "codeSystemName": "Portavita",
                                "displayName": "Physiotherapy"
                            },
                            "component": [{
                                "_mif": "POCD_MT000040",
                                "_clonename": "Component4",
                                "_rimname": "ActRelationship",
                                "organizer": {
                                    "_mif": "POCD_MT000040",
                                    "_clonename": "Organizer",
                                    "_rimname": "Act",
                                    "classCode": "BATTERY",
                                    "code": {
                                        "dataType": "CD",
                                        "code": "Portavita601",
                                        "codeSystem": "2.16.840.1.113883.2.4.3.31.2.1",
                                        "codeSystemName": "Portavita",
                                        "displayName": "Treatment goals"
                                    },
                                    "component": [{
                                        "_mif": "POCD_MT000040",
                                        "_clonename": "Component4",
                                        "_rimname": "ActRelationship",
                                        "organizer": {
                                            "_mif": "POCD_MT000040",
                                            "_clonename": "Organizer",
                                            "_rimname": "Act",
                                            "classCode": "BATTERY",
                                            "code": {
                                                "dataType": "CD",
                                                "code": "Portavita602",
                                                "codeSystem": "2.16.840.1.113883.2.4.3.31.2.1",
                                                "codeSystemName": "Portavita",
                                                "displayName": "Informing patient"
                                            },
                                            "component": [{
                                                "_mif": "POCD_MT000040",
                                                "_clonename": "Component4",
                                                "_rimname": "ActRelationship",
                                                "observation": {
                                                    "_mif": "POCD_MT000040",
                                                    "_clonename": "Observation",
                                                    "_rimname": "Observation",
                                                    "classCode": "OBS",
                                                    "code": {
                                                        "dataType": "CD",
                                                        "code": "Portavita603",
                                                        "codeSystem": "2.16.840.1.113883.2.4.3.31.2.1",
                                                        "codeSystemName": "Portavita",
                                                        "displayName": "information"
                                                    },
                                                    "effectiveTime": {
                                                        "dataType": "IVL_TS",
                                                        "low": {
                                                            "dataType": "IVXB_TS",
                                                            "inclusive": true,
                                                            "value": "20120925230218"
                                                        },
                                                        "operator": "I"
                                                    },
                                                    "id": [{
                                                        "dataType": "II",
                                                        "extension": "1681665382",
                                                        "root": "2.16.840.1.113883.2.4.3.31.3.1"
                                                    }],
                                                    "moodCode": "EVN",
                                                    "statusCode": {
                                                        "dataType": "CS",
                                                        "code": "completed"
                                                    },
                                                    "value": [{
                                                        "dataType": "CD",
                                                        "code": "373066001",
                                                        "codeSystem": "2.16.840.1.113883.6.96",
                                                        "codeSystemName": "SNOMED-CT",
                                                        "displayName": "yes"
                                                    }]
                                                },
                                                "typeCode": "COMP"
                                            }

Note

Use PostgreSQL COPY for bulk load of JSON documents

The PostgreSQL COPY is the command to use for bulk loading of database tables. Refer to the program integration/rabbitmq/transformer_json_copy.py for an example of how to collect documents transformed to JSON from a RabbitMQ message broker, and how to stream these to a PostgreSQL server with a document table.

4.3. RIM map

The RIM map contains a class/attribute map with attribute datatypes. Typically the code datatypes from the RIM classes will also contain the valueset and whether the instances can be “coded with exceptions” or not.

The RIM map is generated once for each required normative edition using rimdb2py.sql. This function detects and exports the structure of the current hl7 schema.

4.4. RIM datatype list

The RIM datatype list contains all datatypes that were defined as xs:complexType or xs:simpleType in the RIM infrastructure xsd. This list is used during message conversion to determine if a particular xml element is a datatype instead of a table. A table would require an row insertion statement, where a xml element expressed as a datatype should be translated to a column statement.

The RIM datatype list is generated once for each required normative edition using xml2dt.xsl.

4.5. MIF maps

HL7v3 Model Interchange Format files model HL7v3 messages using renamed copies of RIM classes (so called cloned classes). The MIF files further define relations between the clones, default and fixed attributes for cloned classes.

MIF maps contain:

  • Rimnames; for each cloned class its corresponding name in the RIM.

    E.g. POCD_HD000040.Birthplace is a Role.

  • Parent; documents the direct (clone) parent of clone classes using inheritance.

    E.g. POCD_HD000040.Encounter -> POCD_HD000040.ClinicalStatement.

  • Target connections; lists rim attribute names with their corresponding cloned class attribute names.

    E.g. POCD_HD000040.RecordTarget.patientRole is a Role.

  • Source connections; lists associations from one cloned class to another, detailing the local and remote attributes that should contain the relational pointers.

    E.g. POCD_HD000040.Patient.guardian <-> POCD_HD000040.Guardian.scoper.

  • Clone class attributes; all known attributes for the cloned classes in the MIF.

  • Fixed class attributes; all known attributes with a fixed value.

    E.g. POCD_HD000040.Patient.classcode = 'PSN'.

  • Default class attributes:

    E.g. POCD_HD000040.ClinicalDocument.classCode = 'DOCCLIN'.

A mif map is generated using the mif12py.xsl or mif22py.xsl transform.

4.6. MIF instance parser

The xsd2parser.py parser generator is used to create a DOM parser from a HL7v3 message XML schema definition.