13. Release History

13.1. Development

  • Switched from Python 2 to 3

13.2. Release 3.4.6 (2019-01-21)

  • Improvements
    • Prevent user from dropping all columns in dataset definition xml
    • Provide better feedback when dataset definition cannot be parsed as xml

13.3. Release 3.4.5 (2018-12-07)

  • Improvements
    • Improve datatype detection on empty bv
    • Order the project list on name
    • Improve deid line error reporting
    • Increased default nginx send and receive timeouts to 8 hours. Added documentation about overriding nginx configuration files.
  • Bug fixes
    • Be able to use DUO in combination with local authentication
    • Be able to login with DUO after logging out

13.4. Release 3.4.4 (2018-09-25)

  • Improvements
    • Fix unresponsiveness of large data uploads
    • Make folderwatcher have multiple workers
    • Add Content Security Policy (CSP) headers to proxy
    • Smaller size of Docker image (avoid chown)
    • Always pull latest version of base Docker image
  • Bug fixes
    • The JWT algorithm is now enforced, whereas previously other algorithms were also allowed. The default algorithm is HS512 if no value for jwt.algorithm is given in the configuration file.
    • Fix for feed.py should not update units when cache is full
    • Prevent logging of keys and passwords in provisioning call

13.5. Release 3.4.3 (2018-03-26)

  • Improvements
    • Split DSB documentation into admin guide and user guide
    • Add table sample in filter screen to prevent users from accidently filtering too much and ending up with an empty file
  • Bug fixes
    • Add quoting to metrics data to ensure that commas and spaces are handled correctly
    • Known vulnerability in PyCrypto library; replaced with cryptography library
    • Check setting duo_enabled as boolean instead of string to support both “true” and “True”

13.6. Release 3.4.2 (2018-02-27)

  • Improvements
    • Added a user guide
    • Customers use projections in the data definition screen instead of the feature creation and code conversion screens. The functionality is similar, and the projections are easier to use. We have simplified the user interface by removing the feature creation and code conversion screens.
  • Bug fixes
    • Ensure that TDF / Format / Header field is set to False when no value is supplied

13.7. Release 3.4.1 (2018-02-15)

  • Improvements
    • Improved UI warning messages while interacting with DAS
    • Set fullname value in UI each time an LDAP login occurs
    • Restrict SQL functions available for projections
  • Bug fixes
    • Remove auto select for first dataset in a list
    • Check if directory is writeable before transferring to workspace
    • Missing qualification on highvalue func
    • Drop view on FDW export
    • Switch from Alpine to Ubuntu container to eliminate musl-libc. Musl-libc has been observed to lose directory entries when accessing network filesystems while in docker.

13.8. Release 3.4.0 (2018-02-01)

  • Improvements
    • folderwatcher (new name for xaplistener) now supports both structured and unstructured files
    • Increase client_max_body_size to 1GB, increase timeouts
    • At LDAP login retrieve displayname as principal.fullname
    • Emit errors when workspace could not be provisioned
  • Bug fixes
    • Even project users must have das authorisation for dataset reuse
    • Ensure created files are readable by setting umask
    • Context parameter with value of None breaks permission request
    • Uppercase column name in pivot causes SQL errors
    • Ensure deid error messages are strings before masking
    • Admins must also request permission for catalog transfers
    • Link from approver mail does not work
    • Make new uuids for all new datasets
    • Fix caching of AJAX content in IE
    • Fix IE11 form submit on workspace and datasource screens

13.9. Release 3.3.3 (2018-01-15)

  • Improvements
    • More granular export method setup
    • New export method that requires dataset authorisation by owner
    • Allow systemwide CA certificates
    • Allow different styles of configurations in entrypoint
    • Make default dataset aggregation time ‘all-data’
    • Foreign data export creates own remote views
  • Bug fixes
    • Fix post dataset yaml definition
    • More verbose logging for xaplistener
    • More verbose logging for deidentification
    • Remove 128k upload restriction for files
    • Disable project creation screens for normal users

13.10. Release 3.3.2 (2017-11-13)

  • Add Single Sign On button to login page if login with OAuth2 has been configured.
  • Add /info endpoint
  • Improvements
    • If necessary, provision user at DAS permission request
    • Login to DUO using only usernames; use email addresses in DAS operations
    • After transfer, point user in direction of XAP
    • Add catalog UUID as part of the project title
  • Bug fixes
    • Disable request button after catalog permission request
    • Fix for faulty redirects after retry of login
    • Filter: also show operators without descriptions
    • Fix for deeplinking to a non-existing dataset

13.11. Release 3.3.1 (2017-07-28)

  • Added row filtering definitions to YAML definition import and export
  • First and last value window operators now order on time_lowvalue and time_availability instead of only time_lowvalue
  • Security improvements
    • Report incorrect de-identify action in dataset definition instead of silently ignoring the action
    • Show configuration information on the REST interface only if the user has admin privileges
  • Bug fixes
    • Fix for failing LDAP AD authentication
    • Fix expression check for the corner case that there is no data

13.12. Release 3.3.0 (2017-06-30)

  • Added OAuth2 login and provisioning using Gitlab as provider
  • Added row filtering and column selection
  • Improvements
    • timeout authentication cookies and gentle return to login screen
  • Bug fixes
    • FDW export to drop pre-existing foreign table
  • Deprecated
    • defining cohorts

13.13. Release 3.2.9 (2017-05-05)

  • Addition of FHIR Dosing List and Proposal Status
  • Add SQL/MED FDW (Foreign Data Wrapper) table export
  • Improvements
    • return nicer error messages when DDF file contains unknown datatype
    • base external dataset name on user definition
  • Bug fixes
    • fix for external upload to unowned workspace possible
    • fix for no feedback is presented when logging in with a wrong username/password combination
    • always return json, even on error
    • make sensitive ui items only available to admins

13.14. Release 3.2.8 (2017-04-20)

  • Performance improvements
    • buffer INSERT ON CONFLICT UPDATE up to 100 tuples at once
  • Bug fixes
    • fix for upload of multiple datasets
    • warn user when using a non-text datatype for pseudonym and pseudonymize

13.15. Release 3.2.7 (2017-04-07)

  • Support for catalogue flow
  • Support for non-structured data
  • Add filter options to define dataset feature column list
  • Bug fixes
    • fix null measurements
    • fix value expression errors for pivots with unit-event grouping
    • updated deidentification frontend to be in line with backend
    • fix for creation of deid project when updating existing project
    • fix for incorrect application header sent to deidentification server
  • Add aptcli

13.16. Release 3.2.6 (2017-03-17)

  • Performance improvements
    • define screen: only query expression datatype when the expression has changed, instead of at every Save
    • define screen: ensure that the table sample size calculation is based on recent table statistics so it is in the order of ~10 units
    • define screen: do not get the full data rowcount for the audit log message after each definition update
    • define screen: at database table creation time, use datatypes that were calculated at Save time, rather than recalculating the datatypes by running the expressions again
    • check unit expression: improve performance by scanning only the data that is appropriate for the given unit
  • Bug fixes
    • fix bug that would cause duplication of column definition data. While this did not lead to data errors, the duplicated data size would grow at each save, which caused a progressing performance degradation for dataset definition retrieval and updates
  • Improvements in handling dataset imports:
    • datatype handling
    • different character sets (encoding)
    • datestyle handling
    • handling of null and empty pseudonymize/pseudonym attributes
  • Resolving of deeplinks from catalogue entries to datasets in the catalogue store

13.17. Release 3.2.5 (2017-02-09)

  • UI Improvements on Define screen
    • add buttons to duplicate unit and feature column definitions
    • filter already used source features from ‘Add Feature’ window
    • table sample scroll to the last row
    • improvement of Revert/Save buttons on slow networks
  • Add usage of bulk deidentification of the deidentification server
  • Add Import YAML Definition to Dataset screen
  • Add support for two-factor authentication
  • Bug fixes
    • Preserve column ordering for CSV export
    • Fix dimension table specified more than once that occurred for certain corner cases
    • Check unit expression to validate an expression on a unit that corresponds to the role (e.g. RegistrationProcess) instead of only the class (such as Act)
    • Fixes in automatic column naming: better match of original name and usage of maximum allowable length of 63 characters
    • Fix object_value_from..() workspace functions to copy with input cases that do not conform to expected input: instead of error now return NULL. This is the desired operating mode when not all input values have a non empty object list.
    • Fix unit expressions with more than one reference to the value.

13.18. Release 3.2.4 (2017-01-23)

  • Improvements for long names
    • Column names < 64 characters
    • Wider feature list screen
  • Improvements in dataset handling
    • Carry across dataset column description in TDF
    • Only use CSV filename and disregard XML filename
    • Fix for NULL handling in deidentification
  • Easier migration to older and newer versions
  • Enable all datatypes supported by the database server, instead of only a fixed number of datatypes

13.19. Release 3.2.3 (2016-12-21)

  • Support for XAP web upload (token API)
  • Added Feature List screen
    • Provides insight into associations between features and units
  • Add YAML export and import of dataset definitions
  • Several small bugfixes

13.20. Release 3.2.2 (2016-12-07)

  • Security additions
    • Added measures against SQL injection
    • Policy drop for datasets
  • Added publication of metrics
  • Added pseudonymization functions
  • Several refactorings

13.21. Release 3.2.1 (2016-11-11)

  • Improved CSV reader
  • Updated measurement API
  • Added documentation for dataset import
  • Bugfixes for TDF and deidentification

13.22. Release 3.2.0 (2016-11-01)

  • Adapted to Aridhia de-identification API 1.8
  • Added HSTS and best practice security headers
  • Provide secure storage for sensitive credentials in internal database
  • Exporting Table Definition File together with CSV file
  • Moved to single datasource format
    • Removed restriction of fixed unit of observation
    • Moved FHIR support to XFM

13.23. Release 3.1.0 (2016-10-08)

  • Refactored Provisioning API v2
    • Allows idempotent put
    • JWT authentication
    • Added DELETE method
  • Added Deidentification tab for datasets in UI
  • Added per project LDAP group authorisation
  • Added Aridhia XAP SFTP Export
  • Added daemon to look for new datasets on an ingress SFTP server
    • new datasets will be added to pre-provisioned projects

13.24. Release 3.0.2 (2016-08-05)

  • Added CSV dataset import support
  • Added Aridhia Table Definition Format ingress support
  • Added Aridhia XAP Deidentification Server Support v1.7
  • Added Provisioning API v1

13.25. Release 3.0.1 (2016-07-01)

  • Improved packaging
    • support for CentOS 6
  • Improved feature overview graphs
    • auto binning
    • show more feature types
    • show more values
    • consistent ordering on most frequent value
  • Convert FHIR ingest to basevalues
  • Added Kafka client
  • Added support for nested properties of dimensions
  • Added front-end job monitoring
  • Added LDAP authentication

13.26. Release 3.0.0 (2016-05-03)

  • Initial public release