13. Release History¶
13.1. Release 3.5.0 (2019-06-20)¶
Switched from Python 2 to 3
Run programs within container as non-root user. Note that this requires setting the user in stack.yml: ‘user: mgrid’. Also note that the port number used inside the container has changed from 443 to 8443.
Update to latest permissions API of the Dataset Authorisation Service
13.2. Release 3.4.6 (2019-01-21)¶
Improvements
Prevent user from dropping all columns in dataset definition xml
Provide better feedback when dataset definition cannot be parsed as xml
13.3. Release 3.4.5 (2018-12-07)¶
Improvements
Improve datatype detection on empty bv
Order the project list on name
Improve deid line error reporting
Increased default nginx send and receive timeouts to 8 hours. Added documentation about overriding nginx configuration files.
Bug fixes
Be able to use DUO in combination with local authentication
Be able to login with DUO after logging out
13.4. Release 3.4.4 (2018-09-25)¶
Improvements
Fix unresponsiveness of large data uploads
Make folderwatcher have multiple workers
Add Content Security Policy (CSP) headers to proxy
Smaller size of Docker image (avoid chown)
Always pull latest version of base Docker image
Bug fixes
The JWT algorithm is now enforced, whereas previously other algorithms were also allowed. The default algorithm is HS512 if no value for jwt.algorithm is given in the configuration file.
Fix for feed.py should not update units when cache is full
Prevent logging of keys and passwords in provisioning call
13.5. Release 3.4.3 (2018-03-26)¶
Improvements
Split DSB documentation into admin guide and user guide
Add table sample in filter screen to prevent users from accidently filtering too much and ending up with an empty file
Bug fixes
Add quoting to metrics data to ensure that commas and spaces are handled correctly
Known vulnerability in PyCrypto library; replaced with cryptography library
Check setting duo_enabled as boolean instead of string to support both “true” and “True”
13.6. Release 3.4.2 (2018-02-27)¶
Improvements
Added a user guide
Customers use projections in the data definition screen instead of the feature creation and code conversion screens. The functionality is similar, and the projections are easier to use. We have simplified the user interface by removing the feature creation and code conversion screens.
Bug fixes
Ensure that TDF / Format / Header field is set to False when no value is supplied
13.7. Release 3.4.1 (2018-02-15)¶
Improvements
Improved UI warning messages while interacting with DAS
Set fullname value in UI each time an LDAP login occurs
Restrict SQL functions available for projections
Bug fixes
Remove auto select for first dataset in a list
Check if directory is writeable before transferring to workspace
Missing qualification on highvalue func
Drop view on FDW export
Switch from Alpine to Ubuntu container to eliminate musl-libc. Musl-libc has been observed to lose directory entries when accessing network filesystems while in docker.
13.8. Release 3.4.0 (2018-02-01)¶
Improvements
folderwatcher (new name for xaplistener) now supports both structured and unstructured files
Increase client_max_body_size to 1GB, increase timeouts
At LDAP login retrieve displayname as principal.fullname
Emit errors when workspace could not be provisioned
Bug fixes
Even project users must have das authorisation for dataset reuse
Ensure created files are readable by setting umask
Context parameter with value of None breaks permission request
Uppercase column name in pivot causes SQL errors
Ensure deid error messages are strings before masking
Admins must also request permission for catalog transfers
Link from approver mail does not work
Make new uuids for all new datasets
Fix caching of AJAX content in IE
Fix IE11 form submit on workspace and datasource screens
13.9. Release 3.3.3 (2018-01-15)¶
Improvements
More granular export method setup
New export method that requires dataset authorisation by owner
Allow systemwide CA certificates
Allow different styles of configurations in entrypoint
Make default dataset aggregation time ‘all-data’
Foreign data export creates own remote views
Bug fixes
Fix post dataset yaml definition
More verbose logging for xaplistener
More verbose logging for deidentification
Remove 128k upload restriction for files
Disable project creation screens for normal users
13.10. Release 3.3.2 (2017-11-13)¶
Add Single Sign On button to login page if login with OAuth2 has been configured.
Add /info endpoint
Improvements
If necessary, provision user at DAS permission request
Login to DUO using only usernames; use email addresses in DAS operations
After transfer, point user in direction of XAP
Add catalog UUID as part of the project title
Bug fixes
Disable request button after catalog permission request
Fix for faulty redirects after retry of login
Filter: also show operators without descriptions
Fix for deeplinking to a non-existing dataset
13.11. Release 3.3.1 (2017-07-28)¶
Added row filtering definitions to YAML definition import and export
First and last value window operators now order on time_lowvalue and time_availability instead of only time_lowvalue
Security improvements
Report incorrect de-identify action in dataset definition instead of silently ignoring the action
Show configuration information on the REST interface only if the user has admin privileges
Bug fixes
Fix for failing LDAP AD authentication
Fix expression check for the corner case that there is no data
13.12. Release 3.3.0 (2017-06-30)¶
Added OAuth2 login and provisioning using Gitlab as provider
Added row filtering and column selection
Improvements
timeout authentication cookies and gentle return to login screen
Bug fixes
FDW export to drop pre-existing foreign table
Deprecated
defining cohorts
13.13. Release 3.2.9 (2017-05-05)¶
Addition of FHIR Dosing List and Proposal Status
Add SQL/MED FDW (Foreign Data Wrapper) table export
Improvements
return nicer error messages when DDF file contains unknown datatype
base external dataset name on user definition
Bug fixes
fix for external upload to unowned workspace possible
fix for no feedback is presented when logging in with a wrong username/password combination
always return json, even on error
make sensitive ui items only available to admins
13.14. Release 3.2.8 (2017-04-20)¶
Performance improvements
buffer INSERT ON CONFLICT UPDATE up to 100 tuples at once
Bug fixes
fix for upload of multiple datasets
warn user when using a non-text datatype for pseudonym and pseudonymize
13.15. Release 3.2.7 (2017-04-07)¶
Support for catalogue flow
Support for non-structured data
Add filter options to define dataset feature column list
Bug fixes
fix null measurements
fix value expression errors for pivots with unit-event grouping
updated deidentification frontend to be in line with backend
fix for creation of deid project when updating existing project
fix for incorrect application header sent to deidentification server
Add aptcli
13.16. Release 3.2.6 (2017-03-17)¶
Performance improvements
define screen: only query expression datatype when the expression has changed, instead of at every Save
define screen: ensure that the table sample size calculation is based on recent table statistics so it is in the order of ~10 units
define screen: do not get the full data rowcount for the audit log message after each definition update
define screen: at database table creation time, use datatypes that were calculated at Save time, rather than recalculating the datatypes by running the expressions again
check unit expression: improve performance by scanning only the data that is appropriate for the given unit
Bug fixes
fix bug that would cause duplication of column definition data. While this did not lead to data errors, the duplicated data size would grow at each save, which caused a progressing performance degradation for dataset definition retrieval and updates
Improvements in handling dataset imports:
datatype handling
different character sets (encoding)
datestyle handling
handling of null and empty pseudonymize/pseudonym attributes
Resolving of deeplinks from catalogue entries to datasets in the catalogue store
13.17. Release 3.2.5 (2017-02-09)¶
UI Improvements on Define screen
add buttons to duplicate unit and feature column definitions
filter already used source features from ‘Add Feature’ window
table sample scroll to the last row
improvement of Revert/Save buttons on slow networks
Add usage of bulk deidentification of the deidentification server
Add Import YAML Definition to Dataset screen
Add support for two-factor authentication
Bug fixes
Preserve column ordering for CSV export
Fix dimension table specified more than once that occurred for certain corner cases
Check unit expression to validate an expression on a unit that corresponds to the role (e.g. RegistrationProcess) instead of only the class (such as Act)
Fixes in automatic column naming: better match of original name and usage of maximum allowable length of 63 characters
Fix object_value_from..() workspace functions to copy with input cases that do not conform to expected input: instead of error now return NULL. This is the desired operating mode when not all input values have a non empty object list.
Fix unit expressions with more than one reference to the value.
13.18. Release 3.2.4 (2017-01-23)¶
Improvements for long names
Column names < 64 characters
Wider feature list screen
Improvements in dataset handling
Carry across dataset column description in TDF
Only use CSV filename and disregard XML filename
Fix for NULL handling in deidentification
Easier migration to older and newer versions
Enable all datatypes supported by the database server, instead of only a fixed number of datatypes
13.19. Release 3.2.3 (2016-12-21)¶
Support for XAP web upload (token API)
Added Feature List screen
Provides insight into associations between features and units
Add YAML export and import of dataset definitions
Several small bugfixes
13.20. Release 3.2.2 (2016-12-07)¶
Security additions
Added measures against SQL injection
Policy drop for datasets
Added publication of metrics
Added pseudonymization functions
Several refactorings
13.21. Release 3.2.1 (2016-11-11)¶
Improved CSV reader
Updated measurement API
Added documentation for dataset import
Bugfixes for TDF and deidentification
13.22. Release 3.2.0 (2016-11-01)¶
Adapted to Aridhia de-identification API 1.8
Added HSTS and best practice security headers
Provide secure storage for sensitive credentials in internal database
Exporting Table Definition File together with CSV file
Moved to single datasource format
Removed restriction of fixed unit of observation
Moved FHIR support to XFM
13.23. Release 3.1.0 (2016-10-08)¶
Refactored Provisioning API v2
Allows idempotent put
JWT authentication
Added DELETE method
Added Deidentification tab for datasets in UI
Added per project LDAP group authorisation
Added Aridhia XAP SFTP Export
Added daemon to look for new datasets on an ingress SFTP server
new datasets will be added to pre-provisioned projects
13.24. Release 3.0.2 (2016-08-05)¶
Added CSV dataset import support
Added Aridhia Table Definition Format ingress support
Added Aridhia XAP Deidentification Server Support v1.7
Added Provisioning API v1
13.25. Release 3.0.1 (2016-07-01)¶
Improved packaging
support for CentOS 6
Improved feature overview graphs
auto binning
show more feature types
show more values
consistent ordering on most frequent value
Convert FHIR ingest to basevalues
Added Kafka client
Added support for nested properties of dimensions
Added front-end job monitoring
Added LDAP authentication
13.26. Release 3.0.0 (2016-05-03)¶
Initial public release