Vonk FHIR Server in public Beta

Take a ride

Furore has built a new FHIR Server named Vonk. In keywords: enterprise grade, support available, SQL Server and MongoDB, FHIR Facade, cross-platform, Dockerized, .Net Core, STU3. As of now it is in public Beta. Try it at http://vonk.furore.com or download a trial at Simplifier.

Inspiration

FietsCK-01

This is my bike. After being a simple city bike for the first 25 years, I decided to keep the good parts and improve the rest. Now it suits all my many bike needs. It is clean. No gears, no cables, no chain cover, just the items that I really need. It is strong but not heavy. Steel frame, steel carrier, able to carry my wife on a romantic trip and it doesn’t break on the many potholes and loose tiles in the city. It is versatile. It can ride both dirt- and tarmac roads. Run lightly on my way to work, carry all the groceries for a week on Friday. It has the right size. Large enough for my 1.95m length, but it still easily fits in a bicycle rack. It never gets in my way in the shed. Maybe the best part is the carbon belt drive that I had mounted recently. Unbreakable and yet free of maintenance. And now that the bike is 27 years old and still my daily commuting bike I can faithfully say that it’s proven technology. And my inspiration for developing Vonk.

The old bike

Back in 2012 when FHIR had just started, Furore started implementing the server part of the specification in .Net, and we called it Spark. A reference implementation to test the spec and provide the world with a real FHIR server to develop and test against. It has gained a lot of attraction, well beyond these purposes. Many requests for e.g. production use, SQL Server, STU3 and company support reached us.

The new bike

So we decided to respond with a new incarnation of Spark that would be able to serve as a very versatile yet production ready FHIR server. And it deserved it’s own name: Vonk. Vonk packs together our extensive experience with FHIR, the HL7 FHIR .Net library and Spark in a fully new solution based on Microsoft .Net Core.

fietsck-03.jpg

New, robust technology

Several of you may have seen my talk on building a FHIR Server (> 19:30) at FHIR DevDays last November. By now we have realized many of the ideas that I presented in that talk. As of today we present the official Beta release of Vonk.

Ready to ride

My bike is always ready to ride. For a quick schoolrun or an extended birdwatching ride – it will just run. Vonk FHIR Server is the same. Install and run. It is a stand alone FHIR server with MongoDB storage. Or SQL Server and then truly support transactions. Or run a few quick tests on the in-memory store. The storage API that we created to accommodate these very different storage engines enables us to add other other back ends easily, either with or without support of Entity Framework.

Change the wheels

A few weeks ago my birdwatching buddy lured me into a dirt road. Dirt became mud and then deep mud. Even my bike has limits. It meant I had to continue walking. But with Vonk you just change the wheels! With Vonk FHIR Facade you can leverage the power of the storage API to plug it on top of your existing database. Thus Vonk acts as a FHIR REST facade on top of your (legacy) system. And although Vonk is enterprise grade, it is not heavy. So you can unleash your systems small and large all in the same way. All you need to do is create an implementation of the storage abstraction for your use case. We already did so ourselves: the Simplifier.net platform runs on Vonk. The same abstraction even allows you to address other (web) services for retrieving or storing the actual data, thereby using Vonk as a hub, possibly backed by other Vonk instances that are facades to your internal systems. When creating a facade you program the new wheels in .Net, with all the power that comes with that. On the roadmap is creating facades by configuration.

Assemble your own

No gears, no handbrakes, clean as can be. But a bit too clean if you need to climb a hill. In the Netherlands we don’t do hills, but Vonk can be used around the globe. Vonk FHIR Components gives you all the good parts of Vonk, for you to assemble your own perfect FHIR Server. Or integrate it into a larger web server. Or turn it into a hometrainer. Just add and leave out parts as you like. Vonk FHIR Components will become available as NuGet packages.

Run all the tasks you need

From groceries to commuting: even my bike is not as versatile as Vonk. In this beta release it will do STU3 and DSTU2, JSON and XML, and most of the regular FHIR interactions plus a few bonus ones:

  • capability statement (metadata)
  • create, read, update, delete
  • search on all specified parameters, also chained, choices, multiple resource types, include related resources
  • full resource version management, enabling history on resource, resource type and system level
  • process batches of interactions, in a transaction if you wish (requires transactional storage)
  • validate a resource against the spec or a custom profile
  • extensible to custom operation (‘$your-operation’)

The road ahead

More features are on their way. Priority is volatile and can be influenced by customer requests.

  • support for several more options, e.g. _summary, _elements and conditional operations
  • subscriptions
  • documents ($document)
  • proper support for ‘Binary’ resources

Lock it

fietsck-06.jpg

Security in Amsterdam is not about riding with a helmet, but putting at least two solid locks on your bike. We know a thing or two about not letting your bike get stolen. Or your data. So Vonk runs on https, can be fronted by a reverse proxy (e.g. Apache or NGINX) and exposes just the FHIR RESTful API endpoints. But a fully locked bike brings you nowhere. You need access, but limited access if needed. Therefore Vonk can participate in an OAuth2 single sign-on environment. Access to resources can be based on the claims in the OAuth2 access token. You can use the preconfigured scopes and launch contexts as defined by SMART-on-FHIR, or configure your own.

Ride anywhere

Although my bike won’t ride on water (in Amsterdam we have canalbikes for that) or deep mud, just about any other surface will do. And Vonk is no different: it runs on Windows, Linux, OSX, as an executable or in a Docker container, on premise or in the cloud, as an Azure App Service – you choose.

Ride fast

When my colleague Marco showed me Vonk running on Docker I was so thrilled I even overtook e-bikes at 30 km/h on my way home. With Vonk we also made performance a number one priority. Using the newest Microsoft ASP.Net technology and proven storage solutions we managed to implement a very performant FHIR Server. Being lightweight and fully asynchronous you can scale out or up as you see fit.

Be in control

In the Netherlands there is wind. Usually in your face. You need to lie low. So my first upgrade was the handlebar that allows me to lie comfortably on it and still keep control over the bike. Extensive semantic logging to just about any target and a connection to Microsoft Application Insights allows you to control Vonk just as well. Integrated validation can be used as a gatekeeper to accept only compliant resources. On the roadmap is also an extensible management API to:

FietsCK-05

  • add profiles, or restrict to specific profiles
  • add or restrict the search parameters
  • enable or disable operations

Where is the blueprint?

I’d be happy to tell you how I built this bike. But Vonk is not open source. Vonk is a commercial product based on license fees and paid support. Nevertheless we want to continue to support the FHIR community as we already do with the FHIR .Net API and Forge. In short it is possible to use Vonk for free if you support the FHIR community in any way with that usage. Just contact us to see if you’re proposed usage fits this goal.

What happens to the old parts?

I kept them in my shed as spares. As for Spark: This is up to the community. Furore has put a lot of effort in Spark and provides all of this effort to the community. It’s open source, available on GitHub. But Furore will neither port Spark to STU3 nor provide support for DSTU2.

Name

I never gave my bike a name. But ‘Vonk’ means two things. First, it is the Dutch word for Spark. After struggling with our company name Furore we thought the FHIR community was up for the next Dutch challenge. Second it reminds us of our dear collegue Hans Vonkeman who has passed away far too young and thus it is a kind of a tribute.

And now I’ve been at the keyboard long enough. Off for a ride. Do you ride along? Visit the getting started to see how you can get a trial Vonk FHIR Server as binaries or in a Docker container.

Prefer to ride along face to face? I’ll be at the HL7 WGM in Madrid next week, albeit not by bike. Or visit the FHIR DevDays here in Amsterdam in November. There I will be by bike.

Cheers, Christiaan
FietsCK-07

Advertisements

Simplifier and Forge Migration to STU3

by Martijn Harthoorn

TL;DR
Simplifier, Forge, and Vonk (the successor of Spark) will be migrated to STU3 before the Madrid WGM in May.

The week after March 20th, the next version of the FHIR specification STU3 will go live. This will be the first STU. Notice the D is gone? You can find out why here.

As the tools that we provide to the FHIR community are quite popular and heavily used, we need a very good plan to migrate to STU3. Of course we want to bring you STU3 as soon as possible. But we are not going to drop DSTU2 support just like that. Looking at several of our customers, we expect some companies will remain dependent on DSTU2 for a while. Here’s what we’ll do with our tooling.

Forge

Forge has gathered a significant user base. And since Forge is distributed via a ClickOnce installer, we will create a separate side-by-side release for STU3. This means that you will be able to install and run both the current DSTU2 and STU3 Forge releases on the same machine. We will not distribute the STU3 release as an auto-update of the current DSTU2 release; otherwise existing Forge users would no longer be able to manage any DSTU2 profiles. We will announce the STU3 release via the Furore news feed on Simplifier and also on the FHIR chat groups on Zulip.

Forge depends heavily on the FHIR .NET API, which we are currently upgrading to STU3. STU3 brings changes in some of the core resources and datatypes. STU3 also introduces some important changes in the meta data. Therefore DSTU2 and STU3 profiles are not directly interchangeable.

Simplifier

The migration of Simplifier is more complex. In addition to introducing support for STU3, we are also migrating Simplifier to a newer technology stack (ASP.NET core for anyone who’s interested). An additional complication is that Simplifier needs to be able to host both DSTU2 profiles and STU3 profiles at the same time. We will make a strong push towards STU3 after march 20th, but that does not imply that we will abandon DSTU2.

We will make projects on Simplifier FHIR version specific to help you, and our tooling,  figure out how and what we should render. Of course, it makes no sense to try and render a DSTU2 profile with a STU3 render engine. It would just fail.

The strategy that we will follow is what we call a graceful fallback strategy. This means that we will move all our internal Simplifier tooling to STU3. And we will move all the rendering logic to STU3. But we will make it possible to still host and even upload DSTU2 profiles. This is not as simple as it may seem. After careful consideration, we decided to implement FHIR validation as a separate step, instead of being tightly integrated as a mandatory step to the import. Currently you are not able to upload DSTU1 or STU3 resources, because the FHIR engine in Simplifier  considers it to be invalid input (does not conform to FHIR DSTU2). We are going to change the upload procedure, so that the server will accept invalid FHIR input. And instead of rejecting invalid input, Simplifier will inform you that it is not valid STU3 FHIR.

But this is where our graceful fallback strategy comes into play: we can’t keep everything live for all versions of FHIR. We think it’s not beneficial for the standard either. Whenever you can migrate to STU3, you should. But we will make an effort to do what we can to keep DSTU2 working for you. We keep the existing tool set for DSTU2 alive. And the more we manage to do, the better your DSTU2 experience you will be.

Schedule

This is our schedule

February 13th – API
We started work on our libraries and packages on NuGet to be as ready as we can for STU3. This includes the model (base resources) parsing and serialization, validation, snapshot generation.

March 20th – FHIR
FHIR STU3 is live.

April 1st – Simplifier
This is not a precise date yet, but around this time we expect our first internal release for BETA STU3 . This is also when we start working on making Simplifier still support DSTU2.

April 17th – Simplifier & Forge
Around this time Simplifier and Forge for will go live with STU3 BETA for the daredevils.

May 6th – Simplifier
Before this date, a week before the Madrid WGM starts, Simplifier and Forge will be battle tested for STU3 and you should be ready to use them.

When something goes live, we’ll tweet about it.

Migration of your profiles

Whether you manage your profiles on Simplifier or on your local disk, migrating structure definitions automatically is no task that can be done without human supervision. But there are several tools in the community to help you. And of course our consultancy team can offer to help you with that too. They have helped a wide range of customers around the globe with migrating their profiles to STU3.

 

 

 

Profile versioning

by Martijn Harthoorn

In the Simplifier team we think a lot about Profiles. We try to think of the places where they will be used and how they will be used. I will share with you one recent discussion we had about versioning of profiles.

The discussion started after a comment in the FHIR community from Lloyd McKenzie of the FHIR core team that a lot of profiles have references to other profiles (profiled references), that when updated, cause cascading updates. Let’s explore this. Assume we are working with a profile on Patient for our Hospital X. Let’s give the first published version of this profile the following canonical URI:
http://hospitalx/Patient-V1 (and lets abbreviate it to P1)

Now let’s assume we also have DiagnosticReports in our hospital, so we create a profile for those, and we give it this canonical URI:
http://hospitalx/DiagnosticReport-V1. (abbreviating it to D1)

As you might know a DiagnosticReport references a Patient. It’s not unlikely to assume here that we want our DiagnosticReports to only refer to Patient resources from our own hospital. For that FHIR allows you to profile the reference to Patient: a constrained reference. In your profile you tell that whatever patient your diagnostic report is about, that patient has be conformant to a Patient profile with the given canonical URI. In our case, the profile on DiagnosticReport (D1) will have a reference to Patient constrained to only allow P1.

This might seem a logical thing to do, but it turns out that it can be quite a burden on the authors of these profiles. If you publish a new version of your Patient profile (let’s call it P2), your existing diagnostic report profiles (D1) won’t allow referencing it because they have the reference definition constrained to only accept references to patients profiled on P1.

profileonpatientchanged

If you would update the DiagnosticReport profile, references to Patients of the P1 profile are not allowed either.

2017-01-05-13_43_40-drawing1-visio-professional

In some situations, this can cascade to an update of 50 profiles with a breaking change.

To be more precise: the problem that profile authors face is that they have to update the canonical URI in all profiles with a constrained reference. It also means that they want to be able to republish their profiles at once as a batch. You don’t want to have your other profiles reference an older version.

For these two steps, changing the references and republishing, it is possible to build tooling that can do most of the work automatically. We’re working hard on getting such capability within Simplifier for you.

But our discussion brought two other interesting versioning challenges to light.

The first challenge concerns authoring and deployment collaboration. Let’s say that the central administration of our hospital X upgrades their patient profile. But the lab creates their own DiagnosticReport profiles, and these DiagnosticReport profiles (of course) constrain the reference to Patient to only allow patients from within the hospital. Again D1 references P1. Now after the upgrade of the central administration for Patient from P1 to P2, the lab won’t be able to accept new P2 patients that are sent from within their own hospital, unless they upgraded their servers at the exact same time. This might be a good thing, because a breaking change is a breaking change for a reason. Note here that a breaking change is the only reason to change your canonical URI. But our presumption is that in a lot of cases, the profiled reference is not a matter of exact conformity. The lab doesn’t care about the all the exact fields of the patient. The only reason why the Patient reference was profiled, was to be sure they would not send or receive Patient data from outside the hospital. For example: they possibly only cared that the Patients they receive all have an identity given out by the central administration. For this likely scenario, a  canonical URL change creates a huge unnecessary burden that might trigger a very expensive cascade of downstream updates in profiles.

There are solutions here. For example: the hospital could start of with a profile “Patient Zero” (P0) that only has the hospital identity field. Then, have the ‘real’ profiles (P1 and P2) be derived profiles from “Patient zero”. This is not very obvious and of course requires a forward thinking profile author that had the smarts to take this step as his first action of business. It won’t be easy fix afterwards.

The second challenge that surfaced in our discussion is about our own tool Simplifier.net and possibly any FHIR registry. Once you have published one of your profiles in Simplifier, you have assigned it a unique canonical URI, you marked it as [Active] and by that you made a promise to your profile consumers that you won’t introduce any breaking changes for that profile. But of course at some point you are going to have to change something. Whether or not you are introducing a small update or a breaking change, you would like to continue working with that same profile. In the case of your Simplifier project, you don’t want to lose your version history, issue tracking and activity log. However you also don’t want to change the status of your published resource from “active” back to “draft”. Currently, you will have to create a new resource and work from there. That’s something we realized we have to change as creators of the tool.

For now our plan for the future is to allow creating draft versions of a resource even after it’s been published, instead of making you create a new resource. This will also require registering a canonical URI for each published version of your resource instead of how it is now: just once, which is automatically the latest version.

Your feedback here is appreciated as always. There is a feedback button on the top of the page in Simplifier.net. Our support team always reads it and responds to it.

Happy new year and happy profiling to everyone.

Forge

Forge 14.4 technical overview

Introduction

Last Friday (November 25th) we published a new Forge release, Forge 14.4 for DSTU2 – FHIR DevDays 2016 edition. You can download the new Forge release for free from www.simplifier.net. The previous Forge release (13.2) dates from June 10th, so clearly it’s been a while since we’ve published an update. In this blog post I’ll elaborate on the new features.

Profiles on profiles

Profiles created in Forge 13.2 and earlier always express a set of constraints on a FHIR core datatype or resource. However the FHIR specification also allows you to author so-called profiles-on-profiles or derived profiles. A derived profile represents a set of constraints on another, existing (non-core) base profile. This allows you to define a hierarchy of profiles that are based on each other.

In a typical situation, a national standards organization publishes a set of official national FHIR profiles defining some common country-wide constraints. Organizations within this country create and publish their own profiles that are derived from the official national profiles. For example, the Dutch HL7 affiliate publishes a national Dutch Patient profile with a specific constraint on Patient.identifier for the Dutch social security number (BSN). Dutch organizations can now author and publish their own, more specialized profiles with additional organizational constraints based on (derived from) the national profiles. Because different organizational profiles all conform to a common set of national profiles, they provide a certain well-defined level of interoperability within the country.

profile-hierarchy

Hierarchy of Profiles

The rules for derived profiles are similar to the rules for profiles on core datatypes or resources. A profile is valid if all resources that conform to the profile also conform to the associated base profile. This implies that a profile is only allowed constrain the base profile. A profile is not allowed to relax/remove constraints defined by the base profile.

Note: In a sense, we can interpret a profile as a filter or predicate on the resources space, cf. a WHERE clause in a SQL expression. The number of constraints that a profile expresses is inversely proportional to the total amount of conforming resources. As derived profiles become more and more specific, the total set of conforming resources becomes smaller and smaller. The extreme case being a profile that matches no resources at all; such a profile may very well be valid according to the FHIR spec, albeit meaningless. Similar to the SQL predicate WHERE 1=0 which is perfectly valid but will always evaluate to an empty set.

new-derived-profile

In Forge release 14.4, you can now create and edit profiles based on other (non-core) profiles. The command New Derived Profile creates a new derived profile based on an existing base profile on disk. The initial implementation is still somewhat limited and provides support for fairly simple base profiles (e.g. without slicing). Future updates will add support for more advanced scenario’s such as reslicing.

Resolving resources

Profiles created in previous Forge releases until version 13.2 were always based on a core datatype or resource definition.The Forge application directory contains an archive “specification.zip” that contains the official definitions of all FHIR core datatypes and resources. Upon application startup, Forge scans the ZIP archive and loads all the available definitions. When creating a new profile, Forge extracts the selected core base definition from the ZIP file and initializes a new profile from the selected base.

In the new 14.4 release we’ve added support to create and edit profiles based on other existing profiles. Forge now needs to resolve the custom base profile from a custom location, as it can no longer be resolved from the default ZIP archive that contains only the core definitions.

A profile can also express constraints on element types via external profiles. For example, a Patient profile may introduce a custom Identifier profile on the Patient.identifier element. In this case, Forge also needs to resolve the specified external profile in order to dynamically generate user interface components for profile child elements and verify the specified constraints.

The same holds for extension elements in a profile. Each extension element is mapped to an external extension definition. Forge needs to resolve the specified extension definitions in order to generate UI and verify constraints.

Forge should be able to resolve external profiles from the containing (working) folder on disk, from a separate external directory, from a FHIR server or from the FHIR registry Simplifier.net. To implement this, Forge requires a generic and flexible system to resolve external (user defined) profiles on the fly, separate from the standard core datatype and resource definitions.

Fortunately (and not coincidentally…) the FHIR .NET API library provides a flexible and efficient resolving mechanism based on the generic IResourceResolver interface, as explained by Ewout in his recent article Validation – and other new features in the .NET API stack. The library exposes default implementations that support resolving resources from disk and online. The library also exposes convenient decorators for caching and resolving from multiple sources based on priority. Forge 14.4 now leverages the FHIR .NET API to implement a flexible multi-step mechanism for resolving external (conformance) resources.

resourceresolver

Resource Resolvers

Upon opening an existing profile in Forge 14.4, the application creates an associated resolver that is bound to the containing folder on disk (including subfolders). The resolver provides cached access to all external profiles in the same folder structure. Future updates will introduce support for the configuration of additional sources, e.g. on local disk or online.

Note: for now, please keep related profiles together in a common folder and Forge should find them. Be aware that the resource resolvers cannot resolve profiles with duplicates, i.e. multiple profiles in the same folder that share a common canonical URI. When Forge detects duplicate profiles, it will display an error message and list the conflicting files. In that case, manually delete/move all the obsolete duplicates from the folder and retry.

The current Session Explorer in Forge provides a conceptual/logical view of resources and resource containers. However the new resource resolving approach introduces a new folder-based workflow that is fundamentally different from the earlier approach and does not fit well with the current Session Explorer. Therefore the Session Explorer will soon be phased out in favor of a new folder-based approach (cf. Visual Studio Solution Explorer).

Snapshot generation

One of the largest changes in the Forge 14.4 release involves a complete new implementation of the logic involving the generation of snapshot and differential components. Let’s take a closer look at these concepts.

snapchat-glasses

Snapshot Spectacles

A FHIR profile defines a structured set of elements. Each element definition has a set of properties. The combination of all the element definitions and associated properties uniquely defines a specific datatype or resource structure. This holds for both a core datatype or resource definition, as well as for a (derived) profile. FHIR defines this complete representation as the so-called “snapshot” component. The official name might be a bit misleading in that it suggests a captured state at a specific moment in time; while this is not incorrect, the key aspect of the snapshot component is that it intends to provide a complete and full representation of a certain data structure. An application that receives a profile snapshot has all the available information and does not need to resolve any external profiles.

A snapshot component can be quite large (~ thousands of lines of XML/JSON). And for a derived profile, the snapshot component contains a lot of redundant information: complete definitions of all the unconstrained elements and properties that are inherited from the base profile. Provided an application has access to the base resource, then it is possible to (re-)generate all the inherited redundant information. So we can also represent a profile by only describing the actual profile constraints and omitting all the information that is implicitly inherited from the base profile. Such a sparse representation of profile constraints is defined by FHIR as the “differential” component. The differential component only expresses the actual constraints with respect to the associated base profile. Generally, the differential component is (much) smaller than the snapshot component. Also the differential provides a convenient serialized representation for profile authors to browse and inspect, contrary to the snapshot component that contains a lot of redundant noise which makes it difficult to navigate.

The FHIR specification requires that a profile should contain either a snapshot component or a differential component or both (invariant sdf-6). A serialized profile with only a differential component has a smaller size and is therefore cheaper to persist and transmit. However this puts the burden on the receiving side to (re-)generate the snapshot component from the associated base profile. A serialized profile that contains a snapshot component is much larger but also self-sufficient; it contains all the available information and the receiver does not need access to any external profiles.

Forge releases up until version 13.2 contained custom application logic to generate the snapshot component from the differential. The old logic was fairly simple but useful as long as Forge was limited to profiles on core datatypes and resources. However in order to implement support for profiles on profiles (a.k.a. derived profiles), the existing snapshot generation logic was no longer sufficiently capable and needed to become a lot smarter. As we now had to completely re-implement snapshot generation anyway, we decided to move the logic out of Forge and into the open source FHIR .NET API library.

The implementation of full-fledged snapshot generation proved to be quite an undertaking. During the process we discovered that some low-level aspects were not yet clearly and/or fully defined by the FHIR specification. So we cooperated with the FHIR core team and profiling community to fill in the blanks and complete the missing parts of the specification. Especially Chris Grenz‘s contributions proved to be invaluable – Kudos! Working closely together with Chris, who works at Mayo Clinic, we managed to define how to unambiguously express some remaining advanced scenario’s and subsequently to align both our implementations.

Unfortunately the snapshot component is not completely deterministic, e.g. element id’s may vary, list order may vary etc. So FHIR cannot and does not define a “canonical” snapshot format that would allow us to perform a byte-wise or node-by-node comparison of two snapshot versions. Therefore a snapshot comparison algorithm needs some custom logic to handle special FHIR rules.

Although snapshot generation certainly isn’t the sexiest topic within the FHIR spec, it is a crucial aspect for any system dealing with profiles (including validation). And feature-complete snapshot generation proved to be quite challenging to implement. For example, in order to generate snapshots for core type definitions, the algorithm must be able to detect and handle recursive type relations, e.g. Element.id and Extension.extension. Also the correct implementation of different scenario’s involving (re-)slicing proved to be non-trivial. So we’re quite pleased that this fundamental part of the FHIR specification is now clearly defined and completely implemented (barring some unknown unknowns…). If you’re interested in the implementation (you are definitely a FHIR nerd like us and) you can inspect the Hl7.Fhir.Specification component source on Github. Fortunately usage is very simple:

IResourceResolver resolver = ...
StructureDefinition structureDef = ...
var generator = new SnapshotGenerator(resolver, settings);
generator.Update(structureDef);

The snapshot generator uses the specified IResourceResolver instance to resolve references to external profiles and/or extension definitions. If this argument equals null, then the generator automatically falls back to the set of FHIR core datatype and resource definitions. The optional settings parameter allows the caller to specify some custom configuration options for the generator.

simplifierSimplifier.net is not yet capable of automatically (re-)generating the snapshot component. Currently, when you publish a profile, you have to explicitly submit the snapshot component in order for Simplifier.net to render the complete structure. Note that if you publish a profile to Simplifier from within Forge, the application automatically handles this. Once the new release of the FHIR .NET API is ready and stable, we will update Simplifier.net and integrate the new snapshot generator. From then on, Simplifier will be able to accept profiles with only a differential component and automatically (re-)generate the snapshot. Our team is also working on implementing a dynamic differential rendering in Simplifier, so the user can activate a filtered view of only the differential profile constraints.

Differential generation

We’ve seen how snapshot generation transforms a sparse tree of constraints into a complete representation of the target structure. This functionality is now provided by the FHIR .NET API library, in the form of an atomic operation (cf. pipeline). Of course, there also exists an inverse transformation that generates a sparse differential component from a complete snapshot representation of a structure. The differential representation only contains nodes that are constrained with respect to the base resource. Unconstrained nodes are excluded from the differential.

differential

Differential Structure

Forge generates the differential representation on-the-fly and keeps it in sync with the profile while it is being edited. This approach is completely different from the snapshot generation. Each element definition and property in a user profile is associated with the corresponding node in the underlying base profile. This allows Forge to determine which profile nodes are actually constrained (not empty and not equal to base). To no surprise, the process to find the corresponding nodes in the base profile turns out to be quite involved, esp. for slicing.

Originally, up until version 13.2, Forge contained custom logic to resolve corresponding nodes in the base profile. The original logic was limited to handling only fairly simple profiles, esp. core datatype and resource definitions and was not very efficient on memory usage. As the application developed further, we were slowly stretching the limits of the original algorithm. Users started reporting unpredictable behavior (i.e. differential was not always properly synchronized) that was hard/impossible to fix. Eventually this was no longer maintainable. So we decided to also completely rewrite the differential generation logic.

Fortunately, the new snapshot generator logic in the FHIR .NET API library already needs to resolve corresponding nodes in the base profile anyway (because each differential constraint is merged onto the corresponding base node). So Forge does no longer have to perform that same complex exercise itself. Instead, Forge now receives all the base profile associations as a useful by-product of the snapshot generator. This is quite efficient and increases the loading speed of a profile. Also we no longer need to maintain and harmonize separate but similar implementations in Forge and the .NET API library, as all common and reusable logic has now been moved into the API.

Since each node in the loaded profile is now associated with the corresponding node in the base profile, it is now trivial to determine which nodes are actually constrained with respect to the base resource. Only constrained nodes are actually included in the differential. Unconstrained element properties (either empty or equal to base) are set to null in the underlying PoCo. Also complex nodes and element definitions having all child properties equal to null are cleared in the PoCo and their state is automatically re-evaluated whenever a child node has changed. This way, all unconstrained nodes are effectively excluded from the differential component. The whole process is quite efficient as after a change, only the affected nodes are re-evaluated.

storagehandlers

Generate Differential

The serialized xml representation is (re-)generated in the background and rendered whenever the user activates the Xml tab. This ensures that the displayed xml is always up to date without impeding the application performance. The Forge UI only displays the differential, by design, in order to save memory. By default, profiles are also saved to disk with only the differential component. The Options menu provides a configuration setting to toggle the serialization of snapshots components when saving profiles to disk. Needles to say that profiles with snapshots are significantly larger, therefore in Forge the snapshot option is disabled by default.

Conclusion

This article is way too long… hats off if you managed to read and process all this information. And I still feel like I just scratched the surface. Time permitting, maybe I’ll author another article about the specifics of the snapshot generator implementation, for those few FHIR die hards that care (the usual suspects).

sad-tree2

Sparse Tree

Looking back to 2016, the Furore FHIR team has invested a lot of time and effort in (re-)designing and (re-)implementing the fundamental infrastructural components of a FHIR ecosystem, thereby laying the groundwork for a plethora of useful productivity-enhancing features that we can now start building and enhancing in 2017. I must confess that I really enjoyed working  on some of the hard core abstract logic. But the nerdy stuff can also be quite demanding, time consuming and eventually goes at the expense of basic social behavior and interpersonal relations… So I’ve learned a long time ago that, even though I take joy in higher abstract thinking, I shouldn’t bury myself in it for all too long, as there is a personal price to pay. Now the Forge user interface desperately needs some love – as well as my estranged girlfriend, who’s totally fed up with me rambling on and on about snapshots and such. So I am eagerly looking forward to next year, when I can focus again on improving usability, workflow and my personal love life. But first let’s celebrate the holidays, decorate a tree, spend some quality time with family and friends and hopefully regain some of those long lost social skills.

Happy profiling!

Michel Rutten

Forge

Forging Profiles on Profiles

Introduction

Recently (Friday November 25th) we published a new Forge release, Forge 14.4 for DSTU2 – FHIR DevDays 2016 edition. You can download the new Forge release for free from Simplifier.net. The previous Forge release (13.2) dates from June 10th, so clearly it’s been a while since we’ve published an update. In this blog post I’ll elaborate on the most important new feature: support for profiles on profiles, also known as derived profiles.

What is a derived profile?

Out of the box FHIR resources are designed to cover 80% of the common use cases. In order to cover the remaining 20%, FHIR introduces profiles. A FHIR profile defines a set of constraints on a standard FHIR resource or datatype. This allows you to define a specialized variant of a standard resource for a specific use case. And because profiles are based on common FHIR resources, they provide a certain basic level of global interoperability with other (external) FHIR systems.

Obviously, as we limit the context to a specific country, region or organization, we can probably harmonize more aspects and provide increased interoperability within that limited context. FHIR supports this common use case by allowing you to define so-called profiles on profiles a.k.a. derived profiles. A derived profile defines a set of constraints on another existing profile, the so-called base profile. The derived profile implicitly inherits all the existing constraints from the associated base profile. A derived profile can only further constrain the base profile, similar to profiles on core resources.

Example:

  • The HL7 affiliate for the Netherlands publishes a national profile for a Dutch patient. The profile is based on the standard FHIR Patient resource and defines a number of additional constraints, e.g. in order to identify Dutch patients by their unique national social security number (BSN). The profile also constrains the generic Patient.careProvider reference to a specific Dutch practitioner profile.
  • A Dutch hospital authors a derived Patient profile for use within the organization. The organizational Patient profile is based on the national Dutch patient profile and defines some additional constraints that only apply within the organization. The hospital’s custom Patient profile defines additional constraints for the unique Patient identifier that is assigned by the hospital EHR system. The profile also prohibits the Patient.animal element, as it does not apply.
Hierarchy of Profiles

Hierarchy of profiles

As the example demonstrates, this allows you to create a hierarchy of profiles, e.g. on a national, regional, organizational and/or departmental level. As we navigate deeper down the profile hierarchy, we encounter more specific profiles for more limited use contexts.

Authoring a derived profile in Forge

New Derived ProfileUsing the new Forge 14.4 release, you can now easily author a derived profile. Forge 14.4 introduces a new command “New Derived Profile“. You can find the new command directly on the start screen, in the File menu and also in the Session Explorer toolbar. The new command resembles the existing “New Profile” command, but with one important difference: you can select an existing profile as the underlying base profile. Forge will display a File Open dialog window to select a suitable base profile from disk. After selecting the base profile, Forge will create and display the new derived profile.

For example, let’s assume that Mayo Clinic wants to create their own derived Patient profile based on the US DAF Patient profile which defines a set of national constraints for the US Patient. Below, we can see the original DAF Patient base profile loaded in Forge. All the constraints defined by the base profile are clearly indicated by the yellow pen icon:

DAF Patient Profile

Notes:

  • The blue exclamation icon indicates a “must support” element
  • The blue paperclip icon indicates extension elements
  • The yellow pen icon indicates a constrained element

Now Mayo Clinics creates a new derived Patient profile based on the DAF Patient profile. The new derived Patient profile inherits all constraints that are defined by the base DAF Patient profile. Initially, the new derived profile does not introduce any new constraints, as is clearly visible in the Forge Element Tree:

Mayo Clinic Patient Profile

The screenshot demonstrates that the derived Patient profile indeed inherits all the constraints defined by the base DAF Patient profile (e.g. you can see the inherited extensions such as ethnicity). Because the implicitly inherited constraints are not defined by the derived profile itself, they are not considered to be part of it. This is clearly visible in the Forge element tree, as none of the elements in the new derived profile show a yellow pen icon (except on the root element, because the derived profile has a new name & url). In other words, the new derived profile shown above is (almost) empty.

Now Mayo Clinic can define additional specific constraints in the derived profile. Any newly defined constraints will actually be part of the derived profile and consequently they will be indicated by a yellow pen in Forge. For example, the screenshot below shows how the derived profile defines an additional constraint on the Patient.photo element:

Mayo Clinic Patient Profile with constraint on photo

Below you can see the serialized XML of the derived Patient profile. We can clearly see the newly introduced constraint on the Patient.photo element. Note that the root element definition is mandatory in DSTU2, even if the element is not constrained. This requirement will be deprecated in STU3. Also note the (mandatory) reference to the underlying DAF Patient base profile.

<?xml version="1.0" encoding="utf-8"?>
<StructureDefinition xmlns="http://hl7.org/fhir">
  <meta>
    <lastUpdated value="2016-12-01T16:47:22.41+01:00" />
  </meta>
  <url value="StructureDefinition/MayClinicPatientProfile" />
  <name value="Mayo Clinic Patient Profile" />
  <status value="draft" />
  <date value="2016-12-01T16:28:17+01:00" />
  <fhirVersion value="1.0.2" />
  <kind value="resource" />
  <constrainedType value="Patient" />
  <abstract value="false" />
  <base value="http://hl7.org/fhir/StructureDefinition/daf-patient" />
  <differential>
    <element>
      <path value="Patient" />
    </element>
    <element>
      <path value="Patient.photo" />
      <max value="0" />
    </element>
  </differential>
</StructureDefinition>

Conclusion

This concludes our introduction of the new support for derived profiles in Forge. The example demonstrates that the authoring process is actually quite easy for end users and quite similar to creating profiles on core resources. After all, in FHIR we try to move complexity away to the developers of tools and servers.

As simple as this all seems to the end user, the actual implementation entailed a long and winding road straight into the bowels of the FHIR specification. In a separate blog post, I will provide some more in-depth background information about the technical challenges we faced when trying to implement support for derived profiles, and how we addressed these.

If you have questions concerning (derived) profiles or need a bit of assistance, then you are strongly encouraged to join the online FHIR discussion groups. The FHIR community is very active, friendly and helpful and usually you quickly receive helpful responses from FHIR users all over the world. The conformance stream is specifically intended to discuss FHIR profiling and related tools including Forge. Also if you’ve implemented FHIR in a production environment, then please inform the FHIR community and share your story in the implementers stream!

Two final notes:

  1. Today we have published a minor hotfix release (14.4.1) with a couple of bugfixes. Should you encounter any issues while trying to repeat the exercise above, then please make sure that you have upgraded to the most recent version. Users of release 14.4 will receive an auto-update notification. Users of earlier releases should manually download and install the new version.
  2. The new Forge release does not yet support base profiles containing slices. If you select a base profile containing slices, snapshot generation will fail and generate a runtime exception. We will introduce support for sliced base profiles in a future application update.

Happy profiling!

Michel Rutten

FHIR DevDays 2016 in Amsterdam

by Martijn Harthoorn

Whenever I am at the FHIR DevDays in Amsterdam, I always get this vibrant feeling that this is where it happens. And it’s true: significant things happen. For those who could not attend, here are the highlights.

Developments in the FHIR standard

We are not that far away from publishing STU3. And STU4 is on the horizon. It will not be long until parts of the FHIR specification will go normative. But we have also seen at the DevDays that there are still many initiatives to expand the possibilities of FHIR. There are quite a number of proposals of adding domain specific languages. The most obvious one is FhirPath, which will come in STU3. Ewout has provided an extensive look into it with a Birds of a Feather session. There is now sufficient tooling and API’s available to get you up and running.

There have been discussions to extend FHIR to Swagger or RAML, for which Nicolai Ryzhikov of Health Samurai has done some excellent research (see more below). Grahame spoke about a new language, intended for mapping between different standards.  This mapping language is now part of the proposed STU3 specification. And of course, there is a new resource for it: StructureMap. Bryn Rhodes showed a very mature implementation of CQL for decision support, which could be a candidate for the FHIR specification too.

A small change with a big impact: at the DevDays we broke the data type HumanName. Family will get a cardinality of zero to one! The structure of it is now defined in extensions. This will help countries like Germany, The Netherlands and all Spanish speaking countries.

FHIR Community

Grahame as always gave a very insightful and inspirational outlook on what he thinks of the future of FHIR in the next year. We still do not seem to be on the top of the Gartner hype curve. The FHIR community now exceeds a 1000 people actively involved in developing the standard. Grahame told us that HL7 and the FHIR Foundation are currently developing a certification process for FHIR credentialed consultants. There is also an option for FHIR to go broader, since a lot of the FHIR architecture, services and tooling might have great use outside of Health.

WHO’s on FHIR?

The University of Pittsburg is on FHIR! They have started using FHIR to connect their 1000+ clinical systems. They are even discussing to make parts open source. We saw interoperability at its best with Cerner and Epic on one stage together showing how their infrastructure works together with FHIR.  IBM showed us how Watson integrates with their own FHIR server and how they map between FHIR and HealthKit. Patients Know Best have worked out and implemented a very valuable clean way to look at patient consent and are actively involved in sharing their thoughts with the developers of the standard.

Tooling

I will start with the tooling. As Alexander Henket nicely put it: traditionally in health there is a shortage of tooling. With the FHIR standard, for the first time, there is an abundance of it.

The build tool for Implementation Guides

In his talk, Lloyd told us that Implementation Guides can now be published independently of the Specification. This is a big step forward, since the ‘build’ of the spec is no small thing. It would take between 25 and 60 minutes.The new tool runs in about 60 seconds. You can find the documentation here. It’s also good news that Implementation Guides that are not maintained by HL7 can nonetheless be published at fhir.org.

HAPI

HAPI has until recently been a set of API’s that you can build a server with. At the DevDays, James Agnew launched SMILE CDR. This is HAPI server components packaged as a product, that runs on top of database like Postgres, MySQL, Oracle, SQL Server. Of  course still works with HL7v2, where HAPI since long is a big name and includes Smart on FHIR and has monitoring tools and a set of really cool management tools like SMART-on-FHIR apps registration. It’s still work in progress, but James team is pretty far. You can take a peek here.

CQL Compiler

Bryn Rhodes has worked on CQL. It’s a language expressing the logic involved in Quality measures and decision support artifacts. And CQL can now be run against FHIR resources and translates to ELM. Bryn showed us all this with the CQL runner tool.

FHIR schema

Nicolai Rhyzikov from Health Samurai showed a complete Swagger definition for the FHIR REST api. His team has also been working on profiling and validation with JSON schema. See code here. There is also an application here. There is even more. His team has been working on a Test script runner in Clojure.  Mike showed JUTE. It’s a yaml like custom domain language to map HL7v2 messages to FHIR. It has a lot of similarities with FhirPath so they might see how it can be integrated.

FHIR Client for Python and Swift

Pascal Pfiffner from Harvard Medical University showed us their new Python Client for FHIR. You can find the python code here. It has a FHIR parser and even has a good basic Validator. Pascal also showed a FHIR Swift implementation, which you can find here. It has all the FHIR data models and has included the same validation feature as their Python tooling.

Simplifier

Ardon Toonstra showed us all new features of Simplifier.net: our own FHIR registry. Ardon showed how to find profiles and extensions, but also the whole path from uploading a resource to publishing and render them as a tree table or UML to using that same resource when writing a complete Implementation Guide in the new Simplifier IG editor, with all FHIR rendering tooling available.

SPARK

Christiaan Knaap from Furore showed the evolution of SPARK that started as one of the first FHIR reference implementation servers to an enterprise grade FHIR server. SPARK has been rewritten on top of ASP.NET Core. It now consists of a several components and a custom API surface, so that it can handle many different scenario’s, like a validation only server, a front end for different databases, like MongoDb and SQL Server, a hub in an existing health infrastructure or a facade for non FHIR systems.

ART-DECOR

Alexander Henket from Nictiz showed us the FHIR Roadmap for ART-DECOR, and open source multidisciplinary tool stack for authoring and sharing of templates, valuesets and much more. The tooling now has a resource viewer, import/export of FHIR resources. It now collaborates with external FHIR repositories, for starters with Simplifier.net and has terminology services. The  plan for coming year is to include basic profile editing scenarios.

That’s all folks!
Hope to see you next year at the DevDays, 15-17 November 2017 in Amsterdam!

Technology with teeth: #FHIR in the year of Trump and Brexit

For who hasn’t the time to watch the video of Grahame Grieve’s keynote at FHIR Developer Days 2016, here is a summary of the important things he had to say.

The central theme of his address is the FHIR community. What does it mean to be part of that community, what do you get out of it, and what’s the importance of the community for the FHIR standard?

The example of the medical profession works for health IT. There isn’t a clinician who thinks he/she is a good clinician, who is not a member of a professional society and who doesn’t contribute to that society.

What do you get out of getting engaged in the FHIR community, or in any community? What’s the business model? The business model is not about the bottom line for this year, but in three years time, when you retained your best staff.

Grahame proposes the – yet rudimentary – framework of an engagement matrix, that ranks the culture of engagement of any company and helps buyers in their procurement. If you score 0 to 2 in this matrix you are entirely focused on the short term, you won’t retain the best staff and customers will be screwed over. 9 to 11 means the opposite. [After the keynote Grahame announced a more elaborate blog on this subject.]

From a more general community perspective, it’s been a bad year. Not for FHIR, but because of what happened at the ballot box. We’ve seen populations turning away from openness, from engagement, form the community. This is counter to what FHIR is all about. This mentality shift is the single biggest threat to the FHIR community.

But Grahame Grieve is confident just the same:

“We’re open not because of some philosopy or culture, but we’re doing it because it works for what we’re trying to achieve. We’re doing it because it makes sense for our goals. We’re doing it because our values lead to outcomes that other people can’t create.”

Who will benefit from FHIR? “Give the data to the patient”, that’s nice for the digital literati but not for those who are too old or cannot afford a computer. FHIR was never just about getting the patients their data. Instead, the vast majority of patients will benefit from government controled back-end integration. Don’t get too extracted on getting the patients their data.

The final topic Grahame Grieve touches in his most engaged performance I have witnessed so far, is a phenomenon he calls alt-med. This, of course, is a variant of the dubious alt-right movement that has a reputation for being post-truth or post-fact. Facts no longer matter. Alt-med means people moving away from traditional healthcare, guided by rumours and false facts in social media. Getting good clinical information into social media is going to be a big thing for FHIR in the upcoming years.

The times we live in ask for technology that is a counterbalance to modern aberations: technology with teeth.

The video of Grahame Grieve’s keynote is available on our website.