Forge

Forge 14.4 technical overview

Introduction

Last Friday (November 25th) we published a new Forge release, Forge 14.4 for DSTU2 – FHIR DevDays 2016 edition. You can download the new Forge release for free from www.simplifier.net. The previous Forge release (13.2) dates from June 10th, so clearly it’s been a while since we’ve published an update. In this blog post I’ll elaborate on the new features.

Profiles on profiles

Profiles created in Forge 13.2 and earlier always express a set of constraints on a FHIR core datatype or resource. However the FHIR specification also allows you to author so-called profiles-on-profiles or derived profiles. A derived profile represents a set of constraints on another, existing (non-core) base profile. This allows you to define a hierarchy of profiles that are based on each other.

In a typical situation, a national standards organization publishes a set of official national FHIR profiles defining some common country-wide constraints. Organizations within this country create and publish their own profiles that are derived from the official national profiles. For example, the Dutch HL7 affiliate publishes a national Dutch Patient profile with a specific constraint on Patient.identifier for the Dutch social security number (BSN). Dutch organizations can now author and publish their own, more specialized profiles with additional organizational constraints based on (derived from) the national profiles. Because different organizational profiles all conform to a common set of national profiles, they provide a certain well-defined level of interoperability within the country.

profile-hierarchy

Hierarchy of Profiles

The rules for derived profiles are similar to the rules for profiles on core datatypes or resources. A profile is valid if all resources that conform to the profile also conform to the associated base profile. This implies that a profile is only allowed constrain the base profile. A profile is not allowed to relax/remove constraints defined by the base profile.

Note: In a sense, we can interpret a profile as a filter or predicate on the resources space, cf. a WHERE clause in a SQL expression. The number of constraints that a profile expresses is inversely proportional to the total amount of conforming resources. As derived profiles become more and more specific, the total set of conforming resources becomes smaller and smaller. The extreme case being a profile that matches no resources at all; such a profile may very well be valid according to the FHIR spec, albeit meaningless. Similar to the SQL predicate WHERE 1=0 which is perfectly valid but will always evaluate to an empty set.

new-derived-profile

In Forge release 14.4, you can now create and edit profiles based on other (non-core) profiles. The command New Derived Profile creates a new derived profile based on an existing base profile on disk. The initial implementation is still somewhat limited and provides support for fairly simple base profiles (e.g. without slicing). Future updates will add support for more advanced scenario’s such as reslicing.

Resolving resources

Profiles created in previous Forge releases until version 13.2 were always based on a core datatype or resource definition.The Forge application directory contains an archive “specification.zip” that contains the official definitions of all FHIR core datatypes and resources. Upon application startup, Forge scans the ZIP archive and loads all the available definitions. When creating a new profile, Forge extracts the selected core base definition from the ZIP file and initializes a new profile from the selected base.

In the new 14.4 release we’ve added support to create and edit profiles based on other existing profiles. Forge now needs to resolve the custom base profile from a custom location, as it can no longer be resolved from the default ZIP archive that contains only the core definitions.

A profile can also express constraints on element types via external profiles. For example, a Patient profile may introduce a custom Identifier profile on the Patient.identifier element. In this case, Forge also needs to resolve the specified external profile in order to dynamically generate user interface components for profile child elements and verify the specified constraints.

The same holds for extension elements in a profile. Each extension element is mapped to an external extension definition. Forge needs to resolve the specified extension definitions in order to generate UI and verify constraints.

Forge should be able to resolve external profiles from the containing (working) folder on disk, from a separate external directory, from a FHIR server or from the FHIR registry Simplifier.net. To implement this, Forge requires a generic and flexible system to resolve external (user defined) profiles on the fly, separate from the standard core datatype and resource definitions.

Fortunately (and not coincidentally…) the FHIR .NET API library provides a flexible and efficient resolving mechanism based on the generic IResourceResolver interface, as explained by Ewout in his recent article Validation – and other new features in the .NET API stack. The library exposes default implementations that support resolving resources from disk and online. The library also exposes convenient decorators for caching and resolving from multiple sources based on priority. Forge 14.4 now leverages the FHIR .NET API to implement a flexible multi-step mechanism for resolving external (conformance) resources.

resourceresolver

Resource Resolvers

Upon opening an existing profile in Forge 14.4, the application creates an associated resolver that is bound to the containing folder on disk (including subfolders). The resolver provides cached access to all external profiles in the same folder structure. Future updates will introduce support for the configuration of additional sources, e.g. on local disk or online.

Note: for now, please keep related profiles together in a common folder and Forge should find them. Be aware that the resource resolvers cannot resolve profiles with duplicates, i.e. multiple profiles in the same folder that share a common canonical URI. When Forge detects duplicate profiles, it will display an error message and list the conflicting files. In that case, manually delete/move all the obsolete duplicates from the folder and retry.

The current Session Explorer in Forge provides a conceptual/logical view of resources and resource containers. However the new resource resolving approach introduces a new folder-based workflow that is fundamentally different from the earlier approach and does not fit well with the current Session Explorer. Therefore the Session Explorer will soon be phased out in favor of a new folder-based approach (cf. Visual Studio Solution Explorer).

Snapshot generation

One of the largest changes in the Forge 14.4 release involves a complete new implementation of the logic involving the generation of snapshot and differential components. Let’s take a closer look at these concepts.

snapchat-glasses

Snapshot Spectacles

A FHIR profile defines a structured set of elements. Each element definition has a set of properties. The combination of all the element definitions and associated properties uniquely defines a specific datatype or resource structure. This holds for both a core datatype or resource definition, as well as for a (derived) profile. FHIR defines this complete representation as the so-called “snapshot” component. The official name might be a bit misleading in that it suggests a captured state at a specific moment in time; while this is not incorrect, the key aspect of the snapshot component is that it intends to provide a complete and full representation of a certain data structure. An application that receives a profile snapshot has all the available information and does not need to resolve any external profiles.

A snapshot component can be quite large (~ thousands of lines of XML/JSON). And for a derived profile, the snapshot component contains a lot of redundant information: complete definitions of all the unconstrained elements and properties that are inherited from the base profile. Provided an application has access to the base resource, then it is possible to (re-)generate all the inherited redundant information. So we can also represent a profile by only describing the actual profile constraints and omitting all the information that is implicitly inherited from the base profile. Such a sparse representation of profile constraints is defined by FHIR as the “differential” component. The differential component only expresses the actual constraints with respect to the associated base profile. Generally, the differential component is (much) smaller than the snapshot component. Also the differential provides a convenient serialized representation for profile authors to browse and inspect, contrary to the snapshot component that contains a lot of redundant noise which makes it difficult to navigate.

The FHIR specification requires that a profile should contain either a snapshot component or a differential component or both (invariant sdf-6). A serialized profile with only a differential component has a smaller size and is therefore cheaper to persist and transmit. However this puts the burden on the receiving side to (re-)generate the snapshot component from the associated base profile. A serialized profile that contains a snapshot component is much larger but also self-sufficient; it contains all the available information and the receiver does not need access to any external profiles.

Forge releases up until version 13.2 contained custom application logic to generate the snapshot component from the differential. The old logic was fairly simple but useful as long as Forge was limited to profiles on core datatypes and resources. However in order to implement support for profiles on profiles (a.k.a. derived profiles), the existing snapshot generation logic was no longer sufficiently capable and needed to become a lot smarter. As we now had to completely re-implement snapshot generation anyway, we decided to move the logic out of Forge and into the open source FHIR .NET API library.

The implementation of full-fledged snapshot generation proved to be quite an undertaking. During the process we discovered that some low-level aspects were not yet clearly and/or fully defined by the FHIR specification. So we cooperated with the FHIR core team and profiling community to fill in the blanks and complete the missing parts of the specification. Especially Chris Grenz‘s contributions proved to be invaluable – Kudos! Working closely together with Chris, who works at Mayo Clinic, we managed to define how to unambiguously express some remaining advanced scenario’s and subsequently to align both our implementations.

Unfortunately the snapshot component is not completely deterministic, e.g. element id’s may vary, list order may vary etc. So FHIR cannot and does not define a “canonical” snapshot format that would allow us to perform a byte-wise or node-by-node comparison of two snapshot versions. Therefore a snapshot comparison algorithm needs some custom logic to handle special FHIR rules.

Although snapshot generation certainly isn’t the sexiest topic within the FHIR spec, it is a crucial aspect for any system dealing with profiles (including validation). And feature-complete snapshot generation proved to be quite challenging to implement. For example, in order to generate snapshots for core type definitions, the algorithm must be able to detect and handle recursive type relations, e.g. Element.id and Extension.extension. Also the correct implementation of different scenario’s involving (re-)slicing proved to be non-trivial. So we’re quite pleased that this fundamental part of the FHIR specification is now clearly defined and completely implemented (barring some unknown unknowns…). If you’re interested in the implementation (you are definitely a FHIR nerd like us and) you can inspect the Hl7.Fhir.Specification component source on Github. Fortunately usage is very simple:

IResourceResolver resolver = ...
StructureDefinition structureDef = ...
var generator = new SnapshotGenerator(resolver, settings);
generator.Update(structureDef);

The snapshot generator uses the specified IResourceResolver instance to resolve references to external profiles and/or extension definitions. If this argument equals null, then the generator automatically falls back to the set of FHIR core datatype and resource definitions. The optional settings parameter allows the caller to specify some custom configuration options for the generator.

simplifierSimplifier.net is not yet capable of automatically (re-)generating the snapshot component. Currently, when you publish a profile, you have to explicitly submit the snapshot component in order for Simplifier.net to render the complete structure. Note that if you publish a profile to Simplifier from within Forge, the application automatically handles this. Once the new release of the FHIR .NET API is ready and stable, we will update Simplifier.net and integrate the new snapshot generator. From then on, Simplifier will be able to accept profiles with only a differential component and automatically (re-)generate the snapshot. Our team is also working on implementing a dynamic differential rendering in Simplifier, so the user can activate a filtered view of only the differential profile constraints.

Differential generation

We’ve seen how snapshot generation transforms a sparse tree of constraints into a complete representation of the target structure. This functionality is now provided by the FHIR .NET API library, in the form of an atomic operation (cf. pipeline). Of course, there also exists an inverse transformation that generates a sparse differential component from a complete snapshot representation of a structure. The differential representation only contains nodes that are constrained with respect to the base resource. Unconstrained nodes are excluded from the differential.

differential

Differential Structure

Forge generates the differential representation on-the-fly and keeps it in sync with the profile while it is being edited. This approach is completely different from the snapshot generation. Each element definition and property in a user profile is associated with the corresponding node in the underlying base profile. This allows Forge to determine which profile nodes are actually constrained (not empty and not equal to base). To no surprise, the process to find the corresponding nodes in the base profile turns out to be quite involved, esp. for slicing.

Originally, up until version 13.2, Forge contained custom logic to resolve corresponding nodes in the base profile. The original logic was limited to handling only fairly simple profiles, esp. core datatype and resource definitions and was not very efficient on memory usage. As the application developed further, we were slowly stretching the limits of the original algorithm. Users started reporting unpredictable behavior (i.e. differential was not always properly synchronized) that was hard/impossible to fix. Eventually this was no longer maintainable. So we decided to also completely rewrite the differential generation logic.

Fortunately, the new snapshot generator logic in the FHIR .NET API library already needs to resolve corresponding nodes in the base profile anyway (because each differential constraint is merged onto the corresponding base node). So Forge does no longer have to perform that same complex exercise itself. Instead, Forge now receives all the base profile associations as a useful by-product of the snapshot generator. This is quite efficient and increases the loading speed of a profile. Also we no longer need to maintain and harmonize separate but similar implementations in Forge and the .NET API library, as all common and reusable logic has now been moved into the API.

Since each node in the loaded profile is now associated with the corresponding node in the base profile, it is now trivial to determine which nodes are actually constrained with respect to the base resource. Only constrained nodes are actually included in the differential. Unconstrained element properties (either empty or equal to base) are set to null in the underlying PoCo. Also complex nodes and element definitions having all child properties equal to null are cleared in the PoCo and their state is automatically re-evaluated whenever a child node has changed. This way, all unconstrained nodes are effectively excluded from the differential component. The whole process is quite efficient as after a change, only the affected nodes are re-evaluated.

storagehandlers

Generate Differential

The serialized xml representation is (re-)generated in the background and rendered whenever the user activates the Xml tab. This ensures that the displayed xml is always up to date without impeding the application performance. The Forge UI only displays the differential, by design, in order to save memory. By default, profiles are also saved to disk with only the differential component. The Options menu provides a configuration setting to toggle the serialization of snapshots components when saving profiles to disk. Needles to say that profiles with snapshots are significantly larger, therefore in Forge the snapshot option is disabled by default.

Conclusion

This article is way too long… hats off if you managed to read and process all this information. And I still feel like I just scratched the surface. Time permitting, maybe I’ll author another article about the specifics of the snapshot generator implementation, for those few FHIR die hards that care (the usual suspects).

sad-tree2

Sparse Tree

Looking back to 2016, the Furore FHIR team has invested a lot of time and effort in (re-)designing and (re-)implementing the fundamental infrastructural components of a FHIR ecosystem, thereby laying the groundwork for a plethora of useful productivity-enhancing features that we can now start building and enhancing in 2017. I must confess that I really enjoyed working  on some of the hard core abstract logic. But the nerdy stuff can also be quite demanding, time consuming and eventually goes at the expense of basic social behavior and interpersonal relations… So I’ve learned a long time ago that, even though I take joy in higher abstract thinking, I shouldn’t bury myself in it for all too long, as there is a personal price to pay. Now the Forge user interface desperately need some love – as well as my estranged girlfriend, who’s totally fed up with me rambling on and on about snapshots and such. So I am eagerly looking forward to next year, when I can focus again on improving usability, workflow and my personal love life. But first let’s celebrate the holidays, decorate a tree, spend some quality time with family and friends and hopefully regain some of those long lost social skills.

Happy profiling!

Michel Rutten

Forge

Forging Profiles on Profiles

Introduction

Recently (Friday November 25th) we published a new Forge release, Forge 14.4 for DSTU2 – FHIR DevDays 2016 edition. You can download the new Forge release for free from Simplifier.net. The previous Forge release (13.2) dates from June 10th, so clearly it’s been a while since we’ve published an update. In this blog post I’ll elaborate on the most important new feature: support for profiles on profiles, also known as derived profiles.

What is a derived profile?

Out of the box FHIR resources are designed to cover 80% of the common use cases. In order to cover the remaining 20%, FHIR introduces profiles. A FHIR profile defines a set of constraints on a standard FHIR resource or datatype. This allows you to define a specialized variant of a standard resource for a specific use case. And because profiles are based on common FHIR resources, they provide a certain basic level of global interoperability with other (external) FHIR systems.

Obviously, as we limit the context to a specific country, region or organization, we can probably harmonize more aspects and provide increased interoperability within that limited context. FHIR supports this common use case by allowing you to define so-called profiles on profiles a.k.a. derived profiles. A derived profile defines a set of constraints on another existing profile, the so-called base profile. The derived profile implicitly inherits all the existing constraints from the associated base profile. A derived profile can only further constrain the base profile, similar to profiles on core resources.

Example:

  • The HL7 affiliate for the Netherlands publishes a national profile for a Dutch patient. The profile is based on the standard FHIR Patient resource and defines a number of additional constraints, e.g. in order to identify Dutch patients by their unique national social security number (BSN). The profile also constrains the generic Patient.careProvider reference to a specific Dutch practitioner profile.
  • A Dutch hospital authors a derived Patient profile for use within the organization. The organizational Patient profile is based on the national Dutch patient profile and defines some additional constraints that only apply within the organization. The hospital’s custom Patient profile defines additional constraints for the unique Patient identifier that is assigned by the hospital EHR system. The profile also prohibits the Patient.animal element, as it does not apply.
Hierarchy of Profiles

Hierarchy of profiles

As the example demonstrates, this allows you to create a hierarchy of profiles, e.g. on a national, regional, organizational and/or departmental level. As we navigate deeper down the profile hierarchy, we encounter more specific profiles for more limited use contexts.

Authoring a derived profile in Forge

New Derived ProfileUsing the new Forge 14.4 release, you can now easily author a derived profile. Forge 14.4 introduces a new command “New Derived Profile“. You can find the new command directly on the start screen, in the File menu and also in the Session Explorer toolbar. The new command resembles the existing “New Profile” command, but with one important difference: you can select an existing profile as the underlying base profile. Forge will display a File Open dialog window to select a suitable base profile from disk. After selecting the base profile, Forge will create and display the new derived profile.

For example, let’s assume that Mayo Clinic wants to create their own derived Patient profile based on the US DAF Patient profile which defines a set of national constraints for the US Patient. Below, we can see the original DAF Patient base profile loaded in Forge. All the constraints defined by the base profile are clearly indicated by the yellow pen icon:

DAF Patient Profile

Notes:

  • The blue exclamation icon indicates a “must support” element
  • The blue paperclip icon indicates extension elements
  • The yellow pen icon indicates a constrained element

Now Mayo Clinics creates a new derived Patient profile based on the DAF Patient profile. The new derived Patient profile inherits all constraints that are defined by the base DAF Patient profile. Initially, the new derived profile does not introduce any new constraints, as is clearly visible in the Forge Element Tree:

Mayo Clinic Patient Profile

The screenshot demonstrates that the derived Patient profile indeed inherits all the constraints defined by the base DAF Patient profile (e.g. you can see the inherited extensions such as ethnicity). Because the implicitly inherited constraints are not defined by the derived profile itself, they are not considered to be part of it. This is clearly visible in the Forge element tree, as none of the elements in the new derived profile show a yellow pen icon (except on the root element, because the derived profile has a new name & url). In other words, the new derived profile shown above is (almost) empty.

Now Mayo Clinic can define additional specific constraints in the derived profile. Any newly defined constraints will actually be part of the derived profile and consequently they will be indicated by a yellow pen in Forge. For example, the screenshot below shows how the derived profile defines an additional constraint on the Patient.photo element:

Mayo Clinic Patient Profile with constraint on photo

Below you can see the serialized XML of the derived Patient profile. We can clearly see the newly introduced constraint on the Patient.photo element. Note that the root element definition is mandatory in DSTU2, even if the element is not constrained. This requirement will be deprecated in STU3. Also note the (mandatory) reference to the underlying DAF Patient base profile.

<?xml version="1.0" encoding="utf-8"?>
<StructureDefinition xmlns="http://hl7.org/fhir">
  <meta>
    <lastUpdated value="2016-12-01T16:47:22.41+01:00" />
  </meta>
  <url value="StructureDefinition/MayClinicPatientProfile" />
  <name value="Mayo Clinic Patient Profile" />
  <status value="draft" />
  <date value="2016-12-01T16:28:17+01:00" />
  <fhirVersion value="1.0.2" />
  <kind value="resource" />
  <constrainedType value="Patient" />
  <abstract value="false" />
  <base value="http://hl7.org/fhir/StructureDefinition/daf-patient" />
  <differential>
    <element>
      <path value="Patient" />
    </element>
    <element>
      <path value="Patient.photo" />
      <max value="0" />
    </element>
  </differential>
</StructureDefinition>

Conclusion

This concludes our introduction of the new support for derived profiles in Forge. The example demonstrates that the authoring process is actually quite easy for end users and quite similar to creating profiles on core resources. After all, in FHIR we try to move complexity away to the developers of tools and servers.

As simple as this all seems to the end user, the actual implementation entailed a long and winding road straight into the bowels of the FHIR specification. In a separate blog post, I will provide some more in-depth background information about the technical challenges we faced when trying to implement support for derived profiles, and how we addressed these.

If you have questions concerning (derived) profiles or need a bit of assistance, then you are strongly encouraged to join the online FHIR discussion groups. The FHIR community is very active, friendly and helpful and usually you quickly receive helpful responses from FHIR users all over the world. The conformance stream is specifically intended to discuss FHIR profiling and related tools including Forge. Also if you’ve implemented FHIR in a production environment, then please inform the FHIR community and share your story in the implementers stream!

Two final notes:

  1. Today we have published a minor hotfix release (14.4.1) with a couple of bugfixes. Should you encounter any issues while trying to repeat the exercise above, then please make sure that you have upgraded to the most recent version. Users of release 14.4 will receive an auto-update notification. Users of earlier releases should manually download and install the new version.
  2. The new Forge release does not yet support base profiles containing slices. If you select a base profile containing slices, snapshot generation will fail and generate a runtime exception. We will introduce support for sliced base profiles in a future application update.

Happy profiling!

Michel Rutten

FHIR DevDays 2016 in Amsterdam

by Martijn Harthoorn

Whenever I am at the FHIR DevDays in Amsterdam, I always get this vibrant feeling that this is where it happens. And it’s true: significant things happen. For those who could not attend, here are the highlights.

Developments in the FHIR standard

We are not that far away from publishing STU3. And STU4 is on the horizon. It will not be long until parts of the FHIR specification will go normative. But we have also seen at the DevDays that there are still many initiatives to expand the possibilities of FHIR. There are quite a number of proposals of adding domain specific languages. The most obvious one is FhirPath, which will come in STU3. Ewout has provided an extensive look into it with a Birds of a Feather session. There is now sufficient tooling and API’s available to get you up and running.

There have been discussions to extend FHIR to Swagger or RAML, for which Nicolai Ryzhikov of Health Samurai has done some excellent research (see more below). Grahame spoke about a new language, intended for mapping between different standards.  This mapping language is now part of the proposed STU3 specification. And of course, there is a new resource for it: StructureMap. Bryn Rhodes showed a very mature implementation of CQL for decision support, which could be a candidate for the FHIR specification too.

A small change with a big impact: at the DevDays we broke the data type HumanName. Family will get a cardinality of zero to one! The structure of it is now defined in extensions. This will help countries like Germany, The Netherlands and all Spanish speaking countries.

FHIR Community

Grahame as always gave a very insightful and inspirational outlook on what he thinks of the future of FHIR in the next year. We still do not seem to be on the top of the Gartner hype curve. The FHIR community now exceeds a 1000 people actively involved in developing the standard. Grahame told us that HL7 and the FHIR Foundation are currently developing a certification process for FHIR credentialed consultants. There is also an option for FHIR to go broader, since a lot of the FHIR architecture, services and tooling might have great use outside of Health.

WHO’s on FHIR?

The University of Pittsburg is on FHIR! They have started using FHIR to connect their 1000+ clinical systems. They are even discussing to make parts open source. We saw interoperability at its best with Cerner and Epic on one stage together showing how their infrastructure works together with FHIR.  IBM showed us how Watson integrates with their own FHIR server and how they map between FHIR and HealthKit. Patients Know Best have worked out and implemented a very valuable clean way to look at patient consent and are actively involved in sharing their thoughts with the developers of the standard.

Tooling

I will start with the tooling. As Alexander Henket nicely put it: traditionally in health there is a shortage of tooling. With the FHIR standard, for the first time, there is an abundance of it.

The build tool for Implementation Guides

In his talk, Lloyd told us that Implementation Guides can now be published independently of the Specification. This is a big step forward, since the ‘build’ of the spec is no small thing. It would take between 25 and 60 minutes.The new tool runs in about 60 seconds. You can find the documentation here. It’s also good news that Implementation Guides that are not maintained by HL7 can nonetheless be published at fhir.org.

HAPI

HAPI has until recently been a set of API’s that you can build a server with. At the DevDays, James Agnew launched SMILE CDR. This is HAPI server components packaged as a product, that runs on top of database like Postgres, MySQL, Oracle, SQL Server. Of  course still works with HL7v2, where HAPI since long is a big name and includes Smart on FHIR and has monitoring tools and a set of really cool management tools like SMART-on-FHIR apps registration. It’s still work in progress, but James team is pretty far. You can take a peek here.

CQL Compiler

Bryn Rhodes has worked on CQL. It’s a language expressing the logic involved in Quality measures and decision support artifacts. And CQL can now be run against FHIR resources and translates to ELM. Bryn showed us all this with the CQL runner tool.

FHIR schema

Nicolai Rhyzikov from Health Samurai showed a complete Swagger definition for the FHIR REST api. His team has also been working on profiling and validation with JSON schema. See code here. There is also an application here. There is even more. His team has been working on a Test script runner in Clojure.  Mike showed JUTE. It’s a yaml like custom domain language to map HL7v2 messages to FHIR. It has a lot of similarities with FhirPath so they might see how it can be integrated.

FHIR Client for Python and Swift

Pascal Pfiffner from Harvard Medical University showed us their new Python Client for FHIR. You can find the python code here. It has a FHIR parser and even has a good basic Validator. Pascal also showed a FHIR Swift implementation, which you can find here. It has all the FHIR data models and has included the same validation feature as their Python tooling.

Simplifier

Ardon Toonstra showed us all new features of Simplifier.net: our own FHIR registry. Ardon showed how to find profiles and extensions, but also the whole path from uploading a resource to publishing and render them as a tree table or UML to using that same resource when writing a complete Implementation Guide in the new Simplifier IG editor, with all FHIR rendering tooling available.

SPARK

Christiaan Knaap from Furore showed the evolution of SPARK that started as one of the first FHIR reference implementation servers to an enterprise grade FHIR server. SPARK has been rewritten on top of ASP.NET Core. It now consists of a several components and a custom API surface, so that it can handle many different scenario’s, like a validation only server, a front end for different databases, like MongoDb and SQL Server, a hub in an existing health infrastructure or a facade for non FHIR systems.

ART-DECOR

Alexander Henket from Nictiz showed us the FHIR Roadmap for ART-DECOR, and open source multidisciplinary tool stack for authoring and sharing of templates, valuesets and much more. The tooling now has a resource viewer, import/export of FHIR resources. It now collaborates with external FHIR repositories, for starters with Simplifier.net and has terminology services. The  plan for coming year is to include basic profile editing scenarios.

That’s all folks!
Hope to see you next year at the DevDays, 15-17 November 2017 in Amsterdam!

Technology with teeth: #FHIR in the year of Trump and Brexit

For who hasn’t the time to watch the video of Grahame Grieve’s keynote at FHIR Developer Days 2016, here is a summary of the important things he had to say.

The central theme of his address is the FHIR community. What does it mean to be part of that community, what do you get out of it, and what’s the importance of the community for the FHIR standard?

The example of the medical profession works for health IT. There isn’t a clinician who thinks he/she is a good clinician, who is not a member of a professional society and who doesn’t contribute to that society.

What do you get out of getting engaged in the FHIR community, or in any community? What’s the business model? The business model is not about the bottom line for this year, but in three years time, when you retained your best staff.

Grahame proposes the – yet rudimentary – framework of an engagement matrix, that ranks the culture of engagement of any company and helps buyers in their procurement. If you score 0 to 2 in this matrix you are entirely focused on the short term, you won’t retain the best staff and customers will be screwed over. 9 to 11 means the opposite. [After the keynote Grahame announced a more elaborate blog on this subject.]

From a more general community perspective, it’s been a bad year. Not for FHIR, but because of what happened at the ballot box. We’ve seen populations turning away from openness, from engagement, form the community. This is counter to what FHIR is all about. This mentality shift is the single biggest threat to the FHIR community.

But Grahame Grieve is confident just the same:

“We’re open not because of some philosopy or culture, but we’re doing it because it works for what we’re trying to achieve. We’re doing it because it makes sense for our goals. We’re doing it because our values lead to outcomes that other people can’t create.”

Who will benefit from FHIR? “Give the data to the patient”, that’s nice for the digital literati but not for those who are too old or cannot afford a computer. FHIR was never just about getting the patients their data. Instead, the vast majority of patients will benefit from government controled back-end integration. Don’t get too extracted on getting the patients their data.

The final topic Grahame Grieve touches in his most engaged performance I have witnessed so far, is a phenomenon he calls alt-med. This, of course, is a variant of the dubious alt-right movement that has a reputation for being post-truth or post-fact. Facts no longer matter. Alt-med means people moving away from traditional healthcare, guided by rumours and false facts in social media. Getting good clinical information into social media is going to be a big thing for FHIR in the upcoming years.

The times we live in ask for technology that is a counterbalance to modern aberations: technology with teeth.

The video of Grahame Grieve’s keynote is available on our website.

Doing #FHIR at DevDays university

The most interesting track of FHIR Developer Days 2016 is the Students Track. This is a one day track, running on Thursday 17 November, in which some 40 students from 15 universities (from Germany, France, Austria, Chile, Netherlands, Switzerland, US) get their teeth into an exercise described in a FHIR Implementation Guide.

The track starts off with a presentation about interoperability by one of the FHIR core team members. After the lecture the students engage with other students and compare solutions in a hackathon session. Yes, hackathon: DevDays is all about developers in action.

A jury of FHIR experts evaluate the solutions the students come up with at the end of the day. The winners will present their solution in front of an audience of 250 participants of DevDays.

After that, the students will join us on a social event in down-town Amsterdam. That’ll be the real test…

Custom workflow in Simplifier.net

Recently we added the custom workflow feature to Simplifier.net. The FHIR specification allows for basic workflow statuses:

  • Draft: This resource is still under development.
  • Active: This resource is ready for normal use.
  • Retired: This resource has been withdrawn or superseded and should no longer be used.

These basic categories not always fit the nuances of how people collaborate on FHIR profiles. Imagine an initial draft version of a profile, a revised version, a rejected version, a published draft version etc. etc.

This kind of refinement is precisely what HL7 Affiliates a.o. need when they work on profiles with standards professionals within their realm. For this reason we created custom statuses, that can be designed and configured by Simplifier.net users according to their own worklow. There is an unlimited number of statuses, each with its own name, description and color.

Custom workflows are created at the organizational level by the administrator of the organization account on Simplifier.net. Each project can have its own set of statuses and can be shared between projects. Workflow management in the account portal of a user looks something like this:

capture2

The below screen shot shows how we implemented the custom workflow statuses to represent the maturity level of the FHIR core resources.

capture1

Custom workflows are available in the Enterprise Plan and in the HL7 Affiliate Plan. More on the HL7 Affiliate Plan in a future blog.

Validation – and other new features in the .NET API stack

During the long and rainy summer here in Amsterdam our team has been working hard to expand the .NET API to support more advanced usecases like full profile validation. Though validation may be The Most Wanted Feature for many of you, it is certainly not the only exciting new feature we like to showcase, so I’ll give you a brief overview on what’s coming up in the newest version of the .NET API in the sections below.

Poco-less parsing

We all love the (generated) .NET classes like Patient and Observation to manipulate and create FHIR data, including the parsers and serializers that turn them into xml and json. In fact, this is what the .NET API started out with some years ago. There is a growing class of tools however, that only need a subset of information, and for which the all-or-nothing nature of working with POCOs is a hindrance. Some of the reasons you may want to start working with poco-less parsing are:

  • Performance Our generic FHIR REST server Spark does not really need to parse data into POCOs just to index and store the data into its internal format.
  • Flexibility Tools may want to work with partially incorrect or incomplete resources, which would never be parseable into the generated classes. Our profile editor Forge, for example, would gladly try to read StructureDefinitions that are incomplete or incorrect so the author can correct and save them using that tool.
  • Version independence You may like to work with FHIR data from multiple FHIR versions, and write code that deals with the differences. With the POCO classes you would commit yourself to a specific FHIR version. Using Poco-less parsing allows you to do that. This will be used by the registry tool Simplifier to support profiles in both DSTU2 and STU3.

The central abstraction for working with FHIR data without using POCOs is IElementNavigator, which includes a set of extension methods to do LINQ-to-Xml-like navigation on the data:

instance.GetChildrenByName("meta").ChildrenValues("profile").Cast()

Many of the newers parts of the .NET API are built on top of this interface and since implementing IElementNavigator is pretty straightforward, you could implement this interface top of your own (proprietary) model and have it participate in the functionality the .NET FHIR stack offers, like validation. Out of the box, we provide an implementation for POCOs, XML/Json and probably RDF.

FhirPath evaluation

In the meantime, work has been going on on a HL7-designed navigation and extraction language called FhirPath (formerly known as FluentPath). FhirPath looks a lot like XPath and can be used to walk trees, extract subtrees, formulate invariants, just like XPath. In fact, almost all XPath invariants in the spec have now been replaced by their FhirPath equivalent.

In parrallel, we have been adding a FhirPath compiler and execution environment to the API. It’s built on top of IElementNavigator described above, so you could now say:

IElementNavigator instance = XmlDomFhirNavigator.Create("...");
var names = instance.Select("Patient.name.where(first = 'Ewout')");

Since we’ve also implemented IElementNavigator on top of the POCOs, you can now quickly select any part of an in-memory instance or run FhirPath invariants on top of POCOs.

Underneath, a FhirPath compiler will turn these FhirPath statements into ordinary .NET lambdas, so if you execute the same statement multiple times, you’ll be running native lambdas, not re-interpreted strings.

Retrieving conformance resources

Most of us will sooner or later need the metadata for Resources, called the conformance resources in FHIR. These allow you to get information about which elements are members of a given resource type, which datatypes exist, cardinality of elements etcetera. The .NET API has a new abstraction (based on previous work that’s been around for a while), the IResourceResolver.

It’s main method is ResolveByCanonicalUri() and it will get you the conformance resource (StructureDefinition, ValueSet, OperationDefinition etc) with a given uri. All core datatypes have uri’s like http://hl7.org/fhir/StructureDefinition/HumanName, but these could of course also be more specific profiles, like Argonaut’s ‘DAF Patient’.

We have provided implementations for locating these resources within a zip file (like validation.zip that is available on the FHIR website) and within a directory where your application is running. Resolvers can be cached and combined, so you can have a resolver that first tries your local directory, then the standard zip, then goes out to the web.

This is how you would locate the core profiles for, say, Patient:

IResourceResolver zipResolver = ZipSource.CreateValidationSource();
StructureDefinition pat =
zipResolver.ResolveByCanonicalUri("http://hl7.org/fhir/StructureDefinition/Patient")
as StructureDefinition;

Of course, you can write your own implementations for these interfaces. We did so for the registry Simplifier, where we needed an IResourceResolver that resolves uris to resources using the database of profiles present in the registry.

These abstractions are used by the validator and the terminology modules to retrieve referenced profiles and valuesets when they are encountered.

Navigating StructureDefinition’s hierarchy

If you have worked with StructureDefinition, you know the pain of dealing with its flat list of Elements: even though the StructureDefinition expresses a (deeply) nested tree of elements, these are represented as a flat list (with hierarchical paths). You’ll generally need a lot of smart string manipulation to just find “the next sibling” for an element, or move back to its parent. To avoid this kind of code creeping all over the place in the API, we developed an internal class that we have now made public, the ElementDefinitionNavigator. It’s highly optimized for speed and low-memory use so you can now move around the hierarchy expressed by the StructureDefinition with ease:

StructureDefinition def = zipResolver.FindStructureDefinitionForCoreType(FHIRDefinedType.Patient);
ElementDefinitionNavigator nav = new ElementDefinitionNavigator(def);

nav.JumpToFirst("Patient.name");
nav.MoveToChild("family");
nav.MoveToNext();

Creating Profile snapshots

Maybe not a feature everyone will need, but if you are authoring profiles you will need at some point to turn your changes to a profile (“the differential”)  into a snapshot, that can be used as input for e.g. rendering and validation tools. This is a pretty complex job, and Grahame Grieve, Chris Grenz and our own Forge developer Michel Rutten have been busy to make sure our tools can read each others outputs. Mind you, that means long nightly sessions on the subtleties of merging differentials into bases – but Michel packed it all in the new SnapshotGenerator. It builds on the resolver and ElementDefinitionNavigator above to do its work and using it looks deceptively simple:

StructureDefinition def = myResolver.FindStructureDefinition("http://hl7.org/fhir/....");
var gen = new SnapshotGenerator(myResolver);

// Regenerate the snapshot for the StructureDefinition
gen.Update(def);

And yes, it will handle your re-sliced re-slices.

Terminology services

Terminology is a vast and complex area, and we have no intention to provide a full-fledged terminology server in the API, but there’s now at least ITerminologyServer and its lightweight, in-memory implementation LocalTerminologyServer. It supports ValueSets with define and compose statements, however it does not support filters. It’s designed to handle most common uses of ValueSet, but will happily throw NotSupportedException if you push it too far or you reach a certain maximum number of concepts in your expansion.

The LocalTerminologyServer depends on two pieces of new functionality:

  • The `ValueSet` class now has methods to search by code/system in an expansion
  • There is a new class ValueSetExpander, which would expand a valueset in-memory (with the caveats mentioned above)

Most likely we will still add another implementation called MultiStrategyTerminologyServer that would first use this local server, and when that fails, reach out to a real terminology server and use the FHIR REST Terminology operations to get what it needs.

Validation

That brings us to the most interesting addition for most people, the validator. It’s been a long-standing wish to add this to the .NET API, but thanks to support from UK’s NHS Digital, we have been able to work on this as well.

If you have been reading about the new features above, you can see these are all pillars that make validation possible:

  • A flexible poco-less way to efficiently read any kind of instance FHIR data (from file, xml, json, memory, database) – the `IElementNavigator`
  • A way to retrieve StructureDefinitions and other conformance resources to validate the instance against – the `IResourceResolver`
  • Generate snapshots for these definitions that serve as input to the validator
  • Be able to run constraints formulated in FhirPath
  • Be able to validate bindings using `ITerminologyService`

Mix these together, add some smart validation logic and you get the Validator.

It is not completely done yet -of course we left slice validation till the end- but it does most other constraints, including binding and simple extension validation. It handles Bundles and aggregation modes as well.

Learning more

While writing this blog I realized how much functionality we have added, and of course I had to leave lots of details out of the description above. Best way to get started with it is by running the validator demo and looking at the code and unit-tests. No, indeed, we have no online documentation about all of this yet. Another great way to learn more is to visit the upcoming HL7 FHIR DevDays where I will have a session on these (advanced) uses of the FHIR .NET API.

For now, let me get back to my Visual Studio and try to get it all polished and finished up for you.