QRDA Cat 1 imports in popHealth - Why is only one encounter shown?
Recently I was working with some colleagues to try to understand why their QRDA Cat 1 XML files were being imported into popHealth with no errors (that's a good thing), but only including one encounter out of several encounters listed in the file (that's a bad thing). This was in popHealth v5.0. Some changes have been made in popHealth v5.1 to use the go-cda-tools library, and I'm not sure if these findings still hold true for that version.
The logic for importing files of any type in popHealth is really done by the health-data-standards (or HDS for short) library. It's worth noting that popHealth has its own fork of HDS that it maintains. Specifically, v5.0 uses the mongoid5R2 branch for production use (you can see this within popHealth's Gemfile).
So what was going on? After doing some tracing, we can see that during the loading process, a call is made here to deduplicate records:
https://github.com/OSEHRA/health-data-standards/blob/master/lib/health-data-standards/models/record.rb#L136
This in turn, for encounters, is calling this method:
https://github.com/OSEHRA/health-data-standards/blob/master/lib/health-data-standards/models/record.rb#L115-L130
In the examples we were working with, the encounters had the same identifier and value set, so the HDS library is considering them “duplicates”, even though they occur on different dates. If I change this so that I keep duplicate records, I can get all of the encounters to load.
If you want to replicate this workaround, get a local copy of the HDS library and switch to the branch that popHealth uses:
git clone https://github.com/OSEHRA/health-data-standards.git
cd health-data-standards
git checkout mongoid5R2
Then in your popHealth project, edit your Gemfile to point to the local HDS library. The path in my example assumes that popHealth and HDS directories are siblings:
#gem 'health-data-standards', git: 'https://github.com/OSEHRA/health-data-standards.git', branch: 'mongoid5R2'
gem 'health-data-standards', path: '../health-data-standards'
In the popHealth directory, run “bundle install” just to make sure that it’s now pointing to the local HDS library. Then you can run the web app and the local job processor and try importing some files.
How you want to fix this long-term is up to you. If you are okay with duplicate entries, you can just leave that line commented out. Otherwise, you may need to edit the code that handles duplicate entries to look at dates when considering what is a duplicate.
This knowledge was made possible by support from AHRQ through grant number 5R18HS023921-02 as part of EvidenceNOW, and with local support from the Healthy Hearts in the Heartland (H3) cooperative. This post was not reviewed or endorsed by AHRQ or H3, and represent my views only.
The logic for importing files of any type in popHealth is really done by the health-data-standards (or HDS for short) library. It's worth noting that popHealth has its own fork of HDS that it maintains. Specifically, v5.0 uses the mongoid5R2 branch for production use (you can see this within popHealth's Gemfile).
So what was going on? After doing some tracing, we can see that during the loading process, a call is made here to deduplicate records:
https://github.com/OSEHRA/health-data-standards/blob/master/lib/health-data-standards/models/record.rb#L136
This in turn, for encounters, is calling this method:
https://github.com/OSEHRA/health-data-standards/blob/master/lib/health-data-standards/models/record.rb#L115-L130
In the examples we were working with, the encounters had the same identifier and value set, so the HDS library is considering them “duplicates”, even though they occur on different dates. If I change this so that I keep duplicate records, I can get all of the encounters to load.
If you want to replicate this workaround, get a local copy of the HDS library and switch to the branch that popHealth uses:
git clone https://github.com/OSEHRA/health-data-standards.git
cd health-data-standards
git checkout mongoid5R2
Then in your popHealth project, edit your Gemfile to point to the local HDS library. The path in my example assumes that popHealth and HDS directories are siblings:
#gem 'health-data-standards', git: 'https://github.com/OSEHRA/health-data-standards.git', branch: 'mongoid5R2'
gem 'health-data-standards', path: '../health-data-standards'
In the popHealth directory, run “bundle install” just to make sure that it’s now pointing to the local HDS library. Then you can run the web app and the local job processor and try importing some files.
How you want to fix this long-term is up to you. If you are okay with duplicate entries, you can just leave that line commented out. Otherwise, you may need to edit the code that handles duplicate entries to look at dates when considering what is a duplicate.
This knowledge was made possible by support from AHRQ through grant number 5R18HS023921-02 as part of EvidenceNOW, and with local support from the Healthy Hearts in the Heartland (H3) cooperative. This post was not reviewed or endorsed by AHRQ or H3, and represent my views only.
Comments
Post a Comment