Multi-source Clinical Data Reconciliation
A research team pulling from an EHR warehouse of hundreds of tables, registries and claims extracts spends 3 to 4 months getting from the pull to an analysable dataset. Race and ethnicity categories do not match across sources, single pulls run 12 hours, and cohorts are refined mid-project.
Once the extracts land, teams need them reconciled into one documented dataset, the joins, mappings and exclusions on record, and an analysis whose steps a reviewer can see, so the clinical discussion starts from interpretable results.
Done by hand, reconciliation consumes the analysts' time, the mapping rules live in notebooks, and the results arrive as raw output that still needs interpreting.
One documented dataset in a day, not months
Extracts up to 20 MB per file and 30 sheets per workbook are reconciled into a single analysis dataset, with every join, mapping and exclusion applied as a counted, approved operation.
Category maps on record
Race, ethnicity and site categories that differ across sources are mapped by recorded rules, with values outside the mapping passed through unchanged and flagged.
Interpretable results, not raw output
The analysis is planned, run and independently validated, and the report carries the methods, the key numbers with intervals and a plain-words section.
How it works
Gather the extracts: warehouse pulls, registry extracts and claims files as they come, screened for identifier classes on upload.
Reconcile and document: joins, code mappings and exclusions are proposed as operations with counts and approved at the cleaning gate.
Analyse with review: the plan stops at review, objections are answered, and the analysis runs with every step and output visible.
Interpret with the team: the report explains what was found and what it means, alongside the formal methods and results.