Research with EHR data - Pitfalls

The rule used to decide who counts as exposed. Too broad or too narrow, and it misclassifies who actually got the exposure.

The list of codes standing in for one clinical idea, often built from a parent code plus its descendants. Pulling descendants in bulk without reviewing them sweeps in codes that do not belong.

Translating codes from one coding system into a different standard vocabulary; distinct clinical meanings can collapse into one broader code.

Whether a code existed and was in use for the whole study period. A code introduced partway through can't capture anything before it existed.

Different sites coding the same clinical event differently. Pooling sites without accounting for this can mistake coding differences for differences in disease.

Bias from how the cohort was assembled: among the people selected, the exposure-outcome relationship differs from the one in the population they stand in for. It arises when entry depends on something tied to both the exposure and the outcome, such as reaching a particular care setting.

Who or what the exposed group is measured against. An unrelated or non-comparable comparator can't isolate the exposure's effect.

Observation begins only when a person or site enters the data source, not when the clinical story started. Anyone whose event happened before that date never appears at all, and for those who do appear, earlier history is missing.

Care received outside the data source, such as a different health system or unlinked records, that the database simply can't see.

Outcomes are detected and coded more often in people with more healthcare contact. When one group is watched more closely than the other, it looks like more disease when it may be only more looking.

Formally checking how well a code-based definition identifies true cases (sensitivity, specificity, PPV) against a reference standard.

A methodological threshold or rule the study states it will use but never actually applies to its own results.

Whether a cited reference actually supports the specific claim it's attached to.

The point where follow-up starts, which should also be where eligibility is met and treatment is assigned. Anchoring the two arms to different kinds of events makes them non-comparable from the first day.

A stretch of follow-up counted as exposed even though anyone who had the outcome during it could never have been classified as exposed. Because events in that window cannot land in the exposed group, the treatment looks more protective than it is.

The time each person is actually under observation and at risk, which is the denominator of an event rate. Reducing years of follow-up to a yes/no lets someone followed six months count the same as someone followed five years.

Follow-up ending before the outcome is seen, whether the person leaves the system, switches care, or the study period closes. Ignoring it treats "unobserved" as "unaffected."

The outcome, or its early signs, actually preceded and possibly caused the exposure, rather than the other way around.

A variable that affects both the exposure and the outcome but is not captured in the data, so no amount of adjustment can remove its effect on the estimate.

Treatment is chosen because of the patient's underlying condition or prognosis, so the treated differ from the untreated before anyone is treated. Matching on demographics does not touch it, because the reason for treatment is itself a risk factor for the outcome.

An association reported without adjusting for confounders: the raw, unadjusted comparison.

Wording that asserts X causes Y when the design and its stated assumptions cannot support it. Observational data can support causal claims, but only when the design is built for that and the assumptions are named.

Whether a patient actually took or used a treatment as prescribed. Assuming a prescription equals full adherence overstates true exposure.