In 30 seconds
- Fraud models compare providers with their peers, so they need one identity per clinician.
- Live claims split one clinician across several NPIs, TINs and addresses; training cases were stitched together by investigators.
- Adjustments that overwrite original claim lines hide what was actually billed.
- Grade provider data before funding the model; resolving identities is usually the first fix.
Payment integrity teams know the arc. A model is trained to spot fraud, waste and abuse: billing patterns that don't fit a provider's peers, services that cluster in suspicious ways, networks of providers that refer to each other more than they should. Trained on closed investigations, it scores beautifully in the steering committee demo. In production, investigators drown in false positives, the real schemes slip through, and within a few quarters the scores are ignored.
The usual post-mortem blames the model. Usually the model was fine. It was trained on data that had been cleaned by investigators, and deployed on data that hadn't.
What a fraud model actually needs
FWA detection is a comparison problem. To call a provider unusual, the model has to see everything that provider billed, compare it with true peers, and judge it against what was covered and authorized at the time. Every one of those depends on the data agreeing about who the provider is and what really happened.
Here is the gap in one table.
| What the model uses | In live claims | In the training cases |
|---|---|---|
| Provider identity | Several NPIs, TINs and addresses | Stitched together by investigators |
| Specialty and peer group | Taxonomy codes mapped three ways | Confirmed for the case |
| What was billed | Adjustments overwrite the original line | Reconstructed in the case file |
| Member eligibility | Monthly snapshot | Verified for the date of service |
| Prior authorization | Matched to claims by hand, if at all | Pulled into the case file |
Provider identity
In live claims: Several NPIs, TINs and addresses
In the training cases: Stitched together by investigators
Specialty and peer group
In live claims: Taxonomy codes mapped three ways
In the training cases: Confirmed for the case
What was billed
In live claims: Adjustments overwrite the original line
In the training cases: Reconstructed in the case file
Member eligibility
In live claims: Monthly snapshot
In the training cases: Verified for the date of service
Prior authorization
In live claims: Matched to claims by hand, if at all
In the training cases: Pulled into the case file
Why the demo lies
Closed investigations are the obvious training set: they come with labels. But by the time a case is closed, the special investigations unit has done the hardest data work by hand. They have worked out that three NPIs, two tax IDs and four addresses belong to one clinician. They have rebuilt the claim lines as originally billed. They know which peer group the provider really belongs to.
So the model learns to recognize schemes in data where one provider looks like one provider. Live claims don't look like that, and the demo never tested whether the model could cope.
A fraud model trained on investigated cases is being tested on a provider network it has never seen assembled.
What production looks like
On go-live, the model meets provider data as it really is:
Several identities
The same clinician bills under an individual NPI, a group NPI and more than one tax ID.
Addresses that don't verify
Directory addresses are stale, so locations can't be used to link records.
Specialty mapped three ways
Each source feed maps taxonomy codes differently, so peer comparisons put providers in the wrong groups.
Claim lines overwritten
Adjustments replace the original line, so the model can't see what was first billed.
The result cuts both ways, and investigators pay for both:
False alarms
Unremarkable billing gets compared with the wrong peers and flagged. Investigators spend their week on it.
Missed schemes
Genuinely suspicious billing is split across records that never meet, so no single record looks unusual.
The grades that predict it
We grade every system behind a use case on five dimensions: data quality, documentation, lineage, access and security, and AI consumability. For FWA detection, provider data decides the outcome before a line of model code is written.
AI consumability: F
Whether a model can read a provider as one entity at the moment it scores a claim. If identity is only resolved inside case files, it's an F, however good the investigated data looks.
Documentation and lineage: D
If taxonomy codes mean different things in different feeds, and credentialing updates arrive by spreadsheet with no trace to their source, nobody can build or defend a peer group.
In our sample report card for a fictional health plan, the Provider Directory scores F on data quality and lineage, and D on documentation and AI consumability. The same split and stale provider records would sink a fraud model, which can't see the networks it was built to find. See the report card →
What to do instead
- 1
Grade the systems before you fund the model.
Score provider data, claims, eligibility and authorizations against what the use case needs. It takes weeks, and it tells you whether the model can work at all before the budget is spent.
- 2
Resolve provider identity first.
One record per provider across NPI, TIN and address, with specialty mapped one way. It is the narrow path FWA detection depends on, and it pays off far beyond fraud.
- 3
Keep claims as billed, and eligibility as of the day.
Preserve original claim lines through adjustments, and reconstruct eligibility for the date of service. The model has to see what the plan saw when it paid.
- 4
Write the precision target down, sized to your investigators.
Agree how many alerts the team can work, and how many must be right, before work starts. It turns a demo into a commitment investigators can trust.
None of this is exotic. It is the data work investigators already do by hand, done once and upstream, before the model is built. The plans that fix provider data first will find the fraud their models were bought to find.

