Skip to content
Insights

P&C Insurance · Claims AI

Why FNOL triage pilots die

The model is rarely the problem. The data at first notice of loss usually is, and you can see it coming before you fund the pilot.

Emerald Isle Consulting · 4 min readPublished

FNOL triage model · accuracy

Illustrative pilot
IN PRODUCTION100%80%60%92% demo58% liveShelvedWeek 0Week 8Week 12Week 19Week 26

In 30 seconds

  • Triage decides at first notice, on the least structured data in a claim's whole life.
  • Demos are scored on closed, adjuster-coded claims, so their accuracy doesn't survive go-live.
  • The failure usually shows up as two grades on the intake system: documentation and AI consumability.
  • Fix structured loss capture, an intake data dictionary and a shared claim key before you build.

The pattern is familiar to anyone who has sat on a claims AI steering committee. A vendor or internal team builds a model to triage first notices of loss: flag the complex claims, fast-track the simple ones, route the likely litigation to the right adjuster. The demo is excellent. Funding follows. A few months after go-live, adjusters have stopped looking at the scores, and the project is quietly shelved.

It is a common story. Capgemini's 2026 research on P&C insurers found two numbers that belong together:

60%

of P&C insurers are still stuck in AI pilot or proof-of-concept mode

12%

report high maturity in AI data readiness

They're related. Most pilots that never ship don't fail on the model. They fail on the data underneath it.

What a triage model actually needs

Triage happens at first notice, which is the whole point: the earlier a complex claim reaches the right handler, the more it saves. That means the model has to work with what exists at the moment the claim is reported, not what exists six weeks later.

Here is the problem in one table.

Loss type and cause

At first notice: Free text from the caller

In the historical data: Coded by the adjuster

Loss location

At first notice: Often missing or free text

In the historical data: Verified and geocoded

Injuries and parties

At first notice: Partial, unconfirmed

In the historical data: Confirmed, structured

Coverage in force

At first notice: Weekly batch, may be stale

In the historical data: Verified by the adjuster

Vehicle or property details

At first notice: Scanned PDFs, photos

In the historical data: Estimate entered

Why the demo lies

Demos are trained and tested on historical claims. Historical claims are clean in exactly the way first notices are not: by the time a claim is closed, an adjuster has coded the cause of loss, confirmed the parties, verified coverage, and entered the estimate. The model learns from all of it.

So the demo is quietly answering an easier question than the one production asks. It isn't predicting how a claim will develop from a phone call. It is predicting how a claim developed, using details that were written down weeks after the call. Accuracy in the 90s is real, and also irrelevant.

A triage model trained on closed claims is being tested on information it will never see at first notice.

What production looks like

On go-live, the model meets the real intake stream:

Notes typed under pressure

Call-center notes and web forms where the loss description is a paragraph of free text.

Scanned PDFs

Agent submissions arrive as images, with nothing a model can read as fields.

No shared claim key

Call-center and web notices land in separate tables that don't join.

Weekly coverage

Coverage comes from a weekly batch, so recently changed policies look wrong.

None of that is unusual. It is just what first-notice data looks like at most carriers. But a model trained on structured, adjuster-coded fields has almost nothing it recognizes, and accuracy slides accordingly.

The grades that predict it

When we look at the systems behind a use case, we grade each one on five dimensions: data quality, documentation, lineage, access and security, and AI consumability. For FNOL triage, two of those decide the outcome before a line of model code is written.

Documentation

If nobody can say what each intake field means, nobody can map it to what the model expects. An F here usually means the field meanings live with the original vendor team.

AI consumability

Whether a model can read the data at the moment it needs to. Free-text loss descriptions and scanned PDFs at first notice are an F, however good the historical data looks.

F

In our sample report card for a fictional carrier, FNOL Intake scores F on both. That's where the triage pilot dies. See the report card →

What to do instead

  1. 1

    Grade the systems before you fund the model.

    Score every system the use case depends on. It takes weeks, and it tells you whether the pilot can work at all before the budget is spent.

  2. 2

    Test only on what exists at first notice.

    Rebuild the training and test sets from data as it looked at the moment of reporting. If accuracy holds, you have a real model. If it collapses, you've learned that cheaply.

  3. 3

    Fix the narrow path, not the whole estate.

    Structured loss capture at intake, extraction from documents, a shared claim key across channels, and current coverage lookup. That's the path triage needs; the rest of the roadmap can wait.

  4. 4

    Write the accuracy target down before you build.

    Agree what "good enough" means, measured on first-notice data, before work starts. It turns a demo into a commitment.

None of this is exotic. It is simply doing the data work before the model work, in the order the use case needs it. The carriers that fix the data first will ship AI first.