Grading rubric
What earns an A, and what earns an F.
Every grade on a Data Foundation Report Card comes from this rubric. Each system the use case depends on is graded A to F on five dimensions, and every grade cites the evidence behind it: a profiling result, a document, or an interview.
WhyWe publish it so you can check our work. You know before we start what a D means, and you can hold every grade in the read-out to the same standard.
- A
Ready
Supports the use case as it stands.
- B
Ready, small gaps
Supports it, with known fixes that don't hold up the build.
- C
Remediate
Good enough to pilot, not to scale. Fix before production.
- D
Blocks
Blocks the use case on the path it sits on.
- F
Blocks, foundational
Blocks it, and the fix is foundational work, not a patch.
Dimension 1
Data quality
Are the values right, and does anyone measure whether they are?
WhyA model learns whatever errors the data holds, at scale. Measuring quality is what turns a hidden pattern into a fixable one.
Evidence we use
- Profiling real extracts: null rates, duplicates, invalid values and broken keys on every field the use case uses
- The checks that already run, and who is alerted when they fail
- Interviews with the people who correct the data by hand
From a sample read-out · PolicyCore
Duplicate policy records for 6% of insureds since the last platform migration.
- A
Automated checks on every field the use case depends on. Failures alert a named owner, and error rates are tracked over time.
- B
Checks on most critical fields. Profiling finds a few known gaps, small enough to clean in the pipeline.
- C
Some checks, but not on all the fields the use case needs. Profiling finds errors a model would learn from, still fixable at the source.
- D
No routine measurement. Profiling finds structural problems on the path: duplicate records, missing keys, values that contradict each other.
- F
Nobody measures, and the fields the use case needs are too unreliable to train on or score with.
Dimension 2
Documentation
Is what each field means written down?
WhyIf no one can say what a field means, no one can map it to what a model expects, and training labels quietly disagree.
Evidence we use
- Reading the dictionaries, glossaries and specs that exist
- Asking each team to define the same core terms, then comparing answers
- Checking coded fields against their documented values
From a sample read-out · FNOL Intake
No data dictionary. Field meanings are known only to the original vendor team.
- A
A data dictionary and business glossary cover the use case's fields, each with an owner, and teams use the same definitions.
- B
Most fields are defined and owned. A few definitions are out of date.
- C
Key terms are defined, but some definitions differ between teams or live in more than one place.
- D
Documentation is old or scattered across spreadsheets and wikis, and core terms mean different things to different teams.
- F
No data dictionary. What fields mean is known only to a few people or to the original vendor.
Dimension 3
Lineage
Can each number be traced to its source, and each change traced to what it affects?
WhyRegulators and auditors ask where every model input came from. In insurance and healthcare, the answer decides whether a model can be used at all.
Evidence we use
- Tracing sample figures from a report back to the source system, step by step
- Checking whether each table keeps history or overwrites it
- Interviews with the people who build and run the pipelines
From a sample read-out · PolicyCore
Endorsement history is overwritten nightly; prior policy states can't be recovered.
- A
Every input the use case needs traces to its source system, and history is kept, so you can say what the data looked like on any past date.
- B
The main flows are documented and traceable. History is kept for the tables the use case needs.
- C
Flows can be traced by hand, with effort and the right experts. Some history is overwritten off the use case's path.
- D
Figures can't be traced past a warehouse load, or history on the path is overwritten.
- F
No one can say where key inputs come from, and past states can't be recovered.
Dimension 4
Access and security
Can the right people reach the data, quickly and safely?
WhyAI teams that wait months for data never get past the pilot. Teams that get it fast without controls on personal and medical fields create a bigger problem.
Evidence we use
- How long our own access request took, measured from the day we asked
- Field-level classification of personal, claimant and medical data
- The approval path, and how access is reviewed and removed
From a sample read-out · ClaimsCore
Role-based access in place; analyst requests approved in about a week.
- A
A standard path to approved access in days. Sensitive fields are classified, masked where they aren't needed, and access is reviewed.
- B
Role-based access in place, with approvals in about a week. Sensitive fields are classified.
- C
Access is granted case by case and takes weeks, or sensitive fields are only partly classified.
- D
Access takes months, or personal and medical fields on the path have no field-level classification.
- F
Both: access runs request by request through one person, and sensitive fields are unclassified.
Dimension 5
AI consumability
Can a model read the data at the moment it needs to decide?
WhyData can be accurate, documented and traceable and still be unusable, if it arrives too late or in a form a model can't read.
Evidence we use
- When each input is available, compared with when the model has to decide
- The share of key inputs held as free text, PDFs or images
- Whether the input existed at decision time, or was filled in later
From a sample read-out · FNOL Intake
Loss descriptions arrive as free text and scanned PDFs. A triage model has nothing structured to read at first notice.
- A
Governed, structured feeds available when the model decides, at the speed the use case needs.
- B
Structured and close to ready. A new extract or interface on existing data closes the gap.
- C
Structured, but on the wrong schedule (a daily batch where the model needs hourly), or some inputs need light parsing.
- D
Weekly batch only, or key inputs sit in free text a model would have to interpret.
- F
Key inputs exist only as free text and scanned documents, or aren't captured at the moment of decision at all.
From grades to verdict
The verdict follows the grades on the use case's path.
Every system is graded on all five dimensions, but not every grade decides the pilot. A fraud model depends on provider data being one identity per provider; a triage model depends on intake data it can read at first notice. We name the dimensions the use case depends on at the start, and weight each system's overall grade toward them.
WhyAn average hides the one F that kills the pilot. Weighting to the use case puts that F at the top of the fix list, and a D on a system the use case never touches can wait.
Go
When
No D or F on the dimensions the use case depends on, in any system on its path.
What you do
Fund the pilot as planned. The C and B items go on the roadmap.
Fix, then go
When
A D or F on the path, with fixes that can be named, owned and finished before the build needs them.
What you do
Fund those fixes first, in the order on the ranked fix list, then the pilot.
No-go
When
An F on the path that no fix closes in the time the use case allows, or data the use case needs that doesn't exist.
What you do
Keep the budget. The read-out says why, and which use case the data can support instead.
Profiled, not surveyed.
Grades come from real extracts and the people who run each system, not from a questionnaire about how things should work.
Evidence beside every grade.
Each grade on the Report Card carries the finding behind it, so anyone can challenge it with better evidence.
Same rubric, every client.
A C means the same thing for a carrier as for a health plan, so grades compare across systems and over time.
See the rubric applied to your pilot.
Map the systems your AI use case depends on in five minutes, free. Or see every grade above applied to a fictional carrier in the sample read-out.