A new study graded an AI comorbidity reader against billing codes. Coders get graded against the note.

On July 8, 2026, the International Journal of Obesity published a validation study that ran a large language model across 350 hospital discharge summaries and asked it one question per chart: is this comorbidity present in the note? The model did well. What matters for anyone who codes for a living is what the researchers scored it against.

They scored it against the billing codes.

What the study actually measured

The researchers pulled 350 discharge summaries covering 341 adult patients with obesity out of MIMIC-IV, a public database of de-identified hospital records. The study reported a mean patient age of 59.7 years and a cohort that was 60.4% female. For each summary, the model marked four comorbidities present or absent: type 2 diabetes, hypertension, hyperlipidemia, and obstructive sleep apnea.

The primary reference standard was the administrative ICD-9 and ICD-10 codes already attached to those admissions. A secondary sensitivity analysis compared the model against a manual review of all 350 summaries.

That design choice is the story. When the paper says the model agreed with the reference standard, it is mostly saying the model agreed with what somebody already billed.

The per-condition numbers, and what they mean at the desk

Prevalence and accuracy below are as the study reported them. The right-hand column is our own arithmetic on the study's manual-review accuracy figures, turning each one into how often a human reviewer landed somewhere different from the model.

Hyperlipidemia is the outlier the abstract lets you skim past. A 0.863 accuracy sounds close to the others until you read it as roughly one disagreement in every seven summaries. The three scores above 0.96 carry their own trap, which is that a number that high is the one nobody goes back and checks.

Agreeing with a billing code is not the same as supporting it

The study set out to detect, in its own words, "the presence" of a condition "documented within discharge summaries." Presence is a real question, and the model answered it well. The question a coder has to answer sits one layer down.

A records-based reviewer does not ask whether a diagnosis appears somewhere in the chart. They ask what the encounter itself supports. None of that was tested here.

A tool that reliably reproduces the codes on a claim will reliably reproduce the weak ones too. If a diagnosis was carried forward without support and then billed, a model trained to agree with that claim has no reason to flag it. Our explainer on what counts as supporting evidence walks through the difference between a diagnosis appearing in a chart and a diagnosis a reviewer will accept.

The setting is probably not your desk

MIMIC-IV is an inpatient database, and a discharge summary is a dense, end-of-stay document. It is a different animal from the progress note and the problem list that most outpatient risk-adjustment work runs on. The cohort here was patients with obesity, which shapes how common each of these four conditions was in the sample.

None of that transfers automatically to your charts. Treat the numbers as evidence about one model, on one document type, in one dataset. Vendors will quote the top of that F1 range. Ask them what document type it came from.

Where a human coder is still required

Three places, concretely.

The encounter test comes first. A model that reads a whole chart will surface conditions the provider never touched at the visit you are coding. Confirming that is a person's job, and it is the one an extraction tool structurally cannot do for you, because it is a judgment about what the provider did rather than about what the text says.

Specificity comes second. Presence is binary. A code isn't. A note can say a condition exists and still miss the specificity the code demands, and then it isn't codeable.

The disagreement case comes third. When the tool and the note point different directions, somebody has to decide, and the decision has to be written down. Our guide to MEAT criteria is the standard most coders already apply here, and it is the standard to apply to a machine-generated list too.

What the authors say about using this today

The paper does not oversell itself. Its conclusion states that "further research, including validation against clinical expert review and investigation into reasons for discrepancies, is necessary before considering practical applications."

That is the researchers, not us. It is worth holding onto the next time a demo puts a comorbidity list on the screen and calls it finished. The list is a place to start looking, and you can check the code path once you have confirmed the documentation is actually there.

For the adjacent problem of AI-written notes that look complete but are not, see our explainer on why AI-generated notes still need coder QA, and our earlier coverage of how AI documentation tools are moving coding intensity.