Smash Notes

Back to The Missing Infrastructure for AI Agents: Documents | Reducto’s Adit Abraham, with Adit Abraham & Jennifer Li.

Can Agents Do Healthcare Work Without Reliable Document Parsing?

Jennifer Li asks how accuracy changes whether customers will trust an agent with their documents. Adit Abraham says a document error in healthcare is not a small miss. One large customer gives doctors a search tool over physical packets that are inches thick. A misread can make a history look absent, and an overworked doctor who trusts the tool may then treat the patient as someone who maybe does not need the drug. If an agent made that decision, everything after it would depend on inputs you can trust.

On those harder packets he still sees cases below 50 percent accuracy, where the best the tool can do is approximate where information sits and a person has to look. One patient's packet can run to tens of thousands of pages, more than a doctor can re-read. He also cites Medicare prior authorization: millions of claims are rejected every year, and in 80 percent of appeals the original decision was overturned. He reads that as inefficiency in the first decision, and he sees the same kind of paper bottleneck in immigration files at USCIS.

Transcript

The Missing Infrastructure for AI Agents: Documents | Reducto’s Adit Abraham ·

From the transcript

Let's talk more about the consequences of accuracy and how does that impact your customers trusting, I guess, using agent and AI to process the documents. Yeah. I remember we have this customer that works in healthcare. They're a really large provider of what's effectively a search tool for doctors to be able to capture information. And they work with some of the harder situations in healthcare to the point where these are physical documents.

They will have packets that are inches in height. And the weird thing, like, It's hard to appreciate how much an error actually matters to the individual. Where we were talking about individual cases where like, because the initial pipeline read something as chest pain instead of chest pain, somebody's claim gets rejected because the doctor searches for, do they have a history of this? And if they don't, it's like maybe they don't need the drug.

See in transcript

a16z

More from this episode

  1. How Does Reducto Read a Messy Document?Adit Abraham breaks document reading into a few epochs, ending with Reducto R1, one unified model that reads a page from end to end.
  2. Why Do Document Mistakes Compound Across a Dozen Files?Adit Abraham says a loan packet is more than reading the page.
  3. Why Did Coding Tools Reach a Billion in Revenue First?Adit Abraham says coding tools reached a billion dollars in annual revenue while professional-services AI has not, even though healthcare, insurance, legal…
  4. Why Won't the Big Model Labs Solve Document Parsing?Jennifer Li asks why the big model labs, already focused on knowledge work and on vision, will not simply solve document parsing.
  5. Do Frontier Models Silently Drop Facts From Long Documents?Adit Abraham says frontier models were more precise than many specialized document parsers on a long-extraction benchmark: when they pulled a fact, it was…
  6. What Should You Check First in a Document Benchmark?Adit Abraham says the first thing to check is whether the distribution of data matches the work you want done.

People

More answers like this

The other side

Related episodes

Plays from this note through the rest of the episode.

Weekly highlights

One letter a week.

The notes worth hearing, with the minute they were said.