Why Do Document Mistakes Compound Across a Dozen Files?
Adit Abraham says a loan packet is more than reading the page. The job splits the files, extracts individual fields, and may edit the application at the end, and each of those steps has to be accurate. Across a dozen files, 90 percent on the individual pages is not acceptable, because those mistakes compound.
Jennifer Li says a lot of traditional OCR still lands around 70 to 80 percent, short of the high 90s. For parsing itself, Adit says the vast majority of use cases can already reach the high 90s, aside from corner cases such as handwriting he cannot read. The newer work is a harness around the model. Deep Extract is where he finds an agent can beat humans at structured extraction on many tests, because a human-made answer key drops rows or fills in only part of the page. The harness extracts, then reviews and corrects its own output.
The Missing Infrastructure for AI Agents: Documents | Reducto’s Adit Abraham ·
From the transcript
What are you shooting for? Like, how do you think about that trade-off of still having human in the loop versus you can actually use agent to replace a whole sort of document processing pipeline? Yeah, and this actually goes to the work that we're doing beyond parsing. I think parsing is like this core building block of how do you even understand what was on the page? And for the vast majority of use cases, we can actually be in the high 90s today.
There are corner cases, like there's doctor's handwriting that I can't read, and so I don't even know if we did a good job. But if you think of an end-to-end task, like you're taking in a loan packet, it's not just parsing, you're splitting the contents, you're extracting individual fields, maybe you're editing a loan application form at the end.