How Does Box Test AI on Real Work?

Aaron Levie says Box's Complex Work eval is a document-centered test of domain-specific work in life sciences, financial services, the public sector, and tech. Five documents and a set of problems are given, and the test asks what the answers would be.

Box runs hundreds of tests on every model and keeps two holdback evaluations. The Complex Work eval is one of them, and every model is scored on it with Box's agent. The second uses Box's own instance and how Box employees use their data. A couple of personal use cases are tracked on the side. A move of half a point registers as model improvement.

Default models roll out from cost and accuracy thresholds. Customers can also choose any model from the model garden.