What Did Warp First Pay Per Pull Request?

Zach Lloyd says the factory approach at Warp first cost about 80 dollars per pull request. The team was using Opus on work that did not need it. The triage agent ran a high-powered model, and the agent setup spawned too many sub-agents instead of fewer agents, so a lot of the work was redundant.

They ran internal tuning and testing with the built-in features and brought the cost per pull request down to about 20 dollars or 15 dollars. Over the last month, pull request throughput went up 40 percent. They also track the revert rate, and it has not gone up.

He puts his confidence in observe, measure, tune. Watch how the agents spend money, change which models run and how many agents fire, then check cost and the revert rate before treating the factory as cheaper.