Harrison Chase

Harrison Chase says AI bills rise because agents burn far more tokens, even while frontier models get cheaper on price for performance. Cost is one of the main things companies ask about. Frontier models are relatively cheaper over time than what existed a year ago, but agents talking to agents, and multi-agent systems, burn so many more tokens that the bill still goes up. Coding costs have skyrocketed, and many organizations, including LangChain, are looking for ways to pull those costs back.

Coding agents get three controls. First, show admins what a run will cost and what it has cost. Second, cap a task, such as getting a reservation without spending more than $5. Third, change the harne

Harrison Chase says the agent harness should stay neutral, so the memory and context an agent builds are not locked to one model lab. A neutral harness holds state that accumulates before the agent runs and while it runs, including short-term and long-term memory. Some of the learning that makes an agent better over time lands in the model weights. A lot of it stays outside the model, in that context. If the context is tied to one model provider, you cannot switch when a better model appears, and you cannot leave if that provider raises rates.

The harness can be tied to one model or kept neutral. Right now the context is tied to the harness, and the harness connects the model to data, APIs,

Harrison Chase says taking money from Benchmark changes a company's trajectory completely, and a discount is absolutely worth it for Sequoia and for Benchmark. On other tier one firms, a discount is only probably worth it, and Sequoia and Benchmark are the firms he has actually worked with.

The Benchmark partnership invested when the company was two people, and he treated the firm like a third co-founder. The team is now about 400 people, and Benchmark has been the most consistent partner on the journey from two people to that size. He is not a venture investor and has never done venture capital. Other market dynamics probably matter as well, because a strong Benchmark round sends a signal.

Harrison Chase does not think fear of open source Chinese models is justified. Some enterprises are fearful of working with open source Chinese models. Others do not care.

The concerns are not the same. One is that the data a model runs on will somehow go back to the Chinese labs. Using Fireworks or hosting the model yourself makes that worry ridiculous. A different fear is that the stuff the models were trained on will cause them to respond in a way that is not consistent with the values of the enterprise. Evals are where that second fear gets checked.

The fear is largely unfounded, and he calls it probably largely unfounded overall. On the response issue he is less sure: for some domains

Harrison Chase says they spend roughly 30% of engineering salary on coding tools. The share has gone down over the last two months, while the number of tokens used has gone up.

They did it in three steps. First, they tracked all coding agent usage in Langsmith so they could tell who was spending how much, what they were spending it on, and run analytics. Second, they set a budget. If someone hits the budget, they can talk to the VP of engineering, who almost always increases it. The budget is a safeguard against agents used incorrectly or stuck in a bad loop. Third, they control the harness. They built an in-house cloud coding agent called OpenSWE. People can still use whatever they want fo

Harrison Chase says most apps probably need a vertical agent of their own. The app already knows how a person uses it and can learn that pattern better than a general horizontal agent. In the consumer space both kinds of agents will exist, and the horizontal agent will talk to the vertical one.

Music use on Spotify is very different from use on Strava, so the apps are actually probably best at learning the person, and an agent is probably the best way to expose what they learned. Spotify also has a ton of APIs. A better experience is an agent that wraps the APIs and offers one natural language interface for a horizontal agent to call.

When agents interact, the number of requests goes way u

Harrison Chase says the hardest part of AI memory is the intelligence that decides what to keep, not the database. A context layer for agents is an agent wrapped around a database, and the data market is still early. ClickHouse is doing incredibly well and stays largely focused on a traditional database, storing and querying data in traditional ways. ClickHouse existed before AI. ClickHouse, Mongo, and Elastic are great for data that software systems produce. Ingesting learnings still needs intelligence, and that intelligence is really hard. A number of companies store memory in a single file. The hard part is taking observations from a user in traditional software, or from a user with an ag

Harrison Chase says a founder should start a vertical agent at the harness layer, doing context engineering before any post-training of the model, because domain experts who can verify the work capture the value.

People say agent value sits in verification, whether the agent did the job well, because verification unlocks the next step, automation. The winners are the domain experts who can quantify what the value means, which is very hard. Building evals is a very hard job, so a lot of the value will accrue to whoever can write the verifiers and capture the domain knowledge. Vertical agents for different domains follow from that, because the domain experts best suited to write the verifiers

Harrison Chase says the hardest part of building agents is getting the system to behave reliably. Models are non-deterministic black boxes. They take natural language in and send natural language out, and the inside of the box is non-deterministic too.

Someone in front of a chatbot can type anything, so the input space is infinite and it is hard to know what will arrive. Agents are a series of model calls interacting with tools, not a single model call. Small black boxes sit inside a large system, and errors cascade. Two similar inputs can go in and arrive at completely different answers. The model is very smart, but builders do not know exactly what it will do, and it changes answers after

Harrison Chase says it is absolutely worth taking a discount for Sequoia and Benchmark. LangChain spends probably roughly 30% of engineering salary on coding tools. Over the last two months the amount spent on coding tools has gone down. Across the same two months the number of tokens used has gone up.

He really underestimated the demand for inference. Everything just works. The market is very large, and there is a lot of demand for inference. He does not know whether demand is going to get any less anytime soon.

The two-month result is more tokens and a smaller coding-tool bill as a share of engineering salary. Demand for inference is still very large, and a discount for Sequoia and Bench

Chase believes that we are very malleable creatures and we can adopt. As a photographer he used to travel a lot, and learned to understand that if he wanted to continue working that life, he'd have to find ways to quickly adopt to new places.