Smash Notes

Back to 20VC: Horizontal vs Vertical Agents: Who Wins | Model Routing: Commodity or Increasing in Value | The Future of the Inference Market | Why Founders Should Take Discounts for Tier 1 Firms with Harrison Chase, LangChain, with Harrison Chase.

What Is the Hardest Part of Building AI Agents?

Harrison Chase says the hardest part of building agents is getting the system to behave reliably. Models are non-deterministic black boxes. They take natural language in and send natural language out, and the inside of the box is non-deterministic too.

Someone in front of a chatbot can type anything, so the input space is infinite and it is hard to know what will arrive. Agents are a series of model calls interacting with tools, not a single model call. Small black boxes sit inside a large system, and errors cascade. Two similar inputs can go in and arrive at completely different answers. The model is very smart, but builders do not know exactly what it will do, and it changes answers after small changes to the input. The system ends up non-robust.

Agents today are a lot better than agents three years ago or four years ago. The models have gotten better, and builders have figured out how to harness them. LangChain focuses on that harness.

Grokipedia·Harrison Chase

Transcript

20VC: Horizontal vs Vertical Agents: Who Wins | Model Routing: Commodity or Increasing in Value | The Future of the Inference Market | Why Founders Should Take Discounts for Tier 1 Firms with Harrison Chase, LangChain ·

The Twenty Minute VC: Venture Capital | Startup Funding | The Pitch

More from this episode

  1. Should I Take a Sequoia and Benchmark Discount?Harrison Chase says it is absolutely worth taking a discount for Sequoia and Benchmark.
  2. Will Model Labs Eat the Agent Harness?Harrison Chase says the agent harness should stay neutral, so the memory and context an agent builds are not locked to one model lab.
  3. Why Do Domain Experts Win With Vertical Agents?Harrison Chase says a founder should start a vertical agent at the harness layer, doing context engineering before any post-training of the model, because…
  4. Why Hasn't Anyone Solved AI Memory?Harrison Chase says the hardest part of AI memory is the intelligence that decides what to keep, not the database.
  5. What Kind of Agent Do Most Apps Need?Harrison Chase says most apps probably need a vertical agent of their own.
  6. Why Are AI Bills Rising Even as Models Get Cheaper?Harrison Chase says AI bills rise because agents burn far more tokens, even while frontier models get cheaper on price for performance.
  7. What Share of Engineering Salary Goes to Coding Tools?Harrison Chase says they spend roughly 30% of engineering salary on coding tools.
  8. Should Companies Fear Open Source Chinese Models?Harrison Chase does not think fear of open source Chinese models is justified. Some enterprises are fearful of working with open source Chinese models.
  9. Should Founders Discount for Sequoia and Benchmark?Harrison Chase says taking money from Benchmark changes a company's trajectory completely, and a discount is absolutely worth it for Sequoia and for Benchmark.

People

More answers like this

More with Harry Stebbings

Related episodes

Plays from this note through the rest of the episode.

Note

Weekly highlights

One letter a week.

The notes worth hearing, with the minute they were said.