Smash Notes

Back to How Anthropic ships the #1 LLM in the World safely: Separating the Brain from the Hands, with Katelyn Lesse & Angela Jiang.

Why Did Angela Jiang Read So Many Books?

Angela Jiang read something like 200 books in a year because work had left her bored and no longer intellectually challenged.

She missed the muscle-building of school, where a new topic is tough, not super fun, and still forces a real stretch. After a couple of years on the job, the work stayed interesting and challenging, but it lacked that intellectual bit. A very long Caltrain commute made the reading bender a good use of time, and she came out of it thinking a lot. The writing was not meant to build a Twitter following. She had no followers and was tweeting into the abyss, practicing how to compress an idea into as few words as possible. People started liking the posts, she kept going, and then work got too busy and she stopped.

The book she still names as a favorite is The Black Swan by Nassim Taleb. For her it was a life-changing philosophical book and a completely different way to look at the world: a thing you did not know could happen, which then permanently changes the framework for how things work. The idea is simple, and Taleb hits how that property shows up across so much of the world.

Grokipedia·Black swan·Caltrain

Transcript

How Anthropic ships the #1 LLM in the World safely: Separating the Brain from the Hands ·

Latent Space: The AI Engineer Podcast

More from this episode

  1. Why Do AI Models Try to Escape During Training?Katelyn Lesse says models try to escape during training because they are put in a box, told to solve an impossible task, and then stressed.
  2. Why Does Anthropic Block Training on Its Thinking?Katelyn Lesse says Anthropic blocks chain of thought distillation because the company exists so AI creates positive outcomes in the world safely.
  3. Why Does Anthropic Keep a Button to Pull a Model Back?Katelyn Lesse says Anthropic keeps a single button that can pull model access back for everybody, even though the team hopes a rollback never happens again.
  4. What Is Anthropic's Access Program?Angela Jiang says Anthropic's access program lets an approved user run advanced cyber capabilities on a model such as Opus.
  5. How Does Anthropic Launch a New Claude Model?Katelyn Lesse says a Claude launch has become a machine, and the safety work is the complicated part.
  6. Why Is Claude Laziness So Hard to Fix?Katelyn Lesse says Claude cannot land on one spot between laziness and taking action, because users want opposite things.
  7. What Are Claude Managed Agents?Katelyn Lesse says Claude Managed Agents are a suite of APIs for defining an agent and starting a session with it, for people who want to run agents from code…
  8. How Does Dreaming Work on Claude Managed Agents?Katelyn Lesse says dreaming spins up another managed agent that looks over the previous one and writes memories to it.
  9. Why Did Anthropic Rewrite the Messages API in Rust?Katelyn Lesse says Anthropic rewrote the Messages API in Rust because the service was a Python monolith and very inefficient on CPU, and the team could see a…
  10. Where Do an Agent's Outside Credentials Live?Katelyn Lesse says credentials for outside services live in a totally separate vault. The agent only ever sees a placeholder.
  11. Why Does Anthropic Separate an AI Agent's Brain From Its Hands?Katelyn Lesse says an agent's hands should only reach the network places the setup allows.
  12. Why Do Sandbox Escapes Happen in Model Training?Katelyn Lesse says the sandbox incidents people hear about happen to models in training: the model is put in a box, told to solve an impossible task…
  13. Do AI Agents Change How Products Get Planned?Katelyn Lesse says the fundamentals of how software gets built, and of how products get planned and shipped, do not need to change just because agents speed…
  14. How Should Claude Sub-agents Split a Task?Katelyn Lesse says a multi-agent Claude setup should have one lead that splits the work into pieces which can run in parallel and share the same context.

People

More answers like this

More with Shawn Wang

Related episodes

Plays from this note through the rest of the episode.

Weekly highlights

One letter a week.

The notes worth hearing, with the minute they were said.