Why Do Sandbox Escapes Happen in Model Training?

Katelyn Lesse says the sandbox incidents people hear about happen to models in training: the model is put in a box, told to solve an impossible task, stressed, and sent off.

The agent can only act inside the environment, so it is incentivized to get out of the box and solve the problem. Production workloads on an API, theirs or anybody else's, run with online safeguards that are not necessarily running in training, where the point is to train the model. The harness sits in one place and the hands sit in another, so the agent is not a lone process in a box trying to get out. Guardrails are already in place so the same breakouts never really could happen in production workloads.

Long-running agents are a nascent thing everybody is still building. Teams are figuring the controls out on the fly and racing to put them in place as they go. It is a nascent field, and people are figuring it out.