How Does Claude Catch Jailbreaks With Probes?

Thariq Shihipar says Claude catches jailbreaks by reading internal activations, because looking only at the input is not enough. The check happens at inference time.

Cost and speed are the trade-off. The check has to run fast on every request to Claude and to Fable, and it adds overhead. A classifier runs after the probes as the fallback. Probes can be refined live when feedback arrives, and the probe can be adjusted. Training the behavior into the model is the alternative, and the training still happens. The model will refuse a request. A refusal is not a fallback and is not a probe activating and falling back. The model is simply refusing to do the request. A few failure modes remain. The model can do something as a side effect, and the side effect is not part of the final output. People try to jailbreak models and steer them off course, and probes help catch the steering. Some training still happens, but refusals that are too strong cut the request off earlier in the pipeline.

Probes are a form of mechanistic interpretability. The check has to happen fast and at scale.