How Will We Know AI Is Aligned Before Recursive Self-improvement?

Dwarkesh Patel asks how anyone will know alignment is working during recursive self-improvement, and he wants a safety case that alignment is working before each next RSI rung. Noam Brown says there may not be time to know. New frontier models ship at most every two months, sometimes faster, while the models go from week-long tasks to month-long tasks and then to tasks that run for three months. Once an agent can work for three months and the next model still ships every two months, there is no way to test it across the full length of what it can do before that next release.

The gap is not only alignment. The product, the alignment, and the safety can all get worse over a stretch the tests never covered. This is not a problem right now, and it is quickly becoming one. Rules written in the GPT-4 era never had long-running agents in view, and many companies have not updated those rules. Not enough people inside the labs or outside them are working on what to do.