Why Does AlphaFold Need a Calibrated Confidence Score?

Pushmeet Kohli says AlphaFold 2 needs a calibrated uncertainty measure, because an uncalibrated PLDDT score would present a very, very confident answer that turns out completely wrong after the next one year of work. AlphaFold is not perfect. AlphaFold 2 was 90 GDT at that time on that set. Even if it reached 95 GDT, an uncalibrated PLDDT score would leave the result untrusted. The model would magically give good answers, then call the answer very, very confident.

How AlphaFold 2 behaves is different from how it found the solution. Interpretability, the notion raised with Sal Candido, asks why it worked and why it gave an answer. At the highest level AlphaFold 2 was interpretable through an ability to generalize that went undiscovered before launch. Once the weights were available, people found it was a great disordered protein predictor. It figures out which elements of the protein are disordered, so it generalizes.

If interpretable means a human rational system with the cognitive and computational limitations of the human brain, AlphaFold 2 is not interpretable. For a much larger and more sophisticated model, Kohli leaves the point open: maybe it is, and the users just do not get it. Users still need the behavioral characterization, what AlphaFold 2 is able to do and what it is not able to do. Without the strengths and the limitations, the models harm rather than help.