Sal Candido

Latent Space: The AI Engineer Podcast · 🔬 Why AlphaFold Didn't Solve Protein Folding — Pushmeet Kohli…

• Why Should a Biology Problem Come Before the Model?

  Pushmeet Kohli says the problem comes first, because treating yourself as only a modeler or only a data person can make the work fail. Be flexible about the solution. Do the modeling when the problem needs modeling, and collect data when the problem needs data.

  Machine learning often showed up with a training set and a test set already fixed, then only optimized the model. That approach is broken when the goal is to solve the problem. What goes into the work is data, modeling, and expertise.

  AlphaFold used the datasets that already existed. The team did not have the expertise or the resources to enlarge the PDB by a significant order of magnitude, and the investment already in that data wa

  ▶ at 5:25 Share Transcript

• What Is the Bitter Lesson for Biological Data?

  Sal Candido says there is a bitter lesson for data, but scaling laws are not everywhere and they do not always exist. Brandon Anderson asked the question after defining the bitter lesson as the claim that methods which scale eventually win. A lot of the work is finding the situation where more compute and more data actually produce a better result. Once that happens, the rest can become an engineering problem.

  The result still depends on the data having the right information for the problem. Without that, more scale does not yield a model with the capabilities or the understanding you want. You can only pull the information out of the data you have, then use it to generalize beyond that dat

  ▶ at 0:42 Share Transcript

• Why Does Biohub Chase a 10x Approach?

  Sal Candido says Biohub chases a 10x approach because the mission is to cure all disease, and a 10x goal shows solutions a 10% goal leaves hidden.

  One thing he learned a long time ago in his career, back at Google, is that it is sometimes easier to approach a problem by asking what it will take to make a 10x improvement rather than a 10% improvement. The path is not necessarily easier. A 10x goal lets you take a broader view, see solutions you have not been approaching, and go back to first principles to ask how the work will really get done. Both the 10% approach and the 10x approach are valuable, and both should be done.

  He is happy that Biohub has a lofty mission statement to cure all d

  ▶ at 30:03 Share Transcript

• Why Does AlphaFold Need a Calibrated Confidence Score?

  Pushmeet Kohli says AlphaFold 2 needs a calibrated uncertainty measure, because an uncalibrated PLDDT score would present a very, very confident answer that turns out completely wrong after the next one year of work. AlphaFold is not perfect. AlphaFold 2 was 90 GDT at that time on that set. Even if it reached 95 GDT, an uncalibrated PLDDT score would leave the result untrusted. The model would magically give good answers, then call the answer very, very confident.

  How AlphaFold 2 behaves is different from how it found the solution. Interpretability, the notion raised with Sal Candido, asks why it worked and why it gave an answer. At the highest level AlphaFold 2 was interpretable through an

  ▶ at 24:09 Share Transcript

• Did AlphaFold Solve Protein Folding?

  Pushmeet Kohli says the protein folding problem has been solved only at a conceptual level. There have been some advances, and a protein's shape might change depending on the context. Science operates by isolating one thing and then making progress step by step.

  Proteins are not blocks, and they do not act as blocks. He says he calls proteins the building blocks all the time, and he does not believe it. Proteins are extremely complex and disordered. He and John Jumper used to ask what they were trying to solve. The true ground state proteins take is unknown, and so is the actual distribution of structure proteins take.

  What the work did was take a structure someone deposited in the PDB and

  ▶ at 15:29 Share Transcript