What Is the Bitter Lesson for Biological Data?

Sal Candido says there is a bitter lesson for data, but scaling laws are not everywhere and they do not always exist. Brandon Anderson asked the question after defining the bitter lesson as the claim that methods which scale eventually win. A lot of the work is finding the situation where more compute and more data actually produce a better result. Once that happens, the rest can become an engineering problem.

The result still depends on the data having the right information for the problem. Without that, more scale does not yield a model with the capabilities or the understanding you want. You can only pull the information out of the data you have, then use it to generalize beyond that data.