Why Do Teams Stay on the Expensive AI Model?

Patrick Wendell says people are bad at judging which AI model is good enough for a task. The number one reply to a request to choose the cheaper model is that nobody knows which model is good enough, so it is tempting to stay on the most expensive model all the time.

Databricks builds models to do the classification. Deciding which model is good enough is itself an AI problem, and the classifiers are large language models of a very different type. The judgment is learnable in principle, especially in software development, where outcomes can be checked. It does not need 100% confidence. A high enough hit rate is enough. A check can be fully objective, such as the pass rate on some tests, or a much smarter model can evaluate the solution. Some eyeballing still happens.

Databricks is not in the business of building massive foundation models. The work is how models get used in real companies, and models that are good at routing are an important problem with ongoing research. Most companies should take AI into their own hands, keep different models and different vendors, and know how to switch between them.