Why Does AI Need Auditors Like Finance?

Demis Hassabis thinks AI needs third party auditors or evaluators, on the model of auditors in the finance industry, because the investment going into AI should fund a healthy ecosystem of startups and companies with enough revenue to do the work. Which third parties come in is still an open question. Leading labs generally agree on bringing auditors in, and he can see the ecosystem forming. Accenture is one of the big players moving in, and other leaders are working with the firm. He thinks Accenture is a little ahead of the game because the company acquired a good startup.

He thinks an industry-run standards body, with some kind of government oversight, is going to be needed as well. Ideally the body is US led and then becomes international. An essay he wrote over the summer set out the idea. The essay did have an impact in the White House, in the administration, and with colleagues. He thinks some of the energy from the essay became the accord, the famous letter everyone signed. International institutions are very weak just when they need to be strongest, so the work has to start somewhere.

The letter is better than nothing and a start, and he does not think it goes far enough. It acknowledges from all of the main players that the issues are there and need to be addressed. It does not say too much about what to do about them, and that has to come next. He thinks the letter gives air cover for teams to work together on benchmarks, best practices on safety, and analysis of what can happen with the systems.