David Silver

Training Data · David Silver and Lasse Espeholt of Ineffable Intelligence: A…

• Why Did Ineffable Keep the Team Small?

  Ineffable has hired 20 people and is keeping the team small on purpose. Ineffable is not in a hurry to build a giant organization, because a small number of incredibly talented people can do an awful lot now.

  Hiring looks for four things together: exceptional talent, kind and collaborative people, alignment with the mission, and the right set of skills. Standards and expectations for the people who come in are very high. The small headcount also keeps compute per person high. One person can run a lot of experiments, and the group is not fighting for resources from day one.

  ▶ at 1:01:07 Share Transcript

• How Did Lasse Espeholt's DeepMind Agent Get So Popular?

  Lasse Espeholt built a reinforcement learning agent at DeepMind that utilized GPUs and scaled linearly with compute. It became the most popular reinforcement learning agent at the company.

  When he joined the deep learning team at DeepMind, most people were running single-machine CPU reinforcement learning, and the scale was quite small. With colleagues he made an agent that really utilized GPUs. Lasse Espeholt says it scaled linearly with compute. He thinks that came in part because the agent was very efficient, very good at scaling, and very easy to extend and modify. At one point most of the team used the agent or derivatives of it.

  After Lasse Espeholt left DeepMind to found the Google

  ▶ at 58:16 Share Transcript

• Why Does Ineffable Pace the Real World but Rush Simulation?

  Sonya Huang asked about pacing the frontier. One panelist is very much supportive of pacing what happens at the frontier of artificial intelligence in the real world, and of moving as fast as possible in simulation.

  In simulation, the science of superintelligence can be developed, along with solutions that help the real world go well and lead to human flourishing. Simulation work should happen as fast as possible so that, when events really happen in the real world, human flourishing has the best possible opportunity. Accelerating artificial intelligence until superintelligence arrives will lead to a really great future for humanity. Things are moving really quickly, and the speed is not so

  ▶ at 51:03 Share Transcript

• Why Will Simulation Build Superintelligence Faster?

  Sonya Huang asked where the ceiling on simulation sits and when an agent should enter the real world. Sonya Huang said Rich Sutton is very dismissive of simulation. One panelist says the right question is whether a superintelligence can be built in simulation. If a superintelligence can be built in simulation, one panelist believes it will happen 10 times or a hundred times faster, because simulation can be infinitely parallelized.

  MetNet is the example. A weather forecasting model from the Google Brain Amsterdam office, built together with Google Research, used machine learning to predict short-term weather forecasts. MetNet was way better than the physics-based simulations. The physics si

  ▶ at 38:13 Share Transcript

• Why Does David Silver Believe a Super Learner Will Work?

  David Silver strongly believes the approach behind a super learner will be feasible at some point. The human brain, he says, is the only existence proof of something that has discovered a very general intelligence, works in an embodied world, and learns how to do all kinds of different things.

  Sonya Huang asks how confident he is that a super learner will emerge from Ineffable at the broadest level, and whether the nearer goals are a 50-50 bet too. He says great moments in science come down to timing. Whether the approach succeeds or fails is about timing, in the same way a startup has to be at the right time.

  ▶ at 33:57 Share Transcript

• How Fast Did AlphaZero Overtake Two Years of AlphaGo?

  David Silver says the moment the team let go of human data, AlphaZero outperformed two years of human-in-the-loop AlphaGo work in a few hours. Giving up human data is what let the algorithm discover for itself and shoot past the competition.

  David Silver says the team spent about two years building AlphaGo. They put in a lot of human data and stayed in the loop, taking the previous generation and choosing how to improve the next one. It took two years to get to AlphaGo. When AlphaZero was right, they gave up on the human data and focused on an algorithm that could discover for itself. Once the algorithm was right, it took a few hours to outperform the two years of work done with a human in

  ▶ at 30:15 Share Transcript

• Was AlphaGo Zero a Coin Flip for David Silver?

  David Silver says he was about 50-50 on whether removing human priors would produce even smarter intelligence. He likes research questions that sit on a knife edge. Work where he is already 95% certain of the answer feels boring, because good research asks something he is really not sure about in advance.

  He was not confident, and seeing the result was exciting. Silver says they are making a bet: go further by giving up the priors everyone else relies on today, the human data and the whole internet of knowledge other systems are built on. The choice is a short-term sacrifice. They give up a certain amount of performance to reach something potentially far greater.

  For Silver, a super learne

  ▶ at 27:31 Share Transcript

• Why Did a Second Agent Make Go Endless?

  David Silver says a game of Go with one agent is the most boring game you could imagine, and a second agent turns the board into emergent complexity with an endless learning curve. One agent only places stones until the grid is full, and there is no strategy whatsoever. The moment a second agent enters, one agent has to reason about all the possible behaviors of the other, and the complexity of the environment rises sharply.

  Go is a 19 by 19 grid where the aim is to maximize territory. David Silver says that, to the best of their knowledge, the compute they sank into Go while they were working on it still left an endless learning curve. They could continue to discover new things in the game

  ▶ at 24:21 Share Transcript

• What Is Ineffable's Super Learner?

  David Silver says a super learner has incredible capacity in depth and in breadth. Depth means a system that starts from not knowing anything and discovers things that go way beyond what humans know. People worked out languages, technology, science, and mathematics for themselves. Nobody handed down a blueprint, and nobody handed down a trillion element data set. He says nobody has cracked that in AI yet, and a system that keeps learning and discovering knowledge forever is the essence of super intelligence. Breadth means you can drop the learner into any stream of experience, a simulated world, a digital world, or a physical world, and it keeps learning.

  Sonya Huang asks what first contact

  ▶ at 9:50 Share Transcript

• What Surprised David Silver About Tiny Extra Data?

  David Silver says the right approach to solve AI is learning from experience, through systems that discover things for themselves. The systems learn through reinforcement learning and get better and better, and he says that path leads to the emergence of intelligence and ultimately superintelligence. He showed the approach in the game of Go, and the conviction has held throughout his career. He is now applying the same conviction at Ineffable, trying to show that superintelligence can emerge all the way through systems that learn from experience.

  Ineffable is not pursuing the human data path that the LLM world has pursued, and David Silver still takes lessons from it. A system trained on a

  ▶ at 2:03 Share Transcript