Smash Notes

Back to The $2B AI Startup Simulating Human Behavior with 85% Accuracy, with Joon Sung Park.

What Was the Smallville AI Experiment?

Joon Sung Park ran Smallville in 2022 and 2023 to see whether a language model could simulate a whole person, then a whole town.

The models were GPT-3 and then GPT-3.5. They were still rough, but training on the web, Wikipedia, and social media made it possible to pull out human behavior and generate new actions, not only write text or classify a task. Smallville was a game town in the style of Pokemon and The Sims, filled with 25 agents. Each agent had a name, a life story, and an architecture for memory, planning, and reflection. They woke up, went to work, and formed relationships. The simulated day was the day before Valentine's Day. One character organized a party, and others asked each other out. None of that was scripted. It came out of the simulation.

That 2023 experiment introduced simulations of people and a core architecture for agents built on generative models. It was early for agent workflows. The same idea has since been pushed toward simulations that are more accurate, more diverse, and larger.

Transcript

The $2B AI Startup Simulating Human Behavior with 85% Accuracy ·

From the transcript

And the part that's always complex and chaotic is what actually gets born in their interactions. So that's ultimately, I think, what building simulation here in the societal scale really is going to look like. And so this all started with Smallville? It did. OK, explain what that is and your whole background behind that. For sure. So Smallville was an experiment that we ran back in 2023 and 2022, where we had the initial observation that these large language models, initially we were using GPT-3 and then GPT-3.5, the models were still fairly janky.

But you could see a glimpse of opportunity where these models were trained on such broad human data. that when you poke at the right angle, you can actually extract a lot of human data that appeared in its training data. This might be from the web, Wikipedia, some of the social media. And what we really start to ask ourselves is, can we actually use this to create an interaction paradigm that is truly new, that goes beyond just simply generating context or classifying different tasks? But can we actually generate truly new behaviors? And we decided that the most ambitious version of this is actually to create a simulation of entire person.

See in transcript

Sourcery

More from this episode

  1. Why Does Joon Sung Park Think We Live in a Simulation?Joon Sung Park thinks we live in a simulation because any civilization will recreate itself to understand itself.
  2. How Did Simile's Digital Twins Reach 85% Accuracy?Joon Sung Park says agents built as digital twins of real people predicted behavior with 85% or better accuracy by the end of 2024.
  3. When Is a Simile Simulation Confident Enough to Act on?Joon Sung Park says an AI simulation of a person is strong enough for a very high-stakes decision when confidence is higher than 90 percent, and usually when…
  4. Why Did Winning Electric Vehicle Messaging Lower Sales?Joon Sung Park says a message that looked like a clear winner for a new electric vehicle also shifted how people saw non-electric vehicles and lowered sales…
  5. Are Universal Basic Income Simulations Worth Running?Joon Sung Park calls universal basic income simulations good ones to run.
  6. Why Does Predicting Behavior Fail to Show What to Do?Joon Sung Park says a forecast is not what people came for.
  7. Is AI Behavior Simulation in Its GPT-4 Era?Joon Sung Park says AI behavior simulation is in its GPT-4 and GPT-4.5 era.
  8. Why Doesn't a Buyout Offer End Simile?Joon Sung Park says a $15 billion offer to sell Simile does not end the work when he has every reason to believe an ambitious idea exists that only Simile can…

People

More answers like this

Related episodes

Plays from this note through the rest of the episode.

Weekly highlights

One letter a week.

The notes worth hearing, with the minute they were said.