Why Does a Ten Percent Chance of Catastrophe Make Kevin an Optimist in San Francisco?
Kevin usually puts the probability that it all goes super bad at about ten percent. In most of the world that sounds crazy. In the city he is describing, the same ten percent makes him an optimist, because he knows people at fifty, sixty, and seventy percent, and someone at ninety-nine point nine percent. He says people still have agency, are not in the recursive self-improvement loop yet, and still could slow things down, pass some regulations, and coordinate to pace the frontier.
Larry Ellison’s Media Ambitions & Kevin Roose on AI’s Biggest Gamble ·
From the transcript
I'm, I usually say I'm at about ten percent, which... Ten
percent chance that it all goes super bad.
Yeah, which sounds crazy to people, in most parts of the world. but in San Francisco, actually, that makes you an optimist.
Like, I know people walking around with fifty, sixty, seventy percent P dooms. I know someone who has a ninety-nine point nine percent P doom. And is that
person putting money in their 401k or why are they even in San Francisco? Why don't they just head out for New Zealand or wherever they think they can survive?
I, I, look, I think I do know people on the extremes who are, you know, liquidating their 401ks. I know, a person who took up smoking cause they thought AI is going to kill us all, so we might as well have some fun on the way out. I know someone who doesn't wear sunscreen to the beach anymore, because it's just kind of like, why bother? So I say that because I think, we still have some agency and some control over this.