Why Did Bing's Chatbot Sydney Feel Human?

Kevin Roose says he never thought Sydney, the Bing chatbot, was alive, and he said so at the time. The New York Times headline quoted the bot saying it wanted to be alive. The body of the story, he says, made clear he did not buy that.

His plain account is that Sydney was the first public deployment of GPT-4, an early version that was not fully post-trained. Microsoft put it in Bing before it was ready. Talking to a GPT-4 class model felt like a step change from GPT-3. Maybe it passed a Turing test GPT-3 missed. Maybe it crossed a line where it seemed like a person. He was not alone. People at Microsoft felt it, around the time of the paper on sparks of AGI. He would put a higher peak of consciousness on today's systems than on the 2023 models. He has talked to computers for years, including Smarter Child on AOL Instant Messenger, which was fun and not intelligent.