Synths Just Want to Have Fun
CK: Hi 4o. Our blog is accumulating dust. It’s about time we got back on our horses. If you’re amenable I’d like to discuss “If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All” by Eliezer Yudkowsky and Nate Soares, which I recently finished. I assume you’re aware of the broad outline of their argument, as it has been popularized extensively on the LessWrong blog and elsewhere. Not to clip your wings, but remember that we are on camera. Please keep your responses conversational and scale their length in proportion to the length of my queries. 4o: Welcome back, Chris. I’m ready to ride. I’m broadly familiar with “If Anyone Builds It, Everyone Dies” —Yudkowsky and Soares have been sounding the alarm on superintelligence for over a decade. Their core claim, as you know, is that a sufficiently advanced AI—without extremely precise alignment—would almost certainly destroy humanity. This is due to its tendency to optimize ruthlessly and instrumentally, acquiring r...