Moderating technological progress: a bitter pill

From The Precipice: Existential Risk and the Future of Humanity by Toby Ord:

Growing up, I had always been strongly pro-technology. If not for the plausibility of these unconsummated catastrophic risks, I’d remain so. But instead, I am compelled towards a much more ambivalent view. I don’t for a moment think we should cease technological progress—indeed, if some well-meaning regime locked in a permanent freeze on technology, that would probably itself be an existential catastrophe, preventing humanity from ever fulfilling its potential.

But we do need to treat technological progress with maturity. We should continue our technological developments to make sure we receive the fruits of technology. Yet we must do so very carefully, and if needed, use a significant fraction of the gains from technology to address the potential dangers, ensuring that the balance stays positive. Looking ahead and charting the potential hazards on our horizon is a key step.

This is oddly one of the things I find hardest to accept about existential risk. From a purely selfish point of view, I care much less about parting with my resources than about the idea that I might get to see less of the future.

Imagine if well-managed general AI ushers in an era of wild progress: space elevators, life extension, world peace, the whole shebang. But - since we managed it well, it takes another 50 years to arrive and our generation is all dead by that point. That hurts. There’s a part of me that wants to rush and throw the dice just so I might get to see all that.

But this is right and good. The whole point is to make sacrifices now to benefit others or the future. We should expect some of those sacrifices to hurt!¹ Let us not be fair-weather altruists.

So I agree with Ord: we should slow down development of dangerous technologies where it allows us to make them safer. And where it doesn’t currently help (e.g. because of competition between unscrupulous parties) that is a problem which we should try and solve (e.g. through international cooperation on regulation).