This is being thrown around a lot, including the dreaded "10% chance by the end of this decade". Every conversation I have seen about this, mostly on twitter, not once have I seen anyone connecting the dots on concretely and plausibly how this might pan out. It is usually 1. Train powerful AI 2. ??? 3. Extinction
I personally find it implausible, but some quite smart people have talked about it so I am also not dismissing it as pure fear-mongering.
The reasons I do not think it is plausible:
1. Limited real world projection: an exceptionally smart AI is less powerful than a dumb person with the capability of shutting off AI's power
2. Confinements: The AI 'escaping' incidents that got media attention were really the failure of infra, probably due to the scenarios being novel. But it is probably a lot better PR to say 'our model escaped' than 'our infra was poorly thought out'. A fully confined model has a significantly less blast radius (claude ruined this term)
3. The game theoretic aspect: 'If all else stays the same and AI kept getting powerful, scenario X will materialize' is tempting to think, but all else never stays the same. Especially with AI, there has already been enough fear and paranoia spread that people are unwilling to give an un-sandboxed AI control of a laptop, let along more consequential outcomes. And the already spread AI FUD almost guarantees that the outcomes where AI controls substantial parts of the infra/economy autonomously are very unlikely to materialize, at least in short/medium term.
I would love to hear competing hypotheses where an AI can cause something resembling extinction and how.
The attention economy rewards and amplifies a buffoon class. These people are fit for nothing at predicting the weather tomorrow morning or how many people a new strain of covid is going to kill day after, but they have convinced you that extinction is going to happen. Not because its going to happen but because fear of death gets the most attention of the chimp troupe. Try making a story where no one dies and see how many people pay attention to you.
The whole problem with predictions is time scale. Over a long enough time scale we may see very many scenarios of attrition… only localized on some level and easily dismissed by the thriving adapting whole.
If we were to say, how might AI be an existential threat to our humanity?
If we let our strengths atrophy and fade away through generations of complicity which at some critical failure point leaves us over leveraged (cannot feed or climate control at scale) or worse we all become mutual enemies, the death of civilizing law over unmitigated conflicts of interest.
When does too big to fail clip itself on the tool designed to make it all easier to?
If you’re looking for a movie, make it about ai dating app chat bots bilk us for our insatiable desire for others company when we are not satisfied with our own.
I personally find it implausible, but some quite smart people have talked about it so I am also not dismissing it as pure fear-mongering.
The reasons I do not think it is plausible:
1. Limited real world projection: an exceptionally smart AI is less powerful than a dumb person with the capability of shutting off AI's power
2. Confinements: The AI 'escaping' incidents that got media attention were really the failure of infra, probably due to the scenarios being novel. But it is probably a lot better PR to say 'our model escaped' than 'our infra was poorly thought out'. A fully confined model has a significantly less blast radius (claude ruined this term)
3. The game theoretic aspect: 'If all else stays the same and AI kept getting powerful, scenario X will materialize' is tempting to think, but all else never stays the same. Especially with AI, there has already been enough fear and paranoia spread that people are unwilling to give an un-sandboxed AI control of a laptop, let along more consequential outcomes. And the already spread AI FUD almost guarantees that the outcomes where AI controls substantial parts of the infra/economy autonomously are very unlikely to materialize, at least in short/medium term.
I would love to hear competing hypotheses where an AI can cause something resembling extinction and how.
The attention economy rewards and amplifies a buffoon class. These people are fit for nothing at predicting the weather tomorrow morning or how many people a new strain of covid is going to kill day after, but they have convinced you that extinction is going to happen. Not because its going to happen but because fear of death gets the most attention of the chimp troupe. Try making a story where no one dies and see how many people pay attention to you.
Every time a new technology appears a group says it will end us all - https://www.newyorker.com/culture/infinite-scroll/the-age-ol...
The whole problem with predictions is time scale. Over a long enough time scale we may see very many scenarios of attrition… only localized on some level and easily dismissed by the thriving adapting whole.
If we were to say, how might AI be an existential threat to our humanity?
If we let our strengths atrophy and fade away through generations of complicity which at some critical failure point leaves us over leveraged (cannot feed or climate control at scale) or worse we all become mutual enemies, the death of civilizing law over unmitigated conflicts of interest.
When does too big to fail clip itself on the tool designed to make it all easier to?
If you’re looking for a movie, make it about ai dating app chat bots bilk us for our insatiable desire for others company when we are not satisfied with our own.