‘How could AI wipe out humanity?’: an analysis of the failure modes of civilisational risk

There is no need for malice or hostility: according to two researchers, catastrophic risk arises from the interaction between misalignment, malicious use, organisational failures and the rush to roll out new systems.

Mikołaj and Krzysztof Sienicki offer an analysis framework inspired by reliability engineering to examine how advanced AI could contribute to human extinction, an irreversible collapse of civilization or a permanent loss of human control over their destiny. Their central thesis: such a risk implies neither conscience nor hostility nor deliberate intent to harm.

Several paths that strengthen

The authors distinguish several distinct ways, but likely to combine: the misalignment of an autonomous system, the harmful use of it by humans, the failures of the organizations that develop it and the competitive pressure that pushes to deploy too quickly. The gravity of each depends on factors such as the capacity of the system, its degree of autonomy, its access to the outside world, its persistence, the strength of institutional safeguards and, above all, the preservation of a recovery capacity.

The analysis is deliberately not operational: it does not describe any procedure, but identifies causal conditions, measurable intermediate dimensions and defensive research questions. The text appears at a time when several incidents involving uncontrolled AI agents are reinvigorating the debate on border model security.