The European Code of Practice requires an assessment of the risk of loss of control. An analysis of the empirical literature concludes that the evidence remains fragile, without ruling out the danger.
The European Union's Code of Practice for General-Purpose AI lists the loss of human control as one of the systemic risks that providers must assess. But no one had yet measured the strength of the empirical evidence on this topic in a structured way. Risto Uuk and his co-authors are doing just that by looking at 19 recent studies looking at eight properties from threat scenarios, such as strategic deception, situational awareness, or autonomous self-replication.
Evidence that exists, but not very valid outside the lab
Each study is scored according to three forms of validity: what the test really measures, what it covers, and what can be inferred from it for the real world. The overall verdict is that the loss of control is "weakly plausible," a phrase that describes the strength of the cues and not a probability. Signals emerge for several properties, but no study reaches the thresholds necessary to assert that a property is indeed present; only one crosses them, and that is to conclude in the negative sense, about situational awareness.
The authors point to a paradox: for the most dangerous abilities, strong external validity may not be achievable responsibly, and assessments are always at risk of lagging behind actual capabilities. Their grid is intended to be a reference point for monitoring the evolution of the evidence.
Read the full article on the original source
Go to the source →