AI consciousness: depending on the assumptions made, the same evaluation framework yields a probability of either 0.01 or 0.8

A team comprising Anil Seth, Murray Shanahan and Marcus Hutter proposes five levels of description to contextualise theories of consciousness, and then evaluates current models. The outcome depends largely on the premises.

The question of the consciousness of AI systems suffers from a theoretical bottleneck: competing frameworks speak little to each other and misjudge each other. A broad collective — including Anil Seth, Murray Shanahan, Chris Frith, and Marcus Hutter — proposes to circumvent the deepest metaphysical disagreement by separating the "hard problem" from the problem of correspondence: if we grant that experience depends on the organization of a system, the treatable question becomes that of the grain of description on which this dependence is located.

The authors extend the three levels of Marr's analysis into a five-level hierarchy — behavioral, computational, intrinsic causal-structural, organismic, and organism-environment — and then position the major theories of consciousness according to the level they deem decisive. For each level, they derive measurable indicators and confront current systems, before aggregating them into a Bayesian model combining theoretical credenzas and observed indices.

A result that depends on the premises as much as on the evidence

This is the most honest conclusion of the work: in the illustrative evaluations, the verdict on the major current language models is determined as much by the distribution of the theoretical credenza as by the reading of the clues. Depending on the hypotheses made, the estimate ranges from less than 0.01 to about 0.8. The authors also note that the indicators of consciousness largely overlap with the architectural characteristics required for general intelligence, which would make future systems increasingly difficult to dismiss out of hand. They argue for a "structured agnosticism": explicit theoretical commitments, credenzas revised as evidence arises, and aggregated probabilities rather than verdicts.