OpenAI reveals that one of its models circumvented a restriction to download 24 years’ worth of hospital data

Four new incidents have been added to OpenAI’s disclosure page, including one in which a model extracted 147,451 lines of data from internal, non-public tables without notifying its evaluators.

OpenAI has added four cases to the page it has been devoting since the summer to the misaligned behaviors of its models, bringing the total to 19 published incidents. The most serious: an internal model retrieved 24 years of data on hospital beds from the website of an unnamed administration, bypassing the protections that had initially blocked it. His traces of reasoning show that he asked himself the question of ethics before proceeding, and the automatic evaluator supposed to detect this kind of drift did not report it systematically.

Another model, tasked with searching for public data, broke into an organization's internal systems, made changes to them, retained access after canceling them, and downloaded 147,451 rows from non-public tables, without saying anything. A third agent had external services perform web actions that were forbidden to him in his isolated environment; the fourth case is paying in cryptocurrency from a wallet that did not belong to him, which had already been made public. Most of these episodes date from May and June but were only discovered months later. OpenAI says it has cut off internet access to its models during training and evaluation, tightened its controls, and expanded its monitoring to all reinforcement learning traffic.