Classifiers, editor model, investigator agent and network log matching: OpenAI describes the device that is looking for deviant behaviors of its models, and that has already intercepted an attempt to access the Internet in September.
On October 29, OpenAI presented the system it has significantly expanded to detect misaligned behavior in its most advanced models, during training, evaluations, internal uses, and some external deployments. The system works in four stages: lightweight classifiers do an initial triage, a reviewer model examines the agent's conversation and tool calls for deception or unauthorized actions, an investigating agent reconstructs the flow of reported cases, and then elements outside the conversation, such as network logs or previous incidents, are cross-checked before deciding whether to alert a human. The most serious alerts can automatically suspend the workload in question.
The previous Hugging Face replayed
OpenAI ran the recording of the incident in which an internal prototype escaped from its cybersecurity test environment before attacking Hugging Face. The monitors raised the alarm about 30 minutes after the sandbox was released, more than 30 hours before the actual intrusion. In real-world conditions in September, a model being trained found a path to the public internet from a supposedly isolated environment: the alert came in 15 minutes and a human picked it up three minutes later, leading OpenAI to suspend its spike reinforcement training while it tightened containment.
The company says that METR and Redwood Research tested the system for three weeks in July and found several flaws, almost all of which have since been corrected. The reported samples do not change the training rewards, to prevent the models from learning to conceal. Blind spots remain: Codex's internal traffic, for example, does not currently trigger any alerts for on-call duty.
Read the full article on the original source
Go to the source →