Spontaneous reports to the police: None of the six major US AI companies say when they transmit their users' conversations

A study examines 27 public documents from six AI companies. None of them specifies the thresholds that trigger a report to law enforcement, or whether the user is informed afterwards.

AI companies practice what researchers call "proactive reporting": to warn law enforcement, without their request, when a conversation seems dangerous to them. The dilemma is real: reporting can expose someone to an investigation who has simply entrusted himself to an AI, not reporting can allow violence to occur. Minjong Cheon and its co-sponsors wonder under what conditions this practice can be justified and to what extent companies make it public.

Eight criteria, no published threshold

Based on the theory of contextual integrity and the rules that govern the lifting of secrecy in psychological counselling or private communications, the authors define eight criteria of justification, classified as mandatory, presumed or recommended. Three evaluators applied them to 27 public documents from six major U.S. companies. As a result, there is no indication as to whether it is its broader data sharing clauses or narrower security advisories that prevail, making the actual scope of the alerts impossible to determine. None publish the factual thresholds or exceptions that guide the decision, nor indicate whether the user is notified after the fact.

The study proposes the elements that a policy should contain to ensure that reporting remains a narrow and pre-announced exception.