A test set of 10,000 false statements shows that four commercial generators reproduce them accurately more than seven times out of ten; an automated attack raises this rate to 95 per cent.
Image generation models now know how to produce text-rich visuals (press clippings, manual pages) that are difficult to distinguish from authentic documents. A team of researchers has measured the extent to which this ability can be used to manufacture visual disinformation, and notes a paradox: when questioned directly, models recognize that a claim is false, but then agree to stage it as credible evidence. Guardrails judge what an image shows, not what it affirms.
10,000 false claims, four business models
The EpiReal-Bench brings together 10,000 instructions based on false claims and as many generated images, spread over ten categories of subjects and ten formats deemed credible. Of the four business models tested, including GPT-Image-2, more than 70% of the instructions give an image that faithfully reproduces the false information. The authors also propose EpiReal-Attack, a black-box optimization method that looks for formulations that bypass protections while maintaining the realism and readability of the text: it increases the rate to 95%.
The researchers point out that these tools are accessible at the click of a button, that their productions are distributed at a lower cost and are difficult to doubt, and that this aspect of the alignment remains largely unprotected to date.
Read the full article on the original source
Go to the source →