On more than 5 million pages, the .gov and .edu sites and the most central web pages remain significantly less affected than the .com pages in the periphery.
In less than five years, generative AI has profoundly transformed the public web. A team of researchers measured its presence on a sample of more than 5 million pages accessible to web crawlers: the share of pages containing AI-generated text increased abruptly, but very unevenly across languages, domain types, and page position in the link network.
Pages in English contain more than those in other languages, and .gov and .edu domains contain much less than .com, even after adjusting for typesetting effects. Above all, growth is concentrated in the least central pages of the web, which explain much of the overall increase; the most central pages are the least likely to contain them.
A method that corrects detectors
Rather than relying on a single detection tool, the study combines several of them into a hierarchical Bayesian model that takes into account their false positives and false negatives, and variable performance across groups of pages. The authors' conclusion: AI has become an important, but very heterogeneous, source of public information online.
Read the full article on the original source
Go to the source →