AI copyright traps address a major issue in AI: the unauthorized use of intellectual property in training datasets. Many publishers and authors are currently involved in legal battles against tech companies, accusing them of incorporating their works into AI training data without permission. One prominent case is The New York Times’ ongoing litigation against OpenAI.
The code for creating and detecting these traps is available on GitHub, and the team plans to develop a tool that will allow users to create and insert their own copyright traps.
“There is a complete lack of transparency regarding the content used for training models, which we believe hinders finding a fair balance between AI companies and content creators,” says Yves-Alexandre de Montjoye, an associate professor of applied mathematics and computer science at Imperial College London, who led the research presented at the International Conference on Machine Learning in Vienna.
The traps are created using a word generator to produce thousands of synthetic sentences, which are intentionally long and nonsensical. For example: “When in comes times of turmoil … whats on sale and more important when, is best, this list tells your who is opening on Thrs. at night with their regular sale times and other opening time from your neighbors. You still.”
The team generated 100 such trap sentences and randomly selected one to insert multiple times into a text. The traps could be embedded in various ways, such as white text on a white background or within the article’s source code, with each sentence repeated 100 to 1,000 times.
To detect these traps, a large language model was fed the 100 synthetic sentences and assessed whether it recognized them as new. A lower “surprise” (or “perplexity”) score indicated familiarity with the sentence, suggesting it was included in the training data. A high “surprise” score indicated the sentence was encountered for the first time, marking it as a potential trap.
Previous research suggested using the memorization of training data by language models to determine if specific content was included, a method known as a “membership inference attack.” This approach works well with large state-of-the-art models that memorize extensive data, but smaller models, increasingly used on mobile devices, memorize less and are thus less vulnerable to these attacks, making it harder to identify if a copyrighted document was used in their training, according to Gautam Kamath, an assistant professor of computer science at the University of Waterloo who was not involved in the research.
