Tracing Model Contamination: Detecting LLM Radioactivity

In an article recently posted to the Meta Research Website, researchers investigated whether large language model (LLM)-generated texts retained detectable traces of their use as training data. They found that conventional methods like membership inference could carry out this detection with some accuracy. However, they showed that watermarking training data left traces easier to detect and much more reliable than membership inference.

Study: Tracing Model Contamination: Detecting LLM Radioactivity. Image credit: MDV Edwards/Shutterstock
Study: Tracing Model Contamination: Detecting LLM Radioactivity. Image credit: MDV Edwards/Shutterstock

The researchers linked the contamination level to factors such as the watermark's robustness, its proportion in the training set, and the fine-tuning process. Even with as little as 5% of the training text containing watermarks, the researchers demonstrated high-confidence detection of training on watermarked synthetic instructions (p-value < 10−5). Researchers quickly identified if the outputs of a watermarked LLM had been used to fine-tune another LLM, thanks to LLM watermarking, initially designed for detecting machine-generated text.

Related Work

Past studies have shown that fine-tuning LLMs with human prompts can be costly and challenging due to the need for expert knowledge and manual annotations. Practitioners sometimes train on synthetic data generated by already instructed models like Bard, chat generative pre-trained transformer (ChatGPT), or Claude to alleviate these issues. However, this raises questions about the originality of the fine-tuned model.

Detecting synthetic texts has become increasingly complex, especially concerning potential malicious uses. Watermarking has emerged as a solution to identify the generating model, particularly in the context of LLMs, with recent techniques ensuring minimal impact on output quality.

Exploring LM Radioactivity Detection

The detection of radioactivity in LMs, both with and without watermarking, is explored across different settings. In the absence of watermarking, methods like membership inference attacks (MIA) are employed to evaluate the radioactivity of individual samples by analyzing the loss of the LM on selected inputs. However, scenarios where specific training samples are unknown or access to the fine-tuned model is restricted impose limitations on these methods.

Watermarking offers a solution by embedding a trace in the generated text, enabling detection even in scenarios where specific training samples are unidentified or access to the fine-tuned model is limited. Researchers conduct detection tests across open and closed-model settings for fine-tuning data. For the open-model setting, where the fine-tuned model is accessible, watermark detection tests yield significant results, even with minimal proportions of watermarked data in the training set.

Conversely, in the closed-model setting, where the fine-tuned model is only accessible through an application programming interface (API), watermark detection tests still exhibit effectiveness, particularly when combined with filtering techniques to focus scoring on relevant k-grams. These methods demonstrate robustness in detecting radioactivity, even in scenarios with limited access to the fine-tuned model or uncertainty about specific training samples.

Experimental results confirm the efficacy of watermark-based detection in identifying radioactivity in fine-tuned LMs across various scenarios. While MIA provides robust detection in settings with open access to the fine-tuned model and known training data, watermark-based detection proves effective even in scenarios with restricted access or unknown training samples. It highlights the potential of watermarking as a reliable method for detecting the use of specific LM outputs in fine-tuning, contributing to the understanding and mitigating potential risks associated with model contamination.

Illuminate Key Factors

Demonstrating robust detection of watermark traces in a practical scenario, researchers provide high confidence in the effectiveness of the detection method. Researchers conduct further investigations to understand the factors influencing radioactivity in LMs. This analysis delves into three key aspects: fine-tuning, the watermarking algorithm, and data distribution.

Starting with fine-tuning, researchers analyze its impact on radioactivity detection using the same setup. They examine variables such as learning rate, fine-tuning algorithm, number of epochs, and model size. Results indicate that the degree to which the model fits the fine-tuning data influences the ease of detecting radioactivity. For instance, increasing the learning rate can significantly affect the model's radioactivity detection.

In the subsequent analysis, researchers explore the influence of the watermarking method and data distribution on radioactivity detection. They introduce more diversity into the data by prompting LM A with the beginnings of Wikipedia articles in English, generating the following tokens with and without watermarking. Researchers fine-tune LM B on these prompts and answers.

One aspect investigated is the watermark window size, with findings suggesting that smaller window sizes lead to higher confidence in radioactivity detection. Additionally, researchers consider the impact of data distribution, particularly in the unsupervised setting where Alice lacks prior knowledge about the distribution of the data used to fine-tune B. By running detection tests on text generated in different languages, researchers demonstrate the importance of data distribution in influencing radioactivity detection.

Despite Alice's potential need for more awareness regarding specific data distributions used in training B, researchers propose combining p-values using Fisher's method. This approach allows Alice to test radioactivity across various distributions and combine significance levels, providing insights into potential model contamination across different data sources and languages. Overall, these investigations shed light on the complex interplay between fine-tuning, watermarking methods, and data distribution influencing radioactivity detection in LMs.


Summing up, this study introduced methods to detect traces of "radioactivity" in LLMs by analyzing their generated texts used as training data. Non-watermarked texts posed challenges in detection, particularly in realistic scenarios. However, watermarked texts showed significant radioactivity, contaminating models during fine-tuning. It provided high-confidence identification of whether fine-tuning another model had utilized outputs from a watermarked model, although it did not detect the use of the other model itself.

Journal reference:
Silpaja Chandrasekar

Written by

Silpaja Chandrasekar

Dr. Silpaja Chandrasekar has a Ph.D. in Computer Science from Anna University, Chennai. Her research expertise lies in analyzing traffic parameters under challenging environmental conditions. Additionally, she has gained valuable exposure to diverse research areas, such as detection, tracking, classification, medical image analysis, cancer cell detection, chemistry, and Hamiltonian walks.


Please use one of the following formats to cite this article in your essay, paper or report:

  • APA

    Chandrasekar, Silpaja. (2024, March 27). Tracing Model Contamination: Detecting LLM Radioactivity. AZoAi. Retrieved on May 27, 2024 from

  • MLA

    Chandrasekar, Silpaja. "Tracing Model Contamination: Detecting LLM Radioactivity". AZoAi. 27 May 2024. <>.

  • Chicago

    Chandrasekar, Silpaja. "Tracing Model Contamination: Detecting LLM Radioactivity". AZoAi. (accessed May 27, 2024).

  • Harvard

    Chandrasekar, Silpaja. 2024. Tracing Model Contamination: Detecting LLM Radioactivity. AZoAi, viewed 27 May 2024,


The opinions expressed here are the views of the writer and do not necessarily reflect the views and opinions of AZoAi.
Post a new comment

While we only use edited and approved content for Azthena answers, it may on occasions provide incorrect responses. Please confirm any data provided with the related suppliers or authors. We do not provide medical advice, if you search for medical information you must always consult a medical professional before acting on any information provided.

Your questions, but not your email details will be shared with OpenAI and retained for 30 days in accordance with their privacy principles.

Please do not ask questions that use sensitive or confidential information.

Read the full Terms & Conditions.

You might also like...
Novel Algorithm for Accurate Weather Monitoring and Enhancing Highway Safety