Community Notes on X Significantly Reduce Spread of Misinformation

"We found that Community Notes are effective when attached, especially in reducing engagement that signals support for the content, such as reposts and likes," said senior author Martin Saveski, a UW assistant professor in the Information School. "But the spread of misinformation on social media is complex and multifaceted, and it requires multiple approaches working together to effectively curb it. Systems like Community Notes are an important addition to the platforms' toolbox."

The team published its findings Sept. 18 in the Proceedings of the National Academy of Sciences of the United States of America.

Study Design

Between March and June of 2023, researchers tracked 40,000 posts for which a note was suggested. Of those, 6,757 notes were deemed helpful and were attached. The team tracked posts for 48 hours after receiving a note and compared them with notes to those without, focusing on two key aspects: engagement, such as likes and reposts, and diffusion.

Diffusion accounts for how a post spreads through the social network, essentially its virality. For example, do only people who follow an account engage with a post?

Impact on Engagement

"We know from other studies that false information typically spreads faster, broader and more virally, than true information does," said lead author Isaac Slaughter, a UW doctoral student in the Information School. "We found that Community Notes significantly change the way information spreads through a network. People who are distant in the social network from the person that posted the misinformation are much less likely to interact with the post. But people close to the source, followers, for instance, tend to be less affected by the note."

On average, the team found that engagement dropped by 46% for reposts, 44% for likes, 22% for replies, and 14% for views after notes were added. Over posts' whole lifespans, including engagement before notes were attached, the drops were 12% for reposts, 13% for likes, 7% for replies, and 6% for views.

"We think views were less affected because what users see is mostly decided by X's feed algorithm," Saveski said. "From the public release of the algorithm, we know that X does not explicitly deemphasize posts with notes attached, but that could change in the future."

Timing and Effectiveness

The study was also able to obtain granular data on what factors affected the spread of posts. Notes added to altered media, such as fake photos and videos, had a greater impact on those posts than on text-based posts. Notes on very popular posts led to greater reductions in engagement. And getting notes appended quickly was vital.

"Content spreads rapidly across X, and if a note comes too late, few users will get a chance to see it," Slaughter said. "Notes that take 48 hours or so to go up have almost no effect."

Saveski's lab at UW is now developing potential tools to speed up the process of attaching notes to posts, thereby increasing their effectiveness.

Limitations and Future Challenges

The authors only examined posts that had notes proposed in early 2023, and X has since significantly updated its Community Notes methods. But it also ended free access to its API, making further academic studies infeasible. The paper also focused solely on X, rather than examining other social media platforms.

"Whether this kind of moderation is sustainable as many separate systems across different platforms, as it's now being used, is really an open question," Saveski said. "If someone is adding notes on X, does that make them less likely to do so on TikTok or Instagram? There's also the question of how much platforms should collaborate and share data, which could help this scale. X has made its code and data available, but none of the other platforms have committed to opening up their systems yet."

Research Team and Funding

Co-authors include Axel Peytavin of Stanford University and Johan Ugander of Yale University. This research was funded in part by a UW Information School Strategic Research Fund award and an Army Research Office Multidisciplinary University Research Initiative award.