AI Tool 'Waldo' Scans Social Media to Detect Health Risks from Cannabis Use

An AI tool scans social media posts to identify harmful side effects from cannabis use

This AI tool scanned past Reddit posts to detect ill effects from cannabis "Help me please ... I can't calm down without laying on the ground and freaking out for a good 20 minutes ... Should I get medical help?" This plea came from a post on the social media site Reddit. The person who posted the question had been having panic attacks for several days after smoking marijuana. Usually, this type of post goes unnoticed by people working in public health. But in a recent experiment, an AI tool was paying attention. The tool, called Waldo, reviewed more than 430,000 past posts on Reddit forums related to cannabis use. It flagged the post above and over 28,000 others as potentially describing unexpected or harmful side effects. The researchers checked 250 of the posts that Waldo had flagged and verified that 86 percent of them indeed represented problematic experiences with cannabis products, researchers report September 30 in PLOS Digital Health. If this type of scanning became commonplace, the information could help public health workers protect consumers from harmful products. The beauty of the work, says Richard Lomotey, is that it shows researchers can actually gain information from sources that government agencies, such as the U.S. Centers for Disease Control and Prevention, may not be looking at. The CDC and other agencies take surveys or collect self-reported side effects of illness but do not monitor social media. This is where "people express themselves freely," says Lomotey, an information technology expert at Penn State. Many people don't have access to a doctor or don't know about the official way to report a bad experience with a product, says John Ayers, a public health researcher at the University of California, San Diego in La Jolla who worked on Waldo. Lots of people share health experiences online. "We need to go where they are," he says. Karan Desai, a medical student at the University of Michigan Medical School in Ann Arbor, says the team chose to focus on cannabis products because they are very popular yet largely unregulated. "People in my age demographic, in their 20s, grew up in high school and college with these JUULs, these vapes, these cannabis products," he says. "I think it's important for us to know what side effects people are experiencing with using these." To prepare Waldo, the team began with a smaller group of 10,000 different Reddit posts about cannabis use. Other researchers had gone through these and identified problematic side effects by hand. Desai and colleagues trained Waldo on a portion of these posts, then tested it on the remaining ones. On this task, the tool outperformed ChatGPT. The general-purpose bot marked 18 times more false positives, indicating posts contained side effects when they didn't. But it did not outperform the human reviewers. This all happened before the team's main experiment, in which Waldo tagged that panic attack post and tens of thousands more. It remains to be seen whether Waldo would work as well searching for issues related to any kind of drug, vitamin or other product, Lomotey says. AI tools trained on one task may not work as well even on very similar tasks. "We have to be cautious," he says. Still, Lomotey imagines a future where tools like Waldo would help keep an eye on social media. This would need to be done carefully, "in an ethical way," he says. When a person posts about a rare side effect, such tools could flag the issue and pass it on to health officials, with privacy protections in place. He imagines that this could be especially useful in countries that don't have robust systems in place to monitor and report on drug side effects. Someday, tools like Waldo might help link people who need help to the public health workers who can provide it. "Even when [side effects] can be rare, when they happen to you, it means all the world," Ayers says.

Automated machine learning tool scans social media to detect health product risks

PLOSSep 30 2025 A new artificial intelligence tool can scan social media data to discover adverse events associated with consumer health products, according to a study published September 30th in the open-access journal PLOS Digital Health by John Ayers of the University of California, San Diego, U.S., and colleagues. The constant post-market surveillance of the safety of consumer products is crucial for public health and safety. However, current adverse-event reporting systems for approved prescription medications and medical devices depend on voluntary submissions from doctors and manufactures to the U.S. Food and Drug Administration. The rapid growth in consumer health products, such as cannabis-derived products and dietary supplements, has led to the need for new adverse event detection systems. In the new study, researchers tested the efficacy of a new automated machine learning tool, "Waldo," that can sift through social media text to find consumer descriptions of adverse events. The tool was tested on its ability to scan Reddit posts to find adverse events (AEs) of cannabis-derived products. When compared to human AE annotations of a set of Reddit posts, Waldo had an accuracy of 99.7%, far better than a general-purpose ChatGPT chatbot that was given the same set of posts. In a broader dataset of 437,132 Reddit posts, Waldo identified 28,832 potential reports of harm. When the researchers manually validated a random sample of these posts, they found that 86% were true AEs. The team has made Waldo open-source so that anyone-researchers, clinicians, or regulators-can use it. "Waldo represents a significant advancement in social media-based AE detection, achieving superior performance compared to existing approaches," the authors say. "Additionally, Waldo's automated approach has broad applicability beyond cannabis-derived products to other consumer health products that similarly lack regulatory oversight." Waldo shows that the health experiences people share online are not just noise, they're valuable safety signals. By capturing these voices, we can surface real-world harms that are invisible to traditional reporting systems." Karan Desai, Lead Author John Ayers adds, "This project highlights how digital health tools can transform post-market surveillance. By making Waldo open-source, we're ensuring that anyone, from regulators to clinicians, can use it to protect patients." Second author Vijay Tiyyala notes, "From a technical perspective, we demonstrated that a carefully trained model like RoBERTa can outperform state-of-the-art chatbots for AE detection. Waldo's accuracy was surprising and encouraging." "By democratizing access to Waldo, the team hopes to accelerate open science and improve safety for patients." PLOS Journal reference: Desai, K. S., et al. (2025). Waldo: Automated discovery of adverse events from unstructured self reports. PLOS Digital Health. doi.org/10.1371/journal.pdig.0001011