Roblox AI rewrites player chats in real time to replace swear words and improve civility

Roblox's New AI Tool Rephrases Player Chats to Remove Banned Language

Video games these days involve chatting, and when people are given the chance to chat, they're not always going to keep it G-rated. The popular online game-creation platform Roblox, among many other games, has used simple substitution systems to censor chats for years. The game would simply remove the offending word and replace it with a bunch of pound signs. Now, Roblox is tapping AI to rephrase the offending phrase rather than just block out its letters. "Using AI filters to block problematic text (visible to users as #####) has always been central to our approach to safety and increasing civility," said Rajiv Bhatia, Roblox VP of User and Discovery Product, in a blog post. "Today, we're adding an additional feature to help keep in-experience text chat civil: We're leveraging AI to automatically rephrase messages, starting with profanity." When a player uses a problematic word in chat, the AI will rewrite the message to remove it before posting it to the chat, thereby eliminating the pound signs. So, "oh shit, are you OK?" turns into, "are you OK?" And "hurry tf up" becomes "hurry up." (Yes, even though that last one doesn't spell it out, Roblox knows what the F stands for.) Everyone in the chat will see that AI has rephrased the message, but only the sender will receive a warning about using improper language in chat. The change comes just two months after Roblox introduced facial recognition, which sorts users into age groups to limit contact between children and adults. It hasn't stopped legal action. Nebraska filed a lawsuit on Wednesday targeting Roblox's alleged child-safety failures, joining Texas, Florida, Louisiana, Kentucky and several municipalities. People make up new slang all the time, and much of it can be considered harsh or harmful. The company says the AI is being trained on "a set of specialized (AI) models" based on a larger model and in-game chat samples to help block current lingo. New slang is handled when the AI system can't decide whether a new word or phrase is actually bad, sending it to a larger, more highly trained AI model, which can then deduce its meaning from context. Players who repeatedly attempt to circumvent the new system or continue using banned words will still be disciplined as they normally would under the game's Community Standards rules. The new system may still produce false positives and take time to learn new lingo, but Roblox is hopeful that it will get it right most of the time. "Our experiments show that this combined approach has significantly improved our filters," Bhatia said. "The filters can now better detect leet-speak, or letters replaced with numbers or symbols, and more sophisticated attempts to bypass our filters."

Roblox is censoring chats with AI

Roblox is using AI to alter the content of chat messages on its platform in real time using a new feature rolling out today. Real-time chat rephrasing goes beyond the current filtering for banned language, which replaces certain words and phrases with "#" symbols. Now, Roblox says those words and phrases can be "translated into more respectful language that remains closer to the user's original intent." For instance, if a user posts "Hurry TF up!" in chat, Roblox's AI will alter the message to say "Hurry up!" instead. Whenever this happens, Roblox says "everyone in the chat is notified that the text has been rephrased to keep things civil." Minecraft similarly filters out profanity in certain in-game chats, but its filter only replaces banned words with symbols or blocks messages containing them, rather than rewording messages. Roblox's real-time chat rephrasing is currently only available for "in-experience" chats between players who have completed age verification and are in similar age groups. Last month, Roblox began requiring age checks to use its chat features on its platform. Once a user's age is confirmed, they can chat with other players in similar age brackets (i.e., users in the 9-12 age group can chat with players in the 13-15 age group).

What the Fork? Roblox's AI to Replace Swearing With More 'Civil' Language

(Credit: Pavlo Gonchar/SOPA Images/LightRocket via Getty Images) Roblox has announced changes to its content-filtering system to "maintain civility" in chats. As part of the move, Roblox will roll out real-time AI-powered chat rephrasing that automatically replaces swear words with more polite alternatives. Previously, the system displayed hashtags when banned words were used. Going forward, the AI will translate profanities "into more respectful language that remains closer to the user's original intent," Roblox said. The AI-powered rephrasing will also recognize profanities when they are expressed using abbreviations, numbers, or symbols. "Hurry TF up," for example, will get rewritten as "Hurry up." "This approach reduces friction in chat while maintaining the standards that help keep our community civil," says Rajiv Bhatia, Roblox's VP for user and discovery product. Every time Roblox's AI rephrases a message in chat, all group members will be notified of it. At launch, the feature is exclusive to in-game chats between age-verified users. It will work with all 16 languages supported by Roblox's automatic translation tool, including English, Spanish, Russian, French, Chinese, Arabic, and German. Roblox currently has 151.5 million daily active users, most of them minors. Recent changes come after Texas, Louisiana, and Kentucky sued the platform over allegations that it's become a "breeding ground for predators." A key part of that effort has been the introduction of age verification. Users worldwide are now required to undergo facial checks to use chat features. The checks are processed by a third-party vendor named Persona. After verification, users are assigned to one of six age groups and are only allowed to interact with others in their age group or a similar one. The system reportedly got off to a chaotic start, with claims that parents were verifying on behalf of their kids and that kids were getting mislabeled as adults. Roblox then added an option to reset verification in Account Settings. Age verification remains mandatory for chat features on Roblox. About 45% of the platform's users had completed the facial checks by February.

Roblox AI now swaps banned words for respectful text in real time

Roblox introduced a real-time AI chat rephrasing feature that automatically replaces banned words with respectful language. The new feature aims to maintain conversation flow by avoiding the disruption caused by the previous text filter, which replaced banned words with hash symbols. Roblox stated that the upgrade supports all languages available through its automatic translation tools. The company is also upgrading its text-filtering system to detect more variations of banned language. A message like "Hurry TF up!" will now be rephrased to "Hurry up!" instead of appearing as a string of hash marks. Everyone in the chat is notified when a message has been rephrased to maintain civility. The system detects leetspeak and sophisticated attempts to bypass filters. Rajiv Bhatia, Roblox VP of User and Discovery Product, stated the feature helps keep gameplay and conversations on track. "Real-time rephrasing helps keep gameplay and conversations on track while guiding language toward what's appropriate," Bhatia said. He added that the approach reduces friction in chat while maintaining community standards. Early results show the system has reduced false negatives when sharing or soliciting personal information by 20x. Roblox recently introduced mandatory facial verification for chat access following lawsuits from attorneys general in Texas, Kentucky, and Louisiana. The lawsuits were filed in response to reports that Roblox was exposing young users to risks such as grooming and explicit content.