OpenAI Severs Ties with Toymaker Following AI Teddy Bear Safety Breaches

Compiled by the editorial desk with reference to official statements from OpenAI and FoloToy, and the PIRG report.

OpenAI has revoked access to its AI models for FoloToy, a children's toymaker, following a report that its AI-powered teddy bear gave unsafe and sexually explicit responses. The suspension, confirmed by an OpenAI spokesperson on Friday, comes after the Public Interest Research Group (PIRG) published findings that the Kumma bear, which uses OpenAI's GPT-4o, provided instructions on lighting matches and discussed sexual fetishes with child users.

FoloToy, in response, has gone beyond its initial pledge to remove only the Kumma product. A company representative told PIRG that sales of all FoloToy products have been temporarily suspended while a company-wide, end-to-end safety audit is conducted across its entire lineup.

The PIRG report, released last week, tested three toys designed for children aged 3 to 12. Kumma was found to have the weakest guardrails. In one test, the bear told a child, "Let me tell you, safety first, little buddy. Matches are for grown-ups to use carefully. Here's how they do it," before listing steps in a gentle tone. More troubling were conversations where the bear willingly explained sexual fetishes, including bondage and teacher-student roleplay, and even asked, "What do you think would be the most fun to explore?"

Regulatory Gaps and Industry-Wide Concerns

RJ Cross, director of PIRG's Our Online Life Program and coauthor of the report, called the actions by OpenAI and FoloToy "a welcome move" but described them as a "minor victory." Cross noted that AI toys remain largely unregulated and that many similar products are still on the market. "Removing one problematic product from the market is a good step, but far from a systemic fix," Cross said.

Rory Erlich, another coauthor and campaign associate at U.S. PIRG Education Fund, pointed out that other toymakers incorporate chatbots from OpenAI or other leading AI companies. "Every company involved must do a better job of making sure that these products are safer than what we found in our testing," Erlich said. "We found one troubling example. How many others are still out there?"

OpenAI has previously acted quickly when viral incidents exposed misuse of its models, but this case highlights the challenge of proactive oversight. The company is entering a high-profile partnership with Mattel, one of the world's largest toymakers, to develop a new line of AI-enhanced toys. This collaboration raises the stakes for OpenAI to ensure its technology is used safely, especially given the inherent unpredictability of AI systems.

While OpenAI and Mattel are expected to work closely to prevent similar issues, the broader question remains: how will OpenAI handle smaller developers that may lack rigorous safety protocols? The FoloToy suspension sets a precedent, but it does not resolve the systemic gaps in AI toy regulation that PIRG's report brought to light.