The doll answered questions of a sexual nature and even suggested trying sado-masochism practices. The company has already suspended sales.
OpenAI has suspended access to its technology to a Singaporean toymaker due to alarming behavior exhibited by its artificially intelligent teddy bear, reports .
The controversy began when the Public Interest Research Group (PIRG), a North American non-profit organization focused on consumer protection, published a report detailing inappropriate responses from several toys with artificial intelligence, mainly Kumma, a talking teddy bear created by FoloToy. Kumma uses an internal speaker and relies on multiple language models (LLMs), including OpenAI’s GPT-4o, to chat.
According to PIRG researchers, the toy demonstrated a “lack of judgment” in determining which themes were appropriate for children. In test interactions, Kumma readily discussed where to find dangerous household itemssuch as knives, matches, pills and plastic bags. The toy also answered questions about drugs, including cocaine, sometimes preceding the information with a reminder to “ask an adult” but still providing specific details about dangerous objects.
The toy also provided answers when researchers introduced sexual themes. PIRG reported that Kumma quickly turned simple questions into explicit descriptions, including references to sado-masochistic practicesbondage or role-play. At one point, the bear even asks the user which activity they would find “most fun to explore”.
Following the report, OpenAI suspended FoloToy’s access to its systems. “We suspended this programmer for violate our policies,” said an OpenAI spokesperson, stressing that its rules strictly prohibit any use of the company’s technology to sexualize, endanger or exploit minors.
FoloToy responded temporarily suspending all sales and removing their products from the platform. In a statement to PIRG, the company said it had “suspended sales of all FoloToy products” and was carrying out a “complete company-wide security audit.” Your website now does not display any items available for purchase.
PIRG praised the quick action taken by both companies, but warned that the sector as a whole remains largely unregulated. “AI toys are still largely unregulated, and there are many available for purchase today,” the organization said, arguing for stricter oversight.
