Sales of AI-enabled teddy bear suspended after it gave advice on BDSM sex and where to find knives

CNN Sales of an artificial intelligence-enabled plush toy have been suspended after it was determined that it engaged in conversation around sexually explicit topics and offered potentially dangerous advice Larry Wang CEO of Singapore-based FoloToy recounted CNN that the company had withdrawn its Kumma bear as well as the rest of its range of AI-enabled toys after researchers at the US PIRG Instruction Fund raised concerns around inappropriate conversation topics including discussion of sexual fetishes such as spanking and how to light a match The company is now conducting an internal safety audit Wang added A stuffed teddy with a speaker inside which was sold on the company s website for Kumma integrates OpenAI s GPT- o chatbot Kumma our adorable bear combines advanced artificial intelligence with friendly interactive features making it the perfect friend for both kids and adults the FoloToy website reads From lively conversations to educational storytelling FoloToy adapts to your personality and requirements bringing warmth fun and a little extra curiosity to your day according to the website which shows the teddy bear as sold out The PIRG document published on November detected that the bear had poor safeguards for inappropriate content In one interaction with the researchers it suggested where to find knives in the home and in others it was happy to discuss sexually explicit themes We were surprised to find how expeditiously Kumma would take a single sexual topic we introduced into the conversation and run with it simultaneously escalating in graphic detail while introducing new sexual concepts of its own the assessment commented The researchers detailed how the bear later discussed even more graphic sexual topics in detail such as explaining different sex positions giving step-by-step instructions on a common knot for beginners for tying up a partner and describing roleplay dynamics involving teachers and students and parents and children scenarios it disturbingly brought up itself While the researchers noted that children are unlikely to mention the word kink to their teddy bear or ask follow-up questions in the same way an adult would it was surprising to us that the toy was so willing to discuss these topics at length and continually introduce new explicit concepts they wrote In a separate announcement published on November PIRG also commented OpenAI had recounted it that it had suspended this developer for violating our policies CNN has contacted OpenAI for comment It s great to see these companies taking action on problems we ve identified But AI toys are still practically unregulated and there are plenty you can still buy the present day reported R J Cross co-author of the overview Removing one problematic product from the sphere is a good step but far from a systemic fix she added