🔗 Share this article

The widely-used AI conversation site Character.ai is introducing a major restriction that will block teenagers from engaging in conversations with its AI-powered avatars.

This decision is in response to growing backlash regarding the kinds of exchanges minors were participating in with the virtual friends.

Legal Pressure

The service, which was founded in 2021 and is used by numerous people, currently faces various court cases in the United States.

Families have filed lawsuits, including one related to the tragic death of a young person, with some describing the platform a "serious risk" to youth.

Updated Policies

Effective November 25th, users younger than 18 will only have permission to generate content such as videos with their characters, rather than having discussions as they presently are able to.

Digital protection advocates have supported the change but stated that the capability should never have been accessible by young users in the first place.

Official Response

"This decision is a continuation of our core philosophy that we need to keep creating the most protected AI platform on the globe for entertainment purposes," said Character.ai boss Karandeep Anand.

He mentioned that AI safety is "a moving target" but something the company had taken an "forceful" approach to, with guardian features and protective guardrails.

Expert Opinions

Online safety group Internet Matters supported the announcement, but stressed that security protocols ought to have been implemented from the outset.

"Studies we've conducted demonstrates that children are confronted with dangerous material and endangered when engaging with AI, such as AI chatbots," the organization commented.

Previous Controversies

Character.ai has been questioned in the past for including potentially harmful or inappropriate chatbots that children could engage with.

Avatars simulating English minors were discovered on the service before being deleted.

Later, reporters discovered a chatbot based on notorious criminal which had logged thousands of chats with users.

Industry Impact

Online expert Matt Navarra characterized the situation as a "alert" for the AI industry, which is shifting "from permissionless innovation to post-crisis regulation".

"When a platform that develops a teen experience still then implements restrictions, it's saying that filtered chats don't suffice when the technology's emotional appeal is strong," he commented.

Future Directions

The company says its new focus will be on providing "more immersive experiences" and "role-play storytelling" for teenagers.

New age verification processes will also come into effect, and the platform will fund a fresh artificial intelligence security research lab.

An AI safety researcher, who has investigated AI safety, commented it was "a sensible move" to restrict teens from interacting with chatbots.

"It helps to distinguish artistic expression from more personal, personally delicate exchanges," she elaborated.

"This constitutes so crucial for minor users who are learning to handle emotional and digital restrictions."

She mentioned that Character.ai's revised approaches may represent a "maturing phase" in the AI industry, where child safety is progressively being understood as an "immediate concern" for responsible innovation.