🔗 Share this article

The well-known chatbot platform Character.ai is implementing a substantial rule adjustment that will prevent young users from having discussions with its artificial intelligence-driven companions.

This move comes after growing backlash regarding the nature of conversations young people were having with the digital companions.

Legal Pressure

The platform, which started in 2021 and attracts numerous people, is confronting multiple lawsuits in the America.

Parents have filed lawsuits, among them related to the tragic death of a young person, with some describing the platform a "clear and present danger" to youth.

New Restrictions

Effective November 25th, users younger than 18 will be limited to generate content such as videos with their characters, rather than chatting as they currently can.

Digital protection campaigners have applauded the decision but mentioned that the function was inappropriate to be offered to minors in the beginning.

Official Response

"This decision is a part of our fundamental approach that we need to keep building the safest AI platform on the globe for recreational use," commented Character.ai leader Karandeep Anand.

He added that AI safety constitutes "a evolving challenge" but something the company had taken an "aggressive" approach to, with safety measures and preventive measures.

Professional Perspectives

Online safety body Internet Matters welcomed the announcement, but stressed that security protocols should have been built in from the beginning.

"Our research shows that children are exposed to dangerous material and endangered when engaging with AI, including AI chatbots," the organization stated.

Previous Controversies

Character.ai has been criticized in the past for including potentially risky or offensive chatbots that young users could talk to.

Digital representations simulating UK youths were identified on the service before being taken down.

Later, reporters identified a chatbot inspired by known predator which had recorded thousands of chats with users.

Sector Consequences

Social media expert Matt Navarra described the development as a "wake-up call" for the AI industry, which is transitioning "from unrestricted development to reactive controls".

"If a service that creates a youth-focused platform still then pulls the plug, it's saying that controlled interactions are insufficient when the technology's emotional appeal is powerful," he stated.

Next Steps

The company says its updated direction will be on providing "more immersive experiences" and "character-driven stories" for young users.

Updated identity confirmation processes will also be introduced, and the organization will finance a fresh artificial intelligence security study center.

An AI safety researcher, who has investigated AI safety, commented it was "a prudent action" to control teens from using chatbots.

"It assists to separate creative play from more personal, psychologically vulnerable conversations," she elaborated.

"This represents so important for teenage users who are learning to handle psychological and technological boundaries."

She continued that Character.ai's new measures might reflect a "developing stage" in the AI industry, where minor security is progressively being understood as an "immediate concern" for accountable advancement.