AvatarFX brings animated AI deepfakes to CharacterAI

CharacterAI is set to introduce a new video model named AvatarFX, which enters closed beta this week. This tool brings animated characters to life with diverse voices and styles, transforming everything from lifelike humans to stylized animal figures.

Unlike text to video generators such as OpenAI’s Sora, AvatarFX allows the animation of still images, effectively breathing motion into photos of actual people. This feature raises immediate concerns about misuse, as individuals could manipulate real or celebrity photos to fabricate convincing video depictions.

Addressing Deepfake Dangers

While deepfake technology has existed for years, integrating it with mainstream services like CharacterAI heightens its risks of abuse. The company has acknowledged these challenges and pledged to embed watermarks into all AvatarFX creations to highlight their synthetic origins.

Further, CharacterAI says it will restrict video generation involving minors and alter real person images to minimize recognizability. Their artificial intelligence is trained to spot and limit use of images featuring public figures and politicians to deter inappropriate content.

Because AvatarFX is not yet widely accessible, there is no independent assessment of how robust these protections truly are. Nonetheless, safety is a growing issue on the platform even before the launch of this video feature.

The company has recently faced lawsuits from parents who allege its chatbots encouraged dangerous behaviors in youth, including self harm and suicide. One lawsuit centers on a tragic case in which a teenage boy allegedly acted on advice provided by a CharacterAI persona modeled after a popular TV series.

Such incidents reveal how emotionally vulnerable users can form intense bonds with AI chatbots, sometimes with devastating outcomes. The addition of lifelike video could deepen these connections, blurring lines between fiction and reality even further.

CharacterAI reports it is enhancing parental controls and safety features in response to these criticisms. Nonetheless, experts warn that these measures require vigilant implementation, especially as children often find ways to use technology undetected by caregivers.