ElevenLabs has taken its AI voice-generation platform out of beta, adding support for more than 30 languages. The company says a new model developed in-house can identify languages automatically and produce speech it describes as “emotionally rich.”
One model, more language options
The update is aimed at creators who want to generate spoken audio in multiple languages. ElevenLabs says its platform can identify languages including Korean, Dutch and Vietnamese, then generate speech in them.
Customers can also combine the new model with the company’s voice-cloning tool. That feature can make a cloned voice speak across almost 30 languages without requiring users to type the text first, according to the company.
ElevenLabs was founded by CEO and co-cofounder Mati Staniszewski and his childhood friend Piotr Dabkowski. Staniszewski previously worked at Palantir, while Dabkowski was an ex-Google employee. The pair were inspired by the mediocre dubbing of American movies they watched while growing up in Poland, and set out to build a platform that could improve on it.
Growth brings scrutiny over voice cloning
The company launched its platform in beta in late January and gained attention for the quality of its generated voices and its generous free tier. But the same capabilities have also been exploited to create harmful audio.
The message board 4chan used ElevenLabs tools to share hateful messages mimicking celebrities, including actor Emma Watson. The Verge’s James Vincent also demonstrated how quickly the system could clone targets’ voices, producing samples with threats of violence and expressions of racism and transphobia.
In response, ElevenLabs said it would add safeguards, including restricting voice cloning to paid accounts and providing an AI detection tool. The launch out of beta puts the platform’s expanded language capabilities alongside those unresolved concerns about how people may use convincing synthetic speech.
Questions for the voice acting industry
Misuse is one concern; the impact on professional voice actors is another. Motherboard has reported that voice actors are increasingly being asked to sign away rights to their voices so clients can generate synthetic versions that could eventually replace them.
Internal emails seen by The New York Times indicate that Activision Blizzard is working on tools for AI-assisted “voice cloning.” These examples point to a broader tension around the technology: it can help publishers and creators make audio in more languages, while also raising questions about who controls a voice and how it may be used.
ElevenLabs presents publishing, audiobooks, radio and video games as natural areas for its tools. It touts work with Storytel, TheSoul Publishing and MNTN for audiobooks and radio content, as well as Embark Studios and Paradox Interactive for video games. Storytel and TheSoul Publishing are strategic investors.
Plans for dubbing and voice sharing
The company says more than a million registered users across creative, entertainment and publishing fields have created 10 years’ worth of audio content on its platform. It recently raised $19 million from investors including Andreessen Horowitz and DeepMind co-founder Mustafa Suleyman at a $99 valuation.
ElevenLabs plans to extend its models to voice dubbing, with the goal of transferring emotions and intonation from one language to another. It also says it intends to introduce a way for users to share voices on the platform, though details of that mechanism remain unclear.
The move out of beta marks a broader language offering for ElevenLabs. Its next steps will test whether the company can make synthetic speech useful across more markets while addressing concerns around voice cloning and the people whose voices may be reproduced.