Translating a video can mean more than replacing its spoken words. HeyGen’s Video Translate tool combines a new-language voice with adjustments to the speaker’s lip movements, aiming to make the result look like a continuous recording. The open beta brings that process to short clips for businesses and home users.
One tool changes both speech and picture
HeyGen says its tool can translate videos up to five minutes long. It also clones the voice of the person in the recording and modifies their lip movements to fit the translated speech. The goal is to preserve a connection between what viewers hear and what they see.
A test shared by Twitter user Jon Finger showed a clip translated into French and German. The original footage was recorded on a busy street, and the edited version filtered out distracting background sounds. The synthetic voice retained a strong resemblance to the original speaker and reproduced the intonation, though it still sounded somewhat robotic or mechanical.
The lip changes were difficult to spot with the naked eye in that test. One visible difference was the speaker’s face: it appeared much brighter in the translated version than in the original. A German native speaker described the translation as somewhat monotonous but authentic, with intonation that might seem unusual without sounding unnatural.
Language support and clip requirements
Video Translate is part of HeyGen Labs’ experimental offerings and is in open beta. At launch, users can provide videos in English, Spanish, French, Chinese, German, Italian, Portuguese, Dutch, Hindi, and Japanese. The tool can translate into English, Spanish, French, Hindi, Italian, German, Polish and Portuguese. HeyGen said “Many more” languages would be added in the coming weeks.
The beta has technical limits that shape which clips are suitable. Videos must be at least 30 seconds long and use MP4, Quicktime or Webm file types. Supported resolution runs from 480 x 480 pixels to 1920 x 1920 pixels.
HeyGen’s recommendations also point to the importance of clear footage and sound. The company advises that each scene show only one person’s face from one angle. Background noise or music can make a clean translation more difficult, even though Finger’s test showed the software filtering noise in one busy-street clip.
Access starts with a small free allowance
HeyGen, formerly known as Movio.la, gives new users two free credits after signing up. Those credits can be used to edit or create up to two minutes of video. An additional 15 credits are available starting at $29 per month, and can be used across HeyGen’s browser-based software offerings.
Video translation sits alongside the company’s main focus on virtual AI avatars. Making voice and lip synchronization available to smaller companies and home users broadens access to a technique associated with large film productions. The same capability raises questions for voice actors, whose work and livelihoods are affected as synthetic dubbing becomes more accessible.
From experiments to screen productions
Synchronizing speech and lip movements is not a new idea, but the technology is drawing attention in the film industry. An Israeli start-up made headlines with a similar service in 2021, while major productions using AI dubbing had not yet been released, according to the source article.
There was, however, an example of AI being used in a mainstream movie: “Top Gun: Maverick” used AI to bring back Val Kilmer’s voice. Together, these examples suggest a technology moving from specialized services toward broader use. HeyGen’s beta makes the process available for everyday clips, while its limits on length, languages, image conditions and background sound show that the results still depend on the material being translated.