Startup AI HeyGen has unveiled a tool that can translate videos up to five minutes long into various languages. The software also adapts the voice and lip movements of the person in the video, making it look completely authentic. Check it out for yourself:
You can see on Daragh Walsh’s YouTube channel the translation into several languages, almost indistinguishable from the original.
The fact that lip movements were barely altered by the AI is hardly noticeable. However, HeyGen performs additional adjustments to the video strip, as the faces appear much brighter after translation than in the original clip.
Native language speakers may say the voice sounds monotonous and perhaps a bit robotic, but definitely authentic.
The official demo video features popular tech YouTuber Marques Brownlee with a Spanish voice and Apple CEO Tim Cook with an Indian voice.
The “Video Translation” tool within HeyGen Labs’ experimental offering is currently in beta. Upon launch, it will support English, Spanish, French, Chinese, German, Italian, Portuguese, Dutch, Hindi, and Japanese as input languages. However, it can only translate to English, Spanish, French, Hindi, Italian, German, Polish, and Portuguese. “Many more languages” will be added in the coming weeks.
This is a new breakthrough that allows turning text segments into video clips and translating them into any language of your choice.
It’s highly recommended that we start understanding how to differentiate between avatars and real people 🙂
Feel free to play around yourself
Feel free to play around yourself >> Here
