AI Multi-Speaker Lip-Sync Enhances Real-Time Translation- written by Aishee Biswas (Managing Editor, Bisjhintus News)

Rask AI, a leading video and audio localization tool, has unveiled its groundbreaking Multi-Speaker Lip-Sync feature. With this advancement, over 750,000 users can now translate their content into 130+ languages, ensuring it sounds as fluent as if spoken by a native speaker.

 

Bridging the Synchronization Gap

 

The Challenge of Lip Sync:

Historically, dubbed content has suffered from a lack of synchronization between lip movements and voices. This asynchrony often makes dubbing unpopular in English-speaking countries. Realistic lip movements enhance localized content, making it more appealing to audiences.

 

The Power of AI Lip-Sync:

Rask’s new feature leverages AI-powered visual adjustments to create natural lip movements. The algorithm restructures the lower face based on references, considering both the speaker’s appearance and their spoken words. The result is dubbed videos that appear more authentic and engaging.

 

How It Works:

Content creators can upload videos with one or more people in the frame. After translating the video into another language, they press the ‘Lip Sync Check’ button. The algorithm evaluates the video for lip sync compatibility. If successful, creators can proceed with the lip sync process and download the final video.

 

Expanding Audience Reach:

According to Maria Chmir, founder and CEO of Rask AI, this feature empowers content creators. By visually adjusting lip movements, characters now appear to speak the language fluently. The underlying technology relies on generative adversarial network (GAN) learning, where a generator creates content (lip movements), and a discriminator ensures quality control.

 

Conclusion:

The beta release is available to all Rask subscription customers. As AI continues to enhance localization and communication, this feature opens new possibilities for global content distribution.