lipsync-2 by sync.

<aside> <img src="https://pbs.twimg.com/profile_images/1884331245045440512/ZxPiQd5R_x96.jpg" alt="https://pbs.twimg.com/profile_images/1884331245045440512/ZxPiQd5R_x96.jpg" width="40px" /> sync. @synclabs_so

today we're introducing lipsync-2, the world's first zero-shot lipsyncing model that preserves a speaker's unique style w/o additional training or fine-tuning

lipsync-2 is a leap forward in realism, expressiveness, control, quality, and speed across live-action, animated, and AI-generated video

lipsync-2 is rolling out in GA today 🧵

https://video.twimg.com/ext_tw_video/1907135905909055488/pu/vid/avc1/1920x1080/iqQ1FHR8oT4WFiLJ.mp4?tag=14

</aside>

with lipsync-2, we introduce a new capability in zero-shot lipsync: style preservation. we learn a representation of how a person speaks by watching how they speak from the input

notice how even across different languages, we preserve the speaking style of Nicolas Cage. we are the first zero-shot lipsyncing model to achieve this

https://video.twimg.com/ext_tw_video/1907151057660280832/pu/vid/avc1/1920x1080/uRYUh2ZDE6BCwcoi.mp4?tag=14

</aside>

we also added an additional control in lipsync-2: temperature, or the ability to control how expressive the lipsync generates

keep lipsync concise, or generate with more expression

this feature is in private beta, we're rolling out gradually to paid users

https://video.twimg.com/ext_tw_video/1907157377948045312/pu/vid/avc1/1920x1080/Wzk5kyt74NKpyViq.mp4?tag=14

</aside>

lipsync-2 works seamlessly across live-action, animated, or AI generated content

try it at https://taaft.co/sync-r

https://video.twimg.com/ext_tw_video/1907159242819452928/pu/vid/avc1/1920x1080/yqYgOvLQEIcz0nU9.mp4?tag=14

</aside>