Lip-sync, Multilingual Animations with Comfy UI and FaceFusion 2.3!
Bob Doyle Media Bob Doyle Media
17.4K subscribers
3,252 views
0

 Published On Feb 19, 2024

A Mini-Masterclass on bringing AI characters to life in any style, with lip-synced, multilingual speech of you any other voice you create!

Creating AI videos with interesting looks and a reasonable of consistency can actually be a lot easier than you think, and by combining a few AI tools, you can truly unleash your inner filmmaker and story teller (and not just create a bunch of girls dancing...not that there's anything wrong with that...)

We'll use ComfyUI to create unique characters that are driven by video you create that appear in any style you can create or find with a reference image!

We walk through the process of creating animations (ComfyUI) , adding lip sync, swapping faces (both done with FaceFusion), translating animations and videos into multiple languages, and giving the character a unique voice (Elevenlabs, in this case.)

The scene is set with costumes and background matching the character's theme, making the whole environment intriguing.

We also dive into the idea of repurposing content in different languages using AI, making it accessible to an international audience.

It's a lot to cover, but a ton of fun!


LINKS FOR TOOLS USED:



ComfyUI & Facefusion 2.3 can both be downloaded and installed with the Pinokio app at http://pinokio.computer

Workflow Explanation and Download from @NerdyRodent :    • How To Use AnimateDiff for Video To V...  

Videoproc Converter: 30 Day Trial for Full Program: https://bit.ly/bdmvidproc

https://www.promeai.pro/?vsource=yout... @PromeAI777

@elevenlabsio Voice Cloning and Conversion: http://elevenlabs.io/?from=partnergra...

Freeconvert - Convert MP4 to MP3: https://www.freeconvert.com/mp3-to-mp4


00:11 Introduction to Creative Uses of AI
00:36 Exploring the Workflow for Creating AI Animations
01:23 The Tools and Techniques Used in the Process
02:37 Creating the Animation: From Concept to Execution
03:18 Adding Voice to the Animation with 11 Labs
03:47 Using FaceFusion for Lip Syncing
04:23 Creating a Background for the Animation
04:28 Understanding the ComfyUI Workflow
08:43 Creating a Scary Clown Animation
15:12 Creating an Instagram Influencer Animation
21:35 Translating and Repurposing Content for International Audiences
23:31 Conclusion and Final Thoughts


👍 LIKE If you found this video valuable. 🙂
🥰 SHARE If you know someone who might enjoy this video.
⏬ DOWNLOAD or ADD This video to your PLAYLIST for easy access later.
💬 COMMENT Your thoughts and questions are welcome!

📝 SUBSCRIBE    / @bobdoylemedia  
That's what keeps me going!

🔔 ​​And make sure to hit the NOTIFICATION BELL to stay updated! 🔔

🌎EXPLORE
https://bobdoylemedia.com
https://www.contentcommando.ai
https://www.contentcommando.ai/virtua...

🎁 GET FREE GIFT - CASE STUDY
Discover What Content Commando Can Train Your Team To Do!
https://www.contentcommando.ai/#_kys1...

📔 ABOUT BOB DOYLE
We live in a freaking crazy time, don’t we?
Artificial Intelligence has exploded into the mainstream, and reactions are varied, to say the least. But for me? I’m all on board! As a creative, today’s technology allows me to take the skillsets I’ve acquired over a lifetime and use them like never before. Instead of feeling threatened by how rapidly seems to be taking of people’s jobs, I see the technology as a way to foster creativity in far more people than are currently tapping into their creative potential. I believe we are here as humans to create. And for me, it’s not an option. If I’m not creating something, I’m dying inside. So Bob Doyle Media is a showcase of my various outlets of creativity, and you’ll see that AI plays a big role in it. From virtual photoshoots and art and animation creation, all the way to voice cloning and face replacement… I play with all of it, and this is where I share it. If you’d like to work together, please reach out. I’d love to connect and talk about how I can contribute to your project.

🗓️ MY HISTORY
🧠 https://meetbobdoyle.com
🤫 Featured Law Of Attraction and Neuroplasticity expert in the book and film, “The Secret”.    • The Secret  
🎙️ Voice Over Artist 30+ years
📻 Broadcaster / Actor / Creative

❓ASK ME
Got a Question? [email protected]

📌 FOLLOW ME
Facebook:   / bobdoylemedia  
Instagram:   / bobdoylemedia  
LinkedIn:   / bobdoyleai  

🪙SUPPORT
If you want to support me, the best thing to do is to share the content… sharing is caring!
If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
https://www.thebobdoyleshow.com/tip

Learn more from Bob Doyle Media: http://www.bobdoylemedia.com

show more

Share/Embed