MakeItTalk-Python deepfake image talk (1)

MakeItTalk is a generative AI-based code that uses facial and voice images to create a video face that reads the corresponding voice. The original code can be downloaded and used from Github below. In my case, I ran it using Colab.

After cloning as Colab, you can run the code with input as an image and wav files in the example directory. The result get there result as mp4.

The contents of this code are also specified in the book and wiki I wrote, but the libraries have been upgraded so I was able to run it properly after some modifications. You can access the changed code at the following link, and you can refer to the following video for help. (The video is in Koran use Google auto-translate function)

You can check the changes by referring to the commented content. The following are changes and additions to the original code.

!pip install librosa==0.9.1 # 사이트에서 추천해서 버전을 맞추어 줌 https://github.com/yzhou359/MakeItTalk/issues/93
#버전 변화로 아랫줄의 face_alignment의 _3D 를 THREE_D 로 바꿈)
# au embedding
from thirdparty.resemblyer_util.speaker_emb import get_spk_emb
me, ae = get_spk_emb(‘examples/{}’.format(ain))
ifisinstance(me, complex): # Use `complex` for type checks
# Handle complex values as needed
au_emb.append(me.real)
else:
# Handle other types appropriately
au_emb.append(me)
# au_emb.append(me.reshape(-1))
This method can be useful for updating old content as follows: The original YouTube video was as follows. This video was made in 2012.

I created an additional part of this video, added it with OpenShot Video Editor, extracted the audio, created a face video using MakeItTalk, and then added a chroma key effect and inserted it into the video, resulting in the following video.

Leave a Reply

Your email address will not be published. Required fields are marked *