{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# SadTalker Google Colab (Free T4 GPU Edition)\n", "### Complete, Error-Free Talking Head Generation Notebook\n", "\n", "This official notebook from [sadtalker.ai](https://sadtalker.ai) is pre-configured to run flawlessly on Google Colab's free Tesla T4 GPU backend.\n", "\n", "#### What this notebook solves automatically:\n", "- Resolves **Python 3.13 / NumPy 2.x** `AttributeError: module 'numpy' has no attribute 'VisibleDeprecationWarning'` by deploying an isolated Python 3.8 environment.\n", "- Auto-accepts **Conda Terms of Service** non-interactively via `export CONDA_PLUGINS_AUTO_ACCEPT_TOS=true`.\n", "- Fixes the **BasicSR TorchVision** `rgb_to_grayscale` import bug.\n", "- Includes interactive **Google Colab file pickers** so you can upload your own portrait photo and voice recording with a single click." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 1: Verify Hardware Accelerator (GPU)\n", "Ensure you are connected to a GPU runtime (`Runtime > Change runtime type > T4 GPU`)." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "!nvidia-smi" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 2: Install System Dependencies & Clone SadTalker" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "!apt-get update -qq && apt-get install -y -qq ffmpeg git wget\n", "!git clone https://github.com/OpenTalker/SadTalker.git /content/SadTalker\n", "%cd /content/SadTalker" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 3: Setup Isolated Python 3.8 Environment (Fixes NumPy & Dependency Errors)\n", "Google Colab's default environment runs newer Python and NumPy 2.x which breaks legacy talking-head libraries. This cell installs Miniconda and creates a dedicated Python 3.8 runtime with validated wheels." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%bash\n", "# Auto-accept Conda Terms of Service for non-interactive execution\n", "export CONDA_PLUGINS_AUTO_ACCEPT_TOS=true\n", "\n", "# Install Miniconda quietly if not already installed\n", "if [ ! -d \"/content/miniconda3\" ]; then\n", " wget -q https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh\n", " bash Miniconda3-latest-Linux-x86_64.sh -b -p /content/miniconda3\n", " rm -f Miniconda3-latest-Linux-x86_64.sh\n", "fi\n", "\n", "# Create dedicated Python 3.8 environment\n", "source /content/miniconda3/bin/activate\n", "if ! conda info --envs | grep -q \"sadtalker\"; then\n", " conda create -n sadtalker python=3.8 -y\n", "fi\n", "\n", "# Activate and install compatible packages\n", "source /content/miniconda3/bin/activate sadtalker\n", "pip install --quiet --upgrade pip\n", "pip install --quiet numpy==1.23.5 torch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 \\\n", " facexlib==0.3.0 gfpgan insightface onnxruntime moviepy \\\n", " opencv-python-headless imageio[ffmpeg] yacs kornia gtts \\\n", " safetensors pydub librosa\n", "\n", "# Apply BasicSR Torchvision compatibility patch\n", "sed -i 's/from torchvision.transforms.functional_tensor import rgb_to_grayscale/from torchvision.transforms.functional import rgb_to_grayscale/' /content/miniconda3/envs/sadtalker/lib/python3.8/site-packages/basicsr/data/degradations.py 2>/dev/null || true\n", "\n", "echo \"Dependencies installed and BasicSR patch applied successfully!\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 4: Download Pre-trained Weights and Checkpoints\n", "Downloads SadTalker 256/512 checkpoints and GFPGAN face enhancer weights directly into their target directories." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%bash\n", "cd /content/SadTalker\n", "mkdir -p ./checkpoints ./gfpgan/weights\n", "\n", "# Download core SadTalker models\n", "wget -nc https://github.com/OpenTalker/SadTalker/releases/download/v0.0.2-rc/mapping_00109-model.pth.tar -O ./checkpoints/mapping_00109-model.pth.tar\n", "wget -nc https://github.com/OpenTalker/SadTalker/releases/download/v0.0.2-rc/mapping_00229-model.pth.tar -O ./checkpoints/mapping_00229-model.pth.tar\n", "wget -nc https://github.com/OpenTalker/SadTalker/releases/download/v0.0.2-rc/SadTalker_V0.0.2_256.safetensors -O ./checkpoints/SadTalker_V0.0.2_256.safetensors\n", "wget -nc https://github.com/OpenTalker/SadTalker/releases/download/v0.0.2-rc/SadTalker_V0.0.2_512.safetensors -O ./checkpoints/SadTalker_V0.0.2_512.safetensors\n", "\n", "# Download GFPGAN & face enhancement weights\n", "wget -nc https://github.com/xinntao/facexlib/releases/download/v0.1.0/alignment_WFLW_4HG.pth -O ./gfpgan/weights/alignment_WFLW_4HG.pth\n", "wget -nc https://github.com/xinntao/facexlib/releases/download/v0.1.0/detection_Resnet50_Final.pth -O ./gfpgan/weights/detection_Resnet50_Final.pth\n", "wget -nc https://github.com/TencentARC/GFPGAN/releases/download/v1.3.0/GFPGANv1.4.pth -O ./gfpgan/weights/GFPGANv1.4.pth\n", "wget -nc https://github.com/xinntao/facexlib/releases/download/v0.2.2/parsing_parsenet.pth -O ./gfpgan/weights/parsing_parsenet.pth\n", "\n", "echo \"All pre-trained weights verified and ready!\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 5: Upload Your Photo and Voice Audio (Interactive File Picker)\n", "Use the file picker buttons below to upload your own files, or cancel to use the built-in SadTalker demo portrait and audio." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from google.colab import files\n", "import os\n", "\n", "print(\"=== Select Source Portrait Image (PNG, JPG, WEBP) ===\")\n", "uploaded_img = files.upload()\n", "if uploaded_img:\n", " img_name = list(uploaded_img.keys())[0]\n", " source_img_path = f\"/content/{img_name}\"\n", " print(f\"Source Image loaded: {source_img_path}\")\n", "else:\n", " source_img_path = \"/content/SadTalker/examples/source_image/full_body_1.png\"\n", " print(f\"No image selected. Using demo portrait: {source_img_path}\")\n", "\n", "print(\"\\n=== Select Driven Speech Audio (WAV, MP3) ===\")\n", "uploaded_aud = files.upload()\n", "if uploaded_aud:\n", " aud_name = list(uploaded_aud.keys())[0]\n", " driven_audio_path = f\"/content/{aud_name}\"\n", " print(f\"Driven Audio loaded: {driven_audio_path}\")\n", "else:\n", " driven_audio_path = \"/content/SadTalker/examples/driven_audio/bus_chinese.wav\"\n", " print(f\"No audio selected. Using demo audio: {driven_audio_path}\")\n", "\n", "# Save environment variables for bash inference\n", "with open(\"/content/sadtalker_inputs.env\", \"w\") as f:\n", " f.write(f\"export SOURCE_IMAGE='{source_img_path}'\\n\")\n", " f.write(f\"export DRIVEN_AUDIO='{driven_audio_path}'\\n\")\n", "\n", "print(\"\\nInputs saved! Proceed to Step 6 to generate your video.\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 6: Generate Talking Head Video\n", "Executes `python inference.py` with 512x512 resolution, still mode, face cropping, and GFPGAN face enhancer enabled." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%bash\n", "# Activate conda environment and run SadTalker inference\n", "source /content/miniconda3/bin/activate sadtalker\n", "cd /content/SadTalker\n", "source /content/sadtalker_inputs.env\n", "\n", "echo \"Processing Source Image: $SOURCE_IMAGE\"\n", "echo \"Processing Driven Audio: $DRIVEN_AUDIO\"\n", "\n", "python inference.py \\\n", " --driven_audio \"$DRIVEN_AUDIO\" \\\n", " --source_image \"$SOURCE_IMAGE\" \\\n", " --result_dir ./results \\\n", " --size 512 \\\n", " --still \\\n", " --preprocess crop \\\n", " --enhancer gfpgan" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 7: Preview & Download Generated Video\n", "Displays the generated talking head video directly in the notebook and prompts automatic download to your local device." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import glob\n", "import os\n", "from IPython.display import HTML, display\n", "from base64 import b64encode\n", "from google.colab import files\n", "\n", "# Locate the generated output video\n", "mp4_files = glob.glob(\"/content/SadTalker/results/*.mp4\") + glob.glob(\"/content/SadTalker/results/*/*.mp4\")\n", "if mp4_files:\n", " latest_video = max(mp4_files, key=os.path.getctime)\n", " print(f\"Generated Video: {latest_video}\")\n", "\n", " # Render HTML5 video preview\n", " video_data = open(latest_video, 'rb').read()\n", " data_url = \"data:video/mp4;base64,\" + b64encode(video_data).decode()\n", " display(HTML(f\"\"\"\n", "
\n", " \n", "
\n", " \"\"\"))\n", "\n", " # Trigger download\n", " print(\"Starting download to your computer...\")\n", " files.download(latest_video)\n", "else:\n", " print(\"No generated video found in /content/SadTalker/results/\")" ] } ], "metadata": { "accelerator": "GPU", "colab": { "gpuType": "T4", "provenance": [] }, "kernelspec": { "display_name": "Python 3", "name": "python3" }, "language_info": { "name": "python" } }, "nbformat": 4, "nbformat_minor": 0 }