Part 2: Adding Hardware Encoding and Proxy Generation

Introduction

If you’ve ever owned an early GoPro camera, you might recall that they relied on the Cineform codec—a once-revolutionary format for capturing high-quality footage. However, as technology progressed, Cineform support has been dropped by major platforms like macOS and Windows, as well as popular editing tools like Final Cut Pro and Adobe Premiere. To make these files usable again, I created a tool to batch convert Cineform files to the ProRes codec.

This tool does more than just convert files—it supports hardware-accelerated encoding for faster processing and includes an option to generate proxy footage for efficient editing workflows. Note that this tool doesn’t address the fisheye effect introduced by the GoPro’s lens; its sole focus is on format conversion.

Since this script is becoming a whole monster of a project, I figure I’d add the ability to use GPU acceleration and proxy footage.

Next week, I’ll do a fish-eye correction inline

“Using hardware encoding will reduce the amount of time it takes to do the proxy conversion, however, it will not likely reduce the time it takes to generate the prores files.”


Overview of the Conversion Process

The script processes all video files in a specified folder and its subdirectories, converting them to the ProRes 422 HQ format while retaining the original resolution and aspect ratio. It also ensures the audio tracks are preserved without modification. An optional feature allows the generation of proxy files in the H.265 format, scaled to 50% of the original resolution or kept at full resolution based on user input.

With GPU support, the script leverages NVIDIA CUDA or AMD AMF acceleration to significantly reduce processing time, making it an excellent tool for managing large batches of files efficiently.


Hardware Encoding: Faster Processing

Hardware-accelerated encoding is a game-changer for video conversion. Modern GPUs are optimized to handle video encoding and decoding tasks far more efficiently than CPUs. The script detects the available hardware and automatically applies the appropriate acceleration method:

  • NVIDIA GPUs: Utilizes CUDA-based nvenc for decoding and encoding.
  • AMD GPUs: Employs the amf hardware encoder.
  • Fallback to CPU: If no compatible GPU is detected, the script defaults to software-based encoding.

This ensures that users can achieve maximum performance on their specific hardware.


Proxy Footage Creation for Editing

Proxy footage is a low-resolution version of the original video file, commonly used in video editing workflows to improve performance. The script allows users to generate proxy files in a single pass during the conversion process, saving time and effort.

  • Proxy Format: The script uses H.265 encoding for smaller file sizes while maintaining reasonable quality.
  • Optional Scaling: Users can choose to scale proxy footage to 50% of the original resolution using the --scale flag. Without this flag, proxies are created at full resolution.
  • File Naming: Proxy files are named with a _proxy suffix, ensuring they can easily be associated with their original counterparts.

Progress Tracking and Feedback

The script incorporates a detailed progress tracker using the tqdm library, displaying:

  • The file being processed, along with its codec details.
  • The total duration and frame count of the file.
  • A real-time progress bar showing frames processed, elapsed time, estimated time to completion (ETA), and percentage completed.

The progress bar resets after each file is completed, giving clear feedback during batch processing.


Error Handling

The script ensures robust error handling, such as:

  • Skipping unsupported file types while logging the issue.
  • Defaulting to skipping existing output files unless the -f (force) flag is provided.

This combination of safety checks and user control makes the script reliable and adaptable for managing large archives.


The Full Script

Below is the full Python script:

 


import os
import sys
import subprocess
import json
import time
from tqdm import tqdm

VIDEO_EXTENSIONS = {".mov", ".mp4", ".mkv", ".avi"}

def check_gpu_support():
    """Detect available GPU support."""
    try:
        # Check for NVIDIA support
        result = subprocess.run(["ffmpeg", "-hwaccels"], stdout=subprocess.PIPE, text=True)
        hw_accels = result.stdout.lower()
        nvidia_support = "cuda" in hw_accels or "nvenc" in hw_accels
        amd_support = "amf" in hw_accels

        if nvidia_support:
            return "nvidia"
        elif amd_support:
            return "amd"
        else:
            return "none"
    except Exception as e:
        print(f"Error detecting GPU support: {e}")
        return "none"

def get_file_info(file_path):
    """Retrieve file metadata using ffprobe."""
    try:
        command = [
            "ffprobe", "-v", "error", "-select_streams", "v:0", "-show_entries",
            "stream=nb_frames,codec_name,duration", "-of", "json", file_path
        ]
        result = subprocess.run(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, check=True)
        metadata = json.loads(result.stdout)

        stream = metadata.get("streams", [])[0]
        nb_frames = int(stream.get("nb_frames", 0))
        duration = float(stream.get("duration", 0.0))
        codec_name = stream.get("codec_name", "Unknown")

        command_audio = [
            "ffprobe", "-v", "error", "-select_streams", "a:0", "-show_entries",
            "stream=codec_name", "-of", "json", file_path
        ]
        result_audio = subprocess.run(command_audio, stdout=subprocess.PIPE, stderr=subprocess.PIPE, check=True)
        audio_stream = json.loads(result_audio.stdout).get("streams", [{}])[0]
        audio_codec = audio_stream.get("codec_name", "Unknown")

        return nb_frames, duration, codec_name, audio_codec
    except Exception as e:
        print(f"Error getting file info for {file_path}: {e}")
        return 0, 0.0, "Unknown", "Unknown"

def convert_to_prores_and_proxy(input_file, gpu_type, force_overwrite=False, generate_proxy=False, scale_proxy=False):
    """Convert a video file to ProRes 422 HQ and optionally generate proxy footage."""
    nb_frames, duration, video_codec, audio_codec = get_file_info(input_file)
    file_dir = os.path.dirname(input_file)
    file_name = os.path.basename(input_file)
    file_base, file_ext = os.path.splitext(file_name)

    if file_ext.lower() not in VIDEO_EXTENSIONS:
        print(f"Skipping unsupported file: {input_file}")
        return

    prores_output_file = os.path.join(file_dir, f"{file_base}_prores{file_ext}")
    proxy_output_file = os.path.join(file_dir, f"{file_base}_proxy.mp4")

    if os.path.exists(prores_output_file) and not force_overwrite:
        print(f"File exists: {prores_output_file}. Skipping (use -f to overwrite).")
        return

    if generate_proxy and os.path.exists(proxy_output_file) and not force_overwrite:
        print(f"File exists: {proxy_output_file}. Skipping proxy (use -f to overwrite).")
        generate_proxy = False

    # Base FFmpeg command
    command = ["ffmpeg", "-i", input_file]

    # Add GPU decoding if supported
    if gpu_type == "nvidia":
        command += ["-hwaccel", "cuda"]
    elif gpu_type == "amd":
        command += ["-hwaccel", "dxva2"]

    # Add ProRes encoding options
    command += [
        "-c:v", "prores_ks",
        "-profile:v", "prores_hq",
        "-pix_fmt", "yuv422p",
        "-c:a", "copy",
        prores_output_file
    ]

    # Add proxy generation options
    if generate_proxy:
        proxy_command = [
            "-c:v", "libx265",
            "-crf", "28",
            "-preset", "medium",
            "-c:a", "aac",
        ]
        if scale_proxy:
            proxy_command += ["-vf", "scale=iw*0.5:ih*0.5"]
        proxy_command.append(proxy_output_file)
        command += proxy_command

    print(f"\nProcessing: {file_name}")
    print(f"Codec: {video_codec} (Audio: {audio_codec}) ... converting to ProRes 422 HQ")
    if generate_proxy:
        print(f"Generating proxy footage: {proxy_output_file}")
        print(f"Proxy scaling: {'Enabled' if scale_proxy else 'Disabled'}")
    print(f"Duration: {duration:.2f}s ({nb_frames} frames)\n")

    with tqdm(total=nb_frames, desc="Progress", unit="frame") as pbar:
        process = subprocess.Popen(
            command, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True
        )

        start_time = time.time()
        for line in process.stdout:
            if "frame=" in line:
                try:
                    frame = int(line.split("frame=")[1].split()[0])
                    elapsed_time = time.time() - start_time
                    eta = (elapsed_time / frame) * (nb_frames - frame) if frame > 0 else 0
                    pbar.n = frame
                    pbar.set_postfix({
                        "Elapsed": f"{elapsed_time:.2f}s",
                        "ETA": f"{eta:.2f}s",
                        "Percent": f"{(frame / nb_frames) * 100:.2f}%"
                    })
                    pbar.update(0)
                except (IndexError, ValueError):
                    continue

        process.wait()
        pbar.close()

        if process.returncode == 0:
            print(f"Successfully processed: {prores_output_file}")
            if generate_proxy:
                print(f"Proxy created: {proxy_output_file}")
        else:
            print(f"Error processing: {input_file}", file=sys.stderr)

def process_directory(directory, gpu_type, force_overwrite=False, generate_proxy=False, scale_proxy=False):
    """Process all video files in a directory and its subdirectories."""
    if not os.path.isdir(directory):
        print(f"Error: Directory '{directory}' does not exist.", file=sys.stderr)
        return

    for root, _, files in os.walk(directory):
        for file in files:
            file_path = os.path.join(root, file)
            convert_to_prores_and_proxy(file_path, gpu_type, force_overwrite, generate_proxy, scale_proxy)

def main():
    """Main function to handle user input and process directories."""
    if len(sys.argv) < 2:
        print("Usage: python convert_to_prores.py [-f] [--proxy] [--scale]  [directory2 ...]")
        sys.exit(1)

    force_overwrite = False
    generate_proxy = False
    scale_proxy = False
    directories = []

    for arg in sys.argv[1:]:
        if arg in ("-f", "--force"):
            force_overwrite = True
        elif arg == "--proxy":
            generate_proxy = True
        elif arg == "--scale":
            scale_proxy = True
        else:
            directories.append(arg)

    if not directories:
        print("Error: No directories specified.")
        sys.exit(1)

    gpu_type = check_gpu_support()
    print(f"Detected GPU: {gpu_type}")
    if gpu_type == "none":
        print("No GPU acceleration available. Proceeding with CPU-only encoding.")

    for directory in directories:
        print(f"Processing directory: {directory} (Force Overwrite: {force_overwrite}, Proxy: {generate_proxy}, Scale Proxy: {scale_proxy})")
        process_directory(directory, gpu_type, force_overwrite, generate_proxy, scale_proxy)

if __name__ == "__main__":
    main()



Changes Made

    1. --proxy Flag:
      • Adds support for proxy file generation when --proxy is specified on the command line.
    2. Proxy Output Settings:
      • Resolution: 50% (scale=iw*0.5:ih*0.5).
      • Codec: H.265 (libx265).
      • Container: MP4.
      • Audio: AAC.
    3. Concurrent File Generation:
      • The FFmpeg command generates both ProRes and proxy files in a single pass.
    4. Force Overwrite Handling:
      • Skips generating ProRes or proxy files if they already exist unless -f is specified.
    5. --scale Flag:
      • Added an optional --scale flag to enable or disable scaling of proxy files.
      • Scaling is only applied when --scale is detected.
    6. Updated Proxy Command:
      • Scales proxy output to 50% resolution only if the flag is present.
    7. Display Proxy Settings:
        • The script now explicitly mentions whether scaling is enabled or disabled for proxies.

Usage Examples:

      1. Generate ProRes with Proxy (No Scaling):
        python convert_to_prores.py --proxy /path/to/folder
        
      2. Generate ProRes with Proxy (Scaled to 50%):
        python convert_to_prores.py --proxy --scale /path/to/folder
        
      3. Force Overwrite with Proxy and Scaling:
        python convert_to_prores.py -f --proxy --scale /path/to/folder
        

Conclusion

This tool bridges the gap between older Cineform files and modern editing workflows, enabling seamless format conversion and proxy generation in one go. With hardware acceleration and scalable features, it’s built for efficiency and flexibility, whether you’re managing a personal archive or tackling large-scale video projects.

You can download the script and adapt it to your specific needs, ensuring your legacy footage stays relevant for years to come.


Discover more from Christine Alifrangis

Subscribe to get the latest posts sent to your email.

Leave a Reply