Audio Summary with Local LLM (Apple Silicon / MLX)

This tool provides a quick and concise summary of audio and video files. It supports summarizing content either from a local file or directly from YouTube. Everything runs fully on Apple Silicon through Apple's MLX framework — no NVIDIA/CUDA, no PyTorch CUDA stack, no external server required.

• Speech-to-Text: Mistral Voxtral via mlx-audio (with an mlx-whisper fallback for memory-constrained Macs). Voxtral outperforms Whisper large-v3 on most transcription benchmarks and natively handles long audio with streaming.
• Summarization: Qwen3 via mlx-lm — the recommended family for summarization on Apple Silicon.

Automatic model selection

At startup the tool detects the machine's unified memory and automatically selects the best models for the available RAM. The detected memory and chosen models are printed before any work begins.

RAM	STT model	Summarization model
8 GB	whisper-large-v3-turbo (mlx-whisper)	Qwen3-4B-4bit
16 GB	Voxtral-Mini-4B-Realtime-2602-4bit	Qwen3-8B-4bit
24 GB	Voxtral-Mini-4B-Realtime-2602-fp16	Qwen3-8B-8bit
32 GB+	Voxtral-Mini-4B-Realtime-2602-fp16	Qwen3-30B-A3B-4bit (MoE)

The selection table lives in src/audio_summary/config.py and can be edited without touching the core logic. The highest tier whose RAM requirement fits the detected memory is selected.

Note

Models are downloaded from the Hugging Face Hub on first use and cached locally (~/.cache/huggingface). The first run for a given model may take a while.

Features

• YouTube Integration: Download and summarize content directly from YouTube.
• Local File Support: Summarize audio/video files available on your local disk.
• Transcription: Converts audio to text with Voxtral (or Whisper fallback).
• Summarization: Generates a concise summary with Qwen3.
• Transcript Only Option: Only transcribe, without generating a summary.
• Automatic model selection: Picks the best models for your Mac's unified memory.

Prerequisites

• An Apple Silicon Mac (M1 or newer)
• Python 3.12 (and lower than 3.13)
• ffmpeg (required for audio extraction/processing)
• uv for package management

Install ffmpeg with Homebrew:

brew install ffmpeg

Installation

Clone the repository and install dependencies with uv:

git clone https://github.com/damienarnodo/audio-summary-with-local-LLM.git
cd audio-summary-with-local-LLM

uv sync
source .venv/bin/activate  # activate the virtual environment

Usage with uvx (no installation required)

You can run the tool directly without cloning the repository using uvx:

# Summarize a YouTube video
uvx --from git+https://github.com/damienarnodo/audio-summary-with-local-LLM.git audio-summary --from-youtube <YouTube-Video-URL>

# Summarize a local audio file
uvx --from git+https://github.com/damienarnodo/audio-summary-with-local-LLM.git audio-summary --from-local <path-to-audio-file>

# Transcribe only (no summary)
uvx --from git+https://github.com/damienarnodo/audio-summary-with-local-LLM.git audio-summary --from-youtube <YouTube-Video-URL> --transcript-only

# Specify language and output file
uvx --from git+https://github.com/damienarnodo/audio-summary-with-local-LLM.git audio-summary --from-local <path-to-audio-file> --language fr --output my_summary.md

Usage

The CLI options are:

• --from-youtube: Download and summarize a video from YouTube.
• --from-local: Load and summarize a local audio or video file.
• --output: Output markdown file path (default: ./summary.md).
• --transcript-only: Only transcribe, do not summarize.
• --language: Language code for transcription (e.g. en, fr, es, or auto for automatic detection). Default: en.

Examples

# Summarize a YouTube video
audio-summary --from-youtube <YouTube-Video-URL>

# Summarize a local audio file
audio-summary --from-local <path-to-audio-file>

# Transcribe only (no summary)
audio-summary --from-youtube <YouTube-Video-URL> --transcript-only

# Custom language and output file
audio-summary --from-local <path-to-audio-file> --language fr --output my_summary.md

When running from a clone you can also use uv run audio-summary ... or uv run python -m audio_summary.cli ....

Output

The summary is written to a markdown file (default: summary.md) in the current working directory, with a title and concise summary. The transcript is saved to tmp/transcript.txt.

Project structure

The code is split into focused modules under src/audio_summary/:

Module	Responsibility
config.py	Model selection table (RAM tiers → STT/summarization).
device.py	Unified-memory detection and tier selection (psutil).
download.py	YouTube audio download (yt-dlp + ffmpeg).
transcription.py	Speech-to-Text (Voxtral via mlx-audio, Whisper fallback).
summarization.py	Summarization with Qwen3 via mlx-lm.
cli.py	Argument parsing and orchestration (entrypoint).

Customizing the models

Edit the TIERS table in src/audio_summary/config.py. Each tier declares a minimum RAM threshold, an STT model (engine + Hugging Face repo), and a summarization repo. For example, to use a different Qwen3 size or a different Voxtral quantization, just change the relevant repo string — no other code changes needed.

Troubleshooting

ffmpeg not found

If you encounter:

yt_dlp.utils.DownloadError: ERROR: Postprocessing: ffprobe and ffmpeg not found.

Install ffmpeg with brew install ffmpeg.

Out of memory

If a model is too large for your machine, lower the relevant tier's model in config.py (e.g. switch a fp16 STT model to its 4bit variant, or pick a smaller Qwen3 size). The smallest tier uses the lightweight mlx-whisper engine.

Sources

• MLX — Apple's array framework for Apple Silicon
• mlx-audio — TTS/STT on MLX (Voxtral)
• mlx-lm — LLM inference on MLX (Qwen3)
• mlx-whisper — Whisper on MLX
• yt-dlp — YouTube downloader