Update project to use MLX for Apple Silicon support

Update dependencies for MLX models Add automatic model selection based on available RAM Rewrite README with MLX-specific documentation Update .gitignore for MLX project structure
2026-06-17 09:00:35 +02:00
parent bda2b516a4
commit 47f2eda4f6
12 changed files with 994 additions and 743 deletions
--- a/.gitignore
+++ b/.gitignore
@@ -1,6 +1,7 @@
-src/__pycache__
+__pycache__/
 *.pyc
 .ruff_cache/
-src/audio_summary_with_local_LLM.egg-info/
+*.egg-info/
 # Virtual Env
 .venv
--- a/README.md
+++ b/README.md
@@ -1,60 +1,77 @@
-# Audio Summary with Local LLM
+# Audio Summary with Local LLM (Apple Silicon / MLX)
-This tool is designed to provide a quick and concise summary of audio and video files. It supports summarizing content either from a local file or directly from YouTube. The tool uses Whisper for transcription and a local version of Llama3 (via Ollama) for generating summaries.
+This tool provides a quick and concise summary of audio and video files. It supports
 summarizing content either from a local file or directly from YouTube. Everything runs
 **fully on Apple Silicon** through Apple's [MLX](https://github.com/ml-explore/mlx)
 framework — no NVIDIA/CUDA, no PyTorch CUDA stack, no external server required.
-> [!TIP]  
+- **Speech-to-Text**: Mistral [Voxtral](https://huggingface.co/mistralai) via
-> It is possible to change the model you wish to use.
+  [`mlx-audio`](https://github.com/Blaizzy/mlx-audio) (with an
-> To do this, change the `OLLAMA_MODEL` variable, and download the associated model via [ollama](https://github.com/ollama/ollama)
+  [`mlx-whisper`](https://github.com/ml-explore/mlx-examples) fallback for
  memory-constrained Macs). Voxtral outperforms Whisper large-v3 on most transcription
  benchmarks and natively handles long audio with streaming.
 - **Summarization**: [Qwen3](https://huggingface.co/Qwen) via
  [`mlx-lm`](https://github.com/ml-explore/mlx-lm) — the recommended family for
  summarization on Apple Silicon.
 ## Automatic model selection
 At startup the tool detects the machine's **unified memory** and automatically selects
 the best models for the available RAM. The detected memory and chosen models are printed
 before any work begins.
 | RAM     | STT model                                  | Summarization model       |
 |---------|--------------------------------------------|---------------------------|
 | 8 GB    | `whisper-large-v3-turbo` (mlx-whisper)     | `Qwen3-4B-4bit`           |
 | 16 GB   | `Voxtral-Mini-4B-Realtime-2602-4bit`       | `Qwen3-8B-4bit`           |
 | 24 GB   | `Voxtral-Mini-4B-Realtime-2602-fp16`       | `Qwen3-8B-8bit`           |
 | 32 GB+  | `Voxtral-Mini-4B-Realtime-2602-fp16`       | `Qwen3-30B-A3B-4bit` (MoE)|
 The selection table lives in [`src/audio_summary/config.py`](src/audio_summary/config.py)
 and can be edited without touching the core logic. The highest tier whose RAM requirement
 fits the detected memory is selected.
 > [!NOTE]
 > Models are downloaded from the Hugging Face Hub on first use and cached locally
 > (`~/.cache/huggingface`). The first run for a given model may take a while.
 ## Features
 - **YouTube Integration**: Download and summarize content directly from YouTube.
 - **Local File Support**: Summarize audio/video files available on your local disk.
- **Transcription**: Converts audio content to text using Whisper.
+- **Transcription**: Converts audio to text with Voxtral (or Whisper fallback).
- **Summarization**: Generates a concise summary using Llama3 (Ollama).
+- **Summarization**: Generates a concise summary with Qwen3.
- **Transcript Only Option**: Option to only transcribe the audio content without generating a summary.
+- **Transcript Only Option**: Only transcribe, without generating a summary.
- **Device Optimization**: Automatically uses the best available hardware (MPS for Mac, CUDA for NVIDIA GPUs, or CPU).
+- **Automatic model selection**: Picks the best models for your Mac's unified memory.
 ## Prerequisites
-Before you start using this tool, you need to install the following dependencies:
+- An Apple Silicon Mac (M1 or newer)
-
+- Python 3.12 (and lower than 3.13)
- Python 3.12 and lower than 3.13
+- `ffmpeg` (required for audio extraction/processing)
 - [Ollama](https://ollama.com) for LLM model management
 - `ffmpeg` (required for audio processing)
 - [uv](https://docs.astral.sh/uv/getting-started/installation/) for package management
 Install `ffmpeg` with [Homebrew](https://brew.sh):
 ```bash
 brew install ffmpeg
 ```
 ## Installation
-### Using uv
+Clone the repository and install dependencies with [uv](https://github.com/astral-sh/uv):
 Clone the repository and install the required Python packages using [uv](https://github.com/astral-sh/uv):
 ```bash
 git clone https://github.com/damienarnodo/audio-summary-with-local-LLM.git
 cd audio-summary-with-local-LLM
 # Create and activate a virtual environment with uv
 uv sync
-source .venv/bin/activate  # On Windows: .venv\Scripts\activate
+source .venv/bin/activate  # activate the virtual environment
 ```
 ### LLM Requirement
 [Download and install](https://ollama.com) Ollama to carry out LLM Management. More details about LLM models supported can be found on the Ollama [GitHub](https://github.com/ollama/ollama).
 Download and use the Llama3 model:
 ```bash
 ollama pull llama3
 # Test the access:
 ollama run llama3 "tell me a joke"
 ```
 ## Usage with uvx (no installation required)
-You can run this tool directly without cloning the repository using `uvx`:
+You can run the tool directly without cloning the repository using `uvx`:
 ```bash
 # Summarize a YouTube video
@@ -70,172 +87,84 @@ uvx --from git+https://github.com/damienarnodo/audio-summary-with-local-LLM.git
 uvx --from git+https://github.com/damienarnodo/audio-summary-with-local-LLM.git audio-summary --from-local <path-to-audio-file> --language fr --output my_summary.md
 ```
 > [!NOTE]
 > `uvx` automatically creates a temporary virtual environment and installs all dependencies. On first run, this may take a moment. Ensure `ffmpeg` and [Ollama](https://ollama.com) are installed and running on your system.
 ## Usage
-The tool can be executed with the following command line options:
+The CLI options are:
- `--from-youtube`: To download and summarize a video from YouTube.
+- `--from-youtube`: Download and summarize a video from YouTube.
- `--from-local`: To load and summarize an audio or video file from the local disk.
+- `--from-local`: Load and summarize a local audio or video file.
- `--output`: Specify the output file path (default: ./summary.md)
+- `--output`: Output markdown file path (default: `./summary.md`).
- `--transcript-only`: To only transcribe the audio content without generating a summary.
+- `--transcript-only`: Only transcribe, do not summarize.
- `--language`: Select the language to be used for the transcription (default: en)
+- `--language`: Language code for transcription (e.g. `en`, `fr`, `es`, or `auto` for
  automatic detection). Default: `en`.
 ### Examples
-1. **Summarizing a YouTube video:**
+```bash
 # Summarize a YouTube video
 audio-summary --from-youtube <YouTube-Video-URL>
-   ```bash
+# Summarize a local audio file
-   uv run python src/summary.py --from-youtube <YouTube-Video-URL>
+audio-summary --from-local <path-to-audio-file>
   ```
-2. **Summarizing a local audio file:**
+# Transcribe only (no summary)
 audio-summary --from-youtube <YouTube-Video-URL> --transcript-only
-   ```bash
+# Custom language and output file
-   uv run python src/summary.py --from-local <path-to-audio-file>
+audio-summary --from-local <path-to-audio-file> --language fr --output my_summary.md
-   ```
+```
-3. **Transcribing a YouTube video without summarizing:**
+When running from a clone you can also use `uv run audio-summary ...` or
-
+`uv run python -m audio_summary.cli ...`.
   ```bash
   uv run python src/summary.py --from-youtube <YouTube-Video-URL> --transcript-only
   ```
 4. **Transcribing a local audio file without summarizing:**
   ```bash
   uv run python src/summary.py --from-local <path-to-audio-file> --transcript-only
   ```
 5. **Specifying a custom output file:**
   ```bash
   uv run python src/summary.py --from-youtube <YouTube-Video-URL> --output my_summary.md
   ```
 The output summary will be saved in a markdown file in the specified output directory, while the transcript will be saved in the temporary directory.
 ## Output
-The summarized content is saved as a markdown file (default: `summary.md`) in the current working directory. This file includes a title and a concise summary of the content. The transcript is saved in the `tmp/transcript.txt` file.
+The summary is written to a markdown file (default: `summary.md`) in the current working
 directory, with a title and concise summary. The transcript is saved to
 `tmp/transcript.txt`.
-## Hardware Acceleration
+## Project structure
-The tool automatically detects and uses the best available hardware:
+The code is split into focused modules under `src/audio_summary/`:
- MPS (Metal Performance Shaders) for Apple Silicon Macs
+| Module             | Responsibility                                            |
- CUDA for NVIDIA GPUs
+|--------------------|-----------------------------------------------------------|
- Falls back to CPU when neither is available
+| `config.py`        | Model selection table (RAM tiers → STT/summarization).    |
 | `device.py`        | Unified-memory detection and tier selection (`psutil`).   |
 | `download.py`      | YouTube audio download (`yt-dlp` + `ffmpeg`).             |
 | `transcription.py` | Speech-to-Text (Voxtral via `mlx-audio`, Whisper fallback).|
 | `summarization.py` | Summarization with Qwen3 via `mlx-lm`.                    |
 | `cli.py`           | Argument parsing and orchestration (entrypoint).          |
-### Handling Longer Audio Files
+## Customizing the models
-This tool can process audio files of any length. For files longer than 30 seconds, the script automatically:
+Edit the `TIERS` table in [`src/audio_summary/config.py`](src/audio_summary/config.py).
-
+Each tier declares a minimum RAM threshold, an STT model (engine + Hugging Face repo),
-1. Chunks the audio into manageable segments
+and a summarization repo. For example, to use a different Qwen3 size or a different
-2. Processes each chunk separately
+Voxtral quantization, just change the relevant repo string — no other code changes needed.
 3. Combines the results into a single transcript
 ## Sources
 - [YouTube Video Summarizer with OpenAI Whisper and GPT](https://github.com/mirabdullahyaser/Summarizing-Youtube-Videos-with-OpenAI-Whisper-and-GPT-3/tree/master)
 - [Ollama GitHub Repository](https://github.com/ollama/ollama)
 - [Transformers by Hugging Face](https://huggingface.co/docs/transformers/index)
 - [yt-dlp Documentation](https://github.com/yt-dlp/yt-dlp)
 ## Troubleshooting
 ### ffmpeg not found
-If you encounter this error::
+If you encounter:
 ```bash
-yt_dlp.utils.DownloadError: ERROR: Postprocessing: ffprobe and ffmpeg not found. Please install or provide the path using --ffmpeg-location
+yt_dlp.utils.DownloadError: ERROR: Postprocessing: ffprobe and ffmpeg not found.
 ```
-Please refer to [this post](https://www.reddit.com/r/StacherIO/wiki/ffmpeg/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button)
+Install ffmpeg with `brew install ffmpeg`.
-### Audio Format Issues
+### Out of memory
-If you encounter this error:
+If a model is too large for your machine, lower the relevant tier's model in
 `config.py` (e.g. switch a `fp16` STT model to its `4bit` variant, or pick a smaller
 Qwen3 size). The smallest tier uses the lightweight `mlx-whisper` engine.
-```bash
+## Sources
 ValueError: Soundfile is either not in the correct format or is malformed. Ensure that the soundfile has a valid audio file extension (e.g. wav, flac or mp3) and is not corrupted.
 ```
-Try converting your file with ffmpeg:
+- [MLX](https://github.com/ml-explore/mlx) — Apple's array framework for Apple Silicon
-
+- [mlx-audio](https://github.com/Blaizzy/mlx-audio) — TTS/STT on MLX (Voxtral)
-```bash
+- [mlx-lm](https://github.com/ml-explore/mlx-lm) — LLM inference on MLX (Qwen3)
-ffmpeg -i my_file.mp4 -movflags faststart my_file_fixed.mp4
+- [mlx-whisper](https://github.com/ml-explore/mlx-examples) — Whisper on MLX
-```
+- [yt-dlp](https://github.com/yt-dlp/yt-dlp) — YouTube downloader
 ### Memory Issues on CPU
 If you're running on CPU and encounter memory issues during transcription, consider:
 1. Using a smaller Whisper model
 2. Processing shorter audio segments
 3. Ensuring you have sufficient RAM available
 ### Slow Transcription
 Transcription can be slow on CPU. For best performance:
 1. Use a machine with GPU or Apple Silicon (MPS)
 2. Keep audio files under 10 minutes when possible
 3. Close other resource-intensive applications
 ### Update the Whisper or LLM Model
 You can easily change the models used for transcription and summarization by modifying the variables at the top of the script:
 ```python
 # Default models
 OLLAMA_MODEL = "llama3"
 WHISPER_MODEL = "openai/whisper-large-v2"
 ```
 #### Changing the Whisper Model
 To use a different Whisper model for transcription:
 1. Update the `WHISPER_MODEL` variable with one of these options:
   - `"openai/whisper-tiny"` (fastest, least accurate)
   - `"openai/whisper-base"` (faster, less accurate)
   - `"openai/whisper-small"` (balanced)
   - `"openai/whisper-medium"` (slower, more accurate)
   - `"openai/whisper-large-v2"` (slowest, most accurate)
 2. Example:
   ```python
   WHISPER_MODEL = "openai/whisper-medium"  # A good balance between speed and accuracy
   ```
 For CPU-only systems, using a smaller model like `whisper-base` is recommended for better performance.
 #### Changing the LLM Model
 To use a different model for summarization:
 1. First, pull the desired model with Ollama:
   ```bash
   ollama pull mistral  # or any other supported model
   ```
 2. Then update the `OLLAMA_MODEL` variable:
   ```python
   OLLAMA_MODEL = "mistral"  # or any other model you've pulled
   ```
 3. Popular alternatives include:
   - `"llama3"` (default)
   - `"mistral"`
   - `"llama2"`
   - `"gemma:7b"`
   - `"phi"`
 For a complete list of available models, visit the [Ollama model library](https://ollama.com/library).
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -4,32 +4,26 @@ build-backend = "setuptools.build_meta"
 [project]
 name = "audio-summary-with-local-LLM"
-version = "0.1.0"
+version = "0.2.0"
-description = 'Sum up your local or remote files with a local LLM'
+description = 'Transcribe and summarize audio/video files fully on Apple Silicon via MLX'
-keywords = ["audio", "summary", "local-llm", "ollama", "whisper"]
+keywords = ["audio", "summary", "local-llm", "mlx", "apple-silicon", "voxtral", "qwen3"]
 readme = "README.md"
 requires-python = ">=3.12, <3.13"
 authors = [
    { name = "darnodo", email = "sepales.pret0h@icloud.com" },
 ]
 dependencies = [
-    "ffmpeg>=1.4",
+    "mlx-lm>=0.21.0",
-    "ollama>=0.4.7",
+    "mlx-audio>=0.2.0",
-    "openai-whisper>=20240930",
+    "mlx-whisper>=0.4.0",
-    "torch>=2.6.0",
+    "psutil>=6.0.0",
    "torchaudio>=2.6.0",
    "torchvision>=0.21.0",
    "transformers>=4.50.2",
    "yt-dlp>=2025.3.27",
 ]
 [project.scripts]
-audio-summary = "summary:main"
+audio-summary = "audio_summary.cli:main"
-[tool.setuptools]
+[tool.setuptools.packages.find]
-py-modules = ["summary"]
+where = ["src"]
 [tool.setuptools.package-dir]
 "" = "src"
 [tool.ruff]
 # Exclude a variety of commonly ignored directories.
@@ -66,8 +60,8 @@ exclude = [
 line-length = 88
 indent-width = 4
-# Assume Python 3.8
+# Match the project's required Python version.
-target-version = "py38"
+target-version = "py312"
 [tool.ruff.lint]
 # Enable Pyflakes (`F`) and a subset of the pycodestyle (`E`)  codes by default.
--- a/src/audio_summary/init.py
+++ b/src/audio_summary/init.py
@@ -0,0 +1,3 @@
 """Audio Summary with local MLX models on Apple Silicon."""
 __version__ = "0.2.0"
--- a/src/audio_summary/cli.py
+++ b/src/audio_summary/cli.py
@@ -0,0 +1,102 @@
 """Command-line entrypoint.
 Download (optional), transcribe, and summarize audio/video files fully on
 Apple Silicon via MLX. Models are selected automatically from the detected
 unified memory (see ``config`` and ``device``).
 """
 import argparse
 from pathlib import Path
 from .device import select_models
 from .download import download_from_youtube
 from .summarization import summarize_text
 from .transcription import transcribe_file
 DEFAULT_LANGUAGE = "en"  # Use "auto" on the CLI for automatic detection.
 def _resolve_language(arg_language: str | None) -> str | None:
    """Map the CLI ``--language`` value to a code or ``None`` (auto-detect)."""
    language = arg_language if arg_language else DEFAULT_LANGUAGE
    if language and language.lower() == "auto":
        return None
    return language
 def main() -> None:
    parser = argparse.ArgumentParser(
        description="Download, transcribe, and summarize audio or video files "
        "on Apple Silicon (MLX)."
    )
    group = parser.add_mutually_exclusive_group(required=True)
    group.add_argument("--from-youtube", type=str, help="YouTube URL to download.")
    group.add_argument("--from-local", type=str, help="Path to the local audio file.")
    parser.add_argument(
        "--output",
        type=str,
        default="./summary.md",
        help="Output markdown file path.",
    )
    parser.add_argument(
        "--transcript-only",
        action="store_true",
        help="Only transcribe the file, do not summarize.",
    )
    parser.add_argument(
        "--language",
        type=str,
        help="Language code for transcription (e.g. 'en', 'fr', 'es', "
        "or 'auto' for detection).",
    )
    args = parser.parse_args()
    language = _resolve_language(args.language)
    # --- Automatic model selection based on detected unified memory ---
    ram_gb, tier = select_models()
    print("=" * 60)
    print(f"Detected unified memory : {ram_gb:.1f} GB")
    print(f"Selected hardware tier  : >= {tier.min_ram_gb} GB")
    print(f"STT model               : {tier.stt.repo} ({tier.stt.engine})")
    print(f"Summarization model     : {tier.summarization_repo}")
    print("=" * 60)
    # --- Working directory for intermediate files ---
    data_directory = Path("tmp")
    if not data_directory.exists():
        data_directory.mkdir(parents=True)
        print(f"Created directory: {data_directory}")
    # --- Resolve input file ---
    if args.from_youtube:
        print(f"Downloading YouTube video from {args.from_youtube}")
        file_path = download_from_youtube(args.from_youtube, str(data_directory))
    else:
        file_path = Path(args.from_local)
    # --- Transcription ---
    print(f"Transcribing file: {file_path}")
    transcript = transcribe_file(
        str(file_path),
        str(data_directory / "transcript.txt"),
        tier.stt,
        language,
    )
    if args.transcript_only:
        print("Transcription complete. Skipping summary generation.")
        return
    # --- Summarization ---
    print("Generating summary...")
    summary = summarize_text(transcript, tier.summarization_repo)
    with open(args.output, "w") as md_file:
        md_file.write("# Summary\n\n")
        md_file.write(summary)
    print(f"Summary written to {args.output}")
 if __name__ == "__main__":
    main()
--- a/src/audio_summary/config.py
+++ b/src/audio_summary/config.py
@@ -0,0 +1,58 @@
 """Model configuration for Apple Silicon (MLX).
 This table maps the amount of unified memory detected at runtime to the best
 Speech-to-Text (STT) and summarization models. It is intentionally kept in its
 own module so the model line-up can be tuned without touching the core logic.
 Each tier declares a ``min_ram_gb`` threshold. At runtime the highest tier whose
 threshold is *less than or equal to* the detected memory is selected (see
 ``audio_summary.device.select_models``).
 STT engines:
    * ``"voxtral"``  -> Mistral Voxtral via ``mlx-audio`` (streaming, long audio).
    * ``"whisper"``  -> ``mlx-whisper`` (lightweight fallback for low-RAM Macs).
 """
 from dataclasses import dataclass
@dataclass(frozen=True)
 class STTModel:
    """A Speech-to-Text model definition."""
    engine: str  # "voxtral" or "whisper"
    repo: str  # Hugging Face repository id
@dataclass(frozen=True)
 class Tier:
    """A hardware tier: minimum RAM and the models it unlocks."""
    min_ram_gb: int
    stt: STTModel
    summarization_repo: str
 # Ordered from the smallest to the largest hardware tier.
 TIERS: list[Tier] = [
    Tier(
        min_ram_gb=8,
        stt=STTModel("whisper", "mlx-community/whisper-large-v3-turbo"),
        summarization_repo="mlx-community/Qwen3-4B-4bit",
    ),
    Tier(
        min_ram_gb=16,
        stt=STTModel("voxtral", "mlx-community/Voxtral-Mini-4B-Realtime-2602-4bit"),
        summarization_repo="mlx-community/Qwen3-8B-4bit",
    ),
    Tier(
        min_ram_gb=24,
        stt=STTModel("voxtral", "mlx-community/Voxtral-Mini-4B-Realtime-2602-fp16"),
        summarization_repo="mlx-community/Qwen3-8B-8bit",
    ),
    Tier(
        min_ram_gb=32,
        stt=STTModel("voxtral", "mlx-community/Voxtral-Mini-4B-Realtime-2602-fp16"),
        summarization_repo="mlx-community/Qwen3-30B-A3B-4bit",
    ),
 ]
--- a/src/audio_summary/device.py
+++ b/src/audio_summary/device.py
@@ -0,0 +1,32 @@
 """Hardware detection and automatic model selection.
 Detects the amount of unified memory available on the Apple Silicon machine and
 picks the most capable model tier that fits (see ``config.TIERS``).
 """
 import psutil
 from .config import TIERS, Tier
 def detect_ram_gb() -> float:
    """Return the total unified memory in gibibytes."""
    return psutil.virtual_memory().total / (1024**3)
 def select_tier(ram_gb: float) -> Tier:
    """Select the highest tier whose RAM requirement fits ``ram_gb``.
    Falls back to the smallest tier if the machine has less memory than any
    declared threshold (the models simply run slower / may swap).
    """
    eligible = [tier for tier in TIERS if tier.min_ram_gb <= ram_gb]
    if eligible:
        return max(eligible, key=lambda tier: tier.min_ram_gb)
    return min(TIERS, key=lambda tier: tier.min_ram_gb)
 def select_models() -> tuple[float, Tier]:
    """Detect memory and return ``(ram_gb, selected_tier)``."""
    ram_gb = detect_ram_gb()
    return ram_gb, select_tier(ram_gb)
--- a/src/audio_summary/download.py
+++ b/src/audio_summary/download.py
@@ -0,0 +1,27 @@
 """YouTube audio download helper (yt-dlp + ffmpeg)."""
 from pathlib import Path
 import yt_dlp
 def download_from_youtube(url: str, path: str) -> Path:
    """Download the best audio track from ``url`` as an mp3 into ``path``.
    Returns the path to the resulting ``to_transcribe.mp3`` file.
    """
    ydl_opts = {
        "format": "bestaudio/best",
        "outtmpl": str(Path(path) / "to_transcribe.%(ext)s"),
        "postprocessors": [
            {
                "key": "FFmpegExtractAudio",
                "preferredcodec": "mp3",
                "preferredquality": "192",
            }
        ],
    }
    with yt_dlp.YoutubeDL(ydl_opts) as ydl:
        ydl.download([url])
    return Path(path) / "to_transcribe.mp3"
--- a/src/audio_summary/summarization.py
+++ b/src/audio_summary/summarization.py
@@ -0,0 +1,39 @@
 """Text summarization with Qwen3 on Apple Silicon via mlx-lm."""
 from mlx_lm import generate, load
 SYSTEM_PROMPT = "I would like for you to assume the role of a Technical Expert."
 USER_PROMPT_TEMPLATE = """Generate a concise summary of the text below.
 Text : {text}
 Add a title to the summary.
 Make sure your summary has useful and true information about the main points of the topic.
 Begin with a short introduction explaining the topic. If you can, use bullet points to list important details,
 and finish your summary with a concluding sentence."""
 def summarize_text(text: str, model_repo: str, max_tokens: int = 2048) -> str:
    """Summarize ``text`` using the Qwen3 model at ``model_repo``."""
    print(f"Loading summarization model: {model_repo}")
    model, tokenizer = load(model_repo)
    messages = [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": USER_PROMPT_TEMPLATE.format(text=text)},
    ]
    # Qwen3 supports a "thinking" mode; disable it for clean, direct summaries.
    prompt = tokenizer.apply_chat_template(
        messages,
        add_generation_prompt=True,
        enable_thinking=False,
    )
    summary = generate(
        model,
        tokenizer,
        prompt=prompt,
        max_tokens=max_tokens,
        verbose=False,
    )
    return summary.strip()
--- a/src/audio_summary/transcription.py
+++ b/src/audio_summary/transcription.py
@@ -0,0 +1,68 @@
 """Speech-to-Text on Apple Silicon.
 Two engines are supported, selected through ``config.STTModel.engine``:
 * ``voxtral`` -> Mistral Voxtral via ``mlx-audio``. Outperforms Whisper
  large-v3 on most benchmarks and natively handles long audio with streaming.
 * ``whisper`` -> ``mlx-whisper`` fallback for memory-constrained Macs.
 """
 from .config import STTModel
 def _transcribe_voxtral(file_path: str, model_repo: str, language: str | None) -> str:
    """Transcribe using Mistral Voxtral through mlx-audio."""
    from mlx_audio.stt.utils import load
    model = load(model_repo)
    # Voxtral handles long audio natively; ``generate`` returns an object with a
    # ``.text`` attribute. ``language`` is optional (auto-detected when None).
    kwargs: dict = {}
    if language:
        kwargs["language"] = language
    result = model.generate(file_path, **kwargs)
    return result.text.strip()
 def _transcribe_whisper(file_path: str, model_repo: str, language: str | None) -> str:
    """Transcribe using the mlx-whisper fallback engine."""
    import mlx_whisper
    result = mlx_whisper.transcribe(
        file_path,
        path_or_hf_repo=model_repo,
        language=language,  # None triggers auto-detection
    )
    return result["text"].strip()
 def transcribe_file(
    file_path: str,
    output_file: str,
    stt_model: STTModel,
    language: str | None = None,
 ) -> str:
    """Transcribe ``file_path`` and persist the transcript to ``output_file``.
    ``language`` may be a language code (e.g. ``"en"``, ``"fr"``) or ``None`` for
    automatic detection. Returns the full transcript text.
    """
    print(f"Transcribing with {stt_model.engine} model: {stt_model.repo}")
    if language:
        print(f"Transcription language: {language}")
    else:
        print("Using automatic language detection")
    print("Starting transcription (this may take a while for longer files)...")
    if stt_model.engine == "voxtral":
        text = _transcribe_voxtral(file_path, stt_model.repo, language)
    elif stt_model.engine == "whisper":
        text = _transcribe_whisper(file_path, stt_model.repo, language)
    else:
        raise ValueError(f"Unknown STT engine: {stt_model.engine!r}")
    with open(output_file, "w") as tmp_file:
        tmp_file.write(text)
    print(f"Transcription saved to file: {output_file}")
    return text
--- a/src/summary.py
+++ b/src/summary.py
@@ -1,167 +0,0 @@
 import ollama
 import argparse
 from pathlib import Path
 from transformers import pipeline
 import yt_dlp
 import torch
 OLLAMA_MODEL = "llama3"
 WHISPER_MODEL = "openai/whisper-large-v2"
 WHISPER_LANGUAGE = "en"  # Set to desired language or None for auto-detection
 # Function to download a video from YouTube using yt-dlp
 def download_from_youtube(url: str, path: str):
    ydl_opts = {
        'format': 'bestaudio/best',
        'outtmpl': str(Path(path) / 'to_transcribe.%(ext)s'),
        'postprocessors': [{
            'key': 'FFmpegExtractAudio',
            'preferredcodec': 'mp3',
            'preferredquality': '192',
        }],
    }
    with yt_dlp.YoutubeDL(ydl_opts) as ydl:
        ydl.download([url])
 # Function to get the best available device
 def get_device():
    if torch.backends.mps.is_available():
        return "mps"
    elif torch.cuda.is_available():
        return "cuda"
    else:
        return "cpu"
 # Function to transcribe an audio file using the transformers pipeline
 def transcribe_file(file_path: str, output_file: str, language: str = None) -> str:
    # Get the best available device
    device = get_device()
    print(f"Using device: {device} for transcription")
    # Load the pipeline model for automatic speech recognition
    transcriber = pipeline(
        "automatic-speech-recognition", 
        model=WHISPER_MODEL, 
        device=device,
        chunk_length_s=30,  # Process in 30-second chunks
        return_timestamps=True  # Enable timestamp generation for longer audio
    )
    # Transcribe the audio file
    # For CPU, we might want to use a smaller model or chunk the audio if memory is an issue
    if device == "cpu":
        print("Warning: Using CPU for transcription. This may be slow.")
    # Set up generation keyword arguments including language
    generate_kwargs = {}
    if language and language.lower() != "auto":
        generate_kwargs["language"] = language
        print(f"Transcribing in language: {language}")
    else:
        print("Using automatic language detection")
    # Transcribe the audio file
    print("Starting transcription (this may take a while for longer files)...")
    transcribe = transcriber(file_path, generate_kwargs=generate_kwargs)
    # Extract the full text from the chunked transcription
    if isinstance(transcribe, dict) and "text" in transcribe:
        # Simple case - just one chunk
        full_text = transcribe["text"]
    elif isinstance(transcribe, dict) and "chunks" in transcribe:
        # Multiple chunks with timestamps
        full_text = " ".join([chunk["text"] for chunk in transcribe["chunks"]])
    else:
        # Fallback for other return formats
        full_text = transcribe["text"] if "text" in transcribe else str(transcribe)
    # Save the transcribed text to the specified temporary file
    with open(output_file, 'w') as tmp_file:
        tmp_file.write(full_text)
        print(f"Transcription saved to file: {output_file}")
    # Return the transcribed text
    return full_text
 # Function to summarize a text using the Ollama model
 def summarize_text(text: str, output_path: str) -> str:
    # Define the system prompt for the Ollama model
    system_prompt = "I would like for you to assume the role of a Technical Expert"
    # Define the user prompt for the Ollama model
    user_prompt = f"""Generate a concise summary of the text below.
    Text : {text}
    Add a title to the summary.
    Make sure your summary has useful and true information about the main points of the topic.
    Begin with a short introduction explaining the topic. If you can, use bullet points to list important details,
    and finish your summary with a concluding sentence."""
    # Use the Ollama model to generate a summary
    response = ollama.chat(
        model=OLLAMA_MODEL,
        messages=[
            {
                "role": "system",
                "content": system_prompt,
            },
            {
                "role": "user",
                "content": user_prompt,
            },
        ],
    )
    # Print the generated summary
    return response["message"]["content"]
 def main():
    # Parse command line arguments
    parser = argparse.ArgumentParser(description="Download, transcribe, and summarize audio or video files.")
    group = parser.add_mutually_exclusive_group(required=True)
    group.add_argument("--from-youtube", type=str, help="YouTube URL to download.")
    group.add_argument("--from-local", type=str, help="Path to the local audio file.")
    parser.add_argument("--output", type=str, default="./summary.md", help="Output markdown file path.")
    parser.add_argument("--transcript-only", action='store_true', help="Only transcribe the file, do not summarize.")
    parser.add_argument("--language", type=str, help="Language code for transcription (e.g., 'en', 'fr', 'es', or 'auto' for detection)")
    args = parser.parse_args()
    # Determine language setting
    language = args.language if args.language else WHISPER_LANGUAGE
    if language and language.lower() == "auto":
        language = None  # None triggers automatic language detection
    # Set up data directory
    data_directory = Path("tmp")
    # Check if the directory exists, if not, create it
    if not data_directory.exists():
        data_directory.mkdir(parents=True)
        print(f"Created directory: {data_directory}")
    if args.from_youtube:
        # Download from YouTube
        print(f"Downloading YouTube video from {args.from_youtube}")
        download_from_youtube(args.from_youtube, str(data_directory))
        file_path = data_directory / "to_transcribe.mp3"
    elif args.from_local:
        # Use local file
        file_path = Path(args.from_local)
    print(f"Transcribing file: {file_path}")
    # Transcribe the audio file
    transcript = transcribe_file(str(file_path), data_directory / "transcript.txt", language)
    if args.transcript_only:
        print("Transcription complete. Skipping summary generation.")
        return
    print("Generating summary...")
    # Generate summary
    summary = summarize_text(transcript, "./")
    # Write summary to a markdown file
    with open(args.output, "w") as md_file:
        md_file.write("# Summary\n\n")
        md_file.write(summary)
        print(f"Summary written to {args.output}")
 if __name__ == "__main__":
    main()
--- a/uv.lock
+++ b/uv.lock
		`@@ -0,0 +1,3 @@`
							`"""Audio Summary with local MLX models on Apple Silicon."""`

							`__version__ = "0.2.0"`