audio-tldr

Any video, audio, or podcast → key takeaways. Transcribed locally, cached forever.

English | 繁體中文

An agent skill — open SKILL.md standard, works in Claude Code and Codex — that turns long-form media into 3–7 key takeaways + a summary. Transcription runs locally with whisper and is cached by content hash — while a cache entry exists, the same source is not transcribed again (unless you --force). Ask for a different angle later and it re-digests from cache in seconds.

First-run transcription time depends on your hardware, model, and backend — after that, the cache answers.

Why?

Watching a 90-minute talk to extract 5 useful points is a bad trade. Sending audio to a cloud API costs money and leaks content. And summarizing the same episode twice — because the first summary had the wrong focus — means paying the transcription cost all over again.

Without audio-tldr                    With audio-tldr
──────────────────                    ───────────────
watch the whole video                 paste the URL
take notes by hand                    get takeaways + summary
"summarize it differently…"           re-digest from cache, instant
re-upload, re-transcribe, re-pay      transcribe once, reuse from cache

Features

• ✓ YouTube, podcasts, and any yt-dlp-supported URL — or local audio/video files
• ✓ Local media pipeline: download, transcription, cache all run on your machine — audio is never uploaded (see Privacy)
• ✓ Content-hash cache: re-summarizing (any angle) reuses the transcript while the entry exists
• ✓ Whisper backend auto-detection: mlx-whisper / faster-whisper / whisper.cpp / openai-whisper
• ✓ Language auto-detection; optional Simplified→Traditional Chinese conversion (OpenCC)
• ✓ Cache management built in: list, clear one, clear all, opt-in retention
• ✓ Timeline for long content (> 20 min)
• ✓ Digests saved to an output folder as Markdown or HTML — transcripts stay in the cache
• ✓ Conversational digest prompt: no request stated? The agent asks in plain text, listing the template menu
• ✓ Digest templates: meeting minutes, key summary, analysis report — or save your own reusable format
• ✓ Opt-in fully-local digest: digest_model: ollama:<model> runs Phase 2 through your own Ollama server — transcript text never leaves the machine
• ✓ Translation at the digest layer: digests in any language, or a faithful full-transcript translation
• ✓ Optional preferences file for standing habits — zero setup required
• ✓ Interpreter auto-selection: backend installed in another Python (e.g. Homebrew) is found and used automatically; --doctor diagnoses the environment
• ✓ Apple Podcasts fallback built in: when yt-dlp's extractor fails, episodes resolve via the iTunes lookup API — cache identity stays on your original link; a show link (no episode id) automatically uses the latest episode
• ✓ Opt-in frame extraction for video sources: scene-detection slide capture, or stills at exact timestamps — video fetched at ≤720p and deleted after extraction; frames share the transcript's cache entry
• ✓ Subtitle export: --format srt / --format vtt write a standard subtitle file with segment timestamps alongside the transcript, on any of the four backends
• ✓ Install by copy, as a Claude Code plugin, or into Codex (open SKILL.md standard)

Install

Option A — copy the skill (simplest):

git clone https://github.com/AugustusW/audio-tldr-skill.git
cp -r audio-tldr-skill/skills/audio-tldr ~/.claude/skills/

Invoke with /audio-tldr, or just ask Claude to summarize a video — it auto-triggers.

Option B — install as a plugin:

/plugin marketplace add AugustusW/audio-tldr-skill
/plugin install audio-tldr@audio-tldr-skill

Invoke with /audio-tldr:audio-tldr. Both options can coexist — plugin skills are namespaced.

Option C — Codex CLI / ChatGPT app:

The skill follows the open SKILL.md standard, so it works in Codex as-is. Copy the skill folder into Codex's skills directory:

git clone https://github.com/AugustusW/audio-tldr-skill.git
cp -r audio-tldr-skill/skills/audio-tldr ~/.codex/skills/audio-tldr        # personal
# or, per-project: cp -r audio-tldr-skill/skills/audio-tldr <repo>/.codex/skills/audio-tldr

Invoke it with a $audio-tldr mention, or let Codex pick it implicitly when you ask to summarize audio/video. The transcript cache (~/.cache/audio-tldr/) and the preferences file (~/.config/audio-tldr/preferences.md) are shared with Claude Code — transcribe once, digest anywhere.

Prerequisites

The media pipeline — download, transcription, cache — runs entirely on your machine.

Whisper backends, in the order the skill auto-detects them:

Local files don't need yt-dlp — only a whisper backend.

For URL sources, make sure you have the right to download and process the content, and comply with the source site's terms and your local copyright law.

Choosing a model

The default is large-v3-turbo on every backend (whisper.cpp excepted — its model is the AUDIO_TLDR_WHISPER_CPP_MODEL file). On CPU-only machines this favors quality over speed — drop to small if transcription is too slow. Override per run with --model, or persistently with AUDIO_TLDR_MODEL (the flag wins). Bare names are mapped per backend (mlx gets the mlx-community/whisper- prefix automatically; a full HF repo path is used as-is; named aliases map to the backend's community conversion — see below):

python3 scripts/transcribe.py --model small "<source>"   # per run
$env:AUDIO_TLDR_MODEL = "large-v3"    # persistent; PowerShell (bash/zsh: export AUDIO_TLDR_MODEL=large-v3)

The cache is keyed by source, not model — to re-transcribe an already-cached source with a different model, add --force.

Taiwanese Mandarin (breeze-asr-25): MediaTek Breeze-ASR-25 is a Whisper-large-v2 fine-tune optimized for Taiwanese Mandarin and Mandarin-English code-switching (Apache 2.0). The alias resolves to a community conversion per backend — mlx-whisper: eoleedi/Breeze-ASR-25-mlx, faster-whisper: SoybeanMilk/faster-whisper-Breeze-ASR-25 (~3 GB download on first use). There is no openai-whisper conversion — the script reports that instead of guessing. whisper.cpp users can point AUDIO_TLDR_WHISPER_CPP_MODEL at one of the community GGML conversions on Hugging Face.

Trade-offs, from one A/B run we did on a 52-minute Taiwanese-Mandarin finance podcast (Apple M4 Pro, mlx-whisper backend, single episode — not a benchmark suite): Breeze-ASR-25 was noticeably better on Taiwanese names, finance terms, and mixed zh/en passages, but ran about 3× slower than large-v3-turbo (≈5× realtime vs ≈15×, its backbone is the bigger large-v2), produced almost no punctuation, and emitted English words in lowercase. The default stays large-v3-turbo; reach for breeze-asr-25 when getting Taiwanese-Mandarin names and terminology right matters more than speed and punctuation.

Optional — Traditional Chinese: whisper often emits Simplified Chinese. pip install opencc and Chinese transcripts are converted to Taiwan Traditional automatically — including common-phrase localization (s2twp, e.g. 軟件→軟體) — plus the model is biased toward Traditional vocabulary. Not installed → transcripts are left as-is.

The conversion is applied per segment, and only to segments that actually contain Simplified characters. A segment already written in Traditional is passed through untouched, so words that are valid in both scripts (干, 里, 吃) and phrases the localization table would otherwise rewrite (文件 as "document") keep what the speaker said.

Windows notes

Windows is supported by the underlying Python stack, but the full flow has not yet been verified on Windows — reports welcome. Install with PowerShell:

# prerequisites (winget shown; Chocolatey: choco install ffmpeg yt-dlp)
winget install Gyan.FFmpeg
winget install yt-dlp.yt-dlp
py -3 -m pip install faster-whisper      # recommended backend on Windows

# install the skill (manual copy)
git clone https://github.com/AugustusW/audio-tldr-skill.git
$skillsDir = "$env:USERPROFILE\.claude\skills"
New-Item -ItemType Directory -Force -Path $skillsDir | Out-Null
Copy-Item -Recurse -Force "audio-tldr-skill\skills\audio-tldr" $skillsDir

Manual copy does not auto-update, and -Force overwrites an existing audio-tldr folder — prefer the plugin install if you want managed versions.

• Python command — if python3 isn't recognized, use python or the py launcher (py -3); the skill tells Claude to fall back automatically, but substitute accordingly when running the script yourself.
• Skill path — Claude Code on Windows reads skills from %USERPROFILE%\.claude\skills\ (plugin install works identically to macOS/Linux).
• GPU (optional) — faster-whisper runs on CPU out of the box. NVIDIA acceleration goes through CTranslate2; check that a CUDA device is visible with py -3 -c "import ctranslate2; print(ctranslate2.get_cuda_device_count())". Non-zero means CTranslate2 can see the GPU — it does not guarantee the CUDA runtime, cuBLAS/cuDNN DLLs, and GPU model loading all work; run one short real transcription to confirm. Required CUDA/cuDNN versions: see the faster-whisper README.
• mlx-whisper is Apple-Silicon-only. whisper.cpp on Windows needs a whisper-cli.exe on PATH plus AUDIO_TLDR_WHISPER_CPP_MODEL.
• Smart App Control may block yt-dlp.exe — the prebuilt binary is unsigned, and Smart App Control silently refuses to run it (Event Viewer shows a CodeIntegrity event 3077 for the file). It does not prompt; yt-dlp just never runs. Workaround: point yt-dlp's shim at your venv's signed python.exe instead of the unsigned exe — create yt-dlp.cmd on PATH:

  @echo off
  "%~dp0python.exe" -m yt_dlp %*
  

  The .cmd file must have CRLF line endings (Windows batch parsing is picky about this; saving it as LF-only from a Unix tool can make it fail silently). Editors that default to LF (or git config core.autocrlf set to input) will need an explicit CRLF save.
• Quote comma lists in PowerShell — frames.py --at 90,215,10:05 gets split by PowerShell into three separate arguments before Python ever sees it (commas are not special-cased inside quotes). Always quote: --at "90,215,10:05".

Usage

> summarize https://www.youtube.com/watch?v=xxxx
> give me the key points from this podcast: https://podcasts.apple.com/...
> /audio-tldr ~/Downloads/meeting-recording.m4a
> summarize this talk for a beginner — action items only: https://youtu.be/xxxx
> (later) same video, but focus only on what they said about pricing

State your needs in the request — focus, audience, format, length, language — and the digest follows them instead of the default takeaways+summary structure. The last one re-uses the cached transcript — instant, no re-transcription.

Pairs well with sharedoc-mcp — once the digest is written, "share this as a doc" turns it into a link you can hand to anyone (append, extend, or revoke it later).

Frames (optional)

> summarize this talk and grab the slides: https://youtu.be/xxxx
> screenshots at 1:30 and 12:05 from ~/Videos/keynote.mp4

Asking for screenshots / slides runs scripts/frames.py: ffmpeg scene detection captures frames where the picture visibly changes (--threshold, --min-gap, --max-frames, JPEG quality via --quality), or --at 90,12:05 extracts stills at exact timestamps (e.g. to illustrate a digest timeline). Off by default — plain summarize requests never download video. For URLs the video is fetched at ≤720p and deleted right after extraction (--keep-video keeps it); local files are used in place and never modified. Frames and their manifest.json live in the same cache entry as the transcript, so repeated requests are instant. Pure logic is covered by automated tests; real-video extraction is verified manually per release.

Subtitles (SRT/VTT)

> transcribe this and give me subtitles: https://youtu.be/xxxx
> srt file for ~/Videos/keynote.mp4

python3 scripts/transcribe.py --format srt "<source>"   # transcript.srt, standard SRT
python3 scripts/transcribe.py --format vtt "<source>"   # transcript.vtt, standard WebVTT

--format defaults to txt (unchanged). srt/vtt additionally write a subtitle file with segment-level timestamps next to transcript.txt — the plain transcript is unaffected either way. All four whisper backends support it. Segment data is cached once, so asking for the other subtitle format later (vtt after an earlier srt run, or vice versa) reformats from the cached segments instantly — no re-transcription. A cache entry from before this feature (or last transcribed with --format txt) has no segment data to build subtitles from: the script reports that clearly and names the fix (--force --format srt, or vtt, to re-transcribe with timestamps) rather than silently guessing or downgrading the request.

How it works

Two phases, deliberately separated:

1. Transcribe (scripts/transcribe.py) — resolves a cache key (normalized URL or file content hash), returns instantly on a hit; otherwise downloads via yt-dlp, transcribes with the best available whisper backend, and caches transcript.txt + meta.json under ~/.cache/audio-tldr/<sha256>/.
2. Digest — the agent reads the cached transcript and produces takeaways, a summary, and (for long content) an approximate timeline. If your request didn't say how to digest, it asks first — in plain conversational text, never a clickable menu, so it works over plain-text messaging channels too. Every digest is also saved to the output folder (default ./audio-tldr-output/) as <title>-<date>-<style>.md (or .html). Re-digesting with a different focus skips phase 1 entirely.

Privacy

Be precise about what stays local and what doesn't:

• Your audio/video never leaves the machine. No third-party transcription service is used, and the scripts in this repo contain no telemetry. Network access still happens where you'd expect: yt-dlp fetches URL sources from the source site, and whisper backends may download their model on first use (dependency behavior is governed by those projects).

• The digest phase sends the transcript text (never the audio) to the model, inside your own Claude session — exactly like asking Claude to read any local file. Exception: set digest_model: ollama:<model> and the digest runs through your own local Ollama server instead — the transcript text never leaves the machine either, for a fully local pipeline (traded against a small local model's lower digest quality). See Digest templates.

• Cached transcripts are unencrypted plaintext, kept indefinitely by default, under ~/.cache/audio-tldr/. After processing sensitive content, --clear that entry, or configure a retention period.

• Digests persist in the output folder (default ./audio-tldr-output/, relative to your working directory) — including full-transcript translations, which carry essentially the whole transcript. The output folder has no clearing or retention mechanism; delete files manually, and add the folder to .gitignore if you run the skill inside a git-tracked directory.

• Phase 1 only (sensitive recordings): transcribe without ever handing the text to Claude — run the script yourself; stdout is metadata JSON only, and the transcript stays at the returned transcript_path until you delete it:

  # macOS/Linux
  python3 ~/.claude/skills/audio-tldr/scripts/transcribe.py "/path/to/recording.m4a"
  

  # Windows PowerShell
  py -3 "$env:USERPROFILE\.claude\skills\audio-tldr\scripts\transcribe.py" "C:\path\to\recording.m4a"
  

Preferences (optional)

Create ~/.config/audio-tldr/preferences.md to set standing habits — every field is optional and everything works without the file:

output_dir: ~/Documents/audio-digests
timeline: off
auto_delete_audio: off
output_format: html
model: large-v3

The file is read by the agent (Claude Code and Codex share it) — the install never asks you to set it up, and defaults apply whenever it's absent.

Digest templates

Three built-in templates shape the digest output — name one ("digest this as meeting minutes") or pick from the menu when asked:

The timeline in key-summary appears only when the timeline preference is not off, the source runs over 20 minutes, and the transcript has clear topic shifts.

Build your own: drop a markdown file in ~/.config/audio-tldr/templates/ — frontmatter (name, description) plus section instructions; same name overrides a built-in, a new name becomes a new menu option. Easiest start: copy a built-in from skills/audio-tldr/templates/ and edit (e.g. change the timeline threshold). You can also just describe a format in conversation — the skill offers to save it for reuse. Your templates live outside the skill folder, so skill updates never touch them.

Cheaper digests: on platforms with subagents the digest runs on a cheaper model by default (Claude Code: sonnet; Codex: GPT-5.6 Terra) — see the digest_model preference to pin a model or turn this off (off = digest inline).

Fully-local digest (Ollama): set digest_model: ollama:<model> (e.g. ollama:llama3.2) and Phase 2 runs through your own local Ollama server via scripts/digest.py instead of any agent subagent — the transcript text never leaves the machine either, on top of the already-local transcription pipeline. Requires Ollama installed, ollama serve running, and the model already pulled (ollama pull <model>) — same user-responsibility rule as the whisper backends, nothing is installed or pulled automatically. Server address defaults to http://localhost:11434; override with AUDIO_TLDR_OLLAMA_HOST if Ollama runs on another machine on your network. If the server is unreachable or the model isn't pulled, the agent reports the specific error and stops — it never silently falls back to an agent-session digest, since that would defeat the point of choosing this mode. Tradeoff: a small local model's digest quality is generally below the subagent path (sonnet / GPT-class models).

Cache & configuration

The cache is kept forever by default — nothing is auto-deleted unless you opt in.

Ask Claude, or run scripts/transcribe.py directly:

Environment variables:

Develop

git clone https://github.com/AugustusW/audio-tldr-skill.git
cd audio-tldr-skill
python3 -m pytest tests/   # 160 unit tests, no network or model needed

Versioning: every release bumps version in .claude-plugin/plugin.json and .claude-plugin/marketplace.json (kept identical), adds a CHANGELOG entry, and is published as a git tag + GitHub Release. tests/test_docs.py enforces that: the suite fails if either README's Status version, either README's stated test count, the marketplace version, or the newest CHANGELOG entry falls out of step with plugin.json. To get update notifications: Watch this repo (Custom → Releases), or — if you installed as a Claude Code plugin — run /plugin and update from the marketplace (it compares the version above). Manual-copy installs have no auto-update: re-copy the skill folder after a new release. Your preferences, custom templates (~/.config/audio-tldr/), and cache (~/.cache/audio-tldr/) all live outside the skill folder — updating never touches them.

Status

v0.7.3 (CHANGELOG) — core logic is covered by 160 offline unit tests (yt-dlp, whisper backends, cache, OpenCC, ffmpeg/ffprobe frame extraction, and the Ollama HTTP endpoint are mocked; no network or models needed). The full flow has been manually verified (2026-07-19: real YouTube download, transcription, cached re-digest, Chinese conversion, --keep-audio, output-folder digests in md/html, transcript translation, interpreter auto-selection from /usr/bin/python3, and the Apple Podcasts fallback end-to-end — a real 53-min episode resolved via iTunes lookup, transcribed, and cache-hit on the original Apple URL) on:

Newer dependency versions may behave differently. Not yet covered by automated tests: real downloads, the other three backends, and Windows. Codex support follows the open SKILL.md standard; the transcription core was verified end-to-end inside Codex on 2026-07-19 (a real 53-min podcast downloaded, transcribed, and cache-hit, including the interpreter auto-selection path). Digest-layer features (output folder, translation, preferences) have so far been exercised in Claude Code only. SRT/VTT subtitle export (v0.6.0) is covered by unit tests on all four backends' segment-capture and formatting logic; end-to-end subtitle output from a real transcription has not yet been manually verified on every backend. The Ollama local digest mode (v0.6.0) is covered by unit tests against a mocked HTTP endpoint (request shape, response parsing, unreachable-server and model-missing errors); it has not yet been manually verified against a real Ollama server. Frame extraction (v0.5.0) is covered by unit tests that stub ffmpeg and ffprobe rather than invoking them; the past-end-timestamp behavior fixed in v0.7.1 was measured against ffmpeg 8.1, and other versions may exit differently. Possible next: speaker diarization. Issues and PRs welcome.

License

MIT. See LICENSE.

Long content is worth hearing once — by your machine, not by you.