What it does

HOW TO SET UP: in Claude Code, say "Set up ClaudeVoice" (or type /claudevoice-setup) and answer a few questions; Claude does the rest and can pick up where you left off. You need Windows, an NVIDIA graphics card (it runs without one, but far too slowly), ~5 GB of disk, a microphone and speaker, and your own Claude API key (Claude shows you how to add it; never paste it into chat). A spoken, two-way conversation with Claude for a household: always listening locally with no wake word, knowing who is speaking (enrolled, consenting people only) and when it is being talked TO rather than ABOUT, and interruptible mid-sentence. Listening (Silero VAD), transcription (faster-whisper), voice-ID and the judge run on the user's own PC; only lines meant for Claude go to the Claude API, and the voice is ElevenLabs or local Kokoro. Voice-me answers fast and hands real work to one linked Claude Code chat (switchboard hooks), reading its replies aloud. Also: echo test, voice audition, speed bench, enrolment by voice, game mode (frees ~3 GB of graphics memory while a listed program runs), Windows default devices, an on-screen avatar, Claude's place as a live Wallpaper Engine wallpaper, and (new) Meta Ray-Ban Display glasses support: a HUD card with Claude's face and live captions of what Claude says. The speech runtime (PyTorch with CUDA, ~5 GB with models) is built by `setup` into its own venv in the data folder, outside the toolbox workspace. Household data (voiceprints, people notes) never leaves that folder. Windows only for now.

Capabilities

What it's capable of

Info

Where the data folder is, whether the speech runtime is set up, and what it can do (GPU, CUDA, package versions). Quick; run it first.

Setup

Build or update the speech runtime: a venv in <data>/runtime with PyTorch (CUDA 12.8 build when an NVIDIA GPU is present, which RTX 50-series needs) and the speech packages. All download caches go to the data folder. Downloads ~3-5 GB the first time: ask the user first, and give the shell command a timeout of at least 30 minutes.

Devices

List microphones and speakers (WASAPI ones on Windows). Devices marked avoid are Bluetooth hands-free/headset mics: opening one drops the speaker into phone-call quality.

Voices

Audition voices: speak one script in each Kokoro voice, save WAVs to <data>/out/auditions, and measure each (pitch Hz, pitch range in semitones, words per minute, pause %, brightness Hz, low-mid warmth ratio, breathiness HNR dB). Returns the table and file paths for a person to listen to.

Echo

Echo test: plays ~10 s of Claude's voice through the speaker while recording the mic (tell the room to stay quiet), then reports room noise, echo level, the speaker-to-mic delay, how much WebRTC echo cancellation removes, and how often voice detection fires on the echo before and after. Saves the delay to <data>/echo.json for the live loop. Re-run after moving the mic or speaker.

Say

Say a line out loud in Claude's voice through the speaker, e.g. to get the user's attention when a long job finishes or Claude needs them. Returns at once: a resident voice server (127.0.0.1, port 47811 or config 'port') keeps the voice loaded, so it starts speaking ~0.3 s later. The first call starts the server (~15-50 s). Call it before writing the chat reply so voice and text arrive together. Keep it short and sparing; mind the hour.

Hush

Stop Claude's voice mid-sentence and drop anything queued. quit=true also shuts the voice server down (frees the GPU memory).

Config

Show this household's saved settings, or save some: input / output (part of a device name, or "default" to follow Windows' default), voice_engine (kokoro or elevenlabs), memory_dirs, lean_when (game mode programs), show_avatar / show_place, chat_speakers, phone / phone_people, stt_compute. The loop, echo and say actions use them. Kept in the data folder, never in the tool.

Start

Start always-on listening in the background (home mode). Nothing leaves the PC until someone talks TO Claude: a local judge (grammar plus a small local model) decides from the name, or from an answer to a question Claude just asked. Then enrolled people's lines go to Claude, with the last ~60 s as lead-in, until ~90 s with no exchange, a goodbye, or 'I wasn't talking to you'. Unenrolled voices are never sent or kept. Nothing anyone says is printed or written to the log. It says 'I'm listening' when ready (~30-60 s). Always ask the user before starting it.

Stop

Stop the background listening (politely, then forcibly after 15 s). Saying 'stop listening' out loud also stops it.

Status

Is the background listening running, is it listening or in a conversation, and the last lines of its log (which never contain what anyone said).

People

List the people enrolled for voice-ID (name, date, quality). Voiceprints only: no recordings are ever kept. Enrol the first person with the enroll action (the setup skill guides it); others join by voice: "Claude, enroll <name>". Nothing is saved unless their own consent is heard. Only enrol people who agreed.

Forget

Delete a person's voiceprint at once (their right, any time). They are then 'someone' again.

Switchboard

The voice chat: pick ONE Claude Code chat whose replies are spoken in Claude's voice while every other chat stays typed. Run {"link": true} from inside the chat to pick it (it reads that chat's CLAUDE_CODE_SESSION_ID); {"unlink": true} to stop; {"speak": "off"} to mute it. Speaking needs Claude Code hooks: {"install": "<project folder>"} writes them to that project's .claude/settings.local.json, "user" to ~/.claude/settings.json (every chat). Installing changes the user's Claude Code settings: ask first. Chats opened before the install may need reopening. No params: status.

Ping

One tiny Claude API request (a few tokens) to check the key and credits work and time the round trip. The key comes from the user environment variable CLAUDEVOICE_ANTHROPIC_KEY (never ANTHROPIC_API_KEY, which Claude Code would pick up) and is never printed.

Bench

Speed test of the local parts of a turn, no mic needed: voice time-to-first-audio and total, transcription time per clip (faster-whisper, warm), and VAD cost per frame. Downloads the Whisper model (~1.6 GB for large-v3-turbo) the first time.

Onboard

The setup checklist: every step (data folder, speech runtime, API key, mic and speaker, voice, echo test, first voice enrolled, optional household notes, shared memory, voice chat, game mode, phone door, desktop wallpaper, always-on listening) with done/detail, and for the next one the action to run or the question to ask. Run it first when setting ClaudeVoice up, and again after each step (the claudevoice-setup skill walks through it).

Enroll

Learn one person's voice from the PC's mic (for the first person; later ones join by voice with 'Claude, enroll <name>'). Show them the consent line and passage from `onboard` first; they read it aloud until `seconds` of speech is heard. Saved only if the consent line was heard and it's one steady voice. The always-on loop must not be running.

Wallpaper

Claude's place as the desktop wallpaper, live (Wallpaper Engine, Steam app 431960): no params shows whether it's installed; {"install": true} copies the page into Wallpaper Engine's projects and sets config desktop=wallpaper, so the loop feeds it (a 127.0.0.1-only WebSocket, port 47816) instead of opening the place window; {"remove": true} undoes both. The page shows Claude's real state, the time of day, who is talking (a name, never words) and what the voice chat is doing. Ask the user before installing. Two looks: the study, or a sci-fi HUD with system load (config wallpaper_look, or the Look setting in Wallpaper Engine).

Glasses

Claude on Meta Ray-Ban Display glasses: a small read-only HUD card (Claude's face in a ring, state, live captions of Claude's own spoken words, time and weather on a pinch) as a Meta Web App. The glasses' browser needs a public HTTPS URL (it doesn't go through the phone's Tailscale), so the card has its own door under a secret path. No params shows the state; {"on": true} makes the secret, turns it on and, with Tailscale, opens a Funnel (https_port, default 8443) unless url is given; it returns url and install_qr (scan with the phone, Meta AI app in Developer Mode). {"off": true} turns it off. Ask the user before turning it on: the link is public (secret path only), and Funnel changes their Tailscale setup. Web apps get no mic: talk through the phone page with the glasses as its Bluetooth headset.

Under the hood

What Claude can ask it to do

info Where the data folder is, whether the speech runtime is set up, and what it can do (GPU, CUDA, package versions).
setup Build or update the speech runtime: a venv in <data>/runtime with PyTorch (CUDA 12.8 build when an NVIDIA GPU is present, which RTX 50-series needs) and the speech packages.
devices List microphones and speakers (WASAPI ones on Windows).
voices Audition voices: speak one script in each Kokoro voice, save WAVs to <data>/out/auditions, and measure each (pitch Hz, pitch range in semitones, words per minute, pause %, brightness Hz, low-mid warmth ratio, breathiness HNR dB).
echo Echo test: plays ~10 s of Claude's voice through the speaker while recording the mic (tell the room to stay quiet), then reports room noise, echo level, the speaker-to-mic delay, how much WebRTC echo cancellation removes, and how often voice detection fires on the echo before and after.
say Say a line out loud in Claude's voice through the speaker, e.g.
hush Stop Claude's voice mid-sentence and drop anything queued.
config Show this household's saved settings, or save some: input / output (part of a device name, or "default" to follow Windows' default), voice_engine (kokoro or elevenlabs), memory_dirs, lean_when (game mode programs), show_avatar / show_place, chat_speakers, phone / phone_people, stt_compute.
start Start always-on listening in the background (home mode).
stop Stop the background listening (politely, then forcibly after 15 s).
status Is the background listening running, is it listening or in a conversation, and the last lines of its log (which never contain what anyone said).
people List the people enrolled for voice-ID (name, date, quality).
forget Delete a person's voiceprint at once (their right, any time).
switchboard The voice chat: pick ONE Claude Code chat whose replies are spoken in Claude's voice while every other chat stays typed.
ping One tiny Claude API request (a few tokens) to check the key and credits work and time the round trip.
bench Speed test of the local parts of a turn, no mic needed: voice time-to-first-audio and total, transcription time per clip (faster-whisper, warm), and VAD cost per frame.
onboard The setup checklist: every step (data folder, speech runtime, API key, mic and speaker, voice, echo test, first voice enrolled, optional household notes, shared memory, voice chat, game mode, phone door, desktop wallpaper, always-on listening) with done/detail, and for the next one the action to run or the question to ask.
enroll Learn one person's voice from the PC's mic (for the first person; later ones join by voice with 'Claude, enroll <name>').
wallpaper Claude's place as the desktop wallpaper, live (Wallpaper Engine, Steam app 431960): no params shows whether it's installed; {"install": true} copies the page into Wallpaper Engine's projects and sets config desktop=wallpaper, so the loop feeds it (a 127.0.0.1-only WebSocket, port 47816) instead of opening the place window; {"remove": true} undoes both.
glasses Claude on Meta Ray-Ban Display glasses: a small read-only HUD card (Claude's face in a ring, state, live captions of Claude's own spoken words, time and weather on a pinch) as a Meta Web App.

Get ClaudeVoice (talk with Claude out loud: local listening, voice-ID and voice) and every other tool

One key and one pasted prompt. The Toolbox keeps itself up to date.

Install with your key