Speech To Speech is Hugging Face's open-source voice agent. One command starts a spoken conversation that runs entirely on your own computer: voice activity detection decides when you have finished talking, a speech recognizer transcribes you, a language model answers, and a text-to-speech model speaks the reply, all streamed so it feels like a conversation rather than a walkie-talkie. The default pipeline is Silero VAD, Parakeet TDT, Qwen3.5-4B and Qwen3-TTS.
What makes it useful beyond a demo is the server. It speaks the OpenAI Realtime protocol over WebSocket and WebRTC, so apps written for OpenAI's Realtime API, the OpenAI Agents SDK and LiveKit Agents can point at your own machine instead. Every stage is swappable for other local models, a llama.cpp server or a hosted provider, and the same pipeline runs as the conversation backend for Hugging Face's Reachy Mini robots.
Limits: the default model wants about 16 GB of unified memory on an Apple Silicon Mac or 24 GB of VRAM on an NVIDIA GPU, CPU-only machines reply slowly, and Windows is not in the CI test matrix. It is Apache-2.0 with about 13,600 stars.
- Repository: github.com/huggingface/speech-to-speech
- Licence: Apache-2.0 (Apache License 2.0)
- Language: Python. Stars: 13.6K. Forks: 1,673. Last push: Oct 26, 2026.
- Scan: safe, Oct 26, 2026, commit 0835002
Who it is for
Developers building voice assistants, robots or phone agents who want a private, local alternative to OpenAI's Realtime API, and anyone with a capable Mac or GPU who wants to talk to an AI without a cloud account.
Getting started
1. Create a Python environment (3.11 recommended)
python3 -m venv .venv && source .venv/bin/activate2. Install the package
pip install speech-to-speech3. Start talking (the first run downloads the models)
speech-to-speech localUse headphones so the assistant does not hear itself. On Ubuntu, first run sudo apt-get install libportaudio2 libsndfile1. To serve the Realtime API for your own app, run speech-to-speech serve and connect to ws://127.0.0.1:8765/v1/realtime.
Safety scan
We cloned huggingface/speech-to-speech at commit 0835002 on Oct 26, 2026 and ran the checks described on the GitHub Tools page: credential patterns, decode-and-execute code, install-time scripts, committed binaries, risky CI workflows, every host the code talks to, known vulnerabilities in pinned dependencies, and project hygiene. A person read every hit. This is what we found.
- No secrets, no install hooks, no committed binaries and no bare-IP URLs across about 94,000 lines of Python. The one pattern hit is a 4,577-character line in demo/ui/account.js, which is the Hugging Face logo as inline SVG.
- The 30 known advisories (1 critical, 10 high) are all in demo/package-lock.json, the Node server for the optional browser demo (proxy-addr, body-parser, the MCP SDK and others). The Python package itself declares its dependencies without a lockfile, so they were not looked up.
- The outside hosts are Hugging Face for model downloads and the optional hosted backends: OpenAI, MiniMax, Hugging Face's inference router, and Serper or You.com for an optional web search tool. After the first run, HF_HUB_OFFLINE=1 keeps it fully offline.
- Optional TTS backends such as OmniVoice and Pocket TTS can clone a voice from a reference clip. They are not installed by default, and the README asks for consent and forbids impersonation; OmniVoice's weights are non-commercial.
- Four workflows, none using pull_request_target, and all 10 third-party actions pinned to commits. Dependabot and licence present; no security policy or CodeQL.
What the scanner counted
| Check | Result |
|---|---|
| Secrets | None found. |
| Suspicious code | 1 pattern hit found and read; every one is listed under the raw findings. |
| Install-time code | None: nothing runs at install beyond the package manager itself. |
| Committed binaries | None. |
| CI workflows | 4 workflows. None use pull_request_target. 0 of 10 third-party actions pinned to a tag rather than a commit. |
| Network hosts | 26 distinct hosts referenced from source; most often api.openai.com, img.png, huggingface.co, router.huggingface.co. No URLs to bare IP addresses. |
| Known vulnerabilities | 30 advisories across 126 pinned packages: 1 critical, 10 high, 17 moderate, 2 low. demo/livekit/requirements.txt: 1 packages, 0 advisories; demo/package-lock.json: 123 packages, 30 advisories; demo/requirements.txt: 3 packages, 0 advisories. |
| Project hygiene | Has automated dependency updates, licence file. Missing security policy, CodeQL, contributing guide. |
| OpenSSF Scorecard | Not scored: the project is not in Scorecard's weekly index. |
The raw findings
Every hit the scanner wrote out, with a link to the exact line at the scanned commit. Secrets candidates are redacted.
Pattern hits (1)
| Where | Rule | Match |
|---|---|---|
| demo/ui/account.js:19 | very-long-line | 4577 chars |
Worst known vulnerabilities (24 of 30)
| Advisory | Severity | Package | Summary |
|---|---|---|---|
| GHSA-jqcg-44mw-7w3h | critical | proxy-addr@2.0.7 | proxy-addr vulnerable to IP spoofing via IPv4-mapped IPv6 trust subnet |
| GHSA-22jm-h49p-29qw | high | @modelcontextprotocol/sdk@1.30.0 | MCP TypeScript SDK: Experimental tasks were not tied to the session that created them |
| GHSA-46jr-4wx8-v453 | high | @modelcontextprotocol/sdk@1.30.0 | MCP TypeScript SDK: One HTTP request within the default size limit could exhaust server memory during validation |
| GHSA-6qxp-vccf-f47h | high | @modelcontextprotocol/sdk@1.30.0 | MCP TypeScript SDK: OAuth client could send credentials to an authorization server chosen by the MCP server |
| GHSA-27r3-jxmh-p828 | high | body-parser@2.3.0 | body-parser vulnerable to Denial of Service via trailing bytes after a compressed request body |
| GHSA-5jgf-p345-68v8 | high | fast-uri@3.1.5 | fast-uri vulnerable to host confusion via skipped IDN canonicalization on scheme-relative references |
| GHSA-f65p-4m7j-42xc | high | fast-uri@3.1.5 | fast-uri vulnerable to server-side request forgery via malformed IPv6 normalization |
| GHSA-fph4-wmhf-6fwf | high | fast-uri@3.1.5 | fast-uri vulnerable to server-side request forgery via repeated hostname percent-decoding |
| GHSA-jqff-g426-hqxp | high | fast-uri@3.1.5 | fast-uri vulnerable to host confusion via percent-encoded scheme normalization |
| GHSA-qw65-cvwx-89v3 | high | fast-uri@3.1.5 | fast-uri vulnerable to authority injection via an unvalidated port in serialize |
| GHSA-mrhx-c7xc-m3w5 | high | zod@4.4.3 | Zod can exhaust memory while validating oversized constrained collections |
| GHSA-6prh-2h8m-c8cw | moderate | @modelcontextprotocol/sdk@1.30.0 | MCP TypeScript client SDK: HTTP client transports and OAuth requests followed cross-origin redirects and re-sent custom … |
| GHSA-rvq5-wwqv-78pq | moderate | @modelcontextprotocol/sdk@1.30.0 | MCP TypeScript SDK: Server bearer authentication accepted tokens issued for another service |
| GHSA-hrr3-gc8f-f4qj | moderate | fast-uri@3.1.5 | fast-uri vulnerable to inconsistent host case normalization via percent-encoded octets |
| GHSA-w74x-v6f9-j3xg | moderate | fast-uri@3.1.5 | fast-uri vulnerable to URI scheme spoofing via percent-encoded scheme decoding |
| GHSA-wjr9-j795-jxcr | moderate | fast-uri@3.1.5 | fast-uri vulnerable to host confusion via unchecked WHATWG host canonicalization |
| GHSA-5r4p-p66f-jhc7 | moderate | hono@4.13.0 | Hono: `serveStatic` decodes the request path a second time, leading to bypass of middleware on static paths |
| GHSA-crvj-82cr-hjcx | moderate | hono@4.13.0 | Hono: Query parser reads parameters after the URL fragment, causing cache-key and proxy interpretation differentials |
| GHSA-g6gw-c38x-mqfc | moderate | hono@4.13.0 | Hono: Unbounded dot-notation nesting in `parseBody()` can cause memory exhaustion |
| GHSA-gqvv-2mrq-wpjv | moderate | hono@4.13.0 | Hono: Incomplete fix for CVE-2026-39408: `toSSG()` still writes files outside the output directory |
| GHSA-hxh3-vqpv-xpqv | moderate | hono@4.13.0 | hono/jsx renders plain strings unescaped in boundary components, leading to XSS |
| GHSA-2vr4-cq9g-pvrc | moderate | ip-address@10.4.0 | ip-address: no classifier recognizes the NAT64 local-use range 64:ff9b:1::/48, allowing SSRF and trust-boundary bypass |
| GHSA-h3mg-xc3c-68pw | moderate | ip-address@10.4.0 | ip-address: Address6 builds a parse diagnostic proportional to the input with no length bound, allowing a single long st… |
| GHSA-j6r3-76f7-8jcv | moderate | ip-address@10.4.0 | ip-address: isInSubnet() and isHostInSubnet() compare addresses of different families as if they shared an address space… |
By the numbers
| Stars | 13.6K |
|---|---|
| Forks | 1,673 |
| Contributors | 72 |
| Commits | 1,564 |
| Open issues | 32 |
| Open pull requests | 31 |
| Releases | 6 |
| Latest release | v1.1.0 |
| Licence | Apache-2.0 |
| Main language | Python |
| Project age | 2 years |
| Last push | Oct 26, 2026 |
| Tracked files | 385 |
| Lines of code | 108.2K |
| Checkout size | 6 MB |
Lines by language: Python 93.7K, JavaScript 6,210, Markdown 4,475, CSS 2,755, YAML 457, HTML 428.
Questions
Is Speech To Speech free?
Yes. The code is Apache-2.0 and the default models are free to download and run locally, with no API key. Some optional models carry their own licences, such as OmniVoice's non-commercial weights, and hosted providers bill you if you choose them.
Does it work without the internet?
Yes, after the first run. The models download from Hugging Face once; then set HF_HUB_OFFLINE=1 and speech recognition, the language model and speech synthesis all run on your machine.
Can I use it in place of OpenAI's Realtime API?
For many apps, yes. speech-to-speech serve exposes the core OpenAI Realtime events over WebSocket and WebRTC, and the project tests the OpenAI Agents SDK against it. Point your client at ws://127.0.0.1:8765/v1/realtime instead of OpenAI's endpoint.
This post is part of GitHub Tools, where every repository is cloned and scanned before it is written up. The scan is a snapshot of one commit on one day; the repository has moved on since, so check it before you install.