feat(stt): phase 1-2 — commande, backend vocal, routeur cerveau, HUD MVP

- cli.py : commande `stt` (--setup, --mode, --no-tts)
- config.py : défauts embarqués + ~/.config/stt/stt.toml
- voice/engine.py : refactor de hermes-voice en classe avec callbacks d'état
- brain/router.py : 3 modes (hermes SSH / local LiteLLM / claude API) + auto-détection LAN
- server/app.py : HTTP statique (HUD) + websocket (états → HUD)
- memory/store.py : tier local SQLite (Qdrant + sync GitHub = phase 4)
- hud/index.html : HUD MVP (visualiseur d'état + conversation)

Vérifié hors-LAN : py_compile, --help, config, routeur (→ claude), mémoire SQLite.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013FmcxGsyXZXogiAHQLjnZT
This commit is contained in:
Claude 2026-06-17 09:11:00 +00:00
parent 8096ba208d
commit 6c680c4f8b
No known key found for this signature in database
15 changed files with 799 additions and 33 deletions

View file

@ -143,16 +143,22 @@ Pilotable depuis `stt/config/` + l'écran de réglages du HUD, sans recompiler :
## Roadmap
| Phase | Objectif | Livrable |
|---|---|---|
| **0 — Cadrage** | Conception validée | ✅ ce doc + squelette `stt/` |
| **1 — Commande + backend** | `stt`/`stt --setup` lancent le moteur vocal + websocket | `stt/pyproject.toml`, `stt/stt/` (hermes-voice + ws + états) |
| **2 — Routeur cerveau** | 3 modes + auto-détection LAN | module `brain` (hermes/local-direct/claude-direct) |
| **3 — HUD MVP** | Visualiseur + conversation à l'écran | `stt/hud/` relié au websocket |
| **4 — Mémoire 3 tiers** | local + GitHub (+ Qdrant si dispo) | module `memory` + sync git + distillation |
| **5 — Personnalisation** | Thèmes / avatar / voix / wake word | écran réglages + `stt/config/` |
| **6 — Auto-start** | Démarre au boot du poste | `install-service.sh` + kiosk |
| **7 — Cluster (opt.)** | STT/TTS sur GPU | Wyoming sur gpu-01 + `k8s/apps/stt/` |
| Phase | Objectif | Livrable | État |
|---|---|---|---|
| **0 — Cadrage** | Conception validée | doc + squelette `stt/` | ✅ fait |
| **1 — Commande + backend** | `stt`/`stt --setup` lancent le moteur vocal + websocket | `pyproject.toml`, `stt/` (cli, config, voice, server) | ✅ fait |
| **2 — Routeur cerveau** | 3 modes + auto-détection LAN | module `brain` (hermes/local/claude) | ✅ fait |
| **3 — HUD** | Visualiseur + conversation | `stt/hud/index.html` (MVP) → visualiseur avancé | 🟡 MVP fait |
| **4 — Mémoire 3 tiers** | local ✅ + GitHub + Qdrant | `memory` : SQLite ✅ ; distillation/sync git + Qdrant | 🟡 local fait |
| **5 — Personnalisation** | Thèmes / avatar / voix / wake word | écran réglages + `stt/config/` | ⏳ |
| **6 — Auto-start** | Démarre au boot du poste | `install-service.sh` + kiosk | ⏳ |
| **7 — Cluster (opt.)** | STT/TTS sur GPU | Wyoming sur gpu-01 + `k8s/apps/stt/` | ⏳ |
### État phase 1-2 (testé hors-LAN)
`py_compile` OK sur tout le package ; `stt --help`, chargement config, routeur cerveau
(auto-détection → `claude` hors-LAN) et mémoire SQLite locale vérifiés. Le test bout-en-bout
audio (Whisper/Piper/micro) se fait sur le poste après `pipx install` + `stt --setup`.
---

View file

@ -1,13 +1,14 @@
# Configuration STT — copier vers ~/.config/stt/stt.toml (via `stt --setup`)
# Configuration STT — copier vers ~/.config/stt/stt.toml (`stt --setup` le génère).
# Ces valeurs reflètent les défauts embarqués (stt/stt/config.py : DEFAULT_CONFIG).
[brain]
# Mode cerveau : "auto" (détecte le LAN), "hermes", "local", "claude"
# Mode cerveau : "auto" (ping s01 → hermes si LAN, sinon claude), "hermes", "local", "claude"
mode = "auto"
# Hôte testé pour décider LAN vs hors-LAN
lan_probe_host = "192.168.1.200"
lan_probe_port = 4000 # LiteLLM — présent dès que s01 est joignable
[brain.hermes] # mode hermes (LAN) — agit sur Funk
url = "http://192.168.1.200:8080"
[brain.hermes] # mode hermes (LAN) — agit sur Funk (via SSH, éprouvé)
ssh_host = "s01"
profile = "funk-ai"
[brain.local] # mode local-direct (LAN) — chat Qwen3 sur g01
@ -16,25 +17,28 @@ api_key = "lm-studio"
model = "hermes-default"
[brain.claude] # mode claude-direct (hors-LAN) — chat seul
# Clé via variable d'env ANTHROPIC_API_KEY (ne JAMAIS committer la clé ici)
model = "claude-sonnet-4-6"
model = "claude-sonnet-4-6" # clé via env ANTHROPIC_API_KEY (jamais ici)
[voice]
wake_word = "hermes" # "Ok Hermès"
whisper_model = "large-v3" # STT (faster-whisper, CPU int8)
piper_voice = "fr_FR-upmc-medium" # TTS — voix dans config/voices/
whisper_model = "large-v3" # STT faster-whisper (CPU int8)
piper_voice = "fr_FR-upmc-medium" # TTS — ~/.local/share/piper/<voix>.onnx
language = "fr"
silence_sec = 1.2 # silence avant fin d'enregistrement
chat_timeout_sec = 60 # retour en veille sans interaction
silence_sec = 1.2
min_speech_sec = 0.6
chat_timeout_sec = 60
[ui]
theme = "arc-reactor" # thème dans config/themes/
avatar = "default" # image dans config/avatars/
kiosk = true # ouvre le HUD en plein écran au lancement
theme = "arc-reactor"
avatar = "default"
kiosk = true
ws_host = "127.0.0.1"
ws_port = 9300 # websocket états → HUD (doit matcher hud/index.html)
http_port = 9301 # HTTP statique qui sert le HUD
[memory]
enabled = true
local_db = "~/.local/share/stt/memory.sqlite" # tier local (gitignoré)
distill_on_exit = true # distille → GitHub à l'arrêt
qdrant_url = "http://192.168.1.200:6333" # tier s01 (si LAN + Qdrant up)
distill_on_exit = true # phase 4
qdrant_url = "http://192.168.1.200:6333" # tier s01 (phase 4)
qdrant_collection = "stt-memory"

82
stt/hud/index.html Normal file
View file

@ -0,0 +1,82 @@
<!doctype html>
<html lang="fr">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>STT — Funk</title>
<style>
:root { --accent:#36d1ff; --bg:#04080f; --dim:#1b2a3a; }
* { box-sizing:border-box; }
body {
margin:0; height:100vh; background:radial-gradient(circle at 50% 40%, #0a1626, var(--bg));
color:#cfe9ff; font-family:system-ui, sans-serif; overflow:hidden;
display:flex; flex-direction:column; align-items:center; justify-content:center;
}
#core {
width:240px; height:240px; border-radius:50%; position:relative;
display:flex; align-items:center; justify-content:center; transition:all .4s;
box-shadow:0 0 60px -10px var(--accent);
}
#core::before, #core::after {
content:""; position:absolute; border-radius:50%; border:2px solid var(--accent);
inset:0; opacity:.5; animation:pulse 3s ease-in-out infinite;
}
#core::after { inset:30px; opacity:.25; animation-delay:.6s; }
#state { font-size:1.1rem; letter-spacing:.25em; text-transform:uppercase; }
.idle #core { box-shadow:0 0 50px -15px var(--accent); filter:saturate(.6); }
.listening #core { box-shadow:0 0 80px 0 var(--accent); }
.thinking #core { box-shadow:0 0 70px 0 #ffd23f; }
.speaking #core { box-shadow:0 0 90px 5px #36ff9e; }
@keyframes pulse { 0%,100%{transform:scale(1);opacity:.5} 50%{transform:scale(1.08);opacity:.15} }
#log {
position:absolute; bottom:0; left:0; right:0; max-height:32vh; overflow:auto;
padding:1rem 1.5rem; font-size:.95rem; line-height:1.5; backdrop-filter:blur(4px);
}
.turn { margin:.3rem 0; }
.user { color:#9fd0ff; }
.asst { color:#aef5cf; }
.err { color:#ff8c8c; }
.tag { opacity:.5; font-size:.75rem; margin-left:.5rem; }
#conn { position:absolute; top:12px; right:16px; font-size:.75rem; opacity:.6; }
</style>
</head>
<body class="idle">
<div id="conn"></div>
<div id="core"></div>
<div id="state">veille</div>
<div id="log"></div>
<script>
const WS_PORT = 9300; // doit matcher ui.ws_port
const LABELS = { idle:"veille", listening:"écoute", thinking:"réflexion", speaking:"réponse" };
const body = document.body, stateEl = document.getElementById("state");
const logEl = document.getElementById("log"), connEl = document.getElementById("conn");
function setState(s) {
body.className = s;
stateEl.textContent = LABELS[s] || s;
}
function addTurn(cls, text, tag) {
const d = document.createElement("div");
d.className = "turn " + cls;
d.textContent = text;
if (tag) { const t = document.createElement("span"); t.className="tag"; t.textContent=tag; d.appendChild(t); }
logEl.appendChild(d);
logEl.scrollTop = logEl.scrollHeight;
}
function connect() {
const ws = new WebSocket(`ws://${location.hostname}:${WS_PORT}`);
ws.onopen = () => connEl.style.color = "#36ff9e";
ws.onclose = () => { connEl.style.color = "#ff8c8c"; setTimeout(connect, 1500); };
ws.onmessage = (e) => {
const ev = JSON.parse(e.data);
if (ev.type === "state") setState(ev.state);
else if (ev.type === "user") addTurn("user", "🗣️ " + ev.text);
else if (ev.type === "assistant") addTurn("asst", "🤖 " + ev.text, ev.mode);
else if (ev.type === "error") addTurn("err", "⚠️ " + ev.text);
};
}
connect();
</script>
</body>
</html>

View file

@ -1,15 +1,15 @@
# stt/stt — package backend (à implémenter)
# stt/stt — package backend
Cœur Python de la commande `stt`. Réutilise le moteur vocal de `tools/hermes-voice.py`.
```
stt/
├── cli.py # entrypoint `stt` (--setup, --mode, --no-tts, --stop) [phase 1]
├── voice/ # wake word + VAD, STT faster-whisper, TTS Piper [phase 1]
├── brain/ # routeur 3 modes : hermes / local-direct / claude-direct
│ # + auto-détection LAN (ping s01) [phase 2]
├── memory/ # mémoire 3 tiers : SQLite local, Qdrant s01, git GitHub [phase 4]
└── server/ # serveur websocket : pousse états + transcript + réponses au HUD [phase 1]
├── cli.py # entrypoint `stt` (--setup, --mode, --no-tts) ✅ phase 1
├── config.py # défauts embarqués + chargement ~/.config/stt/stt.toml ✅ phase 1
├── voice/ # wake word + VAD, STT faster-whisper, TTS Piper ✅ phase 1
├── brain/ # routeur 3 modes hermes/local/claude + auto-détect LAN ✅ phase 2
├── server/ # HTTP statique (HUD) + websocket (états → HUD) ✅ phase 1
└── memory/ # tier local SQLite ✅ ; Qdrant s01 + sync GitHub ⏳ phase 4
```
## États exposés au HUD (via websocket)

3
stt/stt/__init__.py Normal file
View file

@ -0,0 +1,3 @@
"""STT — assistant vocal Jarvis du homelab Funk."""
__version__ = "0.0.1"

View file

@ -0,0 +1,5 @@
"""Routeur cerveau — 3 modes + auto-détection LAN."""
from stt.brain.router import BrainRouter
__all__ = ["BrainRouter"]

113
stt/stt/brain/router.py Normal file
View file

@ -0,0 +1,113 @@
"""Routeur cerveau — décide où envoyer le texte transcrit.
3 modes :
- hermes : SSH vers s01 `hermes --profile -z` (agit sur Funk). Éprouvé dans
tools/hermes-voice. Le gateway HTTP :8080 pourra le remplacer plus tard.
- local-direct : HTTP LiteLLM :4000 (Qwen3 sur g01) chat seul.
- claude-direct : API Anthropic depuis le poste (hors-LAN) chat seul.
mode "auto" : ping TCP s01 joignable hermes, sinon claude.
"""
from __future__ import annotations
import socket
import subprocess
from collections import deque
from typing import Any
VOICE_INJECT = (
"[Réponse vocale : courte, 1-2 phrases max, sans markdown ni listes] "
)
SYSTEM_PROMPT = (
"Tu es Hermes, l'assistant vocal du homelab Funk. "
"Réponds toujours en français, de façon concise (2-3 phrases maximum)."
)
MAX_HISTORY = 10
class BrainRouter:
def __init__(self, cfg: dict[str, Any]):
self.cfg = cfg
self._configured = cfg.get("mode", "auto")
self._history: deque[dict] = deque(maxlen=MAX_HISTORY * 2)
self.mode = self.resolve_mode()
# ── détection de mode ────────────────────────────────────────────────
def lan_available(self) -> bool:
host = self.cfg.get("lan_probe_host", "192.168.1.200")
port = int(self.cfg.get("lan_probe_port", 4000))
try:
with socket.create_connection((host, port), timeout=1.5):
return True
except OSError:
return False
def resolve_mode(self) -> str:
if self._configured != "auto":
return self._configured
return "hermes" if self.lan_available() else "claude"
def refresh_mode(self) -> str:
"""Re-évalue le mode (ex. changement de réseau). Retourne le mode courant."""
self.mode = self.resolve_mode()
return self.mode
# ── dispatch ─────────────────────────────────────────────────────────
def ask(self, text: str) -> str:
if self.mode == "hermes":
return self._ask_hermes(text)
if self.mode == "local":
return self._ask_openai(text, self.cfg["local"])
return self._ask_claude(text)
def _ask_hermes(self, text: str) -> str:
c = self.cfg["hermes"]
query = (VOICE_INJECT + text).replace("'", "'\\''")
cmd = [
"ssh", c.get("ssh_host", "s01"),
"sudo -i -u hermes bash -c "
f"'HERMES_HOME=/srv/data/hermes hermes --profile {c.get('profile', 'funk-ai')} "
f'-z "{query}"\'',
]
r = subprocess.run(cmd, capture_output=True, text=True, timeout=90)
out = r.stdout.strip()
if not out and r.returncode != 0:
raise RuntimeError(r.stderr.strip() or "hermes -z a échoué")
return out
def _ask_openai(self, text: str, c: dict) -> str:
import requests
self._history.append({"role": "user", "content": text})
r = requests.post(
c["url"],
json={
"model": c["model"],
"messages": [{"role": "system", "content": SYSTEM_PROMPT}]
+ list(self._history),
"max_tokens": 200,
"temperature": 0.7,
},
headers={"Authorization": f"Bearer {c['api_key']}"},
timeout=30,
)
r.raise_for_status()
resp = r.json()["choices"][0]["message"]["content"].strip()
self._history.append({"role": "assistant", "content": resp})
return resp
def _ask_claude(self, text: str) -> str:
from anthropic import Anthropic
self._history.append({"role": "user", "content": text})
client = Anthropic() # lit ANTHROPIC_API_KEY
msg = client.messages.create(
model=self.cfg["claude"]["model"],
max_tokens=300,
system=SYSTEM_PROMPT,
messages=list(self._history),
)
resp = "".join(b.text for b in msg.content if b.type == "text").strip()
self._history.append({"role": "assistant", "content": resp})
return resp

67
stt/stt/cli.py Normal file
View file

@ -0,0 +1,67 @@
"""Entrypoint de la commande `stt`."""
from __future__ import annotations
import argparse
import asyncio
import sys
def _cmd_setup() -> int:
from stt.config import write_default_config
path = write_default_config()
print(f"✓ Config : {path}")
print("\nÀ installer séparément sur le poste :")
print(" • modèle Whisper (téléchargé au 1er lancement par faster-whisper)")
print(" • Piper + voix → ~/.local/share/piper/<voix>.onnx")
print(" • clé ANTHROPIC_API_KEY (env) pour le mode claude-direct")
return 0
def _cmd_run(args) -> int:
from stt.brain import BrainRouter
from stt.config import load_config
from stt.memory import LocalMemory
from stt.server import serve
from stt.voice import VoiceEngine
cfg = load_config()
if args.mode:
cfg["brain"]["mode"] = args.mode
brain = BrainRouter(cfg["brain"])
print(f"Cerveau : mode '{brain.mode}' (configuré : '{brain._configured}')")
memory = None
if cfg["memory"].get("enabled"):
memory = LocalMemory(cfg["memory"]["local_db"])
def make_engine(emit):
return VoiceEngine(cfg["voice"], brain, emit, memory=memory, no_tts=args.no_tts)
try:
asyncio.run(serve(cfg, make_engine))
except KeyboardInterrupt:
print("\n👋 Au revoir")
finally:
if memory:
memory.close()
return 0
def main(argv: list[str] | None = None) -> int:
p = argparse.ArgumentParser(prog="stt", description="Assistant vocal Jarvis de Funk")
p.add_argument("--setup", action="store_true", help="Écrit la config par défaut")
p.add_argument("--mode", choices=["hermes", "local", "claude"],
help="Force le mode cerveau (sinon auto-détection LAN)")
p.add_argument("--no-tts", action="store_true", help="Réponses texte seulement")
args = p.parse_args(argv)
if args.setup:
return _cmd_setup()
return _cmd_run(args)
if __name__ == "__main__":
sys.exit(main())

92
stt/stt/config.py Normal file
View file

@ -0,0 +1,92 @@
"""Chargement / écriture de la configuration STT.
Les défauts sont embarqués (pas de data-file robuste pour pipx). `stt --setup`
écrit ces défauts dans ~/.config/stt/stt.toml ; au lancement on fusionne le fichier
utilisateur par-dessus les défauts.
"""
from __future__ import annotations
import tomllib
from pathlib import Path
from typing import Any
CONFIG_DIR = Path.home() / ".config" / "stt"
CONFIG_PATH = CONFIG_DIR / "stt.toml"
DATA_DIR = Path.home() / ".local" / "share" / "stt"
DEFAULT_CONFIG: dict[str, Any] = {
"brain": {
"mode": "auto", # auto | hermes | local | claude
"lan_probe_host": "192.168.1.200",
"lan_probe_port": 4000, # LiteLLM — présent dès que s01 est joignable
"hermes": { # mode hermes (LAN) — agit sur Funk
"ssh_host": "s01", # accès SSH éprouvé (cf. tools/hermes-voice)
"profile": "funk-ai",
},
"local": { # mode local-direct (LAN) — chat Qwen3 g01
"url": "http://192.168.1.200:4000/v1/chat/completions",
"api_key": "lm-studio",
"model": "hermes-default",
},
"claude": { # mode claude-direct (hors-LAN) — chat seul
"model": "claude-sonnet-4-6", # clé via env ANTHROPIC_API_KEY
},
},
"voice": {
"wake_word": "hermes",
"whisper_model": "large-v3",
"piper_voice": "fr_FR-upmc-medium",
"language": "fr",
"silence_sec": 1.2,
"min_speech_sec": 0.6,
"chat_timeout_sec": 60,
},
"ui": {
"theme": "arc-reactor",
"avatar": "default",
"kiosk": True,
"ws_host": "127.0.0.1",
"ws_port": 9300,
"http_port": 9301,
},
"memory": {
"enabled": True,
"local_db": str(DATA_DIR / "memory.sqlite"),
"distill_on_exit": True,
"qdrant_url": "http://192.168.1.200:6333",
"qdrant_collection": "stt-memory",
},
}
def _deep_merge(base: dict, over: dict) -> dict:
out = dict(base)
for k, v in over.items():
if isinstance(v, dict) and isinstance(out.get(k), dict):
out[k] = _deep_merge(out[k], v)
else:
out[k] = v
return out
def load_config() -> dict[str, Any]:
"""Défauts + surcharge du fichier utilisateur s'il existe."""
if CONFIG_PATH.exists():
with CONFIG_PATH.open("rb") as f:
user = tomllib.load(f)
return _deep_merge(DEFAULT_CONFIG, user)
return DEFAULT_CONFIG
def write_default_config(force: bool = False) -> Path:
"""Écrit la config par défaut dans ~/.config/stt/stt.toml (idempotent)."""
import tomli_w
CONFIG_DIR.mkdir(parents=True, exist_ok=True)
DATA_DIR.mkdir(parents=True, exist_ok=True)
if CONFIG_PATH.exists() and not force:
return CONFIG_PATH
with CONFIG_PATH.open("wb") as f:
tomli_w.dump(DEFAULT_CONFIG, f)
return CONFIG_PATH

View file

@ -0,0 +1,8 @@
"""Mémoire STT — tier local (SQLite) en phase 1.
Tiers Qdrant (s01) et distillation/sync GitHub : phase 4 (voir admin/ia/stt.md).
"""
from stt.memory.store import LocalMemory
__all__ = ["LocalMemory"]

49
stt/stt/memory/store.py Normal file
View file

@ -0,0 +1,49 @@
"""Tier mémoire local : journal de conversation en SQLite (brut, gitignoré).
Phase 1 : on persiste juste les tours de conversation. La distillation vers GitHub
et l'embedding Qdrant (s01) arrivent en phase 4.
"""
from __future__ import annotations
import sqlite3
import time
from pathlib import Path
class LocalMemory:
def __init__(self, db_path: str):
self.path = Path(db_path).expanduser()
self.path.parent.mkdir(parents=True, exist_ok=True)
self._db = sqlite3.connect(str(self.path), check_same_thread=False)
self._db.execute(
"""CREATE TABLE IF NOT EXISTS turns (
id INTEGER PRIMARY KEY AUTOINCREMENT,
ts REAL NOT NULL,
role TEXT NOT NULL,
content TEXT NOT NULL,
mode TEXT
)"""
)
self._db.commit()
def log(self, role: str, content: str, mode: str | None = None) -> None:
self._db.execute(
"INSERT INTO turns (ts, role, content, mode) VALUES (?, ?, ?, ?)",
(time.time(), role, content, mode),
)
self._db.commit()
def recent(self, limit: int = 20) -> list[dict]:
cur = self._db.execute(
"SELECT ts, role, content, mode FROM turns ORDER BY id DESC LIMIT ?",
(limit,),
)
rows = [
{"ts": ts, "role": role, "content": content, "mode": mode}
for ts, role, content, mode in cur.fetchall()
]
return list(reversed(rows))
def close(self) -> None:
self._db.close()

View file

@ -0,0 +1,5 @@
"""Serveur : websocket (états → HUD) + petit HTTP statique pour servir le HUD."""
from stt.server.app import serve
__all__ = ["serve"]

95
stt/stt/server/app.py Normal file
View file

@ -0,0 +1,95 @@
"""Serveur asyncio : sert le HUD (HTTP statique) et diffuse les états (websocket).
Le moteur vocal tourne dans un thread (audio bloquant) et pousse ses événements via
`emit()`, relayés sur la boucle asyncio puis diffusés à tous les clients HUD connectés.
"""
from __future__ import annotations
import asyncio
import json
import threading
from functools import partial
from http.server import SimpleHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
from typing import Any
HUD_DIR = Path(__file__).resolve().parent.parent.parent / "hud"
def _start_http(host: str, port: int) -> ThreadingHTTPServer:
handler = partial(SimpleHTTPRequestHandler, directory=str(HUD_DIR))
httpd = ThreadingHTTPServer((host, port), handler)
threading.Thread(target=httpd.serve_forever, daemon=True).start()
return httpd
async def serve(cfg: dict[str, Any], make_engine) -> None:
"""make_engine(emit) -> VoiceEngine. emit est thread-safe (depuis le thread voix)."""
import websockets
ui = cfg["ui"]
loop = asyncio.get_running_loop()
clients: set = set()
last_state = {"type": "state", "state": "idle"}
def broadcast(event: dict) -> None:
nonlocal last_state
if event.get("type") == "state":
last_state = event
data = json.dumps(event, ensure_ascii=False)
for ws in list(clients):
asyncio.create_task(_safe_send(ws, data))
async def _safe_send(ws, data: str) -> None:
try:
await ws.send(data)
except Exception:
clients.discard(ws)
# emit appelé depuis le thread voix → on repasse sur la boucle asyncio
def emit(event: dict) -> None:
loop.call_soon_threadsafe(broadcast, event)
async def handler(ws):
clients.add(ws)
try:
await ws.send(json.dumps(last_state, ensure_ascii=False))
async for _ in ws: # le HUD n'envoie rien pour l'instant (phase 5 : réglages)
pass
finally:
clients.discard(ws)
_start_http(ui["ws_host"], ui["http_port"])
engine = make_engine(emit)
# chargement Whisper + boucle audio dans un thread dédié
threading.Thread(target=_run_engine, args=(engine, emit), daemon=True).start()
async with websockets.serve(handler, ui["ws_host"], ui["ws_port"]):
url = f"http://{ui['ws_host']}:{ui['http_port']}/"
print(f" HUD : {url}")
if ui.get("kiosk"):
_open_browser(url, kiosk=True)
await asyncio.Future() # tourne indéfiniment
def _run_engine(engine, emit) -> None:
try:
engine.load()
engine.run()
except Exception as e: # noqa: BLE001
emit({"type": "error", "text": f"moteur vocal : {e}"})
def _open_browser(url: str, kiosk: bool) -> None:
import shutil
import subprocess
import webbrowser
for browser in ("chromium", "google-chrome", "chromium-browser"):
if shutil.which(browser):
flag = "--kiosk" if kiosk else "--new-window"
subprocess.Popen([browser, flag, url])
return
webbrowser.open(url)

View file

@ -0,0 +1,5 @@
"""Moteur vocal — wake word, STT (faster-whisper), TTS (Piper)."""
from stt.voice.engine import VoiceEngine
__all__ = ["VoiceEngine"]

232
stt/stt/voice/engine.py Normal file
View file

@ -0,0 +1,232 @@
"""Moteur vocal STT.
Refactor de tools/hermes-voice.py en classe avec callbacks d'état (`emit`), pour que
le serveur websocket puisse pousser l'état au HUD. Boucle :
veille wake word "Ok Hermès" conversation (parole libre) retour veille.
Dépendances lourdes (sounddevice, faster_whisper, webrtcvad) importées à la volée :
`stt --help` et la config fonctionnent sans micro ni modèle installé.
"""
from __future__ import annotations
import queue
import subprocess
import tempfile
import time
import unicodedata
from collections import deque
from pathlib import Path
from typing import Any, Callable
SAMPLE_RATE = 16000
PRE_ROLL = 15
CHAT_EXIT = {"au revoir", "bonne nuit", "stop hermes", "fin"}
PIPER_BIN = Path.home() / ".local/share/piper-runtime/piper"
PIPER_VOICE_DIR = Path.home() / ".local/share/piper"
def _deaccent(s: str) -> str:
return (
unicodedata.normalize("NFD", s)
.encode("ascii", "ignore")
.decode("ascii")
.lower()
)
class VoiceEngine:
"""Écoute le micro, transcrit, appelle le cerveau, synthétise la réponse.
`emit(event: dict)` est appelé à chaque changement d'état / message, pour le HUD.
Événements : {"type": "state", "state": idle|listening|thinking|speaking},
{"type": "user", "text": }, {"type": "assistant", "text": , "mode": },
{"type": "error", "text": }.
"""
def __init__(
self,
cfg: dict[str, Any],
brain,
emit: Callable[[dict], None],
memory=None,
no_tts: bool = False,
):
self.cfg = cfg
self.brain = brain
self.emit = emit
self.memory = memory
self.no_tts = no_tts
self._whisper = None
self._stop = False
# ── chargement modèle ────────────────────────────────────────────────
def load(self) -> None:
from faster_whisper import WhisperModel
self._whisper = WhisperModel(
self.cfg["whisper_model"], device="cpu", compute_type="int8"
)
# ── STT ──────────────────────────────────────────────────────────────
def _transcribe(self, audio) -> str:
import numpy as np
audio_f32 = audio.astype(np.float32) / 32768.0
segments, _ = self._whisper.transcribe(
audio_f32,
language=self.cfg.get("language", "fr"),
task="transcribe",
beam_size=3,
vad_filter=True,
initial_prompt="Transcription en français.",
)
return " ".join(seg.text for seg in segments).strip()
def _detect_wake(self, text: str) -> tuple[bool, str]:
wake = self.cfg.get("wake_word", "hermes")
words = text.strip().split()
low = [_deaccent(w).strip(",.!?«»") for w in words]
try:
idx = next(i for i, w in enumerate(low) if wake in w)
return True, " ".join(words[idx + 1:]).strip()
except StopIteration:
return False, ""
# ── TTS ──────────────────────────────────────────────────────────────
def _speak(self, text: str) -> None:
if self.no_tts:
return
voice = PIPER_VOICE_DIR / f"{self.cfg['piper_voice']}.onnx"
if not voice.exists() or not PIPER_BIN.exists():
return
with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as f:
out = Path(f.name)
try:
subprocess.run(
[str(PIPER_BIN), "--model", str(voice), "--output_file", str(out)],
input=text.encode(),
capture_output=True,
check=True,
env={"LD_LIBRARY_PATH": str(PIPER_BIN.parent)},
)
subprocess.run(["aplay", "-q", str(out)], check=False)
except FileNotFoundError:
pass
finally:
out.unlink(missing_ok=True)
# ── cerveau ──────────────────────────────────────────────────────────
def _respond(self, text: str) -> None:
self.emit({"type": "user", "text": text})
if self.memory:
self.memory.log("user", text, self.brain.mode)
self.emit({"type": "state", "state": "thinking"})
try:
resp = self.brain.ask(text)
except Exception as e: # noqa: BLE001 — on remonte au HUD, on ne crash pas
self.emit({"type": "error", "text": str(e)})
self.emit({"type": "state", "state": "idle"})
return
self.emit({"type": "assistant", "text": resp, "mode": self.brain.mode})
if self.memory:
self.memory.log("assistant", resp, self.brain.mode)
self.emit({"type": "state", "state": "speaking"})
self._speak(resp)
# ── boucle principale (VAD + wake word) ──────────────────────────────
def run(self) -> None:
import numpy as np
import sounddevice as sd
import webrtcvad
vad = webrtcvad.Vad(mode=3)
frame_ms = 20
frame_samples = int(SAMPLE_RATE * frame_ms / 1000)
silence_sec = self.cfg.get("silence_sec", 1.2)
min_speech = self.cfg.get("min_speech_sec", 0.6)
chat_timeout = self.cfg.get("chat_timeout_sec", 60)
required_silent = int(silence_sec * 1000 / frame_ms)
audio_q: queue.Queue = queue.Queue()
pre_roll: deque = deque(maxlen=PRE_ROLL)
chat_mode = False
last_activity = 0.0
def callback(indata, frames, time_info, status):
chunk = (indata[:, 0] if indata.ndim > 1 else indata.flatten()).copy()
audio_q.put(chunk)
def record_until_silence() -> "np.ndarray | None":
captured, silent = [], 0
while not self._stop:
try:
frame = audio_q.get(timeout=3.0)
except queue.Empty:
break
captured.append(frame)
try:
speech = vad.is_speech(frame.tobytes(), SAMPLE_RATE)
except Exception:
speech = True
silent = 0 if speech else silent + 1
if silent >= required_silent:
break
if not captured:
return None
audio = np.concatenate(captured)
return audio if len(audio) / SAMPLE_RATE >= min_speech else None
self.emit({"type": "state", "state": "idle"})
with sd.InputStream(
samplerate=SAMPLE_RATE, channels=1, dtype="int16",
blocksize=frame_samples, callback=callback,
):
while not self._stop:
if chat_mode and time.time() - last_activity > chat_timeout:
chat_mode = False
self.emit({"type": "state", "state": "idle"})
frame = audio_q.get()
pre_roll.append(frame)
try:
if not vad.is_speech(frame.tobytes(), SAMPLE_RATE):
continue
except Exception:
continue
self.emit({"type": "state", "state": "listening"})
captured = list(pre_roll)
tail = record_until_silence()
if tail is None:
self.emit({"type": "state", "state": "idle" if not chat_mode else "listening"})
continue
audio = np.concatenate([np.concatenate(captured), tail])
text = self._transcribe(audio)
if not text:
continue
if chat_mode:
if any(w in _deaccent(text) for w in CHAT_EXIT):
chat_mode = False
self.emit({"type": "user", "text": text})
self._speak("À bientôt !")
self.emit({"type": "state", "state": "idle"})
continue
last_activity = time.time()
self._respond(text)
self.emit({"type": "state", "state": "listening"})
else:
found, command = self._detect_wake(text)
if not found:
self.emit({"type": "state", "state": "idle"})
continue
chat_mode = True
last_activity = time.time()
if command:
self._respond(command)
self.emit({"type": "state", "state": "listening"})
def stop(self) -> None:
self._stop = True