mirror of
https://github.com/Alkatrazz24/Funk-lab.git
synced 2026-07-08 14:24:42 +02:00
feat(voice): ajouter client vocal Hermes (hermes-voice)
- tools/hermes-voice/ : pipeline VAD → faster-whisper → LiteLLM → piper TTS - mot-clé "hermes" détecté dans la transcription Whisper (pas de wake word model) - modes : VAD continu + push-to-talk + --no-tts pour debug - nftables : ouvrir port 4000 LiteLLM vers LAN domestique (192.168.1.0/24) - admin/ia/hermes-voice.md : documentation installation et utilisation Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
alkatrazz
parent
eca28a8dc1
commit
6fc5b5984e
4 changed files with 387 additions and 2 deletions
119
admin/ia/hermes-voice.md
Normal file
119
admin/ia/hermes-voice.md
Normal file
|
|
@ -0,0 +1,119 @@
|
|||
# Hermes Voice Client
|
||||
|
||||
Client vocal local pour interagir avec Hermes via la voix depuis le poste perso.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Micro → VAD (énergie) → faster-whisper (STT) → LiteLLM :4000 → piper (TTS) → Haut-parleur
|
||||
```
|
||||
|
||||
- **STT** : faster-whisper `small` sur CPU (9950X3D) — ~0.3s de latence
|
||||
- **Détection** : mot-clé "hermes" dans la transcription Whisper (pas de wake word model)
|
||||
- **LLM** : modèle `hermes-default` via LiteLLM sur storage-01
|
||||
- **TTS** : piper avec voix `fr_FR-upmc-medium`
|
||||
|
||||
## Installation
|
||||
|
||||
### Dépendances Python
|
||||
|
||||
```bash
|
||||
cd tools/hermes-voice/
|
||||
python3 -m venv .venv
|
||||
source .venv/bin/activate
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
### Piper TTS
|
||||
|
||||
```bash
|
||||
# Télécharger la release piper depuis https://github.com/rhasspy/piper/releases
|
||||
# ex : piper_linux_x86_64.tar.gz
|
||||
|
||||
mkdir -p ~/.local/share/piper-runtime ~/.local/share/piper
|
||||
tar xf piper_linux_x86_64.tar.gz -C /tmp
|
||||
cp -r /tmp/piper/* ~/.local/share/piper-runtime/
|
||||
|
||||
# Voix française
|
||||
wget -O ~/.local/share/piper/fr_FR-upmc-medium.onnx \
|
||||
"https://huggingface.co/rhasspy/piper-voices/resolve/main/fr/fr_FR/upmc/medium/fr_FR-upmc-medium.onnx"
|
||||
wget -O ~/.local/share/piper/fr_FR-upmc-medium.onnx.json \
|
||||
"https://huggingface.co/rhasspy/piper-voices/resolve/main/fr/fr_FR/upmc/medium/fr_FR-upmc-medium.onnx.json"
|
||||
```
|
||||
|
||||
### Test piper
|
||||
|
||||
```bash
|
||||
echo "Bonjour, je suis Hermes." | \
|
||||
LD_LIBRARY_PATH=~/.local/share/piper-runtime \
|
||||
~/.local/share/piper-runtime/piper \
|
||||
--model ~/.local/share/piper/fr_FR-upmc-medium.onnx \
|
||||
--output_file /tmp/test.wav && aplay /tmp/test.wav
|
||||
```
|
||||
|
||||
Si `aplay` manque : `sudo dnf install alsa-utils`
|
||||
|
||||
### Réseau
|
||||
|
||||
Le port 4000 (LiteLLM) doit être accessible depuis le LAN domestique (`192.168.1.0/24`).
|
||||
Déjà ouvert dans `ansible/roles/gateway/templates/nftables.conf.j2` depuis le 2026-06-02.
|
||||
|
||||
Vérification :
|
||||
```bash
|
||||
curl -s http://192.168.1.200:4000/v1/models -H "Authorization: Bearer lm-studio"
|
||||
```
|
||||
|
||||
## Utilisation
|
||||
|
||||
```bash
|
||||
cd tools/hermes-voice/
|
||||
source .venv/bin/activate
|
||||
|
||||
# Mode VAD — parler naturellement, commencer par "Hermes, ..."
|
||||
python hermes-voice.py
|
||||
|
||||
# Mode push-to-talk — Entrée pour démarrer/terminer
|
||||
python hermes-voice.py --ptt
|
||||
|
||||
# Sans TTS (réponses texte uniquement — pour déboguer)
|
||||
python hermes-voice.py --ptt --no-tts
|
||||
```
|
||||
|
||||
## Mode VAD
|
||||
|
||||
Le client écoute en continu. Quand de l'énergie vocale est détectée :
|
||||
1. Enregistrement jusqu'à 1.5s de silence
|
||||
2. Whisper transcrit
|
||||
3. Si "hermes" apparaît dans les 5 premiers mots → la commande est envoyée à LiteLLM
|
||||
4. Sinon → ignoré (conversations ambiantes non captées)
|
||||
|
||||
Exemple : *"Hermes, quel est l'état du cluster ?"*
|
||||
|
||||
## Configuration (`hermes-voice.py`)
|
||||
|
||||
| Variable | Défaut | Description |
|
||||
|---|---|---|
|
||||
| `LITELLM_URL` | `http://192.168.1.200:4000/...` | Endpoint LiteLLM |
|
||||
| `HERMES_MODEL` | `hermes-default` | Modèle LiteLLM |
|
||||
| `WHISPER_SIZE` | `small` | Taille modèle Whisper |
|
||||
| `VOICE_THRESH` | `0.012` | Seuil RMS détection voix |
|
||||
| `SILENCE_SEC` | `1.5` | Délai silence pour terminer |
|
||||
| `KEYWORD` | `hermes` | Mot-clé déclencheur |
|
||||
| `KEYWORD_POS` | `5` | Position max du mot-clé |
|
||||
| `PIPER_VOICE` | `~/.local/share/piper/fr_FR-upmc-medium.onnx` | Voix TTS |
|
||||
|
||||
## Dépannage
|
||||
|
||||
**Port 4000 inaccessible** : ouvrir un tunnel SSH temporaire
|
||||
```bash
|
||||
ssh -L 4000:localhost:4000 user@192.168.1.200
|
||||
# puis changer LITELLM_URL → http://localhost:4000/...
|
||||
```
|
||||
|
||||
**libpiper_phonemize.so introuvable** : tous les `.so` de piper doivent être dans
|
||||
`~/.local/share/piper-runtime/` (le script gère `LD_LIBRARY_PATH` automatiquement).
|
||||
|
||||
**Whisper ne comprend pas** : parler plus lentement, augmenter `WHISPER_SIZE` → `medium`.
|
||||
Ou baisser `VOICE_THRESH` si le micro est peu sensible.
|
||||
|
||||
**Faux positifs VAD** (bruit ambiant déclenche) : augmenter `VOICE_THRESH` (ex: `0.02`).
|
||||
|
|
@ -44,8 +44,8 @@ table inet filter {
|
|||
# Hermes dashboard (poste admin uniquement)
|
||||
tcp dport 9119 ip saddr {{ hermes_dashboard_allowed_ip }} accept
|
||||
|
||||
# LiteLLM proxy (pods k8s 10.42/16 + nœuds cluster)
|
||||
tcp dport 4000 ip saddr { 192.168.10.0/24, 10.42.0.0/16 } accept
|
||||
# LiteLLM proxy (pods k8s 10.42/16 + nœuds cluster + LAN domestique pour client vocal)
|
||||
tcp dport 4000 ip saddr { 192.168.1.0/24, 192.168.10.0/24, 10.42.0.0/16 } accept
|
||||
|
||||
# Postfix relay SMTP (pods k8s 10.42/16 + nœuds cluster)
|
||||
tcp dport 25 ip saddr { 192.168.10.0/24, 10.42.0.0/16 } accept
|
||||
|
|
|
|||
262
tools/hermes-voice/hermes-voice.py
Normal file
262
tools/hermes-voice/hermes-voice.py
Normal file
|
|
@ -0,0 +1,262 @@
|
|||
#!/usr/bin/env python3
|
||||
"""
|
||||
Hermes Voice Client
|
||||
VAD → Whisper STT → keyword "hermes" → LiteLLM → piper TTS
|
||||
|
||||
Usage:
|
||||
python hermes-voice.py # parlez naturellement, commencez par "Hermes, ..."
|
||||
python hermes-voice.py --ptt # Entrée pour commencer/terminer (plus simple)
|
||||
python hermes-voice.py --no-tts # réponses texte seulement (pour tester)
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import queue
|
||||
import subprocess
|
||||
import tempfile
|
||||
from collections import deque
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
import requests
|
||||
import sounddevice as sd
|
||||
|
||||
# ── Configuration ──────────────────────────────────────────────────────────────
|
||||
# Si port 4000 inaccessible : ouvrir un tunnel SSH avant de lancer :
|
||||
# ssh -L 4000:localhost:4000 user@192.168.1.200
|
||||
# puis changer LITELLM_URL → http://localhost:4000/...
|
||||
LITELLM_URL = "http://192.168.1.200:4000/v1/chat/completions"
|
||||
LITELLM_KEY = "lm-studio"
|
||||
HERMES_MODEL = "hermes-default"
|
||||
SYSTEM_PROMPT = (
|
||||
"Tu es Hermes, l'assistant vocal du homelab Funk. "
|
||||
"Réponds toujours en français, de façon concise (2-3 phrases maximum)."
|
||||
)
|
||||
|
||||
SAMPLE_RATE = 16000
|
||||
BLOCK_SIZE = 512 # 32ms par bloc
|
||||
PRE_ROLL = 10 # blocs à conserver avant la détection (début de phrase)
|
||||
VOICE_THRESH = 0.012 # seuil RMS normalisé pour détecter la parole
|
||||
SILENCE_SEC = 1.5 # silence pour terminer l'enregistrement
|
||||
MIN_SPEECH = 0.8 # durée minimale traitée (filtre les bruits courts)
|
||||
|
||||
WHISPER_SIZE = "small" # base=rapide, small=équilibré, medium=précis
|
||||
KEYWORD = "hermes"
|
||||
KEYWORD_POS = 5 # le mot-clé doit être dans les N premiers mots
|
||||
|
||||
# Piper TTS — binaire : https://github.com/rhasspy/piper/releases
|
||||
# Voix FR : wget .../fr_FR-upmc-medium.onnx → ~/.local/share/piper/
|
||||
PIPER_BIN = Path.home() / ".local/share/piper-runtime/piper"
|
||||
PIPER_VOICE = Path.home() / ".local/share/piper/fr_FR-upmc-medium.onnx"
|
||||
|
||||
# ──────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def beep(freq: int = 880, duration: float = 0.12):
|
||||
t = np.linspace(0, duration, int(SAMPLE_RATE * duration), endpoint=False)
|
||||
wave = (0.25 * np.sin(2 * np.pi * freq * t)).astype(np.float32)
|
||||
sd.play(wave, samplerate=SAMPLE_RATE)
|
||||
sd.wait()
|
||||
|
||||
|
||||
def rms(chunk: np.ndarray) -> float:
|
||||
return float(np.sqrt(np.mean(chunk.astype(np.float32) ** 2)) / 32768.0)
|
||||
|
||||
|
||||
def detect_keyword(text: str) -> tuple[bool, str]:
|
||||
"""Vérifie si KEYWORD est dans les premiers mots et retourne la commande sans le mot-clé."""
|
||||
words = text.strip().split()
|
||||
stripped = [w.lower().strip(",.!?«»") for w in words]
|
||||
found = any(KEYWORD in w for w in stripped[:KEYWORD_POS])
|
||||
if not found:
|
||||
return False, ""
|
||||
try:
|
||||
idx = next(i for i, w in enumerate(stripped) if KEYWORD in w)
|
||||
command = " ".join(words[idx + 1:]).strip()
|
||||
except StopIteration:
|
||||
command = text
|
||||
return True, command or text
|
||||
|
||||
|
||||
def transcribe(whisper_model, audio: np.ndarray) -> str:
|
||||
audio_f32 = audio.astype(np.float32) / 32768.0
|
||||
segments, _ = whisper_model.transcribe(
|
||||
audio_f32, language="fr", beam_size=3, vad_filter=True
|
||||
)
|
||||
return " ".join(seg.text for seg in segments).strip()
|
||||
|
||||
|
||||
def ask_hermes(text: str) -> str:
|
||||
resp = requests.post(
|
||||
LITELLM_URL,
|
||||
json={
|
||||
"model": HERMES_MODEL,
|
||||
"messages": [
|
||||
{"role": "system", "content": SYSTEM_PROMPT},
|
||||
{"role": "user", "content": text},
|
||||
],
|
||||
"max_tokens": 300,
|
||||
"temperature": 0.7,
|
||||
},
|
||||
headers={"Authorization": f"Bearer {LITELLM_KEY}"},
|
||||
timeout=30,
|
||||
)
|
||||
resp.raise_for_status()
|
||||
return resp.json()["choices"][0]["message"]["content"].strip()
|
||||
|
||||
|
||||
def speak(text: str):
|
||||
voice = Path(PIPER_VOICE).expanduser()
|
||||
bin_path = Path(PIPER_BIN).expanduser()
|
||||
if not voice.exists():
|
||||
print(f" [TTS] voix introuvable : {voice}")
|
||||
return
|
||||
if not bin_path.exists():
|
||||
print(f" [TTS] piper introuvable : {bin_path}")
|
||||
return
|
||||
with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as f:
|
||||
out = Path(f.name)
|
||||
# Les .so de piper doivent être dans le même répertoire que le binaire
|
||||
lib_dir = str(bin_path.parent)
|
||||
env = {"LD_LIBRARY_PATH": lib_dir}
|
||||
try:
|
||||
subprocess.run(
|
||||
[str(bin_path), "--model", str(voice), "--output_file", str(out)],
|
||||
input=text.encode(),
|
||||
capture_output=True,
|
||||
check=True,
|
||||
env=env,
|
||||
)
|
||||
subprocess.run(["aplay", "-q", str(out)], check=False)
|
||||
except FileNotFoundError:
|
||||
print(" [TTS] piper non trouvé — voir https://github.com/rhasspy/piper/releases")
|
||||
finally:
|
||||
out.unlink(missing_ok=True)
|
||||
|
||||
|
||||
def handle(whisper_model, audio: np.ndarray, no_tts: bool, require_keyword: bool = True):
|
||||
if len(audio) / SAMPLE_RATE < MIN_SPEECH:
|
||||
return
|
||||
|
||||
print(" ⏳ Transcription...", end=" ", flush=True)
|
||||
text = transcribe(whisper_model, audio)
|
||||
if not text:
|
||||
print("(rien compris)")
|
||||
return
|
||||
|
||||
if require_keyword:
|
||||
found, command = detect_keyword(text)
|
||||
if not found:
|
||||
print(f"(ignoré : \"{text[:60]}\")")
|
||||
return
|
||||
query = command
|
||||
else:
|
||||
query = text
|
||||
|
||||
print(f"\n 🗣️ Vous : {text}")
|
||||
print(" ⏳ Hermes...", end=" ", flush=True)
|
||||
try:
|
||||
response = ask_hermes(query)
|
||||
except requests.RequestException as e:
|
||||
print(f"\n ❌ Réseau : {e}")
|
||||
return
|
||||
print(f"\n 🤖 Hermes : {response}\n")
|
||||
|
||||
if not no_tts:
|
||||
speak(response)
|
||||
|
||||
|
||||
def run_vad(whisper_model, no_tts: bool):
|
||||
"""Écoute continue par VAD — commencer par 'Hermes, ...' pour déclencher."""
|
||||
print(f"🎙️ En écoute — commencez par \"{KEYWORD.capitalize()}, ...\" (Ctrl+C pour quitter)\n")
|
||||
|
||||
audio_q: queue.Queue = queue.Queue()
|
||||
pre_roll: deque = deque(maxlen=PRE_ROLL)
|
||||
|
||||
def callback(indata, frames, time_info, status):
|
||||
chunk = (indata[:, 0] if indata.ndim > 1 else indata.flatten()).copy()
|
||||
audio_q.put(chunk)
|
||||
|
||||
with sd.InputStream(samplerate=SAMPLE_RATE, channels=1, dtype="int16",
|
||||
blocksize=BLOCK_SIZE, callback=callback):
|
||||
while True:
|
||||
chunk = audio_q.get()
|
||||
pre_roll.append(chunk)
|
||||
|
||||
if rms(chunk) < VOICE_THRESH:
|
||||
continue
|
||||
|
||||
# Voix détectée — enregistrer jusqu'au silence
|
||||
print(" 🔴 ", end="", flush=True)
|
||||
captured = list(pre_roll)
|
||||
silent_blocks = 0
|
||||
required_silent = int(SILENCE_SEC * SAMPLE_RATE / BLOCK_SIZE)
|
||||
|
||||
while True:
|
||||
try:
|
||||
chunk = audio_q.get(timeout=3.0)
|
||||
except queue.Empty:
|
||||
break
|
||||
captured.append(chunk)
|
||||
if rms(chunk) < VOICE_THRESH:
|
||||
silent_blocks += 1
|
||||
else:
|
||||
silent_blocks = 0
|
||||
print(".", end="", flush=True)
|
||||
if silent_blocks >= required_silent:
|
||||
break
|
||||
|
||||
print()
|
||||
beep(440)
|
||||
handle(whisper_model, np.concatenate(captured), no_tts, require_keyword=True)
|
||||
|
||||
|
||||
def run_ptt(whisper_model, no_tts: bool):
|
||||
"""Push-to-talk : Entrée pour commencer, Entrée pour terminer."""
|
||||
print("🎙️ Mode push-to-talk — Entrée pour parler, Entrée pour terminer (Ctrl+C pour quitter)\n")
|
||||
|
||||
while True:
|
||||
input("[ ↵ Entrée pour parler ]")
|
||||
audio_q: queue.Queue = queue.Queue()
|
||||
|
||||
def callback(indata, frames, time_info, status):
|
||||
chunk = (indata[:, 0] if indata.ndim > 1 else indata.flatten()).copy()
|
||||
audio_q.put(chunk)
|
||||
|
||||
print(" 🔴 Enregistrement... (Entrée pour terminer)")
|
||||
with sd.InputStream(samplerate=SAMPLE_RATE, channels=1, dtype="int16",
|
||||
blocksize=BLOCK_SIZE, callback=callback):
|
||||
input()
|
||||
chunks = []
|
||||
while not audio_q.empty():
|
||||
chunks.append(audio_q.get_nowait())
|
||||
|
||||
if not chunks:
|
||||
continue
|
||||
beep(440)
|
||||
handle(whisper_model, np.concatenate(chunks), no_tts, require_keyword=False)
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="Hermes Voice Client")
|
||||
parser.add_argument("--ptt", action="store_true", help="Mode push-to-talk")
|
||||
parser.add_argument("--no-tts", action="store_true", help="Réponses texte seulement")
|
||||
args = parser.parse_args()
|
||||
|
||||
print("Hermes Voice Client")
|
||||
print("=" * 40)
|
||||
print(f" ⏳ Chargement Whisper ({WHISPER_SIZE})...", end=" ", flush=True)
|
||||
from faster_whisper import WhisperModel
|
||||
whisper = WhisperModel(WHISPER_SIZE, device="cpu", compute_type="int8")
|
||||
print("✓\n")
|
||||
|
||||
try:
|
||||
if args.ptt:
|
||||
run_ptt(whisper, args.no_tts)
|
||||
else:
|
||||
run_vad(whisper, args.no_tts)
|
||||
except KeyboardInterrupt:
|
||||
print("\n👋 Au revoir")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
4
tools/hermes-voice/requirements.txt
Normal file
4
tools/hermes-voice/requirements.txt
Normal file
|
|
@ -0,0 +1,4 @@
|
|||
faster-whisper>=1.0.0
|
||||
sounddevice>=0.4.6
|
||||
numpy>=1.24.0
|
||||
requests>=2.31.0
|
||||
Loading…
Add table
Add a link
Reference in a new issue