mirror of
https://github.com/Alkatrazz24/Funk-lab.git
synced 2026-07-12 20:14:42 +02:00
feat(llama_server): watchdog auto-réparation du wedge ROCm
Service systemd local sur gpu-01 qui sonde une vraie génération sur :1234 (pas juste /health, qui ment quand le slot d'inférence est figé). Sur N échecs consécutifs → systemctl restart llama-server en local (root, sans SSH/sudo distant). Gère le 503 "Loading model" post-restart sans le compter comme échec. Complète le llm-heartbeat (rôle litellm) : le heartbeat empêche la péremption par inactivité de la connexion ; le watchdog répare le figeage du serveur lui-même. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
6e6dd45ac5
commit
0cb886acf7
7 changed files with 157 additions and 5 deletions
|
|
@ -123,3 +123,31 @@
|
|||
state: enabled
|
||||
immediate: true
|
||||
when: llama_embed_enabled
|
||||
|
||||
# --- Watchdog auto-réparation (wedge ROCm) — optionnel ------------------------
|
||||
|
||||
- name: Deploy llama-watchdog script
|
||||
ansible.builtin.template:
|
||||
src: llama-watchdog.sh.j2
|
||||
dest: /usr/local/bin/llama-watchdog
|
||||
mode: '0755'
|
||||
notify: Restart llama-watchdog
|
||||
when: llama_watchdog_enabled
|
||||
|
||||
- name: Deploy llama-watchdog service
|
||||
ansible.builtin.template:
|
||||
src: llama-watchdog.service.j2
|
||||
dest: /etc/systemd/system/llama-watchdog.service
|
||||
mode: '0644'
|
||||
notify:
|
||||
- Reload systemd
|
||||
- Restart llama-watchdog
|
||||
when: llama_watchdog_enabled
|
||||
|
||||
- name: Enable llama-watchdog service
|
||||
ansible.builtin.systemd:
|
||||
name: llama-watchdog
|
||||
enabled: true
|
||||
daemon_reload: true
|
||||
state: started
|
||||
when: llama_watchdog_enabled
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue