Operating Narwhal¶
Each model fleet runs one router pair in a shared lease domain. The lease holder admits and places requests, and its peer runs as standby.
clients
|
TLS, authentication, WAF, model routing
|---------------- router pair A ---------------- fleet A, model A
|---------------- router pair B ---------------- fleet B, model B
`---------------- router pair C ---------------- fleet C, model C
Operator tasks¶
-
Production boundary and router pair
Set the production boundary, keep one deployment set, configure the client path, and start a router pair.
-
Router state and placement monitoring
Interpret router state and monitor placement and control.
-
Engine restart and process replacement
Restart one engine or an engine wave, activate replacement profiles, and detect process replacement.
-
Upgrade routers across compatible or changed handoff versions and roll back.
-
Validate every release with restart, whole-wave, and failover drills.
-
Run engine actions, configuration overlays, and load jobs from the control service and its console, and read each session's run record.
-
Choosing admission, queue and retry settings
Choose the admission mode, queue, retries, failure quarantine, in-flight limit and decode-gap limit from the client outcomes of each value.
-
Diagnose overload, engine failures, or router failures from the first visible symptom.
-
Profile the fleet, calibrate SLOs, and measure a target workload.