Narwhal documentation¶

Narwhal runs disaggregated LLM inference and reassigns prefill and decode roles as demand changes. Model weights stay loaded during role changes.

From a management workstation, Narwhal inspects the fleet, derives its configuration, prepares the remote engines, and verifies routed inference over the private deployment path.

Narwhal, adaptive disaggregated inference on a role-free fleet

Start with the task you need to complete¶

Goal	Guide
Install the router commands from PyPI and check their version	Install from PyPI
Run a local NVIDIA GPU fleet on Ubuntu or WSL2	Narwhal dev
Bring up a new fleet and confirm that requests are routed correctly	Deploy a fleet
Profile the fleet, select measurement ranges, calibrate SLOs, and measure a target workload	Measure a fleet
Export metrics to Prometheus and inspect the fleet in Grafana	Set up observability
Manage ingress, routers, engines, and software upgrades	Operate Narwhal
Diagnose overload, engine failures, or router failures from the first visible symptom	Troubleshoot a fleet

Understand and configure the system¶

System model¶

Core concepts describes the contracts that govern engines, request placement, role control, and fleet state.

Fleet configuration¶

Configuration defines fleet configuration fields, environment inputs, and their defaults.

Command-line interface¶

CLI reference documents available commands and options, including command exit behaviour.

HTTP interfaces¶

HTTP API reference covers completion, inspection, and lifecycle endpoints.

Runtime data and persisted artifacts¶

Telemetry and artifact reference documents journals, profiles, metrics, and persisted contract versions.

Development¶

To build or modify Narwhal itself, see Contributing for development environment setup, the local test workflow, and the pull request process.