Skip to content

Accessing dashboards and isolating listeners

Accessing the dashboard from a workstation

An open deployment tunnel already forwards both monitoring ports.

To open a monitoring tunnel:

  1. Open a terminal in the management checkout with the workstation .env loaded.
  2. Open the tunnel:

    python3 tools/deployment/deploy_hosts.py tunnel --role router \
      --forward 13000:3000 --forward 19090:9090
    
  3. Keep the tunnel terminal open.

The dashboard opens at http://127.0.0.1:13000/d/narwhal-router/narwhal-orchestrator, and Prometheus opens at http://127.0.0.1:19090. Grafana grants anonymous Viewer access through the local tunnel.

To add the fleet control service port to this tunnel, follow Reaching the console through the tunnel.

The dashboard's Fleet control link opens the fleet control console in a new tab. To set it up, follow Using the console from Grafana.

Isolating a second monitoring stack

Grafana listens on NARWHAL_GRAFANA_BIND_ADDRESS, default 127.0.0.1. Prometheus listens on NARWHAL_PROMETHEUS_LISTEN_ADDRESS, default 127.0.0.1:9090.

When another monitoring deployment shares the router host, run make observe with distinct loopback listeners:

NARWHAL_GRAFANA_BIND_ADDRESS=127.0.0.2 \
NARWHAL_PROMETHEUS_LISTEN_ADDRESS=127.0.0.2:19090 \
make observe

Grafana binds port 3000 and the image renderer binds port 8081 on the Grafana address. When that address is a wildcard (0.0.0.0 or ::), Grafana binds the wildcard and the image renderer binds loopback.

The Grafana Prometheus datasource and the readiness probes use the Prometheus address. When that address is a wildcard (0.0.0.0 or ::), Prometheus binds the wildcard and the datasource and probes use loopback.

Tunnel to the 127.0.0.2 listeners:

python3 tools/deployment/deploy_hosts.py tunnel --role router \
  --remote-address 127.0.0.2 \
  --forward 13000:3000 --forward 19090:19090

Read the dashboard