Waiting for a Server That Never Comes: Diagnosing SGLang Startup Failure After IOMMU-Induced P2P Corruption

The Message

ssh root@10.1.230.174 'for i in $(seq 1 60); do
  if curl -s http://localhost:30000/v1/models 2>/dev/null | grep -q qwen; then
    echo "SERVER READY after ${i}0s"; exit 0;
  fi;
  stat=$(systemctl is-active sglang-qwen 2>/dev/null);
  if [ "$stat" = "failed" ] || [ "$stat" = "inactive" ]; then
    echo "SERVICE $stat";
    journalctl -u sglang-qwen --no-pager -n 20;
    exit 1;
  fi;
  mem=$(nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits 2>/dev/null | paste -sd,);
  echo "${i}0s ($stat) mem=$mem";
  sleep 10;
done;
echo "TIMEOUT";
journalctl -u sglang-qwen --no-pager -n 20'

Output:

10s (active) mem=757,757,757,757
20s (active) mem=59607,59607,59607,59607
30s (active) mem=59607,59607,59607,59607
40s (active) mem=59607,59607,59607,59607
50s (active) mem=59607,59607,59607,59607
60s (active) mem=59607,59607,59607,59607
70s (active) mem=59607,59607,59607,59607
80s (active) mem=59607,59607,59607,59607
90s (active) mem=59607,59607,59607,59607
100s (active) mem=59607,59607,59607,59607
110s (active) mem=59607,59607,59607,59607
120s (active) mem=59607,59607,59607,59607
130s (active)...

This message, sent by the assistant in an opencode coding session, is a deceptively simple bash command that tells a dramatic story. On its surface, it is a health-check polling loop — a standard pattern for waiting on a server to become ready. But the output reveals that something has gone profoundly wrong. The SGLang inference server, tasked with serving a Qwen3.5-122B-A10B BF16 model across four NVIDIA RTX PRO 6000 Blackwell GPUs, loads its model (memory jumps from 757 MB to 59.6 GB per GPU at the 20-second mark) and then simply... stops. The service remains "active" — the process hasn't crashed — but the HTTP endpoint never responds. The assistant waits through 130 seconds of silence before the output is truncated, the server never having become ready.

The Context: A Chain of Failures

To understand why this message exists, we must trace the events that led to it. The session had been working through a complex deployment pipeline for large language models on a Proxmox VE host with 8× RTX PRO 6000 Blackwell GPUs. Earlier in segment 40, the assistant had reconfigured the GPU topology, splitting the 8 GPUs so that 4 remained bound to the NVIDIA driver for an LXC container running SGLang, while the other 4 were moved to vfio-pci for passthrough to an SEV-SNP confidential VM.

This reconfiguration triggered a catastrophic side effect. The SEV-SNP (Secure Encrypted Virtualization with Secure Nested Paging) configuration required the AMD IOMMU to operate in full translation mode (amd_iommu=on on the kernel command line), rather than the passthrough mode (iommu=pt) that had previously been used. In full translation mode, the IOMMU intercepts and translates every DMA transaction. GPU-to-GPU Peer-to-Peer (P2P) DMA — the mechanism by which GPUs directly exchange data across the PCIe bus without involving host memory — relies on physical addresses that the IOMMU does not know how to translate. The result, as the assistant discovered in preceding messages ([msg 6216]), was that every P2P transfer produced silently corrupted data. A CUDA diagnostic showed that torch.cuda.can_device_access_peer() returned True (the driver believed P2P was functional), but actual data copies between GPUs resulted in mismatches on every single pair. The host kernel log was filled with IO_PAGE_FAULT events from the AMD-Vi IOMMU, confirming that the DMA engine was attempting to access unmapped addresses.## The Fix That Preceded This Message

In the messages immediately before this one ([msg 6221], [msg 6223]), the assistant had implemented a workaround. Recognizing that NCCL (NVIDIA Collective Communications Library) was hanging during init_torch_distributed because it tried to use P2P DMA for all-reduce operations and got back corrupted data, the assistant disabled P2P entirely. It updated the Python sitecustomize.py on the container to set NCCL_P2P_DISABLE=1, forcing NCCL to fall back to shared memory (SHM) transport for inter-GPU communication. It also tuned NCCL parameters for the PCIe Gen5 topology: NCCL_PROTO=LL (low-latency protocol), NCCL_ALGO=Ring (ring all-reduce), NCCL_MAX_NCHANNELS=16, NCCL_BUFFSIZE=16777216, and NCCL_NTHREADS=512. Additionally, it edited the systemd service file (sglang-qwen.service) to include NCCL_P2P_DISABLE=1 and CUDA_DEVICE_MAX_CONNECTIONS=1 in the environment.

With these changes in place, the assistant killed any lingering SGLang processes, freed the GPU devices, copied the updated service file to the container, reloaded systemd, and started the service. The command systemctl start sglang-qwen returned "starting..." ([msg 6225]), indicating the service had been launched.

What This Message Reveals

The polling loop in message 6226 is the assistant's attempt to verify that the fix actually worked. It is a carefully constructed diagnostic script that serves multiple purposes simultaneously:

  1. Health check: It polls the SGLang HTTP endpoint (/v1/models) to detect when the server is ready to accept requests. This is the standard readiness probe for OpenAI-compatible inference servers.
  2. Service monitoring: It checks systemctl is-active sglang-qwen to detect if the service has crashed. If the service transitions to "failed" or "inactive", it dumps the journal logs and exits immediately — a faster path to debugging than waiting for a timeout.
  3. Memory tracking: It records nvidia-smi memory usage across all four GPUs at each 10-second interval. This is a powerful diagnostic signal: the jump from 757 MB to 59,607 MB per GPU at the 20-second mark tells us the model weights were loaded into GPU memory. The fact that memory stays flat at 59.6 GB for the remaining 110+ seconds tells us the model loaded successfully but the server initialization got stuck after that point.
  4. Timeout with evidence: If the server never becomes ready after 60 iterations (600 seconds), it dumps the journal logs. In this case, the output was truncated at 130 seconds, but the pattern is clear — the server is stuck. The output reveals a specific failure mode: the model loads (memory allocation succeeds), but the server never opens its HTTP port. This is consistent with a hang during the NCCL distributed initialization phase that occurs after model loading. In SGLang's startup sequence, the model is first loaded onto the GPUs, then the distributed runtime (tensor parallelism across 4 GPUs) is initialized. If NCCL hangs during init_process_group or the first collective operation, the process remains alive (hence "active") but never reaches the HTTP server startup.

The Thinking Process Behind the Script

The assistant's reasoning is visible in the structure of the script. It chose a polling interval of 10 seconds — long enough to avoid hammering the server during normal startup (which typically takes 30-90 seconds for a 122B parameter model), but short enough to detect failures quickly. The 60-iteration limit (10 minutes) is generous, acknowledging that this was the first launch after a significant configuration change and might be slower than usual.

The conditional branch for service failure (journalctl -u sglang-qwen --no-pager -n 20) shows that the assistant anticipated the possibility of a crash. The fact that the service remained "active" throughout ruled out a segfault or Python exception — the process was alive but blocked. This is a harder problem to debug than a crash, because there is no error message to read.

The memory tracking is the most insightful part of the script. By recording GPU memory at each interval, the assistant created a timeline of model loading progress. The flat memory after 20 seconds tells us that the model weights were successfully allocated and presumably loaded. If the memory had stayed at 757 MB, it would indicate a failure to even begin loading. If it had slowly increased, it would indicate progressive loading. The sudden jump followed by stasis points to a post-loading hang.

Assumptions and Potential Mistakes

The assistant made several assumptions in this message. First, it assumed that the NCCL_P2P_DISABLE=1 fix would be sufficient to unblock SGLang startup. This was a reasonable hypothesis based on the NCCL test ([msg 6219]) which showed all-reduce working correctly with P2P disabled. However, SGLang's initialization path may involve other code paths that also use P2P DMA — for example, custom all-reduce kernels, CUDA graphs, or direct GPU memory copies that bypass NCCL entirely.

Second, the assistant assumed that the environment variables set in sitecustomize.py and the systemd service file would take effect. The sitecustomize.py approach is a Python-level mechanism that runs before any user code, but it depends on Python being invoked in a way that loads sitecustomize. If SGLang uses a subprocess or a different Python interpreter, those settings might not propagate.

Third, the assistant assumed that the driver version mismatch (container userspace at version 565 vs host kernel at version 590) had been resolved. Earlier in the chunk, the assistant had installed matching 590 userspace packages, but the fix might not have been complete — certain CUDA libraries might still be from the 565 installation.

A potential mistake was not checking the SGLang logs directly before launching the polling loop. The journalctl dump at timeout would have shown the server's stdout/stderr, but the assistant could have saved time by tailing the log file immediately after startup. The polling loop is a "wait and see" approach, which is appropriate when the expected outcome is success, but less efficient when the fix is speculative.

Input and Output Knowledge

To understand this message, the reader needs knowledge of: the SGLang inference server architecture (model loading followed by distributed initialization), the NCCL communication library and its P2P vs SHM transport modes, the systemd service management model, the NVIDIA driver and CUDA stack, and the AMD IOMMU behavior under SEV-SNP. The reader also needs the context from the preceding messages: the GPU topology reconfiguration, the IO_PAGE_FAULT diagnosis, the P2P corruption test, and the NCCL_P2P_DISABLE=1 fix.

The message creates new knowledge: it confirms that the NCCL_P2P_DISABLE=1 fix is insufficient to make SGLang start successfully. The server loads its model but hangs during post-loading initialization. This output knowledge drives the next phase of debugging — the assistant will need to investigate whether there are other P2P-dependent code paths in SGLang, whether the NCCL settings are being properly applied, or whether the hang has a different root cause entirely.

A Deeper Look at the Failure

The 59.6 GB memory allocation per GPU is notable. The Qwen3.5-122B-A10B model in BF16 precision requires approximately 122 billion parameters × 2 bytes = 244 GB of memory for the weights alone, plus additional memory for KV cache and activations. With 4 GPUs at 96 GB each (384 GB total), the model fits comfortably. The 59.6 GB per GPU represents roughly 238 GB total across 4 GPUs, which is consistent with the model weights plus some overhead. The fact that this allocation succeeded tells us the CUDA memory management layer is functional — the problem is not at the memory allocation level but at the communication layer.

The hang is almost certainly in NCCL's init_process_group or the first collective operation. With P2P disabled, NCCL falls back to SHM (shared memory) transport, which copies data through host memory via CPU. This is slower but should work. However, NCCL's SHM transport may itself depend on DMA capabilities that are broken under the IOMMU configuration. Alternatively, SGLang may have custom CUDA kernels that perform direct GPU-to-GPU copies outside of NCCL, bypassing the NCCL_P2P_DISABLE flag entirely.

The message ends with the output truncated at 130 seconds, but the polling loop would have continued to 600 seconds before dumping the journal. The assistant eventually would have seen the SGLang logs, which would reveal the exact point of the hang — whether it's in NCCL initialization, CUDA graph compilation, or some other initialization step. That information would determine the next debugging step.

Conclusion

Message 6226 is a moment of tension in the coding session. The assistant has diagnosed a subtle and complex problem (IOMMU-breaking P2P DMA under SEV-SNP), implemented a seemingly correct fix (disabling P2P and tuning NCCL), and is now waiting to see if the server will start. The output shows it won't — at least not yet. The model loads, the memory allocates, and then everything stops. The assistant's carefully constructed polling script captures this failure mode with precision, turning a silent hang into structured diagnostic data. The message is a testament to the reality of systems engineering: a correct diagnosis does not guarantee a complete fix, and the gap between "NCCL works in a test script" and "SGLang starts successfully" can be vast. The debugging continues.