The Diagnostic Pivot: How a One-Sentence Question Uncovered IOMMU-Broken P2P DMA

"Can you run some light cuda test programs to see if the GPUs actually work correctly? Look if p2p transfers still work (they did before host changes), etc."

At first glance, this is a disarmingly simple question. The user is asking the assistant to run a quick health check on the GPUs. But in the context of the preceding 40+ message debugging spiral — where SGLang servers consistently hung at "Init torch distributed begin" across multiple configurations, NCCL env vars, and reinstallations — this single sentence represents a fundamental shift in debugging strategy. It is the moment the conversation pivots from software configuration tweaking to hardware-level diagnosis, and it is the message that ultimately breaks the deadlock.

The Debugging Spiral That Preceded It

To understand why this message was written, one must first understand the agony of the preceding 35 messages ([msg 6156] through [msg 6191]). The assistant had been trying to deploy the Qwen3.5-122B-A10B model across 4 RTX PRO 6000 Blackwell GPUs using SGLang with tensor parallelism (TP=4). Every attempt ended the same way: the server would log "Init torch distributed begin" and then hang indefinitely. GPU memory stayed flat at ~1100 MiB per card — barely enough for NCCL initialization, nowhere near the 234 GB the model required.

The assistant tried every conceivable software fix. NCCL environment variables were toggled: NCCL_P2P_LEVEL was set to PHB, SYS, and unset. NCCL_PROTO was switched to Simple. NCCL_ALGO was forced to Ring. The --disable-custom-all-reduce flag was added. The SGLang source was pulled and reinstalled from the latest main branch. The sitecustomize.py file that injected NCCL defaults was identified and worked around. Strace revealed all four TP worker processes blocked on futex and restart_syscall — classic symptoms of a distributed barrier that never completes. Yet TCP sockets between the ranks were fully established. The hang was happening after NCCL socket setup, somewhere inside torch.distributed.init_process_group or initialize_model_parallel.

Each failed attempt produced the same output, the same hang, the same frustration. The assistant was deep in a local optimum of software debugging, tweaking parameters that could never fix a hardware-level problem.

The User's Insight

The user had been watching this unfold. And they noticed something the assistant had overlooked. In [msg 6191], the user shared kernel logs from the Proxmox host:

[73784.343781] nvidia 0000:01:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x002c address=0x24000000000 flags=0x0030]
[73784.343803] nvidia 0000:11:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x003f address=0x34000001000 flags=0x0030]

These IO_PAGE_FAULT messages from the AMD IOMMU (AMD-Vi) are not normal. They indicate that the NVIDIA GPUs are attempting direct memory access (DMA) operations that the IOMMU is actively blocking. The flags 0x0030 suggest the fault occurred during a DMA read or write operation to an address that the IOMMU's page tables do not have mapped for that device.

The user's message in [msg 6192] connects these dots. By asking the assistant to "run some light cuda test programs" and specifically to check "if p2p transfers still work (they did before host changes)", the user is articulating a hypothesis: the GPU reconfiguration that split the 8 Blackwell GPUs between the LXC container and a SEV-SNP VM likely enabled full IOMMU translation (amd_iommu=on), which in turn broke GPU-to-GPU peer-to-peer DMA. This is not a software configuration issue — it is a fundamental hardware access control problem.

Why This Message Was Necessary

The message was necessary because the assistant's debugging had reached a dead end. The assistant was operating within the assumption that the software stack (SGLang, NCCL, PyTorch) was correctly configured and that the hang must be caused by a subtle compatibility issue, a port conflict, or a race condition in distributed initialization. Every tool in the assistant's debugging arsenal — strace, NCCL debug logging, socket inspection, env var manipulation — was aimed at the software layer.

The user's message reframes the problem. It says: stop debugging the software and verify the hardware. This is a classic debugging maneuver — when higher-level abstractions fail in inexplicable ways, drop down a layer and test the primitives. CUDA P2P transfers are a primitive. If they don't work, nothing built on top of them (NCCL, torch.distributed, SGLang's model parallelism) can work either.

The parenthetical "(they did before host changes)" is crucial. It establishes a baseline: P2P was verified working earlier in the session, before the GPUs were split between containers. The host changes — binding 4 GPUs to vfio-pci for VM passthrough, enabling SEV-SNP with full IOMMU translation — are the most likely cause of the regression. The user is pointing the assistant directly at the diff.

Assumptions Embedded in the Message

The message makes several assumptions, all of which turn out to be correct:

  1. P2P DMA was working before the host changes. This is stated as fact and serves as the control condition. The user had presumably seen earlier benchmarks or tests that confirmed GPU-to-GPU direct transfers functioned.
  2. The host changes are the most likely cause of the regression. This is a reasonable application of Occam's razor — the one thing that changed between "working" and "broken" is the GPU topology reconfiguration and IOMMU settings.
  3. CUDA test programs can definitively diagnose the issue. This is true: a simple CUDA program that allocates memory on two GPUs and attempts a cudaMemcpyPeer or uses cudaDeviceEnablePeerAccess will either succeed or fail with a clear error message, providing unambiguous evidence.
  4. The GPUs themselves are physically functional. The user does not suspect faulty hardware — they suspect a configuration or access control issue. The IO_PAGE_FAULTs point to IOMMU policy, not GPU silicon defects.

The Knowledge Required to Understand This Message

A reader needs substantial context to grasp the full weight of this message:

The Knowledge Created by This Message

This message creates several critical outputs:

  1. A testable hypothesis: P2P DMA is broken due to IOMMU translation. This can be verified with a simple CUDA test.
  2. A direction for the assistant: Run CUDA peer-to-peer tests, check cudaDeviceCanAccessPeer, and verify that direct GPU-to-GPU transfers fail.
  3. A potential fix path: If P2P is indeed broken, the solution is either (a) disable NCCL P2P and force SHM transport, or (b) fix the IOMMU configuration to allow P2P DMA.

The Aftermath

The assistant's response in [msg 6193] shows immediate recognition of the significance: "Those IO_PAGE_FAULTs from AMD-Vi (IOMMU) on the nvidia GPUs are a big red flag — the GPUs are trying to do P2P DMA but IOMMU is blocking it." The assistant then runs CUDA P2P tests, confirms the hypothesis, and ultimately fixes the issue by setting NCCL_P2P_DISABLE=1, forcing NCCL to use shared memory transport instead of broken P2P DMA. The SGLang server then loads successfully, achieving 108 tok/s single-stream and up to 2,800 tok/s at high concurrency.

This one-sentence message — barely 20 words — is the turning point of the entire debugging session. It demonstrates the value of domain knowledge (recognizing IO_PAGE_FAULTs as P2P DMA failures), the importance of questioning assumptions (the assistant assumed a software bug when the problem was hardware configuration), and the power of a well-timed diagnostic question. In the end, the fix was not a code change or a library upgrade — it was a single environment variable, NCCL_P2P_DISABLE=1, that told NCCL to stop trying to use a hardware feature the IOMMU would not allow.