The Moment of Arrival: Diagnosing and Defeating IOMMU-Corrupted P2P DMA on Blackwell GPUs

Introduction

In any complex systems engineering effort, there is a moment when the fog clears and the machine finally responds. Message [msg 6227] captures exactly such a moment in an opencode coding session deploying large language models across eight NVIDIA RTX PRO 6000 Blackwell GPUs. After hours of debugging hangs, silent data corruption, and kernel-level page faults, the assistant writes: "Server is up! It loaded the model (~60 GB per GPU for weights), then allocated KV cache (77 GB per GPU). The P2P disable was the fix. Let me smoke test."

This short message is a culmination — a victory lap after one of the most subtle and consequential debugging sequences in the entire session. It represents the moment when a deep, architectural incompatibility between AMD's IOMMU virtualization and NVIDIA's peer-to-peer GPU DMA was identified, isolated, and worked around, allowing a 122-billion-parameter language model to serve requests across four Blackwell GPUs. To understand why this message matters, we must trace the reasoning that led to it.

The Debugging Context: Silent Data Corruption at the Kernel Level

The story begins with a seemingly routine deployment task. The assistant had been asked to reconfigure the GPU topology on a Proxmox host, splitting eight RTX PRO 6000 Blackwell GPUs between two environments: four GPUs bound to the NVIDIA driver for an LXC container running SGLang (the inference serving framework), and four GPUs passed through to a SEV-SNP confidential VM via vfio-pci. The model in question was Qwen3.5-122B-A10B, a 122-billion-parameter Mixture-of-Experts model in BF16 precision, requiring approximately 234 GB of memory — comfortably fitting across four 96 GB GPUs with tensor parallelism.

The initial server launch failed. SGLang would hang during init_torch_distributed, the NCCL-based distributed initialization that sets up communication between GPUs. The assistant's first clue came from the Proxmox host kernel logs, which the user helpfully provided in [msg 6191]: repeated IO_PAGE_FAULT events from the AMD-Vi IOMMU, targeting the NVIDIA GPU PCIe devices. The flags 0x0030 indicated write operations to unmapped DMA addresses — the GPUs were trying to write to each other's memory through the IOMMU, and the IOMMU was blocking every attempt.

The assistant's reasoning at this point was critical. It understood that the SEV-SNP (Secure Encrypted Virtualization-Secure Nested Paging) configuration required amd_iommu=on in full translation mode, as confirmed by reading the Proxmox host's kernel command line in [msg 6209]. Previously, the IOMMU had likely been in passthrough mode (iommu=pt), which would have allowed GPU P2P DMA to bypass address translation entirely. But SEV-SNP cannot work with passthrough IOMMU — it needs full translation to enforce memory encryption and isolation boundaries for confidential VMs. This was not a configuration mistake; it was a fundamental architectural conflict between the security requirements of SEV-SNP and the performance requirements of GPU P2P communication.

Systematic Isolation: From Symptoms to Root Cause

The assistant's diagnostic approach was methodical. First, it confirmed that single-GPU operations worked correctly — a simple PyTorch tensor creation on cuda:0 succeeded ([msg 6214]). Next, it checked CUDA's peer access reporting using torch.cuda.can_device_access_peer(), which returned True for every GPU pair ([msg 6215]). This was a red herring: the NVIDIA driver believed P2P was available, but the actual data transfers told a different story.

The decisive test came in [msg 6216]: a straightforward GPU-to-GPU copy using PyTorch's .to() method. Every single transfer produced a MISMATCH — the data arriving at the destination GPU was corrupted. All 12 directed pairs across 4 GPUs failed. The assistant's comment — "Every P2P transfer produces data mismatches" — was the moment of diagnostic clarity. The IOMMU was corrupting every P2P DMA transaction silently. The driver reported canAccessPeer=True because the PCIe topology supported it, but the IOMMU translation layer was scrambling addresses and data.

This is a particularly dangerous class of bug: silent data corruption. Unlike a hard crash or an explicit error message, corrupted P2P transfers can manifest as subtle numerical errors, random hangs, or inexplicable convergence failures in distributed training. In this case, the corruption caused NCCL's all-reduce to compute garbage values, which in turn caused the distributed barrier to deadlock as ranks received inconsistent data.

The Fix: NCCL_P2P_DISABLE=1

With the root cause identified, the assistant formulated a workaround. NCCL, NVIDIA's collective communications library, supports multiple transport mechanisms: P2P (direct GPU-to-GPU via NVLink or PCIe), SHM (shared memory via CPU), and network (via InfiniBand or Ethernet). By setting NCCL_P2P_DISABLE=1, NCCL would fall back to SHM transport, routing GPU-to-GPU communication through the CPU's shared memory rather than attempting direct DMA.

The assistant validated this approach with a dedicated NCCL test script in [msg 6219]. With NCCL_P2P_DISABLE=1, all four ranks completed all_reduce successfully, producing the correct result (10.0 = 1+2+3+4). The fix worked.

But the assistant didn't stop there. It recognized that this environment variable needed to be set everywhere — not just in the current shell, but in the Python runtime and in the systemd service that would manage the production deployment. In [msg 6221], it updated /usr/lib/python3.12/sitecustomize.py to set NCCL_P2P_DISABLE=1 (along with other NCCL tuning parameters like NCCL_PROTO=LL, NCCL_ALGO=Ring, NCCL_MAX_NCHANNELS=16, NCCL_BUFFSIZE=16777216, and NCCL_NTHREADS=512). In [msg 6223], it edited the systemd service file to include the same environment variables. This belt-and-suspenders approach ensured that whether SGLang was launched manually or via systemd, the P2P workaround would be active.

The Message Itself: What It Reveals

Message [msg 6227] is deceptively brief. It contains three pieces of information, each significant:

"Server is up!" — After repeated failures, the SGLang server has successfully completed distributed initialization and is listening for requests. This is the first confirmation that the NCCL_P2P_DISABLE workaround is sufficient to unblock the entire deployment pipeline.

"It loaded the model (~60 GB per GPU for weights), then allocated KV cache (77 GB per GPU)." — These numbers reveal the memory distribution. The Qwen3.5-122B-A10B model's weights occupy approximately 60 GB per GPU (240 GB total across 4 GPUs, consistent with BF16 storage of ~122B parameters plus MoE overhead). The KV cache allocation of 77 GB per GPU is generous — this is the maximum cache size configured for the server, representing the total memory remaining after weights are loaded. In practice, the KV cache will be consumed incrementally as requests arrive.

"The P2P disable was the fix." — This sentence encapsulates the entire debugging arc. It is the assistant's explicit attribution of causality, confirming that the IOMMU-P2P conflict was the sole obstacle to deployment.

The todowrite block shows the assistant's task tracking: three items completed (research the model, download it to /shared, update the SGLang service) and one in progress (smoke test). This structured approach to task management is characteristic of the opencode session, where complex multi-step operations are decomposed into tracked subtasks.

Assumptions and Their Implications

The assistant made several assumptions in this message, most of which were reasonable but worth examining:

Assumption 1: NCCL SHM transport is performant enough for 4 GPUs on PCIe. P2P DMA is significantly faster than SHM for GPU-to-GPU transfers because it bypasses the CPU entirely. On PCIe Gen5, P2P can achieve ~64 GB/s per direction, while SHM is limited by memory bandwidth and CPU involvement. However, for a 4-GPU inference deployment where the primary communication pattern is all-reduce of activations (relatively small tensors compared to weights), the SHM fallback may be adequate. The assistant implicitly assumed that the throughput penalty was acceptable — an assumption that would need to be validated through benchmarking.

Assumption 2: The IOMMU configuration cannot be changed. The assistant accepted the constraint that amd_iommu=on in full translation mode was required for SEV-SNP and did not attempt to modify it. This was correct given the user's explicit instruction: "Do not reboot the proxmox host or change bios settings yourself, another tenant is active on the machine so changes will need coordination" ([msg 6196]). The workaround was the only viable path.

Assumption 3: The corruption affects all P2P transfers uniformly. The assistant's test covered all 12 directed pairs across 4 GPUs, and all failed. This suggested a systematic issue rather than a device-specific fault. The assumption held.

Assumption 4: The model will serve requests correctly with P2P disabled. This was the critical assumption awaiting validation. The P2P disable fixed the distributed initialization hang, but did it fix the model's numerical correctness? Corrupted P2P during initialization could have left behind silent state corruption even if the server started. The smoke test in the subsequent message ([msg 6228]) — a simple arithmetic query returning "91" — confirmed basic functional correctness, but deeper validation would require more comprehensive testing.

Input Knowledge Required

To fully understand this message, one needs knowledge spanning multiple domains:

Output Knowledge Created

This message produces several pieces of actionable knowledge:

  1. A confirmed workaround: NCCL_P2P_DISABLE=1 resolves the IOMMU-induced P2P corruption for Blackwell GPUs under AMD-Vi full translation mode.
  2. Memory budget figures: Qwen3.5-122B-A10B in BF16 consumes ~60 GB per GPU for weights across 4 GPUs, leaving ~77 GB per GPU for KV cache.
  3. A validated deployment configuration: The combination of NCCL_P2P_DISABLE, NCCL_PROTO=LL, NCCL_ALGO=Ring, and tuned channel/buffer parameters works for 4× Blackwell GPUs over PCIe Gen5.
  4. A diagnostic methodology: The sequence of tests — single-GPU sanity, peer access capability, actual data transfer, NCCL all-reduce — provides a template for diagnosing P2P issues in any GPU cluster.

The Thinking Process Visible in the Message

While the message itself is terse, the thinking process is visible in its structure. The assistant leads with the good news ("Server is up!"), immediately provides the quantitative evidence of success (memory allocation numbers), states the causal attribution ("The P2P disable was the fix"), and then pivots to the next action ("Let me smoke test"). This is classic engineering communication: state the result, provide supporting data, explain the root cause, and move to validation.

The todowrite block reveals the assistant's awareness of the broader workflow. Three items are completed, one is in progress. The assistant is not celebrating prematurely — it recognizes that smoke testing is still needed to confirm the server actually produces correct outputs. This disciplined approach prevents the kind of false victory where the server starts but serves garbage.

Conclusion

Message [msg 6227] is a turning point in a complex debugging journey. It represents the moment when a deep, architectural incompatibility between AMD's SEV-SNP IOMMU configuration and NVIDIA's GPU P2P DMA was successfully worked around, allowing a 122-billion-parameter language model to deploy across four Blackwell GPUs. The message is brief, but it carries the weight of hours of diagnostic work: kernel log analysis, systematic test design, environment variable tuning, and service configuration. It is a testament to the value of methodical debugging — and a reminder that sometimes the most elegant fix is not to fix the root cause at all, but to route around it.