The Silent Corruption: Diagnosing P2P DMA Failures Under IOMMU Translation

Introduction

In the high-stakes world of large language model deployment, few things are more frustrating than a server that hangs during initialization with no clear error message. This article examines a single diagnostic message from an opencode coding session — message 6216 — where an AI assistant discovered the root cause of a persistent NCCL hang by running a simple GPU-to-GPU data transfer test. The finding was devastating: every P2P DMA transfer between GPUs was silently corrupting data, despite the driver confidently reporting that peer access was available. This message represents a textbook example of systematic debugging, where a seemingly simple test exposed a deep hardware-software interaction failure.

The Message

The assistant wrote:

P2P is reported as available (the driver thinks it can do it). Let me test actual P2P data transfer:

>

``bash ssh root@10.1.230.174 '~/ml-env/bin/python3 -c " import torch print(\"Testing GPU-to-GPU copy...\") for i in range(4): for j in range(4): if i == j: continue src = torch.randn(1024, device=f\"cuda:{i}\") dst = src.to(f\"cuda:{j}\") torch.cuda.synchronize(j) ok = torch.allclose(src.cpu(), dst.cpu()) print(f\" GPU {i} -> GPU {j}: {\"OK\" if ok else \"MISMATCH\"}\") print(\"Done\") " 2>&1' ``

>

Output:

>

`` Testing GPU-to-GPU copy... GPU 0 -> GPU 1: MISMATCH GPU 0 -> GPU 2: MISMATCH GPU 0 -> GPU 3: MISMATCH GPU 1 -> GPU 0: MISMATCH GPU 1 -> GPU 2: MISMATCH GPU 1 -> GPU 3: MISMATCH GPU 2 -> GPU 0: MISMATCH GPU 2 -> GPU 1: MISMATCH GPU 2 -> GPU 3: MISMATCH GPU 3 -> GPU 0: MISMATCH GPU 3 -> GPU 1: MISMATCH GPU 3 -> GPU 2: MISMATCH Done ``

Every single GPU-to-GPU transfer across all 12 possible pairs produced corrupted data. Not a single transfer succeeded.

Context: The Road to This Discovery

To understand why this message was written, we must trace the chain of failures that preceded it. The session was in the middle of deploying Qwen3.5-122B-A10B BF16, a 122-billion-parameter language model, across 4 NVIDIA RTX PRO 6000 Blackwell GPUs using SGLang. The server had been reconfigured to split 8 GPUs between an LXC container (4 GPUs for SGLang) and a SEV-SNP confidential VM (4 GPUs for another tenant).

Every attempt to start the SGLang server resulted in a hang at init_torch_distributed. The log showed:

[2026-03-09 12:32:57 TP2] Init torch distributed begin.

...and then nothing. The process was stuck. The assistant spent several messages investigating the hang — checking socket connections, examining strace output, verifying that NCCL sockets were established, and confirming that the TCPStore was operational. All four TP workers were blocked on futex and socket reads, waiting for something that never arrived.

The user then provided a crucial clue: IO_PAGE_FAULT errors from the AMD-Vi IOMMU driver in the host's dmesg logs:

[73784.343781] nvidia 0000:01:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x002c address=0x24000000000 flags=0x0030]
[73784.343803] nvidia 0000:11:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x003f address=0x34000001000 flags=0x0030]

These IO_PAGE_FAULTs were the smoking gun. The GPUs were attempting DMA transfers to addresses that looked like other GPUs' BAR (Base Address Register) memory regions — the telltale sign of P2P GPU-to-GPU direct memory access. But the IOMMU, now in full translation mode (amd_iommu=on) to support SEV-SNP confidential computing, was blocking these transfers.

Why This Message Was Written

The assistant wrote this message to answer a specific question: Is P2P DMA actually working, or is it silently failing? This question arose from a critical observation about the system configuration.

The Proxmox host's kernel command line included amd_iommu=on and mem_encrypt=on — both required for SEV-SNP. But there was no iommu=pt (passthrough) flag, which would have allowed devices to bypass IOMMU translation for DMA. In passthrough mode, the IOMMU still provides protection but with minimal translation overhead. In full translation mode, every DMA transaction goes through the IOMMU's page tables, and if those tables aren't properly configured for GPU P2P BAR mappings, the transfers fail.

The assistant had already confirmed that torch.cuda.can_device_access_peer(i, j) returned True for all GPU pairs. This function queries the NVIDIA driver's P2P capability — it checks whether the GPU hardware supports direct peer-to-peer access over the PCIe fabric. The driver said yes, P2P is available. But the question was whether the IOMMU was silently corrupting or blocking those transfers.

The assistant's reasoning was: if NCCL is hanging during distributed initialization, and the hang happens after all sockets are established, then the problem is likely in the first actual data exchange. NCCL uses P2P DMA for fast GPU-to-GPU communication during init_process_group and all_reduce operations. If P2P transfers produce corrupted data, NCCL would either hang (waiting for data that never arrives correctly) or crash.

The Diagnostic Method

The test the assistant wrote is elegant in its simplicity. It creates a random tensor on one GPU, copies it to another GPU via .to(), synchronizes, copies both back to CPU, and compares them with torch.allclose. The .to() call on CUDA tensors triggers the P2P DMA path — it's not a host-mediated copy through system memory. This is the same path NCCL uses.

The test covers all 12 directed pairs among 4 GPUs. The result is unambiguous: every single transfer produces corrupted data. Not a single GPU-to-GPU copy succeeds. The corruption is total, not intermittent.

This is a critical diagnostic finding because it explains the NCCL hang. NCCL's init_process_group typically performs a barrier or all-reduce operation to synchronize ranks. If every P2P transfer corrupts data, the all-reduce will produce garbage results, the barrier may never complete because ranks see inconsistent data, and the entire initialization deadlocks.

Assumptions Made

Several assumptions are visible in this message and its surrounding context:

Assumption 1: The driver's P2P capability report is trustworthy but incomplete. The assistant correctly assumed that can_device_access_peer returning True only means the GPU hardware supports P2P, not that the IOMMU will allow it. This is a subtle but important distinction — the NVIDIA driver reports what the hardware can do, not what the platform's DMA translation layer will permit.

Assumption 2: The .to() call uses P2P DMA. This is a correct assumption for CUDA-aware PyTorch. When both source and destination are CUDA devices with P2P access enabled, PyTorch's .to() operation uses cudaMemcpyPeer or similar P2P DMA paths rather than staging through host memory.

Assumption 3: The corruption is consistent and reproducible. The test runs each pair once and gets MISMATCH every time. The assistant implicitly assumes this is a deterministic failure, not a transient error. This is reasonable given the IO_PAGE_FAULT evidence — IOMMU page faults are deterministic when the translation tables are misconfigured.

Assumption 4: The IOMMU is the cause. The assistant connects the IO_PAGE_FAULTs from dmesg with the P2P corruption. The addresses in the faults (0x24000000000, 0x34000001000, etc.) are consistent with GPU BAR mappings — these are physical addresses in the PCIe address space that map to other GPUs' framebuffers. The IOMMU is intercepting these DMA transactions and failing them because it doesn't have valid page table entries for those addresses.

Mistakes and Incorrect Assumptions

The assistant initially assumed the hang was a software issue. Before this message, the assistant spent considerable effort investigating SGLang code paths, checking socket connectivity, verifying TCPStore operation, and even pulling the latest SGLang commits. These were reasonable steps, but they were based on the implicit assumption that the hardware was functioning correctly. The IO_PAGE_FAULT clue from the user was essential to redirect the investigation.

The assistant assumed NCCL was using P2P DMA. This turned out to be correct, but it's worth noting that NCCL has multiple transport modes: P2P (NVLink/NVSwitch), SHM (shared memory within a node), and IB (InfiniBand). On a multi-GPU node without NVLink (these are PCIe-connected Blackwell GPUs), NCCL would prefer P2P DMA over PCIe for GPU-to-GPU transfers. The hang confirmed this assumption — NCCL was indeed trying to use P2P.

The test assumes all GPUs are on the same PCIe root complex. On a dual-socket AMD EPYC system, GPUs may be split across two PCIe root complexes (one per socket). P2P DMA across root complexes requires IOMMU involvement on AMD systems. The IO_PAGE_FAULT addresses span multiple PCIe domains (0x002c, 0x003f, 0x000c), confirming that GPUs on different root complexes are trying to communicate.

Input Knowledge Required

To fully understand this message, one needs:

  1. Understanding of GPU P2P DMA: Knowledge that GPUs can directly access each other's memory over PCIe without host CPU involvement, and that this is critical for distributed training performance.
  2. IOMMU basics: Understanding that an IOMMU (I/O Memory Management Unit) translates DMA addresses from devices, similar to how a CPU's MMU translates virtual addresses. In passthrough mode (iommu=pt), translation is minimal. In full mode, every DMA goes through page tables.
  3. SEV-SNP requirements: AMD's Secure Encrypted Virtualization with Secure Nested Paging requires full IOMMU translation (amd_iommu=on) and memory encryption (mem_encrypt=on). This conflicts with GPU P2P DMA, which typically requires IOMMU passthrough or special ACS (Access Control Services) overrides.
  4. NCCL initialization flow: Understanding that NCCL's init_process_group performs a barrier/all-reduce as part of rank discovery, and that this requires working P2P DMA on multi-GPU nodes.
  5. PyTorch CUDA semantics: Knowledge that .to() between CUDA devices triggers P2P DMA, and that torch.allclose compares tensors element-wise.
  6. PCIe topology on AMD EPYC: Understanding that dual-socket AMD systems have two PCIe root complexes, and P2P DMA across them requires platform support that may be broken under IOMMU translation.

Output Knowledge Created

This message created several critical pieces of knowledge:

1. Definitive proof of P2P DMA corruption: The test conclusively demonstrates that every GPU-to-GPU transfer produces corrupted data. This is not a transient or intermittent issue — it's a systematic failure affecting all 12 GPU pairs.

2. Root cause confirmation: Combined with the IO_PAGE_FAULT evidence, this confirms that the IOMMU is the root cause. The IOMMU is intercepting P2P DMA transactions and either blocking them or corrupting the data because the translation tables don't have valid mappings for GPU BAR addresses.

3. NCCL hang explanation: The P2P corruption explains why NCCL hangs during init_torch_distributed. NCCL's initialization involves data exchange between ranks. If every exchange produces corrupted data, the barrier/all-reduce never completes, and the process hangs indefinitely.

4. Diagnostic methodology: The test itself is a reusable diagnostic tool. A simple 10-line Python script that tests all GPU pairs can quickly determine whether P2P DMA is functional. This is valuable for any multi-GPU deployment, especially on systems with IOMMU enabled.

5. Platform limitation documentation: This finding documents a specific limitation of AMD EPYC platforms with SEV-SNP: GPU P2P DMA is broken under full IOMMU translation. This has implications for anyone trying to run multi-GPU workloads in SEV-SNP confidential VMs.

The Thinking Process

The assistant's reasoning chain, visible across the messages leading to this one, reveals a methodical diagnostic approach:

  1. Observe the symptom: SGLang hangs at init_torch_distributed. All four TP workers are alive but blocked.
  2. Eliminate network issues: Check sockets, TCPStore, NCCL connections. All are established. The hang is not a connectivity problem.
  3. Examine the code: Look at SGLang's init_torch_distributed to understand what happens after socket setup. The code calls NCCL's init_process_group, which performs data exchange.
  4. Receive the clue: The user provides IO_PAGE_FAULT logs from the host. This redirects the investigation from software to hardware.
  5. Understand the configuration: Read the UEFI settings and SEV-SNP documentation. Discover that amd_iommu=on is set (required for SEV-SNP) and iommu=pt is absent.
  6. Form the hypothesis: IOMMU is blocking P2P DMA. The driver reports P2P as available, but the IOMMU is silently corrupting or failing the transfers.
  7. Test the hypothesis: Write a simple CUDA P2P test. Run it on all GPU pairs. Get MISMATCH on every pair.
  8. Confirm the diagnosis: The hypothesis is confirmed. Every P2P transfer is corrupted. The key insight in this message is the decision to test actual data transfer rather than relying on the driver's capability report. The assistant recognized that can_device_access_peer only checks hardware capability, not platform-level DMA translation. This distinction between "can the hardware do it" and "will the platform allow it" is crucial for debugging IOMMU-related issues.

Implications and Aftermath

This finding had immediate practical implications. The NCCL hang was not a software bug in SGLang or PyTorch — it was a fundamental platform limitation. The solution was to disable NCCL's P2P transport and force it to use SHM (shared memory) instead. This was accomplished by setting NCCL_P2P_DISABLE=1 in the environment, which forces NCCL to copy data through host memory rather than using direct GPU-to-GPU DMA.

The SHM path is slower than P2P DMA — it requires two copies (GPU→host, host→GPU) instead of one direct transfer — but it works correctly under IOMMU translation. The assistant later confirmed that with NCCL_P2P_DISABLE=1, the SGLang server loaded successfully and achieved 108 tok/s single-stream and up to 2,800 tok/s at high concurrency.

The article also prompted investigation into BIOS-level options to re-enable P2P DMA under IOMMU, exploring PCIe ACS (Access Control Services), ARI (Alternative Routing-ID Interpretation), and IOMMU coverage settings. These are complex platform-level configuration options that could potentially restore P2P functionality without compromising SEV-SNP security guarantees.

Conclusion

Message 6216 is a masterclass in diagnostic debugging. Faced with a cryptic hang, the assistant systematically eliminated software causes, followed the user's clue about IO_PAGE_FAULTs, understood the IOMMU configuration, and wrote a simple but definitive test that exposed the root cause. The finding — that P2P DMA is silently corrupting data under full IOMMU translation — explained the NCCL hang and pointed to a clear fix (disabling NCCL P2P) while also documenting an important platform limitation for AMD EPYC systems running SEV-SNP.

The message demonstrates a critical principle of systems debugging: when the driver says something should work, but the system doesn't behave accordingly, test the actual data path. The gap between "can access peer" and "data arrives correctly" is where platform-level translation layers like IOMMU can silently break things. In this case, that gap was the entire root cause.