Diagnosing P2P DMA Corruption Under IOMMU: A Pivotal Diagnostic Message

In the course of deploying large language models across a multi-GPU server, few failures are as insidious as silent data corruption. When a distributed tensor parallelism (TP) job hangs during initialization, the symptoms are clear—processes blocked on futex and socket reads, no progress, no error message—but the root cause can hide deep in the hardware abstraction layers. Message [msg 6211] captures the exact moment when an AI assistant, after hours of fruitless debugging, finally identifies the true culprit behind a stubborn NCCL hang: IOMMU-enforced full translation corrupting GPU-to-GPU peer-to-peer DMA transfers.

The Context: A Multi-GPU Deployment Gone Wrong

The session leading up to this message was a complex infrastructure operation. The assistant had been tasked with deploying the Qwen3.5-122B-A10B model (a 234 GB BF16 model) across 4 NVIDIA RTX PRO 6000 Blackwell GPUs on a Proxmox-hosted LXC container. The server had recently been reconfigured: the original 8-GPU topology was split so that 4 GPUs remained bound to the NVIDIA driver for the container, while the other 4 were moved to vfio-pci passthrough for an SEV-SNP confidential VM belonging to another tenant.

This reconfiguration had introduced a critical change to the host kernel boot parameters. The Proxmox host was now booting with amd_iommu=on (full IOMMU translation mode) rather than iommu=pt (passthrough mode), because SEV-SNP confidential computing requires the IOMMU to actively translate and validate all DMA transactions. The assistant did not yet know this when it first attempted to load the model.

The failure was consistent and perplexing. Every attempt to start SGLang with --tp 4 resulted in the same behavior: all four TP processes would print "Init torch distributed begin" and then hang indefinitely. The processes were alive, their NCCL sockets were established, but no progress was made. The assistant spent messages [msg 6165] through [msg 6188] chasing false leads—checking NCCL environment variables, trying different NCCL protocols (Simple, Ring), updating SGLang to the latest commit, reinstalling from source. Nothing worked.

The Breakthrough: IO_PAGE_FAULTs in dmesg

The turning point came when the user provided a critical piece of evidence: IO_PAGE_FAULT logs from the host's kernel ring buffer ([msg 6191]). These showed AMD-Vi (the AMD IOMMU implementation) logging page faults for the NVIDIA GPUs, with addresses like 0x24000000000 and 0x34000010000. The user then directed the assistant to find setup documentation at ~/kpro6-uefi/ ([msg 6196], [msg 6206]), which contained detailed notes about the SEV-SNP configuration and BIOS settings.

Reading those documents ([msg 6208]) and verifying the kernel command line ([msg 6209]) gave the assistant the full picture:

initrd=\EFI\proxmox\6.14.11-5-bpo12-pve\initrd.img-6.14.11-5-bpo12-pve root=ZFS=rpool/ROOT/pve-1 
amd_iommu=on amd_pstate=active processor.max_cstate=0 nmi_watchdog=0 mem_encrypt=on kvm_amd.sev=1 
pcie_aspm=off pci=realloc=off numa_balancing=disable boot=zfs

The critical flag was amd_iommu=on. Combined with mem_encrypt=on (required for SEV-SNP), the IOMMU was now in full translation mode, meaning every DMA operation—including GPU-to-GPU P2P transfers—had to go through the IOMMU page tables. The NVIDIA driver, however, was attempting direct P2P DMA using GPU BAR addresses, which the IOMMU could not resolve, triggering IO_PAGE_FAULTs and silently corrupting data.

Message [msg 6211]: The Diagnostic Pivot

Message [msg 6211] is the assistant's synthesis of this discovery and its plan to confirm the hypothesis. The message reads:

The IO_PAGE_FAULTs confirm it — the GPUs are trying to DMA to addresses like 0x24000000000 and 0x34000010000 (these are GPU BAR addresses of other GPUs). This is P2P GPU-to-GPU DMA that IOMMU is blocking because it's in full translation mode.

Let me write a quick CUDA test to confirm basic single-GPU operations work, and test P2P explicitly: [write] /home/theuser/glm-kimi-sm120-rtx6000bw/gpu_diag.py Wrote file successfully.

LSP errors detected in this file, please fix: ERROR [3:8] Import "torch" could not be resolved ERROR [67:16] Import "torch.distributed" could not be resolved ERROR [68:14] Import "torch.multiprocessing" could not be resolved

This is a short message, but it packs a tremendous amount of reasoning and decision-making into a few lines. Let us unpack what is happening here.

The Reasoning: Connecting the Dots

The assistant's first sentence demonstrates a critical inference. The IO_PAGE_FAULT addresses—0x24000000000, 0x34000001000, 0x34000010000—are not random memory locations. They are GPU BAR (Base Address Register) addresses, which are the physical memory windows that the PCIe bus uses to map GPU memory into the system address space. When one GPU performs a P2P write to another GPU, the NVIDIA driver programs the PCIe transaction to target the peer GPU's BAR address directly. Under IOMMU passthrough mode (iommu=pt), these BAR addresses pass through the IOMMU unmodified, and the PCIe fabric routes them correctly. Under full translation mode (amd_iommu=on), the IOMMU intercepts every DMA transaction and attempts to translate the address through its page tables. Since the GPU BAR addresses are not mapped in the IOMMU page tables, the translation fails, producing an IO_PAGE_FAULT.

The critical insight here is that the IO_PAGE_FAULT does not necessarily kill the process or produce an error code that CUDA can report. Depending on the IOMMU configuration and the NVIDIA driver's error handling, the fault may be silently suppressed, with the DMA write landing in a harmless location or being dropped entirely. The data simply vanishes or gets corrupted. This is why NCCL hangs: it performs a collective operation (like allreduce) that requires all ranks to produce correct results. When one rank's P2P write silently fails, the receiving rank gets garbage data, the computation produces incorrect results, and subsequent synchronization barriers never complete because the expected data never arrives.

The Decision: Write a Targeted Diagnostic

The assistant's decision to write a custom diagnostic script (gpu_diag.py) rather than using existing tools like bandwidthTest or simpleP2P from CUDA samples is telling. The assistant needs to test two things simultaneously:

Single-GPU sanity: Confirm that basic CUDA operations (memory allocation, computation) work on each GPU individually. This rules out driver issues, GPU hardware faults, or memory corruption unrelated to P2P.
P2P data integrity: Explicitly transfer data between every pair of GPUs and verify that the received data matches the sent data. This is the critical test—torch.cuda.can_device_access_peer() might return True (as it did in subsequent testing at [msg 6215]), but the actual data transfer could silently produce garbage. The script is written to the project directory /home/theuser/glm-kimi-sm120-rtx6000bw/gpu_diag.py, indicating that the assistant is treating this as a permanent diagnostic tool for this hardware configuration, not a throwaway test.

Assumptions Made

The assistant makes several assumptions in this message:

The IO_PAGE_FAULT addresses are GPU BAR addresses: This is an educated inference based on the address ranges (0x24000000000, 0x34000010000). GPU BARs on modern NVIDIA GPUs with large BAR1 (up to 64 GB or more) are typically mapped in the 64-bit address space above 4 GB. The assistant assumes these are peer GPU memory windows, not some other DMA target. This assumption is correct, as confirmed by subsequent testing.
P2P DMA is the only thing broken: The assistant assumes that single-GPU operations will work fine. This is a reasonable assumption because single-GPU operations use the GPU's own local memory and do not require P2P transfers. The IO_PAGE_FAULTs only occur when a GPU initiates a DMA write to another GPU's BAR address.
The LSP errors are false positives: The diagnostic tool reports unresolved imports for torch, torch.distributed, and torch.multiprocessing. The assistant correctly recognizes these as local development environment issues (the LSP server does not have access to the remote container's Python environment) and ignores them. The script will run correctly on the target machine.

Input Knowledge Required

To fully understand this message, the reader needs:

Understanding of IOMMU and DMA: Knowledge that the IOMMU (I/O Memory Management Unit) translates DMA addresses from device addresses to system physical addresses, and that full translation mode applies this to all devices while passthrough mode bypasses it for trusted devices.
GPU P2P DMA mechanics: Understanding that NVIDIA GPUs can directly access each other's memory via PCIe BAR addressing, bypassing system RAM. This is critical for NCCL's allreduce and other collective operations in tensor parallelism.
SEV-SNP requirements: AMD's Secure Encrypted Virtualization with Secure Nested Paging requires the IOMMU to be in full translation mode to enforce memory encryption and access control for confidential VMs. This is why amd_iommu=on was set.
NCCL's behavior on data corruption: NCCL does not have built-in data integrity checks for P2P transfers. If a P2P write silently corrupts data, NCCL will compute incorrect results and hang on subsequent synchronization barriers because the corrupted data leads to deadlock in the collective algorithm.
The history of the session: The previous failed attempts to start SGLang with TP=4 ([msg 6165]–[msg 6188]), the IO_PAGE_FAULT logs from the user ([msg 6191]), and the discovery of the SEV-SNP configuration ([msg 6208]–[msg 6209]).

Output Knowledge Created

This message creates several pieces of knowledge:

A confirmed diagnosis: The NCCL hang is caused by IOMMU-blocked P2P DMA, not by NCCL configuration, PyTorch version mismatches, or SGLang bugs. This redirects the debugging effort from software configuration to hardware/IOMMU configuration.
A diagnostic script: gpu_diag.py will provide definitive proof by testing P2P data integrity across all GPU pairs. The results (seen in subsequent messages [msg 6215]–[msg 6216]) show that can_device_access_peer returns True for all pairs, but actual data transfers produce MISMATCH for every pair—confirming the silent corruption hypothesis.
A constraint on solutions: Since the IOMMU configuration cannot be changed (the user explicitly warned against rebooting or changing BIOS settings in [msg 6196] because another tenant is active), the solution must work within the broken P2P regime. This leads directly to the fix implemented in subsequent messages: NCCL_P2P_DISABLE=1, which forces NCCL to use shared memory (SHM) transport instead of P2P DMA.
Documentation of the failure mode: The message implicitly documents that Blackwell GPUs on AMD platforms with SEV-SNP and full IOMMU translation cannot use P2P DMA. This is valuable knowledge for anyone deploying similar configurations.

The Thinking Process: A Window into Diagnostic Reasoning

The assistant's thinking, visible across the conversation leading to this message, follows a classic diagnostic pattern:

Observe the symptom: NCCL hangs during init_torch_distributed with TP=4.
Eliminate easy causes: Try different NCCL protocols, update SGLang, reinstall.
Gather more data: Check strace output (all processes blocked on futex), check socket connections (all established), check log output (stuck after "Init torch distributed begin").
Receive external clue: User provides IO_PAGE_FAULT logs from the host.
Research the context: Find and read the SEV-SNP setup documentation.
Formulate hypothesis: IOMMU full translation is blocking P2P DMA.
Design confirmatory test: Write a script that tests P2P data integrity explicitly.
Execute and confirm: The test proves every P2P transfer is corrupted. This is a textbook example of systematic debugging. The assistant did not jump to the IOMMU hypothesis immediately—it spent considerable effort ruling out software configuration issues first. Only when the user provided the IO_PAGE_FAULT logs did the assistant have the missing piece needed to connect the hang to the hardware configuration change.

Conclusion

Message [msg 6211] is a pivotal moment in a complex infrastructure debugging session. It represents the transition from confusion to understanding—from chasing software configuration ghosts to identifying a fundamental hardware abstraction conflict. The assistant correctly interprets kernel IO_PAGE_FAULT events as evidence of P2P DMA corruption under full IOMMU translation, designs a targeted diagnostic to confirm the hypothesis, and sets the stage for the eventual fix (NCCL_P2P_DISABLE=1). The message demonstrates the importance of understanding the full hardware stack—from GPU BAR addresses to IOMMU page tables to NCCL transport selection—when deploying distributed ML inference at scale.