The Critical Pivot: Investigating Per-Group IOMMU Identity Domains for Blackwell GPU P2P DMA

In the complex dance of deploying high-performance LLM inference on NVIDIA Blackwell GPUs under a SEV-SNP virtualized environment, message 6251 represents a quiet but pivotal moment. It is a message of reconnaissance and planning—the assistant reads the existing GPU split infrastructure to understand how to integrate a novel solution to a fundamental hardware conflict. The message itself is brief, containing only two bash commands and a short commentary, but it encapsulates the culmination of extensive research and sets the stage for a critical experiment that would ultimately reveal a hard limitation of Blackwell GPUs.

The Broader Conflict: IOMMU Translation vs. GPU P2P DMA

To understand why this message matters, one must first understand the conflict at the heart of this deployment. The Proxmox host (kpro6) runs with amd_iommu=on—full IOMMU translation mode—because SEV-SNP (Secure Encrypted Virtualization with Secure Nested Paging) requires it. SEV-SNP is an AMD technology that provides confidential computing protections for virtual machines, and it mandates that the IOMMU remain in full translation mode rather than passthrough (iommu=pt) mode.

However, this creates a severe problem for GPU-to-GPU direct memory access (P2P DMA). NVIDIA GPUs, particularly in multi-GPU configurations, rely on P2P DMA to transfer data directly between GPU memory across the PCIe bus without involving host memory. This is critical for tensor parallelism (TP) in large model inference, where model shards distributed across GPUs must communicate during every forward pass. When the IOMMU is in full translation mode, GPU P2P DMA operations produce IO_PAGE_FAULTs because the IOMMU lacks proper mappings for the GPU BAR (Base Address Register) regions that peer GPUs attempt to access. The transfers complete but with corrupted data, rendering P2P unusable.

The current workaround is NCCL_P2P_DISABLE=1, which forces NCCL to use shared memory (SHM) transport instead of P2P DMA. This works correctly but adds overhead, particularly for all-reduce operations that are the backbone of tensor-parallel inference. The user's explicit question—"Can P2P be restored with IOMMU still in full translation mode?"—drove the investigation that leads to this message.

The Message: Reconnaissance Before Action

Message 6251 reads in its entirety:

Each IOMMU group contains exactly one GPU — no audio devices or other PCI functions. This is ideal. Now let me also check the existing gpu-vfio-split.sh script to understand how we'd integrate the identity domain change:

The assistant then executes two commands: reading the GPU split script and its systemd service file. The results show the complete contents of both files.

On the surface, this is simple: the assistant checks existing infrastructure. But the reasoning embedded in this action reveals a sophisticated understanding of the problem and the solution being contemplated.

The Concept: Per-Group IOMMU Identity Domains

The "identity domain change" referenced in the message is the key insight from prior research (message 6247's task result). Since Linux kernel v5.11, the sysfs interface at /sys/kernel/iommu_groups/<grp_id>/type allows per-group switching between DMA (translation) and identity (passthrough) domain types. This means individual IOMMU groups—each containing a single PCI device—can be placed in passthrough mode while others remain in translation mode.

The plan is elegant: set the four nvidia GPU groups (42, 61, 28, 10) to identity mode, allowing P2P DMA between them to bypass IOMMU translation, while leaving the four vfio-pci GPU groups (101, 117, 90, 72) in DMA-FQ mode to maintain SEV-SNP protection for the confidential VM. This would theoretically give the best of both worlds: high-performance GPU P2P DMA for inference and hardware-enforced isolation for the VM.

The assistant's verification that "each IOMMU group contains exactly one GPU — no audio devices or other PCI functions" is crucial. Many GPUs expose additional PCI functions (audio devices, USB controllers, etc.) that share the same IOMMU group. If those were present, setting the group to identity mode might affect non-GPU functionality. The clean one-device-per-group configuration, a result of ACS (Access Control Services) being enabled in the BIOS, makes the approach straightforward.

Reading the Existing Infrastructure

The assistant reads gpu-vfio-split.sh and its systemd service to understand the boot-time integration point. The script handles binding NUMA 1 GPUs to vfio-pci for VM passthrough. The service runs Before=nvidia-persistenced.service, meaning it executes before the NVIDIA driver initializes.

This timing is critical. The identity domain change must occur before the NVIDIA driver loads and initializes the GPU, because the driver establishes DMA mappings during initialization. If the IOMMU group is in translation mode when the driver loads, the driver will create DMA mappings through the IOMMU, and switching to identity mode afterward might not take full effect or could cause inconsistencies. The existing service infrastructure provides a natural integration point: the assistant could either extend gpu-vfio-split.sh or create a new service that runs at the same point in the boot sequence.

Assumptions Embedded in the Approach

The assistant makes several assumptions, all reasonable given the available information:

First, that writing identity to the IOMMU group type file will actually enable P2P DMA between the GPUs. The kernel documentation and SUSE research (from the task result) suggest this should work, but the interaction with the NVIDIA driver's own P2P infrastructure is untested on this specific hardware.

Second, that the timing constraint is real and critical—that the identity domain must be set before the NVIDIA driver loads. This is a reasonable inference from how IOMMU groups and device drivers interact, but the actual behavior depends on the kernel's IOMMU implementation and the NVIDIA driver's initialization sequence.

Third, that the existing gpu-vfio-split.service is the right integration point. The assistant implicitly assumes that adding the identity domain change to this service (or creating a parallel service with the same timing) will work correctly.

Fourth, that the identity domain change is safe for the vfio-pci GPUs. The VM GPUs must remain in translation mode for SEV-SNP to function, and the assistant's plan respects this boundary.

The Knowledge Flow: Input and Output

The input knowledge required to understand this message includes: the IOMMU architecture on AMD systems, the sysfs interface for per-group domain types (a relatively obscure kernel feature), the NVIDIA driver initialization sequence and its relationship to IOMMU groups, the SEV-SNP requirements for IOMMU translation mode, the PCI topology of the specific machine (which GPUs are on which NUMA nodes and in which IOMMU groups), and the existing GPU split infrastructure.

The output knowledge created by this message is the detailed understanding of the existing boot-time GPU split mechanism. The assistant now knows the exact script contents, the service unit configuration, the timing relative to nvidia-persistenced, and the structure of the device unbinding/binding logic. This knowledge directly enables the implementation of the identity domain change.

The Thinking Process Visible in the Message

The assistant's reasoning is visible in the structure of the investigation. The sequence of prior messages shows a methodical approach: first, comprehensive research on per-group identity domains (message 6247's task); second, verification of the current IOMMU group state and kernel version (message 6248); third, confirmation that the type file is writable and checking running VMs (message 6249); fourth, verification of clean one-device-per-group configuration (message 6250); and finally, reading the existing infrastructure to plan integration (this message).

The comment "This is ideal" reveals the assistant's assessment: the clean group configuration eliminates potential complications. The phrase "understand how we'd integrate the identity domain change" shows forward thinking—the assistant is not just gathering information but actively planning the implementation.

The Significance: A Fork in the Road

This message represents a decision point. The assistant could have pursued other approaches to restoring P2P DMA—BIOS settings like "DMA Protection" or "RMP Coverage for 64Bit MMIO Ranges," kernel parameters like iommu.passthrough, or NVIDIA driver registry settings. Instead, it chose the per-group identity domain approach as the most promising path.

The significance of this choice becomes clear in the next segment (segment 41), where the approach is tested and fails catastrophically: the Blackwell GPU's Firmware Security Processor (FSP) fails to boot with error code 0x177 when IOMMU is in identity mode. The FSP apparently requires specific DMA mappings set up by the kernel's DMA API in translation mode, and identity mode breaks this initialization. This means per-group IOMMU identity domains are fundamentally incompatible with Blackwell GPUs—a hard limitation that no software-level reset (FLR, SBR, CXL bus reset) can clear.

But that failure is not yet known in message 6251. At this moment, the assistant is optimistic, methodically preparing to implement what seems like the ideal solution. The message captures this moment of anticipation before the discovery of a hard hardware limitation—a reminder that even the most elegant software solutions can be defeated by firmware requirements invisible to the application layer.

Conclusion

Message 6251 is a brief but crucial reconnaissance step in a complex optimization journey. It demonstrates the methodical approach required when deploying cutting-edge hardware in constrained virtualization environments: research the solution space, verify the prerequisites, understand the existing infrastructure, and plan the integration. The assistant's reading of the GPU split script and service file provides the knowledge needed to implement the per-group IOMMU identity domain change—a change that, while ultimately unsuccessful due to Blackwell's FSP requirements, was the correct next step based on all available information. The message stands as a testament to the importance of understanding boot-time infrastructure when making low-level hardware configuration changes, and as a snapshot of optimism before encountering a hard hardware limitation.