The Cleanup That Tells a Story: Removing the Artifacts of a Failed P2P DMA Journey

The Message

The subject message, [msg 6337], is deceptively simple:

Good. Now let me also clean up the udev rule we created earlier (it's not needed with the systemd approach):

>

[bash] ssh root@10.1.2.6 'rm -f /etc/udev/rules.d/99-gpu-iommu-identity.rules; rm -f /etc/modprobe.d/temp-nvidia-blacklist.conf; echo "Cleaned up"'

>

Cleaned up

Two files deleted. A single "Cleaned up" echoed back. On its surface, this looks like a trivial housekeeping step — the kind of minor tidying that any engineer does after finishing a task. But in the context of the broader conversation, this message represents a quiet surrender. It marks the moment when a long, creative, and ultimately unsuccessful campaign to restore GPU peer-to-peer (P2P) DMA on a Blackwell-based system was formally abandoned, and the artifacts of that campaign were swept away.

The Long Road to This Cleanup

To understand why this message matters, we must trace the journey that led to it. The system in question is a Proxmox host with 8 NVIDIA RTX PRO 6000 Blackwell GPUs, split between an LXC container (4 GPUs on NUMA0) and a SEV-SNP VM (4 GPUs on NUMA1). The fundamental problem is that GPU P2P DMA — the ability for GPUs to directly read and write each other's memory across the PCIe bus without host CPU involvement — is broken under the AMD IOMMU's DMA translation mode. This is a critical performance issue for multi-GPU machine learning workloads, as P2P DMA is the backbone of NCCL (NVIDIA Collective Communications Library) all-reduce operations.

The assistant had already discovered that NCCL_P2P_DISABLE=1 (falling back to shared memory transport) was the pragmatic workaround, but the desire to restore true P2P DMA drove an extensive investigation spanning dozens of messages. The IOMMU identity domain approach — setting specific IOMMU groups to "identity" mode (which bypasses DMA translation for those devices) — was identified as the most promising solution. It would allow the NUMA0 GPUs to communicate directly while keeping the IOMMU in full translation mode for the SEV-SNP VM's VFIO devices.

The Failed Attempts

The assistant explored multiple strategies to set IOMMU identity domains for the NUMA0 GPUs. In [msg 6323], the assistant grappled with a chicken-and-egg problem: the nvidia driver needs PCI devices present to load, but the IOMMU group type cannot be changed while the nvidia driver is bound to those devices. Ideas included removing one GPU at a time, using driver_override to temporarily prevent nvidia from probing, and setting the kernel's default IOMMU domain type via boot parameters. Each idea was examined and discarded as impractical or blocked.

In [msg 6325], the assistant considered kernel boot parameters like iommu.passthrough=1 but was constrained by the instruction not to reboot or change kernel command-line parameters without coordination. In [msg 6326], the assistant pivoted to examining nvidia driver parameters, discovering that DmaRemapPeerMmio=1 was already enabled. This parameter should theoretically allow the nvidia driver to create IOMMU mappings for peer GPU BARs, making P2P work even under full DMA translation.

A detailed P2P test in [msg 6332] and [msg 6333] revealed a frustrating partial result: DmaRemapPeerMmio=1 worked for some GPU pairs but not others, producing a directional pattern of IO_PAGE_FAULTs. The nvidia driver was creating incomplete IOMMU mappings, with some peer BAR ranges mapped and others missing. This was a dead end — there was no way to fix the nvidia driver's internal mapping logic from outside.

The Boot-Time Gambit

In [msg 6334], the assistant made a strategic decision: since runtime manipulation of IOMMU groups was blocked by the Blackwell GSP/FSP's inability to survive driver rebinding, the only viable path was a boot-time solution. The assistant created a script (/usr/local/bin/gpu-iommu-identity.sh) in [msg 6335] that would set IOMMU identity domains for the NUMA0 GPUs at boot, before the nvidia driver loaded. A systemd service (gpu-iommu-identity.service) was created and enabled in [msg 6336], with careful ordering constraints: it must run after PCI enumeration but before nvidia-persistenced.service and the VFIO split service.

This systemd approach replaced an earlier attempt using a udev rule (99-gpu-iommu-identity.rules). The udev approach was problematic because udev events fire asynchronously and might not execute before the nvidia driver probes a newly discovered device. The systemd service, with its explicit ordering dependencies, offered more reliable timing. The temp-nvidia-blacklist.conf modprobe file was a temporary measure used during experimentation to prevent nvidia from auto-loading while the assistant manipulated IOMMU groups.

The Cleanup as a Decision Point

This brings us to [msg 6337], the subject message. The assistant removes both the udev rule and the modprobe blacklist because they are no longer needed — the systemd service is the chosen approach. But this cleanup also signals something deeper: the assistant is committing to the boot-time strategy and clearing away the debris of earlier, more experimental approaches.

What makes this message particularly interesting is what happens after it. As the chunk summary for segment 41 reveals, the boot-time approach itself ultimately failed. After the next reboot, the Blackwell FSP (Firmware Security Processor) boot sequence failed with error code 0x177 when IOMMU was in identity mode. The FSP apparently requires specific DMA mappings set up by the kernel's DMA API in translation mode, and identity mode breaks this initialization. The systemd service had to be removed, and the system reverted to the working DMA-FQ configuration.

So the cleanup in [msg 6337] was not just tidying up — it was the moment when the assistant committed to a path that would itself prove to be a dead end. The udev rule and modprobe blacklist were artifacts of approaches that were abandoned for good reasons (timing unreliability), but the replacement (systemd service) would also fail, for reasons that could not have been anticipated at this point.

Assumptions and Knowledge Required

To understand this message, the reader needs substantial background knowledge. First, one must understand the IOMMU architecture on AMD systems — the concept of DMA translation domains, the distinction between "identity" and "DMA-FQ" (DMA with Full Queuing) modes, and the sysfs interface for manipulating IOMMU group types at /sys/kernel/iommu_groups/*/type. Second, one needs familiarity with the nvidia driver's module parameters, particularly DmaRemapPeerMmio and its role in peer GPU BAR mapping. Third, knowledge of Linux boot sequencing — the order in which PCI enumeration, module loading, udev events, and systemd services execute — is essential to understand why the systemd approach was chosen over udev. Finally, understanding the Blackwell GPU architecture and its FSP/GSP (Firmware Security Processor / GPU System Processor) boot requirements is necessary to appreciate why the identity domain approach ultimately failed.

The key assumption made in this message is that the systemd service approach is correct and sufficient. The assistant assumes that setting IOMMU identity domains before the nvidia driver loads will work, because the Blackwell FSP will initialize correctly under identity mode. This assumption turned out to be wrong — the FSP requires DMA translation mode during its initialization sequence, and no amount of timing manipulation can change that fundamental hardware requirement.

Output Knowledge Created

This message creates knowledge about the state of the system's configuration. It documents that:

  1. The udev rule approach has been abandoned in favor of the systemd service
  2. The temporary modprobe blacklist is no longer needed
  3. The system is now configured with only the systemd service for IOMMU identity domain management
  4. The assistant has committed to the boot-time strategy More subtly, the message also documents the assistant's reasoning process. By explicitly stating "it's not needed with the systemd approach," the assistant reveals its decision-making framework: the udev rule was a precursor to the systemd service, and the systemd service supersedes it. This is a record of the evolution of the solution.

The Thinking Process

The thinking visible in this message is minimal on the surface — it's a straightforward cleanup operation. But the thinking that led to this message is extensive and visible in the preceding messages. The assistant worked through a systematic exploration of approaches:

  1. Runtime IOMMU manipulation ([msg 6323]): Identified the chicken-and-egg problem with nvidia driver binding
  2. Boot parameter approach ([msg 6325]): Considered but blocked by coordination constraints
  3. Nvidia driver parameter investigation ([msg 6326]-[msg 6327]): Discovered DmaRemapPeerMmio=1 and tested it
  4. P2P testing ([msg 6332]-[msg 6333]): Confirmed partial functionality of DmaRemapPeerMmio
  5. Boot-time service creation ([msg 6334]-[msg 6336]): Designed and deployed the systemd approach
  6. Cleanup ([msg 6337]): Removed artifacts of earlier approaches This progression shows a methodical engineering mindset: explore all runtime options first, then pivot to boot-time configuration, and clean up as you go.

Conclusion

The deletion of two configuration files in [msg 6337] is a small act with outsized significance. It marks the transition from experimentation to commitment, from exploring multiple parallel approaches to betting on a single strategy. The udev rule and modprobe blacklist were the debris of earlier, more tentative attempts — approaches that were tried, found wanting, and now discarded. The systemd service was the chosen path forward, the result of careful reasoning about boot sequencing and driver initialization.

That this path also ultimately failed — that the Blackwell FSP's requirement for DMA translation mode was a hard constraint that no amount of boot-time manipulation could overcome — does not diminish the value of the work documented in this message. Engineering is often about discovering what does not work, and the cleanup in [msg 6337] is a testament to a rigorous, well-documented exploration of a complex systems integration problem. The artifacts were removed, but the knowledge gained — about Blackwell FSP boot requirements, about IOMMU group manipulation, about the limits of nvidia's DmaRemapPeerMmio parameter — remains as a permanent contribution to the understanding of this system.