The Moment of Reversion: When IOMMU Identity Domains Slip Away on Blackwell GPUs
In the course of a marathon debugging session spanning dozens of messages, one brief message stands as a quiet watershed — a moment of apparent success that reveals the fundamental obstacle ahead. Message [msg 6278] is deceptively short, barely a paragraph of text followed by a single bash command and its output. Yet within its few lines lies the crystallization of a critical realization: that restoring GPU peer-to-peer DMA via IOMMU identity domains on NVIDIA Blackwell hardware faces a timing problem that no amount of clever scripting can easily solve.
The Message
Here is the message in full:
[assistant] GPUs are back! All 4 GPUs successfully initialized after the full PCI remove + SBR + rescan. nvidia-smi sees all 4. However, the nvidia driver auto-probed during rescan, which means the IOMMU group types likely reverted to DMA-FQ again. Let me check:
>
[bash] ssh root@10.1.2.6 'for gpu in 0000:01:00.0 0000:11:00.0 0000:61:00.0 0000:71:00.0; do group=$(basename $(readlink /sys/bus/pci/devices/$gpu/iommu_group)); type=$(cat /sys/kernel/iommu_groups/$group/type); echo "$gpu: group=$group, type=$type"; done'
>
0000:01:00.0: group=28, type=DMA-FQ 0000:11:00.0: group=61, type=DMA-FQ 0000:61:00.0: group=42, type=DMA-FQ 0000:71:00.0: group=10, type=DMA-FQ
The tone is one of cautious optimism immediately tempered by expectation. The assistant opens with an exclamation — "GPUs are back!" — reflecting genuine relief after a long struggle to recover four NVIDIA RTX PRO 6000 Blackwell GPUs that had been rendered inoperable by earlier experiments. But the very next sentence acknowledges the likely cost of that recovery: the IOMMU group types have almost certainly reverted to their default DMA-FQ mode, undoing the identity-domain configuration that was the entire point of the exercise.
The Context: A Battle Against Blackwell's Firmware
To understand why this message matters, one must appreciate the battle that preceded it. The assistant had been attempting to enable peer-to-peer (P2P) DMA between GPUs — a critical optimization for multi-GPU model serving where GPUs need to exchange data directly without bouncing through CPU memory. On systems with IOMMU enabled (necessary for virtualization with SEV-SNP), P2P DMA is blocked by default because the IOMMU translates all DMA addresses, and peer-to-peer transactions don't go through the normal IOMMU mapping path.
The solution the assistant pursued was to set individual IOMMU groups to "identity" mode, which bypasses translation for devices in those groups. This is a per-group setting available in Linux kernels with IOMMU domain type support. The approach was elegant: leave the IOMMU enabled globally (for the VFIO GPUs assigned to a VM), but set identity mode only for the four NUMA0 GPUs used directly by the nvidia driver.
However, this ran headlong into a Blackwell-specific problem: the GPU's Firmware Security Processor (FSP) — a dedicated security co-processor that manages GPU initialization — enters a locked state when the nvidia driver is unloaded and reloaded without a full hardware power cycle. The FSP's Write Protected Region 2 (WPR2) remains "already up" from the previous driver session, and no software-initiated reset can clear it. The assistant tried Function Level Reset (FLR), Secondary Bus Reset (SBR), and even CXL bus reset — all failed to clear the FSP state. The only sequence that worked was a full PCI remove followed by SBR on the parent bridge, then a PCI rescan, which effectively re-enumerated the GPUs from a clean hardware state.
The Critical Realization
Message [msg 6278] captures the moment this recovery succeeded — and the moment the assistant understood the fundamental timing constraint. The GPUs were back, all four working, but the IOMMU groups had reverted to DMA-FQ because the nvidia driver auto-probed during the PCI rescan. The identity domain setting, which had been applied in an earlier step, was lost when the devices were removed from the PCI bus and their IOMMU groups destroyed.
The assistant's reasoning is visible in the structure of the message. The opening exclamation reflects the successful outcome of the recovery procedure. But the "However" that follows immediately signals the recognition of a deeper problem. The assistant doesn't even need to run the check to know what it will find — the prediction "likely reverted to DMA-FQ" is stated with confidence before the bash command confirms it. This is the thinking of someone who has internalized the mechanics of the PCI/IOMMU subsystem: when a device is removed from the bus, its IOMMU group is destroyed; when it reappears, a new group is created with the default type. The nvidia driver, matching the PCI vendor/device ID, binds instantly during the rescan, locking in the DMA-FQ type before any user-space intervention is possible.
Assumptions and Their Consequences
Several assumptions underpin the work visible in this message. The first is that IOMMU identity domains could be set at runtime, after the system was fully booted. This assumption proved correct in the narrow sense — the kernel does allow changing the domain type via the sysfs interface — but incorrect in the practical sense because the nvidia driver's auto-probing behavior on Blackwell GPUs makes it impossible to interpose the identity setting between device enumeration and driver binding.
A second assumption was that software resets (FLR, SBR, CXL bus reset) would be sufficient to clear the FSP state on Blackwell GPUs. This assumption was thoroughly disproven over the preceding messages. The FSP on Blackwell is designed to be resistant to software-initiated resets for security reasons — it maintains its state across driver unbind/bind cycles and even across PCI bus resets. Only a full power cycle (D3cold transition or physical power off) can truly reset it.
A third, more subtle assumption was that the IOMMU group sysfs interface would be available during the critical window between device enumeration and driver probing. In practice, the group is created during PCI enumeration and the driver probes almost immediately after. The window is measured in microseconds and is not accessible from user space without kernel-side intervention (such as a udev rule or a custom kernel module).
Input Knowledge Required
To fully understand this message, the reader needs knowledge of several interconnected systems. First, the Linux IOMMU subsystem and its domain type interface (/sys/kernel/iommu_groups/*/type) — understanding that DMA-FQ (DMA with Fast-path Quarantine) is the default translation mode, while identity bypasses translation for peer-to-peer transactions. Second, the PCI device model in Linux, including how devices are enumerated, how drivers match and bind, and how the remove/rescan mechanism works. Third, the NVIDIA Blackwell GPU architecture, particularly the GSP (GPU System Processor) and FSP (Firmware Security Processor) that manage GPU initialization and resist software reset. Fourth, the mechanics of P2P DMA and why IOMMU translation blocks it.
Output Knowledge Created
This message creates several pieces of actionable knowledge. It confirms that the PCI remove + SBR + rescan sequence successfully recovers Blackwell GPUs from a stuck FSP state — a valuable recovery procedure in itself. It establishes that the nvidia driver will auto-probe during rescan, making it impossible to set identity domains through the runtime sysfs interface alone. And it demonstrates that the IOMMU group numbers (28, 61, 42, 10) are stable across remove/rescan cycles, suggesting they are determined by the hardware topology rather than dynamic allocation.
The message also implicitly defines the problem space for the subsequent exploration. Over the next several messages, the assistant would try increasingly creative solutions: using driver_override to prevent auto-probing, unloading the entire nvidia kernel module, creating temporary blacklists, using NVIDIA's own gpu-admin-tools for fundamental reset, and finally crafting a udev rule to set identity domains at device-add time. Each of these attempts builds on the understanding crystallized in [msg 6278].
The Thinking Process
The assistant's thinking in this message follows a clear pattern: observe, predict, verify, conclude. The observation is that the recovery worked. The prediction is that the IOMMU types reverted. The verification is the bash command. The conclusion is implicit in the "However" — that the approach of setting identity domains at runtime is fundamentally blocked by the timing of driver auto-probing.
What makes this message noteworthy is what it doesn't say. The assistant doesn't express frustration or declare defeat. It doesn't enumerate the failed approaches that led to this point. It simply states the situation and moves to the next logical step: checking the current state. This is the hallmark of systematic debugging — treating each result as data rather than as success or failure.
The Broader Significance
In the larger narrative of the session, [msg 6278] marks the point where the assistant accepts a hard constraint of the hardware. The Blackwell FSP cannot be reset by software means, and the nvidia driver cannot be prevented from binding during PCI rescan without kernel-side mechanisms. This realization would eventually lead to the conclusion that IOMMU identity domains are fundamentally incompatible with Blackwell GPUs in this configuration — the FSP requires DMA translation mode during its boot sequence, and identity mode causes it to fail with error code 0x177.
The message also illustrates a broader truth about systems engineering: that the most elegant solutions often fail not because of bugs in the code, but because of timing constraints imposed by the hardware. The assistant's plan was logically sound — set identity domains, then let the driver bind — but the hardware's initialization sequence left no room for the intervention. In the end, the assistant would revert the approach entirely, remove the modprobe hook, and accept that P2P DMA would remain disabled, relying instead on MTP speculation for the throughput improvements that had been the ultimate goal all along.