The Pivot Point: Testing Nvidia's DmaRemapPeerMmio After the Blackwell IOMMU Identity Dead End

Introduction

In the long arc of a complex infrastructure debugging session spanning GPU topology reconfiguration, IOMMU domain manipulation, and Blackwell firmware behavior, message [msg 6328] stands as a quiet but pivotal transition. After discovering that per-group IOMMU identity domains are fundamentally incompatible with NVIDIA Blackwell GPUs — the Firmware Security Processor (FSP) fails with error code 0x177 when IOMMU is in identity mode — the assistant pivots from a failed approach to testing a completely different mechanism: the nvidia driver's built-in DmaRemapPeerMmio=1 parameter. This short message, consisting of a single line of commentary and a file read operation, marks the moment when the assistant accepts the impossibility of the identity domain strategy and turns toward working within the constraints of the existing system.

The Context: A Long Struggle with IOMMU Identity Domains

To understand why this message was written, one must appreciate the exhaustive effort that preceded it. The assistant had been attempting to restore GPU direct peer-to-peer (P2P) DMA access under a system where the AMD IOMMU was operating in full translation mode (required for SEV-SNP confidential computing). The standard approach for enabling P2P under an IOMMU is to set the IOMMU domain type to "identity" (passthrough) for the GPU devices, which bypasses address translation and allows GPUs to directly access each other's BAR (Base Address Register) memory regions.

The assistant had developed an elaborate modprobe install hook that would set identity domains for the NUMA0 GPUs before the nvidia driver loaded, hoping to catch the Blackwell GPUs in a pristine state. After a reboot to test this approach, the results were catastrophic: the nvidia Blackwell FSP boot sequence failed with error code 0x177. The FSP, it turned out, requires specific DMA mappings that are set up by the kernel's DMA API only when the IOMMU is in translation mode. Identity mode bypasses this setup, leaving the FSP unable to initialize.

This was not a timing problem that could be solved by racing the modprobe hook against driver initialization. It was a fundamental hardware compatibility issue. The assistant confirmed this by attempting multiple software-level resets — Function Level Reset (FLR), Secondary Bus Reset (SBR), and even CXL bus reset — none of which could clear the FSP's corrupted state once identity mode had been applied. The only recovery path was to revert to DMA-FQ (the default translated IOMMU mode), remove the GPUs from the PCI bus, perform SBR, reload the nvidia driver, and rescan — a delicate dance that restored functionality but left P2P DMA broken.

The Message Itself: A Simple File Read with Deep Implications

The subject message is deceptively simple:

Good, container sees GPUs. Let me run a quick P2P DMA test: [read] /home/theuser/glm-kimi-sm120-rtx6000bw/gpu_diag.py

The assistant reads a Python diagnostic script that tests GPU P2P access capabilities and performs actual data transfers. The file content is partially displayed, showing a diagnostic tool designed to probe torch.cuda.can_device_access_peer() for each GPU pair and run transfer tests.

The brevity of this message belies its significance. After dozens of commands exploring IOMMU group manipulation, modprobe hooks, SBR timing, and driver reload sequences — all of which led to the same dead end — the assistant is now taking a fundamentally different approach. Instead of trying to change the IOMMU configuration to accommodate P2P, the assistant will test whether the nvidia driver's existing DmaRemapPeerMmio=1 parameter (already enabled in the current driver configuration) actually works correctly under the current DMA-FQ regime.

The Reasoning: Why This Approach Makes Sense

The assistant's thinking, visible in the preceding messages, follows a clear logical progression. In [msg 6326], after discovering that there are no runtime-writable IOMMU parameters and that the kernel command line cannot be changed without coordination, the assistant explicitly states: "Let me try a completely different approach. Since the IOMMU identity domain cannot be set before the nvidia driver probes (because of the Blackwell GSP requirement that nvidia be loaded first), maybe I can work within the nvidia driver to fix P2P."

This is a critical insight. The assistant recognizes that the nvidia driver has a parameter called DmaRemapPeerMmio which, when set to 1, instructs the driver to create IOMMU DMA mappings for peer GPU BAR regions. If this mechanism works correctly, it would solve the P2P problem without requiring identity domains at all — the IOMMU would remain in translation mode, but the nvidia driver would ensure that the necessary mappings exist for peer GPU access.

In [msg 6327], the assistant confirms that DmaRemapPeerMmio is indeed set to 1 in the current driver configuration (driver version 590.48), and notes that EnablePCIERelaxedOrderingMode and EnableResizableBar are both disabled. The assistant then decides to test P2P in the current working state: "Actually, let me just test P2P right now with the current working state (DMA-FQ mode, DmaRemapPeerMmio=1) — maybe the driver has fixed this in 590.48."

Assumptions and Their Validity

The assistant makes several assumptions in this message and the surrounding reasoning:

Assumption 1: DmaRemapPeerMmio=1 might actually work correctly in driver version 590.48. This is a reasonable assumption — driver updates often fix known issues, and the assistant had previously experienced P2P failures with this parameter enabled, but perhaps a newer driver version or a different system state would yield different results. The assumption was partially validated by the subsequent test results ([msg 6332]), which showed that some GPU pairs could communicate while others could not.

Assumption 2: The container environment faithfully reflects the host GPU configuration. The assistant confirms "container sees GPUs" before proceeding, which is an important validation step. The container runs inside an LXC environment on the Proxmox host, and GPU visibility must be verified before running diagnostic tests.

Assumption 3: The existing diagnostic script (gpu_diag.py) is adequate for testing P2P. The assistant reads the existing file rather than writing a new test from scratch. This is efficient but carries the risk that the script might not test exactly what's needed. In the subsequent message ([msg 6330]), the assistant does write a more focused P2P test, suggesting that the existing script was indeed insufficient for the specific diagnostic needs.

Input Knowledge Required

To fully understand this message, one needs knowledge of several domains:

IOMMU Architecture: Understanding that the AMD IOMMU can operate in translation mode (where DMA addresses are remapped through I/O page tables) or identity mode (passthrough, where device addresses pass through untranslated). The DMA-FQ domain type refers to the default translated mode with flush queue optimization.

GPU P2P DMA: Knowledge that NVIDIA GPUs can directly access each other's memory via PCIe BAR mappings, bypassing host memory. This requires either an IOMMU in identity mode or proper DMA remapping entries in the IOMMU page tables.

Nvidia Driver Parameters: Understanding that DmaRemapPeerMmio controls whether the nvidia driver creates IOMMU mappings for peer GPU BAR regions. When set to 1, the driver should call dma_map_resource() or similar kernel APIs to establish these mappings.

Blackwell Architecture: The Blackwell GPU generation introduced the Firmware Security Processor (FSP), which has specific requirements for IOMMU behavior during initialization. The FSP boot sequence apparently uses the kernel's DMA API to set up its own memory mappings, which only work when the IOMMU is in translation mode.

The Broader System Context: The system has 8 Blackwell GPUs split between an LXC container (4 GPUs on NUMA0, using the nvidia driver) and a SEV-SNP VM (4 GPUs on NUMA1, using vfio-pci). The SEV-SNP requirement forces the AMD IOMMU to remain in full translation mode, creating the fundamental tension that the assistant is trying to resolve.

Output Knowledge Created

This message itself does not produce test results — it merely reads a file in preparation for testing. However, it sets the stage for the critical output that follows:

In [msg 6332], the assistant runs the P2P test and discovers a fascinating result: all 12 GPU pairs report canAccessPeer=True (the CUDA API believes P2P is enabled), but actual data transfers reveal a directional failure pattern. Transfers from GPU 0 (the lowest-numbered GPU on bus 01:00.0) to any other GPU fail, while transfers from higher-numbered GPUs to lower-numbered GPUs succeed. This pattern is consistent with incomplete IOMMU mappings — the nvidia driver's DmaRemapPeerMmio mechanism is creating mappings for some BAR regions but not others, likely due to the order in which peer BARs are mapped or a limitation in the driver's mapping logic.

The IO_PAGE_FAULTs logged in dmesg ([msg 6333]) confirm that the IOMMU is rejecting DMA accesses from certain GPU pairs, with the fault addresses pointing to peer GPU BAR regions that lack proper IOMMU mappings.

The Thinking Process: A Detective Story

The reasoning visible in the surrounding messages reveals a methodical, hypothesis-driven debugging process. The assistant:

  1. Identifies the constraint: IOMMU identity domains break Blackwell FSP boot (error 0x177)
  2. Verifies the constraint is fundamental: Multiple reset types (FLR, SBR, CXL) cannot clear the FSP state
  3. Explores alternatives: Kernel boot parameters (iommu.passthrough=1), runtime IOMMU parameter changes
  4. Discovers a new path: The nvidia driver's DmaRemapPeerMmio=1 parameter
  5. Tests the new path: Reads the diagnostic script to prepare for P2P testing This is classic debugging methodology: when one approach is definitively blocked, pivot to a completely different mechanism rather than continuing to fight the same battle. The assistant's willingness to abandon the identity domain approach after discovering the FSP incompatibility — rather than trying to find yet another timing variation or reset sequence — demonstrates disciplined reasoning.

The Broader Significance

This message represents the moment when the assistant accepts a fundamental constraint of the Blackwell architecture and adapts the strategy accordingly. The IOMMU identity domain approach was elegant in theory — it would have restored full P2P DMA without any driver-level changes — but it was defeated by an unexpected hardware requirement. The Blackwell FSP's dependency on DMA translation mode during initialization is a design constraint that no amount of software manipulation can circumvent.

The pivot to testing DmaRemapPeerMmio is not just a tactical shift; it represents a different philosophy. Instead of trying to change the system's configuration to match the ideal setup, the assistant now works with the existing configuration and tests whether the driver's built-in mechanisms can provide an acceptable solution. This is the difference between "making the system what it should be" and "making the best of what the system is" — a distinction that often separates successful infrastructure engineering from endless debugging loops.

The subsequent test results (partial P2P working with directional failures) would lead to further investigation of the nvidia driver's mapping logic, but the fundamental question was answered: P2P DMA under full IOMMU translation is possible on Blackwell, but the nvidia driver's implementation has gaps that prevent it from working universally. This knowledge is valuable for anyone deploying Blackwell GPUs in IOMMU-protected environments, and it sets the stage for potential workarounds or driver-level fixes.

Conclusion

Message [msg 6328] is a quiet turning point in a complex debugging session. In its simple act of reading a diagnostic script, it marks the transition from a failed approach (IOMMU identity domains) to a new investigation (nvidia driver's DmaRemapPeerMmio). The message embodies the discipline of recognizing when a path is truly blocked and pivoting to an alternative strategy. For the reader following this session, it serves as a reminder that the most important debugging skill is not persistence on a single approach, but the wisdom to know when to change tactics entirely.