The Blackwell GSP Trap: When PCI Resets Can't Erase Firmware State

In the high-stakes world of production GPU deployment, few moments are as sobering as watching nvidia-smi return "No devices were found" after what should have been a routine driver rebind operation. Message 6269 captures exactly such a moment — a critical inflection point in an extended debugging session where the assistant attempts to restore GPU peer-to-peer (P2P) DMA by switching IOMMU groups to identity mode, only to discover that Blackwell GPUs have a fundamental architectural constraint that no software-level PCI reset can overcome.

The Context: A Plan to Restore P2P DMA

The message sits within a larger narrative arc spanning dozens of rounds. The assistant had been managing a production inference server running Qwen3.5-122B-A10B BF16 on four NVIDIA RTX PRO 6000 Blackwell GPUs, tensor-parallelism 4, inside an LXC container on a Proxmox host. The host also ran a SEV-SNP confidential VM with four additional GPUs bound to vfio-pci. The system worked, but with a significant performance penalty: NCCL P2P was disabled (NCCL_P2P_DISABLE=1) because the IOMMU, operating in full DMA translation mode (DMA-FQ), caused P2P DMA transfers between GPUs to fault.

The assistant had formulated a plan in earlier messages (msg 6253-6256): set the IOMMU group type to identity for the four nvidia GPU groups (42, 61, 28, 10) while leaving the vfio-pci groups in translation mode for the SEV-SNP VM. This per-group approach was theoretically supported by the Linux kernel since 5.11 and would give the nvidia GPUs passthrough-equivalent IOMMU behavior, enabling P2P DMA without affecting the VM's security isolation.

The execution began in msg 6257-6265. The assistant stopped SGLang, killed zombie processes, unbounded all four GPUs from the nvidia driver, wrote identity to each IOMMU group's type file, and rebounded the driver. The initial results were promising: the type files confirmed identity, and the bind operations reported success. But then came the first ominous sign — nvidia-smi returned "No devices were found" (msg 6266). Kernel logs revealed the cause: RmInitAdapter failed! (0x62:0xffff:2142) and, critically, _kgspBootGspRm: unexpected WPR2 already up, cannot proceed with booting GSP (msg 6270).

The Subject Message: A Diagnosis in Two Sentences

Message 6269 is deceptively brief — just two bash commands and their output — but it represents a critical diagnostic pivot. The assistant writes:

PCI remove/rescan worked — GPUs are back and nvidia driver auto-probed them. But note the IOMMU group types reverted to DMA-FQ after the rescan (new groups were created). And the group numbers changed for some GPUs. Let me check if the GPUs actually work now:

>

`` $ ssh root@10.1.2.6 'nvidia-smi --query-gpu=index,gpu_bus_id,name,memory.total --format=csv,noheader 2>&1' No devices were found ``

This message is the moment the assistant realizes that the problem is deeper than a simple driver rebind timing issue. The PCI remove/rescan sequence successfully re-enumerated the GPUs — they reappeared in sysfs, the nvidia driver auto-probed them — but the GPUs still failed to initialize. The nvidia-smi output of "No devices were found" is the hammer blow: the GPUs are visible to the PCI subsystem but invisible to the nvidia driver stack.

The Reasoning Process Visible in the Message

The assistant's thinking is laid bare in the commentary. Three observations are made:

First, "PCI remove/rescan worked — GPUs are back and nvidia driver auto-probed them." This confirms that the PCIe bus enumeration and driver binding mechanism functions correctly. The GPUs are not physically dead; they respond to PCI configuration cycles, their vendor/device IDs are recognized, and the nvidia driver's probe routine runs.

Second, "But note the IOMMU group types reverted to DMA-FQ after the rescan (new groups were created)." This is a crucial discovery about the lifecycle of IOMMU groups. When a PCI device is removed and rescanned, the kernel creates new IOMMU groups. The previous identity setting, written to the old group's type file, is lost because the group object itself was destroyed and recreated. The assistant had set identity domains before the remove/rescan, but the rescan invalidated that work.

Third, "the group numbers changed for some GPUs." Specifically, GPU 0000:01:00.0 moved from group 42 to group 28, while 0000:61:00.0 moved from group 28 to group 42. This group number swap is a side effect of the order in which devices are re-enumerated during the PCI rescan — the kernel assigns group numbers sequentially based on discovery order, not by PCI address. This means any automation script that hardcodes IOMMU group numbers would break after a rescan.

The final line — the nvidia-smi failure — is the punchline. The assistant had hoped that the PCI remove/rescan would fully reset the GPU state, clearing whatever firmware lock was preventing initialization. It did not.

The Deeper Problem: Blackwell's GSP Firmware Lock

To understand why the GPUs failed to initialize, we need to look beyond this message to the kernel logs revealed in msg 6270. The critical error was _kgspBootGspRm: unexpected WPR2 already up, cannot proceed with booting GSP. The GSP (GPU System Processor) is a dedicated microcontroller on modern NVIDIA GPUs that handles firmware tasks including power management, security, and initialization. WPR2 (Write Protected Region 2) is a memory region that the GSP firmware locks during initialization to prevent tampering.

When the nvidia driver was unbound and rebounded without a proper GPU reset, the WPR2 remained locked from the previous session. The GSP firmware detected this and refused to reinitialize — a safety mechanism to prevent two driver instances from conflicting over GPU state. A PCI remove/rescan, which the assistant tried, does not clear this lock because the GSP retains its state across PCI resets. The GSP is powered by the GPU's auxiliary power rail, which stays active during a PCI reset. Only a full power cycle (D3cold → D0 transition) or a specific firmware-level reset command can clear the WPR2 lock.

Assumptions Made and Lessons Learned

The assistant made several assumptions that proved incorrect. First, it assumed that unbinding and rebinding the nvidia driver would cleanly reset the GPU state. For most GPU operations, this is true — the driver's teardown routine normally releases firmware resources. But the identity domain switch appears to have interrupted this teardown at a point where the GSP firmware was mid-operation, leaving the WPR2 lock in place.

Second, the assistant assumed that a PCI remove/rescan would be sufficient to clear any residual GPU state. This assumption was based on experience with older GPU architectures (Ampere, Hopper) where PCI resets are sufficient to recover from most driver-level hangs. Blackwell introduces the GSP as a more autonomous firmware processor, and its state persistence across PCI resets is a new constraint.

Third, the assistant assumed that setting identity domains before the rescan would persist. The observation that "new groups were created" reveals that IOMMU group objects are ephemeral — they exist only as long as the device is present. Any per-group configuration must be reapplied after a rescan.

Input Knowledge Required

To fully understand this message, the reader needs familiarity with several concepts: IOMMU groups and their domain types (DMA-FQ vs identity), the Linux PCI device model (sysfs, driver binding/unbinding, remove/rescan), NVIDIA's GSP firmware architecture (introduced with Blackwell), and the NCCL P2P DMA mechanism used for GPU-to-GPU communication in multi-GPU inference. Knowledge of the Proxmox virtualization stack (LXC containers, VFIO passthrough, SEV-SNP) is also helpful for understanding why the IOMMU configuration matters in the first place.

Output Knowledge Created

This message generates several critical pieces of knowledge. It establishes that PCI remove/rescan is insufficient to clear Blackwell GSP firmware state — a finding that has implications for any deployment that requires GPU driver rebinding. It documents the ephemeral nature of IOMMU group objects and the need to reapply identity domain settings after a rescan. It reveals that group numbers can change across rescans, breaking any hardcoded group references. And it demonstrates that even when GPUs reappear in sysfs with the correct driver bound, they may still fail nvidia-smi initialization if the GSP firmware is in a locked state.

The Aftermath

The assistant did not give up after this message. In subsequent rounds (msg 6271-6280), it attempted FLR (Function Level Reset), secondary bus reset (SBR) via the parent PCIe bridge's control register, and a combination of PCI remove + SBR + rescan with driver_override tricks. Each attempt revealed more about the Blackwell GSP's resilience. Eventually, the assistant discovered that the only way to clear the GSP lock was a full PCI remove + SBR cycle followed by rescan — and even then, the identity domain setting had to be applied in the brief window between rescan and nvidia driver auto-probe. But ultimately, the assistant would discover a deeper truth: Blackwell's FSP (Firmware Security Processor) boot fails with error 0x177 when IOMMU is in identity mode, making per-group identity domains fundamentally incompatible with Blackwell GPUs regardless of timing.

Message 6269 stands as the moment when a promising optimization hit an unexpected architectural wall — a reminder that in the frontier of GPU hardware, the firmware is as much a part of the system as the silicon itself.