The Four-Word Test: "rebooted, check P2P status"

Subject Message: [user] rebooted, check P2P status — Message 6369 in the conversation

Introduction

In the long arc of a complex systems engineering session spanning GPU topology reconfiguration, IOMMU domain manipulation, and Blackwell firmware behavior, there comes a moment when all the planning meets reality: the reboot. The message [user] rebooted, check P2P status is that moment. It is deceptively short — just four words — but it carries the weight of an entire session's worth of assumptions, debugging, and architectural decisions. The user is not simply reporting an action; they are handing the assistant a verdict on whether the elaborate boot-time configuration they designed together actually works.

Why This Message Was Written

The message exists because of a specific chain of events in the preceding session. The assistant had been attempting to enable GPU Peer-to-Peer (P2P) DMA across four NVIDIA RTX PRO 6000 Blackwell GPUs on a Proxmox host running kernel 6.14.11. P2P DMA is critical for multi-GPU model serving — it allows GPUs to directly access each other's memory without going through system RAM, which is essential for tensor parallelism in large language model inference. However, the host also needed to run a SEV-SNP (Secure Encrypted Virtualization) virtual machine on a separate set of four GPUs, which requires the AMD IOMMU to remain in full translation mode (amd_iommu=on).

The tension is fundamental: P2P DMA requires IOMMU identity domains (where DMA mappings bypass translation), but SEV-SNP requires the IOMMU to be active and translating. The solution devised was per-IOMMU-group identity domain switching — setting only the GPU groups assigned to the host's nvidia driver to identity mode while leaving the VM's GPU groups in DMA-FQ translation mode. This is a sophisticated kernel feature available on Linux 6.14.

The assistant had created a systemd service (gpu-iommu-identity.service) to set these identity domains at boot, and a gpu-vfio-split.service to bind the VM's GPUs to vfio-pci. The plan was: reboot, and the identity domains would be in place before nvidia loaded, enabling P2P without breaking the VM.

The user's message reports that they have performed this reboot and is now asking the assistant to verify the outcome. It is a natural handoff point: the user executed the physical action (rebooting a production server), and the assistant is expected to perform the diagnostic follow-up.

Assumptions Embedded in the Message

Both the user and the assistant carried several assumptions into this moment:

The user assumed that the boot-time configuration files (the systemd service, the modprobe hook, the shell scripts) were correctly authored and would execute in the right order. They trusted that the assistant's design was sound and that a simple reboot would validate it.

The assistant assumed that the systemd service ordering (After=gpu-vfio-split.service, Before=nvidia-persistenced.service) would be respected by systemd's boot sequence. It also assumed that the nvidia driver would not auto-load before the identity service ran — a critical assumption given that udev's modalias mechanism can trigger modprobe nvidia as soon as PCI devices are enumerated.

Both assumed that the Blackwell GPU's Firmware Security Processor (FSP/GSP) would tolerate being unbound from one driver and rebound to another. This assumption had already been challenged in the previous session, where the assistant discovered that the GSP survives all software-level resets (FLR, SBR, CXL bus reset) and that unbinding/rebinding nvidia corrupts the GSP state permanently within a boot session. Despite this knowledge, the hope was that the boot-time approach — setting identity domains before nvidia ever touched the GPUs — would avoid the problem entirely.

What Actually Happened

The assistant's investigation (messages 6370–6409) revealed a cascade of failures rooted in these assumptions:

Boot ordering failure: The gpu-iommu-identity.service ran before gpu-vfio-split.service, despite the After= directive. This happened because the identity service had DefaultDependencies=no (necessary to run early), which apparently bypassed some ordering constraints. The identity service correctly set the IOMMU groups to identity, but then the vfio-split script's echo "10de 2bb5" > new_id registered the GPU PCI ID with vfio-pci, causing all unbound GPUs with that ID — including the NUMA0 ones — to be grabbed by vfio-pci.
The GSP corruption trap: When the assistant tried to fix this at runtime by unbinding the NUMA0 GPUs from vfio-pci and rebinding them to nvidia, the Blackwell GSP was already locked from the initial nvidia probe at boot. The unbind/rebind sequence produced the same kfspSendBootCommands_HAL failure (error 0x62:0xffff:2142) seen in the previous session. The GPUs appeared in sysfs as bound to nvidia with identity domains, but nvidia-smi reported "No devices were found."
SBR's limitations confirmed: The assistant attempted Secondary Bus Reset (SBR) on the parent PCI bridges, hoping to clear the GSP state. This had worked once before in a specific scenario — when nvidia had never been loaded. But in this boot session, the GSP was already initialized, and SBR proved insufficient to reset it. The error _kgspBootGspRm: (the GPU is likely in a bad state and may need to be reset) confirmed that the Blackwell GSP firmware state survives SBR, contradicting the assistant's earlier assumption that SBR was a reliable reset mechanism.
The modprobe install hook pivot: After multiple failed attempts to recover the GPUs with identity domains at runtime, the assistant pivoted to a fundamentally different approach: a modprobe install hook. The file /etc/modprobe.d/nvidia-iommu-identity.conf contains:

   install nvidia /usr/local/bin/gpu-set-identity-before-nvidia.sh

This hook runs instead of the normal modprobe nvidia command. The script sets the IOMMU identity domains for the NUMA0 GPUs, then loads the real nvidia module via exec /sbin/modprobe --ignore-install nvidia. This ensures identity domains are in place during the very first nvidia probe, when the GSP is still clean from power-on.

Recovery without identity: The assistant eventually restored the GPUs to working order by temporarily disabling the modprobe hook, doing a full remove+SBR+rescan cycle, and letting nvidia load normally (with DMA-FQ domains). SGLang was restarted with MTP speculation enabled, and the system returned to a stable state — but without P2P DMA, and with the understanding that another reboot would be needed to test the modprobe hook properly.

Input Knowledge Required

To fully understand this message and its aftermath, one needs:

IOMMU architecture: Understanding of DMA remapping, identity vs. translation domains, and how per-group domain switching works on AMD IOMMU with Linux 6.14.
PCI device model: Knowledge of driver binding/unbinding, PCI rescan, Secondary Bus Reset, and how drivers_probe and new_id work in sysfs.
Blackwell GPU firmware: The critical insight that NVIDIA's Blackwell architecture uses a Firmware Security Processor (FSP/GSP) that initializes during the first driver probe and cannot be reset without a full power cycle. This is a hardware-level constraint, not a software bug.
Linux boot sequence: How systemd services are ordered, how udev triggers module loading via modalias, and how modprobe install hooks can intercept module loading.
SGLang and MTP: The speculative decoding configuration that was already running and needed to be preserved across the reboot.

Output Knowledge Created

This message and its investigation produced several lasting insights:

The modprobe install hook pattern works: The hook successfully set identity domains before nvidia loaded (verified by journalctl logs showing the identity changes). The failure was only because the GSP was already corrupted from an earlier bind in the same boot session.
Blackwell GSP cannot be reset by SBR: This was confirmed definitively. Previous success with SBR was only possible when nvidia had never loaded in that boot session.
The correct boot sequence is: (1) vfio-split binds NUMA1 GPUs, (2) modprobe install hook sets identity domains, (3) nvidia loads and probes clean GPUs. The systemd service approach was fundamentally flawed because it ran too late.
P2P DMA restoration requires a cold boot: The modprobe hook is in place and will be tested on the next reboot. If it works, NCCL_P2P_DISABLE=1 and --disable-custom-all-reduce can be removed from the SGLang service for additional performance.

The Thinking Process

The assistant's reasoning across messages 6370–6409 reveals a methodical diagnostic process. Upon receiving the user's message, the assistant immediately checked the IOMMU group types — finding them correctly set to identity — but then discovered the NUMA0 GPUs were bound to vfio-pci instead of nvidia. This triggered a root-cause analysis: the boot ordering was wrong.

The assistant then attempted runtime recovery through multiple escalating strategies: first simple unbind/rebind, then PCI remove + SBR + rescan, then blocking nvidia autoload with a modprobe blacklist, then trying to load nvidia before rescan. Each failure provided more information about the GSP's resilience. The pivot to the modprobe install hook was the culmination of this learning — a recognition that the timing of identity domain setting relative to nvidia's first probe was the only variable that mattered.

The final recovery (restoring GPUs without identity domains) showed pragmatic engineering judgment: when the ideal solution requires another reboot, restore the system to a working state with the infrastructure for the next attempt already in place.

Conclusion

A four-word message from the user triggered one of the most technically dense investigations in the session. It exposed the gap between a theoretically correct boot-time configuration and the messy reality of systemd ordering, PCI driver interactions, and Blackwell firmware behavior. The modprobe install hook that emerged from this investigation is a elegant solution — it intercepts the module loading at the precise moment when identity domains must be set, before nvidia's probe locks the GSP. But it requires another reboot to validate, leaving the system in a stable-but-suboptimal state with MTP speculation active but P2P DMA still disabled. The story captured in this single message is one of assumptions meeting reality, and engineering adapting in response.