The Verification That Preceded Discovery: A Modprobe Hook Check in the Blackwell IOMMU Saga

Introduction

In the midst of a grueling multi-session effort to deploy NVIDIA Blackwell GPUs for AI inference, a single, deceptively simple message appears: the assistant runs cat on a configuration file to verify its contents. Message <msg id=6398> is barely a line of bash and its output, yet it sits at a critical inflection point in the conversation. This message is the calm before a storm of failure — a verification step that, once passed, leads directly to a definitive discovery about the incompatibility between Blackwell GPUs and IOMMU identity domains. Understanding why this message was written, what it assumes, and what it enables reveals the meticulous engineering discipline required when pushing hardware to its limits.

The Message

The assistant writes:

Now verify the modprobe config is correct: [bash] ssh root@10.1.2.6 'cat /etc/modprobe.d/nvidia-iommu-identity.conf' # Before nvidia module loads, set IOMMU identity domains for NUMA0 GPUs # This runs when modprobe is triggered (e.g., by PCI rescan modalias) install nvidia /usr/local/bin/gpu-set-identity-before-nvidia.sh

That is the entirety of the message. A single remote command to read a file, and the file's contents printed back. On its surface, it is trivial — a sanity check. But in the context of the session, it represents the culmination of hours of debugging, a novel solution to an intractable timing problem, and the final checkpoint before a test that would reveal a fundamental hardware limitation.

The Context: A Desperate Search for P2P DMA

To understand why this verification matters, one must understand the problem the assistant was trying to solve. The system had 8 NVIDIA RTX PRO 6000 Blackwell GPUs split across two NUMA domains. Four GPUs on NUMA0 were bound to the nvidia driver for inference workloads, while four on NUMA1 were assigned to VFIO for a confidential VM. The NUMA0 GPUs needed Peer-to-Peer (P2P) DMA to communicate efficiently during tensor-parallel inference — without it, all GPU-to-GPU traffic had to go through system memory, crippling performance.

The obstacle was the IOMMU (Input-Output Memory Management Unit). With the system running under a virtualized host (Proxmox) using SEV-SNP (Secure Encrypted Virtualization), the IOMMU was in full translation mode by default. This meant every DMA transfer from a GPU went through address translation, which broke the direct memory mappings that P2P DMA requires. The solution, in theory, was to set specific IOMMU groups to "identity" mode — bypassing translation for those devices while keeping protection for others.

But the assistant kept hitting a wall. Every attempt to set identity domains failed because the nvidia driver would load and bind to the GPUs before the identity could be configured. The kernel's modalias mechanism — which automatically triggers modprobe nvidia when a PCI device with matching vendor/device ID appears on the bus — left no window between device enumeration and driver binding.

The Modprobe Install Hook: A Clever Workaround

After numerous failed attempts involving PCI removal, secondary bus reset (SBR), and careful sequencing of module loading, the assistant arrived at a novel approach: the modprobe install hook. The install directive in modprobe.d configuration files allows replacing the normal module loading command with a custom script. By creating /etc/modprobe.d/nvidia-iommu-identity.conf with the line install nvidia /usr/local/bin/gpu-set-identity-before-nvidia.sh, the assistant ensured that whenever the kernel tried to load the nvidia module — whether via explicit modprobe, modalias trigger, or dependency resolution — it would first run the custom script.

The script itself (gpu-set-identity-before-nvidia.sh) iterated over the four NUMA0 GPU PCI addresses, checked if each device existed on the bus, determined its IOMMU group, and wrote "identity" to that group's type file — before the real nvidia module ever touched the hardware. Only then did it execute /sbin/modprobe --ignore-install nvidia to load the actual driver. This was a clever piece of systems engineering: it exploited the modprobe hook mechanism to insert a configuration step into a kernel module loading chain that otherwise offered no intervention point.

Why This Verification Matters

The subject message is the moment the assistant pauses to verify that the hook is correctly installed before triggering the destructive test. This is not idle caution — the test that follows involves removing all four GPUs from the PCI bus, performing a secondary bus reset on their upstream bridges (which takes 15 seconds and carries risk of system instability), rescannning the bus, and hoping the hook fires correctly. If the configuration file had a typo, a syntax error, or was written to the wrong path, the entire test would fail silently, wasting time and potentially leaving the system in a broken state.

The assistant's reasoning is visible in the structure of the verification: it checks the file remotely via SSH, reads its contents, and implicitly confirms that the install directive is syntactically valid and points to the correct script path. The comment in the file — "This runs when modprobe is triggered (e.g., by PCI rescan modalias)" — serves as documentation for any future reader, but also as a self-reminder that the hook must fire during the modalias-triggered load, not just explicit modprobe calls.

Assumptions Embedded in This Message

The message carries several assumptions, some of which would prove incorrect:

The hook would execute before nvidia's PCI probe: The assistant assumed that the modprobe install hook runs early enough in the module loading sequence that the nvidia driver's PCI probe callback would see the identity domains already configured. This turned out to be correct in terms of execution order, but irrelevant because of a deeper issue.
Identity domains would fix the Blackwell FSP boot: The assistant assumed that the GSP (GPU System Processor) firmware initialization failures seen in earlier attempts were caused by the IOMMU being in translation mode when nvidia probed the device. The hope was that identity mode would allow the GSP firmware to boot correctly. This assumption was wrong — as discovered in the subsequent messages, Blackwell's FSP requires DMA translation mode during initialization, and identity mode actually causes the GSP boot failure with error code 0x177.
The modprobe.d mechanism would work reliably on this kernel: The assistant assumed that the Ubuntu 24.04 kernel's modprobe infrastructure would correctly dispatch to the install hook during a modalias-triggered module load. This was a reasonable assumption given that the install directive is a standard feature, but edge cases exist where modalias loading bypasses modprobe entirely (e.g., built-in drivers or direct request_module calls from the kernel).
The script file was correctly deployed: The verification of the config file implicitly trusts that the script at /usr/local/bin/gpu-set-identity-before-nvidia.sh was correctly written and deployed in the previous messages. The assistant had encountered a heredoc parsing error when trying to create the script remotely (msg 6394), then used the write tool to create it locally and deployed it via scp (msg 6397). The verification of the config file does not re-check the script itself.

Input Knowledge Required

To fully understand this message, a reader needs knowledge of:

Linux kernel module loading mechanics: How modprobe, modprobe.d, and the install directive work, and how modalias triggers automatic driver loading when PCI devices are discovered.
IOMMU concepts: The distinction between translation and identity domains, and how the /sys/kernel/iommu_groups/*/type interface controls them.
PCIe topology and resets: How PCI device removal, secondary bus reset, and rescan work, and why the 15-second delay after SBR is necessary for GSP firmware to reset.
NVIDIA Blackwell architecture: The role of the GSP/FSP firmware, and how it initializes during driver probe — knowledge that the assistant was actively developing through trial and error.
The broader deployment context: That this system runs Qwen3.5-122B-A10B BF16 with tensor parallelism across 4 GPUs, that P2P DMA was the optimization target, and that the alternative (NCCL_P2P_DISABLE=1) was already working but suboptimal.

Output Knowledge Created

This message creates several forms of knowledge:

Confirmation of the modprobe hook configuration: The output confirms that the install directive is correctly written and the script path is valid. This is the immediate, practical output.
A documented approach for pre-driver IOMMU configuration: The technique of using a modprobe install hook to configure IOMMU domains before a GPU driver loads is novel and potentially reusable for other scenarios where driver probe timing is critical.
A testable hypothesis: With the hook verified, the assistant can now proceed to the destructive test (msg 6399) that will determine whether the approach works. The verification message is the precondition for that experiment.
Negative knowledge (soon to be discovered): While not yet known at the time of this message, the verification enables the test that will definitively prove that Blackwell GPUs cannot work with IOMMU identity domains — a critical piece of negative knowledge that saves countless future hours of debugging.

What Happened Next

Immediately after this verification, the assistant ran the full remove-SBR-rescan cycle (msg 6399). The result was "No devices were found" — the GPUs disappeared entirely. Checking further (msg 6400), the assistant found the same GSP initialization failure: rm_init_adapter failed, gpuHandleSanityCheckRegReadError_GH100, and the 0xbadf4100 register value that indicates a hardware-level initialization problem. The modprobe hook had executed (journalctl showed the identity messages), but identity mode itself was the problem — Blackwell's FSP requires the DMA mappings set up by the kernel's DMA API in translation mode, and identity mode breaks this initialization.

The assistant would later summarize: "per-group IOMMU identity domains are fundamentally incompatible with Blackwell GPUs — the approach cannot work regardless of timing." The modprobe hook approach was correct engineering, but it was solving the wrong problem. The real limitation was hardware, not timing.

Conclusion

Message <msg id=6398> is a testament to disciplined engineering practice in the face of complexity. In a session filled with elaborate bash scripts, PCI bus manipulations, and kernel debugging, this simple verification step demonstrates the importance of checking assumptions before committing to destructive tests. The assistant did not rush from creating the hook to testing it — it paused, verified, and confirmed. That the test ultimately failed does not diminish the value of this verification; rather, it highlights that even the most careful preparation cannot overcome fundamental hardware limitations. The modprobe install hook technique remains a clever and potentially reusable solution for IOMMU configuration timing problems — it simply met its match in Blackwell's firmware initialization requirements.