The Boot Order Trap: Diagnosing a GPU Driver Race Condition in IOMMU Identity Domain Configuration

Introduction

In the complex world of high-performance GPU computing, the difference between a working configuration and a broken one often comes down to timing. This article examines a single message from an opencode coding session—message index 6373—where an AI assistant diagnosed and corrected a subtle boot-order race condition that left four NVIDIA Blackwell GPUs bound to the wrong driver after a system reboot. The message captures a moment of debugging that reveals deep insights into Linux PCI device management, IOMMU configuration, systemd service orchestration, and the unforgiving hardware requirements of modern GPUs.

Context: The Quest for GPU P2P DMA

The broader session concerns the deployment of a Qwen3.5-122B-A10B large language model using SGLang on a Proxmox host with eight NVIDIA RTX PRO 6000 Blackwell GPUs. Four GPUs are dedicated to the inference server (NUMA0), while the other four are passed through via VFIO to a SEV-SNP virtual machine (NUMA1). A critical performance optimization—GPU peer-to-peer (P2P) DMA—had been disabled because of incompatibilities with the IOMMU's full translation mode (DMA-FQ), which is required for the SEV-SNP VM's security guarantees.

The assistant had devised an elegant solution: use per-IOMMU-group identity domains. By switching only the NUMA0 GPU groups from DMA-FQ to identity mode, P2P DMA would be enabled for the inference GPUs while keeping full IOMMU translation for the VFIO GPUs. A systemd service (gpu-iommu-identity.service) was created to set these identity domains at boot, and a companion service (gpu-vfio-split.service) was responsible for binding the NUMA1 GPUs to the vfio-pci driver.

The user rebooted the host and asked the assistant to verify the P2P status ([msg 6369]). What followed was a cascade of discoveries.

The Subject Message: A Diagnostic Breakthrough

Message 6373 opens with the assistant's realization:

I see what happened: 1. gpu-iommu-identity.service ran first — unbound nvidia from NUMA0 GPUs and set identity domains ✓ 2. gpu-vfio-split.service ran — used echo "10de 2bb5" > new_id which registered the PCI ID with vfio-pci, causing ALL unbound GPUs with that ID (including the NUMA0 ones) to get grabbed by vfio-pci

This is the core diagnostic insight. The assistant had verified in the preceding messages ([msg 6371]) that the IOMMU identity domains were correctly set to identity for all four NUMA0 GPU groups. But the GPUs themselves were bound to vfio-pci instead of nvidia. The assistant immediately connected the dots: the boot order was wrong.

The root cause is a subtle interaction between two systemd services. The identity service runs first, unbinding the NUMA0 GPUs from the nvidia driver and setting their IOMMU groups to identity. Then the vfio-split service runs, using the new_id sysfs interface to register the PCI vendor/device ID 10de:2bb5 (NVIDIA Blackwell GPU) with the vfio-pci driver. The critical detail is that new_id doesn't discriminate—it causes vfio-pci to claim all unbound devices matching that PCI ID, including the NUMA0 GPUs that the identity service had just released.

The Reasoning Process

The assistant's thinking in this message is a model of systematic debugging. The sequence of reasoning is:

  1. Observation: The IOMMU groups are correctly set to identity, but the GPUs are on vfio-pci instead of nvidia.
  2. Hypothesis generation: The vfio-split service's new_id registration grabbed all unbound GPUs.
  3. Root cause identification: The identity service runs first, unbinding GPUs, then vfio-split claims them.
  4. Solution space exploration: The assistant considers two approaches—either change the boot order so identity runs after vfio-split, or update the identity script to rebind to nvidia after vfio-split has run.
  5. Decision: The assistant chooses the more robust approach of updating the identity script to handle the rebinding itself, making it self-contained regardless of boot order. This reasoning is visible in the message's structure: the diagnostic "I see what happened" section, followed by the proposed fix, followed by the implementation. The assistant doesn't just patch the immediate symptom—it redesigns the script to be resilient to the actual boot sequence.

The Script Rewrite: Design Decisions

The assistant proceeds to rewrite /usr/local/bin/gpu-iommu-identity.sh with several important design decisions:

Self-contained lifecycle management: The new script handles unbinding from whatever driver is currently bound (whether nvidia or vfio-pci), setting the identity domain, and then rebinding to nvidia. This makes it order-independent—it works whether it runs before or after vfio-split.

Graceful handling of edge cases: The script checks for device existence, skips devices without IOMMU groups, and verifies the current driver before attempting unbinding. Each step has error handling and diagnostic output.

Use of drivers_probe: The rebinding mechanism uses the sysfs drivers_probe interface (echo "$gpu" > /sys/bus/pci/drivers_probe), which triggers the kernel to probe all available drivers for the device. This is a more robust approach than directly writing to a specific driver's bind file, as it lets the kernel's driver matching logic determine the correct driver.

Timing considerations: The script includes sleep 0.5 after unbinding and sleep 1 after probing, acknowledging that PCI device state transitions are not instantaneous and the kernel needs time to process the change.

Assumptions and Their Consequences

The original design made a critical assumption: that running the identity service before the vfio-split service would work correctly. The reasoning was that the identity service would set up the IOMMU domains for the NUMA0 GPUs, and then vfio-split would only affect the NUMA1 GPUs. This assumption was flawed in two ways:

  1. The new_id mechanism is indiscriminate: Writing a PCI ID to vfio-pci's new_id file causes the driver to claim all unbound devices matching that ID, regardless of which IOMMU group they belong to or which physical NUMA node they're on.
  2. Unbinding creates a window of vulnerability: When the identity service unbinds the NUMA0 GPUs from nvidia, those devices become unbound. Any subsequent driver claiming them (via new_id, driver probe, or hotplug) will succeed. The assistant's updated script corrects both assumptions: it explicitly rebinds the NUMA0 GPUs to nvidia after setting identity domains, closing the window of vulnerability.

Input Knowledge Required

To fully understand this message, one needs knowledge of:

Linux PCI device model: Understanding of /sys/bus/pci/devices/, driver binding/unbinding via sysfs, and the drivers_probe interface.

IOMMU architecture: Knowledge of IOMMU groups, domain types (DMA-FQ vs identity), and how they affect DMA transactions.

vfio-pci driver mechanics: Understanding that new_id registers a PCI vendor/device ID with the driver, causing it to claim all matching unbound devices.

Systemd service ordering: Understanding that systemd services have dependencies and ordering constraints, and that the After= directive doesn't guarantee exclusive access to resources.

NVIDIA GPU driver architecture: Knowledge that the nvidia driver can be unbound and rebound at runtime, and that drivers_probe can trigger driver matching.

Blackwell GPU specifics: Understanding that these GPUs have a firmware security processor (FSP) that imposes constraints on IOMMU configuration—though this particular message doesn't delve into the FSP issue that would later emerge.

Output Knowledge Created

This message produces several important pieces of knowledge:

A corrected boot script: The updated gpu-iommu-identity.sh is a production-ready script that handles the full lifecycle of unbinding, identity domain setting, and rebinding. It's designed to be order-independent and robust.

Boot order insight: The realization that new_id in vfio-pci is indiscriminate and will claim all matching devices, not just the intended ones. This is a subtle but important detail that could affect any system using vfio-pci with multiple GPUs of the same model.

Diagnostic methodology: The systematic approach of checking both the IOMMU group type and the driver binding separately, then reasoning about the interaction between services, is a reusable debugging pattern.

The race condition pattern: The message documents a specific class of boot-time race condition where two services interact through shared kernel state (PCI device bindings). This pattern—where Service A releases a resource that Service B then claims, even though Service A intended to keep that resource—is applicable beyond GPU configuration.

The Broader Narrative

This message is a turning point in the session's P2P DMA saga. The assistant has just discovered that the boot order is wrong and is implementing a fix. However, the analyzer summary for the segment reveals that this entire approach would ultimately be defeated by a deeper hardware issue: the Blackwell GPU's FSP (Firmware Security Processor) fails with error code 0x177 when IOMMU is in identity mode, regardless of timing. The FSP requires specific DMA mappings set up by the kernel's DMA API in translation mode, and identity mode breaks this initialization.

The message doesn't yet know about this FSP issue—that discovery comes later in the segment. At this point, the assistant is still operating under the assumption that per-group identity domains are a viable solution, and the only problem is the boot order. This makes the message a poignant snapshot of debugging in progress: the assistant is solving one problem while a deeper, more fundamental problem lurks undiscovered.

Technical Depth: The new_id Mechanism

The key technical insight in this message is the behavior of the vfio-pci new_id sysfs interface. When a user writes a vendor/device ID pair to /sys/bus/pci/drivers/vfio-pci/new_id, the kernel:

  1. Adds the ID to the driver's dynamic ID table
  2. Iterates over all unbound PCI devices
  3. For each unbound device, checks if its vendor/device ID matches the new entry
  4. If matched, binds the device to vfio-pci The critical detail is step 2: it iterates over all unbound devices, not just devices on a specific bus or in a specific IOMMU group. This means that if you have multiple GPUs with the same PCI ID, and some are intentionally unbound (because you unbind them in a prior service), the new_id write will claim them all. The assistant's fix addresses this by ensuring the NUMA0 GPUs are never in an unbound state when new_id is written. By rebinding them to nvidia immediately after setting identity domains, they remain claimed by nvidia and are invisible to vfio-pci's new_id sweep.

Conclusion

Message 6373 is a masterclass in diagnostic reasoning for complex Linux system administration. It demonstrates the importance of understanding not just individual kernel interfaces, but how they interact across service boundaries at boot time. The assistant's systematic approach—observe, hypothesize, identify root cause, explore solutions, implement—transforms a confusing driver binding failure into a clear, fixable problem.

The message also illustrates a fundamental truth about systems engineering: the order of operations matters. In the tightly choreographed dance of kernel initialization, driver probing, and service startup, a single misplaced step can cascade into a system-wide misconfiguration. The assistant's fix—making the identity script self-contained and order-independent—is not just a patch but a design improvement that eliminates the race condition at its source.

For readers unfamiliar with GPU virtualization and IOMMU configuration, this message offers a window into the intricate world where kernel internals, hardware constraints, and system administration intersect. It shows that even seemingly simple operations—like setting a domain type or writing to a sysfs file—can have far-reaching consequences when multiple services interact through shared kernel state.