The Boot Order Bug: Diagnosing a GPU Driver Conflict After IOMMU Identity Domain Configuration

Introduction

In the complex dance of system initialization, the order in which services start can determine whether a carefully crafted configuration succeeds or fails. Message [msg 6372] captures a pivotal moment in an opencode coding session where an AI assistant, after a host reboot intended to enable GPU peer-to-peer (P2P) DMA via IOMMU identity domains, discovers that its plan has gone awry. The NUMA0 GPUs that should be bound to the nvidia driver are instead sitting on vfio-pci, and the assistant must diagnose why.

This message is a masterclass in systems debugging: the assistant receives raw diagnostic output, immediately recognizes the discrepancy between expected and actual state, traces the root cause to a service ordering issue, and formulates a concrete plan to fix it. The reasoning is concise but complete, demonstrating how deep kernel and driver knowledge translates into rapid diagnosis.

The Context: A Multi-Phase P2P DMA Restoration Effort

To understand this message, one must appreciate the broader effort. The session's host machine has 8 NVIDIA RTX PRO 6000 Blackwell GPUs split across two NUMA nodes. Four GPUs (NUMA0) are dedicated to an SGLang inference server running the Qwen3.5-122B-A10B model with tensor parallelism across 4 GPUs. The other four (NUMA1) are assigned via VFIO to a confidential VM using SEV-SNP.

The challenge is that GPU P2P DMA — essential for efficient NCCL all-reduce operations during tensor-parallel inference — is broken under the default IOMMU configuration. The AMD IOMMU, when operating in full translation mode (DMA-FQ), intercepts and remaps all DMA transactions, which breaks the direct GPU-to-GPU memory access that P2P requires. The standard workaround (disabling IOMMU entirely) is unavailable because SEV-SNP requires amd_iommu=on.

The solution explored in previous messages was to use per-IOMMU-group identity domains: setting specific IOMMU groups to "identity" mode (passthrough) while leaving others in translation mode. This would allow the NUMA0 GPU groups to perform P2P DMA while the NUMA1 GPU groups remain in translation for the SEV-SNP VM. The assistant had set up a systemd service (gpu-iommu-identity.service) to configure these identity domains at boot, before the nvidia driver loaded.

The Subject Message: Discovery and Diagnosis

The message opens with the assistant analyzing the output of a diagnostic command run in [msg 6371]. The command checked the IOMMU group types and driver bindings for all 8 GPUs after a reboot requested by the user in [msg 6369].

The assistant's first observation is that the identity domains are correctly set:

The identity domains are set correctly (type=identity for all 4 NUMA0 GPU groups, DMA-FQ for VFIO groups).

This confirms that gpu-iommu-identity.service ran successfully and configured the IOMMU groups as intended. The kernel's IOMMU subsystem correctly reports identity for groups 42, 61, 28, and 10 (the NUMA0 GPUs), and DMA-FQ for groups 101, 117, and the others (the NUMA1 VFIO GPUs).

But then comes the critical observation:

But the NUMA0 GPUs are bound to vfio-pci instead of nvidia.

This is the crux of the problem. The identity domains are configured, but the GPUs are attached to the wrong driver. Without the nvidia driver, there are no GPUs for CUDA, no SGLang server, and no inference — the entire deployment is non-functional.

The assistant immediately identifies the root cause:

The gpu-vfio-split.service ran after gpu-iommu-identity.service and bound ALL GPUs with matching PCI ID to vfio-pci (because new_id matches all 10de:2bb5 devices).

This is a classic boot-order race condition. The system has two services:

  1. gpu-iommu-identity.service — unbinds NUMA0 GPUs from nvidia, sets identity domains
  2. gpu-vfio-split.service — registers the PCI vendor:device ID 10de:2bb5 with vfio-pci via new_id, causing all unbound GPUs with that ID to be grabbed The problem is that gpu-vfio-split.service runs after the identity service. When the identity service unbinds the NUMA0 GPUs from nvidia to set the IOMMU type, those GPUs become unbound. Then gpu-vfio-split.service registers the PCI ID with vfio-pci, and since the NUMA0 GPUs are now unbound and match the ID, vfio-pci claims them. The assistant's diagnosis is precise: "the vfio-split script grabbed these GPUs too." This shows an understanding of how PCI driver binding works — when a driver registers a new_id, the kernel's driver core attempts to bind all unbound devices matching that ID to the new driver.

The Reasoning Process: A Window Into Systems Debugging

What makes this message particularly valuable is the clarity of the reasoning. The assistant processes the diagnostic output in stages:

  1. Verify the positive: Identity domains are correctly set — the IOMMU configuration worked.
  2. Identify the negative: GPUs are on the wrong driver — the deployment is broken.
  3. Trace causality: The vfio-pci driver claimed the GPUs after they were unbound.
  4. Identify the mechanism: new_id matches all devices with the same PCI ID.
  5. Formulate the fix: Either change the boot order or update the identity script to rebind to nvidia after vfio-split runs. The assistant then takes concrete action by running a command to check the status of both services:
ssh root@10.1.2.6 'systemctl status gpu-iommu-identity.service --no-pager 2>&1; echo "==="; systemctl status gpu-vfio-split.service --no-pager 2>&1'

This command serves to confirm the boot order hypothesis by checking the service timestamps. The output shows gpu-iommu-identity.service started at 20:04:21 and was active. The assistant would compare this with the vfio-split service's start time to confirm the ordering.

Assumptions and Potential Pitfalls

The assistant makes several assumptions in this message:

  1. That the boot order is the sole problem: The assistant assumes that if the services ran in the correct order (identity after vfio-split), the NUMA0 GPUs would end up on the nvidia driver. This assumes that the nvidia driver can successfully claim the GPUs after they've been unbound from vfio-pci, which depends on the nvidia driver being loaded and willing to accept the devices.
  2. That unbinding and rebinding is safe for Blackwell GPUs: Earlier in the session ([msg 6368]), the assistant discovered that Blackwell GPUs have a Firmware Security Processor (FSP) that can become stuck after driver unbind/rebind cycles. The assistant is assuming that at boot time, before the FSP has been initialized by the nvidia driver, unbinding from vfio-pci and rebinding to nvidia will work — but this assumption would later prove incorrect.
  3. That the identity domains persist across driver rebinds: The assistant assumes that once an IOMMU group is set to identity, it stays in that mode even when the driver is changed. This is a reasonable assumption — the IOMMU group type is a property of the IOMMU hardware unit, not the driver — but it's worth verifying.
  4. That the fix is a boot-order issue rather than a fundamental incompatibility: The assistant treats this as a solvable engineering problem (fix the ordering), not as evidence of a deeper incompatibility between identity domains and Blackwell GPUs. This assumption would be challenged in subsequent messages when the Blackwell FSP boot failure with identity IOMMU is discovered.

Input Knowledge Required

To fully understand this message, one needs:

  1. IOMMU architecture: Understanding of DMA remapping, translation vs. identity domains, and how per-group IOMMU configuration works on AMD systems with amd_iommu=on.
  2. PCI device driver model: Knowledge of how Linux binds PCI devices to drivers, how new_id works to register a driver for a device, and how unbinding/rebinding works via sysfs.
  3. Systemd service ordering: Understanding of systemd dependencies (After=, Before=) and how service start order is determined.
  4. NVIDIA Blackwell GPU architecture: Awareness of the FSP (Firmware Security Processor) and its role in GPU initialization, and the implications for driver unbind/rebind cycles.
  5. The SGLang deployment context: Knowledge that the system uses tensor parallelism across 4 GPUs, requiring P2P DMA for efficient all-reduce operations, and that NCCL_P2P_DISABLE=1 is currently the fallback.
  6. VFIO and SEV-SNP: Understanding that VFIO is used to pass through GPUs to a confidential VM with SEV-SNP, which mandates amd_iommu=on.

Output Knowledge Created

This message creates several important outputs:

  1. A confirmed diagnosis: The boot-order race condition between the identity service and the vfio-split service is identified and understood.
  2. A documented problem: The message records that gpu-vfio-split.service using new_id will claim all unbound GPUs with matching PCI ID, which is critical knowledge for anyone maintaining this system.
  3. A clear path forward: The assistant identifies two possible fixes — reorder the services so identity runs after vfio-split, or update the identity script to rebind to nvidia after vfio-split has run. The next message ([msg 6373]) implements the second approach.
  4. Evidence of the system state: The service status output confirms that gpu-iommu-identity.service ran successfully and exited with status 0, which is useful for debugging.

The Broader Significance

This message represents a classic systems integration challenge: when multiple independent subsystems (IOMMU configuration, PCI driver binding, GPU firmware initialization, virtual machine passthrough) interact at boot time, the order of operations matters enormously. The assistant's ability to rapidly diagnose the issue from raw sysfs output demonstrates the value of understanding the Linux driver model at a deep level.

The message also highlights a tension in the design: the identity service needs to run before the nvidia driver loads (to set the IOMMU type before DMA mappings are created), but the vfio-split service also needs to run early to claim the NUMA1 GPUs. The solution of having the identity service run after vfio-split and then rebind the NUMA0 GPUs to nvidia is a pragmatic workaround, but it adds complexity and another opportunity for things to go wrong.

Conclusion

Message [msg 6372] is a concise but powerful example of diagnostic reasoning in a complex systems environment. In just a few sentences, the assistant processes raw diagnostic output, identifies a discrepancy, traces it to a root cause, and formulates a fix. The message demonstrates that effective debugging is not about guessing — it's about systematically ruling out possibilities until the causal chain becomes clear.

The boot-order bug diagnosed here would eventually be superseded by a more fundamental discovery: that Blackwell GPUs cannot initialize under identity IOMMU at all ([chunk 41.0]). But that doesn't diminish the value of this diagnostic step. Each layer of debugging peels back another abstraction, revealing deeper truths about the hardware-software interface. This message captures one such peel — a moment of clarity where a seemingly broken system state becomes perfectly understandable through the lens of boot ordering and PCI driver binding mechanics.