The Chicken-and-Egg of GPU Initialization: Debugging IOMMU Identity Domains on Blackwell

Introduction

In the complex world of high-performance AI inference infrastructure, the interaction between GPU firmware, PCI subsystem, IOMMU virtualization, and kernel drivers creates a delicate dance where timing and ordering are everything. Message [msg 6323] captures a pivotal moment in this dance — a raw, unfiltered reasoning session where an AI assistant wrestles with a seemingly intractable chicken-and-egg problem: how to configure IOMMU identity domains for NVIDIA Blackwell GPUs when the very act of loading the driver and resetting the hardware creates circular dependencies that block every approach.

This message is not about writing code or deploying a model. It is about debugging a hardware-software boundary failure — the kind of problem that separates engineers who can follow a recipe from those who can invent one. The assistant is deep in the weeds of PCI bus resets, IOMMU group types, GPU firmware security processors, and the precise ordering of kernel module loading. It is a masterclass in systematic reasoning under constraint.

The Message in Full

The assistant's message reads:

modprobe nvidia failed with "No such device" — because there are no PCI devices matching nvidia's device table when the module loads. nvidia refuses to load without any matching PCI devices.

>

So the correct order has to be: GPUs present in PCI (from rescan) + nvidia module loaded simultaneously. The working pattern was: nvidia module already loaded, then rescan adds devices which triggers nvidia's probe callback.

>

But the first time we recovered GPUs (successfully), nvidia was already loaded from the previous session. So: I need nvidia loaded, then do remove+SBR+rescan. Let me do that but try to set identity in between:

>

The problem is a chicken-and-egg: - nvidia needs PCI devices to load - We need nvidia loaded before rescan for clean probe - We can't set identity while nvidia is bound - We can't set identity on non-existent groups (after remove)

>

New idea: What if we keep nvidia loaded (from the previous successful probe), remove only 1 GPU at a time, SBR just that one bridge, rescan (nvidia reprobes that one GPU fresh), then set identity? No wait, we still can't set identity while nvidia is bound...

>

Better idea: What about having 3 GPUs on nvidia and 1 removed? Then we have nvidia loaded. We remove GPU, SBR, blacklist nvidia probe for that one device with driver_override, rescan, set identity, clear override, trigger probe?

>

Actually, let me try the driver_override approach. If I can write driver_override to the device sysfs right after rescan but before nvidia binds, I could use a udev rule or tight timing. But udev might be too slow.

>

Wait — there might be a much simpler solution. Let me check: can we set the default domain type for the AMD IOMMU so that NEW groups default to identity?

>

[bash command checking IOMMU group types and nvidia driver status]

Context: The Problem Space

To understand why this message matters, we need to understand the context leading up to it. The assistant has been working on a system with 8 NVIDIA RTX PRO 6000 Blackwell GPUs split across two NUMA domains. Four GPUs (on NUMA0) are assigned to an LXC container running SGLang for AI inference, while the other four (on NUMA1) are bound to vfio-pci for a SEV-SNP confidential VM.

The core challenge is P2P (peer-to-peer) DMA between GPUs. For multi-GPU inference with tensor parallelism, GPUs need to communicate directly via P2P over PCIe. However, the system has IOMMU enabled with SEV-SNP (Secure Encrypted Virtualization), and the default IOMMU domain type is DMA-FQ (DMA with Fine-Grained translation). Under DMA-FQ, P2P DMA between GPUs fails because the IOMMU translates peer addresses and the mappings are incomplete or incorrect.

The solution appears to be setting IOMMU groups to identity mode, which bypasses translation and allows direct P2P access. But as the assistant has discovered over the previous several messages, this is far from straightforward.

The Critical Discovery: nvidia Will Not Load Without PCI Devices

The immediate trigger for this message is a failed attempt. In the previous message ([msg 6322]), the assistant tried a carefully orchestrated sequence:

  1. Blacklist nvidia modules
  2. Unbind GPUs from nvidia
  3. Unload nvidia
  4. Set IOMMU groups to identity
  5. Remove GPUs from PCI
  6. Perform Secondary Bus Reset (SBR) on parent bridges
  7. Remove blacklist and load nvidia before rescanning PCI
  8. Rescan PCI Step 7 was supposed to be the clever trick: load nvidia while the GPUs are still absent from PCI, then rescan so that fresh GPUs appear and nvidia's probe callback handles them cleanly with identity domains already configured. But step 7 failed. modprobe nvidia returned "No such device." The nvidia kernel module refuses to load if there are no PCI devices in the system that match its device ID table. This is a fundamental design constraint: the module's initialization routine apparently checks for compatible hardware and aborts if none is found. This failure is the catalyst for the entire reasoning chain in this message. It forces the assistant to completely re-evaluate the approach.

The Chicken-and-Egg Analysis

The assistant then lays out the four constraints that form the circular dependency:

  1. nvidia needs PCI devices to load — The module refuses to initialize without matching hardware present.
  2. We need nvidia loaded before rescan for clean probe — Previous experiments showed that loading nvidia first, then rescanning, produces working GPUs. If nvidia isn't loaded, the GPUs get claimed by vfio-pci or remain driverless.
  3. We can't set identity while nvidia is bound — The IOMMU group type can only be changed when no driver is bound to any device in the group. Attempting to change it while nvidia controls the GPU fails silently or with an error.
  4. We can't set identity on non-existent groups — When GPUs are removed from PCI, their IOMMU groups are destroyed. There's nothing to configure. These four constraints form a logical square that seems impossible to satisfy simultaneously. The assistant needs: - GPUs present in PCI (to satisfy constraint 1) - But nvidia must not be bound to them (to satisfy constraint 3) - And the IOMMU groups must exist (to satisfy constraint 4) - And nvidia must be loaded before the GPUs appear (to satisfy constraint 2) The only window where all four conditions might be met is the brief instant after a PCI rescan creates new devices but before nvidia's probe callback runs. But that window is measured in microseconds — far too short for manual intervention.

Exploring Alternative Strategies

The assistant doesn't give up. It explores three alternative approaches in rapid succession, each building on the lessons of the previous failure.

Approach 1: One GPU at a Time

The first idea is to keep nvidia loaded (satisfying constraint 2) by leaving 3 GPUs bound, then remove just one GPU, perform SBR on its bridge, rescan, set identity, and repeat for each GPU. This avoids the "nvidia won't load without devices" problem because nvidia stays loaded the entire time.

But the assistant immediately spots the flaw: "we still can't set identity while nvidia is bound." The single GPU being worked on would be unbound during the remove/rescan cycle, but the IOMMU group for that GPU might be shared with other devices, or the act of setting identity might require the entire group to be unbound. And more fundamentally, the rescan would cause nvidia to re-probe the GPU, binding it again before identity can be set.

Approach 2: driver_override with Timing

The second idea is more creative. The assistant proposes using the driver_override sysfs attribute — a mechanism that forces a PCI device to be ignored by all drivers except the one specified. The plan would be:

  1. Keep nvidia loaded with 3 GPUs
  2. Remove one GPU, SBR, rescan
  3. Immediately write driver_override to the fresh GPU device to prevent nvidia from binding
  4. Set identity on the IOMMU group
  5. Clear the override
  6. Trigger nvidia probe manually The assistant correctly identifies the challenge: timing. The window between rescan creating the device and nvidia's probe callback binding it is extremely short. A udev rule might work, but "udev might be too slow." This is a real concern — udev is asynchronous and might not fire before the probe completes.

Approach 3: Default Domain Type

The third idea is the most elegant: instead of trying to set identity on individual IOMMU groups after they're created, can we change the default domain type for the AMD IOMMU hardware itself? If new groups default to identity, then the entire timing problem disappears — the rescan would create groups that are already in identity mode, and nvidia would probe into identity domains directly.

The assistant immediately checks this by running a bash command to inspect the IOMMU class devices (/sys/class/iommu/ivhd*) and the nvidia driver directory. The output shows multiple IOMMU instances (ivhd0 through ivhd7) and confirms that nvidia is not loaded (the driver directory doesn't exist).

Input Knowledge Required

To fully understand this message, one needs knowledge of:

Output Knowledge Created

This message creates several valuable insights:

  1. The nvidia module loading constraint is confirmed: The assistant now knows definitively that modprobe nvidia fails when no compatible PCI devices exist. This is a hard constraint that cannot be worked around.
  2. The working pattern is identified: The only sequence that produces working GPUs is: nvidia loaded → remove GPUs → SBR → rescan. The module must be in memory before the GPUs reappear.
  3. The chicken-and-egg formulation: The four constraints are explicitly articulated, providing a clear problem statement that can guide further investigation.
  4. Three alternative approaches are scoped: Each with identified risks and feasibility concerns. The driver_override approach is tentatively viable but timing-dependent. The default domain type approach is the most promising if it works.
  5. System state is verified: The bash command confirms that nvidia is not loaded, and reveals the IOMMU topology (8 ivhd instances, likely one per IOMMU group or NUMA node).

Assumptions and Potential Mistakes

The assistant makes several assumptions worth examining:

Assumption: Identity domains are the correct solution for P2P. The entire investigation assumes that setting IOMMU groups to identity will enable P2P DMA. While this is generally true, there may be other factors — the DmaRemapPeerMmio=1 nvidia driver parameter was already enabled but produced incomplete mappings. Identity mode might fix this, but it might also introduce security concerns or other issues.

Assumption: The FSP state corruption is permanent without full power cycle. The assistant has tried FLR (Function Level Reset), SBR (Secondary Bus Reset), and CXL bus reset — none cleared the FSP state. The assumption that only physical power-off can reset the FSP seems correct based on evidence, but there might be vendor-specific reset sequences not yet explored.

Assumption: udev would be too slow for driver_override. This is a reasonable engineering judgment, but it's untested. A udev rule with RUN+="/bin/echo ..." might actually fire before the probe completes, especially if nvidia's probe is deferred to a workqueue. The assistant doesn't test this, instead moving on to the default domain type approach.

Assumption: The default domain type can be changed. The assistant asks "can we set the default domain type for the AMD IOMMU?" but doesn't yet know if this is possible. The kernel's IOMMU subsystem may not expose this control, or it may be compile-time configurable only.

The Thinking Process

What makes this message remarkable is the visible thinking process. The assistant doesn't just report results — it thinks out loud, working through the problem step by step.

The reasoning follows a pattern familiar to experienced debuggers:

  1. Observe the failure: modprobe nvidia failed with "No such device."
  2. Generalize the constraint: nvidia refuses to load without matching PCI devices.
  3. Recall the working pattern: The only successful recovery used nvidia already loaded, then rescan.
  4. Formalize the constraints: Four conditions are listed in a clear bullet-point analysis.
  5. Generate alternatives: Three distinct approaches are brainstormed.
  6. Evaluate each: The assistant identifies the flaw in approach 1, the timing risk in approach 2, and the potential elegance of approach 3.
  7. Test the most promising: Approach 3 is immediately tested with a bash command. The language shifts from declarative ("The problem is a chicken-and-egg") to speculative ("What if we keep nvidia loaded...") to interrogative ("can we set the default domain type..."). This mirrors the cognitive shift from analysis to synthesis to verification. The assistant also demonstrates self-correction. The "New idea" is immediately followed by "No wait, we still can't set identity while nvidia is bound" — the assistant catches its own mistake in real-time. Similarly, the "Better idea" about driver_override is tempered with "But udev might be too slow" — an honest assessment of the approach's weakness.

Conclusion

Message [msg 6323] is a window into the messy reality of systems debugging at the hardware-software boundary. It shows that progress is not always linear — sometimes you try a carefully planned sequence, it fails, and you have to go back to first principles. The assistant's response to failure is not frustration but systematic analysis: articulate the constraints, brainstorm alternatives, evaluate trade-offs, and test the most promising path.

The message also reveals something important about the nature of AI-assisted debugging. The assistant is not just executing commands — it is reasoning, hypothesizing, and self-correcting in real-time. The chicken-and-egg problem it identifies is real and non-trivial. The solutions it proposes are creative and grounded in deep knowledge of the Linux kernel, PCI subsystem, and NVIDIA driver architecture.

Whether any of these approaches will ultimately succeed is a question answered in later messages. But this message stands alone as a masterful example of structured problem decomposition under tight constraints — the kind of thinking that separates effective debugging from trial-and-error.