The Architecture of Isolation: Why PCIe Topology Dooms GPU P2P in Virtualized Environments

In the high-stakes world of multi-GPU machine learning inference, every microsecond of cross-GPU communication latency matters. When deploying a model like GLM-5-NVFP4 across 8 NVIDIA RTX PRO 6000 Blackwell GPUs, the difference between a 5µs and a 13µs all-reduce operation can dramatically impact throughput and user experience. This is the story of a single message in an opencode coding session — message index 414 — where an AI assistant delivers a masterclass in PCIe topology, IOMMU architecture, and the fundamental hardware constraints that separate bare-metal performance from virtualized reality.

The message itself is a response to a simple, direct question from the user: "Is the root complex situation down to the actual CPU model? Or also motherboard?" What follows is a nuanced, multi-layered explanation that reveals how motherboard design, CPU architecture, and virtualization technology conspire to create a performance bottleneck that no amount of software tuning can fully overcome. This article examines that message in depth — its reasoning, its assumptions, its knowledge contributions, and the broader lessons it holds for anyone building GPU-intensive workloads in virtualized environments.

The Context: A Long Struggle for GPU P2P

To understand the significance of this message, we must first understand the journey that led to it. The conversation spans multiple segments of an opencode session focused on deploying the GLM-5-NVFP4 model on a Proxmox virtual machine with 8 NVIDIA RTX PRO 6000 Blackwell GPUs. Earlier segments had established the environment — installing NVIDIA drivers, resolving flash-attn build issues, and getting the SGLang inference server running. But a persistent performance problem emerged: cross-GPU communication was bottlenecked at approximately 13µs latency, far worse than the ~5-8µs expected for same-socket GPU pairs on bare metal.

The root cause was identified as a lack of Peer-to-Peer (P2P) DMA between GPUs inside the VM. The nvidia-smi topo -p2p command showed NS (Not Supported) instead of OK or NODE. The assistant hypothesized that disabling ACS (Access Control Services) in the BIOS might merge IOMMU groups and enable P2P. The user attempted this, rebooting the host and checking the results. The IOMMU groups did renumber, but critically, they did not merge — each GPU remained in its own separate IOMMU group. This led to the user's pivotal question: is this a CPU limitation or a motherboard limitation?

The Message: A Taxonomy of PCIe Topology

The assistant's response in message 414 is a model of technical explanation. It begins with a direct, unambiguous answer: "It's both, but primarily the motherboard/platform design." This opening establishes the thesis, then the message proceeds to build a complete picture of how PCIe topology works in modern server platforms.

The CPU Side: EPYC's Internal Architecture

The assistant explains that the AMD EPYC 9335 (Turin) processor contains multiple NBIO (NorthBridge I/O) units, each providing several PCIe root complexes. The CPU itself is capable of routing P2P traffic between its internal root complexes via the Data Fabric (Infinity Fabric). This is why, on bare metal, the NVIDIA driver reports NODE topology for same-socket GPU pairs — the Data Fabric provides a direct path. The CPU is not the limiting factor; it has the internal routing capability.

The Motherboard Side: Deliberate Isolation

The message then pivots to the motherboard, the ASUS ESC8000A-E13. This is where the real constraint lives. The motherboard is designed with an 8x single-GPU-per-root-complex layout. Each GPU slot connects to a dedicated PCIe Gen5 x16 link on a separate root complex. This is a deliberate engineering choice with clear trade-offs:

The VM Problem: IOMMU Groups as Isolation Boundaries

The assistant then explains why this topology is so problematic in a virtualized environment. In a VM with VFIO, each GPU resides in a separate IOMMU group because each is on a separate root complex. ACS (Access Control Services) is irrelevant here — there is no shared PCIe bridge between any two GPUs where ACS could enforce or relax isolation. The IOMMU groups exist because each root complex is inherently a separate IOMMU domain on AMD hardware.

When GPU A attempts to DMA directly to GPU B's BAR address, the host IOMMU has no mapping for this cross-group transaction. The IOMMU maintains separate address spaces per group, and VFIO would need explicit kernel support to allow cross-group P2P — support that does not exist for this topology. The 13µs latency is the cost of the IOMMU mediating every cross-GPU transfer through host memory.

Could a Different Motherboard Help?

The assistant addresses this question directly, explaining that a motherboard with a shared PCIe switch (like a PLX/Broadcom switch) would place multiple GPUs in the same IOMMU group, enabling direct PCIe P2P that would show as PIX or PXB inside the VM. However, such motherboards are rare for 8-GPU configurations. The ESC8000A design of one-GPU-per-root-complex is standard for maximizing per-GPU bandwidth in servers.

The Bottom Line: Practical Recommendations

The message concludes with a realistic assessment and actionable recommendations. The ~13µs latency is fundamentally limited by:

  1. Separate IOMMU groups (due to separate root complexes, not ACS)
  2. VFIO's inability to allow cross-group P2P for this topology The path forward includes: - BIOS power settings (DF C-states, APBDIS) — may shave 1-3µs - TP4+PP2 — halves the number of GPUs in each all-reduce - NCCL tuning — optimize for the SHM transport path - Bare metal — if maximum performance is needed without VM overhead

Reasoning and Decision-Making in the Message

The assistant's reasoning in this message is notable for its clarity and structure. The user's question — "Is the root complex situation down to the actual CPU model? Or also motherboard?" — could have been answered with a simple "both, but mostly motherboard." Instead, the assistant builds a complete explanatory framework.

The reasoning process visible in the message follows a clear pattern:

  1. Direct answer: Establish the thesis upfront
  2. Decompose the problem: Separate CPU-side capabilities from motherboard-side design
  3. Provide comparative context: The table of platform designs is a powerful reasoning tool that lets the reader understand why their specific configuration behaves the way it does
  4. Trace the VM implications: Connect the hardware topology to the IOMMU behavior and the observed performance
  5. Address the hypothetical: Answer whether a different motherboard would help
  6. Synthesize and recommend: Give practical next steps The assistant also demonstrates a sophisticated understanding of the interaction between hardware and virtualization. The key insight — that ACS disable didn't merge IOMMU groups because there's no shared bridge between GPUs — is explained clearly and concisely.

Assumptions and Potential Limitations

The message makes several assumptions worth examining:

Assumption 1: The motherboard design is the primary constraint. The assistant states this clearly, and the evidence supports it. However, it's worth noting that the CPU's root complex topology also plays a role. On some platforms, multiple root complexes might be grouped into the same IOMMU domain by the BIOS or firmware. The assistant acknowledges this nuance by saying it's "both."

Assumption 2: VFIO cannot support cross-group P2P for this topology. This is stated as a fact, and it's generally correct for standard KVM/VFIO configurations. However, there are experimental or insecure kernel parameters (like vfio_iommu_type1.allow_unsafe_interrupts or nv_peer_mem) that might enable cross-group P2P. The assistant doesn't explore these in this message, though they were discussed in later chunks of the session. This isn't a mistake per se — the message focuses on the fundamental hardware topology — but it's a limitation of the scope.

Assumption 3: The comparison table covers the relevant design space. The four rows of the table represent common configurations, but there are others. For example, some platforms use PCIe switches that connect to multiple root complexes, creating more complex topologies. The table is a simplification, but a useful one.

Assumption 4: Bare metal would solve the problem. The assistant states that bare metal would give NODE P2P at ~5-8µs. This is likely true for same-socket GPU pairs, but cross-socket pairs on bare metal would still traverse the Infinity Fabric and might show higher latency. The message doesn't fully explore this distinction.

Input Knowledge Required to Understand This Message

To fully grasp this message, the reader needs:

  1. Understanding of PCIe topology concepts: Root complexes, root ports, PCIe switches, bifurcation, link speeds
  2. Knowledge of IOMMU and VFIO: How IOMMU groups work, how VFIO passthrough creates isolation domains
  3. Familiarity with NVIDIA GPU topology reporting: What NODE, SYS, PIX, PXB, NV# mean in nvidia-smi topo output
  4. Context from the conversation: The earlier ACS disable attempt, the IOMMU group renumbering, the VM boot failure
  5. Understanding of AMD EPYC architecture: NBIO units, Data Fabric, Infinity Fabric
  6. Knowledge of ML inference communication patterns: All-reduce, tensor parallelism, NCCL

Output Knowledge Created by This Message

The message creates significant knowledge value:

  1. A clear taxonomy of GPU interconnect topologies: The comparison table is a reusable reference for anyone designing multi-GPU systems
  2. A diagnostic framework: The explanation of how motherboard design maps to IOMMU groups and P2P capability provides a methodology for debugging similar issues
  3. A realistic assessment of virtualization limitations: The message makes clear that some hardware constraints are fundamental and cannot be overcome by software configuration
  4. Actionable recommendations: The four paths forward (BIOS settings, TP4+PP2, NCCL tuning, bare metal) give the user concrete options
  5. A conceptual model: The explanation of why ACS disable didn't work — because there's no shared bridge between GPUs — provides a deeper understanding of PCIe isolation mechanisms

The Thinking Process: What Makes This Message Effective

The assistant's thinking process, visible in the structure and content of the message, reveals several strengths:

Hierarchical decomposition: The answer is structured as a tree — first the high-level answer, then CPU side, then motherboard side, then the comparison, then the VM implications, then the hypothetical, then the bottom line. Each section builds on the previous one.

Concrete examples: The comparison table is brilliant because it takes an abstract concept (root complex topology) and makes it concrete by showing what different designs look like and what P2P status they produce.

Honest about limitations: The message doesn't promise solutions. It explains why the problem exists and what can realistically be done. This builds trust.

Bridges theory and practice: The explanation connects the hardware theory (root complexes, IOMMU groups) to the practical observation (13µs latency, NS in P2P matrix) to the user's experience (ACS disable didn't help).

Broader Lessons for Virtualized GPU Workloads

This message, while focused on a specific hardware configuration, teaches broader lessons:

  1. Virtualization always adds overhead for tightly-coupled workloads: The IOMMU isolation that makes VMs secure also prevents direct hardware P2P. This is a fundamental trade-off.
  2. Motherboard design is often the binding constraint: Many performance issues attributed to "the CPU" or "the hypervisor" are actually determined by the PCIe topology baked into the motherboard.
  3. ACS is not a magic bullet: Disabling ACS only helps when GPUs share a PCIe bridge. When each GPU is on its own root complex, ACS has nothing to do.
  4. The comparison table framework is reusable: Understanding where your platform falls in the taxonomy of GPU interconnect designs helps predict performance characteristics and debug issues.
  5. Sometimes the answer is "you can't fix this in software": The most valuable technical insight is often recognizing a fundamental hardware limitation and adjusting expectations accordingly.

Conclusion

Message 414 of this opencode session is a masterful technical explanation that transforms a simple question about CPU versus motherboard into a comprehensive lesson in PCIe topology, IOMMU architecture, and the realities of virtualized GPU computing. The assistant's reasoning is clear, structured, and grounded in both theory and practical observation. The comparison table provides a reusable framework for understanding GPU interconnect designs. The honest assessment of limitations — and the actionable recommendations that follow — demonstrate a deep understanding of the problem space.

For anyone building multi-GPU inference systems in virtualized environments, this message offers both a diagnostic methodology and a dose of reality: some performance constraints are baked into the hardware, and the path forward involves either accepting those constraints, optimizing around them, or eliminating the virtualization layer entirely. The 13µs latency that prompted this investigation may never fully disappear in this VM, but understanding why it exists is the first step toward making informed decisions about how to work with — or around — it.