The Research Interlude: Searching for a P2P DMA Workaround in KVM

In the long and winding journey of deploying a 744-billion-parameter GLM-5-NVFP4 model across eight NVIDIA RTX PRO 6000 Blackwell GPUs, there comes a moment that, on its surface, appears unremarkable. Message 421 in the conversation is nothing more than three web search queries fired off by an AI assistant. No commands are executed. No configuration files are edited. No servers are launched. Yet this message represents a critical inflection point — the moment when the assistant, having exhausted every conventional configuration path, turns to the research frontier to find a way around a fundamental hardware limitation.

The message is brief, so let us quote it exactly:

[assistant] [exa_web_search_exa] {"numResults":10,"query":"VFIO KVM GPU P2P DMA across IOMMU groups workaround pcie_acs_override Proxmox"} Title: VFIO tips and tricks: IOMMU Groups, inside and out Author: Posted by Alex Williamson Published Date: 2026-02-16T00:00:00.000Z URL: https://vfio.blogspot.com/2014/08/iommu-groups-inside-and-out.html Text: VFIO tips and tricks: IOMMU Groups, inside and out [exa_web_search_exa] {"numResults":8,"query":"IOMMUFD VFIO device P2P DMA support QEMU KVM multi-GPU passthrough"} Title: IOMMUFD BACKEND usage with VFIO — QEMU 10.2.50 documentation URL: https://qemu.readthedocs.io/en/master/devel/vfio-iommufd.html Text: IOMMUFD BACKEND usage with VFIO [exa_web_search_exa] {"numResults":8,"query":"NCCL P2P not supported KVM virtual machine GPU passthrough workaround 2024 2025"} Title: Quick Fix: Using NCCL with multi-vGPU VMware VMs URL: https://en.vmik.net/2025/11/vmw-run-nccl-on-vgpus/ Text: Quick Fix: Using NCCL with multi-vGPU VMware VMs

Three searches. Three distinct angles on the same problem. To understand why this message exists at all, we must understand the wall the assistant had just run into.

The Problem: Why P2P DMA Matters and Why It Failed

The assistant had been working for hours — across multiple conversation segments — to deploy the GLM-5-NVFP4 model, a massive Mixture-of-Experts transformer that requires all eight GPUs to cooperate via tensor parallelism. In tensor parallelism, the model is sharded across GPUs, and every single token requires collective communication operations (all-reduce, all-gather) that move data between GPUs. The efficiency of these operations depends critically on peer-to-peer (P2P) DMA — the ability for one GPU to directly read from or write to another GPU's memory over the PCIe bus, without staging through host memory.

In a bare-metal machine, the eight Blackwell GPUs (each on its own PCIe root complex, connected through AMD's Data Fabric) show a topology of NODE within the same socket and SYS across sockets. This means P2P DMA works, albeit with different bandwidth characteristics depending on whether GPUs share a CPU socket. But inside the KVM virtual machine, the topology shows PHB (PCIe Host Bridge) for all cross-GPU connections, and NS (Not Supported) for P2P. The VFIO driver that mediates device access in KVM cannot permit DMA transactions between devices in different IOMMU groups — and each GPU, being on its own physical root complex, belongs to its own IOMMU group. This is a security boundary that VFIO enforces by design.

The performance impact is devastating. Small GPU-to-GPU transfers (12KB) show a latency of 13.4 microseconds — an eternity in GPU computing — and bandwidth is capped at less than 1 GB/s. For the all-reduce operations that GLM-5 requires at every transformer layer, this bottleneck limits total throughput to roughly 485 tokens per second, far below the user's target of 1,000+ tokens per second.## The Context: What Led to This Message

To appreciate why the assistant issued these three searches, we must trace the path that led to this moment. The conversation up to this point had been a multi-session saga of environment setup, driver installation, CUDA toolkit configuration, flash-attn compilation wrestling, and ultimately, successful model deployment — only to discover that the VM's GPU communication was crippled.

The user had just asked, in message 415: "Are there hacky/insecure ways to get SYS/NODE in kvm?" This was a pointed question. The user was explicitly asking for workarounds that bypass security boundaries. The assistant's initial response (message 416) had begun researching this, searching for VFIO parameters like allow_unsafe_interrupts and allow_unsafe_p2p, and exploring the nv_peer_mem driver. But that research was interrupted — the conversation moved on to other tasks.

Now, in message 420, the assistant had returned to the problem with a clear plan. It created a todo list with three high-priority items: research P2P workarounds, fix the Proxmox resource mapping (which had broken after a BIOS change), and then proceed to post-reboot verification and benchmarking. The assistant explicitly stated: "Since I can't SSH to the Proxmox host directly — you run those commands manually — let me first check what research I can do independently, and then give you the host commands to run."

This is a crucial detail. The assistant is blocked from acting on the host. It cannot fix the stale IOMMU group mapping itself. So it uses the time productively — researching the deeper architectural question that will determine whether the VM approach is even viable for high-performance inference.

The Three Search Queries: A Strategic Triangulation

The three searches in message 421 are not random. They represent three distinct strategies for solving the P2P problem, each targeting a different layer of the software stack.

Search 1: "VFIO KVM GPU P2P DMA across IOMMU groups workaround pcie_acs_override Proxmox"

This search targets the VFIO layer directly. The pcie_acs_override kernel parameter is a well-known hack that forces PCIe Access Control Services (ACS) to be disabled, which can merge IOMMU groups by making the kernel treat devices behind a PCIe switch as if they were in the same group. The assistant already knew from earlier investigation that this wouldn't work here — each GPU is on its own root complex, not behind a shared switch — but it was worth confirming. The search returned Alex Williamson's canonical blog post on IOMMU groups, the definitive reference for understanding VFIO's group topology. Williamson is the maintainer of the VFIO subsystem, so his blog is the authoritative source. The search essentially confirms what the assistant already suspected: IOMMU groups are determined by physical PCIe topology, and ACS override only helps when devices share a PCIe bridge.

Search 2: "IOMMUFD VFIO device P2P DMA support QEMU KVM multi-GPU passthrough"

This search targets a newer, alternative backend for VFIO. IOMMUFD is a more modern kernel interface that replaces the legacy VFIO type1 IOMMU implementation. It was designed with a device-centric (rather than group-centric) model, which could theoretically allow different P2P semantics. The search returned the QEMU documentation for IOMMUFD backend usage. This was a promising avenue: if IOMMUFD handles DMA mappings differently, it might permit cross-device P2P even when devices are in different IOMMU groups. The assistant was probing whether a software upgrade (newer QEMU, newer kernel, different VFIO backend) could solve the problem without hardware changes.

Search 3: "NCCL P2P not supported KVM virtual machine GPU passthrough workaround 2024 2025"

This search takes a completely different approach. Instead of trying to fix P2P at the VFIO/KVM level, it asks: can we work around the problem at the application layer? NCCL (NVIDIA Collective Communications Library) is the library that sglang uses for multi-GPU communication. If NCCL can be configured to use alternative transports — like NVLink (not available here), SHM (shared memory), or even TCP — then P2P DMA might not be strictly necessary. The search returned a blog post about running NCCL with multi-vGPU VMware VMs, which is exactly the kind of workaround the assistant was looking for: how to make NCCL function (perhaps with reduced performance) when P2P is unavailable.## Assumptions and Knowledge Boundaries

The assistant's research in message 421 operates under several implicit assumptions. First, it assumes that the Proxmox host kernel (6.8.12-9-pve) is recent enough to potentially support IOMMUFD — or at least that the research is worth doing even if it turns out the kernel is too old. Second, it assumes that the user is willing to apply kernel-level or QEMU-level changes, given the user's explicit willingness to disable security features. Third, it assumes that the fundamental hardware topology (eight separate root complexes) is immutable — that no amount of software trickery can change the physical fact that each GPU has a dedicated x16 link to its own PCIe root port.

This last assumption is critical and correct. The AMD EPYC 9335 (Turin) architecture in this server provides each GPU with its own PCIe root complex. This is actually a good thing for bandwidth — each GPU gets a dedicated Gen5 x16 link without sharing — but it is disastrous for VFIO's IOMMU group model. There is no PCIe switch to ACS-override, no bridge to merge. The hardware topology is fundamentally at odds with VFIO's security model, and no amount of kernel parameter tuning can change that.

One could argue that the assistant made a subtle mistake in its prioritization. The three searches are exploratory, but they don't include a search for the most practical workaround: using an LXC container instead of a KVM VM. The LXC approach bypasses VFIO entirely by running on the host kernel, which sees the true PCIe topology. In fact, this is exactly what the assistant pivoted to in the subsequent messages of this chunk (messages 422+), where it guided the user through setting up an LXC container with GPU bind-mounts. The research in message 421, while valuable, ultimately led to a dead end for the KVM approach — and the real solution lay in a completely different virtualization technology.

Input Knowledge Required

To fully understand message 421, a reader needs substantial background knowledge. One must understand what IOMMU groups are and why VFIO enforces group boundaries — that DMA from a passed-through device could corrupt host memory if not properly isolated, and that the IOMMU group represents the smallest set of devices that the hardware can isolate together. One must understand PCIe topology: root complexes, switches, endpoints, and how ACS (Access Control Services) governs P2P transactions. One must understand the NCCL communication patterns used in tensor-parallel inference — that every transformer layer requires all-reduce operations across all GPUs, and that P2P DMA is the fastest transport for these operations. And one must understand the Proxmox/QEMU/KVM stack: how hostpci passthrough works, what the VFIO driver does, and how IOMMUFD differs from the legacy VFIO backend.

The assistant also draws on knowledge accumulated earlier in the conversation: the physical GPU topology (eight separate root complexes, two NUMA sockets), the BIOS settings that had been changed (ACS disabled, which didn't help), the measured P2P bandwidth and latency numbers, and the user's explicit request for hacky/insecure approaches.

Output Knowledge Created

Message 421 produces three pieces of output knowledge, each from a search result. The first is a confirmation: Alex Williamson's IOMMU groups blog post reinforces that IOMMU group topology is determined by hardware and cannot be overridden by software parameters like pcie_acs_override when devices are on separate root complexes. The second is a new lead: the IOMMUFD backend documentation suggests that a newer QEMU and kernel might handle device DMA differently, potentially relaxing the cross-group restriction. The third is a pragmatic workaround: the NCCL VMware blog post shows that it is possible to make NCCL function in virtualized environments without P2P, albeit with performance implications.

None of these results are immediately actionable. The assistant cannot upgrade QEMU or the kernel on the Proxmox host (it has no SSH access). The NCCL workaround would still leave performance far below targets. But the research establishes the landscape of possible solutions, which informs the next decision: abandoning the KVM approach for LXC containers.

The Thinking Process: A Window into Strategic Research

What makes message 421 fascinating is what it reveals about the assistant's reasoning strategy. Faced with a blocking problem and unable to act directly on the host, the assistant chooses to research in parallel. The three searches are dispatched simultaneously (all three exa_web_search_exa calls appear in the same message), covering three different angles. This is not random browsing — it is structured, hypothesis-driven research.

The first search tests the hypothesis "can we merge IOMMU groups via ACS override?" The second tests "does a newer VFIO backend (IOMMUFD) handle P2P differently?" The third tests "can we work around missing P2P at the NCCL level?" These three hypotheses span the entire stack: hardware topology configuration, hypervisor device mediation, and application-level communication library. The assistant is systematically ruling out or validating each layer.

This multi-pronged approach is characteristic of expert debugging. When a problem has no obvious solution, you don't dig deeper in one direction — you survey the entire solution space to find the path of least resistance. The assistant's research in message 421 is a textbook example of this strategy.

The Broader Significance

Message 421 is, in a sense, the turning point of this segment of the conversation. The research conducted here — particularly the confirmation that IOMMU groups cannot be merged for separate root complexes — leads directly to the decision to try LXC containers. That decision, in turn, reveals a new blocker (CUDA initialization failure on the host kernel) but also confirms that the LXC approach shows the correct bare-metal topology. The message is the pivot point between two eras of the conversation: the KVM era, where P2P was fundamentally broken, and the LXC era, where P2P is theoretically possible but blocked by driver compatibility issues.

For a reader unfamiliar with the broader conversation, message 421 might seem like a trivial interlude — just three web searches. But in the context of the full debugging journey, it represents the moment when the assistant stepped back from configuration tweaking and asked the fundamental question: Is this approach even viable? The answer, which emerged from these searches and the subsequent investigation, was no — and that realization sent the entire project in a new direction.