Reading the Blueprint: How a Proxmox PCI Mapping Query Set the Stage for an 8-GPU Blackwell Split

The Message

In a single, deceptively simple bash command, the assistant reached across the network to the Proxmox hypervisor and read the contents of /etc/pve/mapping/pci.cfg, then queried the cluster-level PCI mapping API via pvesh:

[assistant] [bash] ssh root@10.1.2.6 'cat /etc/pve/mapping/pci.cfg 2>/dev/null; echo "---"; pvesh get /cluster/mapping/pci 2>/dev/null'
nv3080
	map id=10de:2204,iommugroup=72,node=kpro4,path=0000:43:00,subsystem-id=1043:87d5

nv3090
	map id=10de:2204,iommugroup=20,node=kpro5,path=0000:53:00,subsystem-id=1043:87d5

nv3090-2
	map id=10de:2204,iommugroup=24,node=kpro5,path=0000:57:00,subsystem-id=1043:87d5

nv3090-3
	map id=10de:2204,iommugroup=50,node=kpro5,path=0000:d6:00,subsystem-id=1043:87d5

nva6000
	map id=10de:2230,iommugroup=16,node=kpro5,path=0000:4f:00,subsystem-id=10de:1459

nva6000-2
	map id=10de:2230,iommugroup=43,nod...

This message (global index 6039) appears at first glance to be nothing more than a routine information-gathering step — a reconnaissance command in a long sequence of infrastructure work. But this single query was the pivotal moment where the assistant transitioned from knowing about the hardware topology to being able to act on it. It is the bridge between understanding and execution, and it reveals a great deal about how the assistant approaches complex systems administration tasks.

The Context: Why This Message Was Written

To understand why this message exists, we must trace back to the user's instruction in [msg 6034]. The user, satisfied with benchmark results from the current configuration, issued a new directive: "I now want to reconfigure the proxmox host such that the lxc container only has 4 GPUs and the other 4 are available for another VM / VMs through passthrough (nvidia driver to 4 on one numa, other 4 to vfio)."

This was a significant topology change. The machine in question — a Proxmox VE host running on a dual-socket AMD EPYC system — had eight NVIDIA RTX PRO 6000 Blackwell GPUs installed. Until this point, all eight were bound to the NVIDIA driver and passed through to a single LXC container running SGLang for LLM inference. The user now wanted to split them: four GPUs (on NUMA node 0) would remain with the LXC container, while the other four (on NUMA node 1) would be detached from the NVIDIA driver and reassigned to the vfio-pci driver for direct passthrough to a separate VM — likely one configured with AMD SEV-SNP for confidential computing workloads.

The assistant had already gathered the foundational data. In [msg 6036], it discovered the NUMA topology: PCI devices 01:00.0, 11:00.0, 61:00.0, and 71:00.0 were on NUMA node 0, while 81:00.0, 91:00.0, e1:00.0, and f1:00.0 were on NUMA node 1. In [msg 6037], it read the LXC container configuration (/etc/pve/lxc/129.conf), which revealed the current GPU bind-mount setup — each GPU device file (/dev/nvidia0 through /dev/nvidia7) was individually bind-mounted into the container. In [msg 6038], it examined an existing VM configuration (/etc/pve/qm/131.conf) that used PCI mappings named nv3090-2 and nv3090-3 for GPU passthrough.

But there was a critical gap. The assistant knew where the GPUs were and how they were currently configured, but it did not yet know what PCI mapping infrastructure existed on this Proxmox host. Proxmox uses a PCI mapping system — defined in /etc/pve/mapping/pci.cfg — to create named aliases for physical PCI devices. These named mappings are then referenced in VM configuration files (e.g., hostpci0: mapping=nv3090-2). Without understanding the existing mappings, the assistant could not create new ones for the Blackwell GPUs. Message 6039 was the reconnaissance step that filled this gap.

The Thinking Process: What the Assistant Needed to Learn

The assistant's reasoning at this point can be reconstructed from the sequence of actions. It had already confirmed the NUMA topology, the LXC config, and the VM config. The natural next question was: What PCI mappings already exist, and what naming conventions are in use? The assistant needed to know:

1. Whether mappings already existed for the RTX PRO 6000 Blackwell GPUs (device ID 10de:2bb5). If they did, the task would be simpler — just assign them. If not, new mappings would need to be created.
2. The naming conventions used in existing mappings. The existing VM config referenced nv3090-2 and nv3090-3, suggesting a pattern of [model]-[index]. For the Blackwell GPUs, something like pro6000-0 through pro6000-3 would be consistent.
3. The IOMMU group assignments. PCI passthrough in Proxmox requires that all devices in an IOMMU group be passed through together. The mapping file records the IOMMU group for each device, which is essential for determining whether GPUs can be individually assigned or must be grouped.
4. The node assignments. The existing mappings were associated with specific Proxmox cluster nodes (kpro4, kpro5). The assistant needed to know which node this host was, to ensure new mappings were correctly attributed. The output revealed that the existing mappings were for older NVIDIA GPUs — RTX 3080 (10de:2204) and RTX A6000 (10de:2230) — on different cluster nodes (kpro4 and kpro5). There were no mappings yet for the RTX PRO 6000 Blackwell GPUs (10de:2bb5). This confirmed that new mappings would need to be created from scratch.

Input Knowledge Required

To fully understand this message, one needs knowledge of several domains:

• Proxmox VE PCI passthrough: Understanding that /etc/pve/mapping/pci.cfg is the configuration file for PCI device mappings, which are used to assign physical PCI devices to VMs. The format includes the device ID (id=), IOMMU group (iommugroup=), cluster node (node=), PCI bus address (path=), and subsystem ID (subsystem-id=).
• NVIDIA GPU device IDs: Recognizing that 10de:2204 is an RTX 3090, 10de:2230 is an RTX A6000, and 10de:2bb5 (from earlier lspci output) is the RTX PRO 6000 Blackwell. This allows the reader to distinguish between existing mappings (for older GPUs on other nodes) and the need for new ones.
• IOMMU groups and PCI topology: Understanding that devices in the same IOMMU group must be passed through together, which affects how GPUs can be split between containers and VMs.
• The cluster context: Knowing that this Proxmox host is part of a cluster (evidenced by the cluster-wide mapping API queried via pvesh), and that PCI mappings are cluster-aware resources.

Output Knowledge Created

This message produced several pieces of critical knowledge:

1. The complete inventory of existing PCI mappings on this Proxmox cluster, including mappings for RTX 3080 and RTX A6000 GPUs on nodes kpro4 and kpro5.
2. Confirmation that no mappings exist for the Blackwell GPUs, meaning the assistant would need to create them.
3. The exact format and conventions used in the mapping file, which the assistant would replicate when creating the new pro6000-vm mapping for the four GPUs destined for VM passthrough.
4. The cluster-aware nature of the mapping system, evidenced by the pvesh get /cluster/mapping/pci query, which showed that mappings are cluster-level resources, not local to a single node. This knowledge directly enabled the subsequent actions in the chunk: creating the pro6000-vm PCI mapping, setting up the gpu-vfio-split.service systemd unit to persist the driver binding across reboots, and updating the LXC configuration to only bind-mount the four NUMA-0 GPUs.

Assumptions and Potential Mistakes

The assistant made several assumptions in this message:

• That the mapping file is the authoritative source: By reading /etc/pve/mapping/pci.cfg directly and also querying the API, the assistant implicitly assumed these two sources would be consistent. In a clustered Proxmox environment, the API is the authoritative source, and the file may be a local cache or representation. The dual query was a prudent cross-check.
• That the existing mappings are relevant to the current task: The assistant read mappings for GPUs on different cluster nodes (kpro4, kpro5). These were not directly relevant to the Blackwell GPUs on the current host, but they established the naming pattern and format.
• That no mappings existed for the Blackwell GPUs: The truncated output shows only the beginning of the nva6000-2 mapping. The assistant assumed the rest of the file contained no Blackwell entries — a reasonable assumption given the device IDs visible, but one that relied on the output being complete enough to make that judgment. One subtle issue: the output is truncated (note the trailing "nod..." at the end of the nva6000-2 line). The assistant was working with incomplete data. In a production environment, this truncation could have hidden a crucial detail — perhaps a Blackwell mapping already existed further down in the file. The assistant implicitly trusted that the most relevant information was at the top, which was a reasonable heuristic but not guaranteed.

Conclusion

Message 6039 is a textbook example of a reconnaissance step in systems administration. It is not flashy — it does not change any configuration, deploy any software, or produce any visible result for the end user. But it is the essential precondition for all the work that follows. Without understanding the existing PCI mapping infrastructure, the assistant could not have created the new mappings, split the GPUs, or deployed the VM. This message represents the moment when the assistant moved from knowing what needed to be done to knowing how to do it — and that transition is the heart of effective infrastructure engineering.