Segment 4
This sub-session focused on implementing an LXC container approach to overcome the VFIO/IOMMU P2P bottleneck that limited GPU communication in the KVM VM. The user installed the NVIDIA driver (590.48.01) on the Proxmox host, converted an existing unprivileged LXC container to privileged, and configured bind-mounts for all 8 GPU device nodes. Inside the container, nvidia-smi topo -m confirmed the true bare-metal topology (NODE within sockets, SYS across sockets), indicating P2P DMA should theoretically work. The model cache was also copied from the VM's ZFS zvol to a shared dataset and bind-mounted to avoid re-download. However, a critical blocker emerged: CUDA runtime initialization (cuInit) fails with error code 3 on both the host and container, despite nvidia-smi detecting all 8 GPUs. Investigation revealed that the open-source kernel module sees the Blackwell GPUs but CUDA cannot initialize, while the proprietary module makes GPUs invisible. The root cause appears to be a driver compatibility issue with the Proxmox VE kernel (6.8.12-9-pve) — the driver lacks Blackwell GSP firmware files, and the older PVE kernel may not support Blackwell's GSP requirements. This is a fundamental difference from the KVM VM, where the guest's own driver stack handled VFIO-passed GPUs without host-level GSP firmware dependencies. The session concluded with the assistant investigating kernel version and firmware limitations, leaving the LXC approach promising for P2P but blocked by host driver initialization issues.
The LXC Pivot: When Bare-Metal GPU Topology Met the Blackwell GSP Firmware Wall