Segment 40

In this sub-session, the assistant reconfigured the Proxmox host's GPU topology to split the 8 RTX PRO 6000 Blackwell GPUs: 4 remained bound to nvidia for the LXC container (running SGLang), while the other 4 were moved to vfio-pci for SEV-SNP VM passthrough. The SGLang service was updated from TP=8 to TP=4, and the Qwen3.5-397B-A17B-NVFP4 model was replaced with Qwen3.5-122B-A10B BF16 on /shared. A critical issue emerged: SEV-SNP's full IOMMU translation broke GPU-to-GPU P2P DMA, causing NCCL hangs due to corrupted data (IO_PAGE_FAULTs). The assistant diagnosed this via dmesg and CUDA P2P tests, then fixed it by adding NCCL_P2P_DISABLE=1 to force SHM transport. A driver version mismatch (container userspace 565 vs host kernel 590) was resolved by installing matching 590 packages. The server then loaded successfully, achieving 108 tok/s single-stream and up to 2,800 tok/s at C=128. The sub-session concluded with investigating BIOS-level options (PCIe ACS, ARI, IOMMU coverage) to potentially re-enable P2P DMA under IOMMU translation.

Split 8 GPUs between LXC and VM with persistent bindingUpdate SGLang service to TP=4Replace Qwen3.5-397B NVFP4 with Qwen3.5-122B BF16Diagnose P2P DMA corruption under SEV-SNP IOMMUFix NCCL hang with NCCL_P2P_DISABLE=1Fix driver version mismatch between container and hostBenchmark Qwen3.5-122B BF16 on 4 GPUsInvestigate BIOS options for P2P DMA under IOMMU

The IOMMU Barrier: GPU P2P DMA Corruption Under SEV-SNP — A Full-Stack Debugging Saga 3935 words

Chunks