The Custom Allreduce Barrier: Why PCIe-Only GPUs Can't Use vLLM's Fastest Communication

In the high-stakes world of large language model inference, every millisecond counts. When serving a 402-billion-parameter model like GLM-5 across eight GPUs, the communication overhead between devices can become a dominant bottleneck. In message [msg 131] of a lengthy optimization session, an AI assistant makes a crucial discovery: vLLM's custom allreduce implementation—a key optimization for multi-GPU communication—is silently disabled for their setup, and the reason lies in a fundamental hardware limitation that no amount of software tuning can overcome.

The Discovery

The message opens with a moment of clarity: "Found it." This simple declaration caps a long chain of investigation that began with throughput plateauing at approximately 57.5 tokens per second. The assistant had already exhausted the NCCL tuning space, experimenting with different protocols (LL, LL128, Simple), shared memory optimizations (SHM_CUDA_MEMCPY, SHM_MEMCPY_ASYNC), and P2P levels (SYS, PIX). None of these moved the needle beyond the 57.5 tok/s ceiling. Something fundamental was limiting performance.

The assistant had noticed earlier, in [msg 127], that fuse_allreduce_rms was False and all fusion passes were disabled. But the deeper question was why. The answer emerges in this message: the custom allreduce—vLLM's optimized replacement for NCCL allreduce—is disabled for their specific hardware configuration.

The Root Cause: A Simple Check with Far-Reaching Consequences

The assistant traces the issue to a specific conditional check in vLLM's source code:

if world_size > 2 and not fully_connected

This seemingly innocuous line determines whether the custom allreduce path is enabled. The fully_connected variable is the result of is_fully_connected(), a function that checks whether the GPUs are interconnected via NVLink—NVIDIA's high-speed GPU-to-GPU interconnect. NVLink provides direct, low-latency communication channels between GPUs, bypassing the PCIe bus entirely.

The assistant's setup uses eight RTX PRO 6000 Blackwell GPUs connected via PCIe. These are powerful workstation GPUs with impressive HBM3e memory bandwidth (~1.5 TB/s per GPU), but they lack NVLink bridges. The is_fully_connected function returns False for PCIe-only configurations, and with world_size=8 (eight GPUs), the condition world_size > 2 and not fully_connected evaluates to True, causing the custom allreduce to be disabled.

Why Custom Allreduce Matters

The assistant's reasoning about why this matters is sharp and well-informed. vLLM's custom allreduce uses shared GPU memory for peer-to-peer (P2P) communication, which provides much lower latency than NCCL for small tensors. In transformer inference, the allreduce operations happen after each attention layer and each MoE layer—that's two allreduces per layer, times 78 layers, totaling 156 allreduce operations per token. At 57.5 tokens per second, that's nearly 9,000 allreduce calls per second.

For small tensors (the typical output of attention and MoE layers), NCCL's overhead—protocol negotiation, buffer management, kernel launches—can dominate the communication time. Custom allreduce, by using direct P2P shared memory access, can reduce this overhead significantly. The assistant estimates that each layer takes approximately 223 microseconds (17.4ms per token / 78 layers), and a significant fraction of that is likely communication.

The Hardware Reality Check

The assistant then executes a bash command to examine the is_fully_connected implementation in vLLM's CUDA platform code:

grep -n "is_fully_connected\|nvlink\|NVLink" /root/ml-env/lib/python3.12/site-packages/vllm/platforms/cuda.py | head -10

The output reveals three implementations of is_fully_connected at different lines (160, 600, 664), each with progressively more detailed NVLink detection. The third implementation at line 664 includes a telling message: "NVLink detection not possible, as con..." — suggesting that on some platforms, NVLink detection simply isn't available.

This exploration reveals an important assumption the assistant had been operating under: that vLLM would automatically use the best available communication backend. The reality is more nuanced. vLLM's custom allreduce is designed primarily for NVLink-connected clusters, and its P2P shared memory approach doesn't scale well across PCIe-only GPUs beyond two devices. The code at line 156-157 of the custom allreduce module explicitly warns about this: "more than two PCIe-only GPUs. To silence this warning, specify disable_custom_allreduce=True explicitly."

The Broader Context: A Session of Incremental Gains

To fully appreciate this message, one must understand the journey that led to it. The session began with setting up an ML environment on Ubuntu 24.04, installing NVIDIA drivers and CUDA Toolkit, and resolving complex build issues with flash-attention. The assistant then deployed the GLM-5-NVFP4 model using vLLM and began a systematic optimization campaign.

Earlier messages show the assistant testing NCCL protocols ([msg 116]), measuring throughput ([msg 118]), estimating memory bandwidth utilization ([msg 119]), and benchmarking HBM bandwidth ([msg 120]). The assistant correctly identified that the system was not memory bandwidth bound (only ~14% utilization), ruling out the simplest explanation for the throughput ceiling.

The discovery in this message represents a pivot point. The assistant had been searching for software-level optimizations—NCCL flags, CUDA graph tuning, flashinfer autotuning—but the real bottleneck is architectural. The custom allreduce is disabled not because of a configuration error or missing library, but because the hardware topology doesn't support it.

Assumptions and Their Consequences

This message reveals several assumptions that shaped the investigation:

Assumption 1: vLLM automatically selects optimal communication. The assistant assumed that vLLM's default configuration would pick the best available communication backend. In reality, vLLM's custom allreduce is gated behind NVLink detection, and when it's disabled, the fallback to NCCL is silent—no error, no warning visible in the logs unless you know where to look.

Assumption 2: PCIe performance is sufficient for 8-GPU inference. The assistant's earlier bandwidth calculations showed that PCIe bandwidth (approximately 32 GB/s per Gen4 x16 lane) should be adequate for the per-token communication volume. But the latency of NCCL allreduce, not the bandwidth, appears to be the limiting factor. Custom allreduce's advantage is in lower latency for small messages, which is exactly what transformer inference requires.

Assumption 3: The throughput ceiling was a software problem. The assistant had been methodically testing NCCL configurations, CUDA graph capture sizes, and fusion passes. The discovery that custom allreduce is fundamentally unavailable shifts the investigation from "which software flag to set" to "what can be done about a hardware limitation."

Input Knowledge Required

To fully understand this message, several pieces of context are necessary:

  1. The vLLM architecture: Understanding that vLLM uses tensor parallelism across GPUs, requiring allreduce operations after each layer to synchronize hidden states.
  2. NCCL vs custom allreduce: NCCL (NVIDIA Collective Communications Library) is the standard multi-GPU communication library. vLLM's custom allreduce is an alternative implementation using direct GPU memory access for lower latency.
  3. NVLink vs PCIe: NVLink provides direct GPU-to-GPU connections with much higher bandwidth and lower latency than PCIe. The RTX PRO 6000 Blackwell cards in this setup lack NVLink, connecting only via PCIe.
  4. The world_size concept: In tensor parallelism, world_size is the number of GPUs participating. With 8 GPUs, world_size=8.
  5. The optimization history: The preceding messages show the assistant systematically testing NCCL protocols, measuring throughput, and ruling out memory bandwidth as the bottleneck.

Output Knowledge Created

This message creates several valuable pieces of knowledge:

  1. A clear diagnosis: The throughput ceiling is caused by the custom allreduce being disabled for PCIe-only GPUs with world_size > 2.
  2. A documented code path: The assistant traces the exact conditional check (world_size > 2 and not fully_connected) and locates the is_fully_connected function in vLLM's source.
  3. A hardware limitation identified: The root cause is the lack of NVLink between the GPUs, not a software bug or configuration error.
  4. A direction for further investigation: With custom allreduce unavailable, the assistant must explore other avenues—perhaps reducing the number of allreduce operations through fusion, or accepting the NCCL overhead and optimizing elsewhere.

The Thinking Process

The message reveals a disciplined, methodical thought process. The assistant starts with a discovery ("Found it"), then immediately explains the mechanism (the world_size > 2 and not fully_connected check). The reasoning about why custom allreduce is disabled for PCIe-only GPUs is concise but accurate: "the P2P shared buffer approach doesn't scale well without NVLink."

The assistant then decides to verify the is_fully_connected implementation, executing a grep command to find the relevant code. This is characteristic of the assistant's approach throughout the session: when a hypothesis emerges, it's immediately tested against the actual source code rather than accepted at face value.

The decision to check the NVLink detection code specifically shows strategic thinking. Rather than trying to patch the custom allreduce to work on PCIe (which would likely fail due to fundamental hardware limitations), the assistant seeks to understand why the check exists. The output confirms that NVLink detection is a platform-level concern, with multiple fallback implementations depending on hardware capabilities.

Implications and Next Steps

This discovery has significant implications for the optimization effort. With custom allreduce unavailable, the assistant must either:

  1. Accept the NCCL overhead and optimize around it
  2. Explore allreduce fusion (combining multiple allreduce operations into one)
  3. Investigate whether the allreduce can be eliminated entirely for certain operations
  4. Consider hardware changes (adding NVLink bridges, though these GPUs may not support them) The message ends without a conclusion—the assistant is still in investigation mode. The grep output shows the is_fully_connected function signatures but doesn't reveal their full implementation. The assistant would need to read more of the source code to understand whether NVLink detection can be bypassed or whether the limitation is truly fundamental. In the broader narrative of this optimization session, message [msg 131] represents the moment when the assistant realizes that software tuning has reached its limits and hardware topology is now the binding constraint. It's a humbling discovery after dozens of experiments, but also a valuable one—knowing the true bottleneck prevents wasted effort on futile optimizations and redirects attention to what might actually make a difference.