The LL128 Hypothesis: A Methodical Search for NCCL Protocol Gains in Multi-GPU LLM Inference

Introduction

In the course of optimizing a 402GB GGUF-quantized GLM-5 model running across 8 NVIDIA RTX PRO 6000 Blackwell GPUs with vLLM, a critical performance inflection point was reached. After a long and largely fruitless exploration of NCCL tuning parameters—including NCCL_NTHREADS, NCCL_BUFFSIZE, NCCL_ALGO=Ring, and channel count adjustments—none of which budged single-request decode throughput beyond ~57.6 tokens per second, the assistant finally isolated a parameter that genuinely mattered: NCCL_PROTO=LL. Message 103 captures the next logical step in this investigation: testing the sibling protocol NCCL_PROTO=LL128 to see if it could squeeze further gains from the same mechanism.

This message is outwardly simple—a single bash command that launches a vLLM server with one environment variable changed—but it represents a pivotal moment in a carefully structured optimization campaign. Understanding why this particular command was issued, what assumptions it carries, and what knowledge it both requires and produces reveals the disciplined, hypothesis-driven methodology at work.

The Discovery That Changed Everything

To appreciate message 103, one must understand the discovery that immediately preceded it. Throughout messages 83–98, the assistant had been operating under an implicit but untested assumption: that the "baseline" performance of ~57 tok/s was already the default, and that NCCL tuning was failing to improve upon it. This assumption had led to a series of dead ends—trying Ring algorithm, reducing channel counts, adjusting buffer sizes—all of which returned the same ~57.6 tok/s ceiling.

The breakthrough came in message 99, when the assistant finally ran a true baseline: a vLLM server started with no NCCL environment variables at all. The result was ~47.3 tok/s. This was a revelation. The assistant had been running all its "tuning" experiments with NCCL_PROTO=LL already set (inherited from earlier in the session), and the ~57.6 tok/s figure was not the baseline—it was already a 22% improvement over the actual default. The assistant's own words in message 100 capture the moment: "Very important finding! Without NCCL_PROTO=LL, we get ~47.3 tok/s. With LL, we get ~57.6 tok/s. So NCCL_PROTO=LL gives a 22% improvement (47.3 → 57.6 tok/s). This is the most significant tuning parameter."

This reframing of the problem was essential. The assistant now understood that NCCL protocol selection was the dominant lever, not algorithm choice or channel count. And with that understanding came a natural follow-up question: if LL is good, could LL128 be better?

The Reasoning Behind LL128

Message 103 is the execution of that hypothesis. The assistant's reasoning, stated explicitly in message 100, was: "Now let me try NCCL_PROTO=LL128 which is another low-latency protocol that might be better for slightly larger messages."

This reasoning draws on knowledge of NCCL's internal architecture. The NCCL (NVIDIA Collective Communications Library) supports multiple protocols for performing allreduce operations across GPUs:

• Simple: The default protocol, which uses a straightforward buffer-based approach. It is reliable and general-purpose but has higher latency.
• LL (Low-Latency): Uses a flag byte mechanism to signal completion without kernel synchronization, reducing per-operation latency. Effective for small message sizes where latency dominates bandwidth.
• LL128: A variant of LL that operates on 128-byte chunks, designed to be more bandwidth-efficient than LL for messages that are somewhat larger than the smallest sizes. It still uses the flag-byte signaling trick but with a different data layout that can improve throughput for medium-sized messages. The assistant's hypothesis was grounded in the specific characteristics of the GLM-5 model's communication pattern. With 78 transformer layers and two allreduce operations per layer (one for the attention output projection, one for the MoE shared expert), each token decode triggers approximately 156 allreduce operations. Each of these allreduces involves a relatively small message—roughly 12KB per GPU, as the assistant had calculated in message 84. The question was whether LL128's different chunking strategy would better match this message size profile than LL's default approach.

The Mechanics of the Command

The command itself is a carefully constructed invocation:

ssh -o StrictHostKeyChecking=no root@10.1.230.174 'NCCL_P2P_LEVEL=SYS NCCL_PROTO=LL128 nohup /root/ml-env/bin/python3 -m vllm.entrypoints.openai.api_server \
  --model /shared/glm5-gguf/GLM-5-UD-Q4_K_XL.gguf \
  --tokenizer zai-org/GLM-5 --hf-config-path zai-org/GLM-5 \
  --tensor-parallel-size 8 --dtype float16 \
  --max-model-len 8192 --gpu-memory-utilization 0.90 \
  --trust-remote-code --port 8000 --disable-log-requests \
  > /tmp/vllm_ll128.log 2>&1 &
echo "Server PID: $!"'

Several design choices are visible here. First, NCCL_P2P_LEVEL=SYS is retained from the earlier successful configuration—this forces peer-to-peer communication through system memory (PCIe) rather than NVLink, which is appropriate for these GPUs since they lack direct NVLink connections across all 8 devices. Second, only NCCL_PROTO=LL128 is changed from the previous LL configuration; all other NCCL variables are left at their defaults to isolate the effect of the protocol change. Third, the server logs are redirected to a distinct file (/tmp/vllm_ll128.log) to keep results separable from previous runs. Fourth, the --disable-log-requests flag suppresses per-request logging to reduce noise.

The assistant also includes echo "Server PID: $!" to capture the process ID for later cleanup—a pragmatic detail born from earlier sessions where orphaned processes had to be hunted down with pkill and nvidia-smi queries.

Assumptions Embedded in the Test

Every experiment rests on assumptions, and this one is no exception. The assistant assumes that:

1. The server will start successfully. This is not guaranteed—different NCCL protocols can expose bugs in the communication backend, especially with the Triton MLA attention backend and GGUF dequantization kernels that this setup uses.
2. The LL128 protocol is available and functional. NCCL protocol availability depends on the NCCL version, the GPU architecture, and the interconnect topology. On Blackwell GPUs with PCIe gen5, LL128 should be supported, but edge cases exist.
3. The performance difference, if any, will be measurable above noise. The assistant has established that single-request decode throughput is stable within ~1 tok/s across trials (e.g., 57.6, 57.6, 56.4 tok/s in message 85). A meaningful improvement would need to exceed this noise floor.
4. The protocol change does not interact negatively with other system components. For instance, if LL128 changes the timing of allreduce operations, it could affect CUDA graph capture behavior, which vLLM uses for optimization. The assistant had noted in message 92 that CUDAGraphMode was running in PIECEWISE mode with 51 graph captures—any change to communication patterns could potentially invalidate cached graphs or trigger recaptures.
5. The cleanup from message 102 was complete. The assistant had killed the previous server process and verified that GPU memory was freed (all showing 0 MiB). If any residual processes or memory allocations remained, they could interfere with the new server's initialization.

Input Knowledge Required

To understand this message fully, one needs knowledge spanning several domains:

NCCL internals: Understanding what LL and LL128 protocols are, how they differ, and under what conditions each is preferable. This includes knowledge of flag-byte signaling, chunk sizes, and the trade-off between latency and bandwidth for different message sizes.

vLLM architecture: Knowing that vLLM uses tensor parallelism across GPUs, which requires allreduce operations after each transformer layer's attention and MLP computations. Also understanding that vLLM uses CUDA graph capture for optimization, which can be sensitive to changes in kernel launch patterns.

GPU interconnect topology: Understanding that 8 GPUs across two NUMA sockets communicate via PCIe rather than NVLink, and that NCCL_P2P_LEVEL=SYS forces system-memory-based communication appropriate for this topology.

GGUF quantization: Knowing that the model is stored in GGUF format with Q4_K quantization, which means weights are dequantized on-the-fly during inference. This affects the compute-to-communication ratio and therefore the sensitivity to allreduce latency.

The specific hardware: Two RTX PRO 6000 Blackwell GPUs per NUMA node (4 per socket, 8 total), each with ~97GB of usable memory, connected via PCIe gen5.

Output Knowledge Created

The immediate output of this message is a running vLLM server with the LL128 protocol. But the knowledge created extends beyond that:

1. A testable hypothesis: The assistant has committed to a specific prediction—that LL128 may outperform LL for the message sizes in this model. The next messages in the session will either confirm or refute this.
2. A controlled experimental setup: By isolating the protocol change as the only variable, the assistant ensures that any performance difference can be attributed to the protocol rather than confounding factors.
3. A baseline for comparison: The log file /tmp/vllm_ll128.log will contain startup information, including any protocol-related warnings or errors, that can be compared against the previous runs.
4. A reproducible procedure: The command structure is consistent with earlier tests (same model path, same vLLM arguments, same server configuration), making it easy to compare results across runs.

The Broader Optimization Context

This message sits within a larger optimization narrative that spans the entire session. The assistant had already:

• Resolved garbage output from the Triton MLA attention backend and GGUF dequantization shard ordering (segment 16)
• Optimized single-request decode throughput from ~20 to ~57 tok/s using CUDAGraph and NCCL tuning
• Demonstrated that aggregate throughput scales well with concurrent requests (144.4 tok/s with 4 concurrent requests)
• Discovered that NCCL_PROTO=LL was the most impactful single parameter The LL128 test represents the next frontier: having established that protocol choice matters, the assistant is now searching for the optimal protocol within the LL family. This is classic hyperparameter optimization—first identify the lever that matters, then fine-tune its setting.

Potential Pitfalls and What Comes Next

The assistant's reasoning, while sound, contains one potential blind spot: the assumption that LL128 is "better for slightly larger messages" may not hold for the specific message sizes in this model. The allreduce messages are approximately 12KB per GPU, which is quite small. LL is typically optimal for messages under a few hundred bytes to a few kilobytes, while LL128 starts to become competitive for messages in the tens-of-kilobytes range. The 12KB message size sits right at the boundary where either protocol could be superior, depending on implementation details and hardware characteristics.

Another subtle issue is that the assistant has not measured time-to-first-token (TTFT) alongside decode throughput. A protocol change that improves steady-state decode speed could potentially increase TTFT due to different initialization or warmup behavior. The assistant's benchmarks in earlier messages (e.g., message 99) measured total request time including both TTFT and decode, but did not separate them. If LL128 improves decode throughput at the cost of increased TTFT, the net effect on end-to-end latency might be neutral or negative.

The server has been launched, and the assistant will now wait for it to become ready (typically ~465 seconds based on previous startups) before running benchmark trials. The results of this experiment will either validate the LL128 hypothesis or send the assistant back to the drawing board—perhaps to explore other optimization avenues like model parallelism strategies, kernel fusion, or memory bandwidth optimization.

Conclusion

Message 103 is a study in disciplined experimental methodology. It captures the moment when a hypothesis—formed from a critical insight about NCCL protocol impact—is translated into a concrete, controlled test. The assistant's journey from dead-end NCCL tuning to the LL128 experiment demonstrates the importance of questioning assumptions, establishing true baselines, and systematically isolating variables. Whether LL128 proves superior or not, the experiment itself is well-designed: one variable changed, all else held constant, with clear success criteria and a reproducible procedure. In the high-stakes world of multi-GPU LLM inference optimization, where every percentage point of throughput improvement translates to significant cost savings, this kind of methodical investigation is the difference between guesswork and engineering.