Reading the Source: How One Bash Command Unlocked the TP+EP Puzzle for MiniMax-M2.5 on 8x Blackwell GPUs

In the midst of an intense benchmarking session on an 8-GPU Blackwell machine, a single bash command stands as a quiet but pivotal moment of investigation. The message at index 2323 in this opencode conversation is deceptively simple: it is nothing more than an SSH command that reads a slice of a Python source file from a remote server. Yet this seemingly mundane action sits at the center of a complex debugging chain, where the assistant is trying to determine whether Expert Parallelism (EP) can bypass a fundamental tensor parallelism (TP) alignment constraint that had just crashed a previous model deployment.

The Context: A Crash and a Hypothesis

To understand why this message was written, we must step back a few messages. The assistant had been deploying the MiniMax-M2.5 model — a 230B-parameter Mixture-of-Experts (MoE) model with FP8 quantization — on a machine with 8 NVIDIA RTX PRO 6000 Blackwell GPUs. An earlier attempt to run the model with --tensor-parallel-size 8 had failed catastrophically with a ValueError:

The output_size of gate's and up's weight = 192 is not divisible by weight quantization block_n = 128.

The root cause was clear: the MoE intermediate size per expert is 1536. With TP=8, each GPU gets 1536 / 8 = 192 elements, but the FP8 block quantization uses blocks of size [128, 128], and 192 is not divisible by 128. TP=4 worked because 1536 / 4 = 384 = 3 × 128. The assistant had settled on TP=4 as the "correct" choice, leaving 4 GPUs idle.

Then the user asked a critical question in message 2303: "can we do tp/ep or tp6?" This question reframed the entire problem. The user was asking whether Expert Parallelism — a parallelism strategy that distributes experts across GPUs rather than sharding their weights — could unlock the use of all 8 GPUs. The assistant immediately recognized the significance: if EP is enabled, the MoE expert weights might not be tensor-sharded at all. Each EP rank could hold a complete set of experts, distributed across GPUs. The FP8 alignment constraint on intermediate_size / TP would then only apply to the attention and dense layers, not to the expert weights.

The Message Itself: Reading the Source

Message 2323 is the assistant's attempt to verify this hypothesis by reading the vLLM source code directly. The command is:

ssh root@10.1.230.174 "sed -n '950,1020p' /root/ml-env/lib/python3.12/site-packages/vllm/model_executor/layers/fused_moe/config.py"

This reads lines 950 through 1020 of the FusedMoEParallelConfig class in vLLM's fused MoE configuration module. The output reveals several properties of the configuration class:

def use_deepep_ll_kernels(self):
    return self.use_all2all_kernels and self.all2all_backend == "deepep_low_latency"

def use_fi_all2allv_kernels(self):
    return (
        self.use_all2all_kernels and self.all2all_backend == "flashinfer_all2allv"
    )

def use_batched_activation_format(self):
    return self.use_deepep_ll_kernels or self.use_pplx_kernels

def use_naive_all2all_kernels(self):
    return self.use...

These properties are all about selecting the communication backend for expert-parallel all-to-all operations. They reveal that vLLM supports multiple all2all strategies — DeepEP low-latency kernels, FlashInfer all2all, and a naive fallback. But critically, the output is incomplete — it cuts off at line 1020, and the key information about how tp_size and ep_size interact in the MoE layer is not in this particular slice. The assistant would need to continue reading further to find the documentation comment that finally answers the question.

The Reasoning Behind the Investigation

This message is a textbook example of hypothesis-driven source code exploration. The assistant had formed a specific hypothesis: "When EP is enabled, the MoE layer uses EP for expert distribution and TP=1 for tensor sharding of individual expert weights." To confirm this, it needed to understand how vLLM's FusedMoEParallelConfig sets up the parallelism configuration.

The investigation leading up to this message had already covered considerable ground. The assistant had:

  1. Checked for EP support in vLLM (msg 2304-2308), discovering the --enable-expert-parallel flag and confirming that EP infrastructure exists in the distributed utilities.
  2. Attempted TP=2 with EP (msg 2310-2311), which crashed with OOM because vLLM's EP group equals the TP group — so --tensor-parallel-size 2 --enable-expert-parallel only launched 2 workers, not 8, and the 230GB model couldn't fit in 2 × 96GB.
  3. Traced the FP8 alignment check (msg 2314-2317) to understand exactly where the 192 % 128 != 0 error originates. The check is in fp8_utils.py and modelopt.py, and it fires when tp_size > 1 and the sharded dimension isn't divisible by the block size.
  4. Hypothesized that with EP, the MoE tp_size might be 1 (msg 2318), meaning expert weights aren't TP-sharded at all — they're distributed whole across EP ranks.

Input Knowledge Required

To understand this message, the reader needs considerable background knowledge:

Output Knowledge Created

This message produced a small but meaningful piece of output: the property methods of FusedMoEParallelConfig that govern which all2all backend is used for expert communication. While this particular slice didn't directly answer the TP/EP question, it confirmed that the codebase has a rich EP infrastructure with multiple backend options.

The real value of this message is not in its output but in its role within the investigative process. It represents the moment when the assistant moved from speculation to source-level verification. Rather than guessing how vLLM handles TP+EP interaction, the assistant went directly to the code to find out.

Assumptions and Potential Mistakes

The assistant made several assumptions during this investigation:

  1. That EP would bypass the FP8 alignment check: This turned out to be correct, as confirmed in subsequent messages (msg 2324-2328) where TP=8+EP loaded successfully in ~90 seconds.
  2. That the FusedMoEParallelConfig class would contain the answer: The class does contain the parallelism configuration, but the key insight about TP_moe={1, 0} when EP is enabled was found in the documentation comments (msg 2324), not in the code properties.
  3. That reading lines 950-1020 would be sufficient: The output cut off mid-property, suggesting the assistant might have needed a wider range. Indeed, the critical documentation was at lines 1020-1090 (msg 2324).
  4. That EP group equals TP group in vLLM: This was confirmed by the OOM crash at msg 2311-2313, where --tensor-parallel-size 2 --enable-expert-parallel only launched 2 workers. The assistant initially hoped for a separate EP sizing mechanism but discovered that vLLM doesn't support that.

The Broader Significance

This message exemplifies a crucial pattern in ML engineering: when a framework's behavior is unclear, the source code is the ultimate documentation. The assistant could have continued guessing or trying random configurations, but instead it systematically read the relevant source files to understand the exact parallelism model.

The investigation that this message is part of ultimately succeeded: TP=8 with EP enabled loaded the MiniMax-M2.5 model across all 8 GPUs, achieving nearly 4,000 tok/s at high concurrency (as documented in the chunk summary for segment 18). The FP8 alignment issue was indeed bypassed by EP because, as the documentation revealed, TP_moe=1 when EP is enabled — expert weights are not tensor-sharded.

This single bash command — sed -n '950,1020p' on a remote config file — is a small but essential step in a larger journey of hardware-aware model selection, where the team systematically eliminated software artifacts to isolate genuine hardware limits on their 8-GPU Blackwell machine.