The Dead End That Forced Invention: A Dry-Run That Changed the Course of GLM-5 Deployment

Introduction

In the long and arduous journey of deploying the GLM-5 model on Blackwell GPUs, there comes a moment that appears, at first glance, to be a trivial failure: a package dependency resolution that returns "unsatisfiable." Yet this single message — a uv pip install --dry-run command executed against the vLLM nightly wheel index — represents a critical juncture where the entire trajectory of the project pivots. The message, indexed as <msg id=1721>, is deceptively simple:

[assistant] [bash] ssh root@10.1.230.174 '~/.local/bin/uv pip install --python ~/ml-env/bin/python3 --index-url https://wheels.vllm.ai/nightly --extra-index-url https://pypi.org/simple --prerelease=allow --index-strategy unsafe-best-match "vllm>=0.16" --dry-run 2>&1 | head -40'
Using Python 3.12.3 environment at: ml-env
  × No solution found when resolving dependencies:
  ╰─▶ Because only vllm<=0.16.0rc2.dev314+gf72061a19 is available and
      you require vllm>=0.16, we can conclude that your requirements are
      unsatisfiable.

What makes this message so significant is not what it achieved — it achieved nothing, returning a dependency error — but what it revealed. The dry-run told the assistant and the user that the latest vLLM nightly wheel was only one commit ahead of their current installation (0.16.0rc2.dev313+g662205d34 vs 0.16.0rc2.dev314+gf72061a19). The Blackwell SM120 attention backend support they desperately needed simply did not exist in any publicly available wheel. This discovery forced a radical strategic shift: instead of waiting for upstream support, they would have to write their own attention backend from scratch.

The Crisis That Prompted the Query

To understand why this dry-run was executed, we must step back into the immediate context. The assistant had been working for hours to deploy the GLM-5 model in GGUF format using vLLM on a machine with 8 RTX PRO 6000 Blackwell GPUs. After successfully patching the GGUF loader and weight utilities, the first real vllm serve launch attempt ([msg 1696]) crashed with a devastating error: no valid attention backend could be found.

The error message listed five MLA (Multi-head Latent Attention) backends and why each failed:

  1. FLASH_ATTN_MLA: "compute capability not supported" and "sparse not supported"
  2. FLASHMLA: "compute capability not supported" and "sparse not supported"
  3. FLASHINFER_MLA: "compute capability not supported" and "qk_nope_head_dim == 128 required, got 192"
  4. TRITON_MLA: "sparse not supported"
  5. FLASHMLA_SPARSE: "dtype not supported" and "compute capability not supported" The core problem was a triple constraint that no existing backend satisfied: the Blackwell SM120 GPU (compute capability 12.0), the sparse MLA attention required by GLM-5's DSA (Dynamic Sparse Attention) indexer, and the non-standard qk_nope_head_dim=192 (DeepSeek models use 128). The GLM-5 architecture, based on the DeepSeek V3 lineage but with significant modifications, fell into a gap in vLLM's attention backend support matrix. The user suggested ([msg 1711]) updating to the latest nightly or master branch, hoping that Blackwell support might have been added in a more recent commit. This was a reasonable suggestion — the installed nightly was from a specific build, and perhaps a newer build would include SM120 support.

The Investigation Preceding the Dry-Run

Before executing the dry-run in the subject message, the assistant conducted a thorough investigation across messages 1696-1720. This investigation revealed several critical pieces of information:

First, the assistant examined the attention backend selector logic ([msg 1698]) and discovered that the use_sparse flag was triggered by the presence of an index_topk attribute in the model config — GLM-5 has this attribute because it uses the DSA indexer. This meant sparse attention was non-negotiable without modifying the model code.

Second, the assistant discovered that TritonMLAImpl.supports_compute_capability returns True for all compute capabilities ([msg 1702]), making it the only backend that would work on SM120. However, it failed because it didn't declare is_sparse() = True — the attention selector checks use_sparse != cls.is_sparse() and rejects backends that don't match.

Third, the assistant checked the sparse backends. FlashMLASparseBackend only supported torch.bfloat16 (not float16 which was needed for GGUF), and its supports_compute_capability only covered major versions 9 and 10, not 12 ([msg 1709]).

Fourth, the assistant checked the installed vLLM version and found it was 0.16.0rc2.dev313+g662205d34 ([msg 1712]), and that the nightly wheel index only had one version ahead: 0.16.0rc2.dev314+gf72061a19 ([msg 1719]). The dry-run in the subject message was the final confirmation that no newer version existed.

The Dry-Run Itself: What It Revealed

The command in the subject message is carefully constructed. It uses uv pip install with several flags:

× No solution found when resolving dependencies:
╰─▶ Because only vllm<=0.16.0rc2.dev314+gf72061a19 is available and
    you require vllm>=0.16, we can conclude that your requirements are
    unsatisfiable.

This is a fascinating error. The resolver says "only vllm<=0.16.0rc2.dev314+gf72061a19 is available" — meaning the nightly index only has versions up to that specific build. But 0.16.0rc2.dev314 is a pre-release version, and PEP 440 version comparison treats 0.16.0rc2 as less than 0.16. So vllm&gt;=0.16 doesn't match 0.16.0rc2.* because the release candidate is considered older than the final release.

This is a subtle version comparison issue. The assistant might have expected &gt;=0.16 to match pre-release versions of 0.16, but PEP 440 specifies that pre-releases sort before the final release. So 0.16.0rc2 &lt; 0.16, and the constraint &gt;=0.16 excludes all pre-releases.

But the deeper truth revealed by this dry-run is more important: there was no newer version available at all. Even if the version constraint were fixed to &gt;=0.16.0rc2.dev314, the resulting version would be functionally identical to what was already installed (one commit ahead). The Blackwell SM120 support simply hadn't been merged into the nightly wheels.

The Strategic Pivot

This dry-run result was the moment the assistant and user accepted that upstream vLLM would not save them. The Blackwell SM120 attention backend support they needed was not coming from a simple package update. They would have to build it themselves.

The chunk summary confirms this pivot: "After confirming that the latest vLLM nightly lacked this support, the assistant and user chose to patch the existing Triton MLA backend to handle sparse attention." The assistant went on to design and implement a TritonMLASparseBackend that reused the existing Triton decode kernel by treating the physical sparse indices as a virtual block table.

This is a beautiful example of engineering under constraint. Rather than waiting for upstream development or attempting to build a full attention backend from scratch, the assistant identified that the Triton MLA backend already supported SM120 (its supports_compute_capability returned True for all capabilities). The only missing piece was sparse attention support. By creating a thin wrapper that declared is_sparse() = True while delegating the actual computation to the existing Triton kernel, the assistant produced a working solution in hours rather than weeks.

Assumptions and Their Consequences

The subject message reveals several assumptions, some of which proved incorrect:

Assumption 1: A newer nightly wheel might exist with SM120 support. This was the core assumption driving the dry-run. It was reasonable — Blackwell GPUs had been on the market for months, and vLLM is actively developed. However, the assumption proved false. The nightly wheel index only had builds from the very latest commit, and SM120 MLA support hadn't been merged yet.

Assumption 2: The &gt;=0.16 version constraint would match pre-release versions. This was a technical error in the version specification. PEP 440 ordering means 0.16.0rc2.dev314 &lt; 0.16, so &gt;=0.16 excludes all 0.16 pre-releases. A more accurate constraint would have been &gt;=0.16.0rc2.dev313 or simply omitting the version constraint entirely (which would pick the latest available). However, this versioning mistake didn't affect the outcome — even with a corrected constraint, the latest available version was only one commit ahead.

Assumption 3: The nightly wheel index would have the most recent development builds. This was correct — the index did have the latest build. The problem was that the latest build simply didn't include SM120 MLA support.

Input Knowledge Required

To fully understand this message, one needs knowledge of:

  1. The GLM-5 model architecture: Specifically that it uses Multi-head Latent Attention (MLA) with a DSA (Dynamic Sparse Attention) indexer, qk_nope_head_dim=192, and is based on the DeepSeek V3 lineage.
  2. Blackwell GPU architecture: The RTX PRO 6000 Blackwell GPUs have compute capability 12.0 (SM120), which is newer than the Hopper (SM90) and Ada (SM89) architectures that most existing attention backends target.
  3. vLLM's attention backend architecture: The attention selector logic, the is_sparse() / use_sparse matching mechanism, and the supports_compute_capability class method pattern.
  4. PEP 440 version comparison rules: Understanding why 0.16.0rc2.dev314 doesn't satisfy &gt;=0.16.
  5. The uv package manager: The --dry-run, --index-strategy unsafe-best-match, and --prerelease=allow flags.
  6. The GGUF deployment context: The assistant had already patched gguf_loader.py and weight_utils.py to support the glm_moe_dsa architecture, and had fixed a speculators_config crash and a dtype incompatibility.

Output Knowledge Created

The dry-run produced two critical pieces of knowledge:

  1. Negative knowledge: The latest vLLM nightly wheel (0.16.0rc2.dev314+gf72061a19) is only one commit ahead of the installed version and does not include SM120 MLA sparse attention support. This eliminated the "wait for upstream" strategy.
  2. Strategic direction: Since no existing wheel would solve the problem, the team would need to either build vLLM from source with custom patches or write a new attention backend. The assistant chose the latter path, creating TritonMLASparseBackend. This output knowledge directly shaped the subsequent messages in the conversation, where the assistant analyzed the sparse MLA architecture, designed the new backend, implemented it, registered it in the attention registry, and deployed it to the container.

The Thinking Process

The reasoning visible in the preceding messages shows a systematic diagnostic approach. The assistant:

  1. Identified the error (no valid attention backend)
  2. Traced the use_sparse flag to the is_v32 / index_topk attribute in the model config
  3. Checked each backend's supports_compute_capability and validate_configuration
  4. Identified Triton MLA as the only backend supporting SM120
  5. Checked the installed vLLM version and the nightly index for newer versions
  6. Executed the dry-run to confirm no newer version existed
  7. Pivoted to a custom backend implementation The dry-run in the subject message is the culmination of step 6 — the final confirmation that external help wasn't coming. It's the moment where the team accepts full ownership of the problem.

Conclusion

The uv pip install --dry-run in &lt;msg id=1721&gt; is a masterclass in negative knowledge discovery. It's a message that appears to fail but actually succeeds brilliantly — not at installing a package, but at revealing the boundaries of what's possible with existing tools. By confirming that no upstream solution existed, it forced the creative leap that produced the TritonMLASparseBackend, a custom attention backend that ultimately allowed the GLM-5 model to run on Blackwell GPUs.

In engineering, the most valuable discoveries are often the ones that tell you what won't work. This dry-run, returning "unsatisfiable," was exactly such a discovery. It closed one door and opened another — the door to building your own solution.