The 1809 Unmapped Tensors: A False Lead in the GLM-5 GGUF Debugging Saga

Introduction

In the long and arduous journey of deploying the GLM-5 model on eight Blackwell GPUs using GGUF quantization and vLLM, few moments are as dramatic—and as instructive—as the one captured in message 1917. After hours of patching vLLM's GGUF loader, fixing weight dequantization errors, and finally getting the model to serve requests, the assistant was confronted with a new and baffling problem: the model generated coherent-looking tokens, but the output was complete garbage. Log probabilities for obvious continuations (like predicting "2" after "1 2 3 4 5 6 7 8 9 10") were around -20, indicating the model's hidden states were essentially random noise.

The assistant had been systematically ruling out possible causes. The GGUF dequantization kernel was verified correct on the SM120 Blackwell architecture. The weight name mapping appeared to be correct. FlashAttention was available. The focus had shifted to the kv_b_proj weight loading, where a tensor parallelism sharding mismatch was suspected. But then came a discovery that seemed to blow the investigation wide open.

The Message

In message 1917, the assistant writes:

ALL 1809 tensors are unmapped!! The gguf-py get_tensor_name_map returns None for everything. This means the auto-mapping is broken for glm-dsa architecture. The only mappings that work are our manual overrides (kv_b splits, expert weights, etc.).

>

This is a huge problem — most weights (attention Q/K/V projections, layer norms, embeddings, dense FFN weights) are NOT being mapped from GGUF to HF names. They're being loaded through a different path that must be working since the model loads without errors...

>

Let me check how the GGUF loader actually creates the name map:

The message then includes a read tool call targeting the patched gguf_loader.py file, specifically lines 217-225 which show how the architecture key is resolved and the name map is created.

The Reasoning Path That Led Here

To understand why this moment felt like a breakthrough, we need to trace the assistant's reasoning over the preceding messages.

In message 1915, the assistant was investigating how expert weights are mapped for the MoE layers. It noted that the manual override in the patched GGUF loader maps ffn_gate_exps.weight → experts.0.gate_proj.weight for layers 3-77 (the MoE layers), but layers 0-2 are dense (non-MoE). The assistant assumed that the auto name map from gguf-py's get_tensor_name_map would handle the dense layers automatically. To verify this, it ran a command that checked the gguf-py name map for glm-dsa architecture.

The output in message 1915 showed mappings like:

blk.0.attn_k_b -> (<MODEL_TENSOR.ATTN_K_B: 124>, 'blk.0.attn_k_b')
blk.0.attn_kv_a_mqa -> (<MODEL_TENSOR.ATTN_KV_A_MQA: 122>, 'blk.0.attn_kv_a_mqa')

But crucially, these are GGUF tensor type ID mappings, not HF name mappings. The assistant didn't catch this distinction yet.

In message 1916, the assistant ran a more definitive test. It iterated over all 1809 tensors in the GGUF file and called nm.get_name(t.name) for each one. The result was devastating: Mapped: 0, Unmapped: 1809. Every single tensor returned None. The output showed:

blk.0.attn_k_b.weight -> None
blk.0.attn_kv_a_mqa.weight -> None
blk.0.attn_kv_a_norm.weight -> None
blk.0.attn_norm.weight -> None
blk.0.attn_output.weight -> None
blk.0.attn_q_a.weight -> None
...

This was the trigger for message 1917. The assistant concluded that the auto-mapping was fundamentally broken for the glm-dsa architecture.

The Critical Mistake: Wrong Direction

The assistant's conclusion in message 1917 was based on a reasonable but incorrect assumption. The get_tensor_name_map in gguf-py creates a bidirectional mapping, but the primary direction is HF name → GGUF name, not GGUF name → HF name. The test in message 1916 was calling nm.get_name(gguf_name) — passing a GGUF tensor name and expecting an HF name back. But the map was designed to be queried in the opposite direction: given an HF parameter name, find the corresponding GGUF tensor name.

This is a subtle but critical distinction. The GGUF loader in vLLM doesn't iterate over GGUF tensors and look up their HF names. Instead, it iterates over the HF model's parameters (obtained by creating a dummy transformers model), and for each one, calls text_name_map.get_name(base_name) to find the GGUF tensor name. The reverse mapping is then built from these lookups.

The assistant's test was essentially asking the map "what HF name corresponds to blk.0.attn_k_b.weight?" — but the map only knows the answer to "what GGUF name corresponds to model.layers.0.self_attn.kv_b_proj?" The two questions are not symmetric, and the map only stores one direction natively.

This mistake is particularly understandable given the context. The assistant had been deep in the weeds of manual name mapping overrides, where it had been writing explicit GGUF→HF mappings like:

gguf_to_hf_name_map[f"blk.{idx}.ffn_down_exps.weight"] = (
    f"model.layers.{idx}.mlp.experts.0.down_proj.weight"
)

These manual overrides are GGUF→HF mappings, so it was natural to assume the auto map worked the same way. But the auto map from gguf-py works in the opposite direction, and the GGUF loader uses it indirectly (HF→GGUF first, then builds the reverse map).

Assumptions and Their Consequences

The assistant made several assumptions in this message:

  1. That get_tensor_name_map returns a GGUF→HF map. This was the root error. The function actually returns a map that can be queried in both directions, but the primary stored direction is HF→GGUF.
  2. That the model loads without errors despite broken mapping. The assistant noted "they're being loaded through a different path that must be working since the model loads without errors." This was a reasonable inference — if the mapping were truly broken, the model should have failed to load or produced shape mismatches. The fact that it loaded silently suggested either the mapping worked through a different mechanism, or the weights were being loaded through a fallback path.
  3. That the garbage output was caused by incorrect weight mapping. This was the natural conclusion: if weights aren't mapped correctly, they can't be loaded, so the model would use random initialization values, producing garbage output. The assistant was about to pivot the entire investigation toward fixing the name mapping.

Input Knowledge Required

To understand this message, the reader needs to know:

Output Knowledge Created

Despite being based on a false premise, this message created valuable knowledge:

  1. Confirmation that the auto-mapping doesn't support GGUF→HF queries directly. This led the assistant to investigate the actual flow in the GGUF loader (reading lines 217-225), which in turn led to the realization in message 1919 that the map works in the HF→GGUF direction.
  2. A clearer picture of the GGUF loader's architecture. By reading the patched loader code, the assistant traced the exact flow: resolve architecture key, get name map, create dummy model, iterate HF parameters, look up GGUF names, build reverse mapping.
  3. Documentation of the tensor count. The discovery that there are exactly 1809 tensors in the GGUF file became a useful reference point for subsequent debugging.

The Aftermath and Resolution

The resolution came quickly. In message 1919, the assistant tested the mapping in the correct direction:

model.layers.0.input_layernorm -> blk.0.attn_norm
model.layers.0.self_attn.q_a_proj -> blk.0.attn_q_a
model.layers.0.self_attn.kv_a_proj_with_mqa -> blk.0.attn_kv_a_mqa
...

The mapping worked perfectly. The assistant immediately recognized its error: "My earlier test was calling nm.get_name(gguf_name) which is the wrong direction!"

This moment of false alarm, while ultimately a dead end, was not wasted effort. The assistant's willingness to question fundamental assumptions — even ones that had been implicitly trusted — is a hallmark of effective debugging. The investigation then pivoted back to the actual cause of the garbage output, which turned out to be a tensor parallelism sharding mismatch in the kv_b_proj weight loading.

Broader Lessons

This episode illustrates several important principles of debugging complex systems:

  1. Test your assumptions about API direction. When working with unfamiliar libraries, always verify the input/output contract of functions before drawing conclusions from their return values.
  2. A model that loads without errors doesn't mean weights are correct. The assistant correctly noted that the model loaded silently despite the supposed mapping failure. This contradiction should have been a stronger signal that the premise was wrong.
  3. False leads are part of the process. The 1809 unmapped tensors moment, while incorrect, led to a deeper understanding of the GGUF loader's internals. In debugging, even wrong turns can produce useful knowledge.
  4. The importance of bidirectional testing. A single test in one direction gave a catastrophic result. Testing the opposite direction immediately resolved the discrepancy. Whenever possible, test mappings in both directions. The 1809 unmapped tensors stand as a monument to the perils of assuming API directionality — and a reminder that in the fog of a complex debugging session, even the most startling discovery deserves a second look before redirecting the entire investigation.