The Moment Everything Unmapped: Discovering a Catastrophic Name Map Failure in GLM-5 GGUF Deployment

Introduction

In the long and arduous process of deploying the GLM-5 model on eight Blackwell GPUs using GGUF quantization, there comes a moment of crystalline clarity — a single diagnostic command that reveals why the model has been producing nothing but garbage. Message [msg 1916] in this opencode session is that moment. After hours of methodically ruling out dequantization kernel bugs, attention backend incompatibilities, and weight value corruption, the assistant finally lands on the root cause: the gguf-py library's automatic tensor name mapping for the glm-dsa architecture is completely broken, returning None for every single one of the 1,809 tensors in the model file. This article examines that message in detail: the reasoning that led to it, the discovery it produced, and the assumptions that had to be overturned along the way.

The Debugging Landscape Before the Discovery

To understand the significance of message [msg 1916], one must appreciate the debugging terrain the assistant had already traversed. The session began in Segment 10 with a performance investigation on the NVFP4 model path, which was eventually abandoned in favor of GGUF quantization using unsloth's UD-Q4_K_XL format. By Segment 12, the assistant had pivoted to deploying this GGUF model on vLLM, only to discover that neither HuggingFace transformers nor gguf-py supported the glm-dsa architecture natively. This forced the assistant to write a comprehensive patch for vLLM's gguf_loader.py, adding manual name mappings for MoE expert weights and implementing the reassembly of the split kv_b_proj tensor from its constituent k_b and v_b parts.

By the time we reach message [msg 1916], the model has been loaded onto the GPUs and the server is serving requests — but the output is incoherent. Logprobs for predictable tokens like "2" following "1" are around -20, indicating the model's hidden states are essentially random. The assistant has already ruled out several plausible culprits:

The Message Itself: Reasoning and Execution

Message [msg 1916] opens with a moment of recognition: "I see the problem." The assistant has been examining the gguf-py name map for glm-dsa and notices that it includes blk.0.indexer.* tensors — but the manual override for MoE expert mapping only covers layers from first_k_dense_replace through num_hidden_layers. The auto name map creates entries for all layers, including indexer and nextn (MTP) tensors.

But the assistant quickly pivots to a more fundamental question: "Let me check if maybe the name map from gguf-py correctly maps the standard layer tensors (non-MoE, non-expert) to HF names. The key question: does the auto name map from gguf-py for glm-dsa correctly map things like blk.0.ffn_down → model.layers.0.mlp.down_proj.weight?"

This is the crucial hypothesis. The assistant has been relying on a hybrid approach: a manual override for the special MoE and kv_b tensors, plus the assumption that the gguf-py auto name map handles everything else. If that assumption is wrong, the entire weight loading pipeline is compromised.

To test this, the assistant crafts a Python script that:

  1. Iterates through gguf.MODEL_ARCH_NAMES to find the arch key for glm-dsa (key 73).
  2. Creates a tensor name map using gguf.get_tensor_name_map(arch_key, 78) — 78 being the number of layers in GLM-5.
  3. Opens the actual GGUF file and iterates over all 1,809 tensors.
  4. For each tensor, calls nm.get_name(t.name) to get the HuggingFace parameter name.
  5. Counts how many tensors map successfully and how many return None.
  6. Prints detailed mappings for layer 0 tensors. The result is devastating:
Mapped: 0, Unmapped: 1809
  blk.0.attn_k_b.weight -> None
  blk.0.attn_kv_a_mqa.weight -> None
  blk.0.attn_kv_a_norm.weight -> None
  blk.0.attn_norm.weight -> None
  blk.0.attn_output.weight -> None
  blk.0.attn_q_a.weight -> None
  blk.0.attn_q_a_norm.weight -> None
  blk.0.attn_q_b.weight -> None
  blk.0.attn_v_b.weight -> None
  blk.0.ffn_down.weight -> None
  blk.0.ffn_gate.weight -> None
  blk.0.ffn_norm.weight -> None
  blk.0.ffn_up.weight -> None
  blk.0.indexer.attn_k.weight -> None
  ...

Zero tensors map correctly. Every single GGUF tensor name — attention weights, layer norms, FFN projections, indexer tensors — returns None when queried against the auto name map. The entire automatic mapping pipeline is non-functional for the glm-dsa architecture.

The Thinking Process Visible in the Message

The assistant's reasoning in this message reveals a sophisticated debugging methodology. Several cognitive patterns are worth noting:

Hypothesis narrowing: The assistant doesn't start with the name map. It has already eliminated dequantization, flash attention, and weight value issues. The name map is the next logical component in the data flow from GGUF file to model parameters.

Explicit hypothesis formulation: The assistant clearly states what it expects to find: "does the auto name map from gguf-py for glm-dsa correctly map things like blk.0.ffn_down → model.layers.0.mlp.down_proj.weight?" This precision is essential — a vague question would produce a vague answer.

Quantitative verification: The script counts mapped vs. unmapped tensors (0 vs. 1809), providing an unambiguous metric. The assistant doesn't just check a few tensors; it checks all of them.

Layered investigation: The assistant first notices a surface-level issue (the auto map includes indexer tensors that overlap with the manual override), then drills deeper to the fundamental question (does the map work at all for standard tensors?). This is characteristic of expert debugging: the surface observation is a symptom, not the cause.

Input Knowledge Required

To fully understand this message, several pieces of background knowledge are necessary:

GGUF format and tensor naming: GGUF files store quantized model weights as named tensors. The naming convention uses a block-oriented scheme: blk.{layer_id}.{component}.{tensor_name}.weight. For example, blk.0.attn_q_a.weight is the query-A attention weight for layer 0. Understanding this naming is essential to interpret the test output.

The gguf-py library: This Python library provides utilities for reading GGUF files and mapping tensor names between GGUF conventions and HuggingFace transformer parameter names. The MODEL_ARCH_NAMES dictionary maps architecture keys to human-readable names (e.g., key 73 → glm-dsa). The get_tensor_name_map() function returns a name mapper for a given architecture and layer count.

vLLM's weight loading pipeline: The GGUF loader in vLLM reads tensors from a GGUF file and loads them into the model's parameters. It uses the name map to determine which model parameter each GGUF tensor corresponds to. If the map returns None, the tensor cannot be loaded.

GLM-5 architecture specifics: GLM-5 uses Multi-head Latent Attention (MLA) and Mixture of Experts (MoE), with layers 0-2 being dense and layers 3-77 being MoE. It also includes an Indexer module for speculative decoding and nextn (MTP) tensors for multi-token prediction.

The manual override context: The assistant had previously patched vLLM's gguf_loader.py to add manual name mappings for MoE expert weights (ffn_gate_exps, ffn_up_exps, ffn_down_exps) and to reassemble kv_b_proj from k_b and v_b. These manual overrides were designed to supplement the auto name map, not replace it entirely.

Output Knowledge Created

This message produces several critical pieces of knowledge:

The auto name map is completely broken for glm-dsa: This is the primary finding. The gguf-py library's get_tensor_name_map() returns None for every tensor in the GLM-5 GGUF file. This means the automatic name mapping is non-functional.

The root cause of garbage output is identified: The model produces incoherent output because most of its weights are never loaded. Only tensors explicitly handled by the manual override in gguf_loader.py (MoE expert weights, kv_b_proj) get loaded. All other parameters — attention projections, layer norms, FFN weights for dense layers, embedding weights — remain in their default/uninitialized state.

The hybrid approach was fundamentally flawed: The assistant's strategy of writing manual overrides for special tensors while relying on the auto map for standard tensors was based on an incorrect assumption. The auto map provides no coverage at all, so the manual overrides were carrying the entire weight of the loading process — and failing to cover most tensors.

A new debugging direction is established: The path forward is now clear: the assistant must either fix the gguf-py name map for glm-dsa or replace the auto map entirely with a comprehensive manual mapping. The previous debugging efforts (dequantization kernel, flash attention, weight values) were chasing symptoms of this deeper issue.

Assumptions and Potential Mistakes

The assistant's reasoning in this message rests on several assumptions, some of which may be worth examining:

The name map is supposed to work: The assistant assumes that gguf.get_tensor_name_map() should return valid mappings for the glm-dsa architecture. This is a reasonable assumption — the architecture is registered in MODEL_ARCH_NAMES — but it may be incorrect. The registration might be incomplete, or the tensor name map might require additional parameters beyond the layer count.

The test script is correct: The assistant's diagnostic script uses gguf.get_tensor_name_map(arch_key, 78). If the function signature or expected parameters differ from what the assistant assumes, the test might produce false negatives. For example, if the function expects a different format for the layer count, or if the arch_key lookup is incorrect, the map might fail for reasons unrelated to the actual architecture support.

The manual override doesn't cover everything: The assistant assumes that only MoE expert weights and kv_b_proj are handled by the manual override. This is likely correct based on the code shown in <msg id=1914-1915>, but it's worth verifying that the override doesn't accidentally handle more tensors than expected.

The weight loader skips unmapped tensors: The assistant assumes that when get_name() returns None, the weight loader simply skips the tensor rather than crashing. This is consistent with the observation that the model loaded without errors — if the loader crashed on unmapped tensors, the server would never have started.

One potential mistake in the assistant's approach is not immediately checking whether the gguf-py library's glm-dsa support was added correctly. The architecture might have been registered in MODEL_ARCH_NAMES without a corresponding entry in the tensor name mapping tables. The assistant's test reveals the symptom (all None), but doesn't yet diagnose why the map fails — it could be a missing mapping table, an incorrect arch key, or a version incompatibility.

Why This Message Matters

Message [msg 1916] is a turning point in the debugging session. It transforms the investigation from a diffuse search across multiple possible causes (dequantization, attention, flash, weight shapes) into a focused engineering problem: the name map is broken and must be fixed. The garbage output is no longer a mysterious symptom; it's a predictable consequence of loading a model with 1,809 unmapped tensors.

The message also illustrates a crucial debugging principle: when all individual components check out, look at the interfaces between them. The dequantization kernel works. The flash attention backend is available. The weight values look reasonable. But the pipeline connecting GGUF file to model parameters — the name map — is completely non-functional. The assistant's methodical elimination of other hypotheses made this discovery possible.

For anyone following this session, message [msg 1916] is the moment the fog clears. The path forward is arduous — fixing the name map for 1,809 tensors across 78 layers is no small task — but at least the problem is now understood. The assistant can stop guessing and start building.