Decoding the GLM-5 GGUF Tensor Map: A Pivot Point in the NVFP4-to-GGUF Migration

Introduction

In the sprawling, multi-day effort to deploy the GLM-5 model on a production GPU server, message [msg 1544] marks a quiet but decisive moment of discovery. Sandwiched between the abandonment of the NVFP4 quantization path and the construction of a comprehensive vLLM GGUF loader patch, this single bash command — probing the internals of the gguf-py library — answers a critical question that determines the feasibility of the entire deployment strategy: What tensors does the glm-dsa GGUF architecture actually define, and how do they map to the HuggingFace model's parameter names?

The answer, encoded in the numeric enum values of gguf.MODEL_TENSOR, reveals both good news and complications. The glm-dsa architecture supports 38 tensor types, including the expected attention components, indexer tensors for the Dynamic Shared Attention (DSA) mechanism, and — crucially — both a combined ATTN_KV_B tensor and separate ATTN_K_B/ATTN_V_B tensors. This split representation of the key-value projection is the central challenge that the subsequent vLLM patch must address.

Context: Why This Message Exists

To understand message [msg 1544], one must appreciate the precarious position the assistant occupies at this moment in the conversation. The user has just pivoted decisively from the NVFP4 quantization path — after days of kernel profiling, throughput tuning, and bottleneck analysis — toward deploying GLM-5 via GGUF UD-Q4_K_XL on vLLM. The 431 GB GGUF download is running in the background ([msg 1527]). vLLM nightly (0.16.0rc2.dev313) has been installed ([msg 1525]). But a blocker has already emerged: the glm-dsa architecture used by GLM-5 is not recognized by the installed gguf-py library (version 0.17.1), nor by transformers (even at version 5.3.0.dev0 from git HEAD).

The assistant has already taken remedial action, installing gguf-py directly from the llama.cpp source repository ([msg 1539]), which adds the glm-dsa architecture (key 73) to the MODEL_ARCH_NAMES enum. In message [msg 1542], the assistant verified that the tensor name map can translate most HuggingFace parameter names to GGUF tensor names — but with a critical gap: per-expert weights and e_score_correction_bias return None. More concerningly, the kv_b_proj maps to blk.0.attn_kv_b, but the assistant suspects the actual GGUF file splits this into separate attn_k_b and attn_v_b tensors.

Message [msg 1543] then runs a command to list the raw tensor types for glm-dsa, but the output is opaque — just a list of 38 integers (1, 8, 5, 9, 17, 12, 120, 121, 122, 123, ...). These are the numeric values of the MODEL_TENSOR enum, but without their symbolic names, they are meaningless.

This is why message [msg 1544] exists: to decode those integers into human-readable enum names, transforming raw data into actionable knowledge.

The Command and Its Output

The message is a single bash command executed on the remote server via SSH:

~/ml-env/bin/python3 -c "
import gguf

# Get the MODEL_TENSOR enum names
for name in dir(gguf.MODEL_TENSOR):
    if not name.startswith('_'):
        val = getattr(gguf.MODEL_TENSOR, name)
        if isinstance(val, int) and val in [120, 121, 122, 123, 124, 125, 126, 127, 185, 186, 187, 188, 284, 285, 286, 287, 288, 289]:
            print(f'  {val}: {name}')
"

The command iterates over all attributes of gguf.MODEL_TENSOR, filters for integer-valued constants that match the specific numeric codes seen in the previous message, and prints the mapping. The output reveals:

  122: ATTN_KV_A_MQA
  127: ATTN_KV_A_NORM
  123: ATTN_KV_B
  124: ATTN_K_B
  120: ATTN_Q_A
  126: ATTN_Q_A_NORM
  121: ATTN_Q_B
  125: ATTN_V_B
  187: INDEXER_ATTN_K
  188: INDEXER_ATTN_Q_B
  185: INDEXER_K_NORM
  186: INDEXER_PROJ
  284: NEXTN_EH_PROJ
  285: NEXTN_EMBED_TOKENS
  286: NEXTN_ENORM
  287: NEXTN_HNORM
  288: NEXTN_SHARED_HEAD_HEAD
  289: NEXTN_SHARED_HEAD_NORM

What This Discovery Means

The decoded tensor names tell a rich story about the GLM-5 architecture as represented in GGUF format:

1. The KV Split Confirmation

The most important finding is the coexistence of ATTN_KV_B (123) alongside ATTN_K_B (124) and ATTN_V_B (125). In the HuggingFace model, the MLA (Multi-head Latent Attention) mechanism uses a single kv_b_proj weight that projects the compressed latent key-value representation into separate key and value spaces. In the GGUF format, this projection is split into two separate tensors: attn_k_b and attn_v_b. The combined ATTN_KV_B entry exists in the architecture's tensor type list, but the actual GGUF file stores the split versions.

This means any vLLM loader patch must handle the reassembly: loading both attn_k_b and attn_v_b from the GGUF file, then combining them into the single kv_b_proj weight that the HuggingFace model expects. This is not a trivial concatenation — the two tensors may need to be interleaved or stacked along a specific dimension, depending on how the original model's kv_b_proj was split during the GGUF conversion.

2. The DSA Indexer Tensors

The four INDEXER_* tensors (INDEXER_ATTN_K, INDEXER_ATTN_Q_B, INDEXER_K_NORM, INDEXER_PROJ) confirm that the GGUF format preserves the Dynamic Shared Attention mechanism's routing components. These tensors are unique to the glm-dsa architecture and do not exist in standard deepseek_v2 or deepseek_v3 GGUF files. The vLLM patch must explicitly map these from GGUF names to the HuggingFace model's indexer.* parameters.

3. The NextN Prediction Head

The six NEXTN_* tensors (NEXTN_EH_PROJ, NEXTN_EMBED_TOKENS, NEXTN_ENORM, NEXTN_HNORM, NEXTN_SHARED_HEAD_HEAD, NEXTN_SHARED_HEAD_NORM) reveal that the GGUF file includes a separate prediction head for the "next token" prediction task — a feature of the GLM-5 architecture that extends beyond the standard language modeling head. These tensors represent an auxiliary prediction pathway that must be loaded and connected correctly.

4. What Is Missing

Notably absent from the decoded tensor list are the per-expert weight tensors (expert gate/up/down projections) and the e_score_correction_bias. This confirms the finding from message [msg 1542] that these tensors require manual mapping in the vLLM loader — they are not handled by the generic gguf-py tensor name map for glm-dsa. The vLLM patch must replicate the existing manual mapping logic used for deepseek_v2/deepseek_v3 models, which already handles expert weight sideloading and the expert scoring bias.

The Thinking Process

The reasoning visible in this message reveals a systematic, hypothesis-driven approach to reverse-engineering an unfamiliar codebase. The assistant does not simply ask "what are the tensor names?" — it already knows the numeric codes from the previous message and constructs a targeted query to decode them. This is classic investigative debugging: first observe the raw data, then formulate a precise question to interpret it.

The choice of which numeric values to filter on (120–127, 185–188, 284–289) is itself informative. The assistant has mentally grouped the tensors into three clusters: attention components (120–127), indexer components (185–188), and nextn components (284–289). This grouping reflects an understanding of the GLM-5 architecture's modular structure — attention, routing, and auxiliary prediction — that the assistant has built up over the preceding messages.

The command also demonstrates careful Python introspection technique. Rather than assuming a particular API exists (e.g., gguf.MODEL_TENSOR.name_for_value()), the assistant falls back to dir() introspection and attribute filtering — a robust approach that works regardless of the library's documentation quality.

Assumptions and Potential Pitfalls

The message makes several implicit assumptions:

1. That the numeric values are stable across gguf-py versions. The assistant is querying the version installed from llama.cpp HEAD, which may differ from the version used to create the GGUF file. If the llama.cpp version used for conversion had different enum values, the mapping could be wrong. However, since the GGUF file embeds the architecture key (73 for glm-dsa), and the tensor type IDs are standardized within the GGUF specification, this risk is low.
2. That all 38 tensor types are actually present in the GGUF file. The architecture definition lists 38 tensor types, but the actual GGUF file may only contain a subset. The assistant will need to verify this when the download completes.
3. That the split ATTN_K_B/ATTN_V_B tensors can be reassembled into kv_b_proj. The exact reshaping logic is not yet known — it depends on how the llama.cpp conversion script (GlmMoeDsaModel) splits the original weight. The assistant has not yet examined the conversion script's source code.
4. That the vLLM loader can handle the reassembly within its existing framework. vLLM's GGUF loader works by mapping HuggingFace parameter names to GGUF tensor names and loading them directly. If a single HF parameter maps to two GGUF tensors, the loader needs special handling — potentially a custom load function rather than a simple name mapping.

Input Knowledge Required

To fully understand this message, one needs:

• Knowledge of the GLM-5 architecture: That it uses Multi-head Latent Attention (MLA) with a kv_b_proj that projects from a compressed latent space, and Dynamic Shared Attention (DSA) with indexer tensors for routing.
• Familiarity with the GGUF format: That GGUF files store tensors with a standardized naming convention (blk.N.attn_*), and that the gguf-py library provides enum-based tensor type definitions.
• Understanding of the vLLM GGUF loader architecture: That vLLM creates a dummy HuggingFace model on meta device, iterates its state_dict(), and maps each parameter name to a GGUF tensor name using the gguf-py name map.
• Context from the preceding messages: The pivot from NVFP4, the GGUF download, the installation of vLLM nightly and transformers from source, and the discovery that gguf-py 0.17.1 lacks glm-dsa support.

Output Knowledge Created

This message produces a concrete, actionable artifact: a decoded mapping of numeric tensor type IDs to symbolic names for the glm-dsa architecture. This mapping directly informs the vLLM GGUF loader patch that the assistant begins writing in the subsequent messages. Specifically, it:

1. Confirms the KV split problem and its exact nature (two separate tensors vs. one combined tensor).
2. Identifies the DSA indexer tensors that must be mapped (four tensors with specific enum values).
3. Reveals the nextn prediction head tensors that add complexity to the loader.
4. Narrows the scope of manual mapping to expert weights and e_score_correction_bias only — everything else is handled by the generic name map.

Conclusion

Message [msg 1544] is a textbook example of methodical reverse-engineering in a complex ML deployment scenario. It takes an opaque list of integers — the raw output of a previous probe — and transforms it into structured knowledge that directly guides the next phase of work. The assistant does not guess or speculate about the tensor structure; it queries the actual library that will be used to load the model, ensuring that the patch it writes is grounded in the real data format.

In the broader narrative of the GLM-5 deployment, this message represents the moment when the assistant fully understands the GGUF tensor layout and can begin writing the vLLM loader patch with confidence. The 431 GB download continues in the background; the patch is about to be written. This small, focused probe is the key that unlocks the next phase of the work.