The GGUF Tensor Name Reconnaissance: How Multi-Source Research Unlocked GLM-5 for vLLM

Introduction

In the high-stakes world of large language model deployment, the difference between a working inference pipeline and a cryptic ValueError often comes down to a single string: the name of a tensor. When the GLM-5 deployment effort pivoted from NVFP4 quantization on SGLang to GGUF UD-Q4_K_XL quantization on vLLM, the team hit a wall. Neither HuggingFace's transformers library nor the gguf-py package recognized the glm-dsa architecture that GLM-5 uses. The user's directive was unambiguous: add GGUF support to vLLM by writing a patch for gguf_loader.py. But before any code could be written, a fundamental question had to be answered: what are the exact tensor names inside the GLM-5 GGUF file, and how do they map to the HuggingFace tensor names that vLLM already understands?

This article synthesizes the work of an entire subagent session (session 387) dedicated to answering precisely that question. Across 19 messages, the assistant conducted a multi-pronged investigation spanning llama.cpp source code, HuggingFace model repositories, conversion scripts, and live GGUF metadata extraction. The result was a comprehensive tensor name mapping that would serve as the foundation for the vLLM GGUF loader patch. This is the story of that investigation — the assumptions, discoveries, pivots, and synthesis that turned an opaque binary format into a navigable map.

The Context: A Pivot That Demanded New Knowledge

The broader session (root segment 12) had been a marathon of environment setup and model deployment. After wrestling with NVIDIA driver installation, CUDA toolkit version conflicts, and flash-attn compilation issues across eight RTX PRO 6000 Blackwell GPUs, the team had finally stabilized a working stack. The original goal was to deploy GLM-5 using NVFP4 quantization via SGLang. That path failed. The pivot was to GGUF format — specifically, the UD-Q4_K_XL quantized variant hosted on HuggingFace at unsloth/GLM-5-GGUF.

But this pivot introduced a critical blocker. When the assistant tried to load the GGUF file through vLLM's existing pipeline, it encountered ValueError: GGUF model with architecture deepseek2 is not supported yet. The root cause was a chain of dependencies: vLLM's GGUF loader relies on the transformers library for architecture metadata parsing, and neither transformers v5.2.0 nor gguf-py v0.17.1 included the glm-dsa architecture. The user explicitly rejected all alternative paths — reverting to SGLang, using llama.cpp directly, or switching to FP8 quantization. The directive was to add GGUF support to vLLM directly.

This is where the subagent session begins. The user's opening message ([msg 0]) lays out a research brief: study the GLM-5 GGUF tensor names to create a mapping from GGUF tensor names to HuggingFace/vLLM tensor names. The message is notable for its self-correcting structure — it begins by proposing direct SSH inspection, immediately realizes the model isn't downloaded, pivots to downloading just the metadata shard, then further pivots to source code analysis. This layered fallback strategy shows an awareness of practical constraints and a willingness to adapt the research approach on the fly.

The Opening Gambit: Parallel Reconnaissance

The assistant's first response ([msg 1]) is a masterclass in parallel information gathering. Recognizing that the fastest way to complete a research task is to gather all potentially relevant data in one shot, the assistant dispatches four independent actions simultaneously:

  1. An SSH command to check what Python packages are installed on the remote machine
  2. A web fetch of the llama.cpp source file llama-model.cpp from GitHub
  3. A web fetch of the GLM-5 HuggingFace config.json
  4. A web fetch of the HuggingFace API listing of the GGUF split files None of these actions depend on each other, so parallelizing them is optimal. The results arrive in the next round and immediately yield valuable intelligence. The HuggingFace config.json reveals that GLM-5 uses the GlmMoeDsaForCausalLM architecture with hidden_size: 6144, kv_lora_rank: 512, q_lora_rank: 2048, and 256 routed experts with 8 experts per token. The HuggingFace API listing confirms the GGUF model is split into 10 files, with the first shard being only 9 MB — suspiciously small for a 431 GB model. The llama-model.cpp fetch, though truncated, confirms the source code contains references to both LLM_ARCH_DEEPSEEK2 and LLM_ARCH_GLM_DSA.

The First Pivot: When Metadata Shards Yield No Tensors

The assistant's next move ([msg 2]) is to install the gguf and huggingface_hub Python packages on the remote machine and download the first GGUF shard. The reasoning is sound: if the first shard is only 9 MB, it might contain the metadata header with tensor names and shapes, avoiding the need to download the full 431 GB model.

But when the assistant runs GGUFReader on the downloaded shard ([msg 6]), the output reveals a startling fact:

GGUF.tensor_count = [0]
Total tensors: 0

The first shard contains zero tensors. It is purely a metadata shard — architecture type, quantization parameters, tokenizer configuration — but the actual weight tensors are distributed across the remaining nine shards. This is a common pattern for multi-shard GGUF files, but it means the direct empirical approach of reading tensor names from the file cannot proceed with just the first shard.

This discovery triggers a strategic pivot ([msg 7]). The assistant's response is immediate and decisive: "The metadata-only shard has 0 tensors (it's just metadata). I need to look at the actual tensor name strings that are constructed from LLM_TENSOR enum values." This is a shift from an empirical strategy (read the actual tensor names from the file) to a deductive strategy (reconstruct the tensor names from the source code that generates them). The assistant dispatches two new fetches in parallel: the llama-arch.cpp file (which maps LLM_TENSOR enum values to their string representations) and the HuggingFace model.safetensors.index.json (which provides the complete set of HuggingFace-side tensor names).

The Deep Dive: Reading the Bones of llama.cpp

With the deductive strategy in place, the assistant begins a systematic exploration of the llama.cpp codebase. The grep results from [msg 2] had already revealed a critical line at position 8003 of llama-model.cpp:

if (arch == LLM_ARCH_DEEPSEEK2 || arch == LLM_ARCH_GLM_DSA)

This line shows that GLM_DSA is treated as a sibling to DEEPSEEK2 in the llama.cpp codebase, sharing much of the same tensor loading logic. It's a crucial finding that shapes everything that follows.

The assistant then reads the GLM_DSA tensor set from llama-arch.cpp ([msg 9]), which reveals the complete list of tensors that the GGUF format expects for a glm-dsa model. This includes standard tensors like LLM_TENSOR_TOKEN_EMBD and LLM_TENSOR_OUTPUT_NORM, attention tensors like LLM_TENSOR_ATTN_Q_A and LLM_TENSOR_ATTN_KV_A_MQA, MoE tensors like LLM_TENSOR_FFN_GATE_EXPS and LLM_TENSOR_FFN_DOWN_EXPS, and DSA-specific indexer tensors.

But knowing what tensors the GGUF format expects is only half the picture. The assistant also needs to know how HuggingFace tensors are transformed into GGUF tensors during conversion. This leads to the discovery of the convert_hf_to_gguf.py script ([msg 11]), where a critical inheritance relationship is revealed:

@ModelBase.register("GlmMoeDsaForCausalLM")
class GlmMoeDsaModel(DeepseekV2Model):
    model_arch = gguf.MODEL_ARCH.GLM_DSA

GlmMoeDsaModel inherits from DeepseekV2Model. This means the tensor mapping for GLM-5 is largely the same as for DeepSeek V2, with additional tensors for the DSA indexer and NextN prediction layers. The assistant reads the DeepseekV2Model class ([msg 13]) and the expert weight merging code ([msg 14]) to understand the full transformation pipeline.

The Second Pivot: When the Package Doesn't Support the Architecture

With the conversion script understood, the assistant attempts to use the installed gguf Python package to get the tensor name map directly ([msg 15]). The command is straightforward:

tensor_map = gguf.get_tensor_name_map(gguf.MODEL_ARCH.GLM_DSA, 79)

But it fails with an AttributeError: type object 'MODEL_ARCH' has no attribute 'GLM_DSA'. The installed gguf package (version 0.17.1) predates the addition of the GLM_DSA architecture to llama.cpp. This is a significant obstacle — the package that should be the authoritative source for tensor name mappings doesn't even know about the architecture.

The assistant adapts by listing all available architectures ([msg 16]) and confirming that GLM_DSA is indeed missing from the enum. The fallback is to use DEEPSEEK2 as a proxy ([msg 17]):

tensor_map = gguf.get_tensor_name_map(gguf.MODEL_ARCH.DEEPSEEK2, 79)

This works because GlmMoeDsaModel inherits from DeepseekV2Model. The DEEPSEEK2 tensor name map provides the baseline mapping for the MLA (Multi-head Latent Attention) components that GLM-5 shares with DeepSeek V2/V3. The DSA-specific tensors and NextN tensors, being unique to GLM-5, must be mapped separately using the information gathered from the llama.cpp source code and the HuggingFace safetensors index.

The Synthesis: A Comprehensive Tensor Name Mapping

The culmination of this multi-source investigation is message [msg 18], where the assistant delivers a comprehensive tensor name mapping between HuggingFace's safetensors format and the GGUF format for GLM-5. The message is structured as a reference document with several sections:

Architecture Summary: Establishes the identity of the model — GGUF arch name glm-dsa, HF model type GlmMoeDsaForCausalLM, and the inheritance relationship from DeepSeek V2.

Key Model Parameters: Lists the hyperparameters that define tensor shapes — hidden_size: 6144, num_attention_heads: 64, num_key_value_heads: 1 (MLA → MQA), q_lora_rank: 2048, kv_lora_rank: 512, n_routed_experts: 256, n_shared_experts: 1, num_experts_per_tok: 8, first_k_dense_replace: 3, and the DSA-specific parameters like indexer_n_heads: 32 and indexer_top_k: 2048.

Complete Tensor Name Mapping: A four-part mapping covering global tensors, attention (MLA) tensors, DSA indexer tensors, and FFN tensors (both dense and MoE variants). Each entry includes the GGUF name, the HuggingFace name, shape annotations, and transformation notes.

The most intricate transformation documented is the kv_b_proj split. In HuggingFace format, there is a single tensor kv_b_proj.weight of shape [n_head * (qk_nope + v_head_dim), kv_lora_rank] = [28672, 512]. During GGUF conversion, this tensor is reshaped to [64, 448, 512], split into k_b (first 192 elements) and v_b (last 256 elements), with k_b transposed to [192, 512, 64] and v_b permuted to [512, 256, 64]. This "MLA absorption optimization" is critical for efficient inference, and the vLLM loader must handle it correctly.

Another major transformation is the merging of MoE expert weights. In HuggingFace format, each of the 256 experts has its own gate_proj.weight, down_proj.weight, and up_proj.weight. In GGUF format, these are stacked into single 3D tensors: ffn_gate_exps.weight of shape [256, 6144, 2048], ffn_down_exps.weight of shape [256, 2048, 6144], and ffn_up_exps.weight of shape [256, 6144, 2048].

The DSA indexer tensors — indexer.k_norm, indexer.proj, indexer.attn_k, indexer.attn_q_b — are unique to GLM-5 and don't exist in standard DeepSeek V2/V3 models. These implement a separate attention mechanism that determines which tokens to attend to, enabling efficient long-context processing.

The Significance: From Research to Implementation

The tensor name mapping delivered in message [msg 18] represents the critical bridge between knowing that a patch is needed and knowing what the patch must do. For the vLLM developer tasked with writing the gguf_loader.py patch, this message provides everything needed to implement the tensor loading logic: the exact GGUF names to look for, the HuggingFace names they correspond to, the expected shapes, and the transformations that must be reversed.

The mapping also serves as a documentation artifact for the GLM-5 architecture itself. By compiling the tensor mappings and transformation rules in one place, it creates a reference that can be used not just for vLLM integration but for any future work involving GLM-5 in GGUF format — whether that's debugging, optimization, or porting to another inference engine.

The investigation also revealed several important details that would be essential for a correct implementation:

  1. The blocker is solely in transformers, not vLLM: vLLM already has manual weight mappings for DeepSeek architectures. The issue is that the transformers library doesn't recognize the glm-dsa architecture, which prevents the GGUF metadata from being parsed correctly.
  2. The gguf-py library from llama.cpp HEAD already defines GLM_DSA: The installed package (0.17.1) is outdated, but the latest unreleased version of llama.cpp's gguf-py already has LLM_ARCH_GLM_DSA with a complete tensor name map. Installing from source resolves the missing architecture issue.
  3. Expert weights use fused gate_up_proj format: The GGUF conversion merges gate and up projections into a single tensor, which must be handled correctly during loading.
  4. e_score_correction_bias needs manual mapping: This special bias tensor used in DeepSeek models for expert score correction has no direct HuggingFace equivalent and requires explicit handling in the loader.

Conclusion

The subagent session dedicated to researching GLM-5 GGUF tensor names is a textbook example of how complex system integration work proceeds in practice. It begins with a clear goal — create a tensor name mapping — and proceeds through multiple phases of investigation, each informed by the results of the previous phase. When the direct approach (reading tensor names from the GGUF file) fails due to the metadata-only first shard, the assistant pivots to a deductive approach (reconstructing names from source code). When the installed Python package lacks support for the target architecture, the assistant uses a related architecture as a proxy and manually adds the missing tensors.

The result is a comprehensive mapping that bridges two naming conventions — the GGUF format's flat blk.{i}.{tensor_name} namespace and the HuggingFace format's hierarchical model.layers.{i}.{module}.{weight_name} namespace. This mapping is the essential prerequisite for writing the vLLM GGUF loader patch that will enable GLM-5 deployment. Without it, any attempt to load the model would fail because the loader would look for tensors by their GGUF names and either find nothing or misinterpret the shapes.

The investigation also demonstrates a key principle of effective research: when multiple independent sources converge on the same conclusion, confidence increases. The llama.cpp source code, the conversion script, the HuggingFace safetensors index, and the GGUF metadata all point to the same tensor structure. This cross-validation is what makes the final mapping reliable enough to build a production system upon.

In the broader narrative of the coding session, this subagent session represents the transition from research to implementation. The assistant has gathered all the necessary information and presented it in a form that can be directly acted upon. The next steps — writing the vLLM patch, testing it against the actual GGUF file, and debugging any issues — will all build on the foundation established in this single, meticulously crafted mapping.