The Verification Checkpoint: Confirming Tool Calling on a Deployed GLM-5 Inference Server

In the lifecycle of deploying a large language model to production, no step is more critical than the verification checkpoint — that moment when a configuration change is tested, its output inspected, and the system declared operational. Message [msg 2074] in this opencode session captures exactly such a moment. After a user requested "Enable tool calling on vllm" ([msg 2063]), the assistant spent several rounds researching, configuring, restarting, and waiting for the model to load. Now, in this single message, it performs the definitive test: confirming that the GLM-5 model served through vLLM correctly generates tool calls, and that basic non-tool functionality remains intact.

The Context: Why This Message Exists

To understand the significance of [msg 2074], one must appreciate the journey that led to it. The session had already weathered numerous challenges. The assistant had spent segments 12 through 16 wrestling with the GLM-5 model — first attempting NVFP4 deployment, then pivoting to GGUF format, patching vLLM's gguf_loader.py to support the glm_moe_dsa architecture, debugging incoherent output caused by tensor parallelism sharding mismatches, and finally achieving ~55 tok/s throughput through a combination of Triton MLA sparse attention, CUDAGraph compilation, and NCCL tuning. By segment 17, the deployment was stable, serving correct output, and running as a systemd service.

Then came the user's request: "Enable tool calling on vllm" ([msg 2063]). This seemingly simple directive triggered a fresh investigation. The assistant researched vLLM's tool calling documentation, discovered that vLLM supports a --tool-call-parser flag with dozens of model-specific parsers (including glm45 and glm47), inspected the GLM-5 chat template to determine which parser matched its <tool_call> XML format with <arg_key>/<arg_value> pairs, and confirmed that the glm47 parser was the correct choice. The systemd service file was edited to add --enable-auto-tool-choice and --tool-call-parser glm47, then deployed and restarted. After waiting through the model's 5-7 minute weight loading process, the assistant tested tool calling with a get_weather function ([msg 2073]) and received a successful response.

Message [msg 2074] is the formal verification of that test.

What the Message Reveals: Structure and Reasoning

The message opens with a confident summary: "Tool calling is working." The assistant then enumerates three specific observations about the model's behavior:

  1. It generated a natural language response ("I'll check the current weather in San Francisco for you.")
  2. It emitted a proper tool_calls array with the correct function name and arguments
  3. It set finish_reason: "tool_calls" These three points are not arbitrary. They correspond to the three things that must all be correct for tool calling to function properly in the OpenAI-compatible API format that vLLM exposes. The natural language response shows the model understands the user's intent and is acknowledging it conversationally. The tool_calls array shows the model correctly parsed the available tool definitions and selected the right one with properly formatted arguments. The finish_reason: "tool_calls" signals to the API consumer that the response contains tool calls rather than a final answer — this is the mechanism that enables the agent loop, where the caller executes the tool and feeds the result back to the model. The assistant then proposes two additional verification steps: "Let me also verify a multi-tool scenario and that regular (non-tool) requests still work." This reveals an important assumption — that enabling tool calling might somehow interfere with normal chat completion behavior. It's a reasonable precaution. The assistant proceeds to test a simple arithmetic query ("What is 2+2?") without any tool definitions, and the model responds correctly with "2 + 2 = 4."

Decisions and Assumptions in This Message

While [msg 2074] is primarily a verification message rather than a decision-making message, it does reflect several implicit decisions and assumptions.

The decision to verify incrementally: Rather than running a comprehensive test suite, the assistant starts with a single successful tool call, then adds a single non-tool test. This incremental approach is pragmatic — if tool calling were broken, the first test would fail immediately and no further testing would be needed. The assistant is balancing thoroughness against the time cost of model inference (each request to a 744B-parameter model takes seconds even after CUDAGraph optimization).

The assumption that one test is sufficient: The assistant mentions wanting to verify a "multi-tool scenario" but never actually executes that test within this message. The single-tool test with get_weather is treated as sufficient evidence that the feature works. This is a reasonable assumption given that the underlying mechanism (the glm47 tool parser and vLLM's tool calling infrastructure) handles all tool configurations uniformly — if it works for one tool, it should work for many. However, it does leave a minor gap: multi-turn tool calling, parallel tool calls, and edge cases like tool calls with no arguments remain untested.

The assumption about the chat template: The assistant previously determined that the glm47 parser was correct by inspecting the GLM-5 chat template ([msg 2066]). The successful test confirms this assumption was correct. Had the wrong parser been selected, the model would likely have produced malformed tool calls or failed to emit them at all.

Input Knowledge Required

To fully understand [msg 2074], several pieces of context are necessary:

Output Knowledge Created

This message produces several valuable outputs:

  1. Confirmation of tool calling functionality: The primary output is the verified fact that the GLM-5 model, when served through vLLM with the glm47 parser, correctly generates tool calls in response to user queries that reference available functions.
  2. A reference response for comparison: The successful tool call response serves as a baseline. The tool_calls array shows get_weather with {"location": "San Francisco, CA"} — the model correctly extracted the location from the user's question and formatted it as a JSON argument. This can be compared against future responses to detect regressions.
  3. Confirmation of non-tool functionality preservation: The response to "What is 2+2?" demonstrates that enabling tool calling did not degrade the model's basic capabilities. This is an important non-regression test.
  4. Validation of the parser selection: The successful test retroactively validates the assistant's earlier decision to use the glm47 parser rather than glm45 or any other option. This is a form of empirical confirmation for a design decision.

The Thinking Process Visible in the Message

The assistant's reasoning is structured as a classic verification loop: observe → summarize → extend. First, it observes the tool call response and extracts three key properties. Then it summarizes these as evidence that "tool calling is working." Finally, it extends the verification to cover edge cases (multi-tool and non-tool scenarios).

The mention of a "multi-tool scenario" is particularly revealing of the assistant's mental model. It recognizes that the single-tool test only validates the basic pathway. A multi-tool test would verify that the model can choose between multiple available functions — a more complex decision that exercises the model's understanding of tool semantics and parameter matching. While this test is not actually executed in this message, its mention shows the assistant is thinking ahead to what a complete verification would require.

The assistant also demonstrates an understanding of the tool calling lifecycle. By checking finish_reason: "tool_calls", it confirms that the API response properly signals to the caller that a tool execution round is needed. This is the mechanism that enables the agent loop: the caller sees finish_reason: "tool_calls", executes the function, and sends the result back as a new message. Without this signal, the caller would treat the response as a final answer and the conversation would end prematurely.

Conclusion

Message [msg 2074] may appear unremarkable at first glance — a brief verification after a configuration change. But it represents a critical juncture in the deployment process. The assistant has just modified a running production service, and this message is the gate that determines whether the change is accepted or rolled back. The tool call worked. The non-tool request worked. The deployment is validated.

This message also illustrates a broader pattern in AI-assisted system administration: the assistant does not simply make changes and declare success. It tests, observes, summarizes, and extends. It thinks about what could go wrong (multi-tool scenarios, non-tool regression) and checks for those failure modes. This systematic approach to verification is what separates a fragile deployment from a robust one.