The Final Piece: Productionalizing GLM-5 as a Systemd Service

A Message of Triumph After a Long Debugging Journey

In the sprawling, multi-session effort to deploy the GLM-5 744B parameter model on 8× NVIDIA RTX PRO 6000 Blackwell GPUs, message [msg 2027] stands as a quiet but significant milestone. It is the moment when a complex, deeply technical debugging and optimization marathon finally crossed the threshold into production. The message itself is deceptively simple—a single bash command and its output—but the context surrounding it tells a story of hardware constraints, software patching, performance tuning, and the careful art of system administration.

The Message

The assistant executed:

ssh root@10.1.230.174 'systemctl enable vllm-glm5.service 2>&1; systemctl start vllm-glm5.service 2>&1; sleep 3; systemctl status vllm-glm5.service 2>&1'

And received the response:

Created symlink /etc/systemd/system/multi-user.target.wants/vllm-glm5.service -> /etc/systemd/system/vllm-glm5.service.
* vllm-glm5.service - vLLM GLM-5 744B GGUF Q4_K_XL Inference Server
     Loaded: loaded (/etc/systemd/system/vllm-glm5.service; enabled; preset: enabled)
     Active: active (running) since Fri 2026-02-20 19:50:41 UTC; 3s ago
   Main PID: 194110 (python3)
      Tasks: 128 (limit: 618378)
     Memory: 426.5M (peak: 435.9M)
        CPU: 14.058s
     CGroup: /system.slice/vllm-glm...

Why This Message Was Written: The Context of Productionalization

This message did not emerge from a vacuum. It was the direct result of a user directive in [msg 2013]: "No keep this config for now, productionalise into vllm-glm5 systemd service." This instruction came after an exhaustive optimization campaign that had consumed multiple sub-sessions and dozens of messages.

The assistant had been exploring paths to push single-request decode throughput from 57 tok/s toward 100 tok/s—investigating allreduce-RMS fusion, NCCL protocol tuning, custom allreduce kernels, and pipeline-parallel configurations. Each avenue had been systematically explored and either exhausted or found incompatible with the hardware topology. The flashinfer allreduce-RMS fusion, for instance, required NVSwitch multicast hardware that simply did not exist on the PCIe-only Blackwell GPUs. The NCCL LL protocol had already delivered a 34% improvement over baseline, but further gains were blocked by the fundamental reality of PCIe Gen5 x16 bandwidth and latency.

When the user said "keep this config," they were implicitly accepting the 57 tok/s ceiling as the best achievable performance on this hardware. The task shifted from optimization to productionalization—making the current configuration reliable, restartable, and manageable as a system service.

The First Attempt and Its Failure

The path to [msg 2027] was not straightforward. In [msg 2024], the assistant had made an initial attempt to deploy the service:

pkill -f "vllm.entrypoints" 2>/dev/null; sleep 2; systemctl daemon-reload && systemctl enable vllm-glm5.service && echo "Service enabled" && systemctl start vllm-glm5.service && echo "Service started"

This command sequence contained a subtle but critical flaw: the pkill command, which was intended to clean up any stale vLLM processes before starting the service, may have created a race condition. If the systemd service started its own vLLM process while the pkill was still matching or after a brief window, the kill could have terminated the freshly started service process. Alternatively, the daemon-reload might not have fully completed before the start command was issued, or the service unit file might not have been properly recognized yet.

The result, shown in [msg 2025], was a service in "inactive (dead)" state—started but immediately failing. The assistant's diagnostic instinct kicked in, and in [msg 2026] it checked the journal, only to find "No entries"—a puzzling outcome that suggested the service had failed before producing any log output, or that journald had not yet flushed the entries.

The Second Attempt: What Changed

In [msg 2027], the assistant made several critical adjustments to the deployment strategy:

  1. Removed the pkill step. This was the most important change. By not killing existing vLLM processes before starting the service, the assistant eliminated the race condition. The stale process from the previous failed attempt may have already terminated naturally, or it may have been irrelevant—what mattered was that the systemd service could start its own process without interference.
  2. Added 2>&1 redirection. This ensured that any error output from systemctl enable or systemctl start would be captured in the command output, providing better diagnostic information if something went wrong.
  3. Added a sleep 3 before checking status. This gave the service time to initialize before the status check, preventing a false "starting" or "failed" reading during the brief window when the process was still booting.
  4. Combined all operations into a single SSH command. This reduced the risk of state changes between separate SSH invocations and ensured a consistent view of the system state. The result was unambiguous success: "Active: active (running) since Fri 2026-02-20 19:50:41 UTC; 3s ago." The service was running, with PID 194110, 128 tasks, and 426.5 MB of memory consumed during initialization.

Assumptions Made

The assistant made several assumptions in crafting this message:

Mistakes and Incorrect Assumptions

The primary mistake was in the first attempt ([msg 2024]), where the pkill command likely sabotaged the service startup. The assistant assumed that killing stale processes before starting the service was a safe cleanup operation, but it failed to account for the timing window where the systemd-managed process could be caught in the same kill pattern.

A secondary issue was the lack of error output capture in the first attempt. The original command used && chaining without 2>&1, meaning any error messages from systemctl would have gone to stderr and been invisible in the SSH output. The assistant learned from this and added the redirection in the second attempt.

The "No entries" result from journalctl in [msg 2026] was also slightly misleading. In retrospect, the service had likely failed so quickly that journald hadn't flushed the log buffer to disk, or the service had never truly started (the systemctl start command may have returned an error that was invisible without stderr capture).

Input Knowledge Required

To understand this message fully, one needs:

Output Knowledge Created

This message created several forms of output knowledge:

The Thinking Process

The assistant's reasoning in this message reflects a methodical, diagnostic approach. After the failure in [msg 2024], the assistant did not simply retry the same command. Instead, it analyzed what went wrong:

  1. The service was "inactive (dead)" — this meant it started but failed, or failed to start at all.
  2. The journal had "No entries" — this was unusual and suggested either a very fast failure or a logging configuration issue.
  3. The most likely culprit was the pkill command, which could have killed the service process immediately after startup. The assistant then redesigned the deployment command to eliminate the race condition, add error capture, and include a startup delay before the status check. This is classic debugging behavior: formulate a hypothesis about the root cause, design a test that eliminates that cause, and observe whether the outcome changes. The choice to combine enable, start, and status into a single SSH command also reflects an understanding of distributed systems debugging. In a remote SSH session, each invocation creates a new connection with potentially different state. By combining operations, the assistant ensured a consistent view of the system and eliminated the possibility of state drift between commands.

Significance in the Larger Narrative

This message represents the culmination of an extraordinary engineering effort. The journey had begun with installing NVIDIA drivers and CUDA Toolkit on a fresh Ubuntu 24.04 system, proceeded through resolving flash-attn build issues, patching vLLM to support the novel GLM-DSA architecture, debugging incoherent model output caused by subtle bugs in the Triton MLA attention backend and GGUF dequantization shard ordering, and optimizing throughput from 20 tok/s to 57 tok/s through CUDAGraph and NCCL tuning.

The systemd service was the final step—the transition from an experimental, manually-started server to a production service that would survive reboots, be manageable through standard systemd commands, and provide a foundation for real-world inference workloads. The 57 tok/s throughput, while below the aspirational 100 tok/s target, represented the best achievable performance given the PCIe-only hardware topology and the complexity of serving a 744B parameter model across 8 GPUs.

In the broader context of opencode sessions, this message exemplifies the transition from development to operations—the moment when a carefully crafted software stack is packaged into a maintainable, restartable service. It is a reminder that deploying machine learning models at scale involves not just model architecture and optimization, but also the unglamorous but essential work of system administration: writing unit files, managing services, and ensuring that complex distributed systems start reliably and stay running.