The Productionalization Command: Deploying GLM-5 as a Systemd Service

Introduction

In the long arc of deploying the GLM-5 744B model on eight NVIDIA RTX PRO 6000 Blackwell GPUs, few messages carry as much symbolic weight as message 2024. After dozens of rounds spent debugging incoherent output, patching vLLM's GGUF loader, fixing Triton MLA attention backends, and squeezing every last token per second from a PCIe-bound inference stack, the assistant finally issued a single bash command to productionalize the entire setup into a systemd service. This message represents the transition from development and debugging to production deployment — the moment when a fragile experimental configuration is hardened into a reliable, auto-starting service.

The message itself is deceptively simple:

[assistant] [bash] ssh root@10.1.230.174 'pkill -f "vllm.entrypoints" 2>/dev/null; sleep 2; systemctl daemon-reload && systemctl enable vllm-glm5.service && echo "Service enabled" && systemctl start vllm-glm5.service && echo "Service started"'

But beneath this single line lies a rich tapestry of context, assumptions, and decisions that merit close examination.

Context and Motivation

The message was written in direct response to the user's instruction at <msg id=2013>: "No keep this config for now, productionalise into vllm-glm5 systemd service." This instruction came after an extensive optimization phase in which the assistant had pushed single-request decode throughput from ~20 tok/s to ~57 tok/s through a combination of CUDAGraph capture and NCCL_PROTO=LL tuning. The user had just rejected further optimization attempts (such as pipeline parallelism with PP=2+TP=4) and wanted the current best configuration locked down as a production service.

The assistant had already completed several preparatory steps before issuing this command. At <msg id=2017>, it verified that the server was stopped and that systemd was available. At <msg id=2018>, it documented all patched files on the container — the GGUF shard ordering fix, the GLM-DSA architecture support in the loader, the force-dequantization logic in weight_utils, and the Triton MLA attention backend fixes. At <msg id=2019>, it confirmed the 402GB GGUF model file existed and the tokenizer cache was in place. At <msg id=2022>, it wrote the systemd service unit file itself. And at <msg id=2023>, it copied that file to the remote server's /etc/systemd/system/ directory.

By the time message 2024 was issued, the assistant had completed all prerequisites. The remaining todo items in its checklist were "Enable and test the service" — and this message executes exactly that.

The Command Structure: A Deliberate Sequence

The command is a carefully constructed pipeline of six operations, chained with shell operators that reflect deliberate reasoning about failure modes and ordering:

  1. pkill -f "vllm.entrypoints" 2>/dev/null: Kill any existing vLLM server processes. The -f flag matches against the full command line, catching any process running vllm.entrypoints.openai.api_server. The 2>/dev/null suppresses errors if no matching process exists — a graceful handling of the "nothing to kill" case.
  2. sleep 2: A brief pause to allow the killed processes to release GPU memory and free ports. This is a heuristic — the assistant assumes two seconds is sufficient for process cleanup, an assumption that would prove incorrect.
  3. systemctl daemon-reload: Instruct systemd to reload its unit files, picking up the newly copied vllm-glm5.service file. This is essential because the file was just placed in /etc/systemd/system/ moments earlier.
  4. systemctl enable vllm-glm5.service: Create the symlink that causes the service to start automatically on boot. This is the "productionalization" step — making the service survive reboots.
  5. echo "Service enabled": A confirmation message to verify step 4 succeeded.
  6. systemctl start vllm-glm5.service && echo "Service started": Actually launch the service, with a confirmation echo conditioned on success. The use of && between steps 3-6 means that if any step fails, the chain stops. This is a sensible design: if daemon-reload fails, there's no point trying to enable or start the service. The assistant was thinking defensively.

Assumptions Made

Several assumptions underpin this message, and some of them turned out to be incorrect:

Assumption 1: pkill would fully clean up the old vLLM process. The assistant assumed that matching the pattern vllm.entrypoints would identify and terminate any running vLLM server. However, as the subsequent messages reveal, this did not work completely. At <msg id=2025>, the service status shows inactive (dead) — the start failed. At <msg id=2029>, the user correctly diagnosed: "Seems previous vllm is still live preventing sysd one from starting." The pkill either missed a process (perhaps one running under a different pattern) or the killed process left behind stale GPU state that prevented the new instance from binding to the GPUs.

Assumption 2: Two seconds is enough cleanup time. The sleep 2 was a guess. In practice, vLLM processes holding 402GB of model weights across eight GPUs may take longer to fully release resources, especially if there are NCCL communicators or CUDA contexts that need to be torn down.

Assumption 3: systemd would surface clear error messages. The assistant expected that if the service failed, journalctl or systemctl status would show meaningful errors. But at <msg id=2026>, the journal shows "-- No entries --", indicating the service produced no output at all. This suggests the failure happened very early — perhaps before the ExecStart command even ran, or in a way that systemd couldn't capture.

Assumption 4: The service file was correct. The assistant had written the unit file at <msg id=2022> and copied it at <msg id=2023>, but had not validated its syntax or tested it in isolation. The service file's correctness was taken on faith.

Input Knowledge Required

To understand this message fully, one needs knowledge of:

Output Knowledge Created

This message produced several important outputs:

  1. A running systemd service (eventually, after a second attempt at <msg id=2027>): The service vllm-glm5.service was enabled and started, appearing as active (running) with PID 194110.
  2. A production deployment: The service was configured to auto-start on boot via the symlink in multi-user.target.wants, meaning the GLM-5 inference server would survive server reboots without manual intervention.
  3. A failure signal: The initial failure (inactive/dead status) created diagnostic output — the empty journal — that informed the next debugging steps. This failure was itself valuable knowledge, revealing that the process cleanup was incomplete.
  4. A validated configuration: The service file encoded the entire optimization journey — CUDAGraph enabled, NCCL_PROTO=LL, tensor parallelism of 8, float16 dtype, 8192 max sequence length, 90% GPU memory utilization — all the parameters that had been empirically determined over the preceding rounds.

The Thinking Process

The assistant's reasoning is visible in the structure of the command and in the sequence of messages leading up to it. The todo list at <msg id=2021> shows a clear progression: "Check current running server config and verify it's correct" (completed), "Document all patched files on the container" (completed), "Create systemd service unit for vllm-glm5" (in_progress), "Enable and test the service" (pending). The assistant was methodically working through a checklist, and message 2024 executes the final step.

The decision to use pkill rather than a more targeted kill (e.g., systemctl stop or kill with a specific PID) reflects an assumption that the old server was started manually (as indeed it was — the earlier sessions used direct python -m vllm.entrypoints... commands, not systemd). There was no systemd service to stop, so the assistant fell back to process-name matching.

The choice to chain commands with && rather than ; shows the assistant thinking about failure handling: if any step fails, the chain stops, and the final echo "Service started" only fires if everything succeeded. This creates a clear success/failure signal in the command output.

The 2>/dev/null on the pkill is another deliberate choice: the assistant anticipated that there might be no existing vLLM process to kill, and didn't want a spurious error message to pollute the output or break the chain (since pkill returns non-zero if no process matches, and without 2>/dev/null this could cascade).

Mistakes and Incorrect Assumptions

The most significant mistake was the assumption that pkill -f "vllm.entrypoints" would reliably terminate the old server. In practice, the old process survived or left behind state that blocked the new one. The user's observation at <msg id=2029> — "Seems previous vllm is still live preventing sysd one from starting" — was correct. The assistant had to re-issue the enable and start commands separately at <msg id=2027> (without the pkill step, since by then the issue was understood) to get the service running.

A secondary mistake was not checking whether the service actually started before moving on. The assistant issued the command and presumably saw the echo output (or lack thereof), but the next message (<msg id=2025>) shows it checking systemctl status and finding the service inactive. This suggests the assistant did not wait for confirmation or did not parse the output carefully enough.

The sleep 2 assumption was also optimistic. For a model of this size, GPU resource cleanup likely takes longer than two seconds, especially when NCCL communicators and CUDA IPC handles need to be released.

Conclusion

Message 2024 is a pivotal moment in the GLM-5 deployment saga. It represents the culmination of an intense debugging and optimization effort — the point at which the assistant stopped tuning and started deploying. The command is a microcosm of the assistant's approach throughout the session: methodical, defensive, and checklist-driven. It also illustrates a common pattern in production deployments: the first attempt often fails, and the failure itself provides the information needed to succeed on the second try.

The message's real significance lies not in what it accomplished (a service that initially failed to start) but in what it represented: the transition from "can we make this work?" to "this works, now make it reliable." After fixing two bugs in the Triton MLA attention backend, correcting GGUF dequantization shard ordering, and optimizing NCCL protocols, the assistant had earned the right to productionalize. The systemd service, once running, would serve as the foundation for whatever application the user intended to build on top of GLM-5 — a 744B parameter model running on eight Blackwell GPUs, delivering 57 tokens per second.