From Manual Process to Production Service: Systemd Deployment of DeepSeek-V4-Flash on Blackwell

Introduction

In the lifecycle of any serious ML inference deployment, there comes a pivotal moment when the server must graduate from a manually-managed background process to a properly supervised system service. Message [msg 12699] captures exactly that transition. After an intensive optimization campaign spanning custom MMA attention kernels, PD-disaggregation architecture, and KV cache tuning, the assistant creates a systemd unit file to manage the SGLang DeepSeek-V4-Flash server on an 8× RTX PRO 6000 Blackwell (sm_120) machine. This message is deceptively brief — a single bash command and a short reasoning block — but it represents the culmination of dozens of earlier decisions and the formalization of the deployment's operational model.

Context: The Road to Production

To understand why this message exists, one must trace the path that led here. The assistant had spent the preceding rounds (segments 63–68 of the conversation) engaged in a deep optimization campaign for the DeepSeek-V4-Flash model on Blackwell GPUs. The journey included building custom MMA sparse-MLA decode kernels with Triton tl.dot tensor-core operations ([chunk 68.0]), discovering and fixing an indexer O(max_context) bottleneck that delivered a ~17× throughput gain ([chunk 68.1]), deploying prefill-decode disaggregation across 8 GPUs with systemd services, and setting up Prometheus/Grafana monitoring ([chunk 68.2]).

In the immediate preceding messages ([msg 12691]–[msg 12698]), the assistant had been working on two concrete tasks: verifying that 512k context length worked correctly, and maximizing KV cache capacity by raising the memory fraction. The KV cache had been increased from 1.63M tokens at mem-fraction 0.70 to 2.58M tokens at mem-fraction 0.85 — a 58% gain with 13 GB/GPU of safety headroom remaining ([msg 12696]). The server was confirmed listening on 0.0.0.0:30001 and returning correct responses ("Paris" for the capital of France) ([msg 12697]). The environment script (dsv4_nccl_env.sh) was verified to contain the necessary CUDA_HOME, PATH, and NCCL tuning variables ([msg 12698]).

At this point, the server was running as a manual nohup process launched from an interactive SSH session. This is fragile: if the SSH session drops, if the machine reboots, or if the process crashes, the server goes down and requires manual intervention. The assistant recognized this and pivoted to systemd — the standard Linux service manager — to make the deployment production-ready.

The Message Itself: Creating the Systemd Unit

The subject message contains two parts: an Agent Reasoning block and a bash command. The reasoning is concise but reveals the assistant's decision-making:

The environment script handles all the CUDA and PATH setup cleanly, so that should work fine with systemd. Now I'm setting up the systemd unit file at /etc/systemd/system/sglang-dsv4.service, then reloading the daemon, stopping the manual server, enabling and starting the service, and verifying it's running properly.

This reasoning shows the assistant performing a dependency check: it verifies that the environment script (dsv4_nccl_env.sh) is self-contained — it sets CUDA_HOME, PATH, NCCL_* variables, PYTORCH_CUDA_ALLOC_CONF, and OMP_NUM_THREADS. Because systemd services run in a minimal environment (they don't inherit the user's shell environment), the script must be fully self-sufficient. The assistant's brief check confirms this requirement is met, and the reasoning explicitly notes this as the precondition for proceeding.

The bash command then creates the unit file using a heredoc:

cat > /etc/systemd/system/sglang-dsv4.service << "EOF"
[Unit]
Description=SGLang DeepSeek-V4-Flash NVFP4 (sm120 MMA+indexer, 512k ctx) on :30001
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
WorkingDirectory=/root
ExecStart=/bin/bash /root/serve_dsv4_final.sh
Restart=always
RestartSec=15
TimeoutStartSec=900
TimeoutStopSec=90
KillMode=control-group
LimitNOFILE=1048576

[Install]
WantedBy=multi-user.target
EOF

Design Decisions Embedded in the Unit File

Every line of this unit file encodes deliberate engineering choices. The Description field is more than documentation — it encodes the deployment's identity: the model (DeepSeek-V4-Flash), the quantization (NVFP4), the custom kernel features (sm120 MMA+indexer), the context length (512k), and the port (30001). This makes systemctl status output immediately informative for anyone inspecting the machine.

The After=network-online.target and Wants=network-online.target directives ensure the service waits for network connectivity before attempting to start. This is critical because the server binds to 0.0.0.0:30001 and may need to initialize NCCL communication — starting before the network is ready would cause failures.

Type=simple is the correct choice here. The ExecStart runs a bash script that ultimately exec's the Python process. Since the script uses exec (as verified in [msg 12694] where the script was rewritten with exec /root/venv_sglang211/bin/python -m sglang.launch_server ...), the Python process replaces the bash process, and systemd tracks the Python PID directly.

Restart=always with RestartSec=15 means the service will automatically recover from crashes after a 15-second cooldown. This is essential for production inference services where uptime matters. The 15-second delay prevents rapid restart loops if the server crashes immediately on startup.

TimeoutStartSec=900 (15 minutes) is a generous timeout that reflects the reality of loading a 671B-parameter model with NVFP4 quantization across 4 GPUs. The model loading, CUDA graph capture, and KV cache allocation can take several minutes, and a shorter timeout would cause systemd to kill the process prematurely.

TimeoutStopSec=90 gives the server 90 seconds to shut down gracefully, allowing ongoing requests to complete or be drained.

KillMode=control-group ensures that if systemd needs to stop the service, it kills the entire process group (the Python server and any child processes), preventing orphaned processes.

LimitNOFILE=1048576 raises the file descriptor limit to 1 million. This is a practical necessity for an inference server handling many concurrent requests — each open connection, each CUDA context, and each log file consumes file descriptors.

Assumptions and Their Validity

The assistant makes several assumptions in this message. First, it assumes that the environment script is truly self-contained for systemd purposes. The verification in [msg 12698] showed that dsv4_nccl_env.sh exports CUDA_HOME, PATH, and NCCL tuning variables, but it did not verify that the Python virtual environment's Python binary is on the PATH or that LD_LIBRARY_PATH includes the CUDA libraries. The ExecStart uses /bin/bash /root/serve_dsv4_final.sh, and the script itself uses exec /root/venv_sglang211/bin/python ... with an absolute path, so the Python binary is resolved correctly. However, the CUDA runtime libraries need to be findable — the script sources the env file which sets PATH to include CUDA_HOME/bin, but LD_LIBRARY_PATH is not explicitly set. The assistant implicitly assumes that the CUDA toolkit's libraries are in the system's default loader path or that the Python process handles this internally (which it does, since PyTorch links against the CUDA libraries at build time).

Second, the assistant assumes that the serve_dsv4_final.sh script is stable and won't require further modification. This is a reasonable assumption given that the preceding rounds had stabilized the configuration, but it does lock in the current set of flags. If the user later wants to change --mem-fraction-static, --context-length, or --cuda-graph-max-bs, they would need to modify both the script and restart the service.

Third, the assistant assumes that WorkingDirectory=/root is appropriate. This means the server's working directory is root's home, which contains the model files (under /root/models/) and the environment script. This is correct for this deployment but would be a security concern in a multi-tenant environment.

What This Message Achieves

The output knowledge created by this message is the systemd unit file itself — a concrete, deployable artifact that transforms the server from a fragile manual process into a managed service. The unit file encodes the operational knowledge of how to run this specific inference stack: the startup command, the resource limits, the restart policy, and the dependencies.

The message also creates a checkpoint in the deployment process. After this message, the assistant can proceed to stop the manual server, enable and start the systemd service, and verify that it comes up correctly. The service will survive reboots (WantedBy=multi-user.target enables it at boot), crashes (Restart=always), and will log to journald for monitoring.

The Thinking Process

The Agent Reasoning block reveals a structured decision process. The assistant first performs a precondition check (environment script self-containment), then outlines the execution plan in chronological order: create the unit file, reload the daemon, stop the manual server, enable and start the service, verify. This is classic operational thinking — verify dependencies, then execute a well-defined sequence of steps.

The reasoning also shows the assistant learning from earlier mistakes. In [msg 12693]–[msg 12694], the assistant had struggled with a combined pkill + sed + nohup launch command that failed because the backgrounded process was killed when the SSH session closed. The systemd approach avoids this entirely — systemd manages the process lifecycle independently of any SSH session, which is precisely why production services use it.

Conclusion

Message [msg 12699] is a small but critical step in the deployment journey. It represents the transition from development-mode operation (manual process management, interactive debugging) to production-mode operation (supervised service, automatic recovery, boot-time startup). The unit file's design reflects the specific requirements of a large language model inference server: long startup times, high file descriptor usage, network dependency, and crash resilience. While the message appears simple on the surface — a single file creation — it encodes the operational maturity that separates a demo from a deployment.