The Pivot That Saved the Deployment: Diagnosing a Custom Docker Entrypoint on DGX Spark

Introduction

In the complex world of deploying large language models across distributed hardware, the smallest misconfiguration can derail an entire session. Message [msg 6664] captures one such moment—a crisp diagnostic pivot that transformed a failing container launch into a successful deployment of Qwen3.5-122B-A10B-FP8 across two NVIDIA DGX Spark nodes. This message, embedded in a multi-hour session spanning GPU topology reconfiguration, NCCL deadlocks, and multi-node orchestration, represents a textbook example of how deep infrastructure knowledge enables rapid problem-solving. The assistant, having just watched a freshly launched vLLM container crash without explanation, pauses, reads the error correctly, and adjusts the approach with surgical precision.

The Context: A Multi-Node Deployment Under Pressure

To understand why this message matters, one must appreciate the preceding struggle. The session had already navigated a gauntlet of infrastructure challenges: splitting eight Blackwell GPUs between LXC containers and virtual machines on a Proxmox host, diagnosing P2P DMA corruption under SEV-SNP IOMMU, and verifying that MTP speculation survived a system reboot. Now the assistant was tackling a new frontier—deploying Qwen3.5-122B-A10B-FP8, a 119-billion-parameter FP8 model, across two DGX Spark systems connected via InfiniBand RoCE.

The DGX Spark is NVIDIA's compact AI supercomputer, built around the GB10 Grace Blackwell system-on-chip with an ARM Cortex-X925 CPU and 120GB of unified memory. These are not standard servers—they run ARM64 architecture with SM121 Blackwell GPUs, and most AI inference frameworks require custom builds to function correctly. The assistant had already discovered that SGLang's multi-node NCCL initialization hung indefinitely on this hardware, forcing a pivot to vLLM. They found hellohal2064/vllm-qwen3.5-gb10, a community image specifically built for Qwen3.5 on DGX Spark, pulled it to both nodes, and attempted a single-node test.

That test failed. The container crashed immediately with no visible error.

The Message: A Moment of Recognition

Message [msg 6664] begins with the assistant's analysis:

The container has its own entrypoint script that sets up default args and then runs vllm serve. Our vllm serve command is getting duplicated. The image expects the model at /models/model and has its own launch wrapper. Let me adjust.

This is the crux of the entire message—a single sentence that encapsulates the diagnostic breakthrough. The assistant had been passing CLI arguments directly to vllm serve in the Docker command, assuming the image would behave like a standard vLLM container. But the specialized DGX Spark image includes a custom entrypoint script that wraps the vLLM server with its own defaults, including a fixed model path at /models/model and environment-variable-based configuration. By passing CLI arguments directly, the assistant was creating a command collision: the entrypoint script ran vllm serve with its defaults, and then the assistant's own vllm serve arguments were appended, resulting in a duplicated or malformed command that caused the immediate crash observed in the previous messages.

The Reasoning Process: What the Assistant Understood

The thinking visible in this message reveals several layers of understanding:

First, the assistant recognized the symptom. The container in [msg 6660] was launched with --rm and disappeared instantly—no logs, no container ID to inspect. The second attempt in [msg 6663] (without --rm) survived long enough to produce logs, which showed the entrypoint script's banner: "vLLM Server for NVIDIA GB10 (SM121 Blackwell)" with default configuration values. This was the critical clue. The banner displayed default settings (port 8000, max model length 8192) rather than the custom values the assistant had passed via CLI arguments, confirming that the entrypoint was overriding or ignoring those arguments.

Second, the assistant inferred the architecture of the container. The image's entrypoint script sets up GPU clock management (the [THERMAL] Setting GPU max clock to 2600 MHz... line), applies a Qwen3 reasoning parser patch, and then launches vllm serve internally. This is a common pattern for hardware-specific Docker images: the entrypoint handles initialization tasks (thermal configuration, kernel patching, environment validation) before handing off to the actual application. The assistant correctly deduced that the entrypoint was consuming the CLI arguments meant for vLLM, causing duplication.

Third, the assistant identified the correct configuration interface. Rather than CLI arguments, the image exposes configuration through environment variables (VLLM_MAX_MODEL_LEN, VLLM_GPU_MEMORY_UTIL, VLLM_HOST, VLLM_PORT) and a fixed mount point (/models/model). This is a design choice that simplifies deployment—users only need to set environment variables and mount their model, without worrying about the underlying vLLM argument syntax.

The Solution: Precision in Execution

The bash command in the message demonstrates the corrected approach:

docker rm -f vllm-qwen35-test 2>/dev/null
docker run -d \
  --gpus all --ipc=host --network host \
  --privileged \
  --ulimit memlock=-1 --ulimit stack=67108864 \
  -v /home/aurora/models/Qwen3.5-122B-A10B-FP8:/models/model:ro \
  --name vllm-qwen35-test \
  -e VLLM_MAX_MODEL_LEN=4096 \
  -e VLLM_GPU_MEMORY_UTIL=0.85 \
  -e VLLM_HOST=0.0.0.0 \
  -e VLLM_PORT=30000 \
  hellohal2064/vllm-qwen3.5-gb10

Every detail matters here. The model is mounted at /models/model (not the original path), matching the entrypoint's expectation. The --gpus all --ipc=host --network host --privileged flags provide GPU access, shared memory for NCCL, host networking for Ray, and the privileged mode needed for GPU clock management. The --ulimit settings prevent memory lock limits from interfering with NCCL. The environment variables replace the CLI arguments entirely.

The output confirms success: the container starts, applies the reasoning parser patch, and begins initializing the vLLM server. The log line Detected GPU with compute capability 12.1 (arch=121) confirms the Blackwell GPU is recognized, and the vLLM ASCII art banner begins to appear—a small but meaningful signal that the server is alive.

Assumptions and Corrections

This message reveals an important assumption that the assistant initially held: that a Docker image wrapping vLLM would accept the same CLI arguments as a bare vLLM installation. This is a reasonable assumption—most vLLM Docker images do work this way. But the DGX Spark image is specialized; it includes hardware-specific initialization that requires an entrypoint wrapper, and that wrapper changes the configuration interface.

The correction was not just about changing syntax. It required understanding the intent of the image designer: to simplify deployment by hiding vLLM's complex argument surface behind a few environment variables. Once the assistant recognized this design pattern, the fix was straightforward.

Input and Output Knowledge

Input knowledge required to understand this message includes: Docker container mechanics (entrypoints, volume mounts, environment variables), vLLM server configuration (tensor parallelism, GPU memory utilization, max model length), the DGX Spark hardware platform (ARM64, SM121 Blackwell GPU, unified memory), NCCL and InfiniBand networking concepts, and the specific model being deployed (Qwen3.5-122B-A10B-FP8, a 119B parameter MoE model with FP8 quantization).

Output knowledge created by this message includes: the correct launch configuration for the hellohal2064/vllm-qwen3.5-gb10 image, the fact that this image uses an entrypoint wrapper with environment-variable-based configuration, the model mount path expected by the entrypoint (/models/model), and the validation that this configuration successfully starts the vLLM server on DGX Spark hardware.

The Broader Significance

This message is a microcosm of the entire session's theme: resilient infrastructure management. The assistant could have tried many other approaches—debugging the crash with docker inspect, searching for error messages, rebuilding the image from source. Instead, they read the logs, recognized the entrypoint pattern, and adjusted in a single message. This efficiency comes from deep experience with containerized ML deployments, where custom entrypoints are common and environment-variable-based configuration is a frequent design choice.

The message also demonstrates the value of incremental testing. Rather than jumping straight to a multi-node deployment, the assistant tested single-node first. This caught the configuration issue early, before it could cause mysterious failures in the distributed setting. The single-node test in [msg 6660] failed, but it provided the diagnostic information needed to succeed in [msg 6664]. This pattern—test small, fail fast, learn, adjust—is fundamental to reliable infrastructure engineering.

Conclusion

Message [msg 6664] is a masterclass in diagnostic reasoning for containerized ML deployments. In a single turn, the assistant identified a subtle configuration mismatch, understood the container's architecture, corrected the approach, and validated the fix. The thinking process is visible and logical: observe the symptom (container crash), read the evidence (entrypoint banner with wrong defaults), infer the cause (CLI argument duplication), design the fix (environment variables and correct mount path), and execute with precision. This message, while brief, represents the kind of deep infrastructure knowledge that separates successful deployments from endless debugging sessions. It is the pivot that saved the deployment—the moment when a failing container became a working server, ready to serve one of the most advanced language models in existence.