The Dockerfile That Bridged Two Worlds: Building a Qwen3.5-Compatible vLLM Image for Multi-Node DGX Spark Deployment

In the sprawling infrastructure odyssey of deploying a 122-billion-parameter Qwen3.5 model across two NVIDIA DGX Spark nodes, one message stands out as a masterclass in pragmatic problem-solving. Message <msg id=6682> captures the moment when the assistant, after a series of failed attempts involving entrypoint conflicts, shell escaping disasters, and missing model registries, finally constructs a working Docker image that bridges two incompatible worlds: the Ray-based multi-node infrastructure of the vllm-node image, and the Qwen3.5 model support found only in the community-built hellohal2064/vllm-qwen3.5-gb10 image. This single bash command—a carefully crafted heredoc that builds a Dockerfile on a remote ARM-based DGX Spark—represents the culmination of a debugging chain that reveals deep truths about container orchestration, model serving architecture, and the art of making incompatible systems cooperate.

The Problem: Two Worlds That Cannot Meet

To understand why this message was written, one must first understand the infrastructure landscape. The assistant was tasked with deploying the Qwen3.5-122B-A10B-FP8 model across two DGX Spark nodes—NVIDIA's compact ARM-based systems built around the GB10 Blackwell GPU (SM121 architecture), each with 120GB of unified memory, connected via InfiniBand RoCE. The existing deployment infrastructure consisted of a custom vllm-node Docker image and a launch-cluster.sh script that orchestrated Ray-based multi-node execution. This infrastructure worked flawlessly for earlier models like Qwen3, but it was built on vLLM 0.14—a version that predated Qwen3.5 model support.

Enter the hellohal2064/vllm-qwen3.5-gb10 image, a community-built container running vLLM 0.17.1rc1 that had been specifically patched to support the Qwen3.5 architecture, including the MoE (Mixture-of-Experts) variant, MTP (Multi-Token Prediction) support, and the custom reasoning parser. This image could load and serve the model on a single node. But it had its own entrypoint script that assumed a single-node deployment, and it lacked the Ray-based multi-node orchestration that the launch-cluster.sh script provided.

The assistant had already tried and failed at multiple bridging strategies. Running the hellohal2064 image directly with launch-cluster.sh failed because the image's entrypoint conflicted with the cluster script's run-cluster-node.sh. Overriding the entrypoint with --entrypoint="" caused the container to fail because the cluster script's helper binary wasn't present in the foreign image. Building a hybrid image by simply copying model files into vllm-node failed because vLLM 0.14's model registry (registry.py) didn't know about the new Qwen3_5ForConditionalGeneration and Qwen3_5MoeForConditionalGeneration architecture classes.

The most recent attempt, in <msg id=6679>, had tried to inline a Python registry-patching script within a Dockerfile RUN command, only to be spectacularly defeated by zsh's aggressive glob expansion on the remote host. The shell interpreted parentheses and quotation marks as glob patterns, producing a cascade of "no matches found" errors that rendered the entire command unusable.

The Solution: Separating Concerns

Message <msg id=6682> represents the assistant's third attempt at the Dockerfile approach, and it demonstrates a critical lesson in infrastructure engineering: when a command becomes too complex to execute inline, factor out the logic into a separate artifact.

The assistant had already written the patch script as a standalone Python file (patch_vllm_qwen35.py) in <msg id=6680> and copied it to the remote node via scp in <msg id=6681>. Now, in the subject message, the assistant constructs a Dockerfile that:

  1. Starts from vllm-node — preserving the Ray-based multi-node infrastructure, the correct entrypoint, and the run-cluster-node.sh script that launch-cluster.sh depends on.
  2. Copies Qwen3.5 model files from the hellohal2064 image using Docker's multi-stage COPY --from syntax. Four files are extracted: the model implementation (qwen3_5.py), the MTP extension (qwen3_5_mtp.py), and two configuration classes (qwen3_5_moe.py and qwen3_5.py). These are placed into the vllm-node image's Python site-packages directory, overwriting nothing (since vLLM 0.14 has no such files).
  3. Runs the patch script — a self-contained Python file that modifies registry.py to add the Qwen3.5 architecture entries and patches the configs __init__.py to import the new configuration classes.
  4. Upgrades transformers to version 5.0 or later, which is required for the Qwen3.5 MoE architecture's configuration class to be recognized by HuggingFace's model registry. The build output tells a nuanced story. The Docker build succeeds: the model files are copied, the patch script runs successfully (printing "Registry patched"), and transformers is upgraded from 5.0.0rc3 to 5.5.0. But a warning appears: vLLM 0.14 requires transformers<5, and the build has installed transformers 5.5.0, which is incompatible according to pip's dependency resolver.

The Assumptions and Their Risks

This message is built on several assumptions, some more precarious than others:

The model files from vLLM 0.17 are compatible with vLLM 0.14. This is the most significant assumption. The qwen3_5.py model implementation likely imports from vLLM's internal modules (vllm.model_executor.layers, vllm.worker, etc.), and the API surface may have changed between vLLM 0.14 and 0.17. If the model file references classes or functions that don't exist in the older version, the import will fail at model loading time.

Patching registry.py is sufficient. The model registry in vLLM maps architecture strings (like "Qwen3_5MoeForConditionalGeneration") to Python module paths. But the model may also need to be registered in the tokenizer configuration, the weight loading logic, or the quantization backend. A missing registration anywhere in the chain could cause silent failures.

The transformers version conflict is benign. The pip warning about transformers>=5.0 being incompatible with vLLM's requirement for transformers<5 is a red flag. vLLM 0.14 may depend on specific transformer APIs that were removed or changed in 5.x. The assistant proceeds anyway, likely assuming that the Qwen3.5 model's configuration class requires the newer transformers registry format, and that the core vLLM serving logic doesn't critically depend on the older transformer APIs.

The vllm-node image has the same Python path structure. The Dockerfile hardcodes /usr/local/lib/python3.12/dist-packages/ as the target path. If the vllm-node image uses a different Python version or a virtual environment, the files would be placed in the wrong location.

Input Knowledge Required

To fully understand this message, one needs knowledge spanning several domains:

Docker multi-stage builds: The COPY --from=hellohal2064/vllm-qwen3.5-gb10 syntax extracts files from another image without running it—a technique for borrowing artifacts without inheriting the source image's entrypoint or runtime behavior.

vLLM's model architecture: vLLM uses a registry pattern where model architectures (strings like "Qwen3_5MoeForConditionalGeneration") are mapped to Python classes. The registry lives in vllm/model_executor/models/registry.py, and each model has its own Python file in the same directory. Configuration classes live in vllm/transformers_utils/configs/.

Qwen3.5 model specifics: The model has two architecture variants—a dense version (Qwen3_5ForConditionalGeneration) and a MoE version (Qwen3_5MoeForConditionalGeneration). It also supports MTP (Multi-Token Prediction) speculation, which requires additional model files (qwen3_5_mtp.py).

DGX Spark hardware constraints: The ARM Cortex-X925 CPU and 120GB unified memory mean that model loading is slow, and memory is tight. The 119GB FP8 model barely fits alongside the OS and display, making TP=2 (tensor parallelism across two nodes) essential.

Shell escaping differences: The previous failure in <msg id=6679> was caused by zsh's default globbing behavior on the remote host. The assistant correctly diagnosed this and moved the complex logic to a separate file.

Output Knowledge Created

This message produces a new Docker image tag vllm-node-qwen35 on the DGX Spark head node. The image combines:

The Thinking Process

The assistant's reasoning is visible in the structure of the command itself. The heredoc delimiter DEOF (instead of the more common EOF) is a deliberate choice to avoid conflicts with the 'EOF' quoting used in previous attempts. The Dockerfile is versioned as v3, acknowledging the two previous failed attempts. The patch logic is separated into a standalone file—a direct response to the zsh escaping failure.

The assistant also shows an understanding of what not to do. Earlier attempts tried to override the entrypoint, mount volumes over the model directory, or use environment variables to reconfigure the hellohal2064 image. Each failed for a different reason. The winning approach—building a new image that cherry-picks the best of both worlds—is the most architecturally sound, even if it requires more build time and carries the transformers version risk.

The pip warning about the transformers version conflict is noted but not acted upon. The assistant implicitly judges that the Qwen3.5 model's need for transformers>=5.0 outweighs vLLM 0.14's preference for <5.0. This is a calculated risk: if the model fails to load due to transformer incompatibility, the error will be immediately visible, and the approach can be adjusted. If the warning is benign, the deployment proceeds.

Conclusion

Message <msg id=6682> is a testament to the iterative nature of infrastructure engineering. It is not the first attempt, nor the most elegant, but it is the one that works. The assistant navigated a maze of incompatible entrypoints, missing registries, and shell escaping pitfalls to arrive at a solution that respects the constraints of both source images while combining their strengths. The Dockerfile it produces is a bridge between two worlds—one built for multi-node orchestration, one built for model compatibility—and its successful build marks the turning point in the Qwen3.5 deployment saga. The next message will transfer this image to the second node and launch the cluster, but the hardest architectural problem has already been solved.