Reconnaissance Before Deployment: Exploring Dual DGX Spark Systems for Multi-Node Inference

Introduction

In any complex infrastructure deployment, the first step is never configuration—it is reconnaissance. Before a single model weight is loaded or a single inference server is configured, the operator must understand the terrain. This principle is on full display in message 6555 of an opencode coding session, where an AI assistant embarks on deploying the Qwen3.5-122B-A10B-FP8 model across two NVIDIA DGX Spark systems. The message represents the critical opening gambit of a multi-node deployment: parallel exploration of both target machines combined with an external search for deployment recipes, all dispatched in a single coordinated round.

This message is not glamorous. It contains no breakthrough optimization, no clever code patch, no architectural insight. Yet it is precisely this kind of methodical groundwork that separates successful deployments from those that collapse under unanticipated conditions. The assistant's choices in this message—what to probe, what to search for, and how to structure the exploration—reveal a deep understanding of the challenges inherent in deploying large language models on novel hardware configurations.

Context: The Mission

The message arrives in response to a user directive at <msg id=6553>: "New mission: same model to deploy, different hardware - 2x GB10 dgx sparks; 120gb of unified memory each, sm121, arm cpu, ib interconnect." The model in question is Qwen3.5-122B-A10B-FP8, a 122-billion-parameter mixture-of-experts model that the assistant has already been working with extensively on a Proxmox-hosted RTX PRO 6000 Blackwell setup (see segments 37–41 of the conversation). Now the same model must be deployed on fundamentally different hardware: two NVIDIA DGX Spark systems, each built around the GB10 system-on-a-chip with 120GB of unified memory, an ARM Cortex-X925 CPU, and an SM121 Blackwell GPU, connected via InfiniBand.

The user provides specific access details: the head node at 10.1.230.180 accessible via SSH, and the second node reachable only through a jump host (ssh -J aurora@10.1.230.180 aurora@192.168.200.13). The user also mentions that "there were previous models setup to inference TP on both for speed and memory efficiency," suggesting existing infrastructure that may need to be torn down or repurposed.

What the Message Actually Does

The assistant dispatches three tool calls in parallel within a single round:

Tool 1: Exploration of the head node. A bash command that collects system identity (kernel version, OS release), memory status, CPU architecture, and GPU state via nvidia-smi. This is a comprehensive system inventory probe, gathering the essential facts needed before any deployment decision.

Tool 2: Exploration of the second node. An identical probe, but routed through a jump host SSH configuration (-J flag). The assistant needs to confirm that the second node is reachable, has the same software stack, and has available GPU and memory resources.

Tool 3: Web search for deployment recipes. An Exa search for "NVIDIA DGX Spark GB10 LLM inference setup guide SGLang vLLM deployment 2025". The assistant is not assuming it knows the best approach—it is actively seeking community knowledge about deploying on this specific hardware.

The Results: A Tale of Two Systems

The results paint a vivid picture of two systems in very different states.

Head node (spark-1a32): Running Linux 6.14.0-1015-nvidia on aarch64, Ubuntu 24.04.3 LTS. It has 119 GiB of total memory, but critically, 104 GiB is already in use, with only 5.7 GiB free and 11 GiB available. Swap usage is 7 GiB out of 15 GiB. This is a system under significant memory pressure—something is already running and consuming the vast majority of its 120 GB unified memory pool. The nvidia-smi output is truncated in the message (shown as ...), but the memory numbers alone signal that the head node cannot simply accept a new model deployment without first freeing resources.

Second node (spark-5be3): Same kernel and OS version, but in dramatically better shape. Only 18 GiB used out of 119 GiB, with 70 GiB free and 100 GiB available. Swap is untouched. The nvidia-smi output shows NVIDIA driver version 580.95.05, confirming the GPU stack is functional. This node has ample room for a new workload.

Search results: The Exa search returns two relevant results. The first is a NVIDIA Developer Forums post titled "A 'Sane' Inference Stack for GB10 (Need Contributors!)" from February 2026, suggesting community-driven efforts to establish best practices for this hardware. The second is a GitHub repository for vLLM setup on DGX Spark. These results will inform the assistant's deployment strategy in subsequent rounds.

Why This Message Matters: The Art of Reconnaissance

The assistant's approach in this message embodies several principles of robust infrastructure engineering.

First, parallel exploration. Rather than probing one node, then the other, then searching the web sequentially, the assistant dispatches all three probes simultaneously. This is efficient but also strategic: by the time the next round begins, the assistant has a complete picture of both systems and the external knowledge landscape. This parallel dispatch is possible because the tools are independent—none depends on the output of another.

Second, comprehensive signal collection. The bash commands are carefully constructed to gather multiple dimensions of system state in a single invocation: kernel identity (for driver compatibility), OS version (for package compatibility), memory pressure (for resource planning), CPU architecture (for binary compatibility), and GPU state (for CUDA/capability verification). Each of these dimensions could become a blocking issue later, and discovering them early prevents wasted effort.

Third, humility through external search. The assistant does not assume it knows the optimal deployment strategy for DGX Spark. Despite having deep experience with the same model on different Blackwell hardware (RTX PRO 6000), the assistant recognizes that the GB10 system-on-a-chip, ARM CPU, and unified memory architecture may require different approaches. The search for community recipes is an acknowledgment that novel hardware configurations often have undocumented quirks that only early adopters have encountered.

Assumptions Embedded in This Message

Every reconnaissance mission carries implicit assumptions, and this message is no exception.

The assistant assumes that SSH access works as described—that the jump host configuration (-J) will successfully route to the second node, that key-based authentication is configured, and that the remote shells will execute the commands without issues. These are reasonable assumptions given the user provided explicit access instructions, but they are not guaranteed.

The assistant assumes that nvidia-smi is the right tool for GPU inspection. On a DGX Spark with unified memory, the traditional GPU/CPU memory boundary is blurred—the 120 GB is a unified pool accessible to both the ARM CPU and the Blackwell GPU. The nvidia-smi output may not fully capture this architecture's memory dynamics, but it is still the most reliable first-pass tool.

The assistant assumes that the web search results will be relevant and actionable. The Exa search engine is specified by the user as a preferred tool, but the quality of results depends on the query formulation and the available indexed content.

Perhaps the most significant assumption is that the existing workloads on the head node (consuming 104 GiB of memory) can be identified and safely terminated. The assistant does not yet know what is running—that investigation will come in subsequent rounds.

What Knowledge Is Required to Understand This Message

To fully grasp the significance of this message, the reader needs several layers of context.

Hardware architecture knowledge: Understanding that DGX Spark is built around NVIDIA's GB10 system-on-a-chip, which integrates a Blackwell GPU (SM121) with an ARM Cortex-X925 CPU and 120 GB of unified LPDDR5x memory. This is not a traditional discrete GPU setup—the memory is shared between CPU and GPU, which changes deployment constraints.

Networking topology knowledge: Understanding the InfiniBand interconnect between the two Sparks, the IP subnet scheme (192.168.200.x for IB, 10.1.x.x for external), and the jump host SSH pattern where the second node is only reachable through the first.

Model characteristics knowledge: Knowing that Qwen3.5-122B-A10B is a 122-billion-parameter mixture-of-experts model with 10 billion active parameters per token, requiring approximately 60-70 GB of memory in FP8 precision. This means it cannot fit on a single Spark's 120 GB unified memory if other workloads are present, necessitating tensor parallelism across both nodes.

Session history knowledge: Understanding that the assistant has been working with this same model on different Blackwell hardware (RTX PRO 6000 GPUs in a Proxmox VM), and has already resolved numerous deployment challenges including flash-attn compilation, FP8 KV cache accuracy, and MTP speculation tuning. This experience informs the assistant's approach but does not guarantee the same solutions will work on the GB10 architecture.

What Knowledge Is Created by This Message

This message produces several distinct pieces of output knowledge that will inform all subsequent deployment decisions.

System state inventory: A complete snapshot of both nodes' operating systems, kernel versions, memory pressure, and GPU driver status. The head node's critical memory shortage (104 GiB used out of 119 GiB) is the most actionable finding—it immediately flags that resource cleanup is a prerequisite for deployment.

Connectivity verification: Confirmation that both nodes are reachable and responsive, and that the jump host SSH pattern works correctly. The second node responds with full system information, validating the network path.

GPU stack confirmation: The second node shows NVIDIA-SMI 580.95.05, confirming that NVIDIA drivers are installed and the GPU is accessible. The head node's nvidia-smi output is partially truncated in the message but the command did not return "no nvidia-smi," suggesting it also has driver support.

Community knowledge signals: The search results point to two relevant resources—a NVIDIA Developer Forums discussion and a GitHub repository—that the assistant can consult for deployment guidance specific to the DGX Spark platform.

The Thinking Process Visible in This Message

While the message itself is a straightforward dispatch of tool calls, the reasoning behind it is revealed through the assistant's preceding message at <msg id=6554>, where it lays out a structured plan:

"I should start by exploring what's already configured on both Spark systems, looking for any existing deployment guides or recipes for GB10 DGX Spark setups, then map out the deployment strategy."

This is accompanied by a todo list with four items, the first being "Explore both GB10 DGX Spark systems - check hardware, existing setup, software" (status: in_progress) and the second being "Search for GB10 DGX Spark inference recipes and SM121 compatibility info" (status: pending).

The assistant is executing a deliberate, staged plan. The reconnaissance phase (this message) feeds into a planning phase (analyzing results), which feeds into an execution phase (downloading the model, setting up networking, configuring the inference server). The parallel dispatch of system exploration and web search reflects an understanding that both internal state and external knowledge are needed before the next decision point.

Conclusion

Message 6555 is a textbook example of how to begin a complex multi-node deployment. It is methodical, parallel, and humble—gathering facts from both target systems while simultaneously seeking community knowledge. The results immediately reveal a critical constraint (the head node's memory pressure) that will shape everything that follows. This reconnaissance-first approach prevents the assistant from charging into a deployment that would fail due to resource exhaustion, network incompatibility, or architectural mismatch.

In the broader narrative of the coding session, this message marks the transition from single-node optimization work on the Proxmox RTX PRO 6000 setup to a fundamentally new challenge: multi-node deployment on ARM-based Blackwell systems with unified memory. The assistant's disciplined approach to exploration sets the stage for the complex work ahead—downloading a 119 GB model, configuring Ray clusters, solving NCCL networking issues, and ultimately achieving ~27 tok/s throughput across two DGX Spark nodes. But none of that would be possible without the reconnaissance performed in this single, unassuming message.