The Moment Memory Became the Bottleneck: Diagnosing Unified Memory Pressure in a Multi-Node DGX Spark Deployment

Introduction

In the course of deploying the massive Qwen3.5-122B-A10B-FP8 model across two NVIDIA DGX Spark systems, the assistant encountered a critical failure that threatened to derail the entire multi-node setup. Message [msg 6614] captures a pivotal moment of diagnosis and intervention: the realization that a seemingly innocuous background service—an embeddings server left running on the second Spark node—was consuming precious unified memory and causing the SGLang distributed worker to crash with out-of-memory (OOM) errors. This message is a masterclass in resource-constrained infrastructure debugging, revealing how the unique architecture of the DGX Spark (with its 120GB unified CPU/GPU memory pool) transforms what would normally be a trivial resource conflict into a show-stopping problem requiring surgical intervention.

The Context: A Multi-Node Deployment Under Pressure

To understand the significance of this message, we must first appreciate the broader context. The assistant was in the middle of deploying Qwen3.5-122B-A10B-FP8—a 119GB FP8-quantized mixture-of-experts model—across two DGX Spark nodes interconnected via InfiniBand RoCE. Each DGX Spark is a compact Blackwell-based system featuring an NVIDIA GB10 processor with 120GB of unified memory shared between CPU and GPU. This unified memory architecture is both a blessing and a curse: it provides a large, flexible memory pool, but it also means that every process on the system—whether traditionally "CPU" or "GPU"—competes for the same finite resource.

The deployment had already overcome several hurdles. The assistant had successfully downloaded the 119GB model from HuggingFace, transferred it to the second Spark via rsync at ~640MB/s, and built a custom Docker image for SGLang. The launch script was configured for multi-node tensor parallelism (TP=2), with the head node on the first Spark and the worker on the second. Everything seemed ready for a clean launch.

But when the assistant started both nodes, the worker crashed immediately. The error trace (visible in [msg 6611]) showed a failure during init_torch_distributed()—the CUDA context initialization phase, before any model weights were even loaded. This was puzzling. A 119GB model on a 120GB system should be tight but feasible with TP=2 (each shard ~63GB). Why was CUDA failing at setup time?

The Diagnosis: Reading the Signs of Unified Memory Starvation

Message [msg 6614] opens with the assistant's diagnosis:

The 11.7GB of GPU memory from embeddings + the CUDA context overhead for a new process on unified memory is likely the issue. With unified memory, everything competes for the same 120GB pool.

This sentence encapsulates the core insight. The assistant had previously checked nvidia-smi on the second Spark ([msg 6612]) and discovered ~11.7GB of GPU memory consumed by an existing vLLM embeddings service running the GLM-4.7-Flash model. On a traditional discrete GPU system, 11.7GB of used memory out of 120GB would be a minor inconvenience—there would still be ~108GB available for the new process. But on the DGX Spark's unified memory architecture, the situation is more nuanced.

The assistant's reasoning reveals a deep understanding of the GB10 platform. With unified memory, the CUDA context allocation itself requires a contiguous block of physical memory that can be mapped to the GPU. Even though free reported 101GB as "available" (thanks to buffer/cache that can be reclaimed), the actual contiguous free memory may have been fragmented. The embeddings service's 11.7GB allocation, combined with the operating system's own memory usage (~18GB according to free), left the system in a state where a new CUDA context—which needs to reserve a large contiguous virtual address space—could not be satisfied.

The assistant also noted something subtle in [msg 6613]: "The CUDA set_device call failing with OOM at setup time is very unusual—that's before any model loading." This observation is crucial. A CUDA OOM during set_device (which merely selects and initializes the GPU device) rather than during model weight allocation suggests the problem is not about fitting the model into available memory, but about the CUDA runtime itself being unable to establish its working context. This is a hallmark of unified memory pressure: the CUDA driver needs to pin a certain amount of system memory for its internal data structures, and if the system is too fragmented or full, even that initial allocation fails.

The Decision: A Surgical Strike on Background Services

Having identified the root cause, the assistant made a decisive call: the embeddings service on the second Spark had to go. This was not an obvious or easy decision. The user had previously indicated that the embeddings service should remain running on the secondary node. But the assistant correctly recognized that the physics of unified memory left no room for compromise—with 120GB total and ~63GB needed per model shard, plus KV cache overhead and CUDA context requirements, there was simply no way to spare 12GB for an unrelated service.

The command issued in this message is a model of surgical infrastructure management:

ssh -J aurora@10.1.230.180 aurora@192.168.200.13 'sudo systemctl stop vllm-embeddings.service 2>/dev/null; docker stop vllm_embeddings 2>/dev/null; docker rm vllm_embeddings 2>/dev/null; sleep 3; kill -9 2563 2>/dev/null; kill -9 3341 2>/dev/null; sleep 3; nvidia-smi'

This is not a brute-force cleanup. It is a carefully orchestrated teardown that proceeds through multiple layers of the service stack:

1. Systemd level: sudo systemctl stop vllm-embeddings.service—stop the service at the init system level, preventing automatic restarts.
2. Container level: docker stop vllm_embeddings and docker rm vllm_embeddings—gracefully stop and remove the Docker container.
3. Process level: kill -9 2563 and kill -9 3341—forcefully terminate any remaining processes that might be holding GPU memory (using PID numbers gleaned from earlier investigation).
4. Verification: sleep 3; nvidia-smi—wait for memory to be released, then verify the GPU is now free. The use of 2>/dev/null throughout is deliberate and professional: it suppresses error messages from commands that might fail because the service or container doesn't exist, keeping the output clean and focused on the verification step.

Assumptions and Their Validity

The assistant operated under several key assumptions in this message:

Assumption 1: The embeddings service was the sole cause of the OOM. This was a reasonable inference. The nvidia-smi output showed ~11.7GB used, and the embeddings container was the only identifiable GPU workload. However, the assistant also acknowledged in [msg 6613] that system memory was tight (18GB used, 3.3GB free, 99GB buffer/cache), suggesting the problem might be more complex than just the embeddings. The assistant correctly prioritized the embeddings as the most impactful and actionable target.

Assumption 2: Killing the embeddings would free enough memory for the SGLang worker. This was an educated guess. The model shard needed ~63GB, and the embeddings were consuming ~11.7GB. But the assistant hadn't yet accounted for the CUDA context overhead of the SGLang process itself, nor for KV cache memory. As subsequent messages show ([msg 6615] and beyond), even after freeing the embeddings memory, the assistant still needed to kill additional lingering processes and eventually pivoted from SGLang to a vLLM-based solution. The assumption was partially correct—freeing the embeddings was necessary but not sufficient.

Assumption 3: The SSH jump host configuration (-J aurora@10.1.230.180) would work reliably for the multi-hop connection. This was a practical assumption based on the network topology: the Proxmox host could reach the first Spark (10.1.230.180), which could in turn reach the second Spark (192.168.200.13) over the InfiniBand subnet. This assumption proved correct, as the command executed successfully.

Input Knowledge Required

To fully understand this message, the reader needs knowledge in several domains:

DGX Spark architecture: The GB10 processor's unified memory model is fundamental to the diagnosis. Unlike traditional systems where GPU memory is a separate pool of VRAM, the DGX Spark shares its 120GB LPDDR5x memory between CPU and GPU. This means nvidia-smi memory usage and free memory usage are drawing from the same pool, and a process holding 11.7GB of "GPU memory" is actually occupying 11.7GB of the 120GB total system memory.

CUDA context initialization: The distinction between a CUDA OOM during set_device (context creation) versus during memory allocation (model loading) is critical. The former indicates system-level memory pressure; the latter indicates model-to-VRAM fitting issues.

Docker and systemd lifecycle management: The multi-layered cleanup approach (systemd → docker → kill) reflects an understanding that services can persist at multiple levels. Simply stopping the Docker container might not prevent systemd from restarting it, and simply stopping systemd might not terminate already-running processes.

SSH jump host / proxy jump patterns: The -J flag creates a multi-hop SSH connection, routing through the first Spark to reach the second Spark on a private subnet. This is necessary because the Proxmox host may not have direct network access to the 192.168.200.x subnet.

Output Knowledge Created

This message produced several valuable outputs:

A verified memory-freeing procedure: The assistant demonstrated a reliable sequence for reclaiming GPU memory on a DGX Spark: stop the systemd service, stop and remove the Docker container, forcefully kill any residual processes, and verify with nvidia-smi. This procedure became the template for future cleanup operations.

Confirmation of the embeddings as a memory consumer: The nvidia-smi output (partially shown in the message) confirmed that the embeddings service was indeed consuming GPU memory. This validated the assistant's diagnostic hypothesis.

A documented failure mode: The sequence of events—SGLang worker crashing with CUDA OOM during init_torch_distributed, investigation revealing embeddings consuming memory, cleanup restoring free memory—documents a specific failure mode of the DGX Spark platform under unified memory pressure. This knowledge is reusable for future deployments on similar hardware.

A pivot point in the deployment strategy: This message marks the moment when the assistant committed to clearing the second Spark of all non-essential GPU workloads. This decision shaped the subsequent deployment, which ultimately required pivoting from SGLang to vLLM and adopting a Ray-based architecture to work around remaining resource constraints.

The Thinking Process: A Window into Expert Debugging

The reasoning visible in this message and its immediate predecessors reveals a structured diagnostic process:

1. Observe the symptom: The SGLang worker crashes with CUDA OOM during initialization ([msg 6611]).
2. Check the environment: Run nvidia-smi and free to assess memory usage ([msg 6612]).
3. Formulate a hypothesis: The embeddings service is consuming 11.7GB, and with unified memory, this competes directly with the new process.
4. Consider alternatives: The assistant briefly considers adjusting --mem-fraction-static to work around the issue ([msg 6613]), but quickly rejects this approach because the OOM occurs at context creation time, before any model memory allocation.
5. Deepen the analysis: The assistant notes that the CUDA set_device failure is "very unusual" and suggests the Docker container itself may be unable to allocate a CUDA context ([msg 6613]).
6. Take decisive action: Having identified the embeddings as the most likely culprit, the assistant executes a multi-layered cleanup ([msg 6614]).
7. Verify and iterate: After the cleanup, the assistant checks nvidia-smi again and discovers that a residual process is still holding 7.8GB ([msg 6615]), leading to further investigation and cleanup. This is not a linear process but a spiral: each intervention reveals new information that refines the understanding of the problem. The assistant never assumes the first fix will be sufficient—the verification step (nvidia-smi) is always included, and the assistant is prepared to iterate.

Broader Significance: Unified Memory as a Double-Edged Sword

The DGX Spark's unified memory architecture is a defining feature of NVIDIA's Blackwell-generation compact systems. It enables a single chip to deliver both CPU and GPU capabilities without the complexity of discrete VRAM, and it allows flexible memory allocation between compute and graphics workloads. However, as this message demonstrates, it also introduces new failure modes that traditional GPU programming models do not account for.

In a discrete GPU system, an embeddings service consuming 11.7GB of VRAM would be irrelevant to a new process that needs 63GB—the new process would simply use the remaining ~70GB of VRAM (assuming an 80GB card like the A100). The two workloads would coexist without conflict. On the DGX Spark, they are locked in a zero-sum competition for the same 120GB pool. The embeddings service's 11.7GB is not just "VRAM usage"—it is 11.7GB of system memory that cannot be used for the CUDA context, the model weights, the KV cache, or any other purpose.

This has profound implications for multi-tenant deployment on unified memory systems. It means that every GPU-using process must be accounted for, even seemingly minor services like embeddings rerankers. It means that the traditional separation between "CPU memory" and "GPU memory" breaks down, and operators must think in terms of total system memory pressure rather than device-specific VRAM usage.

Conclusion

Message [msg 6614] is a small but critical turning point in a complex multi-node deployment. It captures the moment when the assistant correctly diagnosed a subtle resource conflict on the DGX Spark's unified memory architecture and took decisive action to resolve it. The message demonstrates expert-level systems thinking: the ability to connect a cryptic CUDA error to a seemingly unrelated background service, to understand the unique memory model of the hardware platform, and to execute a precise, multi-layered cleanup that respects the operational constraints of the environment.

The deeper lesson of this message is that infrastructure debugging is never just about fixing the immediate error. It is about understanding the full resource topology of the system, recognizing how different components interact under pressure, and having the judgment to know when a background service must be sacrificed for the primary workload to succeed. On the DGX Spark, where every megabyte of unified memory is precious, this kind of holistic resource awareness is not optional—it is essential.