The Ghost in the GPU: Diagnosing Orphaned Memory After Process Termination

In the middle of an intense performance tuning session for a large language model deployment, the assistant encounters a frustrating and all-too-familiar problem: GPU memory that refuses to be freed even after killing all visible processes. Message 76 captures a single investigative step in this debugging journey — a moment where the assistant pauses the performance tuning workflow to ask a fundamental question: "Hmm, still memory allocated. Let me check what's using it."

This brief message, consisting of a single bash command and the beginning of its output, is a pivot point. It represents the transition from assuming that standard cleanup commands work as expected to the realization that something more insidious is happening — orphaned processes, leaked CUDA contexts, or zombie workers that survive their parent's death.

The Road to This Moment

To understand why this message matters, we must trace the path that led here. The session began with a monumental effort: setting up an ML environment on Ubuntu 24.04 with eight NVIDIA RTX PRO 6000 Blackwell GPUs, resolving flash-attn build issues, and deploying the GLM-5-NVFP4 model using vLLM. By the time we reach message 76, the assistant has already achieved a working deployment running at approximately 57 tokens per second — a respectable figure for an 8-way tensor-parallel MoE model with 78 layers and 256 routed experts.

The performance tuning phase had been methodical. The assistant tested NCCL_NTHREADS=64 (result: 57.6 tok/s, no improvement), then NCCL_BUFFSIZE=1048576 (result: 57.5 tok/s, also no change). After each test, the server was killed and restarted — a process that took roughly 10 minutes per iteration due to model loading time. The assistant had just examined the GPU topology via nvidia-smi topo -m (message 70), discovering that the eight GPUs were connected purely through PCIe Gen5 with no NVLink, split across two NUMA nodes. This discovery led to a hypothesis: the bottleneck was allreduce latency rather than bandwidth, since each decode step required approximately 156 small allreduce operations.

Armed with this insight, the assistant decided to test NCCL_ALGO=Ring versus NCCL_ALGO=Tree to see which algorithm performed better for small messages on this PCIe topology. The kill command was issued in message 74:

pkill -9 -f "vllm.entrypoints" 2>/dev/null; sleep 3; fuser -k /dev/nvidia* 2>/dev/null; sleep 5

The Moment of Discovery

Message 75 returned the result of the cleanup attempt — and it was alarming:

0, 92953 MiB
1, 92953 MiB
2, 92953 MiB
3, 92955 MiB
4, 92953 MiB
5, 92953 MiB
6, 92953 MiB
7, 92953 MiB

All eight GPUs still showed approximately 92.9 GiB of memory allocated. The pkill and fuser commands had apparently failed to release the memory. This is the direct trigger for message 76.

The assistant's response — "Hmm, still memory allocated. Let me check what's using it" — reveals several layers of reasoning. First, there is genuine surprise. The assistant had used what should have been a comprehensive cleanup sequence: pkill -9 -f "vllm.entrypoints" to kill any process matching the API server pattern, followed by fuser -k /dev/nvidia* to kill any process holding NVIDIA device files open. The fact that memory remained allocated after both commands suggests that either (a) the processes were not actually killed, (b) the processes were killed but their CUDA contexts leaked, or (c) there were processes that neither command caught.

The Investigation Strategy

The assistant's chosen next step is telling: instead of immediately trying more aggressive kill commands or rebooting the GPUs, it runs nvidia-smi | head -40. This is a diagnostic first move — look before you leap. The head -40 flag is particularly interesting because it captures the full header information of nvidia-smi, which includes the process table at the bottom. The assistant wants to see not just memory usage numbers but the process list — which PIDs are holding memory, what their names are, and potentially what state they're in.

The command output shown in the message is truncated (ending with ...), but the visible portion confirms the driver version (590.48.01), CUDA version (13.1), and the GPU table header. The actual process list would appear below the separator line, which is cut off in the message.

Assumptions and Their Consequences

This message exposes several assumptions that had been guiding the assistant's actions:

Assumption 1: pkill -9 is sufficient to terminate vLLM processes. The assistant had been using pkill -9 -f "vllm.entrypoints" as its standard cleanup command. However, vLLM uses a multiprocess architecture: the API server process spawns an EngineCore process and multiple Worker_TP processes. The pkill pattern only matched the parent API server process (the one started with python3 -m vllm.entrypoints.openai.api_server). The child processes — VLLM::EngineCore and VLLM::Worker_TP0 through VLLM::Worker_TP7 — did not match the pattern and survived.

Assumption 2: fuser -k /dev/nvidia* would catch any remaining processes. The fuser command identifies processes using specified files or sockets and can kill them. However, its behavior depends on the process state and timing. If the orphaned workers were in an uninterruptible sleep state or if the NVIDIA device files were already released by the time fuser ran, the command would have no effect.

Assumption 3: The cleanup sequence was comprehensive enough. The assistant had added fuser -k after discovering orphaned workers in message 65, but the sequence still failed because the orphaned processes from the previous server instance were not the same as the ones from the current instance. Each restart created new PIDs, and the cleanup commands were pattern-based rather than state-based.

Input Knowledge Required

To understand this message fully, the reader needs knowledge of several domains:

NVIDIA GPU architecture and memory management: Understanding that GPU memory is allocated per-process and persists until the process terminates or explicitly frees it. CUDA contexts can leak if processes are killed ungracefully.
vLLM process architecture: Knowing that vLLM uses a multi-process model with an API server, EngineCore, and worker processes. The workers hold the model weights in GPU memory.
Linux process management: Understanding pkill, fuser, process states, orphan processes, and how child processes survive parent termination.
The NCCL tuning context: Recognizing that this cleanup is a necessary prerequisite for testing NCCL_ALGO=Ring, and that each failed cleanup adds ~10 minutes of restart time.
The nvidia-smi tool: Knowing that nvidia-smi without arguments shows a summary including process list, and that head -40 captures the full header including the process table.

Output Knowledge Created

This message produces several important pieces of knowledge:

Confirmation of persistent memory allocation: The 92.9 GiB per GPU confirms that the cleanup failed, and the problem is systematic (all 8 GPUs affected equally).
Timing information: The timestamp "Fri Feb 20 19:00:06 2026" provides a reference point for correlating with log files and understanding the duration of the session.
Driver and CUDA version confirmation: The output confirms the environment is using NVIDIA driver 590.48.01 and CUDA 13.1, which is relevant for understanding NCCL behavior and potential compatibility issues.
A diagnostic starting point: The truncated output sets up the next steps — the assistant will need to run more targeted queries to identify the specific PIDs holding memory.

The Thinking Process

The assistant's reasoning in this message is visible in its concise phrasing. "Hmm, still memory allocated" conveys surprise mixed with analytical curiosity — the "hmm" suggests the assistant did not expect this outcome and is recalibrating its mental model. "Let me check what's using it" indicates a shift from action (kill/cleanup) to investigation (diagnose/understand).

The choice of nvidia-smi | head -40 rather than nvidia-smi --query-compute-apps=pid,name,used_memory --format=csv (which was used later in message 78) is interesting. The assistant opts for the human-readable summary view first, suggesting it wants a broad overview before drilling into specifics. This is a classic debugging pattern: get the full picture, then narrow down.

The truncated output (the ... at the end) is significant for the article's reader — it means the assistant did not see the full process list in this message. The actual discovery of the orphaned workers happens in subsequent messages (78-80), where the assistant runs more specific queries and finds VLLM::Worker_TP0 through VLLM::Worker_TP7 still running with PIDs in the 1847xx range — a different set from the ones killed earlier.

Broader Significance

This message, while brief, illustrates a fundamental challenge in ML infrastructure debugging: the gap between intention and effect in process management. The assistant's cleanup commands were logically sound — kill the parent process, then kill anything holding GPU device files — but failed because of the specific architecture of vLLM's process tree and the timing of process termination.

The problem of "zombie GPU memory" is endemic in ML engineering. CUDA contexts are tied to processes, and when a parent process spawns children that hold GPU allocations, those children must be individually terminated. Pattern-based killing (like pkill -f) is fragile because it depends on matching process names or command lines, which may not include all relevant processes.

The assistant's methodical approach — test a hypothesis, observe the result, investigate anomalies — is the core of effective debugging. Message 76 is the "investigate anomalies" step in action. It's a reminder that even in automated sessions, the most valuable debugging tool is the willingness to say "Hmm, that's unexpected" and dig deeper.

Conclusion

Message 76 captures a single, seemingly mundane moment in a complex ML deployment session. But within that moment lies a wealth of insight about process management, GPU memory allocation, and the art of debugging distributed systems. The assistant's concise question — "Let me check what's using it" — embodies the debugging mindset: when results don't match expectations, stop, observe, and understand before acting.

The message also serves as a cautionary tale about assumptions in automation. Each layer of cleanup commands (pkill, fuser, sleep) was added in response to previous failures, but the cumulative sequence still had gaps. The lesson is that process management in multi-GPU environments requires not just killing processes but verifying that memory is actually freed — and having fallback strategies for when standard cleanup fails.

In the messages that follow, the assistant will discover the orphaned workers, kill them individually by PID, and successfully free all GPU memory. But message 76 is the turning point — the moment the assistant realizes that its cleanup strategy has a blind spot, and begins the investigation that will lead to a more robust approach.