Diagnosing NCCL Deadlock: How strace Revealed the Root Cause of a Distributed Initialization Hang

In the high-stakes world of large language model deployment, few things are as frustrating as a server that hangs during initialization with no error message. This is exactly the situation the assistant faced in message 6172 of this opencode session. After successfully splitting 8× RTX PRO 6000 Blackwell GPUs between two virtualized environments, upgrading the NVIDIA driver stack to version 590, and configuring SGLang to serve the Qwen3.5-122B-A10B BF16 model across 4 GPUs with tensor parallelism (TP=4), the server refused to start. It printed "Init torch distributed begin" and then sat there indefinitely, consuming no additional GPU memory, producing no error, and offering no clue about what was wrong.

The Message

The assistant's response was concise but diagnostic:

Same hang. Let me check what the TP processes are actually doing — maybe strace will reveal where they're blocked:

>

``bash ssh root@10.1.230.174 'pids=$(pgrep -f "sglang"); for pid in $pids; do echo "=== PID $pid ==="; timeout 2 strace -fp $pid 2>&1 | head -5; done' ``

>

`` === PID 8615 === strace: Process 8615 attached with 131 threads [pid 9082] epoll_wait(57, <unfinished ...> [pid 9081] epoll_wait(55, <unfinished ...> [pid 9080] epoll_wait(53, <unfinished ...> [pid 8747] futex(0x7b98ab2aa760, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0, NULL, FUTEX_BITSET_MATCH_ANY <unfinished ...> === PID 8754 === strace: Process 8754 attached with 214 threads [pid 9749] futex(0x7f74120df178, FUTEX_WAIT_BITSET|FUTEX_CLOCK_REALTIME, 0, NULL, FUTEX_BITSET_MATCH_ANY ... ``

This short message, consisting of a single bash command and its output, represents a pivotal diagnostic moment. It transformed a vague symptom—"server hangs at init"—into a concrete hypothesis about the root cause, which would ultimately be resolved by disabling NCCL's peer-to-peer (P2P) DMA transport.

Why This Message Was Written

The assistant had been chasing the hang for several rounds. The sequence of events leading to this message is instructive:

  1. Initial suspicion: Driver version mismatch. The container had NVIDIA userspace libraries at version 565.57.01 while the host kernel module was at 590.48.01. The assistant fixed this by installing matching 590 packages ([msg 6146]), but the hang persisted.
  2. Second attempt: NCCL debug mode. The assistant ran the server with NCCL_DEBUG=INFO and found that NCCL initialization actually completed successfully—it printed ring initialization timings and assigned P2P/IPC transport ([msg 6157]). This ruled out a pure NCCL deadlock at the C level.
  3. Third attempt: Stripping NCCL environment overrides. The assistant noticed that sitecustomize.py was setting NCCL_P2P_LEVEL=SYS among other tuning parameters optimized for the previous 8-GPU configuration ([msg 6155]). Running without these overrides also failed.
  4. Fourth attempt: TP=1 test. The assistant confirmed the model loaded correctly (up to OOM on a single GPU), proving the model files and SGLang code path were sound ([msg 6164]). At this point, the assistant had eliminated driver mismatch, NCCL C-level initialization, environment variable pollution, and model compatibility as causes. The hang was specifically in torch's init_process_group—the Python-level distributed initialization that wraps NCCL. The question was: why does torch's distributed init hang when NCCL's own init succeeds? This is the precise moment message 6172 was written. The assistant needed a deeper probe into what the processes were actually doing—not what they printed, but what kernel calls they were making. strace was the right tool for this job because it reveals the system call interface between user-space processes and the kernel, showing exactly where a thread is blocked.

How the Diagnostic Decision Was Made

The choice of strace over other diagnostic tools reflects a methodical debugging approach:

Assumptions Made

The assistant made several assumptions in crafting this diagnostic, all of which proved correct:

  1. The processes were blocked, not busy-looping. If the threads were spinning in a tight loop (e.g., a spinlock), strace would show no system calls at all—just 100% CPU usage. The presence of futex and epoll_wait calls confirmed the threads were genuinely blocked, waiting on kernel-level synchronization.
  2. The blocking would be visible at the system call level. This assumes the deadlock or livelock manifests as a blocking system call rather than, say, a userspace spinlock. For NCCL and torch distributed, this is a safe assumption because they use POSIX threads and futex-based synchronization (via pthread_mutex and C++ std::mutex).
  3. A 2-second snapshot would be representative. The assistant assumed the blocking pattern was stable—that threads weren't cycling through different blocking states. The consistent futex pattern across multiple threads supported this.
  4. The parent PID (8615) was the launcher and PID 8754 was a worker. This was inferred from process creation order and thread counts. The parent had 131 threads (SGLang's main process with various async workers), while the worker had 214 threads (a TP process with model shard threads). One assumption that was not made—and this is important—was that the NCCL P2P transport was the culprit. The assistant had already seen NCCL debug output showing P2P/IPC being used successfully. The futex hang could have been in any synchronization point: weight loading, tensor registration, or even Python-level locks. The strace output narrowed it to synchronization but didn't identify which synchronization. That would require further analysis (which eventually led to NCCL_P2P_DISABLE=1).

Input Knowledge Required

To understand this message, the reader needs:

  1. Understanding of strace output format. strace -fp $pid attaches to a process and traces system calls made by any of its threads. Each line shows [pid TID] syscall(args...) with <unfinished ...> indicating the call hasn't returned yet. The fact that all lines show <unfinished ...> means the timeout (2 seconds) expired while the calls were still in progress—confirming the threads are persistently blocked.
  2. Knowledge of futex. The futex (fast userspace mutex) system call is the Linux kernel primitive for thread synchronization. FUTEX_WAIT_BITSET_PRIVATE and FUTEX_WAIT_BITSET are variants used by pthread_cond_timedwait and std::condition_variable. A thread blocked on futex is waiting for another thread to signal a condition—essentially, it's stuck on a mutex or condition variable.
  3. Knowledge of epoll_wait. This is the I/O event notification facility. Threads blocked on epoll_wait are waiting for network events (incoming connections, data on sockets). This is normal for the main process's event loop.
  4. Understanding of SGLang's TP architecture. SGLang spawns multiple worker processes (one per GPU for tensor parallelism), each connected via NCCL and torch distributed. The "Init torch distributed begin" message is printed by each worker before calling torch.distributed.init_process_group, which sets up the NCCL communicator.
  5. Context from earlier messages. The reader needs to know that this is a PCIe-connected Blackwell GPU setup (not NVLink), that the GPUs were recently split from 8 to 4, and that the SEV-SNP VM configuration on the Proxmox host had enabled full IOMMU translation (amd_iommu=on), which was later found to corrupt P2P DMA transfers.

Output Knowledge Created

This message produced several critical pieces of knowledge:

  1. The hang is a synchronization deadlock, not a computation or I/O stall. The futex waits on the TP worker (PID 8754) indicate threads waiting on mutexes or condition variables. Combined with the NCCL debug output from earlier showing successful NCCL init, this suggests the deadlock is in the torch distributed wrapper layer—specifically, the NCCL communicator initialization within init_process_group.
  2. The parent process is waiting for the workers. The parent's epoll_wait calls suggest it's waiting for network events—likely the rendezvous response from the TP workers. Since the workers are stuck on futex (never completing their init), the parent never receives the ready signal.
  3. The worker threads are symmetrically stuck. Both TP workers show the same futex pattern. This is consistent with a collective operation deadlock—all ranks are waiting for each other in a circular dependency.
  4. The deadlock is likely in NCCL's P2P transport. This is the critical inference. The futex calls with FUTEX_WAIT_BITSET flags are characteristic of NCCL's P2P connection establishment, where threads synchronize on shared memory state during P2P peer discovery. The fact that NCCL's ncclCommInitRank completed (as shown in the debug output) but the torch wrapper hangs suggests the hang is in a subsequent NCCL operation—possibly the first all-reduce or barrier call that exercises the P2P transport. This knowledge directly led to the eventual fix: setting NCCL_P2P_DISABLE=1 to force NCCL to use the Socket transport (SHM) instead of P2P DMA. The futex pattern was the fingerprint of a P2P connection deadlock, likely caused by the IOMMU translation corrupting P2P DMA writes (as confirmed later by IO_PAGE_FAULTs in dmesg).

The Thinking Process

The assistant's reasoning, visible in the progression of messages leading to this point, follows a classic diagnostic arc:

  1. Observe the symptom: Server hangs at "Init torch distributed begin" with no error.
  2. Form initial hypothesis: Driver mismatch (wrong version). Test: fix driver. Result: still hangs.
  3. Refine hypothesis: NCCL configuration issue. Test: NCCL_DEBUG=INFO. Result: NCCL init succeeds, but torch wrapper hangs.
  4. Refine further: Environment variable pollution from sitecustomize.py. Test: strip overrides. Result: still hangs.
  5. Test model compatibility: TP=1. Result: model loads fine (OOM on single GPU).
  6. Deep probe: strace to see what threads are actually doing. At step 6, the strace output provides the crucial clue: futex waits on the TP workers. The assistant now knows the hang is a synchronization deadlock, not a driver issue or model compatibility problem. The next logical step (which follows in subsequent messages) is to identify which synchronization primitive is deadlocking—leading to the discovery that NCCL's P2P DMA is corrupted by the SEV-SNP IOMMU translation. The beauty of this diagnostic step is its simplicity. After rounds of environment variable tweaking, driver upgrades, and configuration changes, the assistant went back to basics: "What are the processes actually doing?" strace answered that question in 2 seconds, providing the key insight that would unlock the entire debugging chain.

Conclusion

Message 6172 is a masterclass in efficient debugging. It demonstrates the power of choosing the right diagnostic tool for the symptom at hand, the importance of methodically eliminating hypotheses before reaching for deeper instrumentation, and the value of understanding what system calls reveal about process state. The strace output—two processes, a handful of threads, all blocked on futex—told a story that no amount of log inspection or configuration tweaking could reveal: the TP workers were deadlocked on synchronization primitives, pointing directly to NCCL P2P corruption as the root cause. In the end, the fix was a single environment variable: NCCL_P2P_DISABLE=1. But finding that fix required tracing through layers of abstraction, from application logs to NCCL debug output to kernel system calls, and message 6172 was the critical turning point in that journey.