The Symbol Table Tells the Truth: Diagnosing an ABI Mismatch with nm

Message Summary

In this brief but pivotal message (index 5304 in the conversation), the assistant performs a surgical diagnostic operation on a failing CUDA 13 stack upgrade. After multiple rounds of installing and reinstalling PyTorch, sgl-kernel, and flashinfer in various combinations, the sgl-kernel library continues to fail with an "undefined symbol" error. The assistant pivots from trial-and-error reinstallation to low-level binary analysis, using nm to inspect the symbol tables of both the failing shared object and the PyTorch library it links against. The result is a precise identification of an ABI (Application Binary Interface) mismatch: the sgl-kernel binary expects a function signature with an int parameter, while the installed PyTorch provides one with unsigned int — a one-character difference in the C++ mangled symbol name that explains the entire failure cascade.

The Message in Full

[assistant] Still failing with the same undefined symbol. Let me check if the .so was actually replaced when we reinstalled sgl-kernel, or if there's a stale file. Let me also check what symbols are expected:
[bash] ssh root@10.1.230.174 'nm -D ~/ml-env/lib/python3.12/site-packages/sgl_kernel/sm100/common_ops.abi3.so 2>/dev/null | grep c10_cuda_check_implementation'
                 U _ZN3c104cuda29c10_cuda_check_implementationEiPKcS2_ib

[bash] ssh root@10.1.230.174 'nm -D ~/ml-env/lib/python3.12/site-packages/torch/lib/libc10_cuda.so 2>/dev/null | grep c10_cuda_check'
0000000000044380 T _ZN3c104cuda29c10_cuda_check_implementationEiPKcS2_jb

Context: The CUDA 13 Upgrade Saga

To understand why this message was written, we must step back and appreciate the broader context. The assistant is working on a high-performance inference server running on a machine with 8× NVIDIA RTX PRO 6000 Blackwell GPUs connected via PCIe. The project has been pursuing speculative decoding with EAGLE-3 to improve throughput, but has been blocked by a fundamental problem: the verify step of speculative decoding was slower than running the base model directly, making speculation a net-negative.

Earlier in segment 35, the assistant had identified two Blackwell-native optimizations — FlashInfer allreduce fusion and Torch symmetric memory — that could potentially fix the verify-step bottleneck. However, both required CUDA 13 (the Blackwell-optimized CUDA release) rather than the CUDA 12.8 that was currently installed. The assistant therefore embarked on a major infrastructure upgrade: installing CUDA 13.0.1 toolkit alongside the existing 12.8, then rebuilding the entire PyTorch + SGLang software stack on top of it.

The upgrade process (messages 5273–5303) had been a rollercoaster. The assistant successfully installed CUDA 13.0.1, then installed a PyTorch nightly build (2.12.0.dev20260226+cu130), only to discover that the sgl-kernel wheel from the cu130 index was incompatible with the nightly ABI. The assistant then pivoted to stable PyTorch 2.10.0+cu130, which should have been compatible with the sgl-kernel wheel — but the same error persisted. This is where message 5304 begins: "Still failing with the same undefined symbol."

The Reasoning: Why nm?

The assistant's decision to use nm — the Unix symbol table inspection tool — reflects a critical shift in debugging strategy. Up to this point, the approach had been combinatorial: try different PyTorch versions, reinstall packages in different orders, force-reinstall to overwrite stale files. But the error message was opaque: "Could not load any common_ops library" with an undefined symbol reference. The assistant needed to answer two concrete questions:

  1. Was the .so file actually replaced? Perhaps the reinstallation of sgl-kernel had not actually updated the shared object file, and a stale binary compiled against the old PyTorch was still sitting on disk.
  2. What symbol was actually missing? The error message didn't specify which symbol was undefined. By using nm -D (which lists dynamic symbols) and grepping for a likely candidate (c10_cuda_check), the assistant could get the exact mangled C++ symbol name and compare it against what PyTorch actually exports. This is a textbook example of moving up the abstraction ladder: when high-level package management fails, drop to the binary level where the truth is unambiguous.

The Discovery: One Character, Worlds Apart

The two nm commands reveal the root cause with devastating clarity. The sgl-kernel shared object expects:

U _ZN3c104cuda29c10_cuda_check_implementationEiPKcS2_ib

The U prefix means "undefined" — this symbol is referenced by the .so but not defined within it. It must be resolved at load time from a linked library. The mangled C++ name decodes as: c10::cuda::c10_cuda_check_implementation(int, char const*, char const*, unsigned int, bool).

But PyTorch's libc10_cuda.so provides:

0000000000044380 T _ZN3c104cuda29c10_cuda_check_implementationEiPKcS2_jb

The T prefix means "text" — this symbol is defined (exported) by this library. The decoded signature is: c10::cuda::c10_cuda_check_implementation(unsigned int, char const*, char const*, unsigned int, bool).

The difference is the fourth character of the mangled name: i vs j. In the C++ ABI mangling scheme used by GCC/Clang, i represents int (a signed 32-bit integer) while j represents unsigned int (an unsigned 32-bit integer). The sgl-kernel was compiled against a version of PyTorch where c10_cuda_check_implementation took an int as its first parameter, but the installed PyTorch 2.10.0+cu130 uses unsigned int.

This is a classic ABI compatibility break: the function logically does the same thing, but the C++ type system considers int and unsigned int to be different types, so the linker generates different mangled names. The dynamic linker cannot resolve the reference because the symbol names don't match — even though the actual machine code might be functionally identical.

Assumptions and Their Consequences

Several assumptions led to this point. The assistant initially assumed that the sgl-kernel wheel from the cu130 index would be compatible with any PyTorch build for CUDA 13 — but the wheel was built against a specific PyTorch version (likely 2.10.x stable), and the nightly 2.12.0 had introduced ABI changes. When the nightly failed, the assistant assumed that switching to stable PyTorch 2.10.0+cu130 would fix the issue, since it was the same major version. But the error persisted, which was puzzling — and correctly prompted the deeper investigation.

The assistant also implicitly assumed that the package reinstallation was actually replacing the .so file. The nm check confirmed that it had been replaced (the symbol was present, just mismatched), ruling out the "stale file" hypothesis.

A subtle but important assumption was that the sgl-kernel wheel was correctly compiled for CUDA 13 at all. The abi3.so suffix indicates a stable ABI (PEP 384) Python extension, meaning it should work across Python versions — but the C++ ABI linking against PyTorch's internal libraries is a separate concern. The wheel was "for cu130" in the sense that it was compiled against CUDA 13 runtime libraries, but it was also compiled against a specific PyTorch C++ ABI that didn't match the installed version.

Input Knowledge Required

To fully understand this message, a reader needs:

Output Knowledge Created

This message creates several valuable pieces of knowledge:

  1. The exact ABI mismatch: The i vs j difference is now documented. This is the root cause of the sgl-kernel load failure.
  2. Confirmation that the .so was correctly replaced: The symbol is present (not stale), just mismatched.
  3. A clear path forward: The assistant now knows they need a PyTorch version where c10_cuda_check_implementation takes int (not unsigned int). This means either finding a sgl-kernel wheel compiled against the exact PyTorch 2.10.0+cu130 ABI, or using a PyTorch version that matches the sgl-kernel wheel's expectations.
  4. A diagnostic methodology: The use of nm for ABI debugging is demonstrated and can be applied to future compatibility issues.

The Thinking Process

The assistant's reasoning unfolds in a logical sequence:

Step 1 — Frame the problem: "Still failing with the same undefined symbol." The error is reproducible and unchanged despite reinstalling packages. This rules out transient issues and suggests a fundamental incompatibility.

Step 2 — Generate hypotheses: Two specific hypotheses are stated: (a) the .so file wasn't actually replaced (stale file), and (b) the symbol expectations don't match. These are testable with nm.

Step 3 — Test hypothesis (a): The first nm command checks what symbols the sgl-kernel .so expects. The presence of the c10_cuda_check symbol confirms the file was rebuilt/replaced (the symbol is there, just unresolved). If it were a stale file from the cu128 era, it might have had a different symbol entirely.

Step 4 — Test hypothesis (b): The second nm command checks what PyTorch actually exports. The symbol is present (confirmed by the T flag), but the mangled name differs in one character.

Step 5 — Interpret the result: The i vs j difference is immediately recognizable to anyone familiar with C++ name mangling. The assistant doesn't need to decode it explicitly — the one-character difference is sufficient evidence of an ABI break.

What's notable is what the assistant doesn't do. They don't try yet another reinstallation. They don't rebuild sgl-kernel from source. They don't file a bug report. They simply gather the evidence and present it. The message ends with the two nm outputs and no further commentary — the data speaks for itself.

Why This Message Matters

This message is the diagnostic turning point for the entire CUDA 13 upgrade. Before it, the assistant was stuck in a loop of reinstalling packages with no understanding of why they failed. After it, the path forward becomes clear: the assistant needs to find a PyTorch version whose ABI matches the sgl-kernel wheel. In the subsequent messages (not shown here), this discovery leads to the solution: using PyTorch 2.9.1+cu130 (an older stable release that matches the sgl-kernel ABI) instead of 2.10.0+cu130.

The broader lesson is about debugging methodology. When high-level tools (pip, uv, package indices) fail to produce a working configuration, dropping to the binary level with nm, objdump, or ldd can reveal the exact incompatibility. The symbol table doesn't lie — it shows precisely what each binary expects and provides. The one-character difference between i and j is a tiny thing, but it was the difference between a broken stack and a working one.

In the context of the larger project, this message represents the final obstacle before the CUDA 13 upgrade succeeds. Once the ABI mismatch is resolved, the assistant can proceed to enable FlashInfer allreduce fusion and Torch symmetric memory, which together transform the EAGLE-3 speculative decoding from a net-negative 54.1 tok/s to a net-positive 96.1 tok/s — a 77.6% improvement. But none of that would have been possible without first understanding why the sgl-kernel wouldn't load.