The First Build Attempt: Compiling sgl-kernel for Blackwell SM120

Message Overview

In message [msg 5902] of this extended opencode session, the assistant issues a single bash command to build sgl-kernel from source on a remote machine equipped with 8× RTX PRO 6000 Blackwell GPUs. The command is a carefully constructed invocation of uv build that sets environment variables for CUDA 13.0, targets the SM120 architecture (Blackwell), enables FP4 kernel compilation, disables SM90-and-below code generation, and limits parallel compilation to 20 jobs for memory safety. The output shown is merely the first few debug lines from uv — the build has not yet completed, and in fact this particular invocation will fail due to a subtle environment mismatch. The message is a pivotal moment in a larger effort to deploy the Qwen3.5-397B-A17B-NVFP4 model, representing the transition from preparation (patching source code, upgrading dependencies) to execution (compiling the custom kernel library that makes the model actually run on Blackwell hardware).

The Command in Full

ssh root@10.1.230.174 'export CUDA_HOME=/usr/local/cuda-13.0 && export TORCH_CUDA_ARCH_LIST="12.0a" && export MAX_JOBS=20 && export CMAKE_BUILD_PARALLEL_LEVEL=20 && export CMAKE_ARGS="-DSGL_KERNEL_COMPILE_THREADS=16 -DENABLE_BELOW_SM90=OFF -DSGL_KERNEL_ENABLE_FP4=ON -DSGL_KERNEL_ENABLE_FA3=OFF" && export PATH="/root/.local/bin:$PATH" && cd /root/sglang-main/sgl-kernel && uv build --wheel -Cbuild-dir=build-sm120d . --no-build-isolation --verbose 2>&1 | tail -80'

The output shown is truncated debug output from uv 0.10.4 discovering the workspace, searching for Python interpreters, and beginning its resolution process — none of the actual CMake configuration or compilation output is visible in this message.

Why This Message Was Written: Context and Motivation

To understand why this message exists, one must trace the thread of decisions that led to this precise moment. The session had been working for many hours across multiple segments (see [segment 34] through [segment 39]) to deploy large language models on a cutting-edge hardware setup: eight NVIDIA RTX PRO 6000 Blackwell GPUs connected via PCIe, running Ubuntu 24.04 with CUDA 13.0. The user had recently directed the assistant to "update all to nightly," triggering a cascade of upgrades: PyTorch was bumped to 2.12.0.dev20260307+cu130, flashinfer was upgraded to 0.6.5, and the SGLang main branch was pulled to the latest commit.

The critical piece of the puzzle was sgl-kernel — the custom CUDA kernel library that SGLang uses for efficient inference operations including attention, MoE (Mixture-of-Experts) routing, and FP4 matrix multiplications. The pre-built wheels available on PyPI did not include SM120 (Blackwell) support, and the upstream SGLang repository had not yet merged the necessary patches for Blackwell compatibility. The assistant had therefore applied a set of patches authored by "catid" (a community contributor) to the CMakeLists.txt and the FA3 import fallback in flash_attn.py. These patches:

  1. Wrapped the cmake_policy(SET CMP0169 OLD) and cmake_policy(SET CMP0177 NEW) directives in if(POLICY ...) guards, since CMake 3.28 (installed on the system) does not know about CMP0177 (introduced in CMake 3.30).
  2. Added the CUDA 13 cccl include directory path to the build, since CUDA 13 restructured its header layout.
  3. Added a guard so that Flash Attention 3 (FA3) compilation could be explicitly disabled via -DSGL_KERNEL_ENABLE_FA3=OFF, since FA3 does not support SM120.
  4. Made the FA3 Python import fall back gracefully instead of raising a hard ImportError. With the patches applied and verified ([msg 5894], [msg 5895]), the assistant was ready to attempt the build — and message [msg 5902] is that attempt.

How Decisions Were Made

Every environment variable in this command reflects a deliberate choice informed by earlier failures and discoveries in the session:

CUDA_HOME=/usr/local/cuda-13.0: The system had been upgraded to CUDA 13.0 in segment 36 ([msg 36]), which was a prerequisite for Blackwell-native optimizations. The assistant explicitly points to this installation rather than relying on the system default.

TORCH_CUDA_ARCH_LIST="12.0a": This is the architecture string for Blackwell GPUs with tensor core support (the "a" suffix enables SM90a-like features on SM120). Earlier in the session, the assistant had discovered that the standard 12.0 architecture was insufficient for the FP4 kernels needed by the NVFP4 model.

MAX_JOBS=20 and CMAKE_BUILD_PARALLEL_LEVEL=20: These limits reflect a painful lesson learned much earlier in the session (segment 0) when compiling flash-attn with 128 parallel jobs exhausted system memory and caused the build to crash. The assistant is being conservative with parallelism.

CMAKE_ARGS="-DSGL_KERNEL_COMPILE_THREADS=16 -DENABLE_BELOW_SM90=OFF -DSGL_KERNEL_ENABLE_FP4=ON -DSGL_KERNEL_ENABLE_FA3=OFF":

Assumptions Made

The command makes several assumptions, some of which turn out to be incorrect:

That uv build --no-build-isolation uses the virtual environment's Python. This is the critical incorrect assumption. The PATH is set to include /root/.local/bin (where uv lives) and the system paths, but the virtual environment's bin directory (/root/ml-env/bin) is not in PATH. The uv build command, even with --no-build-isolation, discovers and uses the system Python interpreter (/usr/bin/python3) rather than the venv's Python (/root/ml-env/bin/python3). This means scikit-build-core and torch are not found, and the build will fail — though the failure is not visible in this message because the output is truncated.

That the build environment is fully prepared. The assistant assumes that scikit-build-core and nanobind are already installed in the build environment. In fact, the assistant had to install nanobind just one message earlier ([msg 5900]), and scikit-build-core was already present in the venv but not in the system Python.

That CMake will find CUDA 13 automatically via CUDA_HOME. While CUDA_HOME is set, CMake's CUDA detection also typically looks for nvcc in PATH. The PATH in this command includes /root/.local/bin and the system paths but does not include /usr/local/cuda-13.0/bin. This will cause CMake to fail to find the CUDA compiler in a subsequent attempt ([msg 5905]), though it is not the immediate issue here.

That uv build works correctly with --no-build-isolation and a custom build directory. The -Cbuild-dir=build-sm120d flag is a scikit-build-core configuration option passed through uv. This assumes that scikit-build-core will respect this setting and use the specified directory for the CMake build.

That the patches are sufficient for a successful build. The assistant had applied catid's patches and verified them syntactically, but had not yet tested whether they fully resolve all SM120 compilation issues. Subsequent messages reveal additional problems: the dlpack dependency uses an outdated cmake_minimum_required that conflicts with CMake 4.2.1's policy version requirements ([msg 5907]).

Mistakes and Incorrect Assumptions

The primary mistake in this message is the environment isolation issue. The assistant correctly identified in earlier messages that uv build should be used, but did not account for uv's behavior of finding its own Python interpreter rather than using the one from the activated virtual environment. This is a subtle but common pitfall with uv: even with --no-build-isolation, uv uses the Python interpreter it discovers (which may be the system Python) and then adds the venv's site-packages to sys.path. However, the build backend (scikit-build-core) is imported by that interpreter, and if scikit-build-core is not installed in the system Python, the import fails.

A secondary issue is that the PATH does not include the CUDA 13 bin directory. While CUDA_HOME is set, CMake's FindCUDA (and the built-in CUDA language support) typically searches PATH for nvcc as a fallback. Setting CUDACXX explicitly (as done in later messages) is the more robust approach.

The assistant also assumed that the build would proceed without issues after the patches were applied, but the dlpack cmake_minimum_required issue was not anticipated. This is understandable — dlpack is a third-party dependency fetched via FetchContent, and its build system incompatibility with CMake 4.2 is an upstream issue, not something the assistant could have predicted without attempting the build.

Input Knowledge Required

To understand this message, one needs:

Knowledge of the CUDA compilation toolchain: Understanding what TORCH_CUDA_ARCH_LIST does (it tells PyTorch's CUDA extension build system which GPU architectures to generate code for), what the 12.0a architecture string means (SM 12.0 with tensor core enhancements), and why ENABLE_BELOW_SM90=OFF is a valid optimization.

Knowledge of SGLang's architecture: Understanding that sgl-kernel is a separate Python package within the SGLang monorepo that contains custom CUDA kernels for attention, MoE, and quantization operations. The model being deployed (Qwen3.5-397B-A17B-NVFP4) uses FP4 quantization and requires specific kernel support.

Knowledge of the uv build tool: Understanding that uv build invokes the build system defined in pyproject.toml (in this case, scikit-build-core), what --no-build-isolation does, and how -C flags are passed through to the build backend.

Knowledge of Blackwell (SM120) architecture: Understanding that Blackwell is NVIDIA's next-generation GPU architecture after Hopper (SM90), that it requires CUDA 13+ for full support, and that many existing CUDA libraries (like Flash Attention 3) do not yet support it.

Context from the session: Knowing that this machine has 8× RTX PRO 6000 Blackwell GPUs connected via PCIe, that the assistant has been fighting with build issues throughout the session, and that previous attempts to optimize inference throughput had been blocked by the lack of SM120 kernel support.

Output Knowledge Created

This message creates the following knowledge:

A record of the build attempt: The command and its truncated output serve as a log entry documenting the first attempt to build sgl-kernel for SM120. Even though the build fails (as revealed in subsequent messages), this message captures the exact configuration used.

Evidence of the build environment state: The uv debug output reveals that uv 0.10.4 is installed, that it searches for Python interpreters in managed installations and the search path, and that it finds a cpython interpreter. This is useful diagnostic information.

A baseline for debugging: By seeing what failed (the system Python being used instead of the venv Python), the assistant can formulate a corrected approach in the next message ([msg 5903]). The failure mode is identified and addressed.

Confirmation of the toolchain versions: The output shows uv 0.10.4 and the workspace structure, confirming that the build tooling is modern and correctly configured at the project level.

The Thinking Process Visible in the Message

While the message itself is just a bash command and its output, the reasoning behind it is visible through the choices made:

The assistant is thinking in terms of defensive compilation: every flag is set explicitly rather than relying on defaults. MAX_JOBS=20 prevents OOM, ENABLE_BELOW_SM90=OFF prevents wasted compilation, SGL_KERNEL_ENABLE_FA3=OFF prevents a known failure, and SGL_KERNEL_ENABLE_FP4=ON ensures the necessary kernels are built.

The assistant is also thinking iteratively: this is not expected to be the final build command. The tail -80 pipe suggests the assistant expects verbose output and wants to see only the end (where errors or success would appear). The custom build directory build-sm120d suggests the assistant anticipates needing to rebuild and wants to keep artifacts organized.

The choice of uv build over pip install reflects an understanding of modern Python packaging tools. uv is faster and more reliable than pip for build operations, and its --no-build-isolation flag is the standard way to build packages that depend on complex system dependencies like CUDA and PyTorch.

The Broader Significance

This message, while seemingly just a failed build attempt, represents a critical transition point in the session. The assistant has completed all preparatory work — upgrading dependencies, patching source code, verifying toolchain versions — and is now taking the first concrete step toward producing a working Blackwell-compatible sgl-kernel. The failure that follows is not a setback but a discovery: it reveals the uv environment isolation issue that must be addressed before the build can succeed. In the subsequent messages ([msg 5903] through [msg 5908]), the assistant systematically diagnoses and fixes each issue, ultimately producing a working wheel that enables the Qwen3.5-397B-A17B-NVFP4 model to run at ~172 tok/s single-request throughput.

The message also illustrates a fundamental truth about systems engineering with cutting-edge hardware: the path from "theoretically compatible" to "actually running" is paved with failed build attempts, each revealing a new incompatibility or misconfiguration that must be addressed. The assistant's methodical approach — prepare, attempt, diagnose, fix, retry — is the only reliable way to navigate this landscape.