The Delicate Art of the No-Deps Install: Deploying Qwen3.5-397B-A17B-NVFP4 on SGLang Main

A Single Command That Encapsulates Weeks of Infrastructure Work

In the middle of a sprawling coding session spanning GPU driver debugging, CUDA toolkit upgrades, speculative decoding optimization, and production deployment, one message stands out for its deceptive simplicity. At index 5804, the assistant executes a single bash command:

ssh root@10.1.230.174 '~/.local/bin/uv pip install --python ~/ml-env/bin/python3 --no-deps -e /root/sglang-main/python 2>&1 | tail -10'

The output is equally unassuming:

Using Python 3.12.3 environment at: ml-env
Resolved 1 package in 1.31s
   Building sglang @ file:///root/sglang-main/python
      Built sglang @ file:///root/sglang-main/python
Prepared 1 package in 753ms
Uninstalled 1 package in 0.90ms
Installed 1 package in 0.69ms
 - sglang==0.0.0 (from file:///root/sglang/python)
 + sglang==0.5.9 (from file:///root/sglang-main/python)

A package upgrade from version 0.0.0 to 0.5.9, taking under two seconds to install. On the surface, this looks like routine maintenance — upgrade a Python package, move on. But this message sits at the intersection of several converging threads of work, and the specific flags chosen reveal a deep understanding of the infrastructure's fragility. This article unpacks why this particular command was the right tool at the right moment, and what it reveals about the broader deployment strategy.

The Context: A Pivot Between Two Massive Models

The immediate trigger for this message was the user's request at [msg 5789]: "Swapping the model on the llm server; Get and setup nvidia/Qwen3.5-397B-A17B-NVFP4; use latest upstream / main SGLang." The assistant had just finished hardening the deployment of Kimi-K2.5 INT4 — a 1-trillion-parameter Mixture-of-Experts model — into a production systemd service with hierarchical KV cache, tool call parsers, and reasoning parsers. Now the user wanted to pivot to a newer, more efficient model: Qwen3.5-397B-A17B-NVFP4, a 397-billion-parameter MoE model with only 17 billion active parameters per token, quantized using NVIDIA's NVFP4 format.

The NVFP4 quantization format is significant. It uses 4-bit floating point weights that are designed specifically for Blackwell GPUs (SM120 architecture). The model card explicitly requires --quantization modelopt_fp4 and the latest SGLang main branch with PR #18937 merged. This is not a model that can be served with a stable release of SGLang — it requires bleeding-edge code.

Why Not Just pip install -e?

The most interesting decision in this command is the --no-deps flag. A standard pip install -e /root/sglang-main/python would have triggered dependency resolution, which would have examined the pyproject.toml in the SGLang main branch and attempted to satisfy its pinned dependencies. As the assistant discovered in [msg 5802], the main branch's pyproject.toml pins cuda-python==12.9, torch==2.9.1, sgl-kernel==0.3.21, and other packages at specific versions.

The problem is that this environment is running CUDA 13 — not CUDA 12.x. The entire previous segment of work (<msg id=5764-5803>) was dedicated to upgrading the CUDA stack to version 13, patching SGLang for SM120 support, and building custom versions of PyTorch (2.9.1+cu130), flash-attn, flashinfer, and sgl-kernel that are compatible with the Blackwell GPUs. If pip were allowed to resolve dependencies normally, it would have attempted to downgrade cuda-python to 12.9, potentially replace the custom torch build with a standard one, and generally wreak havoc on a carefully tuned environment.

The --no-deps flag is therefore not an optimization — it is a protective measure. It tells the package manager: "Install only this package. Trust that all dependencies are already present and correct. Do not touch anything else." This is the equivalent of performing surgery with a scalpel instead of a chainsaw.

The Choice of uv Over pip

The command also uses uv pip install rather than bare pip install. uv is a fast Python package manager written in Rust, known for its speed and correctness. In this context, speed matters less than control — uv provides better isolation guarantees and faster resolution when it does need to resolve dependencies. But critically, the --python flag explicitly points to the existing virtual environment (~/ml-env/bin/python3), ensuring that the package is installed into the correct environment rather than a system Python or a newly created venv.

The -e (editable) flag is also deliberate. Installing in editable mode means the package is linked directly to the source directory rather than copied into site-packages. This is standard practice during development — it means any changes to the source code are immediately reflected without reinstalling. But in this production context, it also signals that this is a temporary or evolving deployment. The assistant is not committing to a frozen version; the expectation is that further patches may be needed as the Qwen3.5 model is tested and tuned.

What the Output Reveals

The output tells a subtle story. The old package was sglang==0.0.0 — note the version number. This was the previous build from the older SGLang source tree at /root/sglang/python. The version 0.0.0 is the default development version when no version tag is set, confirming that the previous installation was also a source build. The new package is sglang==0.5.9, which is the version baked into the main branch at the time of cloning.

The installation took 753ms to build and 0.69ms to install — remarkably fast for a source build. This suggests that much of the heavy compilation (C++ extensions, CUDA kernels) was either cached from a previous build or handled by prebuilt wheels. The SGLang build system uses setuptools and compiles significant amounts of CUDA code, so a sub-second build is only possible if the build artifacts were already present or if the package was installed without building extensions (which -e mode typically does by linking to the source).

Assumptions Embedded in This Command

This message makes several assumptions, most of them correct:

1. The environment is stable: The command assumes that the existing Python environment at ~/ml-env/bin/python3 has all the necessary dependencies — torch, flashinfer, sgl-kernel, flash-attn, etc. — already installed and compatible. This assumption is validated by the preceding work where the assistant confirmed torch 2.9.1+cu130 was working ([msg 5803]).
2. The source tree is complete: The command assumes that /root/sglang-main/python contains a fully functional SGLang source tree that can be built and installed. This was verified in [msg 5797] where git log showed the latest commits.
3. No dependency conflicts will arise at runtime: By skipping dependency resolution, the assistant is betting that the runtime imports will work even though the declared dependencies in pyproject.toml don't match the installed versions. This is a risk — if SGLang main branch added a new import that requires a newer version of a library that isn't installed, the error would only surface at runtime.
4. The build will succeed without errors: The | tail -10 at the end of the command means the assistant is only capturing the last 10 lines of output. If there were compilation warnings or errors earlier in the output, they would be hidden. The assistant is implicitly trusting that the build succeeded based on the "Installed 1 package" message.

What You Need to Know to Understand This Message

To fully grasp the significance of this command, a reader needs:

• Knowledge of CUDA versioning: Understanding why CUDA 13 vs CUDA 12 matters, and why a pyproject.toml pinning cuda-python==12.9 would be problematic in a CUDA 13 environment.
• Familiarity with Blackwell GPU architecture: The SM120 compute capability requires specific kernel builds and driver support (open kernel modules, not proprietary).
• Understanding of NVFP4 quantization: This is a NVIDIA-specific 4-bit floating point format that only works on Blackwell GPUs and requires specific SGLang patches.
• Knowledge of Python packaging: The difference between pip install, pip install -e, and pip install --no-deps, and why each matters in different contexts.
• Context about the session history: The weeks of work that went into stabilizing the CUDA 13 environment, building custom PyTorch, and getting flash-attn to compile.

What This Message Creates

The output of this message is a production SGLang server capable of serving the Qwen3.5-397B-A17B-NVFP4 model with NVFP4 quantization on Blackwell GPUs. Specifically:

• A new SGLang installation at version 0.5.9 (main branch) with PR #18937 support for modelopt_fp4 quantization.
• A preserved environment — all the custom CUDA 13 builds of torch, flashinfer, sgl-kernel, and flash-attn remain untouched.
• A foundation for the next steps: The subsequent messages in the session show the assistant configuring the FP4 GEMM and MoE backends (--moe-runner-backend flashinfer_cutlass and --fp4-gemm-runner-backend flashinfer_cudnn) to fix NaN outputs, demonstrating that the installation succeeded but further tuning was needed.

The Thinking Process: A Study in Risk Management

The assistant's reasoning, visible across the preceding messages, shows a careful risk calculus. When the user requested the model swap at [msg 5789], the assistant didn't just blindly install. It:

1. Stopped the existing service ([msg 5791]) to free resources and avoid conflicts.
2. Checked disk space to ensure the 397B model would fit.
3. Read the model card ([msg 5792]) to understand the exact requirements: modelopt_fp4, tensor-parallel-size 4, PR #18937.
4. Verified that the required PR was already merged into main ([msg 5798]).
5. Checked the build system (<msg id=5799-5802>) to understand the dependency landscape and identify the CUDA version conflict.
6. Confirmed the existing torch version ([msg 5803]) to ensure compatibility.
7. Only then executed the install with the protective --no-deps flag. This is not the behavior of someone blindly following instructions. It is the behavior of a system administrator who has been burned by pip install silently upgrading critical dependencies and breaking production systems. The --no-deps flag is the scar tissue from past incidents.

Conclusion

Message 5804 is a masterclass in surgical package management. On its surface, it is a one-line pip install. In context, it is the culmination of hours of infrastructure debugging, a deep understanding of CUDA versioning and GPU architecture, and a deliberate choice to protect a fragile environment from automated dependency resolution. The command succeeds because the assistant understood not just what to install, but what not to touch. In a world where package managers are designed for convenience, sometimes the most important skill is knowing when to say "no dependencies."