The Smoke Test That Reveals Everything: Verifying an LLM Server After a Radical GPU Reconfiguration

In the middle of a sprawling, multi-session effort to deploy and optimize large language models on a cluster of NVIDIA RTX PRO 6000 Blackwell GPUs, a single, unassuming message appears. It reads, in its entirety:

Server is up with TP=4. Let me do a quick smoke test:

followed by a curl command that fires a trivial arithmetic question at a freshly started SGLang inference server, and the server's JSON response. At first glance, this looks like a routine check—the kind of "did it work?" verification any engineer runs after starting a service. But in the context of the conversation, this message is a critical inflection point. It represents the culmination of a dramatic infrastructure reconfiguration, a test of assumptions about model serving at reduced scale, and a window into the behavior of a cutting-edge language model that reveals as much about its design as about the health of the system serving it.

To understand why this message matters, we must first understand what preceded it.

The GPU Split: From Eight to Four

The assistant had just executed a high-risk operation on a Proxmox host: splitting the machine's eight NVIDIA RTX PRO 6000 Blackwell GPUs between two virtualized environments. Four GPUs (those on NUMA node 0, at PCI addresses 01:00.0, 11:00.0, 61:00.0, and 71:00.0) were kept bound to the nvidia driver for use by an LXC container running SGLang. The other four (NUMA node 1, at 81:00.0, 91:00.0, e1:00.0, and f1:00.0) were unbinding from the NVIDIA driver and rebound to vfio-pci for passthrough to a separate SEV-SNP VM. This involved updating the LXC configuration to only mount /dev/nvidia0 through /dev/nvidia3, creating a new PCI mapping (pro6000-vm) in Proxmox's device configuration, and restarting the container. The SGLang service itself was updated from --tp 8 (tensor parallelism across all eight GPUs) to --tp 4 (across the four remaining GPUs).

This was not a trivial change. Tensor parallelism is the backbone of serving large language models across multiple GPUs. When you halve the number of GPUs, you halve the available memory bandwidth and compute, and you change the communication topology. A model that comfortably fit across eight GPUs might now strain against memory limits or suffer from increased communication overhead. The assistant's decision to proceed with TP=4 for the Qwen3.5-397B-A17B-NVFP4 model—a 397-billion-parameter mixture-of-experts model in 4-bit NVFP4 quantization—was a calculated risk. The model had been designed for and tested on eight GPUs. Would it even load on four?

The Smoke Test: Deliberate Minimalism

The choice of smoke test is itself revealing. The assistant sends a single curl request to the OpenAI-compatible /v1/chat/completions endpoint with the prompt: "What is 7 13? Answer with just the number."* The parameters are carefully chosen: max_tokens=50 (just enough to get a short response), temperature=0 (deterministic, reproducible output), and a single-turn chat format.

This is not a performance benchmark. It is not a stress test. It is the absolute minimum viable check: Can the model load, process a prompt, and generate tokens? The assistant is following a principle of incremental verification—before testing throughput, latency, or quality, you first confirm that the basic inference pipeline works. A more complex test (e.g., a long context prompt, a multi-turn conversation, or a batch of requests) would risk conflating a fundamental loading failure with a more subtle issue. By choosing a trivial arithmetic question, the assistant ensures that any failure is unambiguous: either the server responds or it doesn't.

There is also a subtle assumption embedded in the choice of question. "7 * 13" is simple enough that any competent language model should answer correctly, but it requires actual computation rather than memorization. The assistant is implicitly testing not just that the model loads, but that the weights are correctly distributed across the four GPUs and that tensor parallelism is functioning. A corrupted weight or a misconfigured communication channel would likely produce a nonsensical answer.

What the Response Reveals

The server responds, and the JSON output is illuminating:

{
    "id": "94754f02040e4c968fb721f43455a577",
    "object": "chat.completion",
    "created": 1772899588,
    "model": "qwen3.5-397b",
    "choices": [
        {
            "index": 0,
            "message": {
                "role": "assistant",
                "content": null,
                "reasoning_content": "Thinking Process:\n\n1.  **Analyze the Request:**\n    *   Operation: Multiplication (7 * 13).\n    *   Constraint: Answer with just the number.\n\n2.  **Perform Calculation:**",
            }
        }
    ]
}

The response is truncated—the content field is null, and reasoning_content cuts off mid-sentence. This is a direct consequence of max_tokens=50: the model's chain-of-thought reasoning consumed nearly all of the token budget before it could produce the final answer. The model is using what appears to be a structured thinking process, enumerating steps, performing intermediate calculations, and only then planning to output the final answer. This is characteristic of models fine-tuned for reasoning (often called "chain-of-thought" or "thinking" models), where the model is trained to externalize its reasoning before producing a final response.

The fact that reasoning_content is a separate field from content is architecturally significant. It indicates that the SGLang server is configured to expose the model's internal reasoning trace as a distinct API field, likely through a feature like --reasoning-parser or a custom model configuration. This is not standard OpenAI API behavior—it is an extension specific to this deployment. The assistant's earlier work (referenced in segment 38's summary) included adding "reasoning parsers" to the service, and this response confirms that integration is working.

A Critical Observation: The Model Name Mismatch

One detail that might escape casual notice: the curl request specifies "model": "qwen3.5-397b", and the response echoes back "model": "qwen3.5-397b". However, the segment summary for this chunk states that "the Qwen3.5-397B NVFP4 model (previously on /data) was replaced with Qwen3.5-122B-A10B BF16 (native precision, 234 GB) stored on /shared." Yet the service file shown in message 6066 still points to --model-path /data/Qwen3.5-397B-A17B-NVFP4. This suggests that at the moment of this smoke test, the server is still running the 397B NVFP4 model, not the 122B BF16 model. The model replacement described in the segment summary likely occurs later in the conversation, after this test.

This is a useful reminder that segment summaries are high-level narratives that may compress or reorder events. The actual conversation unfolds step by step, and the assistant tests each configuration before moving to the next.

The Thinking Process Visible in the Message

The assistant's own reasoning is visible in the structure of the message. The opening line—"Server is up with TP=4. Let me do a quick smoke test"—reveals a clear mental model: the assistant has just confirmed the server is running (message 6071 showed an 80-second startup wait followed by a successful model list query), and now it is executing the next logical step in a verification pipeline. The assistant does not explain why it chose this particular test, because it does not need to—the reasoning is implicit in the action. But we can reconstruct it: the assistant knows the GPU topology changed, knows the tensor parallelism dimension changed, and knows the model is large. The most likely failure mode is that the model fails to load or produces garbage. A single, deterministic, low-token request is the fastest way to detect either failure.

The assistant also demonstrates a pattern of incremental escalation that recurs throughout the conversation: first verify the server is listening (curl to /v1/models), then verify it can process a request (this smoke test), then verify output quality (the follow-up in message 6073 with max_tokens=200), and only then proceed to performance benchmarking. This systematic approach minimizes wasted time—if the smoke test failed, the assistant would know immediately without having invested in a full benchmark setup.

Assumptions and Potential Pitfalls

The message rests on several assumptions that are worth examining. First, the assistant assumes that a single request is sufficient to validate the TP=4 configuration. In practice, tensor parallelism errors can be intermittent or load-dependent—a single request might succeed while a concurrent request triggers a race condition in the distributed communication layer. The assistant implicitly trusts that the SGLang server's initialization checks (which include NCCL distributed setup) are sufficient to catch configuration errors.

Second, the assistant assumes that the model served as "qwen3.5-397b" is indeed the intended model. The model name is passed as a parameter to the curl request, but the server could silently serve a different model if the configuration is wrong. The response echoes back the same model name, but this is just the server reporting what it was configured to report—it does not verify the actual weights.

Third, the assistant assumes that a correct answer to "7 * 13" implies correct model loading. While a wrong answer would certainly indicate a problem, a correct answer does not guarantee that all layers are functioning correctly. Subtle numerical errors (e.g., in FP4 quantization or in the MoE routing) might only manifest on specific inputs.

Output Knowledge Created

This message produces several pieces of actionable knowledge:

  1. The server is operational at TP=4. The model loaded successfully across four GPUs, the NCCL distributed setup completed without errors, and the inference pipeline is functional.
  2. The reasoning parser is working. The reasoning_content field is populated, confirming that the custom parser integration (added in earlier segments) is correctly intercepting and exposing the model's internal reasoning trace.
  3. The model uses chain-of-thought for simple arithmetic. Even for a trivial multiplication, the model externalizes its reasoning process, enumerating steps before producing the final answer. This is a design choice by the model's creators, not something the assistant configured.
  4. The token budget is insufficient. With max_tokens=50, the model cannot complete both its reasoning and its final answer. This is not a bug—it is a predictable consequence of the test parameters—but it informs the assistant's next action (message 6073 retries with max_tokens=200).

Conclusion

Message 6072 is a masterclass in minimal verification. In a single curl command, the assistant confirms that a complex, multi-GPU inference server is operational after a radical reconfiguration. The response reveals not just that the server works, but that the model's reasoning capabilities are intact, the custom parser integration is functional, and the API contract is being honored. For the reader following the conversation, this message is a quiet moment of validation before the conversation pivots to deeper diagnostics, model replacement, and the discovery of P2P DMA corruption under SEV-SNP IOMMU—a cascade of problems that will occupy the remainder of the segment. But for now, the server is up, the model is thinking, and the assistant can move forward with confidence.