The Blackwell Schism: How a Research Subagent Uncovered NVIDIA's Hidden Tensor Core Divide

Introduction

In the course of a marathon ML infrastructure session—spanning driver installations, CUDA toolkit conflicts, flash-attn rebuilds, and multi-GPU benchmarking—an engineer reached a critical inflection point. The GLM-5-NVFP4 model had been deployed on eight RTX PRO 6000 Blackwell GPUs using SGLang, and performance tuning had yielded a respectable 28% throughput improvement. But a nagging question remained: was the FP4 throughput ceiling imposed by the hardware itself, or was there still headroom to be unlocked through better kernel implementations?

The answer to this question would determine the entire trajectory of the optimization effort. If the hardware was the bottleneck, the team would need to accept the limit and focus on higher-level optimizations like pipeline parallelism and expert parallelism. If the software was the bottleneck, the team could invest in kernel development, compiler flags, and framework patches to unlock the hardware's true potential.

To resolve this question, the engineer spawned a research subagent with a precise mission: "Search the web for detailed NVIDIA SM120 tensor core specifications" ([msg 0]). What followed was a methodical, multi-stage investigation that would uncover a startling truth: NVIDIA had shipped two fundamentally different tensor core architectures under the same "5th Generation" branding, and the RTX PRO 6000 had inherited the weaker one.

This article traces that investigation from its inception through its stunning conclusion, examining the reasoning, strategies, and discoveries that transformed a simple research request into a definitive hardware analysis.

The Question That Started It All

The user's message at [msg 0] was a model of precise technical inquiry. It listed five specific information needs: FP4 tensor core throughput in TOPS compared to SM100 (B200/B100), whether SM120 has 5th generation tensor cores or a cut-down version, memory bandwidth and L2 cache size for the RTX PRO 6000, any NVIDIA documentation about SM120 vs SM100 compute differences, and the official FP4 TOPS rating for the RTX PRO 6000 Blackwell Server Edition.

The framing was deliberate and consequential. The user explicitly stated: "This is critical to understand whether the GPU hardware itself is limiting FP4 throughput, or if it's purely a software/kernel tuning issue." This was not idle curiosity—it was a diagnostic pivot point. The engineer had been working empirically (benchmark, tune, measure, repeat) and had achieved meaningful gains. But rather than continuing down that path indefinitely, they recognized the need to establish the theoretical upper bound. As [1] notes, this question embodies a classic engineering move: when iterative optimization plateaus, go back to first principles and verify your assumptions about the hardware's capabilities.

The five questions formed a deliberate diagnostic chain. Question 1 (FP4 TOPS comparison to SM100) established the baseline—if SM120 delivers dramatically less FP4 throughput than SM100, that signals a hardware limitation. Question 2 (5th generation tensor cores or cut-down) got at the architectural lineage—if SM120 lacks 5th gen tensor cores entirely, FP4 operations might be emulated. Question 3 (memory bandwidth and L2 cache) addressed the data movement bottleneck—FP4 tensor core operations are compute-bound only if data can be fed fast enough. Question 4 (NVIDIA documentation) was the most open-ended, acknowledging that the engineer was operating in a documentation vacuum. Question 5 (FP4 TOPS rating) was the concrete number ultimately needed to compute utilization percentages.

The First Wave: Broad Searches and the PDF Problem

The assistant's first response ([msg 1]) launched five parallel Exa web searches, each targeting one dimension of the investigation. The queries were carefully crafted: one for SM120 FP4 throughput, one for the RTX PRO 6000 datasheet, one for SM120 vs SM100 architectural differences, one for memory specifications, and one for the Blackwell architecture whitepaper. This was a funnel strategy—start broad to get an overview, then narrow to specific documents that provide authoritative data.

The search results were informative but incomplete. They returned snippets from NVIDIA's developer blog about Blackwell Ultra, a workstation edition datasheet, a Backend.AI analysis article about DGX Spark, a Fixstars blog post about the Max-Q variant, and a Blackwell datasheet. These were useful signals but not definitive evidence. A blog post is not a specification sheet. A workstation datasheet may differ from the server edition.

Recognizing this limitation, the assistant escalated to direct primary source retrieval ([msg 2]). It attempted to fetch four documents: the NVIDIA RTX Blackwell GPU Architecture PDF, the RTX Blackwell PRO GPU Architecture PDF, the Chips and Cheese analysis, and the RTX PRO 6000 Server Edition datasheet. This was a textbook example of source triangulation—the assistant was building a web of evidence from multiple perspectives.

But here the investigation hit a critical obstacle: the PDF fetches returned raw binary data. The webfetch tool with format:"text" cannot parse PDF files—it dumps the raw bytes, which are useless for extracting structured information. As [2] observes, this is the moment when the assistant had to make a strategic choice: give up on those sources, try a different approach to access the PDFs, or find alternative sources that are guaranteed to be parseable.

The Strategic Pivot: Adapting to Tool Limitations

The assistant's response to the PDF failure was a masterclass in adaptive research methodology ([msg 3]). Rather than retrying the same PDF URLs with different parameters (which would likely fail again), it abandoned those sources entirely and pivoted to three markdown-friendly alternatives: the Backend.AI DGX Spark analysis, the NVIDIA Blackwell Tuning Guide, and an arXiv microbenchmarking paper.

This pivot reveals sophisticated reasoning. The assistant recognized that PDFs were a dead end and chose sources guaranteed to render as markdown or HTML. It selected sources that directly addressed the user's questions: the DGX Spark article explicitly investigates whether consumer/professional Blackwell GPUs use the same architecture as data-center Blackwell—which is exactly the SM120 vs SM100 question. The NVIDIA Blackwell Tuning Guide is the official documentation that should contain architecture specifications. The arXiv paper is a microbenchmarking study of the Blackwell architecture—exactly the kind of empirical data needed.

As [3] notes, this message exemplifies the kind of adaptive reasoning that makes AI-assisted coding sessions powerful—not just executing commands, but recognizing when an approach isn't working and formulating a new strategy. The assistant demonstrated metacognitive awareness of its own failures and pivoted intelligently to alternative sources.

Narrowing the Focus: Targeted Searches and the Lei Mao Discovery

With the primary sources retrieved, the assistant conducted two more targeted searches ([msg 4]). The first sought the official RTX PRO 6000 FP4 TOPS rating, including the number "3352" in the query—a hypothesized value that may have appeared in search snippets or been extrapolated from known specifications. The second search was remarkably technical, targeting "NVIDIA SM120 tensor core mma.sync FP4 throughput per SM compared to SM100 tcgen05." This query reveals that the assistant had absorbed enough context to formulate precise, instruction-level research questions.

The Exa search returned Lei Mao's benchmark article as a result. The assistant recognized this as the most promising source for per-SM throughput data and fetched it immediately ([msg 5]). As [4] explains, Lei Mao's blog is well-known in the CUDA community for its meticulous, reproducible benchmarks of low-level GPU instructions. The article benchmarks MMA instructions at the per-SM level across multiple GPU architectures, providing exactly the kind of data needed: not aggregate GPU-wide TOPS, but the throughput a single streaming multiprocessor can achieve for specific data types using specific MMA instructions.

This was the Rosetta Stone that could translate between architectural capability and observed kernel performance. The assistant's decision to fetch this specific article, and the reasoning chain that led to that decision, exemplify how effective technical problem-solving requires not just access to information but a sophisticated understanding of what information is needed, at what granularity, and from what sources.

The Revelation: SM120's Crippled Tensor Cores

With all the data assembled, the assistant synthesized everything into a comprehensive analysis ([msg 6]) that delivered a stunning verdict. Despite NVIDIA marketing both SM100 and SM120 as "5th generation tensor cores," they are architecturally different in fundamental ways.

The comparison table in the analysis laid bare the divergence. SM100 (B200/B100) uses tcgen05.mma instructions that can be issued by a single thread, with data flowing from shared memory through a dedicated 256 KB Tensor Memory (TMEM) per SM. SM120 (RTX PRO 6000, RTX 5090) uses extended mma.sync instructions that require a full warp of 32 threads, with data sourced directly from registers—exactly like the Ampere architecture from 2020. SM120 has no TMEM at all. It supports only 48 concurrent warps per SM versus SM100's 64. Its shared memory is 128 KB per SM versus SM100's 228 KB.

The assistant quoted the Backend.AI analysis, published the same day: "Datacenter Blackwell turned the tensor core into an autonomous compute unit; consumer Blackwell put new data types on top of the oldest programming model." And: "SM12x's tensor core programming model is closer to Ampere's mma.sync than to datacenter Blackwell's tcgen05."

This was the critical insight. The SM120 tensor cores are not a cut-down version of SM100's tensor cores—they are a fundamentally different design that retains the Ampere-era programming model while adding support for new data types like FP4 and FP6. The absence of TMEM is particularly significant: on SM100, tensor cores have their own dedicated memory that allows them to operate autonomously without competing with CUDA cores for register file access. On SM120, tensor cores must share the register file with CUDA cores, creating resource contention that limits throughput.

The Weighted Conclusion: Hardware vs. Software

The assistant's final synthesis delivered a nuanced answer: the performance limitation is "BOTH, but weighted differently." Hardware limitations are real but bounded: no TMEM, warp-level mma.sync instead of single-thread tcgen05.mma (2-4x lower per-SM throughput), GDDR7 at 1.6 TB/s versus HBM3e at 8 TB/s (memory bandwidth is likely the primary bottleneck for inference), and 128 KB shared memory versus 228 KB (constrains tile sizes).

But software limitations are the dominant factor for FP4 right now. FlashMLA, FlashAttention, and FlashInfer do not support SM120 at all. The specific FP4 MMA instructions that achieve peak throughput on SM120 require specific kernel implementations—generic instructions achieve only ~26% of peak. Triton treats SM120 as SM80 (Ampere), disabling all Blackwell optimizations. vLLM requires --enforce-eager on SM120. CUTLASS 4.x has SM120 support but it's newer and less battle-tested.

The bottom line was sobering but actionable: for LLM inference (which is heavily memory-bandwidth-bound during decode), the theoretical FP4 compute capacity (~2000+ TOPS dense) far exceeds what the 1.6 TB/s memory bandwidth can feed. The practical FP4 throughput is almost certainly memory-bandwidth limited, not compute limited. But software maturity is the bigger gap—the right kernels to exploit SM120's FP4 instructions simply don't exist yet in mainstream inference stacks.

Implications and Aftermath

This investigation fundamentally reshaped the optimization strategy. By establishing that the hardware limitations were real but bounded, and that software maturity was the dominant gap, the assistant provided a clear roadmap: focus on developing SM120-optimized kernels rather than waiting for different hardware or accepting a performance ceiling.

The discovery also revealed something uncomfortable about NVIDIA's product strategy. By marketing both SM100 and SM120 as "5th generation tensor cores," NVIDIA obscured a fundamental architectural divergence. The datacenter Blackwell received a ground-up redesign of the tensor core subsystem with autonomous operation, dedicated TMEM, and a new instruction set. The consumer/workstation Blackwell received incremental improvements to the existing Ampere-era design—new data types bolted onto the old programming model.

As [5] observes, this is the kind of insight that can save weeks of fruitless optimization work. It is also the kind of insight that only emerges when someone takes the time to dig into architecture documentation, read between the lines of marketing materials, and synthesize information from multiple sources. The assistant's work was not just a technical analysis—it was an act of intellectual independence from NVIDIA's product narrative.

Conclusion

The research subagent's investigation into SM120 tensor core specifications is a case study in how careful hardware analysis can demystify performance problems that might otherwise be blamed on software alone. From the initial question through the failed PDF fetches, the strategic pivot to markdown-friendly sources, the targeted searches for per-SM throughput data, and the final synthesis that revealed the Blackwell architectural schism—every step of the journey was methodical, adaptive, and grounded in deep technical knowledge.

The final analysis delivered precisely what the engineer needed: not just data, but a framework for understanding the performance landscape. The hardware limitations were cataloged and quantified. The software gaps were identified and prioritized. The weighted conclusion—that software maturity is the dominant bottleneck—provided clear direction for future work.

In the broader narrative of the coding session, this investigation marks the transition from empirical tuning to strategic optimization. The engineer no longer needs to wonder whether the hardware is the ceiling. They now know exactly what the ceiling is, why it exists, and which parts of it can be raised through software effort. That knowledge is worth more than any single optimization technique—it is the foundation upon which an entire optimization campaign can be built.## References

[1] "Probing the Silicon Ceiling: A Deep Dive into the SM120 Research Request" — Analysis of the initial research request at [msg 0], examining the reasoning, assumptions, and knowledge embedded in the engineer's five questions.

[2] "From Search to Source: The Critical Data-Fetching Pivot in NVIDIA SM120 Research" — Analysis of the primary source retrieval phase at [msg 2], focusing on the strategic decision to fetch architecture documents directly.

[3] "Strategic Source Selection in GPU Architecture Research: Deconstructing a Subagent's Web Fetch Round" — Analysis of the adaptive pivot at [msg 3] after PDF fetches failed, examining the assistant's source selection strategy.

[4] "The Pivot Point: How a Single Web Fetch Reframed a GPU Performance Investigation" — Analysis of the Lei Mao article fetch at [msg 5], examining how targeted retrieval of per-SM benchmark data filled a critical knowledge gap.

[5] "The Blackwell Divide: How SM120's Crippled Tensor Cores Explain the FP4 Performance Puzzle" — Analysis of the final synthesis at [msg 6], examining the revelation that SM120 and SM100 have fundamentally different tensor core architectures.