Segment 24
This sub-session focused on deploying and debugging the pinned memory pool fix for GPU underutilization. The team uncovered two critical issues: budget integration caused silent fallback to heap allocations, and budget exhaustion prevented PCE caching. Removing budget from the pool fixed the fallback, and a GPU queue depth throttle (max_gpu_queue_depth = 8) was implemented to free budget for PCE caching. Further diagnosis revealed a dispatch burst problem causing thundering herd of cudaHostAlloc calls and pinned pool thrashing. The team implemented a semaphore-based reactive dispatch mechanism ensuring exactly one synthesis per GPU completion, which eliminated H2D transfer overhead (ntt_kernels dropped from 1,300–12,000ms to 0ms), reduced per-partition GPU time to ~935ms, improved pool reuse ratio from 12:474 to 48:24, and increased budget headroom to 288 GiB.