The Integral That Wouldn't Float: Tuning a PI Controller for GPU Dispatch Under Memory Pressure

The Message

[assistant] [edit] /tmp/czk/extern/cuzk/cuzk-core/src/engine.rs
Edit applied successfully.

At first glance, this message appears to be little more than a routine confirmation — a file was edited, the tool returned success. But this terse output marks a pivotal moment in an extended debugging session that had been chasing a pathological feedback loop in a GPU dispatch pipeline. The edit it confirms would fundamentally rebalance the relationship between two control-theory parameters — ki and max_integral — and transform a PI controller that was saturating in milliseconds into one that could float freely over minutes, finally giving the integral term room to do useful work.

The Problem: Integral Saturation in a GPU Dispatch Pacer

To understand why this edit was necessary, one must trace the chain of failures that led to it. The system under development was a GPU-accelerated zero-knowledge proof engine (cuzk) that used a PI (proportional-integral) controller to pace the dispatch of synthesis work items to a GPU worker pool. The controller's job was to maintain a target queue depth at the GPU — dispatching faster when the queue was shallow, slower when it was deep, and using the integral term to correct for steady-state drift.

The previous tuning iteration (committed as 2bf16cd6 in [msg 3631]) had introduced asymmetric integral clamping: max_integral_pos = 2.0 and max_integral_neg = -0.5, with ki = 0.02. The intent was to prevent the integral from accumulating deeply negative values that would choke off dispatch after a memory-ceiling "slam" event. But the user observed in [msg 3645] that the integral was still saturating — pinned at its limits rather than floating in a useful range. The clamp was so tight that the integral hit its ceiling in a single update tick and stayed there, providing no meaningful control authority.

The Reasoning: Working Through the Control Theory

The assistant's reasoning in [msg 3646] reveals a careful, step-by-step analysis of the PI controller's dynamics. The key insight was understanding how the integral accumulated relative to its clamp bounds.

The current parameters were: ki = 0.02, max_integral_pos = 2.0, max_integral_neg = -0.5. The maximum integral contribution to the rate multiplier was ki * max_integral = 0.02 * 2.0 = 0.04 on the positive side and 0.02 * -0.5 = -0.01 on the negative side. But the real problem was the rate of accumulation. At norm_error = 1.0 (an empty GPU queue) with a 2-second bootstrap interval, the integral grew by 1.0 * 2.0 = 2.0 per tick — hitting the +2.0 cap in a single update. The integral was useless because it was always pinned to its boundary.

The assistant considered several approaches. One was to bake ki into the accumulation itself — a standard PID formulation where u(t) = kp * e(t) + ki * ∫e(t)dt — so that the integral accumulated ki * norm_error * dt rather than norm_error * dt. This would naturally slow the growth. But the assistant ultimately chose a simpler path: dramatically lower ki and dramatically widen the clamp bounds, so the integral would take minutes to saturate rather than seconds.

The assistant worked through the numbers: with ki = 0.001 and max_integral_pos = 100, the maximum positive correction would be 0.001 * 100 = 0.10 (compared to the old 0.04). At a sustained error of 0.5, it would take about 200 seconds to saturate — a lifetime in the dispatch loop. The negative side got max_integral_neg = -20, giving a max negative correction of -0.02 and a saturation time of about 40 seconds at error 0.5. These were numbers that would let the integral genuinely float.

Assumptions and Their Consequences

The assistant made a critical assumption: that the integral should be a slow trim — a gentle correction that accumulates over many seconds or minutes — rather than an aggressive term that responds to every transient. This assumption was validated by the system's dynamics: the GPU processing time was ~1 second per item, but the synthesis pipeline feeding it had a 30-60 second latency. A fast integral would overcorrect to transient queue-depth fluctuations caused by synthesis batching, not by genuine changes in GPU throughput.

The previous assumption — that aggressive asymmetric clamping (+2.0 / -0.5) would prevent pipeline-draining backoff — turned out to be incorrect. It didn't prevent backoff; it prevented any integral action at all. The clamp was so restrictive that the integral was always saturated, making the I-term effectively a constant offset rather than a dynamic correction. The user's observation in [msg 3645] — "integral still saturates" — was the key signal that the previous approach had overshot.

Input and Output Knowledge

To understand this message, one needs knowledge of PID control theory (proportional and integral terms, saturation, accumulation dynamics), the specific architecture of the cuzk GPU dispatch pipeline (the DispatchPacer struct in engine.rs, the update() method, the bootstrap mechanism), and the history of tuning iterations that preceded this edit (the normalized error formulation, the asymmetric clamping experiment in commit 2bf16cd6).

The output knowledge created by this edit was a new set of PI parameters that would be deployed and tested as pitune3. The assistant summarized the change in [msg 3656] with a clear table:

| Parameter | Old | New | |---|---|---| | ki | 0.02 | 0.001 (20x lower) | | max_integral_pos | 2.0 | 100.0 (50x higher) | | max_integral_neg | -0.5 | -20.0 (40x higher) | | Max I correction | +0.04 / -0.01 | +0.10 / -0.02 | | Time to saturate (error=0.5) | ~4s / ~1s | ~200s / ~40s |

The Thinking Process

The reasoning in [msg 3646] reveals a sophisticated debugging process. The assistant didn't just tweak parameters blindly — it worked through the math, considered alternative formulations (baking ki into accumulation), evaluated the time-to-saturate at different error levels, and explicitly compared the maximum correction values. The thinking shows the assistant catching itself mid-reasoning: "Actually, the current implementation is correct as-is — the real problem is just that the clamp values are too restrictive." This self-correction — recognizing that the accumulation formula was fine and the real issue was the clamp bounds — was the key insight that led to the final solution.

The edit in [msg 3648] was the culmination of this analysis. It was followed by compilation ([msg 3649]), Docker build ([msg 3650]), deployment to the remote server ([msg 3651]), and a clean baseline measurement ([msg 3653]) before the new binary was started ([msg 3654]). The assistant then confirmed the binary was running and summarized the change in a table ([msg 3656]), giving the user clear visibility into what had changed and why.

Significance

This message, for all its brevity, represents a turning point. The previous tuning iteration had been too clever — the asymmetric clamping was an attempt to solve a symptom (pipeline draining after memory slams) by restricting the controller's behavior. But restricting the integral to ±0.5 didn't prevent pipeline drains; it just made the integral useless. The new approach — let the integral breathe, give it room to float, trust the low ki to keep it slow — was a return to first principles. A PI controller's integral term needs room to work. Clamping it to near-zero made it a decoration, not a control term. By widening the bounds by 50x and lowering the gain by 20x, the assistant gave the integral the space it needed to actually do its job.