When Shell Quoting Bites: The F-String That Broke zsh

Introduction

In the long and winding journey of deploying GLM-5 — a 744-billion-parameter Mixture-of-Experts model quantized to GGUF Q4_K_XL format — on 8× RTX PRO 6000 Blackwell GPUs, the assistant had just achieved a major breakthrough. After hours of debugging incoherent output, fixing a Triton MLA attention backend bug and a GGUF shard ordering issue, the model was finally producing correct text. CUDAGraph had been successfully enabled, doubling single-request throughput from 20 tok/s to 43 tok/s. The next logical step was to run a comprehensive benchmark suite to quantify the improvement.

But before the assistant could celebrate, it hit an unexpected wall — not a GPU memory error or a kernel bug, but something far more mundane: a shell quoting problem. Message <msg id=1995> captures this moment of friction, where the assistant's attempt to run a Python benchmark inline via SSH was foiled by zsh's interpretation of f-string curly braces. This short message — barely a dozen words of reasoning followed by a file write and some irrelevant LSP diagnostics — is a fascinating snapshot of the real-world challenges that arise when complex toolchains meet the unforgiving syntax of shell interpreters.

The Message

The subject message reads:

[assistant] The f-string with curly braces is fighting zsh. Let me write a script file instead. [write] /home/theuser/glm-kimi-sm120-rtx6000bw/benchmark.py Wrote file successfully.

>

LSP errors detected in other files: <diagnostics file="/home/theuser/glm-kimi-sm120-rtx6000bw/decode_latency_breakdown.py"> ERROR [17:8] Import "torch" could not be resolved ERROR [18:8] Import "torch.distributed" could not be resolved ERROR [345:12] Import "torch.multiprocessing" could not be resolved </diagnostics>

>

LSP errors detected in other files: <diagnostics file="/home/theuser/glm-kimi-sm120-rtx6000bw/decode_gap_analysis.py"> ERROR [12:8] Import "torch" could n...

At first glance, this seems trivial — a simple workaround for a quoting issue. But understanding why this message exists requires unpacking the chain of events that led to it, the assumptions that were made, and the pragmatic decision-making that resolved the problem.

The Preceding Failure

In the message immediately before this one (&lt;msg id=1994&gt;), the assistant had constructed an elaborate Python benchmark script and attempted to execute it as a one-liner via SSH:

ssh root@10.1.230.174 '/root/ml-env/bin/python3 -c "..."'

The ... contained a full 40-line Python program with imports, function definitions, a benchmark runner, and — crucially — f-strings for formatted output. The f-strings used curly braces for variable interpolation, like {tot_out/wall:7.1f} tok/s. When zsh on the assistant's local machine tried to parse this command, it encountered the { and } characters from the f-strings and attempted to interpret them as shell brace expansions. The result was a parse error: zsh:35: parse error near &#39;tok/s&#39;.

This is a classic problem when embedding complex code inside shell commands. Python f-strings use {...} for expression interpolation, and while these characters are not special in all shell contexts, they become problematic when nested inside quoting layers. The command had three levels of quoting: single quotes for the SSH argument, double quotes for the python3 -c argument, and single quotes again for the Python f-strings. Each layer adds complexity and opportunities for the shell to misinterpret the syntax.

The Assistant's Reasoning

The assistant's response reveals a clear diagnostic process. It immediately identified the root cause: "The f-string with curly braces is fighting zsh." This statement shows that the assistant understood the quoting conflict at a deep level — it recognized that the curly braces {} in the Python f-strings were being interpreted by zsh as shell syntax rather than being passed through to Python.

The decision to "write a script file instead" is a pragmatic and elegant solution. Instead of debugging the quoting further — which could involve escaping every { and } with backslashes, restructuring the quoting, or using heredocs — the assistant sidestepped the entire problem. By writing the Python code to a file and executing that file directly, the shell only needs to handle a simple command like python3 benchmark.py, with no complex quoting required. The Python interpreter reads the file directly, so f-strings work as intended without any shell interference.

This decision reflects a key principle in systems engineering: when a tool (the shell) is fighting you on syntax, change the interface rather than fighting the tool. The script file approach is cleaner, more maintainable, and avoids the fragility of deeply nested quoting.

The LSP Diagnostics: Noise in the Signal

The message also includes LSP (Language Server Protocol) diagnostics from the assistant's IDE environment, reporting unresolved imports for torch, torch.distributed, and torch.multiprocessing in two other files in the project directory. These errors are entirely expected — the assistant's local development machine likely doesn't have PyTorch installed, while the remote server does. The assistant does not acknowledge or act on these diagnostics, which is the correct response: they are false positives from the IDE's static analysis running in an environment that differs from the deployment target.

This is a common source of noise in AI-assisted coding sessions. The assistant's tooling reports diagnostics from the local environment, but the actual execution happens on a remote server with a different software stack. The assistant implicitly understands this and ignores the irrelevant errors, focusing instead on the actual task at hand.

Context and Significance

To appreciate the weight of this seemingly minor message, one must understand what preceded it. The assistant had spent hours debugging a model that loaded successfully but produced garbage output. Two bugs were found and fixed: a Triton MLA attention backend issue where a custom PyTorch op created a phantom output buffer, and a GGUF shard ordering bug where fused projection layers had their columns in the wrong order. After these fixes, the model produced correct output. Then CUDAGraph was enabled, doubling throughput from 20 tok/s to 43 tok/s.

The benchmark that failed due to the quoting issue was meant to quantify this improvement across multiple concurrency levels. The assistant had every reason to be eager to see the numbers — the 2.15× improvement was a significant achievement. But instead of getting results, it got a shell parse error. The response in &lt;msg id=1995&gt; is remarkably calm and methodical given this context: no frustration, no wasted effort trying to fix the quoting, just a clean pivot to a file-based approach.

Assumptions and Lessons

The assistant made one key assumption that turned out to be incorrect: that a complex multi-line Python script could be cleanly passed as a one-liner through SSH. This assumption is reasonable — many developers do this routinely for simple commands. But the combination of f-string syntax (with its curly braces), nested quoting (single inside double inside single), and the specific shell (zsh) created a failure mode that was difficult to predict.

The lesson here is about the fragility of shell quoting for complex code. While one-liners are convenient for simple operations, they become increasingly error-prone as the code grows in complexity. The assistant's decision to write a script file is a best practice that avoids these issues entirely. It also makes the code reusable, editable, and debuggable — all advantages over an inline command.

Conclusion

Message &lt;msg id=1995&gt; is a small but revealing moment in a much larger story. It shows that even after solving deep technical challenges — debugging GPU kernel issues, fixing tensor parallelism bugs, optimizing inference throughput — the mundane problems of tooling and syntax can still bring progress to a halt. The assistant's response demonstrates a mature engineering mindset: recognize the problem, understand its root cause, and choose the simplest solution that eliminates the entire class of issue. Writing a script file instead of fighting with shell quoting is not just a workaround; it's a fundamentally better approach that avoids similar problems in the future.

This message also highlights the layered complexity of modern AI-assisted development. The assistant operates across multiple environments (local IDE, remote server, SSH sessions), multiple programming languages (Python, shell), and multiple tools (vLLM, GGUF, Triton, NCCL, CUDAGraph). Each layer introduces its own failure modes, and the assistant must navigate all of them simultaneously. The ability to quickly diagnose and sidestep a shell quoting issue — rather than getting bogged down in it — is a testament to the assistant's pragmatic problem-solving approach.