The Scrubbing Layer: Why Reasoning Models Are Spawning a Secondary Inference Economy

Open any modern developer environment wired into a frontier reasoning model, and the immediate visual symptom is unmistakable: an unrelenting cascade of discursive tokens scrolling endlessly down the terminal. Before a developer receives a two-line shell script or a refactored function, the model insists on narrating its internal deliberations, qualifying its assumptions, and producing paragraphs of conversational fluff. The friction is no longer just a matter of wasted bandwidth. It is an operational tax on human attention and production budgets.

Rather than waiting for model providers to address this conversational inflation at the training stage, engineers are starting to build crude, client-side interventions. A recent open-source utility by developer Zach Ahn, plainly titled Vomit, highlights the lengths to which practitioners must now go. The tool is designed to intercept the output stream from Claude and immediately pipe it through a secondary, local language model running via runtimes like Llama.app or Ollama. The sole responsibility of this second model is to digest the verbose output and rewrite it into clean, terse English. It is an automated editorial janitor, deployed exclusively to clean up after an upstream intelligence.

The mechanics of the automated janitor

The architecture behind Vomit, written in Go and distributed via GitHub, is deliberately straightforward. It hooks into the terminal session, catches Claude's raw token stream, and routes it to an engine such as a locally hosted GPT-OSS 20B instance. By running commands like 'vomit scrub' or passively following sessions with 'vomit tail', developers can strip out conversational detritus before it hits their visual field or their terminal logs.

There is a profound systemic irony in this arrangement. Upstream, hyper-scale cloud infrastructure burns kilowatts of power and bills users per token to generate thousands of superfluous words. Downstream, on the developer's local machine, another chunk of silicon is spun up to consume those same tokens, discard the bulk of them, and emit a condensed summary. The workflow treats the frontier model not as an author of finished code or analysis, but as an erratic raw material producer whose output must immediately undergo secondary processing.

Compounding latency and hallucination risks

While placing a secondary sanitiser in front of a primary model solves immediate visual clutter, it introduces significant architectural compromises. As Ahn notes in the project documentation, local inference introduces noticeable latency into what should be an interactive development loop. Running a 20-billion-parameter local model on a workstation requires memory and compute, meaning the time-to-first-readable-token is extended considerably.

More critically, this secondary scrubbing layer lacks the environmental context available to the primary agent. The local model can only inspect the raw text sent over the stream, possessing zero visibility into file structures, active terminal processes, or tool executions. Consequently, the scrubbing model inevitably introduces its own hallucinations and summary errors, occasionally obscuring or completely omitting crucial operational details that the frontier model was trying to communicate. To safeguard against this data loss, developers are left needing auxiliary inspection tools to monitor the unscrubbed session files written to temporary directories.

A symptom of misaligned training incentives

The emergence of secondary scrubbing utilities is not merely an eccentric tooling experiment. It exposes a growing rift between the benchmark metrics favoured by frontier AI labs and the ergonomics required by working software engineers. In the race to top public evaluation leaderboards, reinforcement learning techniques reward models for generating extensive chains of thought and exhaustive explanations. For research benchmarks, verbosity correlates with reasoning depth; for production software, it translates directly into bloated latency, inflated API invoices, and degraded user experience.

Because frontier model providers charge by the token on both input and output, there is an inherent commercial disincentive for labs to make their base models intrinsically concise. Every surplus paragraph generated by a cloud-hosted reasoning model represents direct revenue for the provider and an unforced expenditure for the client. When platforms fail to provide granular, reliable controls to throttle this verbosity at inference time, downstream developers are left with no choice but to construct client-side workarounds.

The limits of chained inference

Chaining small, specialised models to clean, validate, or reformat the outputs of larger foundational models is rapidly becoming a standard design pattern across the software industry. Yet treating these secondary layers as a permanent solution to verbose generation is deeply inefficient. Stacking inference calls on top of inference calls simply transfers the burden of model alignment onto the client workstation.

Until frontier model architectures provide native, strictly enforced concision controls that reduce token generation at the source, projects like Vomit will remain an instructive template. They demonstrate that without discipline at the training and inference boundary, the broader AI ecosystem will inevitably be forced to waste compute cycles building an entire economy of digital janitors.