The Engineer

The bug is the last link in the chain

Ben Thompson's write-up of his own compromised Mac mini traces how the host got exposed. TCC (Transparency, Consent, and Control) is the macOS per-app permission system. It draws consent prompts in a protected GUI layer that no software can read. That is the correct design, because a prompt software can see is a prompt malware can approve. His box was headless and ran only Claude and Codex. When agent-written programs touched a gated resource such as an SMB share, Desktop or Downloads, the call failed silently. Clearing it required a human looking at the screen.

So Thompson left Screen Sharing on permanently and clicked prompts from his phone, prompts he had long since stopped reading. Enabling Screen Sharing opens port 5900 in the macOS firewall. A control built to stop malware ended up requiring an always-on remote-control daemon, and that daemon was the entry point. Routers usually block 5900 inbound, but that depends on the network the host sits on, not on anything the host enforces.

The compliance dashboard may be measuring intent

Thompson had "Install… security updates automatically" checked for years. He found it does not install point releases, which is where CVE fixes almost always ship. This bug was fixed in a point release that existed mainly for that purpose. That behavior is the author's finding, not an Apple statement, so test it against the OS versions in the fleet. If it holds, an MDM report built on that checkbox records what admins intended, not what is installed. The signal to trust is the OS build number compared against the fixed release for Tahoe, Sequoia and Sonoma.

The highest-risk group is iOS build farms. Teams leave Screen Sharing on there because remote GUI access is how they clear Xcode, simulator and TCC dialogs, which reproduces the Screen Sharing dependency that exposed Thompson's host. A VPN in front of 5900 limits who can reach the bug. It does not remove it, because the flaw needs no credentials. Patching is the only fix.

What the agent got right

Detection was close to accidental. Claude Code's persistent monitor stops every 30 minutes, and a scheduled restart surfaced an URGENT alert. Claude noticed the account could now run admin commands without a password. It stopped executing commands on its own and told Thompson to stop using it. He used it anyway to remove the malware and pin down a four-second intrusion window, then wiped the machine.

Once an attacker holds root, every binary, config and tool output on that host can be controlled by the attacker. Forensics run from inside it prove nothing.

Claude's advice was textbook incident response, and the wipe is what actually made him safe. The blast radius matters too. With passwordless admin, every agent running as that user effectively ran as root. Thompson calls the box being a low-value, agent-only host his saving grace.

Mac or Linux for agent hosts

Thompson had already bought a Linux server and says he will never rack a Mac again. The split I would use: keep Macs for work that needs the Apple toolchain or Mac GUI automation, and move other headless agent jobs off them.

Why the harness moved the score

The mechanism behind Hugging Face's result matters more than the headline number. Its capture proxy accepts OpenAI, Anthropic and Gemini request formats, forwards calls to vLLM, and records exact token IDs and logprobs for TRL. That turned 10 unmodified harnesses into RL environments. Exact IDs matter because re-tokenizing the harness text afterward produces a different sequence from the one the model actually sampled, and that corrupts on-policy RL. AINews offers a plausible but unattributed explanation for the gap: weights never trained on Claude Code's prompts and tool schema pay a fit tax there. Either way, an eval run in a single harness measures model–harness fit, not the model.

Saturated benchmarks hide the spread

Turing Post rebuilt Reflection's Beam comparison and restored six Qwen and DeepSeek results Reflection had left out. With those back, Beam trails Kimi K3, Qwen 3.8 Max and DeepSeek V4.1 Flash on all seven coding benchmarks that have scores. The gap grows as the suites get harder: −20.2 on SWE-bench Multilingual and −33.4 on SWE Atlas Codebase QnA. Verified is Python-only. If you ship Go or TypeScript, the Multilingual row is the one to read. Reflection's claim of 3–4x less compute than GLM 5.2 leaves out prefill and serving overhead. Those costs dominate an agent loop that re-sends repo context on every turn.

Price the success, not the attempt

Cost per success equals cost per attempt divided by success rate. Turing Post's illustrative math: at DeepSWE-like rates of 44.4% against 74.2%, Beam's per-attempt cost has to fall below 0.60x of DeepSeek V4.1 Flash's just to break even. Retries and engineer time spent on failures come on top. Import AI's SciUniverse data shows the same approach changing a ranking. Claude Fable 5.1 at xhigh effort scored 45.3% at $40.61 per task, about $89.65 per success. GPT-5 Astra came in around $161.14 per success. With 92 tasks, each pass rate carries roughly ±5 points of standard error.

The reasoning budget is part of the harness

Every top SciUniverse result ran at xhigh effort, and Meta benchmarked Muse Glimmer at high reasoning. Production settings rarely match either. One analysis of 43,261 Fable 5 calls over six weeks found the delivered reasoning budget varied sharply while the model name stayed fixed, and many calls showed no thinking at all. The author and method are undisclosed, so trust the direction, not the size. You can check this in-house cheaply by logging reasoning tokens per call.

The fan-out row comes from Toby Ord's data in Import AI. Ten times as many agents returned only 3–5x the performance, which is worse than giving one agent ten times the tokens. Parallel agents buy wall-clock time, not quality.

Three retry layers make 27 attempts

Agent stacks nest retries. The model SDK retries, the framework retries tool calls, and the planner re-attempts failed steps. Multiply three attempts per layer and one logical call becomes 27 network attempts in the worst case. That figure is TLDR IT's own arithmetic, not a measured result. Agents also cancel constantly: user interrupts, superseded plans, deadlines. A library that keeps sleeping through backoff after a cancel still fires the retry when it wakes. The tokens are spent. A non-idempotent tool call can run again with nobody waiting on it. Hedging without deduplication is worse, because hedged duplicates double-execute writes, deploys or payments.

What stateless MCP actually requires

No handshake means no negotiation phase whose results the server can cache for the life of a connection. Every request has to describe itself. Negotiated capabilities, per-connection auth, in-memory caches and progress on long-running tools all have to move. Mid-call interactive prompts become a multi-round-trip input-request pattern. The tool returns a request, the client calls back, the tool resumes. Every interactive tool is now a resumable state machine with idempotent re-entry, backed by a KV store or signed continuation tokens. The payoff is a server that sits behind a load balancer with affinity off and scales to zero. The test is simple. If integration tests pass behind a non-sticky load balancer, the server really is stateless.

Stage guardrails; don't just parallelize them

The Bedrock case in TLDR Data got its latency win by running independent safety checks in parallel. Naive fan-out has a cost. It runs every check on every request and gives up the early exit the serial pipeline had. The shape that works has three stages. Cheap deterministic rejects go first: schema validation, size limits, allowlists, regex. Model-based checks run next, in parallel. The first reject cancels the rest. Two traps remain. A client-side cancel may not stop the provider from billing the call. And if one check consumes another's redacted output, running them in parallel is a safety bypass.

Failover is a deploy

Techpresso makes the related point. An LLM gateway that fails over from model A to model B on a 429 is shipping an untested behavior change to production. Model B has different tool-calling conventions, refusal boundaries and JSON adherence. Cloud spend controls do not help. AWS's monthly limit pauses whole projects. Google Cloud's Spend Caps cover only specific services. Neither returns an error the calling code can handle.

One cost is easy to miss. Task-scoped tokens put the IdP in the hot path of every agent action. When the IdP goes down, the agents stop. It needs an SLO.