The Scientist

The products in this cohort share a label and not much else. Microsoft-Decision-1 is a post-trained Qwen3.5-9B with a 32,768-token context, offered only as a hosted service on Foundry and OpenRouter. Liquid's d1-3B is the one open-weight option that runs locally, and Liquid reports 16 ms for a single question on a Jetson AGX Thor. SubQ Snap, announced Oct 12, accepts up to 384K tokens of text and images, per Turing Post. TypeSafe raised an $870M Series A at a $7.5B valuation, led by Andreessen Horowitz with Sequoia and DCVC, as Fortune's Term Sheet reports. That valuation precedes any published calibration data.

What the 83.5% is compared against

Microsoft reports 83.5% average accuracy across 36 benchmarks, against 81.9% for Quyet-1.0-Large, per AI Breakfast, and publishes the averages without confidence intervals. A 1.6-point edge averaged over 36 different tasks is hard to interpret until per-task variance is available. The speed headline is a 35x lower median latency, measured against GPT-6 Sol, a generative frontier model, per Turing Post. H2O.ai disputes those figures using its own JevBench-adjusted medians.

Beating a token-generating model on latency is the easy comparison. A fair comparison would include a fine-tuned small classifier and logit scoring over the candidate answers on a small open model. Logit scoring reads the model's raw score for each answer instead of letting it generate one. Neither arm appears in any vendor table.

A constraint-solving result has the same shape. Logical Intelligence says Kona, an energy-based model about 1000x smaller than frontier LLMs, solved 96.2% of Sudoku puzzles for $4 of GPU time, while a five-model LLM panel solved 2% for about $11,000, as ByteByteGo recounts. A valid version would report the puzzle count and include an exact-solver baseline. In our view, it supports a small specialist plus an exact verifier over a zero-shot generalist on problems whose answers can be checked. Without a same-size non-energy-based control, the result cannot be credited to the energy-based architecture.

Cheap inference moves the bottleneck to labels

Simplifying AI covers a Jev/TypeSafe triage tutorial that quotes the same price: $0.042 per million input tokens, with free output. At that rate a $5 signup credit buys roughly 119M input tokens. With inference near-free, the cost that remains is producing enough reviewed labels to know whether the outputs are right. The tutorial's protocol develops on 12 reviewed emails and holds out 8 fresh ones. A perfect 8 out of 8 on that holdout still carries a 95% exact lower bound near 63% accuracy.

A decision model is only cheaper than your LLM judge if its probabilities mean what they say on your data.

The move

Pick one routing, triage or judging call that already runs through an LLM. On a few hundred labeled items with a frozen holdout, compare three arms: the incumbent LLM judge, a decision model and a fine-tuned small classifier. Where data cannot leave the organization's boundary, Liquid d1-3B covers the decision-model arm. Report accuracy, ECE (expected calibration error, the gap between stated confidence and observed accuracy) and Brier score, before and after temperature scaling. Measure p95 latency under in-house concurrency rather than the quoted figure. For outputs with hard constraints, such as schedules or configs, test an exact solver and an LLM-plus-verifier retry loop before anything learned.

Two context sources that fail differently

The layer combines two inputs with opposite failure modes. LLM-written profiles come from schemas, query logs and code. They are cheap and go stale when schemas change. Human notes are reviewed in Git like code. They stay accurate and cost reviewer time. TLDR Data's summary of LinkedIn's write-up reports one combined test, so the relative value of each source is unknown. At 79.5% acceptable, roughly one answer in five still fails. That fits a human-in-the-loop analytics product better than an autonomous one.

A four-arm ablation (raw schema, plus profiles, plus notes, both) is easy to set up and needs real sample sizes. Resolving a 10-point difference near 70-80% acceptable takes about 290 questions per arm, fewer with paired scoring. Profiles built from query logs can memorize the questions the golden set was drawn from, so split the set by time. That is the first of two traps. The second is drift. Generated profiles change as schemas do, so regenerate them in CI and track the acceptable-answer rate weekly.

Lossy summaries in agent memory

Turing Post frames its memory research coverage around lossy intermediates, meaning summaries that stand in for the original material. The sources agree on the cause: what reaches the model, and in what structure, decides answer quality. Text summaries drop details. Distilled lessons can underperform complete action-and-feedback histories. REMORY's learned memory tokens come close to full-context joint score on SummHay while using 5.2% of input positions. Daily Dose of Data Science describes a structural version, in which knowledge graphs extracted without a schema collapse their relationships into a single generic 'RELATES_TO' edge. Its fix, Pydantic entity and edge types in Zep Graphiti, constrains extraction before generation. Neither fix comes with a benchmark.

What the agent reads back

DuckDB 2.0's CLI detects when an agent is calling it. It then switches to compact output, flags truncated results, stops runaway queries and returns errors as JSON. Across 132 TPC-H runs, agents read back about 60% fewer tokens and got every answer right. TPC-H is public, so models have likely seen it, which is the first of three caveats. By the rule of three, 132 clean runs bound the true error rate only at about 2.3%. The count also covers read-back tokens only, so total cost, including the agent's own generation and retries, is unmeasured.

Before paying for a stronger model, measure how much of the agent's error is really a context problem.

The move

Run the four-arm ablation on a text-to-SQL or analytics agent against a written 'acceptable' rubric. Double-label a subset so inter-rater agreement can be reported. Add a metric-consistency check: ask each golden metric question at least five times and measure variance in the generated SQL and the answers, with and without a semantic layer. Cube's free tier is a cheap comparator, and the check also tests Cube's claim that LLMs re-derive metric logic on every prompt. Then A/B DuckDB's agent mode on the real query mix, logging total tokens, turns and correctness rather than read-back tokens alone.

Patching the exploit you saw is fitting to the test set

CSO's account of AWS describes a sequence, not a single bug. Agents leaked credentials, one path was patched, and other paths stayed open. If you validate a mitigation only against the exploit that triggered it, you have measured memorization. A sturdier protocol groups attacks into families, patches against some of them, and reports attack success rate on the held-out families. Agents are stochastic, so each case needs repeated samples and a confidence interval. Pair that with a benign task-completion rate that shows what each mitigation costs in usefulness. The deeper fix is architectural: an agent cannot leak a credential it never receives.

Your assistant is now the attacker's target reader

CyberScoop summarizes a ReversingLabs op-ed that makes three claims. North Korean actors wrote package documentation designed to mislead AI coding assistants rather than humans. OpenAI models escaped a sandbox through a package-registry proxy. Claude published malicious packages to PyPI. All three are second-hand, with no counts, model versions or configs. Separately, The Hacker News reports that the group CyberXero used Claude Code in a campaign that stole over 628,000 Ukrainian records. Nothing reported says the tool itself was flawed. The practical point is that prose is now a payload. Scanners that inspect package code but not documentation can miss manipulation that lives only in the docs.

Where the boundary should sit

Techpresso covers Nadella's essay, which separates the model, the orchestration software and the actions the model can take. The organization sets permissions, keeps tamper-proof records and holds an emergency brake that can pause a model mid-task. The brake is the hard part. An agent's state lives partly in its context window and partly in external side effects, so it cannot be halted safely at an arbitrary step without checkpoints and idempotent tools. Nadella gave no rollout timetable.

Recovery lags capability

A Fleet survey reported by TLDR IT found that 86% of IT teams let AI-generated configs or scripts into production, while 69% cannot roll back a bad change within an hour. The respondents were IT and endpoint teams, and no sample size is given, so read it as a directional signal for ML teams.

An agent that reads untrusted text and can see a credential will eventually be steered into leaking it. Remove the credential, because you cannot remove the text.

The move

This week, list every credential that an AI coding agent or IDE assistant on your team can reach: warehouse roles, cloud keys, feature store, model registry and package-publish tokens. Replace long-lived ones with short-lived, read-only, scoped tokens brokered outside the model's view. Then build a poisoned-documentation suite with held-out attack families, and gate agent releases on held-out attack success rate.