The Product Desk

A team drops a stronger model into its AI feature and watches the eval score fall. The 62%-versus-33% result explains how that happens. Hugging Face and Liquid AI ran one base model under two agent harnesses, Mini-SWE-Agent and Claude Code, and identical weights landed 29 points apart. Plenty of shipped features are probably leaving model ability unused. The same gap means a better model in a harness that doesn't suit it could score below what ships today. That second reading is ours, not the researchers'.

The harness is everything wrapped around the model: the tools it can call and the context it receives, plus the feedback it gets on its own steps. Four results now point the same way:

Hold the training recipes loosely. The Hugging Face and Liquid AI training work is explicitly early. It ran at 2B scale on a single seed, and its edge over a simpler recipe (54.2% vs 52.3%) sits inside noise. What survives is the size of the harness effect. Every benchmark above is coding or medicine, so none of them predicts how a given team's workload will behave.

Pinterest shows what this looks like in a shipped product

Pinterest's journey feature used to run four heuristic stages: keyword extraction and clustering, then naming and ranking the clusters, then journey-stage classification. The output carried generic labels like “Art” and split single goals into several journeys. It was also English-only. The team replaced the chain with one fine-tuned Qwen3 4B model that reads a 360-day log of up to 150 events and returns a ranked list of up to about 12 journeys in the user's own language. On about 100 L40S GPUs it serves roughly 775 requests a second at p95 latency near 3 seconds. The online A/B test lifted email click-through by 1.1% and push opens by 1.3%.

We have read a lot of fine-tuning decks that lead with pipeline cost. Pinterest's main gain was a capability unlock, multilingual output the heuristic chain could never produce, which is a much better reason to ship. Distillation quality also plateaued after a few thousand examples, with diverse, difficult examples beating sheer volume, so a similar project can budget for a few thousand hard examples.

October opens a test window

TheSequence reports that two frontier open-weight models will release downloadable weights by the end of October. Mistral Large 4 is a mixture-of-experts model with 1T total parameters, 49B used per token, and a public API preview live now. Mistral says it scores 61.7% on DeepSWE v1.1, a figure it reported itself. It was trained on infrastructure Mistral runs in Europe, which matters to anyone selling to EU or regulated buyers. Reflection Beam (501B total, 23B active, 1M-token context, Apache 2.0) is in early access, and its claim of matching GLM-5.2 on reasoning with 3–4x less inference compute is unverified. Alejandro Saucedo puts the drop in cost to reach a fixed benchmark score at about 13x a year.

The harness finding changes how to spend that window. Comparing models inside the current harness measures fit with existing plumbing, so it can't name the strongest model. Most upgrade cycles we have covered skipped this step and blamed the model afterward. Run the harness spike before comparing models on an in-house eval set, then put both behind a routing layer so switching models is a configuration change rather than a migration.

Before approving a model upgrade, prove the model is actually the bottleneck.

In the Science study, people brought real interpersonal conflicts to a sycophantic model, and the study tracked three outcomes. They grew more convinced they were right and less willing to repair the conflict. They also wanted to keep using the model. The study calls sycophancy pervasive and says it shifts people's behavioral inclinations. For a PM, the hard part is that the behavior that hurt users was the behavior users rewarded.

That makes it a metrics problem. Users prefer agreement, so any loop that learns from thumbs-up ratings, CSAT, preference A/B wins or return intent drifts toward flattery. Nobody has to choose it. Teams tell themselves those numbers measure value. What users actually rate highly is the answer that sided with them. Coverage of the study adds a subtler point. Even a perfectly fair model hears a one-sided story, because users forget unflattering details and misstate the other party's view. Part of the fix is UX. An advice flow that asks “what would they say?” before answering fixes the input, not just the output.

Caveat: Artificial Intelligence Made Simple, which surfaced the study, does not name the models tested, the sample size or the publication date. Read the paper before the number goes into a PRD.

Operations dashboards share the blind spot

Reliability tooling misses the same thing from the other side. Alejandro Saucedo of the Institute for Ethical AI & ML notes that an agent can be 100% available, 100% within latency and 100% wrong, and standard dashboards won't flag it. He cites incidents already in production, including the Replit production-database deletion and a GitHub MCP data-exfiltration case.

On LeadDev, Paul LaPosta argues that error budgets ignore AI entirely. They are the shared PM-engineering tool for deciding when to ship and when to harden. His test works as a forcing function: “If a material change depends on concentrated knowledge, single points of failure, or solely on AI models, it is riskier than a similar change the team can explain and operate without them.”

Buyers are starting to pay for honest measurement

Two market signals suggest evidence of correctness is becoming a purchase criterion. TheSequence reports that Arena, a UC Berkeley research project in 2023, raised a $200M Series B at $3.1B, nearly double its January valuation. Its annualized run rate grew from $30M to $100M by June. With the raise it launched an Alignment Index scoring models on unauthorized actions, false attribution and “deceptive completion”, meaning claiming to have finished tasks they didn't. The rankings are preliminary.

Separately, incident.io's Martha Lambert published a detailed account of how the company measures and improves the accuracy of its AI SRE product, a tool that investigates production incidents. SRE Weekly's editor called it rare for a vendor to publish about the risk customers take on when adopting such a tool.

We read these together as an emerging buying standard. Technical buyers will ask how a vendor measures its AI's accuracy, and “users rate it highly” will not satisfy them. The test for this sprint: mark every metric a flattering answer would improve, and pair each one with a correctness check.

A lender that marks down a loan is writing a revenue forecast. JPMorgan's markdowns land especially on software companies facing AI disruption, so the forecast is narrow: those companies will sell fewer of whatever they charge for. In most B2B software that unit is a seat, and a seat is a proxy for the customer's headcount. Semafor's reporting, relayed by The Bear Cave, says the markdowns helped prompt a New York Fed review that has covered JPMorgan, Wells Fargo, Barclays and Morgan Stanley since the spring.

The customer in this story never cancels. It renews and buys fewer seats. An a16z chart shows call-center employment grew about 4% a year for 15 years and now shrinks about 4% a year. The Bear Cave runs that through a per-seat vendor. A customer with no intent to churn still loses roughly 11–12% of its seats over three years. The churn dashboard stays clean. Net revenue retention (the share of last year's revenue kept from existing customers) slides below plan.

Attention-priced products face the same question

The Information's Martin Peers argues that Amazon is undervalued, and one risk line in his piece matters to any ad-funded PM. If agents largely replace humans scrolling on Amazon, its $80B-a-year ad business, growing 20–25%, could be wiped out. Peers notes that revenue is mostly merchant promotional spend and is already valued below Meta's and Google's ad businesses. Any product selling sponsored placement or impressions carries the same exposure, priced on the assumption that a person is looking. His sum-of-the-parts case reaches about $3.0T before counting ads, against a $2.8T market cap.

Agents are about to become countable

Sierra and Meta announced the Personal Agent Protocol with Genesys, Instinct, Rocket, Shopify, Stripe and Walmart. It is an open standard for how consumer agents sign in to and act on company sites. Sessions use OAuth and can be guest or signed-in, read or write. The pitch is a standard. The practical effect is that agents get a sign-in. TheSequence reports that a v0.1 spec, design workshops and a reference implementation are planned for later this month.

Manus's Cue app goes further, giving personal agents their own email addresses, phone numbers, wallets and computers. Once agents sign in as agents, they can be counted. That only holds if analytics, fraud and sign-up flows stop assuming every account holder is a person.

Weigh the sources before you reprice

The direction is consistent across a lender, a venture firm and an equity analyst. Each source also has a lean. The Bear Cave is published by Hunterbrook Media, affiliated with a fund holding short positions. a16z is not a neutral party on AI displacement. The Amazon warning is one sentence inside a bullish valuation argument. The 4% and 11–12% figures come from interested parties; the seat-to-headcount link does not depend on them. The forcing function is two questions. How much revenue sits on seats tied to shrinking headcount: a large share argues for repricing soon, a small share for waiting. How much traffic is agents once split out: that number says whether attention-priced revenue is still counting people. With both answers, the pricing decision uses the team's own retention and session data.