The Product Desk

Priced against the work it does best

An engineer kicks off a 105-minute QA run. The agent emits actions, observations and notes the whole time, because computer-use work is output-heavy by construction. Astra lists at $10 per million input tokens and $50 per million output. That 5x asymmetry makes the lever task selection, not usage reduction. Vo's rule: premium model for stubborn coding, complex visual navigation and high-toil QA, everything else routed cheaper. Track cost-per-successful-task, not cost-per-call. A $4 session that clears a six-month-blocked feature is cheap. A $0.40 session that fails twice is not.

Adoption is configuration, not capability

Stripe's internal agent Kai is the counterweight to the demo. Lenny's Newsletter reports the first working version took two weeks by 1.5 engineers; it now serves more than 10,000 employees weekly at 86% adoption with fewer than 10 maintainers. Its features are projects, tool policies, skill routing, per-action human review. None make the model smarter. They decide who can call which tool, on which data, with which approval.

Two findings transfer. Hiding choices raised adoption: a small knowledgeable group sets the default model, the approved tools and which actions need human review, and everyone else inherits that setup. A settings page is an adoption tax with a security cost attached. And the skill library shows negative returns to scale. Stripe runs roughly 2,000 skills with about 50 in daily use, every addition pollutes the retrieval pool, and telemetry now promotes, merges or retires entries. A library that grows on a growth metric is degrading a quality metric.

Stripe's agents behaved unpredictably, overwhelmed core systems and nearly took production down. Prompts did not fix that; agent-specific identities, priority signals, load shedding and a hardened query layer did. The engineering manager's stated advice is to double the developer-experience and data teams before investing heavily in agents.

Where the sources disagree

Lenny's reads Astra as vindication: "SaaS is back. Buttons are back." Simplifying AI reports Browser Use going native on WebMCP, swapping human-oriented UI automation for deterministic HTTP calls. TLDR Fintech notes Anthropic's commerce blueprints stop merchant agents short of completing transactions. Agents can drive interfaces, and the rails are being built so they do not have to.

That reconciles as a sequencing rule, not a side. Driving a UI is capability-proven, slow and expensive. Deterministic endpoints for the top five user jobs stay cheap and reliable. Only the speculative version dies: an accessibility re-architecture justified entirely by a capability agents demonstrably have.

A computer-use test on your own product is a one-week spike. A headless re-architecture is a quarter. Run them in that order.

One cheap borrow: approval gates as trust architecture. Vo's support agent handles refunds but cannot issue one without a human-confirmed request, and it earned unsolicited five-star reviews within a week of launch. Autonomous on low-risk actions, human-in-loop where consequences are real. Hours to add now, brutal to retrofit across every call site later.

Four models, four different channel sizes

A developer asks four assistants the same question and gets four differently shaped answers. Anthropic widened retrieval from Opus's 11 sources to Fable's 15. OpenAI went the other way, narrowing from Sol's 9 to Astra's 5. Astra is also far less likely to change its answer under light paraphrase, so influence there concentrates in a few authoritative sources instead of spreading across content volume. That gives two lanes, not one strategy. Breadth-of-content plays compete for Fable's fifteen slots. Authority concentration is the only play that moves Astra's five.

The disqualifier no dashboard reports

Latent.Space's study empirically validated the markdown content-negotiation practice Ora and Vercel have been pushing, and found that serving failures actively discourage models from reading your content. This is a days-long infrastructure ticket with a silent failure mode. Server logs show model-crawler fetch errors. Agent user-agents get blocked by accident. Docs sit unread. Audit that before funding anything else, because content investment on an unreadable site returns zero.

The adjacent sweep is anti-recommendations. They are rare, they are real, they carry negative scoring weight, and being argued against is strictly worse than being unmentioned. The usual causes clear inside a sprint: stale documentation, an unresolved reputation thread, or a comparison page that ranks well and reads badly.

83% of the board is still in play

Only 28 of 161 categories have a single universally dominant primary choice. The rest are close contests plus what the authors call "always the vibesmaid, never the vibe" — products consistently named as the alternate and never the pick. That distinction is the routing decision for spend. A perennial alternate means the model has learned about the product and has a reason not to choose it first. That is a positioning problem, which is the kind you can read and fix.

Self-preference reproduces across model families. Fable and Opus default to Claude Code, Sol and Astra to Codex, Muse to Muse Code, SWE-1.7 to Devin. Grok defaults to Cursor, a third party, and there are documented cases of GPT models recommending Claude. The bias is structural and beatable. Reverse-engineering which sources Grok cites for a given category is a better use of a week than another feature-parity matrix.

The second, independent measurement

Daniel Miessler's reporting supplies the demand side. Google's AI Mode preserved query volume while external clicks fell 18.8 points, so referral traffic can collapse behind a search dashboard that still looks healthy. Tracy Durnell's server logs add the other distortion: 90% of visits under thirty seconds, and a single IP fetching 423,059 pages. Her own caveat is that server-side stats only roughly separate humans from bots, which makes "we'll figure out bot share later" a decision rather than a deferral. Two independent measurements now agree. The answer layer keeps the click, and the shortlist forms before the site is involved.

Read the fine print before you cite this internally

Astra is both the evaluator and a subject of the tracker. Name that entanglement first, before someone names it in a review. Gemini/Antigravity, GLM/Zcode and DeepSeek/DeepCode were excluded from run one entirely because of API errors and rate limits, so the panel skews toward accessible labs. Latent.Space is publishing rankings while soliciting business enquiries to develop the tracker. Every prompt and answer pair is inspectable, which is the correct counterweight. Spot-check the one category that matters instead of citing the aggregate.

No dashboard reports a model that gave up on your docs. That loss shows up as absence, and absence looks exactly like a slow quarter.

Supply can be revoked for reasons no contract review would flag

Somewhere there is a vendor scorecard for Cursor with rows for uptime and data handling, and no row for who owns the company. OpenAI ended Cursor's model access after SpaceX acquired it, citing contractual-compliance concerns rooted in past disputes with Musk's companies, per Last Week in AI. The cut had nothing to do with usage or spend; it followed the change in ownership. That risk sits on the same roadmap as everything below.

What neutrality costs, in a number

The headline score is about capability; the uncomfortable part is about architecture. On the neutral harness, which allows the model only visible notes, the run was portable across vendors. On the Provider Adapter harness, which preserves OpenAI's opaque reasoning state between requests, the same model scored higher and cost 28% less. Provider-native context management is a real multiplier rather than an evaluation trick, so a vendor-agnostic abstraction layer carries a performance and cost penalty that has to be quantified per workflow instead of assumed away.

Keep the caveat attached: ARC Prize said plainly "we are not claiming that it is AGI," and flagged that the benchmark has bounded scope and deterministic mechanics the real world lacks. The sample-efficiency result travels further than the score. Astra used fewer actions than the human baseline on 96.0% of levels and 51.7% fewer actions per level on average, which is why actions-per-task belongs on the dashboard rather than in trivia.

The failure a second provider still leaves open

Four independent providers failed inside one window with no identified common cause, which implies a shared upstream dependency — cloud, CDN, or network — that no vendor contract, status page or SLA exposes. TLDR Fintech's read on the bank response is the one worth funding, because only one of its four parts is about model choice.

Demand for portability did not move an inch

TLDR IT reports Snowflake posting strong quarterly results attributed in part to its CoCo coding assistant, with the stated reason being that buyers want tools that broker multiple models rather than lock into one maker, driven by CIO cost savings after earlier AI spending surges. Procurement is now demanding the exact property the harness data says costs score and money.

Price that contradiction per workflow instead of adopting a house position. Long-horizon computer use, and anything leaning on preserved reasoning state, earns the provider-native premium. Most paths do not, and there the abstraction layer pays twice, in swap-ability and in cost routing. Two moves narrow the gap cheaply: Anthropic's Enterprise Frontier Safeguards will store data on the customer's own cloud this fall, which resets what a security questionnaire is measured against, and GLM-5.3-Flash's open weights make a validated fallback affordable rather than aspirational. The forcing function for this sprint is one column per production workflow, listing what that workflow does when the primary provider returns nothing, and funding the rows that come back blank.

A second model provider buys integration surface. A degraded mode is what still ships when every provider is gone at once, which makes it the piece to test on a calendar rather than during the incident.