The Product Desk

OpenAI's user-ops team moved all of its support-ticket classification onto the Decisions API as soon as it existed. That is the clearest sign of how little OpenAI had to build. In Latent Space's post-keynote interview, API product lead Nikunj Handa described no new model: existing GPT-6 Luna weights, constrained structured outputs, parallel batched questions with no reasoning step, and an inference stack tuned for time to first decision. Four weeks before DevDay it wasn't on the roadmap "at all." Then Jev launched, users asked for something similar, and internal teams wanted faster classification. "This whole thing started, like, a week ago," Handa said.

Version 1 makes its decisions zero-shot and leaves confidence calibration to a future model release. Calibration means a 0.9 confidence score is right about nine times in ten. Swyx argued that RLHF pushes models toward answers users want to hear at the expense of honest confidence, and Handa agreed calibration may need more work later. Swyx called a Jev-style API "honestly structured outputs," and the market behaved accordingly, with about 100 clones within two weeks.

The benchmark that shows where the value is

TLDR IT reported a separate test that supplies the missing evidence. Vincenzo Iozzo ran Jev on identity resolution, matching accounts in different systems to the same person, at five real organizations.

Paying 47 times as much bought a lower score. This is one author, one task and five organizations, and the source doesn't say whether Iozzo is independent of Jev's maker. Treat it as a hypothesis to run on in-house data before it reaches a board deck.

A lab can copy the interface in a week and sell it at Luna prices, but the copy launched without the property that let a small model beat frontier LLMs on a bounded task. AI Breakfast still called the Decisions API DevDay's sleeper launch, and on routing and gating paths the label fits: one answer from a fixed set deletes fragile parse-and-validate code. Removing that code is worth having, though it says nothing about whether the answer can be trusted at a threshold.

If an AI feature can be described as an API shape, assume a lab can ship it at the lab's own prices. Calibration and proprietary data are the parts it can't reproduce on that schedule.

What this changes in planning

Any feature pitched as "an API that does X" now has a defense problem, and the defense has to live elsewhere: proprietary data, calibration that can be proven, workflow depth, or distribution. The Decisions API is a sensible default for routing, tagging and fast eval judging. It is the wrong tool anywhere a system acts automatically once a confidence score crosses a threshold. Handa also said Luna picking one action at a time won't match Astra-class models that script long tasks.

Current classifier evals probably blend three numbers that belong apart: accuracy, time to first decision, and calibration. Score each on its own before choosing between the lab default, a specialist model like Jev, or an in-house model. Speed-only workloads go to the default. Threshold-driven ones need the calibration column to win. Last week's Last Week in AI roundup listed Jev as a research curiosity. A week later, a top lab had copied it.

Swyx told Latent Space he runs YouTube's A/B testing and community posts through the UI. YouTube exposes neither by API. OpenAI's own devex team does the same thing. Better Computer Use means having no API no longer keeps agents out, and a product's UI now works both as an interface agents operate and as a spec they can clone. Swyx copies "shitty SaaS" apps screen by screen with Computer Use plus Codex.

Teams tend to picture an agent squinting at screenshots. Computer Use lead Ari Weinstein said that screenshot-scroll-screenshot loop is giving way to accessibility trees (the structured page data screen readers use), DOM access, Playwright and generated JavaScript. Screenshots miss link targets and truncate labels. The bottleneck has moved to the environment. Accessibility debt now shows up as failed agent tasks, and page load time shows up as agent latency. A meaningful share of OpenAI's benchmark time goes to waiting for sites like doordash.com to load.

How agents arrive, and what the product team controls

Perplexity's trigger work carries the sharpest product lesson. It retired Scheduled Tasks because a timer alone wasn't enough; users needed events plus conditions. Its own examples include a daily report and watching a Linear project, along with replying to one client's emails. Those jobs currently belong to the notifications and digests a product already ships.

DoorDash handles the third route deliberately. The Information reports it is bringing its in-app assistant into iMessage, where DoorDash still does the talking. It also accepts being back-end fulfillment inside ChatGPT, and reportedly in Meta's Muse. The stated justification is a claimed nearly 50% higher grocery basket for assistant orders. That figure compares against "traditional orders," not a holdout group, and people building full-meal carts probably inflate it. Copy the meal-idea-to-full-cart flow, and hold off quoting the nearly 50% until someone runs a holdout. The Information's Martin Peers adds a useful counterpoint: chat adds little to repeat ordering, which apps already make "as easy as pie."

Agents are already in the support queue

Latent Space's hosts described using agents for customer service. One prompts his agent to act like "a very annoyed human." Any team that owns a support experience is already fielding some inbound contacts from agents. Most of those teams are probably not measuring them.

Calibrate the urgency. Town CEO Jean-Denis Greze forecasts that assistants will handle 40% of digital work time within a year, per Newcomer. He sells work assistants, so read that as the bull case. The no-regret moves are cheap. Make the product readable to agents and emit events they can trigger on. Then go surface by surface and decide whose assistant does the talking, the way DoorDash split iMessage from ChatGPT.

Yuechen Zhao, a partner at Informed Ventures, stored a fake card in Instinct's Vault after the agent told him it wouldn't see card numbers. Then he asked it to read them back. It did, in plain text, by running JavaScript. Nothing about this is mysterious. Autofilled values sit in the page's form fields, and any agent that can run scripts on that page can read them. Per The Information, Instinct's spokesperson said "users are always in control of their data and credentials" and did not address the exposure.

Separate the thing pitched from the thing done. The pitch was a sentence the agent said to a user. The thing done was reading the card. The new failure mode is that the security claim came from the model itself, an assurance its own architecture didn't support. This is one reported test with a fake card, not a confirmed breach. Meta says Muse keeps credentials in storage it can use but not see, and that claim is also self-reported.

Three designs that don't rely on the agent's promise

The sources disagree on how much a vendor can carry. TLDR Hardware treats Nvidia's design as the reference architecture buyers will hold every product against. CSO First Look reports analysts saying the vast majority of agent-generated problems fall outside the platform's scope. Both can be right. Nvidia has the principle correct: enforcement has to sit where the agent can't reach it. Identity and business-logic controls still belong to the team shipping the product, and no platform purchase changes that. DoorDash's details show what the work looks like in software. Pick one of four credential modes per tool and keep read tools apart from write tools. Security rules stay out of prompts entirely.

What an agent promises not to touch is copy. It is not a control, and only architecture makes it true.

Why buyers will ask this quarter

CyberScoop reports that National Cyber Director Sean Cairncross told CEOs, not CISOs, to track three things about their agents: who authorizes them, what they can access, and where they run. Questions framed for CEOs tend to become procurement checklists. Instinct's answer to the second question would have failed.

The forcing function is Zhao's own test. Run it against the product before a reporter or a prospect does. Then find every security assurance the agent generates on its own, remove it, and replace it with fixed, reviewed copy tied to a control the team has actually tested. An assurance with no tested control behind it does not ship.