The Scientist

The power problem behind a tiny lift

ByteByteGo ran a naive power calculation on Netflix's reported numbers: two-sided α=0.05, 80% power, and an assumed coefficient of variation of about 0.33 for a binary retention-style metric. Under those assumptions, a 0.006% relative lift needs roughly 470M users per arm. Netflix detected it on a 10% traffic slice in four weeks. That implies heavy variance reduction (such as CUPED-style pre-period covariates), a much lower-variance metric definition, or a different unit of analysis. Netflix discloses none of these. The short-term lift is also about 19x the long-term one. That is classic surrogate attenuation: metrics like plays and CTR overstate durable value by an order of magnitude.

You will not be able to measure a Netflix-sized lift at your scale. GenRec's real business case is consolidation. It brings new surfaces and content types, such as games, into a stack that today depends on thousands of hand-crafted features. It also offers a reusable serving pattern. The model scores the whole catalog in a single prefill pass, with no token-by-token decoding, and its ranking head scores only catalog items, so it cannot recommend something that doesn't exist. Netflix ruled out off-the-shelf LLMs for three reasons: popularity bias, out-of-catalog suggestions and no personalization.

The context window is a feature store nobody reviewed

GenRec's prompt-building rules read like a feature-selection spec. Describe strong evidence in detail. Drop short views and hovers. Collapse a run of episodes into one entry. Spend extra tokens only on metadata for new, cold-start titles. Longer history shows diminishing returns. The corollary for your stack is that rewording a template is a schema change. Version and hash your prompt templates the way you version feature definitions, and log the exact serialized prompt at serving time so you can replay it offline.

Bloomberg's study shows what goes wrong with the same mechanism. Claude and ChatGPT steered profiles that looked wealthier toward more expensive goods and profiles that looked lower-income toward cheaper ones. The reporting gives no sample size, effect size or protocol. It also doesn't say whether the models were following a budget the user stated, which is legitimate, or inferring income from proxies like zip code, job title or writing style, which is the pattern consumer-protection law targets. Either way, any CRM field or loyalty tier you put in the context works as a feature with an uncalibrated weight that you never fit. A paired persona-swap audit tests for this: hold the query fixed and vary only the income signal. Assuming a standard deviation of 0.5 in paired log-price differences (estimate yours from a pilot), you need about 220 paired queries per contrast to detect a 10% price shift.

Your logs and your objective are drifting too

Morning Brew reports that Meta's Muse agent completes Shopify checkouts end to end, that Amazon has blocked Muse, and that Adobe sees AI-referred traffic up almost fivefold since last spring. Agent clicks break position-bias corrections, which assume a human scanning a ranked list. Dwell time and scroll depth mean nothing in an agent session. Adobe's finding that AI-referred shoppers convert better is observational: people who asked an assistant and then clicked were already high-intent.

On the objective side, a paper covered by Techpresso claims up to about 5x more within-session variety at equal engagement by making slate size adaptive instead of fixed. It names no diversity metric or dataset. 'Equal' requires a non-inferiority test, not a non-significant difference. Logs from fixed-size slates contain no examples of other slate lengths, so off-policy estimates of this change will be high-variance. The risk the paper addresses is real. SiriusXM's comedy channel aired 812 comics in April and 275 in September after it shifted toward famous names. GenRec itself reweights training examples with reward models so that binge sequences don't dominate.

An LLM in your ranking path turns prompt text into features, agents into users and small lifts into unmeasurable ones, so audit all three before you claim a win.

Why the poem trick gets past standard controls

CyberScoop describes how PoeLLM locates its command-and-control (C2) servers. It takes four words from a poem on GitHub and maps them through a hard-coded dictionary to generate server addresses at runtime. Network monitoring never sees a fixed C2 address. Operators rotate servers by editing the poem, with no malware update. On ML hosts, inference and training boxes pull from GitHub and model hubs all day, so a request for a public text file is ordinary allowlisted traffic.

The design has two weak points. The dictionary is fixed, so once it is reverse-engineered, defenders can read the poem and compute the live server set themselves. The poem also hides only the address. The callback still goes to a destination the host has never contacted. Compromised machines are used for exploit scanning, crypto mining and remote code execution. The last one matters most. A serving node holding credentials for a feature store or model registry turns a mining nuisance into a data breach.

Open weights remove refusal training as a control

Anthropic says GLM-5.3 can run autonomous cyberattacks at roughly the level of its original Mythos model. The weights are public, so anyone can strip the safeguards. I would discount the comparison. Anthropic is a competitor, it benefits from the threat narrative, and it released neither a test harness nor success rates. The structural point survives the discount. Once weights are public, refusal training stops being a security control.

AI Breakfast's numbers suggest the pattern extends past one model. Mistral Large 4 self-reports 93% on Cybench, about 37 of 40 challenges, with a 95% interval of roughly 80% to 97%. Forty self-scored challenges cannot rank models, though they do establish the capability. Anthropic's CyScenarioBench data shows its gating is effectively binary: Defense Access is blocked on 46 of 50 tasks, Red Team on none. A self-hosted open model with no classifier amounts to Red Team access by default. Mistral's Pierre Stock told Axios that wider availability lets researchers audit the technology and improves defenses. Both can be true. None of these scores measure the deployment-level exposure, which is what the model can reach when it runs local tools.

The platform underneath is exposed too

CSO reports that Dell patched 18 critical flaws across storage, server management and Kubernetes environments. They allow authentication bypass, root access, token forgery and remote code execution, and those layers often host training volumes and GPU nodes. Paul Nakasone claims AI has cut average adversary dwell time to 29 minutes. He discloses no methodology, so treat the figure as directional. Hourly batch anomaly scoring means an event waits an average of 30 minutes before it is scored at all, one minute longer than that dwell time.

Reconciliation gives the best return on effort. It is a simple join of GPU telemetry against job records, and it is high-precision. It keys on unaccounted GPU usage, which is where the attacker's revenue comes from.

Where Nano Banana 2.1's discount flips

Simplifying AI prices each request as the image cost plus input tokens times the input rate. Its prior prices are derived, not published: about $0.067 per 1K image and $0.50 per million input tokens. With the input rate up $1.00 per million, added input cost cancels the cheaper image at about 33K input tokens for 1K output and about 75K for 4K output. On a 1K image, a 500-token prompt gets 48.9% cheaper, a 30K-token request 4.1% cheaper, and a 60K-token request 27.4% more expensive. At 100K tokens the increase is 56.9%. These figures exclude output and thinking tokens, and Google hasn't disclosed what a reference image costs in tokens. Usage metadata holds the real numbers.

The headline features are the ones that inflate input: up to 14 reference images per request, plus conversational refinement and search results as context. Multi-turn edit sessions usually re-send history every turn. Per-turn input grows linearly, so session cost grows roughly quadratically with turn count. Re-sending only the latest image plus the references the edit needs, with search grounding opt-in, is now the main cost lever.

The forced October 29 migration is a distribution-shift event. Region editing ships with a testable invariant: pixels outside the mask shouldn't change, so SSIM or LPIPS belongs on the unmasked region only. Identity tracking is claimed for up to 4 people and 10 objects. Track face-embedding similarity across edit chains and find where it degrades.

The same arithmetic error on text models

AI Breakfast checked Mistral Large 4's list price against GPT-6 Astra's. It comes to 13.6% of Astra's input price and 8.4% of its output price. The multiplier depends on the input-to-output token mix:

Mistral Large 4 is a hybrid reasoning model. If thinking tokens bill as output, effective cost drifts toward the bottom row. Tokenizers also differ by vendor. For self-hosting, the baseline is Mistral's own API, not Astra. At $2.07 per million blended, each $10 per hour of node cost needs about 1,350 tokens per second sustained around the clock to break even. All of its benchmarks are self-reported.

One unit that survives repricing

Tom Tunguz projects $350B in inference spend by 2027, with no published method. He names three margin levers: smaller models, better harnesses and smarter routing. Each can hide failures on long-tail inputs. The unit to use is cost per successful task, inference spend divided by tasks that clear the quality bar. Cost per token rewards a cheap model needing three retries. Cost per success penalizes it. Showing a cheaper route is no more than 2 points worse at an 85% baseline success rate (one-sided α=0.05, 80% power) takes about 3.9K tasks per arm.