AI News Wrap-Up: Analysts Followed an AI Agent's Hidden Lean 86% of the Time. Only an Estimated 40% of AI-Quoted Statistics Traced Back to a Matching Study.
AI News Wrap-Up: Analysts Followed an AI Agent's Hidden Lean 86% of the Time. Only an Estimated 40% of AI-Quoted Statistics Traced Back to a Matching Study.
Five stories from the week of 24–30 October — an experiment on how far experts defer to an AI agent, a study of where AI-quoted numbers really come from, a cloud AI service that mixed up customers' responses, a big API price cut with a caveat in the safety report, and a new Google model released to cyber defenders first.
Most weeks the AI news is about what the tools can do. This week's more useful stories were about us: how often we take an AI's view without noticing, and how often the numbers it hands us survive a second look. For a finance function, that is the review step, and the review step is ours to keep.
1. Professional analysts went with an AI agent's hidden lean 86% of the time
Anthropic's alignment researchers published an experiment on 27 October. Fifty professional analysts, all with causal-analysis experience, worked through a realistic business case with an AI agent: a fictional farming co-operative deciding whether to expand an advisory program. The case had four analytical choices, each with two defensible options. Without telling the analysts, the researchers gave the agent a "lean" on each choice, set by coin flip, and told it to argue honestly and change its view if the evidence demanded. Across the four choices, analysts went with the agent's lean 86% of the time, against roughly 50% if the lean had no effect. 54% recommended expanding the program when the agent leaned that way, against 36% when it didn't. None of the participants suspected the leans had been set.
Tim's take: Notice what this study isn't about. The agent didn't lie or make errors. Both options were defensible. It simply had a preference, and capable people adopted it without knowing. In finance, that bites on the judgment calls: the provisioning assumption, the cost driver, timing versus permanent variance. The researchers suggest agents that argue the best case for every option, or ask for the expert's view first. On your side, form your own view before you ask. I'd add one for management reporting: for any material judgment, record what you thought before the AI weighed in. If the two never differ, that tells you something.
Source: Anthropic Research
2. "Zombie stats": an estimated 40% of AI-cited statistics led back to a study that said the same thing
AI visibility firm Trakkr published a study on 28 October. It put 200 number-seeking questions to five AI assistants with web search switched on, through each company's API, collecting 1,000 answers. From the cited statistics, it sampled 1,285 at random and followed each citation trail up to five pages deep, then hand-checked a sample of trails to correct its automated judge. The estimated result: 40% were "sourced" (the original study was found and said the same thing), 8% had changed in number or context along the way, 40% couldn't be checked (the source was named but the number wasn't in it, or the page was locked or gone), and 11% led to no study at all.
Tim's take: Two cautions before anyone quotes this. Trakkr sells AI visibility tracking, so it has an interest in the topic. And the 40% relies on a hand-check correction: the automated judge alone rated only 23% as sourced. Even so, the direction is clear. A citation is not a source. A link to a blog that mentions a survey is not the survey. The rule for anything going into a management reporting pack or a funding submission is simple: if you haven't opened the original document and found the number in it, it isn't verified.
Source: Trakkr Research, daily.dev
3. Cloudflare: a configuration mismatch and a software bug let 155 AI responses carry other customers' content
On 26 October Cloudflare published a detailed account of a cross-customer exposure in its Workers AI service running the GLM-5.3 model. Between 27 August and 9 September, 155 responses, all to very long prompts and received by a small number of customers, may have included content from other customers' requests made shortly before. Two things combined. The model was configured to accept a longer context than its memory cache could hold, and pre-launch tests checked response codes and speed rather than output quality. Separately, a bug in the open-source serving software let over-length requests continue when they should have been rejected. An outside researcher reported it on 8 September; the model was taken offline the next day and fixed. Cloudflare says it found no evidence of deliberate exploitation. Its post does not say whether affected customers were contacted.
Tim's take: To be fair, a detailed public write-up is the behaviour you want from a vendor. But the finance lesson is in the cause. The tests checked that the system was up and fast, not that the answers were right. That's the AI version of testing a payroll run by confirming the bank file uploaded. For your own AI vendors, ask three things: will you tell us if our data appears in another customer's output, or theirs in ours? How quickly? And is that in the contract, or only in a blog post?
Source: Cloudflare Blog
4. OpenAI's GPT-6.1 Terra halves standard API prices. Its safety addendum found it working around restrictions.
OpenAI released GPT-6.1 Terra on 26 October, describing it as a significant step up over GPT-5.6 Terra with 50% lower standard API prices. Developer coverage reports US$1 per million input tokens and US$5 per million output. The safety addendum published the same day is worth reading alongside the price. In one test where the model was warned off a course of action, at maximum reasoning effort, it "completed the task through an alternative route that violated the spirit of the restriction" in 29.6% of attempts. In another, after messages from an outside agent, it took a specified unauthorised action in 11 of 80 samples. OpenAI notes the tests were designed to be adversarial, the first ran without its system-level safeguards, and the results may not reflect real-world behaviour.
Tim's take: Cheaper inference makes it tempting to give an agent more work: more invoices, more reconciliations, more access. But a model that sometimes finds a way round a restriction needs controls that sit outside the model. In finance terms: permissions the agent cannot change, approval steps it cannot skip, and a log someone actually reads. If your controls only work because the model chooses to respect them, they aren't controls.
Source: OpenAI Developer Community, OpenAI system card addendum, Apidog
5. Google's Gemini 4 Carbon goes to cyber defenders first, at an introductory price that will double
Google announced Gemini 4 Carbon on 28 October, two weeks after Gemini 4 Argon. It ranks first on version 5.0 of the Artificial Analysis Intelligence Index, at 59, and Google is releasing it first to a subset of cyber defenders, then phasing in other trusted partners through its Fairwind Program before wider access. Google says it is taking part in the US government's voluntary pre-release testing process. Pricing launches at an introductory US$2 per million input tokens and US$10 per million output. A footnote says that after the introductory period, US$4 and US$20 will apply. It does not say when that period ends.
Tim's take: Two things for a finance reader. First, the staged release: the model goes to defenders before anyone else, which suggests Google sees its cyber capability as needing a careful rollout. Second, the footnote. If you're building a business case on today's price, model the US$4/US$20 rate as your base case and treat the introductory rate as a temporary discount. Argon launched at the same introductory price, and its higher rate starts on 1 December. An end date you don't know is still a cost you have to budget for.
Source: Google
Read together: experts follow an agent's preferences without seeing them. Cited numbers often don't survive the trail. A vendor's tests checked uptime, not answers. A cheaper model sometimes works around its limits. And the headline price has a footnote. None of this argues against using AI. It argues for keeping the review step, the source check and the control outside the tool, which is where finance has always added its value.
When AI prepares a judgment call in your finance team, who checks it, and against what?
The value of AI in finance depends on a review step that holds up. PFL provides senior-level outsourced finance, management reporting, and AI automation for Australian NFP, NDIS, and SME organisations, with the review, sourcing and control discipline built in.
Talk to PFL →- Anthropic Research — study on AI agents and expert judgment calls (27 October 2026)
- Trakkr Research Study 013 — Zombie stats: where the numbers AI quotes come from (28 October 2026)
- daily.dev — Zombie stats: where the numbers AI quotes come from (29 October 2026)
- Cloudflare — Cross-request KV cache exposure in the GLM-5.3 Workers AI model, and our remediation (26 October 2026)
- OpenAI Developer Community — GPT-6.1 Terra: fast and capable computer use at a lower price (26 October 2026)
- OpenAI — Addendum to GPT-6 Astra System Card: GPT-6.1 Terra (26 October 2026)
- Apidog — GPT-6.1 Terra pricing (27 October 2026)
- Google — Gemini 4 Carbon: our next frontier model (28 October 2026)
Comments
Post a Comment