AI News Wrap-Up: Four Labs Were Asked If They Insure Against Catastrophe, and Nobody Raised a Hand

A leash lying slack on the ground beside an open gate, flat geometric illustration in blues, no people

AI News Wrap-Up: Four Labs Were Asked If They Insure Against Catastrophe, and Nobody Raised a Hand

Five stories from the past week with something in them for a finance function — a council hearing that turned into a liability question, consumer agents already logging into accounts, a model selling jurisdiction rather than benchmarks, a copyright ruling that recognises a market for training data, and 25,000 Danish workers who confirm neither narrative.

There is a question finance people ask instinctively and technologists almost never do: who carries the loss? It came up in a council chamber in New York on Monday, and the answer — a room of silence — is the most useful thing in this wrap.

1. Asked whether they insure against catastrophic AI risk, none of the four labs raised a hand

On Monday 5 October the New York City Council held a Committee of the Whole hearing on a package of AI bills — its first of that kind since 2022. Google, OpenAI, Anthropic and Meta all appeared, three of them only after the council signalled subpoenas. Pressed on how likely a catastrophe is, the witnesses Fortune quotes gave no figure: OpenAI's Morgan Dwyer said any chance at all was "unacceptable" regardless of likelihood, and Google's Alice Friend said forecasting it is "not a perfect science at this stage." Then came the question that travels. Speaker Julie Menin asked the witnesses to raise a hand if their company carried insurance against catastrophic risks. None did, Fortune reported; Menin's reply was that she assumed the public would then absorb the costs.

Tim's take: Skip the extinction debate; that isn't where the information is. The information is that none of four of the world's best-funded AI developers indicated it insures this risk, and the two quoted on its likelihood wouldn't put a number on it. Insurance is how an organisation says "this is quantifiable and someone else will carry the tail." Nobody raising a hand doesn't tell you why — no cover on offer, self-insurance, or caution in front of a council — but each leaves the tail with someone other than an insurer. So read the limitation of liability clause before the feature list; that is where your share is decided. The cap, the exclusions, and what happens when the tool acts rather than drafts matter more than the model version. I'd also stop hearing "we can't quantify it" as evasion. It's a disclosure.

Source: Fortune

2. Meta's Muse is already showing up in web logs, and it doesn't say it's an agent

Meta launched Muse, its consumer AI agent, on 8 September. This week the security vendor Cequence published analysis of 1–24 September traffic across customers in financial services, retail, travel and software: it says Muse traffic appeared at more than half of them within two weeks of launch and grew nearly sixfold from its first week to its most recent, varying by industry. Muse runs a real Chrome browser in the cloud and does not identify itself as an agent, so most tooling reads it as a person; Cequence says it detected the traffic behaviourally. At financial institutions, it says Muse logged into customer accounts and completed multi-factor authentication on users' behalf. And in late September one travel customer began blocking nearly one in five Muse requests — only those that looked abnormal. Treat the figures with caution: this is a paid press release from a vendor that launched a product detecting exactly this alongside it. Separately, on 6 October Meta, Walmart, Stripe and Bret Taylor's Sierra Technologies announced the Personal Agent Protocol, an open standard meant to authenticate agent purchases and show businesses what agents do on their sites.

Tim's take: Two weeks ago I covered "know your agent" — payment networks building a framework, six banks publishing principles — as a prospective governance gap. This is the same problem with a date on it and traffic in it. The question is no longer whether agents will transact with you, but whether you would see them if they already were. So ask whoever runs your website, portal or member system whether they can tell automated sessions from human ones, and what the answer rests on. If it's "we assume a browser is a person," that assumption is now being tested at consumer scale, and any control resting on it inherits the error. The MFA detail is the one I'd escalate. Cequence doesn't say how the second factor was completed, but either way the customer let an agent through both steps and your log shows a clean authenticated session. Nothing in it is false — it just doesn't mean what the control assumed. The 6 October protocol is one industry answer, and it is days old and voluntary.

Source: Cequence via Yahoo Finance, SiliconANGLE

3. Mistral's new flagship is selling jurisdiction, not benchmark scores

French lab Mistral released Mistral Large 4 as a public preview on 6 October: a multimodal mixture-of-experts model of roughly a trillion parameters, about 52 billion of them active, trained on 3,800 Nvidia Grace Blackwell GPUs in Mistral's own European datacentres and served from the same infrastructure. Mistral says it will release the weights by the end of October and has not named a licence — so today it is a proprietary model with an open-weight promise attached, not an open-weight model. Its headline cyber benchmark scores are self-reported, and Mistral itself notes that several closed models score near zero on one of the tests because they refuse the task.

Tim's take: Trained and served in Europe, under European law — Mistral's "built for AI sovereignty" pitch — is a procurement claim, not a capability claim, and it names something most Australian AI purchasing leaves unnamed. Choosing a vendor is choosing a jurisdiction, a data-residency posture and a legal regime you will never negotiate with. For an NFP holding participant data that isn't abstract: it shapes who can compel disclosure, and by what process. A European sovereignty pitch isn't itself relevant to an Australian organisation — the EU isn't our jurisdiction either. The useful part is the question it makes obvious, which belongs on your vendor assessment whether or not Mistral ever comes up: where is this processed, under whose law, and who can be compelled to hand it over? On the self-reported benchmarks, wait.

Source: Mistral AI, LLM Stats

4. A US court just said there's a market for training data — which means there's a price

On 29 September the US Third Circuit affirmed judgment against ROSS Intelligence in Thomson Reuters' copyright case, holding that Westlaw's editorial headnotes were original enough to attract copyright and that copying them to train a legal-research tool was not fair use. The use was highly commercial and only minimally transformative, because ROSS used the headnotes for the same ultimate purpose as Westlaw: building a legal-research platform. On market harm, the court found damage not only in the legal-research market where the two competed, but in a developing market for licensing headnotes as AI training data. It expressly did not decide whether training a generative model on books or other expressive works is fair use, distinguishing ROSS's tool because it returned existing passages rather than generating new text. ROSS has said it will seek Supreme Court review.

Tim's take: Two caveats. This is US law, and Australia has no fair use doctrine at all — we have narrower fair dealing exceptions — so it isn't precedent here. And it's eleven days old, included because the considered analysis followed in the days after. What travels is the market-harm finding. Once a court accepts that a licensing market for training data exists, unlicensed use of someone's proprietary content starts to look less like a grey area and more like an unpaid invoice — at least where, as here, the result competes with the original. For most readers the exposure runs opposite to the headlines: not "will we be sued for training a model," but "what did we agree to when we pointed an AI tool at a paid subscription or a licensed methodology?" That permitted-use clause was probably written with a human reader in mind. It's a contract question this ruling doesn't decide. Worth checking.

Source: Ballard Spahr, Patently-O, LawNext

5. 25,000 Danish workers, no measurable pay effect — and one finding that isn't null

On 6 October Brookings published the authors' write-up of a Danish study, first circulated as a working paper last year, covering around 25,000 workers across about 7,000 workplaces in eleven AI-exposed occupations — accountants among them — and linking survey responses to monthly administrative records. The headline is a precise null: no effect of AI chatbots on earnings or hours, for users or adopting workplaces, with confidence intervals tight enough to rule out average effects above 2%. That is a stronger statement than "we found nothing." Underneath it, plenty moved. Adoption reached 93% where employers paired encouragement with enterprise tooling and training, against about 41% where they took no active steps. And the one non-null result is occupational mobility: adopters were more likely to have switched occupations, mostly into IT support and clerical roles, and switchers' earnings growth ran about 12 percentage points ahead of other Danish workers.

Tim's take: Read the date on the data before quoting the finding. The records run to December 2024, so this measures early chatbot use — people with a chat window — not the agentic tools being sold in 2026. It's strong evidence about a world that has already changed. What carries across is the adoption split, the only number here you control: 93% where tooling and training accompanied encouragement, about 41% where the employer did nothing. That's an argument for budgeting tooling, training and a clear signal together, not licences alone. The mobility finding deserves a quieter note. It's an association, not proof that training makes people leave — but if adopters are more likely to move, and to be paid more for moving, training your team well improves your productivity and your retention risk at once. Have that conversation before someone else does.

Source: Brookings

The thread, if there is one: four of these five are about things already true and not yet visible. Agents in web logs. Permitted-use clauses written for a different kind of reader. A risk nobody in the room would price. Capability built in people who can take it elsewhere. None of that is a decision about which model to buy, and all of it will outlast whichever model you choose.

This post is general commentary based on publicly available information and does not constitute legal or tax advice. Always seek independent professional advice before acting. A standing note for anyone using AI tools on payroll, participant or client data, or financial figures: confirm whether the vendor retains your inputs or uses them to train its models, and where there is a choice, prefer the configuration in which your data isn't retained for training. Check the setting rather than assuming the default, and strip identifying detail out wherever the task doesn't need it. This applies as much to tools staff adopt on their own as to ones the organisation has approved.

Do you know what your AI vendor contracts cap, exclude, and leave with you?

If nobody has read the liability clause since the trial was approved, that's the gap item one describes. PFL provides senior-level outsourced finance, management reporting, and AI automation for Australian NFP, NDIS, and SME organisations — including the vendor, permission and approval records that make AI use defensible before anyone asks to see them.

Talk to PFL →
Timothy, CPA is Managing Director of Professional Financelink (PFL), providing senior-level outsourced finance, management reporting, and AI automation for Australian NFP, NDIS, and SME organisations. 20+ years in finance leadership across NFP, NDIS and SME.

Comments

Popular posts from this blog

Google Gemma 4 Just Launched — And It Might Solve Finance's Biggest AI Privacy Problem

Claude vs Gemini for Australian Finance: An Honest Comparison After 12 Months of Using Both

Why NFP Boards Are Finally Talking About AI — And What the Finance Team Should Do Before They Ask