87% of CFOs Want a Human in the Loop on AI. Here's How You Actually Build That Governance Layer.

Thursday, 4 June 2026  |  By Timothy, CPA — Managing Director, Professional Financelink (PFL)

A piece of research from Workday caught my attention recently. It surveyed Australian CFOs on their approach to agentic AI — the kind of AI that acts autonomously, executes multi-step tasks, and makes decisions without waiting for a prompt at each step. The finding: 87% of CFOs insist on a human-in-the-loop requirement before any autonomous AI action is approved for their finance function.

That's an encouraging number. It suggests finance leaders are approaching AI autonomy with appropriate caution, rather than deploying tools and hoping for the best. But here's the thing: insisting on human oversight and actually building the governance structures that deliver it are two very different things. Most organisations are still doing the former without the latter.

This post is about the gap between the principle and the practice — and what it actually takes to build a governance layer that keeps AI-assisted finance processes accountable.

⚠️ AI & Data Privacy Reminder: When designing AI governance frameworks for finance, be deliberate about what data AI tools can access and act on. Public AI models should not process personally identifiable employee data, client financial records, or commercially sensitive information without appropriate controls in place.

87% Of Australian CFOs require a human-in-the-loop before approving autonomous AI actions in their finance function (Workday, 2026)	79% Of organisations are still in early stages of exploring agentic AI — despite 70% of Australian CEOs ranking AI as a top investment priority
7% Of finance leaders report high impact from their AI use cases — despite nearly three in five finance teams piloting or implementing AI projects (Gartner, 2026)	30/90/365 Day implementation roadmap recommended for finance AI adoption — structured rollout matters more than speed of deployment

Why "Human in the Loop" Is a Principle, Not a Process

The phrase "human in the loop" is used so frequently in AI governance discussions that it's become almost meaningless. Every vendor says their tool keeps humans in control. Every AI governance policy states that human oversight is maintained. But when you look at how AI tools are actually deployed in finance teams, the oversight mechanisms are often absent, informal, or inconsistent.

The gap matters because agentic AI — AI that acts autonomously across multi-step tasks — introduces a qualitatively different kind of risk than AI that simply assists with analysis. When a finance team member uses ChatGPT to draft variance commentary, the human reviews every word before it's used. When an AI agent is configured to automatically categorise transactions, flag exceptions, and push approved items through a workflow, the human touchpoints may be far fewer — and the error propagation potential is much higher.

One CFO quoted in the Workday research put it directly: "If we let it loose now, we'd spend more time fixing problems than getting benefits." That's not technophobia — it's a reasonable risk assessment. The governance architecture has to match the autonomy level of the tool being deployed.

The Three Layers of AI Governance in Finance

Building a functional human-in-the-loop governance structure for a finance team isn't complicated — but it does require intentional design across three distinct layers. Understanding these layers is the starting point; how you build them for your specific environment is where the real work is.

Layer 1: Approval thresholds. Any AI-assisted process that results in a financial output — a transaction posting, a payment instruction, a report distribution, an exception sign-off — needs a defined threshold at which human approval is required. Below the threshold, the AI can proceed autonomously. Above it, a human reviews before action. These thresholds need to be documented, communicated, and enforced by the system design, not by individual discipline. If your AI agent needs a human to say yes for anything over $10,000 but there's no mechanism to actually stop it from proceeding without that approval, the threshold is decorative.

Layer 2: Exception escalation design. AI tools fail in predictable ways — and the governance structure needs to account for those failure modes before they occur. When an AI agent encounters something outside its configured parameters — an unusual transaction pattern, a data quality issue, a workflow step that doesn't resolve cleanly — where does it go? The answer should be "to a named human, via a defined channel, with a defined response time requirement." Vague escalation paths produce ignored exceptions and unresolved errors that compound over time.

Layer 3: Audit trail completeness. Every autonomous action an AI agent takes in a finance context needs to be logged, timestamped, and attributable. Not because someone will review every entry — they won't — but because when something goes wrong (and at some point, something will), the ability to reconstruct exactly what the system did, in what sequence, and based on what inputs is essential. Regulatory expectations are moving toward stronger governance, accountability and explainability around automated decision-making in financial services. For finance teams, that makes audit trails a control requirement, not just an operational convenience.

What We Learned Building Our Own AI Agents

At PFL, we've built AI agents for specific finance workflow applications — including a ChatGPT-based email briefing agent and an onboarding assistant for a new team member. These aren't complex deployments by enterprise standards, but building them from the ground up gave us a close look at where the governance challenges actually sit in practice.

The most consistent finding: the governance design takes longer than the build. Setting up the automation is the easy part. Defining what the agent is allowed to do autonomously, what it must escalate, how outputs are reviewed, and how errors are caught and corrected — that's where the real design effort goes. Any team that skips this step and goes straight to deployment is taking on risk that will eventually materialise.

The second finding: the first version is always incomplete. Not because the build is wrong, but because you can't fully anticipate the edge cases until the tool runs in a live environment. Building in a structured review cycle — checking what the agent actually did against what you expected, adjusting the parameters, and refining the escalation logic — is part of the governance design, not an afterthought.

Note: The scenarios in this post are based on real experiences — mine and those shared by colleagues across the sector. Details have been modified slightly to protect confidentiality, and I've used a first-person perspective throughout for readability.

The CFO's Role as Governance Architect

The Workday research describes a shift in the CFO's role in an agentic AI environment: from producing financial outputs to architecting the frameworks that govern how autonomous systems produce them. That framing is right, and it points to a specific kind of leadership capability that not every finance leader has yet developed.

Governance architecture for AI isn't a technology skill — it's a finance and risk skill applied to a new context. The questions it requires are familiar: what could go wrong, who is accountable when it does, what controls would catch the error, and how would we know if the controls failed? These are the same questions a good finance leader asks about any process. The AI context just adds a layer of complexity around the speed and scale at which autonomous systems can compound an error before it's caught.

The 7% of finance leaders reporting high impact from AI use cases versus the nearly 60% piloting projects tells its own story. The gap isn't about which AI tools are used — it's about whether the governance and integration work was done properly before the tool was deployed at scale. Governance isn't the thing that slows AI adoption. Done right, it's the thing that makes AI adoption sustainable.

RELATED ON FINANCE INTELLIGENCE

On what it actually takes to build and safeguard an AI agent for finance workflows — the lessons from our own builds. Read: We Built a ChatGPT Agent for Finance. Here's What We Learned About Safeguards First →

Building AI governance for your finance function?

PFL provides senior-level outsourced finance, management reporting, and AI automation for Australian NFP, NDIS, and SME organisations. We've done the governance design work on our own AI tools — and we apply that experience to client engagements.

Talk to PFL →

Timothy, CPA

Managing Director of Professional Financelink (PFL). 20+ years in finance leadership across NFP, NDIS and SME sectors. Builds and deploys AI tools for finance workflows — and thinks carefully about the governance structures that make them trustworthy.

SOURCES

• Workday — How CFOs Can Govern the 'Black Box' of AI in Finance (2026)
• Gartner — Finance AI ROI Research (cited via Journal of Accountancy, April 2026)
• ASIC — Key Issues Outlook 2026
• CFO Connect — State of AI in Finance 2026

Tomorrow on Finance Intelligence: The finance leader's job description has changed. You're not just reporting the numbers anymore — you're running the programme. And most finance leaders are already doing it without realising it.