The Data Your Finance Team Shouldn't Paste Into AI (And a One-Page Checklist for What's Safe)
The Data Your Finance Team Shouldn't Paste Into AI (And a One-Page Checklist for What's Safe)
Only 11% of Australian and New Zealand organisations have a formal AI policy. Here's a practical test finance staff can run before anything involving NDIS, aged care or payroll data goes into a prompt box.
The Shadow AI Problem Already Inside Your Finance Team
Your finance team is probably already using AI. Whether your organisation has approved it or not.
That's not a guess. A 2023 survey of Australian and New Zealand organisations by ISACA, the global IT governance association, found that 63% of employees were already using AI at work, while only 11% of their organisations had a formal, comprehensive policy governing its use. More recent data suggests the gap hasn't closed. The Josys Shadow AI Report 2025, based on a survey of 500 Australian technology decision-makers, found that 36% of employees have already input confidential company data into AI tools, including financial information, strategic plans and customer personal details.
Put those two findings together and the picture is clear. Most staff are using AI. Most organisations haven't told them what's off-limits. And more than a third have already put something sensitive into a tool nobody vetted.
This is what security teams call "shadow AI" — the unsanctioned use of AI tools outside any policy, procurement process or vendor agreement. It's the same pattern as shadow IT a decade ago, except the stakes are higher, because generative AI tools are built to ingest whatever you give them.
For a finance team, shadow AI isn't abstract. It looks like pasting a participant's support plan into a chatbot to summarise it before a meeting. Uploading a payroll export to check an award interpretation. Dropping a donor list into an AI tool to draft a segmented appeal. Each of these feels like a small productivity win in the moment. Each one also means participant, client, employee or donor data has left your organisation's control and landed on a server you don't own, under terms you probably haven't read.
|
63%
of ANZ employees are already using AI at work (ISACA survey, 2023)
|
11%
of ANZ organisations have a formal, comprehensive AI policy in place (ISACA survey, 2023)
|
|
36%
of Australian professionals have uploaded confidential company data to an AI tool (Josys Shadow AI Report, 2025)
|
US$670,000
average extra cost of a data breach where shadow AI was a factor — a global figure, not AU-specific (IBM Cost of a Data Breach Report, 2025)
|
Why NDIS, Aged Care and Payroll Data Sit in a Different Risk Category
Not all business data carries the same risk if it ends up somewhere it shouldn't. A budget variance commentary with no names attached is low stakes. NDIS participant data, aged care client records and payroll information are different, because they combine three things at once: individually identifiable people, government-mandated confidentiality obligations, and, in payroll's case, financial account and tax file details.
This isn't a hypothetical harm. IBM's Cost of a Data Breach Report 2025 found that breaches involving shadow AI cost an average of US$670,000 more than breaches at organisations with little or no shadow AI use, largely because unmonitored tools take longer to detect and contain. That figure is global and in US dollars, not an Australian-specific number, but the underlying mechanism applies here too: data that leaves your organisation through an unmanaged tool is data your incident response plan doesn't know exists.
Payroll data deserves particular caution because it sits at the intersection of two separate risks: the confidentiality risk of shadow AI, and the accuracy risk of AI tools that generate plausible-sounding but wrong answers when asked about award interpretation or entitlement calculations.
A One-Page Checklist: Is This Tool Safe for This Data?
Rather than banning AI outright, which tends to just push usage further into the shadows, give your team a simple test they can run in the moment, before they paste anything in.
Ask three questions:
1. Would this data identify a specific person if it leaked? Think names, dates of birth, addresses, NDIS or Medicare numbers, tax file numbers, bank details, or health and incident information. If any of these are in what you're about to paste, the answer is yes.
2. Has your organisation confirmed, in writing, that this specific tool doesn't train its models on your inputs — and is that covered by a signed agreement, not just a settings toggle you found yourself? Free, personal-account versions of consumer AI tools generally can't give you this assurance. Enterprise or business tiers, configured by your IT function with a data processing agreement in place, often can.
3. Could you explain this exact use — this data, in this tool — to your board, an NDIS quality auditor, or the Office of the Australian Information Commissioner, without needing to qualify it?
If the answer to question one is yes, and either question two or three is no, don't paste it in. That's the whole test.
| Data category | Typical fields | Free/consumer AI tools? | What's actually safe |
| NDIS participant data | Name, NDIS number, plan goals, support notes, incident reports | Never | De-identify fully first, or use an enterprise AI tier with a signed no-training agreement and organisational sign-off |
| Aged care client records | Name, Medicare number, care plan, medication and health notes, family contacts | Never | Same as above; check your provider agreement before enabling any AI feature that touches client files |
| Payroll & STP data | TFN, bank details, super fund, salary, leave balances, award classification | Never | Keep AI use inside your payroll platform's own permission-gated tools; never copy exports into a general chat tool |
| Donor & fundraising data | Name, address, giving history, payment details | Never | Use de-identified, aggregated segments only, or your CRM's built-in AI under a vendor data processing agreement |
| Aggregate management reporting | Budget variances, KPI commentary, board narrative with no individual-level data | Generally yes | Once your AI policy allows it — still confirm the tool's training setting before relying on it routinely |
The checklist works the same way regardless of sector. NDIS providers, aged care operators, other NFPs and SMEs all handle some mix of these categories, and the same three questions apply whether the tool in front of you is a general-purpose chatbot, an AI feature bundled into your practice management system, or something a staff member found and started using on their own initiative.
What Changes on 10 December 2026
From 10 December 2026, a new transparency obligation under the Privacy Act 1988 takes effect. Under new APP 1.7 and 1.8, introduced by the Privacy and Other Legislation Amendment Act 2024, APP entities that use a computer program to make, or substantially support, a decision that could reasonably be expected to significantly affect an individual's rights or interests must disclose this in their privacy policy — including the kinds of personal information used and the kinds of decisions involved. This is a confirmed commencement date, already legislated, not a proposal. It's worth being precise about scope, though, in two ways. First, "APP entity" isn't every business — it generally means organisations with annual turnover above $3 million, plus some smaller entities that are always covered regardless of turnover (health service providers, which capture most NDIS and aged care providers, are the main example relevant to this sector). Second, this is a disclosure obligation aimed specifically at automated decision-making, not a general licence or ban on AI tools, and the OAIC's detailed guidance on how to comply is still being finalised, with release expected before the December deadline.
For most finance teams today, day-to-day AI use — drafting, summarising, checking a calculation — sits outside this specific obligation, because a human is still making the decision. But if your organisation is moving toward AI-assisted eligibility checks, payment matching, or anything that could plausibly "substantially support" a decision affecting a participant, client or employee, now is the time to map where that sits, well before the deadline arrives.
Getting this right doesn't require a large governance program. It requires a short, sector-mapped checklist your team can actually use, and a habit of asking the vendor one direct question before anything sensitive goes anywhere near a prompt box. If your payroll AI use extends beyond simple checks into automated award interpretation, the accuracy risk compounds the confidentiality risk covered here — worth reading alongside this piece.
Not sure which of your team's AI habits are actually safe?
Building the governance and reporting discipline around new tools like AI is exactly the kind of gap Professional Financelink closes, alongside senior-level outsourced finance, management reporting, and AI automation for Australian NFP, NDIS, and SME organisations.
Talk to PFL →Shadow AI use in the workplace widespread: survey — ISACA research via Technology Decisions
One in three Australians upload sensitive data to AI tools, report finds — SecurityBrief Australia
'Shadow AI' increases cost of data breaches, report finds — Cybersecurity Dive
Cost of a Data Breach Report 2025 — IBM
Australian Privacy Law Update 2026: What APP entities need to know — Lander & Rogers
Consultation on Guidance for Transparency in Automated Decision Making — OAIC
Next week: what an NDIS provider's month-end close should look like once AI has actually earned a place in the process.
Comments
Post a Comment