Australia NDB scheme: when an AI prompt becomes an eligible data breach

On a Monday morning, an HR manager in Melbourne pastes a spreadsheet into a consumer AI assistant to prepare a summary: names, salaries, sick leave, tax file numbers. She realises her mistake an hour later. The question that then lands on the compliance manager's desk is simple: is this a data breach that must be notified?

In Australia, the answer runs through the Notifiable Data Breaches scheme. It sets out clear steps and a 30-day assessment window. The hard part is knowing how to apply it to an incident involving AI.

The NDB scheme in brief

The scheme sits in Part IIIC of the Privacy Act 1988 and has applied since 22 February 2018. It covers entities bound by the Australian Privacy Principles: Australian Government agencies and most organisations with an annual turnover above 3 million Australian dollars, as well as some small businesses such as health service providers.

A breach is an "eligible data breach" when three conditions are met:

• there is unauthorised access to, unauthorised disclosure of, or loss of personal information;

• it is likely to result in serious harm to one or more individuals;

• the entity has not been able to prevent that likely risk with remedial action.

In that case, the entity must notify the Office of the Australian Information Commissioner (OAIC) and the affected individuals as soon as practicable. The numbers show this is far from theoretical: the OAIC received 1,113 notifications in 2024, a record year for the scheme.

When a prompt becomes an eligible data breach

Pasting personal information into an external AI tool is not automatically a notifiable breach. But it can amount to an unauthorised disclosure, especially if the tool has not been approved by the organisation, if no contract covers the data, or if the provider may retain the conversations or use them to train its models.

Everything then turns on serious harm. A first name and a meeting date carry little weight. Tax file numbers (TFNs), Medicare numbers, health information, bank details or identity documents are a different matter. In its October 2024 guidance on commercially available AI products, the OAIC itself recommends not entering personal information, particularly sensitive information, into publicly available generative AI tools. We covered that guidance in our article on the Privacy Act, the OAIC and generative AI.

The 30-day assessment

When an entity has reasonable grounds to suspect an eligible data breach, the law requires a reasonable and expeditious assessment, and asks it to take reasonable steps to complete that assessment within 30 days. For an AI incident, the assessment has to answer practical questions:

1. Which tool was used, and with what kind of account: personal, free, or business under contract?

2. What data was sent, about how many people, and how sensitive was it?

3. Does the provider keep conversations, and for how long?

4. Can deletion be obtained and confirmed in writing?

5. Could the data have been reused, shared or exposed to other users?

Without a log of what was sent, the second question is very hard to answer. That is often where assessments get stuck.

A four-step response playbook

The OAIC suggests a four-step approach: contain, assess, notify, review. Applied to AI, it looks like this:

• Contain: delete the conversation, switch off chat history, ask the provider to erase the data and suspend access to the tool if needed.

• Assess: reconstruct the data sent, weigh the risk of serious harm and document every decision.

• Notify: if the breach is eligible, prepare the statement to the OAIC and inform affected individuals, with practical advice.

• Review: fix the root cause, adjust AI usage rules and make sure the same action cannot happen again.

Remedial action matters a great deal. If the entity acts quickly enough that serious harm is no longer likely, notification may not be required. But it has to be able to prove it.

The cost of getting it wrong

Since the 2022 reforms, a serious or repeated interference with privacy can attract a penalty of up to 50 million Australian dollars, three times the benefit obtained, or 30% of adjusted turnover for the relevant period, whichever is greater. The 2024 reforms added mid-tier penalties and infringement notices. Beyond the fine, a late or poorly handled notification damages customer trust. Our article on the real cost of shadow AI looks at these effects.

Prevent rather than notify

The best incident is the one that never happens. A control at the moment an employee pastes or uploads data can detect a TFN, a Medicare number or a medical record, then anonymise, warn or block. Every event is logged, which feeds directly into the 30-day assessment if an incident does occur. To see how a leak unfolds step by step, read our article on the anatomy of an AI data leak, and for deployment, our 90-day AI DLP rollout playbook.

Summary

In Australia, a prompt containing personal information can become an eligible data breach under the NDB scheme. Entities then have to assess the incident within 30 days and notify the OAIC and affected individuals if serious harm is likely. A ready-to-use playbook, reliable logs and a control at the moment of pasting make all the difference.

ZeusLock detects and anonymises TFNs, Medicare numbers and other sensitive data before they reach an AI assistant, and keeps the record you need for an NDB assessment. Book a ZeusLock demo to get your incident response ready.