The Nurse & ChatGPT: A Privacy Breach Waiting to Happen
A registered nurse on a busy hospital ward needs to complete a discharge summary before end of shift. Short on time, she opens ChatGPT and types:
What just happened: Identifiable health information has been entered into an unapproved external AI service. Assess the authority for disclosure, service settings, processing location and applicable privacy obligations. The prompt alone does not establish whether consent or another legal basis existed.
Clinical AI Risks
AI in clinical settings introduces patient safety, regulatory, and liability risks that administrative AI governance frameworks alone cannot address.
Diagnostic AI & Clinical Decision Support
AI tools used in radiology, pathology and clinical decision-making may meet the definition of a medical device depending on their intended purpose and functionality. Unless excluded or exempt, a medical device generally needs inclusion in the ARTG before supply. Clinical validation and applicable post-market responsibilities remain important to patient safety.
TGA non-compliance + patient harm liability if AI diagnostic errors are not governed.
AI Treatment Recommendations
Generative AI used to suggest treatment protocols, medication dosages, or care pathways without appropriate clinical oversight creates accountability gaps. When a clinician acts on an AI recommendation that results in patient harm, questions of responsibility, and insurance coverage, become complex without clear governance documentation.
Unvalidated AI treatment recommendations create uninsurable clinical liability gaps.
Autonomous AI in Care Coordination
AI agents increasingly assist appointment scheduling, patient triage, discharge planning and referral coordination. Assess uses in aged care and mental health against clinical safety and duty-of-care obligations. From 10 December 2026, qualifying automated decisions also require privacy-policy transparency under APP 1.
Assess care-coordination AI for clinical safety and applicable ADM transparency obligations.
Administrative AI Risks
The greatest volume of AI-related health privacy breaches comes not from clinical AI, but from staff using general-purpose AI tools in their daily workflows.
ChatGPT for Patient Notes & Discharge Summaries
A nurse in a busy ward uses an unapproved public ChatGPT service to draft a discharge summary, entering the patient’s name, date of birth, diagnosis, medications and treating physician details. This creates serious health-information risks. The hospital should contain the incident and assess the service arrangements, authority for disclosure, applicable privacy laws and any data-breach notification obligations.
AI Medical Scribes & Clinical Documentation
Ambient AI scribes listen to patient-clinician consultations and generate clinical notes. Providers should assess consent, data handling, vendor arrangements and applicable health-privacy obligations, including where recordings are processed. My Health Records Act requirements depend on whether and how the system’s information is involved.
Medical Staff Using Consumer AI for Research
Medical staff searching for treatment evidence, drug interactions, or clinical guidelines using general-purpose AI tools may inadvertently include patient-specific context in their queries, e.g., 'my patient has condition X and is on medication Y, what dose adjustment is appropriate?' This patient-identifiable query is processed by an external AI service not covered by any BAA or data processing agreement.
Administration Staff & Billing AI Tools
Administrative staff at hospitals and clinics are adopting AI tools to process insurance claims, manage Medicare billing, and draft correspondence. These tools often require patient health and financial data as input. Without data classification controls, staff may use unapproved AI tools that expose Medicare numbers, private health insurance details, and sensitive diagnosis codes.
Australian Healthcare AI Regulatory Framework
Healthcare AI in Australia sits at the intersection of multiple regulatory regimes. Understanding each obligation is the first step to effective governance.
Health Data Sovereignty
The My Health Records Act governs collection, use and disclosure within the My Health Record system. Authorised access and handling are required. OAIC explains that, after lawful download to a provider’s local system, most My Health Records Act rules no longer govern subsequent handling; the Privacy Act, state or territory privacy laws and professional obligations apply instead.
Sensitive Information & APP Compliance
Health information is sensitive information under the Privacy Act. Relevant obligations include APP 3 collection requirements, APP 6 use and disclosure, APP 8 overseas disclosure and APP 11 security. From 10 December 2026, qualifying automated decisions using personal information also require privacy-policy transparency about the relevant information and decision types.
AI Medical Device Regulation
The TGA regulates software that meets the medical-device definition unless it is excluded. Medical-image analysis, diagnosis support, treatment recommendations and clinical-outcome prediction should be assessed by intended purpose and functionality. ARTG inclusion is generally required before supply unless an exclusion or exemption applies, with applicable assessment and post-market obligations.
Aged Care AI Obligations
The Aged Care Act 2024 commenced on 1 November 2025, alongside strengthened Aged Care Quality Standards. Providers using falls prediction, care-planning or medication-management AI should assess its role in safe, person-centred care, with appropriate human oversight and consent processes.
Learn more about AI governance and compliance frameworks
Healthcare AI Governance Framework
A practical five-step framework for hospitals, clinics, and aged care providers building AI governance programs that satisfy the OAIC, TGA, and accreditation bodies.
AI Tool Inventory & Risk Classification
Map every AI tool in use across your health service, clinical, administrative, and research. Classify each by risk level: clinical AI (TGA obligations), tools processing health records (Privacy Act/My Health Records Act), and general productivity AI used by staff. Shadow AI discovery is essential, most hospitals significantly underestimate AI tool adoption.
Aona discovers observed AI activity on supported, deployed endpointsPatient Data Protection Policies
Implement controls that prevent patient-identifiable health information from entering unapproved AI tools. This includes technical controls (DLP policies that detect health identifiers, Medicare numbers, diagnosis codes), staff policies, and approved-tool lists. Prevention is far less costly than NDB notification and OAIC investigation.
Block patient data from reaching unapproved AI, automaticallyClinical AI Governance Committee
Establish a governance committee including CMO/CNO, privacy officer, IT security, legal counsel, and clinical leads. This committee approves clinical AI tools, reviews TGA compliance, oversees incident response, and provides the board-level accountability that regulators increasingly expect from health services.
Document governance to support privacy, clinical safety and accreditation reviewsStaff Training & Consent Frameworks
Train all clinical and administrative staff on what health information can and cannot be shared with AI tools. Establish patient consent processes for any AI use that affects care decisions. Document consent in clinical records. Training logs and policy attestations are critical evidence in any regulatory investigation.
Training records + policy attestations reduce regulatory risk significantlyIncident Response & NDB Procedures
Document an AI-specific incident response procedure covering containment, assessment, notification and evidence preservation. Under the Privacy Act NDB scheme, take all reasonable steps to assess a suspected eligible breach within 30 days. Notify the OAIC and individuals at risk of serious harm as soon as practicable when an eligible breach requires notification.
Assess promptly; notify as soon as practicable when requiredHow Aona Protects Australian Healthcare Organisations
Secure clinical and administrative staff’s AI use with visibility, sensitive-data controls and evidence for your governance program.
Shadow AI Discovery in Healthcare Settings
Discover observed staff AI activity on supported browsers and desktop paths, including unapproved tools. Aona’s browser extension or native app must be deployed on the relevant endpoints.
Patient Data Protection Controls
Apply policies to detect and block sensitive patient information in prompts and files on supported AI paths. Configure and validate the controls for your health service’s requirements.
Audit Trails for OAIC & Accreditation
Use observed activity and policy-event records to support privacy investigations, governance reviews and accreditation work. These records complement the health service’s wider evidence and obligations.
Policy Enforcement Across Clinical Workflows
Roll out employee AI-use policies across clinical and administrative teams on supported, managed endpoints. Apply approved-tool access and sensitive-data controls with clear staff guidance.