AI adoption in Australia is booming and so are the risks.
By mid-2025, 84% of enterprises were piloting or deploying generative AI tools (IDC ANZ survey).
Yet ACSC logged over 84,700 cybercrime reports in the same period, with a sharp rise in AI-driven scams and data exposure incidents.
Below are the seven AI security threats most likely to shape Australia’s risk landscape in 2026, backed by recent local data, guidance and industry trends plus pragmatic steps to start hardening now.
2026 won’t just test how much AI we can build. It will test how securely we can scale it.
1️⃣ Prompt injection attacks dominate enterprise AI breaches
As organisations wire LLMs into internal data and actions (RAG, agents, copilots), indirect prompt injection (malicious content that hijacks the model via PDFs, web pages or emails) will drive the majority of production AI incidents. OWASP now lists LLM01:2025 Prompt Injection as a top risk, mirrored by recent guidance from Microsoft and the UK NCSC emphasising defence-in-depth (content isolation, allow-lists, output verification, tool-use fences).
Why it’s urgent: Injection turns read-only tasks into “do” tasks such as exfiltrating data, rewriting tickets, or triggering downstream workflows without tripping legacy DLP or EDR.
2026 control focus:
- Treat all external and untrusted internal content as adversarial; sandbox and strip active instructions before ingestion.
- Separate system prompts from user/context; enforce tool-use allow-lists and deterministic checks on model outputs.
- Continuously red-team LLMs with automated adversarial tests mapped to OWASP LLM Top 10.
2️⃣ AI supply chain attacks rise sharply
From malicious models uploaded to public hubs to stolen tokens and poisoned datasets, the AI supply chain is now an attractive target. We’ve already seen Hugging Face Spaces incidents exposing secrets and research showing transferable backdoors that survive fine-tuning and propagate downstream. Expect attackers to increasingly weaponise community models, datasets, and agent tool integrations in 2026.
Why it’s urgent: Even “approved” models can be swapped or backdoored; agents inherit the blast radius of every tool they can call.
2026 control focus:
- Enforce artifact provenance (signing, checksums, SBOMs for models/datasets) and scan models pre-deployment.
- Lock down tokens/keys; rotate on any upstream incident.
- Vet agent tools like third-party SaaS APIs with the same rigor as code dependencies.
3️⃣ Deepfake scams become mainstream fraud
Regulators report takedowns of 14,000+ scam and phishing sites in two years, with ASIC averaging ~130 scam site removals per week. ACCC/NASC warns of fake news pages and deepfake celebrity endorsements driving investment scams; states have issued parallel alerts. Expect voice-clone phishing and “CEO fraud” to blend with AI assistants and BEC (Business Email Compromise) playbooks becoming in fact faster, personalised, and harder to detect.
Why it’s urgent: Losses are trending up again with $174m reported in H1 2025 to Scamwatch. Social media and fake websites being the dominant lures.
2026 control focus:
- Mandate out-of-band verification for payments/credential resets; adopt voice-clone and deepfake detection in high-risk workflows.
- Kill trust in screenshots and “news” by default; embed URL and domain verification into agent workflows.
- Train staff with AI-augmented simulations of fraud and BEC variants.
4️⃣ Privacy reform & automated decision transparency laws tighten
Australia’s privacy overhaul expands OAIC powers. It creates a tort for serious invasions of privacy, and introduces transparency duties for substantially automated decisions including some of these automated decision provisions carrying a two-year grace period to 10 December 2026. Expect enforcement on disclosures, data minimisation, and cross-border transfers (white-list regimes).
Why it’s urgent: Fines, litigation exposure and mandated disclosures collide with AI logging gaps and “ gray" IT model usage.
2026 control focus:
- Maintain decision registries for high-impact automated decisions (inputs, purpose, human oversight, contestability).
- Apply smart redaction and policy-based blocking for PII/health/financial data going into LLMs; keep audit-ready logs.
- Run Data Protection Impact Assessments (DPIAs) and Privacy Impact Assessments (PIAs) on new AI uses; update privacy notices to describe automated decisions.
5️⃣ AI-powered attacks target critical infrastructure
ACSC’s latest threat picture is showing persistent state actors, rising ransomware, and increased pressure on critical infrastructure. Expect adversaries to pair traditional intrusion with AI-powered phishing, scripting and data analysis which in return will help shortening dwell times and operationalising extortion. APRA’s CPS 230 (operational resilience) locks in upstream/downstream dependency risk expectations alongside CPS 234.
2026 control focus:
- Scenario-test AI outage and model abuse in your business continuity and incident response; measure MTTD/MTTR for AI-specific events.
- Tie model and agent access to criticality levels; require step-up authentication for sensitive actions.
6️⃣ Data & model poisoning move from research to reality
Poisoning pre-training, fine-tuning, and embeddings can subtly bias or booby-trap models. OWASP now tracks LLM04:2025 Data & Model Poisoning, and 2024–2025 research shows attacks that persist across downstream tasks.
2026 control focus:
- Curate training data with source allow-lists, hashing and anomaly detection; run poisoning diagnostics during training.
- Separate and monitor sensitive concept drift; re-verify safety guardrails post-fine-tune.
7️⃣ New safety rules hit AI platforms and enterprises
Australia’s eSafety actions are expanding to AI platforms including recent notices compelling chatbot companies to detail their child-protection measures and the first Australian court actions over deepfakes are moving forward. Enterprise risk is rising for brands whose tools can be misused or whose staff are targeted.
2026 control focus:
- Content safety controls (CSAM, self-harm, abuse) in enterprise AI interfaces; age-gating/abuse detection where relevant.
- Clear misuse reporting paths and takedown workflows.
⚡How to Prepare for 2026
CEO Insight : Bastien Cabirou, Aona AI
“2026 will separate those who use AI and those who understand it.
The winners will treat AI like an operational surface : observable, governed, and coached.”
Bastien Cabirou, CEO of Aona AI, outlines three priorities:
- Guardrails that enable, not block
→ Combine policy enforcement with real-time coaching. - Total visibility
→ Know which tools, data, and users interact with AI including every prompt, every policy. - Proof of control
→ Build audit-ready logs aligned to OAIC and APRA CPS 230.
Where Aona AI Fits
Aona AI delivers safe, smart, scalable AI adoption for Australian enterprises:
- Detects shadow AI across 5,000+ tools.
- Automatically blocks or redacts sensitive data.
- Provides adaptive in-flow coaching through Coach Aona.
- Generates audit-ready dashboards for compliance and ROI.
👉 Explore: How Aona Protects Enterprise AI.
Key Takeaways
- 2026 will be the year of prompt-injection, AI supply-chain abuse, and deepfake-enabled fraud, amplified by privacy and sector enforcement.
- Winning organisations will observe, govern and coach & not just block.
- You don’t need to slow down innovation to stay safe; you need guardrails that travel with the work.
AI security is now business security. And while the threat landscape is expanding fast, the solution isn’t fear. It’s visibility, control, and confidence.
Ready to make AI safe, fast?
Sign-up for Aona's 90 Days Generative AI Risk Discovery Trial. We’ll map your current AI usage, identify top policy gaps (prompt injection, data leakage, supply chain), and deliver a 90-day hardening plan aligned to OAIC reforms and APRA CPS 230/234 so you can scale AI without compromise.