Let’s talk about Use Cases
Mistakes are the portals of discovery. - James Joyce
As discussed in the last Synlaw newsletter, as AI reshapes our economy, the regulatory path we choose will define our global competitiveness and security. Right now, Australia’s AI regulatory landscape is decentralised and includes significant reliance on voluntary AI adoption.
We know that in Australia, national and APS AI plans exist to drive adoption, but the real requirements sit scattered across a patchwork of existing ‘technology-neutral’ frameworks. The DTA’s Policy for the Responsible Use of AI in Government, effective 15 December 2025, does impose conditions on agencies, publish an AI adoption strategy by mid-2026, roll out mandatory training by December 2026, and maintain a registry of in-scope use cases.
But these remain disconnected layers of oversight, not a single standard. Under the Privacy and Other Legislation Amendment Act 2024, agencies must also map their automated decision-making (ADM) processes before the late-2026 deadline, another obligation, not a coordinated system.
This leans on the strengths of established regulators, but it lacks the certainty of a consolidated model. The result is that an AI developer in Australia must navigate privacy, online safety, consumer protection, and anti-discrimination law separately with no single rulebook to follow. In addition, the potential to be stung by evolving ‘tweaks’ to regulation puts the developer on a reactive trajectory, waiting for the next bad use case to swing the regulatory pendulum.
The clearest way forward is a harmonised regulatory, risk-based approach with real guardrails.
To see why, look at the international failures where poor categorisation of AI caused serious harm. We can learn from past mistakes.
What have we seen from use cases?
AI is often sold as a tool to eliminate human bias. Recent experience suggests otherwise. In the United States, a healthcare algorithm from Optum, applied to over 200 million people, used ‘cost of care’ as a proxy for health, and as a result systematically underestimated the needs of African American patients relative to white patients.[1] Amazon had to scrap its AI recruiting tool after it downgraded female applicants, having learned from years of male-dominated hiring data. This is likely not surprising to most, as we humans already know many examples of where data is skewed.
In the Netherlands, SyRI was an automated surveillance system built to predict the likelihood of individuals committing benefit or tax fraud.
It was halted by the courts for violating privacy rights under the European Convention on Human Rights, failing to strike a ‘fair balance’ between fraud prevention and citizens’ privacy. A separate Dutch childcare benefits algorithm wrongly accused 20,000 families of fraud: families lost children to foster care, some took their own lives, and the scandal brought down the government.
Michigan’s MiDAS system shows the risk of treating AI output as a final decision with no human in the loop: a 93% error rate, and 40,000 citizens wrongly accused of unemployment fraud. In the financial sector, an uncontained algorithmic error cost Knight Capital Group $460 million, and Zillow lost $881 million when its pricing model ran outside its validated range unchecked.
What do the regulations say?
The world has split into two regulatory camps. The United States, like Australia, relies on a patchwork of state laws, federal agency enforcement (the FTC, the EEOC), and executive orders. It’s flexible, but it forces international business to chase a moving target, defaulting to the strictest applicable standard just to stay compliant.
The European Union sits on the other side, having implemented the EU AI Act, a single horizontal framework that unifies AI regulation across every member state.
The US model runs on three overlapping layers: federal agency enforcement under existing statutes, a growing patchwork of state laws, and high-level executive orders. There is no federal AI Act, instead, the FTC polices consumer protection and the EEOC polices employment, applying old laws to new algorithms. Meanwhile, states like Colorado have gone further, legislating directly against ‘high-risk’ systems that make ‘consequential decisions’ in housing, healthcare, and credit.
This preserves flexibility but at the cost of volatility, as individual states set their own mandatory guardrails for frontier technology. California, the leading jurisdiction on AI development, has already moved. Governor Newsom signed Senate Bill 53 into law, enacting the Transparency in Frontier Artificial Intelligence Act (TFAIA).
Australia mirrors this ‘technology-neutral’ reliance, governed by a scattered set of laws including the Privacy Act 1988, the Online Safety Act 2021, Australian Consumer Law. But where the US has an executive branch actively preempting ‘onerous’ state laws to force a national framework, Australia has nothing but voluntary alignment between federal and state governments under the National Framework for the Assurance of AI in Government. We have adopted the fragmentation of the US model without adopting any of its correcting mechanisms.
When a US healthcare algorithm used past medical spending as a proxy for sickness, it underestimated the needs of African American patients, a failure of transparency, not intent. Australia has no equivalent rule requiring independent bias audits for high-stakes software, meaning the same blind spot exists here, simply waiting to surface.
The EU and A Risk-Based Approach
The EU’s consolidated model, categorising AI into Unacceptable, High, and Minimal risk, offers Australia a way out of the fragmentation of the US model and the ambiguity of the UKs. A risk-based approach matters for three main reasons:
Addressing Consequential Decisions: Australia is only considering mandatory guardrails for high-risk settings: employment, legal services, welfare, that Colorado has already legislated. Guardrails like these would have stopped a SyRI-style failure before it violated anyone’s rights.
Preventing Automation Bias: Mandatory human-in-the-loop requirements for rights-affecting systems are standard in a risk-based regime. Without them, Australia is exposed to its own MiDAS-style failure.
Ensuring Market Certainty: A single, unified statute lets developers follow one standard calibrated to risk, rather than navigate the maze of overlapping laws Australia currently asks them to work through.
For High-Risk AI (in the employment, education, healthcare sectors) the EU mandates ex-ante conformity assessments, strict data governance, and built-in human oversight to prevent automation bias. Australia is still considering mandatory guardrails for the same settings, and in the meantime relies on the Voluntary AI Safety Standard, voluntary by definition, for the applications that carry the most risk.
The EU takes a zero-tolerance approach to some AI applications outright, banning systems that manipulate, exploit vulnerable groups, or enable social scoring by public authorities. Australia has no banned-practices list at all. The closest equivalent, the National Framework for the Assurance of AI in Government, only encourages agencies to identify and mitigate risk. Encouragement is not, however, a prohibition.
Article 10 of the EU AI Act also requires training datasets for high-risk systems to be ‘free of errors and complete’, with mandatory bias testing before market entry. Australia’s answer is a patchwork: the Privacy Act’s APP 10 ‘reasonable steps’ standard, backed by anti-discrimination law, neither written with AI training data in mind.
This is where static policy and active systems thinking part ways. An agency or business that uses AI to accelerate steps without workflow discipline risks an unmanageable bottleneck, or worse, a breach of evolving protections like the Work Health and Safety (Digital Work Systems) Act 2026.
Compliance has to be built into the system where the decision is made, not bolted on afterward. We need to start from the fact that AI has no innate sense of organisational context or risk appetite and that this has to be designed in.
So what could this look like?
A risk-based approach, the model Australia is still considering, could categorise AI use cases so the level of scrutiny matches the potential for harm.
High-Risk Consequential Decisions
Services Australia already runs over 600 automated processes, from disaster-payment OCR to the myGov Digital Assistant. The Federal Government has committed $105.9 million to automate environmental and housing approvals, this is not hypothetical, the scale is already here. A risk-based Act would classify welfare and infrastructure approvals as ‘High-Risk’, closing the gap that let failures like Michigan’s MiDAS (93% error rate) and the Dutch SyRI system happen elsewhere.
South Australia Medical Imaging already uses Harrison.ai to detect 124 clinical findings in under 20 seconds. Without strict oversight, Australia risks the same failure as the US healthcare algorithm that discriminated against African American patients by proxying health with cost of care. Australia’s current safeguard, the Privacy Act’s APP 10 ‘reasonable steps’ standard, is reactive; a unified framework would make bias audits of training data mandatory before medical software reaches the clinic, not after.
Low-risk Productivity
Procedural applications belong in a ‘Low Risk’ category, protecting genuine productivity gains, like the Department of Parliamentary Services’ video captioning, or the National Library’s transcription of 58,000 hours of oral history.
So Australia can keep navigating a fragmented environment, or adopt a clear, risk-based standard. We could categorise AI by its actual impact, protect citizens from high-stakes harm, clear the way for productivity tools, and avoid the failures already seen overseas, while keeping our agencies safe and efficient.
The aim isn’t to avoid risk and kill innovation. We need to adopt systems thinking in order to harness AI’s potential with confidence. Safety and progress can be achieved with the right framework.