Frontier AI Regulator Announced - but will it make AI safe?

Regulation is only as strong as its reach. A regulator can only oversee the models it can access, and rules can only be enforced against companies with people, assets and operations onshore.

The Prime Minister has announced that Australia will regulate frontier AI companies through a new regulator empowered to impose significant penalties, backed by what the Government describes as world-leading obligations. It is a welcome step. The question is whether it will make Australians materially safer from AI harms, and where the gaps lie.

Gap 1: Reach - the regulator may have little to regulate

For foreign-based AI developers, the practical impact may be limited. For Australian (sovereign) AI developers, however, the obligations are substantial and include maintaining security controls, managing model risks, notifying government rapidly of incidents, submitting to independent evaluation, publishing safety and security frameworks, and keeping pace as standards evolve.

There is a catch, however. Frontier AI models are not currently trained in Australia, nor do they have operations or assets on shore. Much of how these companies operate may therefore sit beyond the regulator’s practical reach, while the compliance burden falls most heavily on local developers.

Attracting frontier model training onshore would give the new regulator real teeth. AI training data centres in Australia would bring development, safety research and evaluation within domestic oversight. Australia has clear advantages including a stable democracy comfortable with safeguards, space to build, and some of the highest solar resources in the world. Pairing regulation with an investment strategy could position Australia as a place for safe, accountable AI development rather than simply a market for imported models.

Gap 2: Scope - most AI harm is not at the frontier

The Government's approach so far concentrates on frontier AI models. This is a narrow view of AI safety. Serious risks to safety, public health and national security are not confined to the frontier.

AI and algorithmic systems are already embedded in healthcare, insurance, housing, finance, employment, education and social security. Most are not frontier models but smaller, specialised tools such as a recruitment screening bot, a personal budgeting app or an automated educational grading algorithm. These are the systems most likely to affect an individual’s job, loan, benefits or educational results.

Independent oversight appears limited to evaluating models themselves, rather than whether their development and deployment in particular contexts is safe, fair, responsible and accountable. A model can pass evaluation and still cause harm when deployed in the wrong setting, on the wrong data or without human oversight. Regulating context, not just capability, is where meaningful protection lies.

Gap 3: Government as a deployer of AI

There is a critical distinction between regulating those who build frontier AI and regulating those who use it. The new regime targets the former. It does not impose binding obligations on how organisations, including government agencies, procure and deploy AI.

Commonwealth agencies are not without guardrails. Beyond the laws that apply to all organisations, they are subject to public sector obligations and, since September 2024, the Digital Transformation Agency’s Policy for the responsible use of AI in government. But that is policy, not regulation backed by an independent regulator. What happens when an agency buys an AI product from a vendor that is not a frontier developer, and so falls outside the new regime?

These questions intersect with existing Commonwealth obligations under the Public Governance, Performance and Accountability Act 2013 (Cth), the Privacy Act 1988 (Cth), the Commonwealth Procurement Rules and administrative law.  In practice, an agency’s procurement contracts, due diligence and assurance processes will need to carry the weight in absence of regulation and ability to enforce.

For everyone operating outside the Government, consultations are occurring on adoption of AI in the workplace. A draft best practice guide on AI adoption in the workplace was issued recently for consultation. Whilst merely a ‘guideline’ information is being sought on how the private sector will adopt AI to understand whether any amendments to the Fair Work Act may be required to protect the workforce.

The Opportunity

Encouragingly, the Government has signalled that voluntary principles are no longer sufficient for the most advanced AI systems. Mandatory reporting, independent testing and potentially criminal liability could materially change the responsibilities of AI developers and their directors.

The real test will be in the detail. The regulator’s independence, expertise, enforcement powers, resourcing and relationship with existing regulators will determine whether it becomes effective or simply another layer of bureaucracy.

Where to from here

  1. Extend oversight to deployment. Pair frontier rules with risk-based obligations for high-impact uses in sectors such as health, finance, employment and social services.

  2. Equip agencies. Provide Commonwealth agencies with practical guidance on AI governance, procurement, assurance, risk allocation and accountability, including model contract clauses.

  3. Build reach through investment. Align regulation with incentives for onshore AI training and evaluation infrastructure so the regulator can oversee what matters.

  4. Coordinate regulators. Define clearly how the new regulator works alongside the OAIC, ACCC, ASIC and sector regulators to avoid gaps and duplication.

Regulating sci-fi frontier risks, such as rogue AI agents attacking government systems or critical infrastructure, is good politics and sound policy. But the harms Australians are most likely to experience are quieter and closer to home. Whether the new regime keeps Australians safe will depend on whether it reaches beyond the frontier.

We will be closely watching the Government’s proposed legislation and the detailed design of the new regulator.

In the meantime, if you would like to discuss how these developments affect your AI contracts, procurement or regulatory obligations, please get in touch.

Next
Next

Lawyers are using AI. But for what?