Companies building the most capable AI models would have to run a working risk management system and be held accountable for whether it works, under rules modelled on bank supervision and workplace safety law, federal Assistant Minister Andrew Charlton said in Sydney on Thursday.
Charlton, the Assistant Minister for Science, Technology and the Digital Economy, told the Sydney Trust and Safety Festival the approach would sit inside the National AI Standards the government has committed to legislate. His speech gave no timetable. The Prime Minister's July announcement said the standards were expected to be legislated early next year. No draft text or bill has been released.
The model Charlton described is what regulators call systems regulation. Instead of listing banned behaviours, the law sets a standard for a company's internal process and then tests whether the process holds. "Systems regulation places the onus on companies to build and run a rigorous process for finding, testing, reporting on and managing the risks of their systems, and then holding them accountable for whether that process works," he said. He named workplace health and safety, critical infrastructure security, prudential supervision of banks and parts of aviation safety as the templates.
He ruled out the two alternatives. Voluntary codes "fail the incentive test", he said, because firms cannot be expected to act against their own interests "amid a manic race involving trillions of dollars". Detailed rules would not keep up either. "At the pace this technology moves, which is months not years, those rules could be out of date before the ink is dry."
That is a change of course. The National AI Plan released in December 2025 said the government would build on existing legal and regulatory frameworks, "ensuring that established laws remain the foundation for addressing and mitigating AI-related risks", and the mandatory guardrails proposed in September 2024 were shelved when it was published. The government set up an AI Safety Institute at the same time, with $29.9 million in funding.
The trigger Charlton pointed to was the Medicare incident. Prime Minister Anthony Albanese disclosed at the United Nations on 23 September that an OpenAI agent had accessed public and non-public data on a Medicare statistics portal in June, and that the company took three months to acknowledge it. "It is why the Government is developing AI standards legislation, and why the lessons of the Medicare incident will shape it," Charlton said.
What the rule would actually require is still open. The speech said the duties would apply to "developers of models with the sharpest, most acute frontier AI risks" but did not define the threshold. A Prime Minister and Cabinet consultation paper published in September proposes minimum safety and security requirements for large-scale AI training in Australia, including reporting defined incidents to authorities. Submissions close on Friday. Neither document says whether a lab training its models overseas and selling access here would be covered, and no estimate of the compliance cost has been published.
Charlton framed the rules as compatible with building an industry, not slowing one. "If Australia wants a say in how these technologies are built and governed, we need to be a country that can host and build them, not only regulate them," he said. "Safety is not the brake on the AI opportunity. It is the ignition."
OpenAI has said it backs mandatory safety requirements, independent assessments and incident reporting, the ABC reported. No response from the Coalition or from industry groups had been published by Thursday afternoon.
The ABC reported the national standards are due by the end of the year, with legislation planned for 2027. Until a draft appears, the definition of a frontier developer and the reach of the duties to offshore labs are the two questions the speech left unanswered.
