An actual AI benefit could be restricted.
More than 40% of Australian doctors use AI scribes — but neither they nor their patients can find out where the recordings go.
Somewhere in a GP clinic right now, a consultation is being listened to by software that will turn that conversation into medical notes before the patient gets home. The doctor gets back an hour of their day. The patient gets a practitioner who looks them in the eye instead of at a keyboard. That is a genuine, unambiguous improvement to a health system whose workforce is burning out under administrative weight. And it could easily be strangled by the uncertainty surrounding what happens to the recording after the appointment ends.
More than four in ten Australian doctors are already using these tools
The scale here is not trivial. According to a report by Digital Rights Watch, AI scribes are now used by more than four in ten Australian doctors. The tools work by recording consultations and feeding the audio into large language models that generate clinical notes. For a system where Medicare compensates doctors for face-to-face time but not the paperwork that follows it, the financial logic is hard to argue with. Neither is the human cost of the alternative: clinician burnout is a structural problem in Australian healthcare, not a personal failing.
But the productivity case has a catch. Early studies found up to 90% of AI-generated clinical notes required some correction. More recent research puts the figure for errors significant enough to affect diagnosis at around 20%. That is not a rounding error. A misattributed symptom, a hallucinated medication, a culturally garbled history for a patient from a non-English-speaking background, any of these can follow a patient through the health system for years. And the productivity gains the technology promises start to evaporate if clinicians must verify every note with the same care they would give to writing it themselves.
Clinicians are being put in the position of obtaining informed consent for something they cannot themselves explain. That is not a fair ask of anyone.
The consent problem sits with the clinician, not the vendor
The deeper problem is not the error rate. It is that nobody, including the clinician asking for your consent at the start of the appointment, can tell you what the AI does with the recording. Most AI scribe vendors do not clearly disclose which model processes the data, who can access it, or where it is stored beyond vague assurances that it stays in Australia. Clinicians are being put in the position of obtaining informed consent for something they cannot themselves explain. That is not a fair ask of anyone.
Australia's TGA classification excludes the tools that are already shaping clinical records
Australia's regulatory framework has not caught up. The Therapeutic Goods Administration currently only classifies AI as a medical device if it provides diagnostic advice, which means AI scribes, classified as documentation tools, avoid the scrutiny applied to other clinical equipment. The United Kingdom has already moved beyond that position. The gap matters practically, not just in principle: software that shapes clinical records, and therefore downstream diagnoses and treatment decisions, is influencing patient outcomes whether or not it is formally providing a diagnosis.
This is where the framing question bites. The government could respond to this gap with mandatory standards and disclosure requirements, treating AI scribes like the clinical infrastructure they functionally are. Or it could overcorrect, burdening adoption with compliance frameworks so complex that smaller practices simply drop the tools and return to doing paperwork, which is a loss borne disproportionately by the overworked GP in a regional clinic rather than the specialist in a well-resourced urban practice.
The Partnered Health breach already showed what unregulated data handling costs
The analogy that fits here is not to AI broadly, but to any other piece of medical equipment introduced into clinical practice. An ultrasound machine is not banned because it could produce a misleading image. It is regulated, tested, and the clinician operating it is trained in its limitations. The Partnered Health breach earlier this year demonstrated clearly enough that patient data held by third-party digital health providers can be exposed in ways neither patient nor clinician anticipated. The lesson there is not to avoid digital health tools. It is that the data handling requirements need to be explicit before deployment, not after a breach.
Australia's broader AI policy challenge, which we have covered in the context of the government's productivity ambitions, is that the stated goals tend to outrun the mechanism. AI scribes are unusual in that the mechanism is visible and the gains are measurable. That makes them worth getting right rather than reflexively restricting, or reflexively leaving alone.
The design question is specific: require vendors to disclose data handling in plain terms, extend TGA classification to cover tools that materially shape clinical records, and set independent testing standards for accuracy and bias. None of that is novel regulation. It is the application of existing principles from medical device governance to software that is already inside the clinic. A patient asked to consent to a recording deserves to know what they are consenting to. That is not a high bar. It is the minimum.
Sources
The Conversation — Clinicians are using AI scribes, but what happens to your medical data?
Digital Rights Watch — Off the Record
The Bearing — Australia's productivity will be boosted by AI, somehow…
The Bearing — Australian businesses race to adopt AI so they can…
Frequently Asked Questions
What do AI scribes actually do in a GP consultation?
AI scribes record the spoken consultation and feed the audio into a large language model that automatically generates clinical notes. The aim is to remove the documentation burden from the clinician so they can focus on the patient rather than a keyboard.
How accurate are AI-generated medical notes?
Early studies found up to 90% of AI-generated clinical notes required some correction. More recent research puts errors significant enough to affect diagnosis at around 20% — meaning roughly one in five notes may contain a clinically meaningful mistake.
Why can't my doctor tell me where my consultation recording goes?
Most AI scribe vendors do not clearly disclose which model processes the data, who can access it, or where it is stored. Clinicians are obtaining patient consent for a data process they cannot themselves explain, because vendors are not required to disclose this information in plain terms.
Are AI scribes regulated as medical devices in Australia?
Not currently. The Therapeutic Goods Administration only classifies AI as a medical device if it provides diagnostic advice. Because AI scribes are categorised as documentation tools, they fall outside the regulatory scrutiny applied to other clinical equipment, even though they materially shape clinical records.
What would proportionate regulation of AI scribes actually look like?
The core requirements are three: mandatory plain-language disclosure from vendors on data handling, an extension of TGA classification to cover tools that shape clinical records, and independent testing standards for accuracy and bias. These are existing principles from medical device governance applied to software that is already in use.