AI-powered scribes are taking the consult room by storm, but a senior medico-legal expert warns there are things to keep an eye on.
Privacy, consent, accuracy, professional obligations, product terms and conditions, quality, safety, due diligence and governance are just some of the issues to consider before using an AI-powered medical scribe, according to senior Avant solicitor Tracy Pickett.
Speaking at the annual RACGP conference in Perth on Thursday, Ms Pickett encouraged delegates to approach the new technology with caution.
“Unlike every other tool that you use in your practice, where you can say the TGA has tested it, or that it has some other certification that tells me it is fit for purpose … there is no such criteria for AI scribes,” she said.
Data collected by Avant indicated that one in two GPs were either already using a scribe or were open to using one in the future and Ms Pickett said the medical defence organisation understood the draw of the technology for its members.
“These AI scribing tools can pre-generate referral letters and [help with] all of the other administrative tasks that do take you a long time to undertake, and that can be an enormous benefit,” she told delegates.
“But there’s a catch in that you must check the output, and there have been some disturbing anecdotes that we’ve received about practitioners starting to use the tool and getting so excited about what is being produced … then becoming a bit complacent and not checking the output of the tool.
“That is a very, very concerning issue, because [those notes are] your record and you will be anchored to that record.
“It will not be a defence to say that it was produced by an AI scribe.”
She cited one case where a physician was conducting a consult ahead of a plastic surgery and had run through the complete list of risks, including fat necrosis, nipple necrosis and various other conditions.
The AI scribe, for reasons unknown, wrote that only risk of vascular injury had been discussed.
“We’re not naysayers, and [AI] is going to change the practice of medicine for the positive, but forewarned is forearmed, and that’s the purpose of talking about all of these issues,” Ms Pickett said.
“We don’t want you to not think about them and then trip over something.”
Another key pitfall with AI scribes, the MDO lawyer said, was that they cannot record the things that go unsaid during a consult, like a patient acting different or looking tired or being reluctant to talk about their home situation.
“Whatever it is that has occurred to you as being slightly off, or a concern that you have about the patient or their treatment – that needs to be manually added to the record,” Ms Pickett said.
One of the key differences between using an AI note-taking scribe and simple dictation, she said, was that the AI scribe is a third entity in the doctor-patient relationship because it is applying an algorithm to the conversation and putting out an output which is different from the actual words that have been used.
Because of this, Ms Pickett said, patient consent needs to be obtained three times: once to cover the privacy aspect of collecting patient data, once for the act of having a device “listen” to the consult and once for the action of using the algorithm to turn a conversation into notes.
GP24 was held at the Perth Convention Centre between 21 and 23 November.