Transcript

PLEASE NOTE: While an effort has been made to correct errors in this AI-generated transcript, some mistakes may have been missed. Users should take responsibility for checking anything drawn from here. Further, the guests are speaking in their own capacity and presenting opinions in an open discussion. This podcast does not represent the RACP’s position and is not the authoritative last word on the subject matter.

MIC CAVAZZINI: Welcome to Pomegranate Health, a podcast about the culture of medicine. I’m Mic Cavazzini for the Royal Australasian College of Physicians. You don’t need me to tell you about the burden of clinical documentation. In the literature I’ve read it seems that every hour of clinical face-time corresponds to an additional two hours of clerical work, and that it’s typical for a physician to give up at least an hour of personal time per shift to complete paperwork.

The frustration is experienced by patients too, who sometimes feel that their doctors are more focused on the computer than they are on them. A 2016 paper titled “How Do Residents Spend Their Shift Time?” gave this answer to its own question. More than half the shift was spent interacting with computers and less than 10% in direct contact with patients. This digital ball and chain is one of the top reasons given for job dissatisfaction in medicine.

No doubt you’ve heard the promise of ambient AI scribes that could slash the time you spend hovering over the keyboard. Uptake has been mostly in general practice where some 40 percent of Australian GPs are using scribes already, and this figure has doubled in the last year. Among private specialists adoption been a fair bit slower and if you’re a publicly employed clinician, chances are that ambient scribes have been prohibited in your hospital until now. So, what is all the fuss? You’ve probably already got access to transcription software like Dragon Medical One. But, of course, it transcribes every single word you say into the transcript. So, you’ve got to be very precise in what you dictate to it, and correcting errors is tedious.

But the Large Language Models we’re talking about today are something else again. They will take a raw transcript of the entire consult and summarise key information for the patient record or draft a letter back to a referring doctor. They’re described as Natural Language Processors because they can weigh up the meaning of different words and categorise concepts such as presentation, history and results. When let off the leash, these AI models could also scour pre-existing medical records to generate discharge letters or other documentation.

A few Australian health services have dipped their toe in and trialled the use of ambient scribes, first in clinics and then ward rounds and even the emergency department. As we’ll hear from today’s guests, there are subtle differences in the ergonomic fit of scribes to those different settings. But like everything AI, the genie is out of the bottle. In the recent NSW Budget, almost $39 million dollars was earmarked for scribe licences for 6000 clinicians. An individual licence costs practitioners $1300 to $1900 dollars a year.

Globally, the AI scribe industry is already worth up to $2 billion US dollars a year, and that’s set to double by the end of the decade. The field is dominated by two vendors known as Heidi AI and Lyrebird Health but there are another dozen out there. Today’s guests will talk about testing these market leaders in their health services, but it’s not the intention of this podcast to review which one is better. The conversation is more generally around how the benefits we’ve heard about translate to the hospital setting and what are some risks to look out for. Let me introduce Professor Ian Scott, Dr Andrew Vanlint, and Dr Adam Brand.

IAN SCOTT: Okay, I'm Ian Scott, I'm a general physician by training. I'm currently a clinical consultant in AI at Metro South Hospital and Health Service and a professorial research fellow at the Queensland Digital Health Centre.

MIC CAVAZZINI:                And Ian's already made several contributions to the RACP education and the podcast. So, every everyone should go back to his explainer on machine learning in episode 95. I first came across Dr Andrew Van Lint from his viral YouTube channel, Coding Matters. Tell us about your day jobs, Andrew.

ANDREW VANLINT:         Yeah, I I've trained as a general physician and haematologist. So, I work eighty percent in public with general medicine and education, and some private work with hematology and a little bit of pall care. And I've unintentionally fallen into being somewhat of an expert doctor in documentation and clinical coding, possibly the most unsexy thing to be a specialist of, but probably getting more spotlight in the last five to ten years than ever before.

MIC CAVAZZINI:                And Dr Adam Brand was behind one of the scribing trials we're going to talk about today at Gold Coast Hospital.

ADAM BRAND:  Hi there. My name's Adam Brand. I'm an emergency physician at Gold Coast Hospital and I'm also the medical director for digital and information. Yeah, a very keen user of new and current technology.

MIC CAVAZZINI:                Thank you very much. Now, before we talk about the pilot studies you guys have led, I want to get some sort of first-hand impressions of using ambient scribes in your consults. Natural language processing requires the intelligence—yes, I'll use that word—to weigh up the significance of each word in this jumbled stream and categorize them. Andrew, is it accurate to say that the first order output from one of these sessions is a patient record, a record of the patient encounter and fill the boxes you see in your anyone can see in their patient management system.

ANDREW VANLINT:         Yeah, yeah. In simple terms, there's an audio recording and then there's one piece of technology to convert that recording into a transcription. In most of the products they then delete the recording so it isn't retained. And then another large language model converts the transcript into whatever kind of format you want or comes preloaded. So, what I use is usually the SOAP note for my clinics or if I use it in private inpatients where it's allowed, it's a SOAP note.

You can then, after you've made any corrections that you may or may not need, depending on what you're using and how complex the discussion was, ask it to produce other outputs like a letter back to the GP would be my most common one, as I work in in haematology and clinic, but also patient summaries or referrals onto other specialists.

MIC CAVAZZINI:                And have you have you had to change the way you conduct yourself in a consultation? Having to articulate out loud some of that thinking that that would normally go in your head and then you type it up?

ANDREW VANLINT:         Yeah. So, I'd say in my particular work, physical examination doesn't feature a lot in haematology. It does, but I go through a lot of patients where it's really about symptoms, blood tests and CT scans, and as long as I'm verbalizing those things— or where someone has maybe explained something using more colloquial terms or indirect language, I may reiterate back to them, “What you're saying is this to make it clearer”. I don't feel I need to do that very much, but I would say that when I'm physically examining someone I deliberately verbalize out loud to the patient and the computer what I'm doing, or I may go back to the computer afterwards and mention something.

You can also, before the patient comes in the room, if you've already consented them previously, or after the patient leaves, tell the scribe a few things that it will then incorporate. I don't do that a lot, but I do it sometimes as well, and that's pretty handy. Just like having a personal assistant writing or a student writing, you would check it and might make a few comments for them to include as well. So, it's very similar to having an actual paid scribe physically in the room with you.

Quality of outputs from ambient scribes

MIC CAVAZZINI: Alright, let’s move onto the scale up in hospital. In mid 2024, Adam’s team at Gold Coast Health conducted a 16 week trial of the Lyrebird scribing tool across 21 specialty outpatient clinics. Ian, you similarly led a trial at a couple of other hospitals in Brisbane, this time using Heidi AI. The results of the Gold Coast trial can be found in an article for BMC medicine published last January. This describes how in the pre-implementation phase, the ambient scribe was set up to run silently in five outpatient clinics of 100 clinicians whose patients had consented. The unedited AI-generated notes were later compared to clinician-generated medical records for each consultation and note quality was evaluated using the Physician Documentation Quality Instrument or PDQI-9 tool. Adam, can you explain what this tool is assessing and what this comparison found about the scribe working without any oversight.

ADAM BRAND:  So, the PDQI assessment tool, it it's one of the validated tools out there. There's perhaps no perfect tool, they all have their strengths and weaknesses, but it was trying to have some objective measure of quality which might be measured in structure, how inclusive it was of important points, how readable it was, you know, grammar, spelling. And what they found was that particularly in cases like this, it definitely is not inferior. And so, it gave us the confidence to understand that much like you would expect with an in-person scribe, which can often be people like medical students, that it produced a good quality of note that was acceptable in clinical practice and particularly when compared to what people were using in their actual practice.

There were some interesting things that kind of came out from that where 58% of the scribed notes didn't require any editing that might seem like a low bar, but when you compare and to talk to Andrew's point earlier, we often don't verbalize everything that we talk about in the in the consultation ourselves. And so, I think as you become more familiar with the environment, as you improve your dictation or your vocalization of your clinical findings, maybe the tests that are there before, you enhance that interaction with the patient in the room, but you also give the opportunity to capture that in the transcript, which typically then results in a higher quality output that may not need any editing.

So yeah, we were very pleased with it and sort of it translated then into the practice where we weren't doing the silent trials, where we were doing it in person. And I think the outcomes and the conclusions probably very similar to other trials out there that it generally was accurate, the quality of the notes was good, and that the experience of the clinician and the patient was improved.

MIC CAVAZZINI:                And you noted you used another assessment tool called the ROUGE Score that tells you how closely the two documents match. And your team observed that: “Across clinical specialties, those such as endocrinology that involve chronic disease management had lower ROUGE score performance… To explain this, these clinicians reported that they routinely refer to historical inputs from a patient’s medical record history to inform their consultation and complete their notes.” Do you want to expand on that?

ADAM BRAND:  And I think that's alluding to what we talked about in in the fact that often when they're writing a note, you might not verbalize their entire medical history to the patient in front of them. That said it can really be—when I've used it in the emergency department, actually doing a summary back to the patient of what you know of them can often help build that bond and trust with them and it and it seeds the consultation with that background whilst trying to also remove the dreaded copy and paste which kind of exists in the in the text world, that there's the opportunity now to actually cut through that and because your verbalizing it and you're synthesizing that in your brain, you're giving the scribe real gold quality.

MIC CAVAZZINI: Ian, in your pilot you had 58 clinicians participating from three different sites. Notes from five patient encounters conducted before and after the introduction of the Heidi scribe were compared. More than half of participants reported that the ambient scribe helped improve the quality of their notes to an average of greater than 3.8 on five-point Likert scale across nine quality axes. Can you tell us a bit more about these quality domains and where the biggest improvements were made.

IAN SCOTT:         Well, it was the same instrument that Adam was referring to before, the PDQI-9. I mean the quality was good. We didn’t rate the previous note, we only just looked at the AI scribe note, so we weren't necessarily doing a direct comparison. But the thing is that we were asking the clinicians themselves to rate each time they generated a note to fill out that questionnaire. So, what we did see is that we had to edit in the vast majority of cases. What we found was the areas of a note where most editing was done was the history of the presenting complaint, the physical exam and the assessment plan. In terms of social history, past medical history, other things, investigation results, they were fairly clear-cut and not much to edit, but those other three areas was where most of the work was done.

We found that, interesting enough, the actual word count of the generated note and the finalized note was really much the same. So, people either were either taking stuff out or rearranging and then perhaps putting stuff in. So, as Andrew mentioned, I think people often then add additional information after the actual encounter perhaps to get their thoughts in order and just to make sure that then what they're writing is as accurate and comprehensive and complete as they can. And we all know that sometimes there are things that you don't necessarily want to say perhaps in front of the patient for good reason, not to not because you're trying to hide things, but you just want to be sensitive to patients' feelings and sometimes the technical jargon too needs to be you know needs to be inserted as well. So, I think that, overall, we were pretty happy with the quality of the notes that were being generated, and that people were looking at the note and editing appropriately.

MIC CAVAZZINI: I’ll go back to Andrew now, but it's a long lead into this question, so bear with me. So, from the Gold Coast pilot study, 80% of participants felt that the AI scribe produced a good quality note, but 47% of respondents reported having observed hallucinations in scribe outputs and a fifth reported frequent occurrences of mistakes. The paper describes these in broadly how this included mistakes about similar sounding medications, or about the presence of drug allergies or smoking status.

But there was an article for Australian Doctor from Jan last year with some more high-stakes examples given by the experience of general practitioners. One in which a scribe that had written that the patient “does” have epilepsy, leaving out the rather important qualifier “not”. Another instance where the occurrence of chest pain was recorded incorrectly, and one in which a cancer was attributed to the wrong breast. These are kind of yes/no, you can understand, slip-ups, but there were others that were clearly introduced from thin air. The wildest example from the AusDoc article was one in which the occurrence of a neurological exam was recorded which had never happened at all in reality.  Some people in Adam’s trial felt that they could cope with these errors and clean them up, but others were put off by them, others were put off by it, saying it took just as long to clean up the mistakes. One user said that he had no way of fact checking the “spiel” that the scribe came up with, as he couldn't remember the whole conversation himself. So, Andrew, from your experience what should we trust more, our own memory or the AI that's always listening but sometimes trips up?

ANDREW VANLINT:         I think it's a bit of both. I think the examples that you've raised there are good ones. And I think it's important when we use this kind of tool to distinguish between what is a hallucination, which I would say that where information has been grossly misunderstood or information has been creatively produced that wasn't there before, and the latter being a really classic hallucination that we see when using large language models. I think things where it's a misinterpretation, so “does have, does not have”, maybe the “not” was not pronounced clearly.

And I think a lot of the time we as a as a society, we seem to have little grace for the AI, but we would have grace for a human in the same position. So, if I had a professional medical scribe or a senior medical student doing my writing—and when I work in public doing rounds as I was this morning in general medicine, I often have a senior student, internal RMO, writing the notes for me—it's still my job to go back and make sure that they've documented the things that I think are important and clarify the plan. But we seem to want the AI to—we're holding it to a higher standard, a near perfect sstandard, and also the question of who's accountable if mistakes are made. In the case of AI scribes, we're accountable. I'm using the tool, I'm choosing to use the tool, I need to check it, just as a consultant overseeing a ward round with juniors, I need to check what's being managed for my patients because I'm ultimately responsible there. So, I think it's important that we retain this idea of agency and accountability, checking over things whilst using these tools.

The most common mistakes are when it interprets personal anecdotes from me, building rapport with the patient, as being part of the patient's personal history. I look out for them quite deliberately and make sure that they don't feature there. And the second one would be names and places, which in the Australian context is quite a broad range of many different cultures that are here, that the pronunciation may not be super clear. I actually find as someone who prescribes some very expensive and complicated sounding drugs as a haematologist, it seems that's a very good database for that. And I've very rarely had inaccuracies or mistranscriptions around drug names, even though they are difficult to pronounce and spell. So, that's been my journey with it. But I think again it comes back to the accountability is mine. This is my note, and so if I'm outsourcing the writing of that note, I need to ensure that I'm quality-checking. And I think for me that quality checking is still a massive time saver and general improvement on quality, particularly when I'm feeling under pressure for time.

Real world time savings

MIC CAVAZZINI:                Let’s speak more about the times savings. Ian, your study was particularly interested in the times saving associated with scribe-supported consults. And you found that from an average consult duration of 37 minutes across all encounters, the median time saved was two minutes. But there were larger time savings for mental health and allied health consults, on the scale of 9 to 12 minutes. What do you think explains this difference?

IAN SCOTT:         Well, I think because in terms of mental health, they have very prolonged consultations and they're complicated consultations and they also have to fill out a template in terms of mood questionnaires, their axes of diagnosis, et cetera. When you look at psychiatry notes even as a consult, you find that they're sometimes some of the biggest and longest things to read, so I think that's just the nature of the game.

So, for them, I think that they found using a scribe was very helpful because it could structure all that information very quickly for them and it also allowed them to then also annotate that more quickly as well. For nursing similarly, the nurses often do fairly complicated nurse assessments. Allied health professionals but not also nurses, who were doing complex patient assessments, things like NDIS assessments, for example, psychopathology re psycho psychopathology reports, again, these can be quite long and very detailed and I think that's why they found then the time savings were even more noticeable for them because it was able to structure the note quickly and allow them to edit more quickly as well.

MIC CAVAZZINI:                And Adam, your study found in a similar vein that specialists who tend to have short consults anyway, like orthopods, were less inclined to find great benefit from the AI. And you know, while two minutes per consult doesn't sound like very much, over a whole clinic that can add up to twenty, thirty minutes, which is significant. And that's consistent with the data from half a dozen serious trials of ambient scribes that Ian presented in a webinar for the college's EVOLVE series. So, Adam, do you have a practical example from about how impactful these savings have been?

ADAM BRAND:  Yeah, look, I would say our one of our first time savers of this was our rheumatology clinics, where the pyjama time, you know, the time that was off the clock writing letters, getting things out the door, went from being unpaid time that just needed to be done to get the job done, to actually finishing clinic on time or earlier. So, really transformational for those that fully embrace it within their clinic. There were scenarios where it was incredibly impactful with long-form consultations, so social workers, psychologists, mental health. It scales incredibly well in that form.

But I would say that those clinicians that haven't found it useful or time-saving, it's quite similar to other trials where they've actually had physical scribes or professional scribes in emergency departments and how there are certain clinicians that thrive from having a scribe, a physical scribe, and there are others where it actually slows them down. And that's personal preference, it's workflows. So for us in orthopaedics, we initially gave the licenses to the consultants, thinking that that would be where the actual benefit would be. But we realized the consultants actually walked from room to room and actually it was the registrars that needed it and that the consultants needed to add the detail in when they came in finally to help them write the note. And that came from the letters normally being dictated by the consultant orthopaedic surgeon. But if you slightly change the workflows that you can have the benefit in multiple places.

Ward Rounds and the ED

MIC CAVAZZINI:                Clinics would have very similar ergonomics to the private practice that we've heard about from Andrew. I imagine that the dynamics of a ward round are very different, the conversation is less structured, there are more voices chiming in, you know, not just the doctor and the patient, but also, junior doctors and nurses who might be taking instruction. Is there any evidence about how well ambient scribes perform in this setting versus the more controlled consultation room?

IAN SCOTT:         Yeah, it is more challenging, there's no doubt and what we found in our service is that some units will use it in inpatient ward rounds, others probably prefer not to. So, I think most people don't mind it in clinics, but in inpatient settings it's perhaps a little more difficult. Having said that though, we have a number of units that do use it on ward rounds. What they try to do is that obviously there's one or two people who are speaking and that's what the scribe picks up, but the other people are asked to sort of just I guess step back so they're not right in front of the microphone and they're not necessarily conversing directly in the conversation at that time. It's between patient the consultant, you know, in those sort of situations. These are normally ward rounds where consultants are seeing new patients for the first time so they're having a discussion with that patient, that's what's actually being transcribed.

And then at the end, you don't have to actually complete the session there and then you can pause it and then come back and then in a room later or perhaps even sometimes the corridor, you then add further information in, right? And then finalise the note there. There is some concern around privacy because, you know, you may pick up some ambient conversation that's happening next door. But I think it's more design to, you know, try to put some shelters between those different beds so you know and you choose your timing in the sense that if you’ve got people running around cleaners and others doing a lot of noisy things, well then you don't go to that patient, you go somewhere else where perhaps it's more quiet. So, I think they people have had to sort of individualize a bit and improvise a little bit to work out what's the best way to deal with the technology.

MIC CAVAZZINI:                Go ahead, Andrew.

ANDREW VANLINT:         I think in the inpatient setting, context is really key. So, if you're on a weekday ward round in most inpatient teams in most public hospitals, you're going to have multiple people on that round. Even if you divided and conquered, you might be rounding in pairs with a student. And so, you have the luxury of having someone on the team document. And it's important to realise in the public setting, that is a teaching opportunity. That's a learning opportunity to refine, receive feedback for students to learn, but students to meaningfully contribute to patient care through documenting and learning and having that feedback and paying more attention because they're invested into writing a good quality note. So, there's a dynamic there that we need to understand will influence the adoption of AI scribes and that we need to do that as well when we're using them, teaching our students how to use them well by example and in and word as well.

If you're in a public setting on a weekend, you might be rounding by yourself if you're in a smaller team. Suddenly, having an AI scribe might make a bigger difference to you because you don't have the luxury of having a second person to go around with you. Or if you were rounding in a pair and it's only the two of you for 20 to 25 patients on a weekend and your colleague gets caught off to a code blue or an urgent family meeting or needs to an urgent discharge script, suddenly you're rounding alone. And in private, we all round alone.

So, I was on this weekend just gone and I had my new personal best of 29 inpatients to see on Saturday. And I got out of there by 4:30 thanks to using an AI scribe on my rounds. And I’ve got to say in private the standard documentation is not usually that high. But I was proud to leave that with very good documented notes, thanks to the AI scribe, and a good amount of time and a good amount of attention paid to the patient. Each one I think felt like I was really present with them not rapidly typing on a computer whilst we chatted. So, that context I think is really important and there'll be higher value propositions in those more solo or limited staffing contexts.

MIC CAVAZZINI:                And Andrew, you're a big educator. There’s been one prediction that AI scribes will lead to what's been described as never-skilling amongst the junior doctors that traditionally would, write out their cognitive thinking as they're taking notes. Whereas on the other side of that, you put me onto the Clinical Documentation Improvement Australia website. And there was a great video from Dr. Felicity Sinclair-Ford titled Junior doctors: paperwork lackeys or key allies for CDI? And she describes her experience as a junior chasing the registrars around trying to document important changes to patient management that sometimes instead of being bedded in pearls of wisdom and teachable moments, were cast around like meagre breadcrumbs. As she puts it, “Junior doctors often don’t have the clinical experience to understand what their seniors imply but do not specify.” So, there’s obviously scope for better teaching, but could a scribe help juniors participate more directly in the round rather than just being paperwork lackeys or is it going to lead to this never-skilling?

ANDREW VANLINT:         Yeah, I think as with any technology, there are gains and losses and we need to be cognizant of both ways and weighing up is that an acceptable price to pay. When you're more junior, and I certainly remember this as a as a junior registrar and basic physician trainee doing a lot of admissions overnight, the process of writing out an admission note was a great way of structuring my thinking and reasoning. I'd often start by going, “I don't know, this person's got a few different things and I'm not sure they're all connected. I'm not sure what my management plan's going to be. I've got a few ideas.” But as I wrote it and as I thought about how I'm going to word my assessment, the plan more came together. And so, there's a there's a question to say, will we lose some or all of that through going to AI? And the answer is probably yes. Should we be replacing some of those learning activities and formulation activities with something else? And the answer is probably, yes.

Is that an acceptable price to pay for improvement of quality and efficiency? Maybe. I don't have a definitive answer for that. But we have often moved through ways of doing things. So, if you went to a GP today and they say, “No, I don't like using electric systems. I like to handwrite all my notes and handwrite all my prescriptions, handwrite all my orders.” You would be like, “I don't know. I don't know if I'm getting good quality out of this GP”. They feel like they haven't caught up that to the standard of practice. I think in five to ten years, the AI scribes are going to be that standard of practice because they're increasingly getting better and you'll start going, “Why aren't you using AI Scribe? I'm concerned that the quality of the documentation and your efficiency and the care I'm receiving may not be as good because you're not using a scribe”. But we're still in the early adoption phase, but I think that's the way we're heading. And we just need to continue progressing and continue analysing what we're losing and gaining and maybe what other educational activities need to be added along that journey to substitute for what was before.

MIC CAVAZZINI: Adam wanted to add something.

ADAM BRAND:  Yeah, look, I think just to add to what Andrew had said; I chair a communicating for safety committee in the hospital and I'm lucky enough to hear about how inpatient teams try and improve their ward-rounding, discharge summaries. And not related to AI scribes initially, where teams reimagined their rounding to try and improve problem-based management plans. And, you know, consultant-led essentially, you know, voicing their internal thinking process out loud in a structured way is incredibly useful for our junior medical staff to learn what good looks like. And I think that kind of verbalization of thought process is where AI scribes really thrive.

And so, for some of those teams now where the most senior person on the ward round is essentially verbalizing their thought process, you suddenly end up with an incredibly high-quality ward round note, the junior doctor realizes the thought process that the consultant or the registrar is going through and then can replicate that in their practice. So, yeah, there's pros and cons. I think in the right set of hands and in the right way, it can be thought of as excellent.

And just as on another use case that I think we've we're just scratching the surface on at the moment is around multidisciplinary team meetings where if you have a high-enough quality microphone that can pick up multiple people, it does an excellent job of taking what can sometimes feel like a very unstructured set of expertise in a room and coalescing it into a highly structured document that can really drive that patient's care forward.

IAN SCOTT:         My feeling is, a bit like Adam, that I think I think on ward rounds, poor old interns are treated as chaff a bit. They're just the clerks to sort of document what's been written and they really don't have a chance. We don't do any sort of bedside teaching anywhere near as much as we should or have done in the past. So, they're trying to sort of interpret sometimes some high-level stuff and put it into a record and I know that when they come to do their discharge summaries, I find that some of our residents really have not understood this patient at all throughout the care episode. You know, they haven't really understood the diagnosis or the rationale for the management or, you know, why we're doing certain things.

And I think that's because they're just so focused on, “I’ve just got to document, I’ve just got to put something into this record”. So, I think if we can use the scribe to relieve them of that scribe as scribing burden and just be able to listen in to what's been said and to ask questions. I mean, the time saving in relation to describing could be made up then in more quality teaching. Not necessarily to push more patience or to finish your word rounds and you can do other things more quickly, but to actually give us back some time and some cognitive bandwidth to actually engage in talking to each other and reasoning and thinking about and reflecting on what we're doing. So, I think this the idea of never-skilling, I'm certainly sensitive to that, and yes we don't want to lose skills, but perhaps there are some skills that are not being developed, and never-skilled, because we don't give people the time and the and the ability to actually engage in that sort of activity.

MIC CAVAZZINI:                Well put. Those are great insights from the educators lens. now let's put aside ward rounds. What about the chaos of the emergency department, Adam? I don't know if you came across a study from the ED at Cabrini Health in Melbourne, where a 110 scribed and non-scribed notes were retrospectively compared by two staff members and rated for quality. If you take the outcomes at face value, there was no difference in quality between human generated notes and AI generated and AI generated notes. But they did identify a pretty poor agreement between raters regarding that PDQI assessment tool. So, Adam, is there a weaker use case for scribes in the ED setting or do we need a better way to assess this? 

ADAM BRAND:  My personal thought on this is it depends. And I think it depends on the patient context. So, I don't use my AI scribe for certain shifts and I use it a lot for others. So, if I'm working in a resus shift where I'm constantly changing from a resus, or a trauma or looking at an ECG, giving advice, that is not a useful tool for me. It might be a useful tool if there's a complex patient where I'm the sole provider for. And so, I think in the same way that we've talked about in terms of ward rounds or in certain areas it might not be as useful as we would think.

In other areas it can be incredibly useful. So, if I'm allocated to a minor injury shift where I'm seeing you know, broken bones, cuts, minor injuries, where my document documentation burden is quite high in comparison to the time I spend with the patient. I use it a lot and it really really helps with the quality of the output. Some of the forms that it can fill in are incredible. That saves many minutes per patient. So, I can start a consultation and generate a note, a letter, a workers' compensation form, a sick certificate all within the space of a few minutes for a patient and they walk out the door very informed and very happy.