Transcript

MIC CAVAZZINI: Welcome to Pomegranate Health, podcast of the Royal Australasian College of Physicians. I’m Mic Cavazzini, and over the next two episodes I’ll be putting CPD under the microscope. Continuing professional development is intended to keep you sharp throughout your career, and it’s undergoing some changes to try and make it as valuable as possible. Stay tuned even if you’re currently an advanced trainee, or you have an interest in medical education.

From 2019, Australian Fellows will be asked to take part in activities like practice review and clinical audit, just as colleagues in Aotearoa have been doing for a few years already. In the next episode, we’ll hear from two New Zealand Fellows about how to organise these activities and what unique insights you can get from them. The requirements of CPD come from the Medical Council of New Zealand and the Medical Board of Australia, with its new Professional Performance Framework.

And this follows a tide of change by health regulators in Canada, the U.K. and the U.S. While some models of relicensing have been opposed by doctors, there’s a good case for a more systematic CPD approach. We’ll hear about this later in the episode from an expert Canadian medical educator. To explain how the RACP will be trying to help Fellows meet these new requirements, with me in the studio is the Dean of the College. Welcome Richard Doherty.

RICHARD DOHERTY: Thanks, Mic.

MIC CAVAZZINI: Richard, can you describe our beautiful studio for the listeners?

RICHARD DOHERTY: Our beautiful studio is ‘Meeting Room 2’ of the new College education facility in the Governor Macquarie Tower.

MIC CAVAZZINI: And can you introduce yourself, your special interest and your role in advising the MBA?

RICHARD DOHERTY: So, I’m the Dean of the College, most of the time. Some of the time I’m a paediatric infectious disease physician based in Melbourne, and I’ve been fortunate to have been asked to serve on the Medical Board of Australia's Expert Advisory Committee on Revalidation, which provided a major report to the Medical Board in 2017. That report has been used as the basis for the professional performance framework.

MIC CAVAZZINI: So the story so far: the whole recertification movement was kicked off in the USA in the 90s, and to remain certified, U.S. doctors now have to undergo a pass/fail assessment every 5 to 10 years. In 2007, the Canadian college implemented a more formative process they called ‘revalidation.’ The U.K. also took up the term ‘revalidation,’ but in the U.K. it does include a high stakes review of fitness to practice. And in 2014 New Zealand implemented its recertification model, which is being emulated in Australia.

So just to be clear, health practitioners in Australia and New Zealand can’t lose their licence to practice if they fail some CPD exam?

RICHARD DOHERTY: No, we don’t have a CPD exam, and this goes to the heart of the question of: ‘What’s the difference between recertification and revalidation?’

I guess in general use, recertification is about showing that you still have the qualities of knowledge expected of a practitioner in your discipline, as if you were just completing training. So in the United States practitioners are expected to re-sit the current board exams, which means you have to update your knowledge and show that you’re still current. Whereas revalidation is broadly considered to be showing that you’re still fit to continue to practice in your chosen scope of practice, that you’ve kept a record of your own continuing education and that your performance is appropriate to allow you to continue to practice.

New Zealand took the very interesting perspective of legislating the term ‘recertification’ in its act that governs this whole process, and choose to implement a model which is much more of a revalidation-type model along the type line of ‘strengthened CPD.’ And it’s a bit like Alice in Wonderland, the term happens to mean precisely what I define it as at any particular time.

MIC CAVAZZINI: That’s helpful. There was a line from a paper by Ian Scott which made the point that if members embrace a sort of transparent and rigorous framework for themselves, that will obviate the need for external regulators and, you know , prying eyes.

RICHARD DOHERTY: I don’t think it'll ever obviate the need for them but, of course, if you look at the literature about what is a profession, one of the key features of a profession is its willingness to self-regulate, and that doesn’t have to be done in a punitive or authoritarian way. One of the things that we’ve been particularly concerned about in the couple of years that this has been evolving has been the idea that colleges might somehow be assigned a regulatory role. That doesn’t appear to be the case, and we don’t wish to take that role on, and it’s obviously important that Fellows don’t get confused about this, so we’ll be trying to make that very clear.

MIC CAVAZZINI: So the College doesn’t want to have its past role conflicted by a regulatory role?

RICHARD DOHERTY: No. So the College does have a pastoral role, but the College is also a standards organisation, and one of our key roles is to articulate and endorse standards—what is appropriate behaviour, what is appropriate learning, what are appropriate skills? But one of the key things about the Professional Performance Framework, of course, is that it is very clear that this is largely seen as a formative exercise, so that remediation and assistance for people whose performance is found to be less than ideal, is built in. And so whilst recognising what a difficult challenge that will be, the College feels that we wouldn’t want anyone else to be doing it and that would hope that the Fellows would see the College as the natural home for that sort of activity.

***

MIC CAVAZZINI: Before we drill down into the requirements of ‘strengthened CPD’ let’s hear the origin story. Why is CPD needed at all, after you’ve already done 10 to 15 years of training?
The idea of continuing professional development is wrapped up with the revolution in medical training during the 1980s. In the classic apprenticeship model, trainees were thrown in the deep end with little structured learning. Over gruelling days they would absorb as much as they could, often practising skills for the first time on real patients.

Critics of this education model observed that simply clocking up hours did not guarantee expertise or safety. They suggested that trainees should instead have to demonstrate each clinical skill before moving on. ‘Competency-based medical education’ was championed in Canada after a 1986 doctor’s strike triggered a conversation about what the public should expect from the profession.

One expectation that emerged was that doctors should have to maintain their knowledge and skills throughout their career. Not just clinical, but professional competencies were mapped out as well, and in 1996, Canada’s Royal College of Physicians and Surgeons published the first CanMeds framework. This described the exemplary doctor not just as medical expert, but also a collaborator, a communicator, and advocate and a scholar—a Renaissance physician you could say. In an article titled ‘A Call to Action,’ authors from the International Competency-Based Medical Education (ICBME) collaboration write: ‘the era of solo practice was waning; the era of team-based care, rapidly evolving practices, quality reforms and patient-centeredness had arrived.’

One of the most published members of the ICBME is Ottawa internist, Craig Campbell.

CRAIG CAMPBELL: Well, I’m Craig Campbell, I’m an internal medicine specialist, I have been a fulltime employee at the Royal College of Physicians and Surgeons of Canada since 2002, where I think about a competency-based approach for CPD and practice.

MIC CAVAZZINI: OK, so let’s start with: what’s the point of continuing professional development? Is it demonstrated that you do lose your skill over your career without continued training?

CRAIG CAMPBELL: Yeah, so I think that systematic review was 2006 in the Annals, which suggested that as one gets older, as one ages, most things that they measured goes down over age. So just experience alone doesn’t entirely protect one and so that’s, you know, the ageing physician literature would also suggest that that’s true, and I’m getting close to that age, so I’m worried about that kind of literature myself.

So I think there’s an educational mandate or rationale for lifelong learning which is that science and the scientific basis of the profession is evolving at such a rapid pace that one would become out of date in a relatively short period of time, and that may not have been the case in the 60s and 70s. you know, if you talk to our surgical colleagues they would say that the things that they are doing in practice today, none of them learned it in residency, right, they’re all new techniques.

MIC CAVAZZINI: Some of the literature in this area makes a distinction between competence and performance. Can you define these terms for us?

CRAIG CAMPBELL: So if you’re talking to an educational person, you know, if you’re using the medical education literature, competence is something that’s demonstrated in an educational setting, in a sim centre, you know, in a classroom setting, those kinds of things. Performance is what you actually do in practice.

MIC CAVAZZINI: A really simple throw-away example might be compliance with hand hygiene requirements, right? Everyone knows that this is important, but adherence is not to so great. And so these examples aren’t about cutting-edge expertise, but about behaviour and culture and leadership.

CRAIG CAMPBELL: Absolutely. So you can actually say is somebody able to wash their hands properly coming in and out of a room. You could sort of simulate that and people would say, ‘Yeah, they know exactly how to do it.’ But whether they actually do it consistently in a hospital setting or in a clinic or in an office setting would be their performance. And that’s why moving the language from CME, which was just about what I was doing in my medical discipline, to CPD, which then embraces these other competencies that are part of the care delivery process—these competency frameworks tend to sort of embrace social responsivity as a foundational principle for systems of CPD.

MIC CAVAZZINI: So for the BMJ you and Tanya Horsley reviewed recertification systems around the world and noted that all the regulatory documents also make reference to concepts like patient safety, quality of care, public demands for accountability and transparency?

CRAIG CAMPBELL: That’s correct.

MIC CAVAZZINI: Should CPD been seen as part of the contract that physicians make with the public?

CRAIG CAMPBELL: Yeah, I think it’s embedded in that, but it’s not well expressed. But I think for the privilege that we have been extended, our part of the contract is that we will maintain our competencies, our skills and abilities. You know, it’s saying that my learning is not just geared to my own interests alone, but largely it should be constructed around addressing patient-population health needs. And in that sense, it’s part of a social contract of what the public expects of us to maintain those things. Like, every time a patient walks into the clinic like I was in today, they expect you to be competent. Just like when you and I walk into a plane we expect the safety concerns to be covered, you know, that shouldn’t be a question.

MIC CAVAZZINI: Yeah. Now, the American and the British medical boards, they go so far as to talk about the restoration of public trust. And you observe in your BMJ article that this has been influenced by catastrophic incidents like the Royal Bristol Infirmary and the Shipman Enquiries, and that these have, I quote, ‘been collectively viewed as a failure of medical regulation to provide necessary assurances that physicians have the competence and ethical integrity to care for the public.’ Now, these scandals are outliers, of course, but do they reflect a need for greater oversight?

CRAIG CAMPBELL: Yeah, you know, when I asked the GMC why they did revalidation that was the first thing that they said, was Harold Shipman, right, I mean, that was right out of the—in the United States it was really centred a lot in the Institute of Medicine’s reports around that the healthcare system was 11th out of 11 in quality of care measures, and they spend the most in healthcare per GDP and yet they have the worst outcomes. And they responded to that in a very legal way by requiring every physician to participate in revalidation by law.

I’ve talked about this as a balance between autonomy—you know, the right to choose—and accountability. And I think the Americans started much more on the accountability framework and were emphasising the proof that you are still good enough to merit the label of general internist or psychiatrist or family physician, and they were much more about the testing and psychometrics of their assessment system. But there was very little emphasis on learning and improvement in practice, which is what the docs have been reacting to recently.

I mean, I still want physicians to be able to make choices about what they should pay attention to, what they should be learning about, et cetera. I just think we have to be more accountable for the choices we have made and demonstrating through practice how you’ve been maintaining your knowledge and skills and abilities against a certain level of expectation.

MIC CAVAZZINI: We should also note that doctors consistently come out as one of the most trusted professions in the 2017 Roy Morgan Image of Professions Survey—they were third placed behind nurses and pharmacists, whatever you make of that.
CRAIG CAMPBELL: Yeah, interesting.

MIC CAVAZZINI: That’s a good segue onto the domains of professionalism that you’ve touched on. You mentioned the CanMEDS framework, it’s a beautiful rosette that the RACP has itself coloured in now, it’s a rainbow-coloured flower with nine domains encompassing communication, teaching and learning, cultural competence, ethics, decision-making, teamwork, health advocacy.

CRAIG CAMPBELL: All these things, yeah.

MIC CAVAZZINI: Yeah, and in a 2017 paper with your colleague Jocelyn Lockyer you argued that competency-based assessment should be applied not just in medical training, but also to professional development. So do you mean that these professional traits could also be assessed or measured in some way?

CRAIG CAMPBELL: Yeah, so I do think that. I think we need some more robust thinking about how to do that well, but I think the one tool that’s in our quiver today that enables us to do this reasonably effectively is multisource feedback. So, peers, colleagues, patients would provide the same—a response from their perspective on these domains. And we’re trying to use then a coaching feedback model because the data itself doesn’t seem to change behaviour very much.

You know, it’s easy to explain away the data, to say, ‘Well, people don’t get my practice, I’ve got tough patients - people don’t understand me.’ There’s no help to that, right? There’s no guidance, there’s no plan that flows from that that’s saying, ‘Okay, I can use this now as a means of framing a CPD strategy to improve in these areas.’ The analogy that I used was if you’re a teacher you get back your annual teaching feedback, and usually you get these numerical sort of five point or seven point scale. And I said to people, ‘When you get those reports what do you read first?’ Almost invariably everybody reads the narrative comments first.

MIC CAVAZZINI: Now we’ve got this beautiful framework, the CanMEDS or the professional standards, but in Canada and New Zealand and Australia there’s no intention to assess these in a pass-fail way

CRAIG CAMPBELL: Like the Americans are doing.

MIC CAVAZZINI: Yeah. And one of the comments in the consultation with the RACP members was that there’s a risk that they just become tokenistic. Do you agree with this or do you feel like having these emblazoned up there in the CanMEDS, it makes them an aspirational target nonetheless?

CRAIG CAMPBELL: So I think there is a risk even in our current system, they can treat that just like a tick box; ‘OK, I did this assessment, I did that assessment, I got a minimal number of credits, so what?’ So how do we change that? I think one of the ways to change it is to think about assessment as a programmatic idea. So there’s things that you can learn from charts or administrative databases. Like a prescribing programme, you know, I can see me and my prescribing for condition X versus my peers.

So that might be the one data source. The other data source would be peers and colleagues and the third data source may be patients. People can pick and choose but they can’t miss a category. So we create this sort of a programmatic approach to assessment that draws different data sources because they tell us different things about different competencies, and collectively it’s like multiple sources of light—you’re sampling enough to say, ‘Oh, I’ve got data and feedback on multiple CanMEDS roles.’ It may not be every role, every cycle, I get that, but changing it from a tick box to something that’s more as a way of guiding the choices I make.

MIC CAVAZZINI: Now, one of the other gripes that practitioners might have about CPD is whether it even makes a difference, and there are hundreds of studies evaluating different types of educational and practice change. There are 40 systematic reviews of the literature; a good one is the 2007 review by Marinopoulos and colleagues who found that CME, as they call it, improved knowledge or skills in 34 of the 43 studies that looked at that, and could change attitudes and behaviour in 83 of the 131 studies, but generally the effect sizes are quite small. And it’s quite hard to show on association with improved patient outcomes. So where do you put your faith in this literature?

CRAIG CAMPBELL: Yeah, the effect size is certainly smaller than we would hope. In part because we’re almost assuming that everybody walking into, say, the group learning experience, that they’re all at the same place, their knowledge base is all the same, they have the same gap, you know, they’re at the same place of change, or consideration of change, which is really not true, right? So if you see somebody who comes in to practice with a significant gap, and feedback can certainly cause significant percent change, versus somebody who is already doing the vast majority of that already, there isn’t much room to change, they’re already at the ceiling for what the expectations are.

So I’ve tried to say, ‘Look, there are some things that are easier to measure even though they may be less important than the ultimate outcome.’ So in other words we can measure compliance on blood pressure control, you know, the percent of patients in one's practice to which blood pressure control is meeting targets, because that might be a sub-correlate to then reducing the incidence of stroke or heart disease, which a randomised-controlled trial 20 years to prove that. I think it’s important to say ‘Here’s the outcomes of our system of CPD,’ you know, in other words, the collectiveness of it. It’s not any one conference or one assessment tool or technique, but the collective responsibility of multiple organisations to achieve these ends.

***

MIC CAVAZZINI: As we’ve heard, there are reams of literature examining the impact of different types of CPD on performance. One standout example published in BMJ Medical Education involved thousands of U.S. physicians who had taken high quality online medical education courses. They were demographically-matched doctors who hadn’t taken the courses and a subset were selected randomly to respond to case-based questions. Those that had taken the courses were 48 per cent more likely to make evidence-based decisions.

But not all CPD is created equal. A 2009 Cochrane review found that educational meetings typically made a 6 per cent difference to compliance with best practice. Two other systematic reviews concluded that interactive methods and small group activities could be more effective—things like hands on workshops, simulation-based training, even grand rounds. Meanwhile, structured feedback and audit have great potential according to a 2012 Cochrane review, but it really depends on how often feedback is given, and how it’s delivered.
But these higher-value activities aren’t always appreciated by physicians. In an RACP survey from a few years ago, Fellows ranked activities such as team-based training and peer review among the least useful, and simulation-based training and clinical audit weren’t far behind. Fellows expressed a preference for seminars and journals, and these are the most frequently logged in the MyCPD records.

With its call for ‘strengthened CPD,’ the MBA will be requiring some participation in more reflective activities, and the RACP will nudge Fellows with a modified MyCPD framework from next year. Where there have been previously five categories of activities, there will now be only three. Category 1 includes all the educational activities that previously dominated MyCPD. Category 2 is known as ‘Reviewing Performance’ and Category 3 is called ‘Measuring Outcomes.’

Of course, the MBA and the MCNZ also regulate practitioners from other specialties, all the colleges are tightening up their CPD programs in much the same way as the RACP.  We’ll talk about performance review and clinical audit in detail in the next episode, but let’s take a quick look at a few educational activities described under Category 1 of MyCPD. The RACP has many resources that fit the bill at its website, including the e‑learning portal and the College Learning Series lectures. And there are several online providers of CME courses such as EdX, Coursera, Medscape and The Frances Foundation. Here’s Richard Doherty with his impression of these.

RICHARD DOHERTY: Yeah—Medscape is one that I’ve used for many years and find mixed but often extremely valuable, particularly where you can link quickly to the primary literature. The flag I’d like to wave here is the role of the speciality societies in Australasia as providers of high quality professional development activity particularly focussed in that medical education sphere—and so their meetings, their webinars, their online activities, their resources, are all enormously valuable.

And many of the collegial interactions, which are so educational, in which there’s behaviour-modelling and various other things that are valuable, are done within the organisations in which people work. And so I’d really like to think that we can develop a collegial approach with the special societies so that the system, if you like, of MyCPD works for everybody and that the societies can actually focus on content, which of course is much more interesting and much more entertaining than having to develop and maintain a system.

MIC CAVAZZINI: There are many other activities that are included in this category one such as academic outreach visits where a trained facilitator will visit your workplace and coach clinicians about particular prescribing practices, for example. There’s a lot already out there which listeners might not recognise is part of their CPD.

RICHARD DOHERTY: Yeah, and this is something I’ve pondered over, over a long time. Physicians tend to go looking for information in the literature and focus, or shape, their learning in CPD to address the issues that are challenging them at the time. And you go away, you log the hour or so that you spent reading that article, and then you do some reflection, and you change your practice. And, in fact, probably what you’ve done is a mental audit of your own practice anyway as part of that reflection. The challenge for us in the College is to make tools available to people to make those mental audits real and to give them validity. Because, of course, when we do those sorts of mental exercises we can run the risk of being incomplete and selective.

MIC CAVAZZINI: And before we go, let’s also mention a few of the other housekeeping aspects of the MBA's Professional Performance Framework, or the PPF as you'll hear it referred to. Under the concept of strengthened CPD there’s also the requirement that all doctors are to have a CPD home. So does this mean that doctors have to record their CPD through a provider like the College?

RICHARD DOHERTY: Yep, this addresses a specific issue in the Australian national law at the moment which allows practitioners to undertake a self-directed CPD programme, and so we know that when some of those people have been selected for random audit by the medical board or by AHPRA that they’re coming to grief because the structure of the programme they’ve followed is not consistent with that of the accredited CPD programme for their specific speciality. But that option is going to disappear.

MIC CAVAZZINI: And the benefit, of course, of operating within an institution like the College, is that at first you'll get a friendly phone call from our CPD team before you hear from AHPRA or anyone up the food chain.

RICHARD DOHERTY: Yep, and the experience we’ve had in the College of Physicians, of course, has been when following up people who've not completed a CPD return by the deadline date is that it’s not that people haven’t done their CPD, it’s that people haven’t done their return. But there is some specific stuff around that, and so in New Zealand it’s very clear that if a Fellow doesn’t submit a CPD return for two years in a row, then we need to report them to the MCNZ. The situation is not nearly so clear in Australia, and we don’t appear to have that same legislative requirement.

I think we can draw very considerable optimism from some of the studies, the one that I particularly like is the study by Goulet, a Quebecois-Canadian researcher in family practice  who showed that basically anything you do is valuable, that it influences your performance. But the subtext in that study was that it’s really only effective if you’ve got a proper collegial association. And so the people who benefitted most were the ones who had an appointment to a hospital or some other formal association with their colleagues, and that goes to the heart of the issue about isolated practice, of course.

MIC CAVAZZINI: And it fits with the theme we’ll get into with the strengthened CPD about having peers to review your practice?

RICHARD DOHERTY: Yeah, indeed.

MIC CAVAZZINI: But that doesn’t just refer to clinical expertise, the MBA talks about personal and professional qualities, and this is where the professional standards fit in, we talked about these with Craig Campbell, and Fellows might be familiar with the previous model, which was the Professional Qualities Curriculum. Many listeners are probably participating in valid activities and not recognising it, and you might have already done some exercises in public speaking which would relate to leadership; courses in clinical ethics or Indigenous culture; and you might be involved in research or developing teaching resources or advising on some policy or advocacy statements, again, these are all relevant to the professional requirements.
RICHARD DOHERTY: Oh, I think that’s absolutely true—a lot of what people do in their day-to-day activity is entirely appropriate as the right sort of professional development and there are very, very few Fellows of this College who have any difficulty whatsoever in meeting their CPD requirements. It’s worth pointing out that the medical board in Australia has articulated very clearly the view that this new Professional Performance Framework should be as efficient and as undemanding as current practice without making it too much more onerous, or without making it more onerous at all.

MIC CAVAZZINI: Thanks to Richard Doherty and Craig Campbell for contributing to this episode of Pomegranate Health. Tune in to part two to hear some good examples of practice review and clinical audit. There’s also an appendix to the podcast at our website racp.edu.au/podcast. Other aspects of the PPF are covered, such as increased scrutiny of ageing physicians and the proposed introduction of professional development plans.

At our website you’ll find a template for designing a professional development plan, as well as all the citations and resources mentioned in the podcast. A good place to start reading is the expert advisory report to the MBA, or the MCNZ’s paper ‘Recertification, the Evidence for Change’ which summarise the academic evidence quite well. And there’s a link to the KeyLIME podcast from the Royal College of Physicians and Surgeons of Canada. The hosts ‘journal club’ some fascinating literature in medical education and are prolific academic authors in their own right. A recent episode discusses feedback of consultants to their residents, and particularly the gender biases that are implicit.

Any advanced trainees of the RACP can request early access to the MyCPD system to get familiar with it before the transition to Fellowship. And don’t forget that just listening to Pomegranate Health earns you credits for educational activity; there’s a MyCPD link for each episode that automatically populates your logbook. Please feel free to continue the conversation about today’s podcast in the comments section. And receive every new episode by subscribing to the mailing list or looking us up in any podcasting app. I’m Mic Cavazzini. Thanks for listening.