We undertake and promote strategic and collaborative research and evaluation that together generate high quality, trustworthy knowledge to drive evidence-informed decisions and improvements to the effectiveness and impacts of RACP training programs.
Education Renewal Evaluation Strategy
Supporting high quality physician education
As a specialist medical training provider, medical education research and evaluation is a key priority and function of the RACP. It guides innovation, ensures the quality improvement of the training programs, embeds change in an evidence-base and is a core requirement of our accreditors.
Our approach
Physician education occurs within non-homogenous, complex environments that are dynamic, non-linear, and non-simplistic, with multiple stakeholders, leading to recursive and probabilistic outcomes. The success of any intervention in this environment is highly context dependent. Drawing on theory-based evaluation approaches, our evaluation approaches acknowledge and explore the impact of these relationships and describe the assumptions, risks and external factors that may influence the outcomes.
Three theoretical lenses
Examining Impacts
Kirkpatrick’s levels of evaluation (adapted for medical education) are a useful framework for classifying the extent of impacts, ranging from proximal to distal outcomes.
Tracking Change Interventions
Theory-of-change (program logic) articulates a program theory by linking an initiative’s inputs, activities, outputs, and outcomes. Explicitly mapping this logic of change integrates the design and evaluation processes. It is a commonly used approach in program evaluation.
Unsurfacing complexity
Contribution analysis using realist evaluation theory helps unpack what works, for whom, and in what circumstances, allowing credible claims of causation to be made in light of situational or contextual challenges. This lens views the program as a part of a ‘causal package’ in which various influences and interventions work together to bring about change (intended and unintended).
Our organising framework
A pragmatic combination of the three theoretical lenses outlined above will be used to enable meaningful evaluative activity that recognises context, and starts to understand the nuance of different outcomes. Multiple, systematic yet discrete evaluation activities will be brought together to tell a broader ‘contribution story’ about distal impacts such as physician performance and healthcare experiences and outcomes.
Read our approach to evaluation and published reports.
Login
Publications
Educational renewal of physician training in Australia and New Zealand: Multiple educational innovations in a complex environment