Exam results
When will I get my results?
Results were released on Tuesday, 1 March 2022 from 3pm AEDT/3pm NZDT.
There was a delay in the release of results for candidates in Australia while we undertook extra checks due to the additional information provided in the results letters.
As candidates who sat the DWE on 14 February needed their exam result to determine if they would sit the backup exam on 8 March, we brought the results release date forward from Thursday, 10 March 2022.
Important quality assurance steps including psychometric analysis of the exam data were carefully and thoroughly completed before results were confirmed and released. We expedited this process as fast as we could.
We acknowledge the implications of the short notice between results release and the backup exam on 8 March. Arrangements were in place for the exam on 8 March noting that further delays would impact candidate preparation for the 2022 Divisional Clinical Examination (DCE).
How is the College setting a pass mark?
The final exam mark is used to determine if the candidates meet the standard to pass the examination.
The College uses ‘criterion based’ standard setting techniques to determine the outcomes of exams. This means the standard to pass the exam is based on the difficulty of the exam rather than a comparison of the candidates that take the exam. As a result, the pass mark will vary between exams. Bell curves and standard deviations are not used to determine the candidates that pass or fail.
The College sets the pass mark based on a review of the questions by a panel of expert Fellows (the Modified Angoff process) and uses a statistical technique called Rasch modelling to account for differences in difficulty between exam forms. This process ensures that exam outcomes meet the same standards for each administration of the exam. Candidates require the same ability level to pass each exam.
For more information, see the DWE page under the Results Tab there is a section for ‘Setting the pass mark’ that outlines the process of ensuring consistency of outcomes between examination forms.
Was there special consideration in the marking of the exam because of what happened?
Exam results processing followed the established robust process including comprehensive psychometric analysis. Adjustment to individual candidate marks to compensate for technical or procedural issues experienced during the exam is not part of that process.
Because of the technical and procedural issues experienced by some candidates during exam delivery on 14 February, all DWE candidates who received a fail result in this exam were given the option to sit the backup exam on 8 March 2022. This provision aligns to our Special Consideration for Assessment Policy (PDF) and possible outcomes for applications for special consideration due to technical or procedural issues experienced during an exam. The provision was granted to all DWE candidates without need to individually apply.
Why can’t candidates who failed the 14 February exam by one or two marks, just be granted a pass?
The College has a rigorous process to determine the pass mark based on the Modified Angoff process and the pass mark is equated using Rasch Modelling to optimise the fairness of the pass mark for each exam. This pass mark is determined without looking at any individual candidate’s scores but rather based on the dataset as a whole so no personal bias is involved when setting the standard.
Once the pass mark has been set, candidates at or above this pass mark have been determined to have met or exceeded the standard required to pass. Candidates under this pass mark have not demonstrated the standard required to pass the exam.
The College can only assess the responses provided by candidates. During analysis, we used CBT data as the primary data source and used paper backup scoresheet data when necessary.
The Special Consideration for Assessment Policy (PDF) states that:
3.2. Where special consideration is granted, the trainee will not be excused from meeting a requirement, a standard for performance or qualification which have been objectively set by the RACP. Marks will not be adjusted. Exam results are not appealable.
When will exam results be released for the backup exam on Tuesday, 8 March?
Exam results will be released on Tuesday, 12 April 2022.
Applications for the DCE close on Monday, 28 March 2022. All candidates registered to sit the backup exam have been given Conditional Enrolment for the DCE, with enrolments confirmed after results release.
How will the pass mark be determined for candidates who resit one exam paper on 8 March and keep their score from their other exam paper from 14 February?
Traditionally the Divisional Written examination has had only one overall pass mark based on the examination as a whole. In 2022, candidates were offered the option to use their results from one paper taken on 14 February and the other paper taken on 8 March.
To enable pass marks for candidates taking papers in different examination administrations in February and March 2022, a modified approach using pass marks for each individual paper will be used for this exam only. The pass mark will still be based on the Modified Angoff score and will use Rasch Modelling to equate the pass mark to previous exams. Marker items will not count toward candidate final scores.
If candidates elected to take a paper on 8 March that they previously sat on 14 February, we will not count their paper taken on 14 February. We cannot consider taking the best of the two papers as this would be unfair to candidates as this is a backup examination. Other unsuccessful candidates from previous exams have not been awarded this same opportunity. This is in line with the decisions of other RACP examinations that have had to split their examinations such as those affected by COVID-19.
Pass marks for each paper of the 14 February DWE for Adult Medicine were set at:
- Clinical Applications: 50 out of a total 85 questions (58.8%)
- Medical Sciences: 39 out of a total 62 questions (62.9%)
Pass marks for each paper of the 14 February DWE for Paediatrics & Child Health were set at:
- Clinical Applications: 56 out of a total 96 questions (58.3%)
- Medical Sciences: 33 out of a total 67 questions (49.3%)
Pass marks and total questions for each paper of the 8 March DWE will be determined after the examination has been administered.
The overall examination pass mark is established by combining the pass marks for each paper undertaken, divided by the total number of questions in those papers. Candidates do not need to pass both papers however their aggregate score needs to be above pass mark for both papers that they have attempted.
For example, to pass the examination, an Adult Medicine candidate who undertook the Clinical Applications and Medical Sciences paper on 14 February would need to have achieved at least a combined mark of 89 (i.e. 50 + 39), out of a possible 147 questions (i.e. 85 + 62), or 60.5%. If a candidate scored 89 or higher, they met the standard to pass.
As another example, an Adult Medicine candidate who undertook the Medical Sciences paper on 14 February but sat the Clinical Applications paper on 8 March, we will need to wait until we have the pass mark for the 8 March Clinical Applications paper to determine the overall pass mark. If the pass mark for the 8 March Clinical Applications paper was, for example, 52^ and was determined out of total 87 questions^, the candidate would need to have achieved at least a combined pass mark of 91 (i.e. 52 from CA 8 March + 39 from MS 14 February), out of a possible 149 questions (i.e. 87 + 62), or 61.1%. Note that the number in the total number of questions is different because in this example the CA 14 February paper was out of 85 questions and the CA 8 March paper was out of 87 questions.
^ figures provided for illustrative purposes only and are not necessarily indicative of the 8 March 2022 scores.
14 February 2022 exam
What happened during the 14 February DWE exam?
A number of trainees sitting the computer-based Divisional Written Examination (DWE) on Monday, 14 February 2022 were affected by delays in logging on to the exam system at the start of one of the two exam sessions. Candidates have also since reported experiencing other technical and process issues with delivery of their computer-based exam.
Some candidates were able to continue on to complete the sessions at the end of the day. Alternatively affected candidates were offered the option to sit the exam on the backup date of Tuesday, 8 March 2022.
We have apologised to affected candidates.
Will there be an independent review of the DWE CBT function?
RACP has appointed KPMG to examine its Divisional Written Computer Based Testing function.
KPMG will be looking at a range of matters including our engagement of suppliers, the events on the day, planning for the exams including contingency planning and risk mitigation.
KPMG were chosen because several of their staff worked for Ferrier Hodgson, who assisted us in 2018 following the exam process for that year and so are familiar with the College and the examination function.
The cost, and terms of reference for the KPMG engagement are confidential.
Given the complexity of the engagement we expect it will take several months.
Once it is complete, we will inform the membership of any further actions we intend to take.
Was any exam data lost?
Since our initial assessment where it appeared that we had data for all candidates, a more detailed analysis has confirmed that 6 candidates in Western Australia were affected with incomplete exam data for one paper. We have double checked and confirmed that this issue is confined to those 6 people. We have now finished contacting them and consequently are able to notify the wider cohort of this issue. Our review of what caused this is still underway and we will provide further information when we have a clear picture of what has occurred.
How many candidates were affected?
Initially we knew that around 100 candidates in Auckland, Melbourne, Brisbane and Perth who sat the computer-based exam were significantly affected. These candidates could not complete a paper and/or had a shorten exam time.
In the days since the exam, we heard about a range of concerning candidate experiences which contributed to an unsatisfactory exam experience for many. We apologise to those affected and recognise the additional stress imposed during this important examination for all candidates.
What caused the log on delays and related issues?
We are looking into what caused the delays and any other reported issues. We are treating this seriously and working now with our exam provider to establish what happened.
Is this the same problem as occurred with the exam held in October 2021?
On Monday, 14 February 2022, some candidates experienced delays in being able to log on to the exam system at the start of the one of the two exam sessions and were unable to complete their session in the allocated time. Candidates have also since reported experiencing other technical issues with delivery of their computer-based exam.
We are still reviewing what happened and will update you when we have more information.
Following the October 2021 problem, further backup measures were incorporated into the exam platform and the paper answer sheet was provided to all candidates.
How did the College not know about this problem before the exam started?
The issues experienced by candidates on Monday, 14 February 2022 were not identified through the extensive exam system testing program implemented by our exam provider and College staff before the examination on 14 February.
As soon as the log on delays were identified our exam provider began working with us and the venue provider to try and enable candidates to complete their exam sessions on the day.
We are working now with our exam provider to establish what happened.
What is the College doing to help trainees who were impacted?
We sent a communication to all DPEs and candidates about what happened. We have apologised to candidates. We are communicating new information as it becomes available and updating the FAQs on our website.
Some candidates were able to continue on and complete their examination. Impacted trainees were given the option of sitting the backup exam on Tuesday, 8 March 2022.
We have individually contacted candidates who were most impacted to identify other support we can provide. We are keeping all candidates informed with updates and answers to frequently asked questions. The RACP Support Program, delivered by Converge International, is also available to all members with contact information available on our website.
Australia 1300 687 327
Aotearoa New Zealand 0800 666 367
Will this exam attempt count as an official exam attempt?
The 14 February 2022 and backup 8 March 2022 Divisional Written Examinations do not count as an official exam attempt for unsuccessful candidates.
The College Education Committee (CEC) had already made this decision in recognition of the notable Omicron impacts on preparation time over December-February for a majority of trainees.
If there were disruptions at my exam but I achieve a Pass for the exam will this count as a Pass, or will I be asked to resit the exam?
All candidates who achieve a Pass will have their Pass counted and will not need to re-sit the exam.
How do I provide feedback to the College about my experience at my exam?
We welcome your feedback about your exam experience. All candidates received a Post Exam Candidate Survey (PECS) to complete and provide feedback about their experience. You can also email us at examinations@racp.edu.au
Why were there not paper backup exams at venues?
Many have queried why paper back-up papers were not available to be handed out. Distributing a full backup paper test to candidates in the middle of a CBT that is underway is potentially disruptive.
Also, suitable venues of appropriate sizes and locations that can secure and release paper-based exams in parallel to a CBT would be necessary and those of sufficient size are set up for one format, or the other, not both.
The collection, secure storage, and secure return of exam papers, in conjunction with candidates continuing to use CBT, could prove highly problematic for invigilators and there is the potential for documents to be missed.
As part of our planning the examinations team and the Fellows who work with us have already studied these issues in detail. The fairest and most practical solution for all is to offer an entire backup paper exam.
All candidates were provided a backup paper scoresheet to complete so that their responses were captured in the event of a system failure.
Will we get a partial or full refunds because of what happened?
We have heard your queries about possible refunds of examination fees.
We are aware that candidates may have experienced difficulties during the Divisional Written Examination undertaken on 14 February 2022, and that the exam delivery issues on the day may have impacted their performance and their ability to complete both exam papers. We regret these circumstances. We also recognise the hard work candidates put into preparing for this exam and therefore, as a gesture of good faith the College has decided to refund the full examination fee for candidates who failed the CBT exam on 14 February 2022.
Candidates who sat and failed the paper based DWE on 14 February 2022 will not receive a refund of their examination fees as their exam delivery was not disrupted. As a special allowance, candidates who failed the paper based DWE on 14 February 2022 were given the option to resit the exam on 8 March without an additional fee.
8 March 2022 backup exam
Who was offered an exam resit on 8 March 2022?
The College is aware that candidates may have been directly or indirectly impacted by disruptions during their exam. Since the exam, we heard about a range of concerning candidate experiences which contributed to an unsatisfactory exam experience for many of our trainees.
All candidates who failed or were unable to complete the DWE on Monday, 14 February 2022 were given the option to sit the backup exam on Tuesday, 8 March 2022.
This provision applied to all candidates who failed regardless of whether they sat the CBT or paper-based test, their exam venue, or the extent to which they were affected during their exam. It was not by exception and did not require special consideration.
Was the backup exam on 8 March computer-based?
The backup exam on Tuesday, 8 March 2022 was a paper-based exam for all candidates. The exam was not offered as a Computer-Based Test (CBT).
If I could not attend the 8 March backup exam due to flooding, COVID or other exceptional circumstances, will there be another resit offered before October?
There is no additional exam scheduled until October 2022.
Any exceptional circumstances preventing exam attendance will be dealt with as special considerations for individual candidates.
If I sat the 8 March 2022 backup exam, will I have my results in time to register for the DCE?
The College is aware of the need to have results so you can register for the Divisional Clinical Examination (DCE).
Applications for the DCE opened on Thursday, 10 March 2022 and close on Monday, 28 March 2022.
Because your result for the 8 March 2022 DWE will not be released until after the close of applications, you have been given conditional enrolment for this exam. This means you can apply online and pay the 2022 DCE exam fee by the closing date of 28 March 2022. Once your DWE result is confirmed, if you are progressing to the DCE you will be advised that your application has been accepted and you will shortly be allocated to an exam date and site. If you do not progress to the DCE, you will be notified and you will be refunded the full amount of the DCE fee.
What options did candidates have for sitting one or both papers on Tuesday, 8 March 2022?
All candidates sitting the backup exam on Tuesday, 8 March 2022 were registered to sit the clinical applications and medical sciences papers.
However, if you sat and failed or did not complete the exam on 14 February, you did have the option to resit only one of the exam papers on 8 March 2022 and carry over your score for the other paper from your exam on 14 February 2022 if you chose.
This decision could be made up until the start of the exam on 8 March 2022.
Candidates were required to sign in and out of each exam session they attended to record the papers they sat again. If a candidate did not attend one of the exam sessions, their exam score from 14 February for that paper will be counted as their final score.
Unless candidates achieved a high score on one paper in the 14 February exam, we recommended they sat the full exam on 8 March to give themselves the best chance of success. If you resat for a paper on 8 March, your score for that paper will be counted as your final score and you cannot revert to your score from 14 February.
October 2022 exam
Will the October exam be delivered by computer-based test?
We have confirmed that the October 2022 DWE will be delivered by paper-based test.