Examiner performance report

The examiner performance report is designed to help you compare the marking profiles of OSCE examiners and to assist in the hawk and dove analysis. 

Running the examiner performance report

To run the exam analysis report:

• Step 1 of 4

  Navigate to your exam then Set standard and View reports.

  

• Step 2 of 4

  Click Create new report and Examiner report.

  

• Step 3 of 4

  Select the date for which you wish to run the report. You can also optionally give the report a name.

  

  Click Save to commit.

• Step 4 of 4

  Click Preview to view the output.

  

  You can also export a copy of this report to PDF.

Report output

The report's output is shown below and has a separate set of metrics for each item in the exam. You can click each examiner's name to drill down to the raw scores they awarded:

Metric	Description
Station mean	The simple average score was achieved on the station as a whole.
Station std. dev.	The distribution of score values on the station as a whole.
Z-score	A measure of how far away the examiner’s mean score lies from the station mean score and expressed in standard deviations. Calculated as the difference between the examiner’s mean score and the item’s mean score divided by the item’s standard deviation.
Mean score	The simple average of scores awarded by the examiner
Std. deviation	A representation of the dispersion of given scores (calculated as the square root of the variance). Larger values mean the scores are more spread out whereas smaller values mean the scores are more concentrated.
Scoring profile curve	The graph displays each examiner's scoring profile represented as a normal distribution and derived from the mean and standard deviation values. The x-axis is the score and the y-axis is the probability density. The area under the curve, and up to each scoring point, is the probability of the examiner awarding that score. The graph attempts to abstract examiner behaviour from the actual cohort data and generalise to the whole ability range. It therefore assumes the marks given are samples from a normal distribution.  The apex of the curve represents the relative mean score and its width indicates the scoring spread. You can toggle each examiner's curve on and off using the tick boxes on the left.