The set standard screen explained

Once the assessment data has been collected and evaluated (i.e. automatically scored, marked or examiner judgements from the OSCE) the act of processing that information into examination outcomes is carried out in the set standard page. 

The set standard page provides a vast amount of information about the data collected from the exam. It's also where we can begin to apply the standard setting methodologies discussed in the earlier article. 

Navigating the set standard page

To navigate to the set standard page, go to Exams > (your exam) View Sessions > Set standard

An example set standard screen is shown here:

Each set of options are explained in the expandable sections below.

• Field/button	Function
  SEm Multiplier	This value acts a multiplier to apply to the SEm in order to create a confidence interval around the cut score. Common values include 1.64 (90% CI) and 1.96 (95% CI). If you wish to not use a confidence interval on the cut score set this value to 0.
  Minimum items to pass	This is the conjunctive criterion, or the minimum number of items in the exam that the candidate must pass in order to pass the exam overall. This is commonly used in OSCE exams alongside the overall passing score to prevent cross compensation of poor general performance across lots of stations with excellent performances in a few.
  Standard setting method	Use the dropdown to select the algorithm to apply to the exam data in order to process the items. Where benchmark or adjustable borderline regression is selected, additional user input is required to process results. See the respective articles for further information.  Where Angoff, Ebel, borderline groups, borderline regression and McManus borderline regression are selected, no further action is required.
  Dimension pass mark	This feature makes it possible to use the item tags present in the exam to create a dimensional passing criterion. For example, in order to pass you may want candidates to achieve the overall exam score as well as a particular score in or across items or stations tagged to particular category.
  Grades	Navigates to the grades view where descriptive grading bands can be applied to the examination outcomes.

• Field	Description
  N	The number of candidates present in the exam.
  No of items	The number of items present in the exam.
  Mean score	The average candidate score across the exam.
  Exam cut score	The exam cut score as generated by the standard setting algorithm applied to the data.
  Exam max score	The maximum possible score achievable on the exam.
  Standard deviation	The standard deviation (measure of spread) of the exam scores.
  Alpha	Cronbach's alpha (measure of exam reliability) value computed from the exam data.
  Sqrt alpha	The square root of the Cronbach's alpha value.
  SEm	The standard error of measurement (an estimate of the spread of measurement error present in the exam).
  SEm * multiplier	The standard error of measurement multiplied by the SEm multiplier set above. This is the size of the confidence interval applied to the cut score.
  Cut score + SEm	The calculated cut score plus the (SEm * multiplier) value. Essentially the cut score + confidence interval.
  Passing score	The final passing score (subject to any rounding and display configuration rules).
  Fail on score	The number of candidates that have failed the exam because their total score is less than the passing score.
  Fail on CSP	The number of candidates that have failed the exam based on having received a critical stop point judgement. This is those who have failed the entire exam based on a CSP, not an individual station.
  Fail on min stations	The number of candidates that have failed the exam because they have not achieved the required number of stations as set above.
  Fail on dimension	The number of candidates that have failed the exam based on any dimensional passing criteria as set above.
  Passed	The number of candidates have have passed the exam.

• Button	Function
  Items	Navigates to the item processing view where you can see individual item performance data as well as amend individual item cut scores and exclude items from the exam, should that be necessary.
  Candidates	Navigates to the candidate view where you can view individual candidate performance as well as, with sufficient permissions, adjust both candidate/examiner interaction data as well as raw response data. You can also download raw candidate response data from here.
  Examiners	Navigates to the examiner view where you can see an overview of the scores provided by each examiner (only available for OSCE).
  Amend	Allows you to amend the final passing score for the exam (i.e. override the system value determined by the standard setting algorithm and any statistical adjustment).
  Normalise	Allows you to normalise the data in the exam to a consistent denominator.

• Button	Function
  View sessions	Navigates back to the view sessions screen.
  View reports	Navigates to the exam reports screen.
  Download standards	Exports the data shown in section 2 to spreadsheet format.
  Export results	Exports the processed candidate result data to spreadsheet format.
  Update item performance	Passes back the processed item performance data generated in the exam to the item bank for display against individual items.
  Finalise exam results	Puts the exam into a finished state so that no further processing can occur on the data.

Updating results

When applying particular steps during standard setting, you may see the following warning bar appear.

This simply requires the data to be reprocessed and you should click update results before proceeding with any further changes.