Item-level processing

The items screen lets you view individual item performance and carry out administrative functions such as adjusting the item-level cut score or excluding poorly-performing items from the exam.

To access the Items screen click Items from the Set standard page.

Items view explained

The image below shows the items view for an OSCE exam, but the output for written exams is identical. The table can be sorted by clicking the column headings.

1. Item: The item reference and title. Click the link to view the item content.
2. Submitted count: The number of submissions/answers present for the item. In the OSCE this is the number of mark sheets received whereas in written exams it would be the number of candidate responses received for each item.
3. Passes: The number of passes (i.e. candidates who have achieved above the cut score) for the item.
4. Mean score: The simple mean score achieved by candidates on the item.
5. Cut score: The system-generated cut score on the item generated by the standard-setting algorithm applied. In this example it's the score generated by the borderline regression calculation, but for written exams might be the Angoff or Ebel value.
6. Std Dev: The standard deviation (measure of spread) of all candidate scores on the item.
7. Alpha (if excluded): This is the Cronbach's alpha value should the item be excluded from the exam. A common post-hoc treatment is to analyse the Alpha (if excluded) value in comparison to the overall test reliability (Cronbach's alpha) metric generated for the test. If the overall alpha value improves significantly if the item is excluded, you may consider removing the item from the exam.  
8. Max score: The maximum score attainable on the item.
9. Amend button: Allows you to amend the item's cut score.
10. Exclude button: Excludes the item from the exam.

Amending the item's cut score

As well as the ability to amend the overall exam passing score, there may be occasions where you need to amend the passing score at the individual item-level. The most common examples could include situations where standard-setting processes are performed off platform and pass marks need to be put back into the system, or if a small manual adjustment is discovered to be needed part-way through the process.

To amend the item's cut score:

• Step 1 of 3

  Select Amend against the item in question

  

• Step 2 of 3

  Input the new cut score as well as a justification for the change and Submit.

  

• Step 3 of 3

  Note that the changed cut score for this item has now been applied and the warning triangle shows that this has taken place.

  

  You will also be required to Update results to reprocess and resynchronise the calculated values. 

If you need to reverse this process, just remove the value and Submit. 

Excluding the item from the exam

If an item is found to be performing adversely in the exam, it is also possible to exclude it from the calculations from this screen.

To exclude the item fro the exam:

• Step 1 of 3

  Click Exclude against the item in question

  

• Step 2 of 3

  Add a justification and click Exclude from marking.

  

• Step 3 of 3

  The item is now excluded from the calculations as shown below.

  

  You will also be required to Update results to reprocess and resynchronise the calculated values. 

To reverse this action click Enable against the item.

As well as excluding items, it is also possible to apply some post-hoc adjustments to it (for example, changing the answer key, adjusting feedback visibility). For further information please see the dedicated article here.