Item response model

The item response model report produces a characteristic curve for each item in the exam which estimates the relationship between the candidate's ability on the trait being measured, and the probability of them endorsing the correct answer.

risr/ assess uses a 2-parameter logistic model which includes the item's difficulty and discrimination attributes.

If you are interested in learning more about item response models, the Wikipedia page is a good place to start: https://en.wikipedia.org/wiki/Item_response_theory 

Running the item response model report

To run the item response model report:

• Step 1 of 4

  Navigate to your exam then Set standard and View reports.

  

• Step 2 of 4

  Click Create new report and Item response model.

  

• Step 3 of 4

  Select the date for which you wish to run the report. You can also optionally give the report a name. Select the items you wish to include in the report (SHIFT+select or CMD/CTRL+select for multi-selection). Alternatively do not select any items in the list to include everything. 

  

  Click Save to commit.

• Step 4 of 4

  Click Preview to view the output.

  

  You can also export a copy of this report to PDF.

Report output

An item characteristic curve is shown for each item in the exam along with Item Response Theory (IRT) and Classical Test Theory (CTT) metrics.

Metric	Description
(IRT) Difficulty	This is the item difficulty parameter derived from the IRT model. The value displayed is the point on the ability scale at which 50% of the test-takers endorse a correct answer. In the above example case the 50% point on the probability scale intersects the ability scale at 1.2113. This can also be observed in the graphical output. The further to the right the curve, the more difficult the item (i.e. a higher ability needed for 50% of respondents to answer correctly).
(IRT) Discrimination	This is the discrimination parameter derived from the IRT model and indicates how the rate of change in ability affects the probability of answering correctly. The higher the value (and the steeper the slope of the curve) the greater the discriminative power of the item.
(IRT) Chi-squared test	This is an indication of model fit. Essentially, how closely the model (blue line) predicts the observed data points (yellow line). The closer the value to zero, the better the model fit.
(CTT) Facility	This is the proportion of candidates that answered the question correctly, essentially the the sum of candidate scores / the sum of maximum possible scores. The higher the value, the higher the proportion of candidates endorsing a correct answer (and therefore the easier the item).
(CTT) Discrimination (point-biserial)	The point-biserial value is a linear correlation between the performance of candidates on the item with their performance across the exam as a whole. The higher the value the greater the strength of the relationship between the item score and the test score.