Short Answer Question (SAQ)

The short answer question (SAQ) item type requires the test-taker to provide an open-ended text response to a prompt.

SAQs always require human marking.

Building an SAQ item

To build an SAQ item:

• Step 1 of 3

  Navigate to Items and Create new item.

  

• Step 2 of 3

  Complete the fields on the new item screen.

  

  1. Item type: Select Short answer question from the dropdown menu
  2. Item title: Give the item a name. This is entirely free-text and at your discretion, but it is required. This is not visible to candidates during the exam, but it can optionally be used later in the candidate feedback report.
  3. Item number: The item number auto populates but can be overwritten. Each item type maintains its own sequential index from 1-n (e.g. ARQ1, ARQ2 / SBA1, SBA2 etc.). By default this field will receive the next available sequential number for that item type.

• Step 3 of 3

  The basic form fields for the item type are now made available.

  

  1. Item number: This is the item number established in the previous step. It can no longer be changed. 
  2. Question author(s): Unless specified, the question author will default to the current user. Alternatively you can select another author if you are inputting the material on someone else's behalf. Start typing a name in the field and the system will suggest names based on the list of users present in the system.
  3. Item title: The item title entered in the previous step. This can be changed here.
  4. Summary: This is the item stem. You can use the WYSIWYG editor to write your question prompts. 
  5. Comments: The comments field is used for notes against the item. These do not appear to candidates and are for administrative use only.
  6. Item notes: The item notes field can be made available to candidates in the candidate feedback report and is designed to be used for any predetermined answer justification.
  7. Maximum Mark: SAQs are manually marked by examiners. During the grading process examiners can select from a dropdown of available marks to assign for each question. Select a maximum mark here to limit this dropdown to a certain range of values. 
  8. Number of lines: specifies the number of expected response lines displayed on the written paper.
  9. Angoff: An Angoff rating for later standard setting can be banked against the item. If a value is given, this will be used if Angoff is selected as the standard-setting method.
  10. Difficulty and relevance: If Ebel standard setting is used, you can bank the difficulty and relevance values agains the item.
  11. Response to this question is mandatory: select if you want a response to be mandatory or not.

  Click Save item to commit the changes.

SAQ Marking Criteria

In the previous section we discussed how it is possible to set the maximum score value for the SAQ item. In the event that you would like to add a more prescriptive scoring rubric for markers to use when evaluating responses, this can be achieved via the Marking criteria function.

This is entirely optional. If no rubric is added, markers will be able to award a single overall score for the item during marking. If you do add a rubric, markers will need to follow its structure to award their marks. 

To add marking criteria:

• Step 1 of 4

  Navigate to the item information tab and select marking criteria.
  
  

• Step 2 of 4

  You will note that the system has added a single marking criterion that corresponds with the maximum mark added in the previous step. In this case 3.0.

  In the example here want to create a rubric that awards 2.0 marks for one element of the response and 1.0 marks for another.

  Click Edit to make changes.

  

• Step 3 of 4

  In the example below we have edited the defaults to the values we want for our first marking criterion.

  

  1. Criteria: This is the marking instruction provided to the marker
  2. Minimum: The minimum possible value that the marker can select for this criterion.
  3. Max: The maximum possible value that the marker can select for this criterion (Note that this cannot exceed 3.0 as the value given for the overall item)
  4. Increment: These are the value increments from which the marker can select. This has been changed from 0.5 to 1.0 so only whole numbers can be selected.

  The site default for marking increments is 0.5 but this can be changed as a site configuration option.

• Step 4 of 4

  Once the second marking criterion has been added, the item should look something like the image below:

  

  The marker will now see two boxes into which they can add their judgements about each marking criterion.

Item preview

In the browser application, the SAQ item appears with the question on the left. Test-takers are required to complete their answer in the free text box on the right.