Updating marksheet criteria

OSCE mark sheets are subject to strict version control safeguards which ensure that any changes made to the schema or criteria are not automatically cascaded into live exams.

The general principle is that, when items travel through the review and approval process, a snapshot of the mark sheet is taken and this is indelibly associated with that version of the approved item. Any mark sheet rows that were applied from global criteria or use global schema effectively become local copies attached to the item version. However, the links to the original global schema and criteria are silently maintained so that when a new version of the item is created, those elements can be resynchronised with their global counterparts and the mark sheet inherits any changes that might have been made at the global level.

As discussed in earlier articles OSCE item scoring components consist of both the measurement criterion and the schema on which it is being measured. Whilst global criteria will always use a global schema, local criteria may use a global schema or a local one designed individually per item mark sheet row.

The below table summarises what happens as the item travels through the review and approval process.

Criterion	Schema	Outcome
Local	Local	No change. Both the criterion and the schema continue to be local to the item.
Local	Global	The local criterion remains unchanged. A local copy of the schema is taken and applied to the version of the item. The link to the global schema is retained in the background.
Global	Global	Copies of both the criterion and the schema are taken and applied to the version of the item. The links to both the global criterion and the global schema are retained in the background.

The item view

This image below shows a newly created (yet to go through the review and approval process) simple OSCE mark sheet with a mix of the three possible criterion/schema formats listed in the table above.

The two highlighted columns give us information about the status of the criterion/schema for each row.

• In the Schema column we can see that rows 1, 3 and 4 are using global schema (these have names, 0/1 and Borderline, that were given during their creation). Row 2 uses a local schema built directly into this criterion.
• The Global column tells us first of all whether the criterion originated from a global schema at all (i.e. rows 1 and 4) and, if so, its current status.

The in-app guidance tells us more about the status of the global criterion:

In this case, because the item is yet to go through the review and approval process, the black dot tells us that this is still a global criterion. Furthermore, at this point, any further site-level global criteria we create (subject to any ApplyIf conditions) would be automatically applied to this mark sheet.

At creation, global and local criteria are grouped and presented separately.

What happens when the item is approved?

In the image below, the item has now been put through the review and approval process, and we can see the approved v1 of the item.

Notice a few key changes to how this item looks:

• In the Schema column, the rows that use global schema (rows 1, 3 and 4) now tell us that they are using a local schema based on the original global one. As discussed in the introduction to the article, a snapshot or copy of the schema has been taken and applied to these mark sheet criteria. There is no change to row 2 which always used a local schema.
• In the Global column the black dots have now changed to green. Again referring to the in-app guidance, this tells us that the a local copy of the global criterion has been taken and applied to these rows.

Notice also that the local/global groupings have now disappeared. 

What happens if I make a change to the global schema?

In the below image we have changed the 0/1 global schema to 0/1/2 because we have decided that we want to instead measure these checklist criteria on a three-point scale.

We are still in the approved v1 of the item, but notice the changes to the mark sheet view.

In the Schema column we now have a warning telling us that changes have been made to the global schema. Hovering over the warning triangle also gives a tooltip description.

Note that no changes have yet been made to the item; only warnings given. Changes cannot be applied until you create a new version of the item. 

What happens if I make a change to the global criterion?

It's a similar situation for any changes to global criteria too. In the image below we have changed the Global borderline criterion to instead read Global judgement based on feedback from our examiners.

Notice two additions to the mark sheet view:

• A warning bar at the top of the page telling us that our global criteria have been changed.
• A warning triangle next to the specific global criterion that has been changed. Referring back to our in-app guidance, we can see this described.  

Notice again that no changes have been made to the item at this point. We are still in the approved v1 of the item and only warnings are issued at this point. 

How do I synchronise my changes?

To synchronise these changes to the item's mark sheet a new version of the item must be created. the image below shows the draft v2 of the item.

We now have:

• Individual refresh buttons agains the schema and criteria that are eligible for resynchronisation
• (From risr/ assess version 8.10) an update all obsolete criteria and schema button.

Once those obsolete schema and criteria have been updated, the mark sheet will be back to its original state having inherited all the changes made to the global schema and criteria.

The schema and criteria are still local copies of the global ones, but they now align with the most recent version of their respective global counterparts.

Notice how row 2 (local schema, local criterion) has not changed throughout.

Synchronising in bulk

The article has so far described the process for updating individual mark sheets, however it is also possible to push out any global criteria changes to items in bulk.

In the example shown here we have again changed the name of the global judgement criterion.

To apply this change to items in bulk:

• Step 1 of 4

  Navigate to Settings > Global criteria then Options > Admin against the criterion.

  

• Step 2 of 4

  Next you will see the criteria management tools screen which gives an indication of how many items are eligible to receive the change you have made

  

  In the example above there are three items in an approved state and 1 archived item that are eligible for updates. You also have an option to download the list of items in each status.

  The update options are to either Create a new version of all items and leave in draft or to Create a new version of all items and immediately approve. Choose the appropriate option.

• Step 3 of 4

  The system will ask you to confirm your choice

  

  Click make changes.

• Step 4 of 4

  This will initiate a background task and once that is complete you will be able to see its results.

  

  Under the recent results dropdown you can download a list of the items that have been changed.