Online Assessment/ Open-book assessments
This page was written in collaboration with Dr Nicola Crewe.
When writing or designing open-book assessments (for delivery in Time Constrained Assessments (TCAs), Multiple Choice Questions (MCQs), etc.) it is important to develop questions that challenges a student’s understanding of the course content rather than their recollection of facts. By choosing question types that allow for synthesis and application of knowledge, the students will show mastery of the subject knowledge. Below you can find some examples of how this can be achieved.
The key to writing effective open-book assessment questions is to challenge students to apply their understanding to a novel situation or scenario.
Remember: To develop robust open-book exam questions students must be challenged to apply their knowledge rather than regurgitate facts. Memory recall questions lack robustness, only access surface level learning, and unfortunately fail to promote active and deep learning that is important for synthesis of knowledge.
Try to ask students to apply their understanding to a novel situation and work through a series of questions that link together, developing their thoughts as they go. This allows for answers with increasing complexity and depth.
For this to be effective, marking schemes may need to be quite broad and open ended. You are able to reward students who analyse an angle of the problem that is entirely appropriate, even though it may not have been considered in the original marking scheme.
Under traditional examination settings, open-book exams would traditionally allow students to utilise their notes to enhance their answers, and it is common practice to advise students to strategically collate concise notes to avoid the time searching through full textbooks in exams. Online open-book exams (such as TCAs) however mean that all online resources are easily available via any internet enabled device. It is essential therefore that your assessment design anticipates this in order to ensure effective and fair assessments.
Scenario-based questions challenge the student to adapt to specific context and detail, making the answers impossible to ‘Google’.
Students can search for specific terms, and even definitions to jog their memory, but the context can only be provided by the students in relation to the scenario. The biggest challenge with this is to ensure students tackle their learning and subsequent revision in a different way. There is very little need to fact learn with an open book exam – although some simple questions with standard answers can be included – sitting and learning notes off by heart will not produce success in the final exam.
Students must understand the concepts and be able to apply them to real-world situations, requiring deeper learning, reading around the subject area, and most importantly, the chance to practice this application.
After providing students with a contextual scenario, consider asking students:
- What is the impact of XXX in this situation?
- What outcome would you expect if… ?
- What alternatives to XXX could be applied in this situation?
This assessment strategy needs to be highlighted to students throughout the teaching period, and embedded within revision strategies in order to be successful. Students should have example questions at regular stages so they can practice their skills. Revision becomes an opportunity to spend time reading articles/journals/opinion pieces and see how the subject area is being applied in real-world situations, rather than creating sets of notes that they will memorise.
“In my experience with open-book MCQ exams, the assessment consistently showed a full, normal distribution of marks, including up to and above 95%. Closed-book exam approaches tend to show a normal distribution but don’t always allow for students to achieve the full range of marks.”