Questions must be designed not only for students to remember knowledge, but also to apply or relate knowledge.
The questions must have different levels of complexity and must be balanced from that point of view. It is recommended 70% of the questions of medium difficulty, 10-15% of easier questions and 10-15% of difficult questions.
In the context of multiple choice, the ideal time to answer a question is dependent on its nature and the need to a more elaborated development/reasoning.
The nature of the questions must be considered carefully. One must consider the need for some critical thinking in order to arrive at the correct answer on the part of the students. Depending on the functionality available on the platform used, the focus should be on reasoning and understanding.
Recommendations for questions preparation:
- The assessment must-have clear instructions;
- The main idea must be in the question avoiding that it is contained in the answer options;
- Avoid very long assessment;
- Use the language that was used throughout the classes or in the materials provided;
- It is suggested that the questions list be written in the form of a question;
- Avoid writing negative statements, but if you use it, ensure that the word “No” is underlined (eg, “Which of the paragraphs is not correct?”);
- The questions list should allow the student to remember which subject to consider in the answer;
- The questions list can refer to the justification of methods and procedures;
- The questions list should refer to the question “what is the most correct option?” and not just “what is the right option?” – so students realize that there are options that have something valid, but that is not the most correct option;
- Questions should be randomized so that each student questions arise in a different order or have a bank of questions. This makes fraud difficult by merely sharing the key solution of the evidence;
- The answer options must have a similar extension since the longest options often correspond to the correct option because it has a greater number of elements.
Recommendations for preparing responses:
- Response options must be globally plausible, avoiding those that are obviously wrong;
- Avoid answer options like “all are correct” or “none of the other options is correct” because it creates unnecessary confusion and the student will only have to realize that most options are correct/wrong;
- Make use of typical student mistakes in writing incorrect options or answers to open questions given by students in assessment instruments from previous years;
- Avoid very long response options;
- Avoid response options with very specific expressions that can be easily found in the UC book or slides by a simple search;
- In the case of multiple-choice questions, the order of answers within each question must also be random.