Exams: call time on the academy’s Hunger Games

Kevin's description highlights the all-too-common situation in assessment within (particularly undergraduate) university courses in which there would appear to be no routine psychometric input. A valid assessment will start with input from a team of individuals with logistical expertise, content expertise and psychometric expertise. For example assessment items are structured in a particular way to minimise construct irrelevant variance and match predefined purpose of the test. Also, there is no such thing as an item that is too easy / too hard. There is only the issue of constructive alignment with the curriculum. Defining the performance standard is a separate exercise (standard setting) as is ensuring the standard of the consistency between years (equating). The proper application of standard setting and equating also deals with 'grade inflation' from a statistical point of view (but not a political one - if the college down the road gives more firsts than you why should you moderate your numbers) Statistically there are optimum numbers of distractors (options) for MCQs (again depending on the purpose of the test). Adding non-functioning distractors will add nothing to your test. 'Normalising' scores has a particular meaning in psychometrics but in practice this term may be used to describe many different and magical procedures that turn unacceptable score distributions into acceptable score distributions. Kevin may very well be correct about assessment needing an overhaul but any / all methods will suffer in the areas mentioned above without the proper input from the appropriate experts. We wouldn't dream of trying to put together an exam without content experts but it is still common to create, administer, generate score and make decisions without appropriate psychometric input.