Inclusion and diversity are hot topics in Higher Education, with concerted efforts being made to ensure equality for all. However, within traditional assessments there are several hidden hurdles which prevent students from receiving the full benefit of these endeavours.
Why the need for reform
In 2020/21 approximately 370,000 UK higher education students declared a disability of some kind, compared to 230,000 in 2014/15. According to The Higher Education Statistics Agency (HESA) that is an increase of over 69% over a 6-year period. It should be noted that these figures only represent those who have chosen to make a declaration of disability.
HESA also reported that more students from ethnic minority backgrounds are studying for degrees. In 2020/21 a total of 28% of all students were from ethnic minority backgrounds, up 1% from the previous year.
As both these numbers continue to rise, universities are beginning to devote more resources into catering for an increasingly diverse student population, including the forming of new divisions and sectors, focused solely on improving fairness across the university as well as tackling the injustice and discrimination minority students face.
Understanding discrimination in assessment
Section 6 of The Equality Act 2010 defines a disability as a physical or mental impairment that has long-term and substantial adverse effect on their ability to carry out normal day-to-day activities. A higher education provider must not discriminate against students in relation to assessment and exams in the same way they do with admissions, teaching, and facilities.
It’s important for everyone to have equal opportunities in the employment market and for this to happen their degrees must be a true reflection of ability and hard work. However, the increased diversity of cohorts we are seeing are too often not represented by the test authors themselves and therefore the opportunity for unconscious discrimination arises.
There is a risk during the exam authoring process for students to be unfairly penalised if the tests are not properly scrutinised to eliminate bias and irrelevant barriers to comprehension.
Here are four recommendations for developing diverse and inclusive assessments:
1. Adopt stringent review process
Before a paper reaches students, it should go through a process of multiple review cycles.
This is essential in ensuring exam content, and associated mark schemes are fair and inclusive. The more reviewers you can use, the less likely you are to discriminate against students with protected characteristics. However, a manual review process makes this difficult.
The number of reviewers seeing a paper is often limited by admin practicalities, and, unless a review process is setup with strict rules to follow, reviewers could all be looking for the same thing and miss other key issues.
If you want to increase the number of review stages exams go through, as well as improving the organisation of feedback, try adding efficiency to your process by digitising your authoring.
At GradeMaker we have seen the enormous benefits of using a custom workflow system for passing papers between reviewers. You can stipulate exactly what the reviewer must look for with custom QA forms, and ensure reviewers are only accessing the material relevant to them with tight user permission controls.
2. Create multiple papers
Another important consideration is the practicalities of exam day.
Not all students will be able to sit a test at the same time, for example due to hospital appointments or religious prayer times.
Exam scheduling is a complex task but taking these needs into account is vital if your qualifications are to be truly accessible.
One solution for this is to create contingency papers, so you can avoid forcing all your cohort to be available at the same time.
In doing this, you will need to ensure that all the papers are a comparable standard, without increasing the workload of your assessment writers. This is another reason to focus on the efficiency of your review cycle, and ensure you have a flexible system in place for generating and managing multiple papers.
The GradeMaker Pro platform has already helped multiple assessment creators to streamline their authoring processes and create large item banks, which, along with our Automatic Paper Generator, allow, for much faster and more flexible paper creation.
3. Pre-test your questions
Even the most rigorous review cycle can overlook hidden discrimination, which is why ‘pre-testing’ of questions is best practice in high-stakes assessment.
By pre-testing your questions, you can calculate different statistics, for example difficulty, item discrimination and Item-total correlation discrimination.
Question Discrimination
The University of Oxford Medical Sciences Division explains that Item-Total Correlation Discrimination is the correlation between the question score and the assessment score.
Higher positive correlation values indicate that participants who obtain high question scores also high overall assessment scores and those that obtain low question scores also obtain low overall assessment scores.
This is what we want, as it shows that the question is a fair judge of ability in relation to the whole assessment. Using question meta-data such as this, you check that questions are fit for purpose. If you get a negative correlation value it indicates that low scoring participants do well on this question or high scoring students do badly, which could indicate an issue with a question, and you can investigate why this is happening.
Question Decoding
One common reason questions fail this test is when there are issues with decoding. This occurs when students must decipher content within the question which is not relevant to the course.
For example, if you were to use an image of a clock with roman numerals in your question, you are assuming that every student can read Roman numerals.
If they are not a required part of the syllabus, then this question may unfairly trip up students who would otherwise be able to answer quickly and correctly.
It is for this reason that pre-testing is useful as it can help flag up possibly discriminative questions.
4. Choose a flexible delivery format
With universities shifting to a digital approach to assessment, sped up by the COVID-19 pandemic, there is an increasing need to deliver assessments digitally.
However, this can lead to equality issues as not all students will have access to the same technology while at home.
Its highly likely that during this transition, some students with different access requirements or in less well-connected locations will still require a printed assessment. Therefore it’s important that you are prepared to deliver dual output, both into a e-test player and print.
In many institutions this means double the work, re-keying and typesetting assessments ahead of the exam. What’s more, this extra work can introduce errors and typos creeping into the test. That is why a dual output authoring system is vital for any futureproof exam preparation system.
By working more flexibility into your authoring system, you can open many opportunities for improving the quality of your exams. A rigorous exam development process will bolster the diversity and inclusion of you institution by helping to discover and mitigate hidden barriers.
What’s Next?
If you want to ensure your assessments fully support your diversity and inclusion policies, GradeMaker can give you a complete, centralised platform for more efficient development of assessments.
Book an online demonstration to see how digitized authoring can enable you to focus more time and expertise, on the quality of your tests. Meaning every student is presented with an equal opportunity for success.
