Examining the Experts in Assessment for Awarding Bodies, with Denise Edens

Denise Edens is an expert in assessment and qualification design, who has supported a number of awarding bodies and training centres in delivering assessment and in developing new qualifications.

Our blog series “Examining the Experts” asks leaders and experts in assessment – practitioners and consultants, authors and solution providers – to share their experience and knowledge on the world of assessment and the impact of new innovations and exam technology.

What piece of advice would you give to anyone planning a career in assessment?

I would advise that you work for a training organisation first. This helps you to understand the importance and impact of qualifications on learners and centres. It helps you to develop qualifications and make assessment decisions based on understanding how real candidates think in the pressurised environment of an exam. It reminds you that a candidate usually only takes your qualification once, so it is a big deal for them as it can make or break their transition to the next stage in their education or career.

So if you write examination questions and you’ve been through the assessment cycle a few times and are feeling a bit bored and think you might to do something slightly off-centre to liven things up in this year’s exam, having the experience in a centre helps you see it from the candidate’s perspective of the assessment being a one-off life-changing opportunity for them, so you keep the questions comparable to previous ones, for fairness to candidates.

Thinking back over your career, what would you rank as the most important innovation in running qualifications at an awarding organisation?

Undoubtedly the move from paper-based assessment to online has been the most important innovation. It can make marking more objective, as examiners might only mark a few questions across a massive cohort, rather than whole examination scripts, so they can’t use a candidate’s performance on a previous question in the paper to influence their decisions on a current question.

Where scripts are chunked up and one question from all candidates is beamed down to a single examiner, it can be fairer for candidates as there is no opportunity for discrepancies in marking by a huge range of different examiners to creep in.

Also, marking online reduces the need to send parcels across the country to examiners, with the risk of them going missing in the post or being left on doorsteps.

What would you say was the biggest challenge facing awarding bodies today?

I think managing malpractice in centres is challenging as there is tension between an awarding organisation and its centres, who on the one hand are its customers, and on the other hand must at all times adhere to quality practices. Malpractice can be particularly difficult to manage in overseas centres as it is more costly to visit them and some countries face difficulties with a culture of not quite playing by the rules. I have experienced an overseas candidate trying to bribe a centre in the UK to give him his certificate at the time of registering for the course, by slipping some money into his registration form. Thankfully the centre rejected it, and forced him to play fair and turn up to class.

Do you think that the use of paper in examinations will be phased out or will it always have a role?

I hope that examinations will become mainly online, because an online examination can be more interactive than pen and paper, and questions can be in colour which can be helpful. We all spend more time on keyboards than writing with a pen these days, so it is more realistic for candidates to sit assessments onscreen. It also enables them to edit and hone their answers more easily, and to do word counts, as in real life.

However, certain candidates, particularly those with vision or motor skills challenges, may not be suited to onscreen assessment, so paper versions will always need to be additionally available. Also, in situations where the use of IT would be a barrier to the assessment of the knowledge or understanding being assessed, we should use paper and pen.

What do you think have been the most useful technical advancements in assessment software?

Online marking systems make it much easier for a Principal Examiner to moderate the marking being done by the team of examiners, due to having a super-user functionality which enables the PE to look into each examiner’s allocation of marking remotely, to check progress and help with any marking issues. This helps ensure accurate marking and standardisation across all the examiners. In the past the PE was reliant on the examiners sending over a batch of sample scripts every so often, which did not give the PE as much information as they can now access. Also marking online helps the awarding organisation to see how the marking across the whole team of examiners is progressing, enabling the awarding organisation to monitor whether sufficient scripts will be marked before the awarding deadline and to reallocate scripts when needed.

Should we be running assessments differently? What would you change if you could?

I think awarding organisations already offer a robust service. If I could change one thing it would be to decouple level 1 from level 2 at GCSE. GCSE awarders use candidate performance statistics to set grade boundaries using a normative approach, so a certain percentage of the cohort is by definition destined to attain lower than average grades. What’s odd about these school leaving qualifications in our country is that the lower grades are still technically a pass, but students and the public tend to perceive the lower grades as failure. Students may have actually performed well at level 1 but this is only ever reflected as lower grades in the GCSE. I would like to see candidates given a positive reward for having achieved level 1, before going on to achieve at level 2. This would prevent unnecessary humiliation and negative attitudes to learning due to perceived past failures.

What’s the most unusual assessment story you have encountered or experienced?

By definition, most awarding organisation work is careful, rigorous and methodical, so unusual things don’t tend to happen. This makes it sound dull but I actually find it very creative, particularly when developing new qualifications, exam questions and assignments.

My current role as a consultant to a variety of awarding organisations has given me the most unusual in-tray: one day I can be writing an assignment for overseas aid workers to manage volcanic eruptions; the next day I can be flying to Saudi Arabia to help develop national occupational standards, and another day I can be in a meeting to review a canine hydrotherapy qualification.

Have you had a professional mentor who was especially influential in your career? If yes, what lessons or advice have proven to be most impactful for you?

I had a mentor while I was studying for a BTEC in management at work. He was great at giving constructive and honest feedback, which was a helpful approach.

I’ve learnt a lot from working with colleagues who were great at their jobs: things like how to chunk work into manageable tasks; why it helps to explain the importance of tasks when delegating them; why fit-for-purpose is better than perfection; and how to deal calmly with emergencies in the exams season.

What social media platforms do you use regularly, and, of these, which one do you find to be the most useful in your professional life?

I find LinkedIn really useful for keeping in touch with people I have met while working in the education sector. It is also useful for being introduced to new people and having people find me to approach me for consultancy work.

What business book or blog have you found most interesting or useful in the past year?

I am a Consultant Member of the Federation of Awarding Bodies and I find their weekly update extremely useful as it provides a synopsis of the latest news, issues and policies affecting awarding organisations. It’s a clearly written digest, which saves me oodles of time. It also has links to useful webinars and seminars.

Visit Edens Education to learn more about services that Denise provides, such as independent freelance support and project management to awarding organisations including qualification development, assessment, support materials and training.

If you’re interested in online assessment for awarding bodies, take a look at the online assessment case studies here for more insights into how organisations have made the transition.