Examining the Experts in Assessment, with Graham Hudson

What piece of advice would you give to anyone planning a career in assessment?

1. This may seem a strange thing to say, but think about what sort of person you are. If you like numbers, get satisfaction from analysis and study, then research and psychometrics is a fascinating field. Not only in relation to cognitive assessment, but also behavioural and personality types.

2. If you prefer to be active and like organising, then the challenges of making assessments work well with good processes is also fascinating. There are not so many people in the business who can do this really well – so picking this field provides a lot of opportunity – not just in the UK but overseas as well.

3. Get some good groundwork in, though, with people who know what they are doing. This would generally be with a larger assessment or awarding organisation – small businesses tend to take a more parochial view of the world.

Thinking back over your career, what would you rank as the most important innovation in management of certification programmes?

For me, the lessons learned running national tests for England and making this happen on a large scale, including both programme managing and risk managing. The development of electronic marking also changed the game not only through process management and improvement, but also the ability to control marking quality. This has been a paradigm shift in marking processes leading to better confidence in outcomes.

What would you say was the biggest challenge facing awarding bodies today?

1. I would say keeping up with the changes in the educational environment. Not only meeting changes created by political drivers, but also the use of technology. There are not many smaller awarding bodies that have the breadth and depth of experience to know how to decide on their future pathway, and they will have to recognise that they cannot do this themselves. They will need input from others who have trodden the road. The challenge is to make the changes sooner rather than later, when circumstances force the issue and others have moved before you.

2. In the vocational world, the biggest challenges have been created by the new apprenticeship programme and assessments. It is extremely difficult to keep up with the complexities of operation, let alone to create assessments that are valid.

Do you think that the use of paper in examinations will be phased out or will it always have a role?

1. This depends upon the context. With less time-critical, large-scale assessments, paper will go sooner rather than later. In addition, where the testing framework requires more than assessment of domain skills and content, e-assessments have a vital role to play.

2. For larger-scale examinations, until the testing approach and paradigm changes (i.e. all students sit the same test on the same day at the same time), paper will remain for quite some time to come.

What do you think have been the most useful technical advancements in assessment software?

1. Digitalisation of examination answers – whether images or data input by students. This has created a new paradigm where the same answer can be viewed many times, processed in different ways and marking quality checked.

2. The move to tablet delivery is a big change, especially for countries where national infrastructure cannot be relied on, so you can make use of the mobile phone network, rather than fixed-line or wireless internet.

3. Advancements in the quality of remote invigilation software has meant that more organisations can now offer professional invigilated exams over the web, avoiding the need for in-person invigilators and test centres, which gives a lot of flexibility when running exams worldwide.

What are the most dramatic changes you have seen in the running of exams over the course of your career?

1. The move from processing on paper to digital.

2. The ability to have on-demand validated items that can be created as tests on demand, with parallel levels of difficulty.

3. The use of digital information in research that provides research processes and data sets that could only have been achieved (if at all) at great expense previously.

What’s the most unusual assessment story you have encountered or experienced?

1. The candidate who wrote to the examiner in an ‘S level’ paper, “My Dad says that if I don’t write anything in this paper, you won’t get paid. Is this enough?”

2. The note put into a packet of scripts for A Level Physics taken in Malawi, which said “During the examination, the roof blew off the hall where the candidates were sitting. We gave them 10 minutes extra time ….”

3. A paper boy, who stole a package from the porch of one of his rounds, only to find it was examination paper. A farmer found it three weeks later in a ditch at the side of one of his fields … discarded!

4. An examiner was using an online marking tool and found that it did not work properly. On close questioning, it was found that she was using a keyboard bought in Russia…

Have you had a professional mentor who was especially influential in your career? If yes, what lessons or advice have proven to be most impactful for you?

Several. The most impactful is to believe that you can achieve what you set out to do and not let your own doubts get in the way. Also, do not to let others who create doubts or who are not in line with your goals stop you. If they do, then get rid of them! Find the people who are successful, talk to them, ask their advice, copy their habits.

What social media platforms do you use regularly, and, of these, which one do you find to be the most useful in your professional life?

LinkedIn – the best for me in this professional environment.

What business book or blog have you found most interesting or useful in the past year?

‘What to say when you talk to yourself’, by Shad Helmstetter. A small book, but a powerful one for dealing with self-doubt and improving your self-esteem.

Biography

Graham Hudson is a long-standing member of the educational assessment community, having worked in the sector for 35 years. He has been involved in many of the significant developments in general qualifications, including the introduction of GCSEs, Curriculum 2000 and National Curriculum at Key Stage 4.

Whilst at QCA he was responsible for the marking and collecting data for the Key Stage tests and after a move to DRS Data Services Limited was part of the team responsible for the first implementation for the live marking of UK examinations using scanned scripts. His interest in eAssessment started whilst at QCA where he led a development change programme involving all the regulatory bodies in the UK looking at the uses of new technologies in assessment, having won several £m of funding from the Treasury. He now runs his own consultancy businesses supporting organisations in the assessment sector in the UK and internationally.

His great passion is to harness technology of all kinds to improve the examination experience that candidates face, the reliability of the assessments and marking accuracy.

If you’ve enjoyed this assessment blog post, follow this link to read more insights from eassessment expert Paula Goddard.

For more information on assessment and computer-based exams, download our 5-Step Guide to Online Examinations here.