Quality Considerations with Remote Proctoring Software

If you are an organisation running secure examinations for professional candidates for certification or recruitment, you will most likely have heard of remote proctoring software. This software became an essential tool for all awarding organisations during the Covid pandemic, to enable high-stakes examinations to be delivered online, but still under secure conditions. Meeting up in large groups at test centres became impossible, and while some examinations could be rescheduled, many more could not be postponed for what was an unknown amount of time. Think of a medical doctor who has to be certified before being allowed to work in a key role in accident and emergency? It’s not an option for that doctor to wait a couple of years until test centres open again! Remote proctoring software has transitioned from an interesting tool that provides flexibility to some exam candidates, to an essential part of any awarding organisation’s software options.

What is Remote Proctoring Software?

Remote proctoring is software that allows an examination to be delivered online to a candidate at any location while still being supervised by an invigilator or proctor. You can think of it as recreating the exam hall experience online but without the travel. If you are based in the UK or Ireland, you may know the term ‘invigilator’ as the person who supervises an exam while in progress, but in most other countries this role is called ‘proctor’. With remote proctoring, the candidate can sit their exam on a computer at their home or office, and the invigilator/proctor will supervise the exam using various communications technologies such as video, audio and remote screenshare, to ensure that the exam is conducted with integrity and there is no attempt to cheat.

Benefits of Remote Proctoring Software

Using remote proctoring removes all the logistical issues around the provision of exam halls and in-person proctors. With remote proctoring the candidate can take their exam in any location, which provides a huge degree of flexibility to both the awarding organisation and the candidate. The candidates’ stress levels are also reduced, as they have not had to travel and can take the exam in a familiar environment. Organisations can use this exam delivery method to increase competitive advantage by offering flexibility to busy candidates, and they can also more easily scale their exam offering over a larger geographic area.

Live Remote Proctoring for High-Stakes Exams

There are various remote proctoring options, which are suited to different types of exams. For high-stakes certification exams, being delivered to professional candidates, the best method to use is live, remote proctoring. With this option, candidates are connected to a supervisor over the internet, who will closely monitor the candidate via video, audio and remote screen share. Within most good systems, everything is recorded and, if required, can be reviewed at a later time by the examining body. Once connected, the supervisor will run through a number of steps to authenticate the candidate and secure the environment. This may involve checking official photo ID or panning their webcam around the room. The steps involved will depend on the specific rules of the exam. The supervisor will then provide access to the exam, following which they continually watch over the candidate for the duration of the test. Most providers will have a maximum ratio of candidates to supervisors, which is usually significantly lower than most test centre conditions.

Advantages of Live Remote Proctoring

An alternative delivery option to the live approach is ‘record and review’ where the exam session is recorded for review at a later time. Sometimes detection technologies are used to flag noises or certain movements outside of calibrated levels, which can then be reviewed after the exam. At this point, live invigilation is considered to offer the best level of security and there are two main reasons for this. Firstly, detecting noise or flagging movement is useful, but is unlikely to pick up subtle eye movements or small alterations in behaviour. When it comes to the processing of moving images, human beings are significantly ahead of computers and will spot suspicious activity, far beyond anything that computers can detect. Secondly, with live invigilation, any suspicious behaviour can be fully investigated by the supervisor and action taken, for example asking a candidate for an explanation and stopping them transcribing questions. With a recorded video, it’s not possible to ask the candidate what they are looking at, or do a scan of the room, and often the results of reviewing a flagged video can be inconclusive. Trained invigilators can also reassure and provide direct support to candidates, for example if they have questions about how to navigate the exam. For these reasons, “record and review” is better suited to lower stakes examinations.

What are the Quality Factors to Consider with Live Remote Proctoring?

So if you choose live proctoring over ‘record and review’, there are still a number factors to take into account.

A Single Exam Proctoring Solution

In most solutions, remote proctoring is provided as one system that ‘bolts on’ to a separate assessment application. Typically the systems are from two different vendors. This is not ideal as there are various issues with using one system for the exam but a separate system for the proctoring. For example:

• Exam candidates are under a lot of pressure at exam time and often they have many questions, some of which may relate to the assessment application, e.g. are my answers being saved? Is the notepad submitted as part of the exam? etc. If the proctors have no underlying knowledge of the assessment application (because they work for a different company), they cannot provide a knowledgeable and supportive service. This can be very frustrating for candidates.
• With a bolt-on proctoring solution, there may be security concerns, where proctors have to take control of the candidates screen to enter a password in the assessment system, or alternatively passwords are given out by the proctor, which may result in them being compromised. This can potentially risk candidates accessing the exam without being remotely proctored. If you use an all-in-one live proctoring and assessment solution, there is no way for a candidate to access the exam paper without going through the validation process with the supervisor.
• With two separate systems there is an integration point, and with any integration there is always the risk of ending up with inconsistent data. For example, 1,000 candidates took the exam on the assessment system, but only 997 were listed in the proctoring application. You then have the problem of determining what actually happened and how to deal with it.
• With separate systems, the exam will typically be running in a browser to enable the remote proctor to view the candidates’ screen. Running an exam in a browser is not ideal, and does not offer a stable environment in which to take a high-stakes exam. It is far too easy for a nervous candidate to close down a browser window and lose exam content, or they can potentially navigate to other browser windows or applications. For an all-in-one solution, a secure application can be used, which removes the risk of candidates accidentally closing a browser, or navigating to a different website. Plus it allows secure computer lockdown on, which prevents the candidate opening other applications or accessing their file system.

Who is Providing the Proctors?

Proctors are a critical part of the process, as they are dealing directly with your candidates and are representing your brand. It’s important to understand key factors such as who they work for, where they are based and how well they are trained and monitored to ensure quality. At TestReach, our team of proctors are employees of the company and fully trained and quality monitored in line with detailed company policies and operational procedures. This means they can give a knowledgeable, friendly and professional service to exam candidates.

Comprehensive Support and Incident Management

You always need to make sure that the supplier has a well-trained support team and associated processes and systems in place, to help to exam candidates if they run into problems. During the pandemic, there have been many problems reported of supplier support desks not being responsive, resulting in remote exam candidates phoning the examining body for help. Does your potential supplier have a proven track record of providing great candidate support and do they have references who will attest to this? It is also important to say that much as suppliers can mitigate risks associated with their own software, they do not have control over all parts of the service, and events such as full internet outages, although infrequent, have to be properly managed. Incident management is a critical component in any service offering, where a supplier should be able to demonstrate robust processes and procedures to deal with unplanned issues, which may be at scale. This is really important to fully explore before you make any decision on who to work with.

We hope that some of the points above will help you in considering the quality factors to keep in mind when evaluating remote proctoring systems. There is now an extensive number of suppliers in this area, who offer very different ways of delivering an exam at varying degrees of quality and integrity. At TestReach, we offer a complete application for creating and delivering exams, and live proctoring is an integral option within the same system. We have a large team of trained proctors who are based in Ireland, and an award-winning helpdesk and support team. If you’re interested in running secure and professional remotely proctored exams, backed by quality support and services offered by a very experienced provider, we’d love to hear from you.