Keelson Marine Assurance

Based in Houston, Texas, Keelson Marine Assurance is a marine consultancy firm that provides a range of services to the shipping industry, from training support to vessel inspections, risk studies and incident investigations. Keelson are known experts in the field of Dynamic Positioning and their work includes providing expert witness services.

Dynamic Positioning (DP) is a computer-controlled system which is used to control propulsion to maintain a vessel’s position at sea, when mooring or anchoring is not feasible due to deep water or infrastructure on the sea bed. DP is used by much of the offshore oil and wind industries and allows operations such as drilling, dive support and personnel transfer amongst others to take place. It is highly complex, requiring system knowledge and understanding of external factors which affect the ability of a vessel to maintain position and heading.

Keelson identified that a lack of secure knowledge among ship crews often was a contributory factor to position loss incidents that they investigated.

A failure in DP either by equipment fault or operator action / inaction can cause a ship to lose heading or position during operations, which can be hazardous to people and the environment as well as being costly to rectify. Keelson identified, through their own experience and wider industry reporting, that human factors were often significant contributory factors in these unwanted position loss events.

With this insight, it is beneficial to find a way to accurately measure the level of knowledge among crews and identify any gaps that could be subsequently addressed. This was a challenge because ship crews are by their nature highly mobile, busy people. Keelson set out to provide a solution which could meet the international challenge to promote safe and predictable DP outcomes by enabling the necessary professional development and improvement for all critical DP staff offshore.

Keelson designed a CPD programme based around repeated assessments. Ship crews take a secure online test, usually onboard, and receive a report summarising performance mapped against key subject areas, with guidance on areas to improve and where to find that knowledge. A second test is taken, as routine, after a period of time, usually around eight weeks, to show if and where knowledge has improved. On successful completion of the second test, Keelson issues a certificate proving engagement and assuring a level of DP expertise. Assessments are designed for different ship roles and learning pathways, and include engineers, who had in the past been overlooked in the area of DP.

This required a technology solution that would:

• enable ship crews to take a secure assessment anywhere in the world at any time;
• provide a question bank with detailed tagging and taxonomies to enable automated creation of exam papers, where papers would include the relevant subject areas, at different difficulty levels and match the learning pathways for the various ship roles involved;
• generate randomised examination papers so that question content was protected while still examining the right subject areas at the right level;
• enable subject matter experts to add questions to the bank with ease;
• allow multiple re-sits of the exam while still randomising the questions in each exam paper;
• enable an assessment window to be as long or short as required, to fit in with shipping schedules;
• provide a detailed reporting tool to pull results data by topic area, by individual, by role, by vessel and by groups of vessels, so that gaps in knowledge could be identified and compared in detail.

Kate Gillespie, Senior Consultant at Keelson Marine, said:“We knew what we wanted to do from an education perspective, but we needed to find a platform that would allow us to do it. TestReach was unique in meeting all our requirements and allowed us to implement our vision. These tools are not available anywhere else.”

“Every question has been created with a correct answer from a referenceable, published document. With TestReach we can tag each question and answer at a granular level with parent and sub-parent tags. We can create question pools according to different topics, difficulty levels, learning pathways and roles, for both the initial and second assessment. Our clients can include their own bespoke compliance questions if wished, and this is very popular. The system pulls the questions to create randomised exam papers for each learning pathway. Even though we are using MCQ questions, the top level of Bloom’s taxonomy is possible.”

“I can then set an assessment window of any length according to the client’s needs, and post-assessment the tagging feeds into the reports with results divided by question pool, so we can see which areas are performing well or poorly. We can report knowledge data by role, vessel or company. The report is very focused and we can quickly point out to clients where the issues are.”

Keelson has applied this programme across multiple marine organisations and the results have been phenomenal. Their client companies were highly impressed with the level of detail given on their crews’ knowledge. While often being shocked to see so many gaps, it highlighted which crews were most at risk in their operations and where they could direct training and support to resolve the issues.

As a result of this success, Keelson has seen an exponential rise (200% in 2022) in the use of its CPD programme. It is now an essential resource for many shipping companies based in Australia, Europe and the US Gulf.

Mat Bateman, CEO at Keelson Marine, explains:“This is effectively about managing risk. There is evidence that human factors are causing dynamic positioning events or failures, and the impact of the programme is that that marine companies get really detailed feedback on the gaps in knowledge and where they need to focus - the power is in the data that comes through. A ship’s crew has to show improvement over time and it has to be measurable. It’s a professional programme, where the aim is to keep the learner engaged and empower them to be responsible for their own learning.”

“What we also like about TestReach, apart from the software itself, is the guidance we receive. If we ever have an issue or a question, the team are always so responsive and a pleasure to work with. In addition, TestReach continues to develop and add new features, so we are now creating bespoke business intelligence dashboards that clients define and utilise themselves in order to drill even further into the data points, inform training and onboard safety drills and get KPI data for upward reporting.”

The programme has armed the marine industry with focused and relevant data points. While the initial set-up looked at top level results by individual and rank, over time Keelson was able to analyse knowledge with regard to one specific vessel, then a fleet of vessels and so on, until they gained insight into overall performance industry-wide. This enables Keelson to inform the wider industry and they are actively sharing these findings at relevant industry forums and with local marine forums and groups. Some examples of the type of data (anonymised) can be seen in Figures 1, 2 and 3, which show the knowledge growth between assessment 1 and 2, the differences between vessels, and knowledge mapping of engineers and deck crew over the two assessments.

Figure 1: Knowledge Curve Moving from Assessment 1 to Assessment 2

There have been other insights from the data, some of which have been surprising. Non-native English speakers are higher performers in the exams than native English speakers, perhaps because they read the question distractors in the MCQs more closely. It became apparent that people in senior roles were sometimes not as well informed as they believed they were, which is particularly problematic as it affects the performance of the entire vessel. However, as Kate Gillespie points out,“this is not a gotcha moment, trying to catch people out. It is true assessment for learning, it is designed to help. Some people are over confident but don’t realise what they don’t know, and the initial assessment helps get engagement and assists individuals to understand that they have areas where they really need to learn. This is a high-risk area in the marine world, and it’s not about pass rates, it’s about getting engagement and moving the knowledge dials in the right direction.”

Figure 2: Knowledge Results from Different Vessels

Figure 3: Knowledge Mapping for Deck vs. Engineering Crew over two Assessments

View downloadable copy of this case study.