Monica Sen Gupta, Chris Vance, Tim Hughes, David Leahy, Ryan Meeks, BAE Systems, MBDA and Frazer Nash
The Ops Room of a RN warship is a complex system which requires a vast array of command and control, communications and information systems and a crew of operators to operate together efficiently, effectively and safely. Numerous Human Factors studies have looked into the Ops Room design from consideration of information flows associated with defence operations to spatial arrangements. This study commissioned by the Defence Systems Technology Laboratory (Dstl) required that we examine what the integrated air defence element spend their time doing, if tasks are being repeated and the timeliness of the availability of information. The study was to identify where information flow pinch points/deficiencies occur, identify the causes and consequences of any such occurrences and provide recommendations on how to address the identified issues. Our approach was to take a combination of Human Factors methods to answer the question. All approaches had the same aim (to determine deficiencies and opportunities for improvement) but use different methods to answer the question.
The first method employed was modelling and simulation. A task network model was developed based upon a thorough task analysis of Ops Room activity. A discrete event simulation tool (IPMETM) was then used to run simulations of the Ops Room activity model with varying levels scenario demand. This enabled the prediction of operator workload and identification of process bottlenecks in the Ops Room activity.
To complement the objective modelling and simulation approach three qualitative methods were also employed.
Critical Decision Method (CDM) and Cognitive Probes The first of these employed the Critical Decision Method (CDM) as an interview framework to examine the critical decision points identified in the task analysis with RN Subject Matter Experts (SMEs).
Knowledge Elicitation of Rich ‘lived’ Experiences. The second method used CDM and knowledge audit retrospective interviewing techniques to tap into the tacit knowledge and rich ‘lived’ experiences of personnel with recent operational experience.
Post Exercise Review Analysis. Finally, the third method used a mix of observations of several air defence training events, analysis of recorded audio from training events and semi structured interview techniques with training personnel to explore HF issues and deficiencies in the Ops Room environment. Techniques used during the semi structured interviews included card sorting tasks.
Each of the applied methods produced a wealth of data and demonstrated the complexity of collecting data of this nature from a complex system. Each technique captured the main human factors issues such as planning and coordination and situational awareness, etc. which is correlated. However each method identified some unique lower level findings as well, demonstrating the variability associated with subjective data collection. This study indicates that to effectively capture complex system issues use of different methods is beneficial.