Bolstad et al assessed the effectiveness of computer-based basic skills training on both situation awareness and flight skills performance. They did an experiment with a group of ab-initio students pilots (around 2003?) and published their results in an article in 20101.
They used students2 already holding a private pilot licence but with experience of lesser than 150 flight hours. These students received computer training on three things:
- Checklist completion - a theoretical training of the importance of completing checklists fully, and techniques on how to do so under different flight conditions
- ATC comprehension - a practical training of ATC speech comprehension, advancing from comprehension at normal speech rates to fast speach rates
- Basic psychomotor skills - a practical training of hand-eye coordination, advancing from low difficulty levels to higher difficulty levels of hand-eye coordination
The performance of the students on above tasks, in particular, as well as their general performance on a flight simulator (which assessed changes in situation awareness and flight skills), were compared before and after receiving the training. A different group of students was used as a control group2. A number of some 36 (independent) variables were measured, with training acting as the research (dependent) variable of interest.
The main research hypothesis was that training would improve the specific skills being trained as well as overall situation awareness (measured with SAGAT3) and flight skills (measured in the flight simulator).
The research results are summarized in table 1. Training seems to improve (significantly) some of the particular skills being trained. It also appears to improve (significantly) some aspects of situation awareness.
However, these results need to be interpreted with care. On the one hand, only 2 out of 15 situation awareness variables were significant, which is not a great deal of improvement if situation awareness were, indeed, improving. On the other hand, the researchers opted for a one-tailed level of significance, and carried some 36 t-tests. We could expect almost two of those tests to be significant by chance alone if a probability of 0.05 was used. (We could further expect three of those tests to be significant by chance alone if a probability of 0.10 was used, and there is suspicions that the researchers may have been using the latter probability instead4. Given that situation awareness has not been trained directly, the improvement in such awareness may be simply incorrect, that is, a chance result. In this case, is safer to assume that the training of particular skills seems to partially improve those skills, but does not seem to have a clear effect neither on situation awareness nor on flight skills performance (in a simulator).
|Effect of training on particular skills|
|Checklist completion (knowledge)||yes|
|Reaction time to ATC communications||no|
|Correct responses to ATC||no|
|Psychomotor skills difficulty||yes|
|Reduction of psychomotor errors||no|
|Effect of training on flight skills and SA|
|Flight skills performance (overall)||no|
|SAGAT: awareness of current heading||no|
|SAGAT: awareness of current altitude||yes|
|SAGAT: awareness of current vs planned altitude||no|
|SAGAT: awareness of current airspeed||no|
|SAGAT: awareness of current attitude||no|
|SAGAT: awareness of current winds||yes|
|SAGAT: awareness of current clearance||no|
|SAGAT: awareness of current vs planned time||no|
|SAGAT: awareness of current fuel||no|
|SAGAT: awareness of special airspace||no|
|SAGAT: awareness of current obstacles||no|
|SAGAT: awareness of conflict with restricted airspace||no|
|SAGAT: awareness of current altimeter setting||no|
|SAGAT: awareness of current vs planned groundspeed||no|
|SAGAT: awareness of current ATC organization||no|
Want to know more?
- BOLSTAD Cheryl A, Mica R ENDSLEY, Anthony M COSTELLO & Cass HOWELL (2010). Evaluation of computer-based situation awareness training for general aviation pilots. The International Journal of Aviation Psychology, 2010, number 20, volume 3, pages 269-294. ISSN 1050-8414.
- This is the original article, with above research described as experiment 1. (People with online access to the publisher can find the original article here.)
- AviationKnowledge - Situational awareness
- You can find information on situation awareness in this page.
- AviationKnowledge - Effectiveness of computer-based training on SA
- There are two other experiments releated to the one described here: the effectiveness of computer-based attention sharing training on SA and the effectiveness of computer-based preflight planning training on SA.
Jose D PEREZGONZALEZ (2011). School of Aviation, Massey University, New Zealand (JDPerezgonzalez).