Bolstad et al assessed the effectiveness of computer-based training of contingency planning and preflight planning on situation awareness and flight skills. They did an experiment with general aviation (GA) pilots (around 2003?) and published it in an article in 20101.
They used GA pilots already holding a private pilot's licence and with experience averaging 270 flight hours2. These pilots received theoretical computer training on contigency planning and preflight planning (the latter also including practical exercises).
The performance of the pilots on a 7-item flight plan knowledge questionnaire, as well as their performance on a flight simulator, were compared before and after receiving the training. A different group of pilots was used as control group2. Some 25 (independent) variables were measured, with training acting as the research (dependent) variable of interest. The main research hypothesis was that training would improve the specific skills being trained as well as overall situation awareness (measured with SAGAT3) and flight skills (measured in the flight simulator).
The research results are summarized in table 1. Training seems to improve (significantly) some preflight knowledge. It also appears to improve (significantly) some aspects of situation awareness.
However, these results need to be interpreted with care. On the one hand, only 2 out of 7 knowledge items and 3 out of 15 situation awareness variables were significant, which is not a great deal of improvement if basic skills and situation awareness were, indeed, improving. On the other hand, the researchers opted for a one-tailed level of significance with a probability of 0.05, and carried some 25 t-tests. We could expect at least one of those tests to be significant by chance alone if a probability of 0.05 was used. Yet, the evidence suggests that the researchers were, indeed, using a probability of 0.10 (or two-tailed tests with a probability of 0.05 for each tail), in which case we could expect up to two of those tests to be significant by chance alone. (The table shows results which are coherent with one-tailed tests and a significance level of 0.05).
Given that situation awareness has not been trained directly, the improvement in awareness may be simply incorrect, that is, a chance result. In this case, is safer to assume that the training of contingency planning and preflight planning seems to partially improve preflight planning but not actually much. Furthermore, it does not seem to have a clear effect neither on situation awareness nor on flight skills performance (in a simulator).
|Effect of training on flight plan knowledge questionnaire|
|Undisclosed item (3/7)||no|
|Undisclosed item (4/7)||no|
|Undisclosed item (5/7)||no|
|Undisclosed item (6/7)||no|
|Undisclosed item (7/7)||no|
|Effect of training on flight skills and SA|
|Flight skills performance (overall)||no|
|SAGAT: awareness of current heading||no|
|SAGAT: awareness of current altitude||no|
|SAGAT: awareness of current vs planned altitude||no|
|SAGAT: awareness of current airspeed||no|
|SAGAT: awareness of current vs planned heading||no|
|SAGAT: awareness of current winds||no|
|SAGAT: awareness of aircraft level||no|
|SAGAT: awareness of current airspace||no|
|SAGAT: awareness of current course||yes|
|SAGAT: awareness of special use airspace||yes|
|SAGAT: awareness of current distance to destination||no|
|SAGAT: awareness of current nearest airport||no|
|SAGAT: awareness of current altimeter setting||no|
|SAGAT: awareness of current system degrades||no4|
|SAGAT: awareness of current ATC organization||no|
Want to know more?
- BOLSTAD Cheryl A, Mica R ENDSLEY, Anthony M COSTELLO & Cass HOWELL (2010). Evaluation of computer-based situation awareness training for general aviation pilots. The International Journal of Aviation Psychology, 2010, number 20, volume 3, pages 269-294. ISSN 1050-8414.
- This is the original article, with above research described as experiment 2. (People with online access to the publisher can find the original article here.)
- AviationKnowledge - Situational awareness
- You can find information on situation awareness in this page.
- AviationKnowledge - Effectiveness of computer-based training on SA
- There are two other experiments releated to the one described here: the effectiveness of computer-based attention sharing training on SA and the effectiveness of computer-based basic skills training on SA.
Jose D PEREZGONZALEZ (2011). School of Aviation, Massey University, New Zealand (JDPerezgonzalez).