UNCLASSIFIED/,'Ten OFFICIAL UGE OHL¥ 15 Decem ber 20 10 ICO D: 8 S eptem ber 20 10 DIA-0 8-110 1-0 0 1 Defense Intelligence Reference Document Defense Futures Cognitive Limits on Simultaneous Control of Multiple Unmanned Spacecraft UNCLASSI FIED // FOR OTriCIAL USE ONW UNCLASSIFIED/ /rOR OFFICIAL USE ONLY Cognitive Limits on Simultaneous Control of Multiple Unmanned Spacecraft The Defense Intelligence Reference Document provides non-substantive but authoritative reference inform ation related to intelligence topics or m ethodologies. Prepared by: Technology Warning Division (DWO-4) Defense Warning Office Directorate for Analysis Defense Intelligence Agency Author: A A P P erson 73 C O PYR IGHT W AR NING : Further dissem ination of the photographs in this publication is not authorized. This product is one of a series of advanced technology reports produced in FY 20 10 under the Defense Intelligence Agency, Defense W arning O ffice's Advanced Aerospace W eapons S ystem Applications (AAW S A) Program , C om m ents or questions pertaining to this docum ent should be addressed to Person 1 AAW S A Program M anager, Defense Intelligence Agency, Al IN: JU1AF - Dl/DW O -3 , Bldg 60 0 0 , W ashington D.C . 20 3 40 -5 10 0 UNCLASSIFIED//EQR QKIMAh USE QNh¥ UNCLASSIFIED / /FOR OFFICIAL USE ONLY Contents Summary....................................................................................................................iv Chapter 1: Introduction...................................................................................... 1 Chapter 2: Measurement of Mental Workload..........................................................3 Subjective Measurements........................ 3 Performance Measures........................................................... 4 Physiological Measures........................................................ 4 Cardiac Function............... 5 CNS Measurements............................................................................................6 Ocular Measurements....................................................................... 6 Skin Measurements................................................ 7 Serum Levels of Hormones....................................................................... 7 Chapter 3: Studies in Cognitive Workload for Air Traffic Controllers......................8 Modeling the Air Traffic Control Task...................................... 15 Chapter 4: Studies in Command of Multiple Semi-Automated Vehicles................18 Chapter 5: Discussion......................................................................................... 22 Chapter 6: Conclusions............................................................................................23 References................................................... 24 Figures Figure 1. One-dimensional Representation of Changes in Performance as Workload Varies..........................................................................................................................2 Figure 2. Typical EKG Signal for a Normal Heartbeat................................................5 Figure 3. Air Traffic Control...................................... 10 Figure 4. Representation of Performance Results from Brookings Study...............13 Figure 5. Information Processing Model for a Human Operator..............................16 Figure 6. Examples of Unmanned Military Vehicles............... 19 Tables Table 1. Variables used in Determining Complexity of the Traffic.....................11 Table 2. Correlations among Physiological Variables.........................................14 iii UNCLASSI FI ED / /FOA OFFICIAL USE ONLY UNCLASSIFIED// FOR OFFICIAL USE ONLY Cognitive Limits on Simultaneous Control of Multiple Unmanned Spacecraft Summary S pace exploration 40 years into the future m ay include m anned m issions to parts of the outer solar system . A possible scenario m ay include sending a sm all fleet of craft w ith different prim ary m issions. For exam ple, a trailing spacecraft of nuclear pow ered electrom agnets designed to shield the m anned part of the fleet from solar radiation; halo spacecraft w ith pow erful radars to scout for incom ing objects; exploration and m ining craft, etc. The fleet could regularly travel out of unaided visual range of each other, joining up w hen necessary for m aintenance, exchange of m aterials such as fuel, or other necessities. Piloting these m ultiple craft could be econom ically accom plished if only one rem ote pilot on station at a tim e w as necessary. The cognitive lim itation of a hum an astronaut and his ability to perform the m ultiple­ vehicle piloting task is the focus of this paper as little w ork has been done in this specific area. How ever, a large body of cognitive research on the lim itations of object supervision and tracking for the task of air traffic control (ATC ) exists. Additionally, there is an em erging body of research concerned w ith m ultiple unm anned vehicle piloting for heterogeneous m issions. These are the tw o areas review ed in detail as they relate to possible spacecraft m issions. Pilots develop an internal m ental representation of the identity, position, m ission, and current direction of relevant objects. This is referred to colloquially as "the big picture." The prim ary research question w e seek to answ er is w hether there is a cognitive lim it to the num ber of objects that can be m onitored and tracked w ithin the big picture. S econdarily, w e seek to find w hether this m axim um num ber is lim ited by the com plexity of interaction; how those lim iting factors are described, w hether there is a real-tim e objective m easure that indicates w hen a pilot is approaching his m axim um capacity, and w hether that capacity has been exceeded. The m axim um num ber of tracked objects is highly dependent on the com plexity of the piloting and m ission tasks at hand. R esearch is lacking in the area of cognitive lim its on the num ber of spacecraft one pilot could control given any m ission scenario. C urrently, tw o m odels are being used to exam ine sim ilar activities in air traffic control and rem ote piloting of m ultiple unm anned vehicles. In both areas it has been show n the cognitive lim its on the num ber of craft capable of sim ultaneous control is 16 for sim ple destination selection, 7 for m oderately com plex piloting and/or m ission task com pletion, and 4 for com plex heterogeneous craft. W hile additional future research m ay help to increase the autom ation com ponent of aircraft and m ission control, no current evidence exists to show that a com plete m ental picture can be m aintained for m ore than about 16 objects at one tim e, even w ith external w orking m em ory augm entation. How ever, it has also been dem onstrated that physiological variables can be objectively em ployed to indicate overload. Nom inal success has been achieved in classifying physiological states near high w orkload thus enabling both prediction and possibly prevention of overload. iv UNCLASSI FIED/ /FOR OFFICIAL UDE ONLY UNCLASSIFIED// reR OFFICIAL USE ONLY Chapter 1: Introduction Due to the com plexity, duration and num erous support requirem ents of future m anned deep-space m issions involving exploration, m ineral exploitation, and possible colonization, a likely scenario w ill be the inclusion of unm anned fleets of support craft. C oupled w ith other requirem ents, an intensive research program is needed to investigate the cognitive lim its on pilots and other operators responsible for the sim ultaneous control of m ultiple unm anned spacecraft m aking up the support fleet; a "fleet' approach is proposed in an effort to optim ize safety and exploratory reach. This research effort w ould also aim at m axim izing the functional efficiency of the m ission and reducing the operation costs of unm anned vehicle fleets. In this scenario there is m uch about the ancillary craft that are autom ated in both navigation and m ission. M any of them w ill not require full-tim e piloting but given that they could be hundreds of m iles from each other at any instant of tim e, they need m onitoring to prevent unseen system failure or collision from letting them just disappear one day during the m ission like a M artian probe. Accom plishing this m onitoring task and the occasional piloting task for m ultiple craft in the fleet could be econom ically accom plished if only one rem ote pilot on station at a tim e w as necessary. W e w ill focus here on the cognitive lim itation of a hum an astronaut to perform the m ultiple-vehicle piloting task. It is not a surprise that there is little w ork in this specific area - in fact there w ere zero peer-review ed articles in the m ajor journals concerning rem ote piloting of m ultiple spacecraft (published in the last 3 0 years). There is how ever, a large body of cognitive research on the lim itations of object supervision and tracking for the task of air traffic control (ATC ). There is additionally an em erging body of research concerned w ith m ultiple unm anned vehicle piloting for heterogeneous m issions. These are the tw o areas review ed in detail as they relate to possible spacecraft m issions. W hen a pilot or ATC operator is in control of several craft they have developed an internal m ental representation of the identity, position, m ission, and current direction of each object tracked. This is referred to colloquially as "the big picture." This m ental representation is also called situational aw areness. Keeping all of the inform ation about all of the objects straight as long as they are in scope is the goal. The prim ary research question w e seek to answ er is w hether there is a cognitive lim it to the num ber of m oving objects that can be m aintained in the big picture. S econdary questions are w hether this m axim um num ber is lim ited by com plexity, how those lim itations m ight be described, w hether there is a real-tim e objective m easure that w ill indicate w hen a pilot is approaching their m axim um capacity, and w hether that capacity has been overloaded. For the current treatise w e w ill consider only traditional hum ans as pilots. C yborg- enhanced astrobots are a topic for another tom e. In discussing cognitive lim itations, it is useful to introduce the concepts of task dem and, m ental w orkload, and a sim plistic m odel of m ultiple resource theory. For a given task, the gross level of neural activity required is a representation of the m ental dem ands of the task. O f course, a com plex task can dem and varied resources such as visual and 1 UNCLASSI FIED //FOR OFFICIAL USE ONLY UNCLASSIFIED//rne nEHGTAI UOC ONW audio processing, and this is w here m ultiple resource theory enters: different resources can be considered independent if dem ands on one resource do not tax the availability of capacity of the other. Perform ance of a task can be high or low : in general if spare capacity is available, perform ance is high, and if capacity is lim ited or exceeded, perform ance is low . M oreover, task dem and enforces a fixed theoretical relationship betw een m ental w orkload and task perform ance: this is show n in Figure 1. Figure 1. One-dimensional Representation of Changes in Performance as Workload Varies. Task dem and increases to the right. In the three A regions, perform ance rem ains unchanged: A2 is optim al, w here a trained operator exerts m inim al effort to m aintain a set level of perform ance. At tim es of low dem and the operator exerts effort to m aintain vigilance (Al) until dem and is so low , the subject disengages from the task (D). The w orkload curve is not w ell defined at very low levels of dem and (dotted lines). At the other end, as dem and increases, the subject can exert effort to keep up w ith dem and (A3 ). Additional effort m aintains perform ance until degradation begins (B), and in the overload condition, region C , perform ance is degraded beyond acceptable levels. W hile effort is m aintained in overload, som e low level of perform ance exists. 2 UNCLASSI FIED// FOR OFFICIAL USE ONLY UNCLASSIFIED//FOK OFFICIAL USE ONLY Chapter 2: Measurement of Mental Workload There are three approaches to m easuring m ental w orkload in a subject perform ing a prim ary task. The first is subjective evaluation, either post-hoc self-report questionnaires concentrating on how "busy" one m ay have felt, or an experim ental observation of activity. The second is perform ance m easures, an objective evaluation of how w ell the subject com pletes the prim ary task, a secondary task, or an experim entally inserted reference task. The final approach is to record real-tim e physiological m easures, w ith the assum ption that increased w orkload increases anxiety and this w ill be exhibited by changes in the autonom ic nervous system (ANS ). SUBJECTIVE MEASUREMENTS S elf-report m easures are appealing because they get inside the m ind that w as perform ing the task. There is no absolute objective scale to m easure one person's "fully occupied" from another person's view of the sam e state; how ever, through rating scales and self-draw n graphs, one can obtain an accurate picture of how perceived w orkload evolved during the experim ent. Perceived w orkload is im portant because it is w hat needs to be m aintained betw een a subjective m inim um , w here attention m ay w ander, and a subjective m axim um , w here increased em otion m ay decrease perform ance capacity. The m ost frequently used standardized self-report tools are the NAS A Task Load Index (NAS A-TLX or just TLX) and the S ubjective W orkload Assessm ent Technique (S W AT).1-2'3 -4 The TLX is a subjective w orkload assessm ent based on a m ulti­ dim ensional rating questionnaire. An overall w orkload score is derived based on a w eighted average of ratings on six subscales: m ental dem ands, physical dem ands, tem poral dem ands, ow n perform ance, effort, and frustration. S W AT is a tw o-step assessm ent of three w orkload factors: tim e load, m ental effort load, and psychological stress load. In the first step, hypothetical activities are ranked according to perceived w orkload. In the second step, the experim ental task is evaluated post-hoc, using a 1-3 rating scale for each of the three dim ensions. An interval scale of w orkload is derived, from 0 -10 0 , based on the reference data collected for each subject in the first step, and the evaluation of the experim ental task. A custom self-report can also be designed by the experim enter to specifically focus on the research questions in a given experim ent. The second type of subjective m easure is evaluation by an expert observer. In this type of m easurem ent, assum ptions are m ade on the m ental activity of the subject based on the activities being perform ed. The advantage of this approach is that there is m inim al variance per-to-person if the sam e evaluator is em ployed. The disadvantage of this approach is the outside observer can m iss w orkload m ultiplying factors such as task com plexity contributing to an overw helm ed feeling by the subject. An exam ple of this approach is the concept of utilization. In calculating utilization it is assum ed that the subject m ust cognitively address one issue at a tim e in serial order. The m easure of utilization is percent tim e busy, or addressing any issue, as opposed to w aiting or m onitoring for the next event. In general is it observed for control and supervisory tasks that at around 70 % utilization perform ance begins to degrade.5 Arguably not a perfect m easure of w orkload, utilization has the advantage of sim plicity, objectivity, 3 UNCLASSI FIED / /EQR OFFICIAL USE ONCT UNCLASSIFIED/ / FOP ngHCTAL USE OH LT and a quantitative scale, allow ing it to be used in threshold detection and prediction, as w ell as in com puter-assisted w orkload balancing. PERFORMANCE MEASURES Laboratory, reaction tim e to an event stim ulus is a typical real-tim e m easure of speed. O utside the laboratory, in a m ore natural environm ent, speed is an indirect m easure of a subject's ability to keep up w ith a given rate of events.3 Accuracy is m easured in both the laboratory and naturalistic environm ents: for exam ple, in an air-traffic control task, properly handing off a plane to the next controller, as it leaves the first controller's m onitored airspace13 , is m easured as a successful task com pletion. Perform ance, w hen utilizing a secondary task, is accom plished follow ing tw o paradigm s. First, in the dual-task paradigm , perform ance on the secondary task is required and prim ary task perform ance is thus an indication of w orkload. For the second paradigm , instruction is given to m aintain the prim ary task perform ance, and perform ance on the secondary task is thus a m easure of "space capacity" for additional w orkload. C are is required in selection of secondary tasks in order to ensure they affect the resources one w ishes to probe. For exam ple, in a driving scenario, a m inim ally intrusive task is to push a button on the floor or steering w heel w hen a light flashes in the field-of-view of the driver. This visual "detect-and-respond" task adds to w hat is prim arily a com plex visual and m otor coordination task that probes the capacity of visual attention.6 This is different than a secondary task utilizing different resources, such as having a conversation w hile driving. W hen planning the paradigm , a m odel needs to be developed that treats (or explicitly ignores) individual tasks and interactions am ong them . R eference tasks are executed before and after prim ary tasks. Typically reference tasks focus on trending in perform ance due to effects such as fatigue. O ne im portant case of reference tasks is to norm alize an individual's current capacity for m ental w orkload. S uch a pre-perform ance m easure can be used to adjust m axim um w orkload to com pensate for day-to-day variation. PHYSIOLOGICAL MEASURES The hum an nervous system is anatom ically divided into the C entral Nervous S ystem (C NS ) - and the Peripheral Nervous S ystem (PNS ). The C NS includes the brain and the spinal cord. The PNS is m ade up of the som atic division, w hich innervates the skin, voluntary m uscle, and joints, and the autonom ic division, w hich m ediates visceral sensation as w ell as executes m otor control of sm ooth m uscle, viscera, and endocrine glands. The autonom ic division consists of sym pathetic, parasym pathetic, and enteric system s. The sym pathetic system m ediates response to stress, w hile the parasym pathetic system w orks to m aintain hom eostasis and conserve body resources. a R eaction tim e is not typically m easured in real-tim e in the field as events occur at unplanned tim es. Post-hoc analysis can resolve reaction tim es dow n to a com parable resolution to innate m otor reaction variance, on the order or 10 's of m sec. b C ontrary to w hat a layperson m ay think, the vast m ajority of errors in air traffic control do not result in collisions or even near m isses; rather they are m istakes in procedure. 4 UNCLASSI FIED // FOR OFFICIAL USE ONLY UNCLASSIFIED/ /FnP PTriSIAL UOC CNET The enteric system executes control of sm ooth m uscle. Although the C NS and PNS are anatom ically distinct, they are functionally intertw ined. W hen discussing the function of the autonom ic division, it is custom ary to refer to it as the Autonom ic Nervous S ystem , or ANS .7 C hanges in global arousal or activation through changing w orkload can result in changes in physiological activity. These m easures are advantageous as changes are m easurable continuously, in real-tim e, and usually unobtrusively in a naturalistic setting. The draw back of using physiology alone is that there is no direct m easure of prim ary task perform ance. Cardiac Function Norm al beating of the hum an heart produces a distinct and repeating pattern of electrical activity m easurable throughout the body for any tw o sam ple points that cross the chest/ The typical electrocardiogram signal is show n in Figure 2. Figure 2. Typical EKG Signal for a Normal Heartbeat. Portions O f the W aveform are labeled P, Q, R , S , and T. Detection of the R -w ave allow s m easurem ent of frequency, tim e,d and am plitude. For continuous m onitoring, heart rate m easurem ents w ill vary considerably and in a non­ linear fashion; therefore, the m easurem ent of inter-beat-interval (IBI), the tim e betw een R peaks, is m ore norm ally distributed in the absence of signal, reducing noise in the m easurem ent.8 Averaging the heart rate over m inutes of task perform ance and com paring to baseline yields a reliable estim ate of increased m etabolic function.9 An additional m easure is the heart rate variability (HR V), calculated by dividing the standard deviation of IBI by an average value of IBI w ithin a sam ple period. Additional m easurem ents can be m ade by decom posing the spectra of HR V into low , m id, and c In fact, a typical introductory physics course for life science students w ill include a laboratory experim ent w here the heart rhythm is m easured betw een electrodes located on the right w rist and left ankle. d This tim e m easurem ent is properly a phase m easurem ent, w ith the phase relative to som e reference event. 5 UNCLASSI FIED/ /TOR OFFICIAL UGE OMhY UNCLASSIFIED/ /EQn QrrTCTAI UDE OMCT high frequency com ponents w hich have different noise contributions (for exam ple, core tem perature changes, blood pressure or speech, and respiration, respectively).10 Blood pressure and its variability can also be m easured. For continuous m onitoring, a finger cuff filled w ith w ater m atch the inner-arterial pressure and can be used to m onitor variability. 11 L Low -count EEG is generally less than 10 electrodes, and usually 3 or 5 . f The Blood O xygen Level Dependent (BO LD) effect is a local change in the oxygen saturation ratio near neural activity due to m etabolic and vascular action. This change is detectible in the infrared spectra. CNS Measurements M easurem ent of brain activity can be unobtrusively recorded using low -count electroencephalography (EEG ),e or the m inim ally obtrusive techniques of EEG , near­ infrared spectrom etry (NIR S ), trans-cranial Doppler sonography (TC DS ), or the non- invasive laboratory techniques of functional M R I (fM R I), m agnetoencephalography (M EG ), or positron em ission tom ography (PET). The latter three techniques are for brain research only and do not have any current naturalistic research studies (although see G enik12 for a prognosis on generation-after-next technologies including NIR S , PET, fM R I, and M EG). EEG m easurem ents are typically divided into spectra and the relative pow er in the bands 0 -4 Hz (A), 4-8 Hz (0 ), 8-13 Hz (a), 13 -3 0 Hz (p), and 3 0 -10 0 Hz (y). NIR S m easures the BO LD effect1 and is related to localized y activity. TC DS m easures C O 2 as the byproduct of localized increased m etabolism and is also an indirect m easure of neural activity w hich has been show n to be related to vigilance.13 For EEG experim ents in m ental w orkload, changes are typically reported in the 0 and a bands, though m ore recently p and y bands have show n sensitivity.14 Event-related potentials (ER P) are peaks of activity m easured at the skin indicative of several tens of thousands of neurons firing coherently for a short tim e. For exam ple, a w ell-studied cognitive ER P is P3 0 0 , a 10 's-of-m sec-w ide bum p in the EEG signal occurring 20 0 -40 0 m sec after an event. S om e success in utilizing a task-irrelevant secondary audio stim ulation and N10 0 has been show n,15 but little further developm ent to this approach has been found in the last 15 years. Ocular Measurements M easurem ents involving eye fixations, dw ell tim e (tem poral length of a fixation), and pupillary changes are w ell established m etrics of w orkload in visual searching tasks.16 Additional m easures of ocular changes include blink rate, blink duration, blink latency, and eye m ovem ent. These are recorded using one of various types of eye-tracking (ET) or electrodes to m easure an electrooculogram (EO G ). ET data w ill include position of a fixation and the tim e of each eye m ovem ent (or saccade), w hereas an EO G only identifies the tim e that the m uscle controlling eye blinks or eye position activated. 6 UNCLASSI FIED / /TOK OFFICIAL UDE ONh¥ UNCLASSIFIED/ /rOR OmCIAL U3E OWET Skin Measurements The ANS controls the opening and closing of sw eat glands in response to stress and anxiety. These changes can be observed by m easuring electroderm al activity, typically skin conductance response (S CR ),17 but also skin potentials (S P) and skin tem perature (S T). A standard score of these m easures can be calculated by collecting resting state data prior to the task interval. Serum Levels of Hormones Horm one levels are direct results of activity in the ANS . It is natural to w ant to continuously m onitor certain stress horm ones such as adrenaline or cortisol. The difficulty lies in m easurem ent tim e - typical fast assays of salivary cortisol still require about 15 m inutes.18 Until reliable in-situ m onitoring is available, serum levels of horm ones w ill only play a supporting role in establishing baseline capacities or post­ incident analysis. UNCLASSI FIED / /R5R UkMUAL UAL UNL^ UNCLASsiFiED//ron official use emy Chapter 3: Studies in Cognitive Workload for Air Traffic Controllers There are a few areas of research applicable as analogues to the piloting of several aircraft. The largest of these areas analyzes the supervisory functions of air traffic controllers (ATC ). In a typical ATC duty environm ent, a single operator is responsible for m any fully-autonom ous aircraft. ATC is either airport-based or en-route. Airport­ based facilities include ground control, local control for active runw ays (Tow er control), and approach control, w hich in the US is called Term inal R adar Approach C ontrol, or TR AC O N. A zone of air-controlled space assigned to a specific control center is that center's sector; sim ilarly, a given controller is responsible for a sector of airspace. Betw een airport-controlled sectors, aircraft are m onitored by an en-route facility. TR AC O N is the m ost cognitively dem anding function in this chain. Term inal controllers, as ATCs are called at TR AC O N facilities, are responsible for departures after take-off, and approaches and flyovers w ithin about a 5 0 -m ile radius of the airports Approaching aircraft need to be vectored by the controller into an appropriate flight path for landing, avoiding all other aircraft in the air or soon to be in the air, and then handed off to the Tow er controller for landing and ground instruction. Departing flights and flyover traffic m ainly need to be m onitored for conflicts. Approaching aircraft are by far the m ost cognitively challenging in the ATC task. Im ages of air traffic controller environm ent and displays are show n in Figure 3 ,h Errors in flight control are called anom alies in the industry. The m ost com m on aircraft anom aly is deviation from flightpath in en-route sectors. This anom aly is corrected by pilots them selves or after instruction from the appropriate ATC . The m ost com m on ATC error is m iscom m unication betw een controllers in an aircraft handoff betw een sectors.19 A study in 1997 by the National Academ ies show ed that ATC errors occurred in both high and low w orkload conditions, as predicted by overload and disengagem ent.20 In looking at the cognitive lim its in ATC, w e should seek w here typical controllers enter the B region of w orkload. Traffic load defined as sim ply the num ber of aircraft does not by itself show the com plete picture of ATC w orkload. In a study of professional controllers in 20 0 6, Boag recorded subjective m easures of w orkload and reaction tim e w hen static air traffic displays w ere presented. Displays included air traffic conflicts of differing com plexity that required resolution. C om plexity in the display w as objectively ranked using the M ethod of Analysis of R elational C om plexity (M AR C ).21 R esults show ed that a relatively sm all num ber of aircraft can greatly increase the perceived w orkload. Boag concludes that perceived com plexity is the num ber one factor in determ ining w orkload, and that although conflicts are the m ajor source of com plexity in the ATC task, and these can be m odeled som ew hat using a com bination of aircraft separation and transition 1 variables, individual differences still have significant im pact on w hen a controller m ay reach overload.22 9 Each facility w ill vary in TR AC O N sector radius. For sm aller airports, TR AC O N functions m ay be perform ed by a nearby en-route facility. h O f historical note in the US Air traffic C ontrol industry w as the industry-w ide strike begun on August 3 , 1981, by the nearly 13 ,0 0 0 ATC specialists. O nly 13 0 0 obeyed a Presidential order to return to w ork under the "peril to national safety" provision of the 1947 Taft-Hartley Act. O n August 5 , 19 81 the rem aining 11,3 45 controllers w ere fired and banned for life from federal service. The FAA rebuilt the force to pre-strike levels during the rest of the 1980 s. This event resulted in a dearth of research during the 1980 s on air traffic control professionals. 1 An aircraft transition is an event such as landing, hand-off to another controller, or entering/leaving a sector, w here a sector is defined as a controller's airspace of responsibility. 8 U N CLASSI FI E D / /£aB^UHCMb-U6£-&tte¥ UNCLAssiFiED//rnp nmciAi uoc onet In a post-hoc exam ination of sources of hum an error in taped controller data, C hang developed a conceptual m odel of ATC task w ithin a system of hum an, softw are, hardw are, and environm ent using the S HEL23 approach of ergonom ics. This study also exam ined several aspects of personnel m anagem ent, but w hen overload w as exam ined as the source of error, it w as show n that all factors need to be considered along w ith their interaction. That is, the hum an and the external task cannot be considered in isolation of the environm ent of the other hum ans, and the augm entation capabilities of softw are, hardw are, and the perform ance environm ent significantly affect error rates.24 O perator overload can result in subsequent errors if sufficient recovery tim e is not allotted. Di Nocera exam ined specific error rates after short periods of overload. The first part of the study successfully proved that these so-called post-com pletion errors existed in tim es im m ediately after peaks in w orkload. The second part of the study exam ined w hether an augm entation tool to assist in m edium term conflict detection could reduce these errors. The subject population included 18 m ilitary ATC of varying experience. W orkload w as based on self-reported NAS A-TLX . R esults show ed that augm entation processes helped junior controllers, but senior controllers w ere unaffected by the assistance.25 9 UNCLASSI FIED / /FOK OFFICIAL UDE ONL¥ UNCLASSIFIED//rQR OFFICIAL UDE CNET Figure 3. Air Traffic Control: a) overview of a large TR AC O N facility, Potom ac is pictured, b) A single ATC TR AC O N station. Beside the radar display is a stack of m em ory aid blocks w hen the controller w rites essential aircraft inform ation - as long as there is a physical block to the right, there should be a corresponding "blip" on the screen, c) TR AC O N display from M inneapolis-S t. Paul. O n this display are aircraft under control (undesignated), as w ell as craft from 3 other controllers, East (E), North (N), and G round (G ). Below each aircraft nam e is controller designation, altitude in hundreds of feet, and airspeed in tens of knots. 10 UNCLASSI FIED // FOR OFFICIAh USE ONEY UNCLASsiFiED//ran orriciAL use oro Lam oureaux researched in 1999 w hether a proposed ATC augm entation system could safely allow sm aller distances betw een aircraft and thus increase traffic capacity w ithout building new airports or runw ays. Pairs of aircraft w ere characterized based on their direction of travel and separation as show n in Table 1. During the task, operators rated their instantaneous perceived w orkload on a five-point scale roughly corresponding to D, Al, A2, A3 /B, and B/C J O perators w ere prom pted for their self­ assessm ent once every tw o m inutes. The experim ent w as able to generate self­ assessm ent values betw een 1 and 4; no overload conditions w ere generated in this study. Using a m odel based on task variance and com plexity of current aircraft under control, the researchers w ere able to predict perceived w orkload 74% of the tim e, but m ore im portantly, w ere able to predict a self-rating of 4 for 80 % of the cases. Predicting the boredom value of 1 w as less successful, at about 60 % . Besides the num erical predictions, the authors conclude that com plexity of the task drives perceived w orkload.26 Table 1. Variables used in Determining Complexity of the Traffic relationship betw een pairs of aircraft. Using four variables w ith three thresholds gives different classes of com plexity. For exam ple, an aircraft pair could be 5 m iles apart (3 to 7 threshold), traveling in the sam e lateral direction, flying w ith 25 0 0 ft of vertical separation (> 20 0 0 ft), and both straight and level. There are 81 such com binations. Threshold Variable Low M id High Lateral separation < 3 m iles 3 to 7 m iles > 7 m iles Lateral direction S am e direction O pposite direction C rossing Vertical separation < 80 0 ft 80 0 to 20 0 0 ft > 20 0 0 ft Vertical direction Both straight and level O ne straight, one clim bing or descending Both clim bing or descending Physiological indicators of stress w ere m easured at tw o low -traffic control centers (Fayetteville, AR and R osw ell, NM ) and one higher traffic center (O klahom a C ity). Heart rates w ere m easures along w ith horm one secretion levels in urine. The urine specim ens w ere pooled before analysis into tw o groups throughout a 5 -day w orkw eek: during the 8-hour w orkday, night tim e after w ork. Additionally, controllers com pleted the S tate­ Trait Anxiety Index before and after each w orkday. R esults show ed that low er traffic centers exhibited low er stress levels, and that the best biological indicator of stress w as epinephrine levels from urine rather than HR . The authors concluded that traffic load and com plexity w ere the m ain sources of stress in the ATC task rather than the nature of the job itself.27 A straightforw ard experim ent to look at physiological responses using EEG and EO G to lapses in attention during ATC tasks w as perform ed by Peiris in 20 0 5 . The goal of the study w as to categorize expert analysis of EEG /EO G data to develop an autom ated analysis program that could read the data online w ithout hum an intervention to alert operators to attention lapses and low levels of alertness. Professional ATC operators w ere given 10 m inute intervals of the psychom otor vigilance task (PVT). R ecorded data w as evaluated by several hum an expert EEG and EO G analyzers. These experts w ere not able to correctly identify alertness or attention lapses (EEG identified only 6 of 10 1 J In this study, the A3 /B rating described "Non-essential tasks suffering, could not w ork at this level for long," w hile B/C described getting behind and losing situational aw areness. 11 UNCLASSI FIED / / TOR OFFICIAL USE ONLY- UNCLASSIFIED/ /EQR QrrTCTftI USE ONCT lapses). Peiris concluded that a built-from -scratch autom ated system is needed to identify subtle features, especially in the low -count electrode EEG (5 electrodes).28 In the definition of com plexity, it is im portant to note that one should not focus entirely on a single aspect of the ATC task in the laboratory. Donald defines com plexity in tw o aspects: task com plexity independent of the event rate, and ancillary aspects of the job in a naturalistic environm ent. The com plexity of the task as a w hole needs to be considered instead of focusing on a single aspect such as m onitor and detect and the rate at w hich this task can be com pleted.29 The naturalistic environm ent w as in fact utilized by Brookings in 1996.3 0 M oreover, in situ m easurem ents w ere conducted by C ollet in 20 0 9.3 1 Both studies exam ined TLX ratings, as w ell as several ANS variables. Brookings additionally utilized EEG . In the Brookings study, three sim ulated TR ACO N sessions w ere conducted. The first session varied traffic volum e betw een low , m edium and high levels, w hile the second session varied task com plexity at a constant rate of aircraft. The third scenario w as conducted w ith an overw helm ing num ber of aircraft, the goal being to take physiological data in the condition w here situational aw areness is lost.k The traffic load variance session lasted 45 m inutes w ith three 15 -m inute sessions w here the controller w as required to handle 6, 12, and 18 aircraft; the order of presentation w as counterbalanced across subjects. O ther com plexity factors, such as the ratio of overflights, arrivals, and departures, w ere kept constant. In the com plexity variation session, the num ber of aircraft w as kept constant at 12, w hile various com plicating factors w ere m odulated. C hanging com plexity factors included: • Altering the ratio of arriving to departing and flyover traffic. • C hanging the probability that a pilot didn't hear or failed to execute a controller's instruction. • Increasing or decreasing the heterogeneity of aircraft type. In the overload session, 15 aircraft w ere presented in 5 m inutes. Physiological variables m onitored included heart activity using tw o electrodes on the chest, EO G using electrodes around the eyes,1 respiration using elastic transducer bands, and 19 channels of EEG using a cap outfitted w ith a standard 10 -20 configuration."1 Task perform ance points w ere aw arded for successfully handing aircraft, m inus any points for operational errors such as separation conflicts, hand-off errors, and m issed approaches. TLX ratings w ere recorded betw een w orkload conditions during a designed 1-m inute lull in traffic. The sim ulation w as considered quite difficult, even for professional Air Force ATC, and participants w ere required to practice until they didn't 1 Electrodes are pointed out here as m ore recent m ethodology could use infra-red optical devices to record heart and ocular activity. m The standard 10 -20 configuration refers to electrodes every 10 % /20 % of the total distance betw een right- left/anterior-posterior anatom ical m arkers, A 10 -10 configuration w ould include tw ice the electrodes, etc. k In colloquial term s, ATC call this "losing the picture." 12 UNCLASSI FIED// TOR OFFICIAL UDE OMLT UNCLASSIFIED// FOR OFFICIAL USE ONLY crash any planes in any of the scenarios - this required approxim ately 6 hours of practice per participant before the experim ent w as conducted. O nly one of the eight controllers in the Brookings study rated the overload condition as a loss of situational aw areness. The results com pared the TLX, prim ary task perform ance, and physiological m easures to low , m edium , and high w orkload conditions, as w ell as the m ax or overload condition. Prim ary task perform ance is represented in Figure 4 (this chart w as recreated visually from the source chart to accurately represent all trends), show ing a trending effect for com plexity but not volum e. Additional results show ed that changes in task difficulty (volum e or com plexity) produced changes in TLX, eye blink rate, respiration rate, and the EEG pow er spectra. The EEG pow er spectra w ere different for changes in volum e versus changes in com plexity. There w ere no observed significant correlations w ith heart rate or heart rate variability. The authors conclude that psychophysiological data can be used to accurately m easure w orkload in real-tim e, an observation they note confirm s earlier w ork on F4 crew m em bers perform ing flight tasks of m odulated com plexity.3 2 The Brookings data w as reanalyzed by W ilson in 20 0 3 using an artificial neural netw ork approach as w ell as a stepw ise discrim inate analysis to classify a physiological state as either operational or overloaded. W ilson successfully classified the overload condition consistently in m ore than 98% of the cases. The authors adm it an issue w ith psychophysiological variation (day-to-day) that w ould need to be norm alized and further research is required.3 3 Figure 4. Representation of Performance Results from Brookings Study. 30 S how n are the three scenarios w ith m odulation of traffic volum e, com plexity, and the overload condition* Note that the Low w orkload entry for the volum e m odulation perform ance (6 planes) w as already 80 % and this w as sim ilar to the 12-plane m edium com plexity perform ance. O nly the low com plexity, 12-plane scenario show ed near 10 0 % prim ary task perform ance. 13 UNCLASSIFIED// TOR OFFICIAL USE ONLY UNCLASSIFIED/ /^OtOEElGiAW^E^ftt-f The m ore recent C ollet study recorded 5 ANS variables from 25 participants during real ATC operations. The population, m ean age of 44, included only fully qualified operators, w ho w ere m onitored for one hour during TR AC O N duty at S aint Exupery International Airport (Lyon, France). C orrelation analyses w ere perform ed w ith the num ber of aircraft the operator w as currently controlling. No adjustm ent w as m ade for task com plexity; how ever, data w ere acquired betw een 6 and 9 PM local tim e to collect m edium and high w orkload data. Each participant handled betw een 1 and 10 aircraft during the study. The results of the correlation analysis are show n in Table 2. The authors conclude that changing the num ber of aircraft for professional ATCs produced correlations in physiological m easures for S C, S BF, and IHR .3 1 Table 2. Correlations among Physiological Variables in a study of air traffic controller w orkload m odulation w ith variable num ber of aircraft. NA: num ber of aircraft; TLX: NAS A self-report w orkload m etric; S td S C : norm alized skin conductance; S td S P: norm alized skin potential; S td S BF: norm alized capillary blood flow m easured through the skin; S td S T: norm alized skin tem perature; S td IHR : norm alized instantaneous heart rate. Bold values show significant correlation. S C , S BF, and IHR show significant correlation w ith changes in NA. Norm alizations (standardizations) w ere perform ed against baseline data per subject to decrease inter-subject noise.3 1 NA TLX S td S C S td S P S td S BF S td S T NA 1 TLX .98 1 p< .0 0 1 Std SC .93 .89 1 p = .0 0 2 p= .0 0 8 S td S P .77 .67 .91 1 NS NS p= .0 0 5 Std SBF -.97 -.94 -.87 -.75 1 p< ,0 0 0 1 p= .0 0 1 P= .O 2 NS S td S T -.79 -.80 -.62 -.43 .82 1 NS NS NS NS NS Std IHR .98 .95 .97 .85 -.93 -.88 pc.0 0 0 1 p< .0 0 0 1 pc-0 0 0 1 p= .O 3 p= ,0 0 2 p= .0 0 5 Adaptive autom ation (AA) is the rebalancing of w orkload betw een the com puter and hum an. Low w orkload levels can be supplem ented w ith usually routine tasks that w ill keep the operator attentive, w hile providing the subject w ith additional m ission inform ation. This "extra inform ation" m ay not be critical, but it w ill keep the subject from disengaging from the overall task. The goal of AA is to m aintain peak perform ance of the system , in the Al to A3 regions. Kaber studied AA in term s of a sim ulated ATC task in 20 0 5 . Forty non-professional participants w ere m onitored for prim ary and a probe secondary task perform ances. R esults show ed that prim ary task perform ance w as greatest w hen AA w as added to the system .3 4 14 UNCLASSI FIED / /FUR UI-Fl LI AL USE ONLY UNCLASSIFIED/ /FOR OFFICIAL USE ON LT MODELING THE AIR TRAFFIC CONTROL TASK Like any profession, ATC personnel experience day-to-day variation in perform ance, and there are natural variations betw een controllers. In order to study these differences, m entioned in m ost of the studies detailed above, a m odel needs to be built of the controller, the environm ent, and the task, w ith the goal of locating w here the m ajority of changes m ay be occurring, and w here any augm entation m ay be best suited to assist in perform ance. S pecific to the air traffic controllers, the m ajor variation source found w hen studying large variations in perform ance w as disruption of the circadian rhythm leading to a disequilibrium condition described as a biological instability. Fortunately, no fancy technology system w as required to solve this particular problem , just proper hum an resource m anagem ent to avoid frequent shift sw itching.3 5 Loft proposed that m odeling the ATC task com plexity and w orkload is insufficient to predict perform ance due to the overriding effect of operator decision strategy. ATC operators can select priorities, m anage their ow n cognitive resources, and thus regulate their ow n perform ance. The prim ary relief for the ATC operator is handing off traffic to another local operator.3 6 O ur overall topic is concerned w ith a single pilot in a space environm ent, w here no room full of colleagues exists to take up the slack; therefore, such group m odeling techniques are outside the scope of the current treatise. W e do note that Loft develops excellent single-task descriptions of tim e pressure, conflict detection, conflict resolution, etc. As show n m ultiple tim es in the preceding section, single-task processing tim e and intensity (difficulty or com plexity) are the prim ary drivers of w orkload. Developing a m odel connecting tim e, intensity, and effort, Hendy show s how decision tim e connects a tim e-intensity-effort loop (Figure 5 ). Hendy contends decision tim e is the single variable dom inant in w orkload. W ithin this loop m odel, increasing the event rate is akin to increasing task difficulty. The adaptation strategies are developed w ith training and experience, a possible explanation of the difference in junior and senior perform ance w ith augm entation aids. Averty contends that ATC w orkload cannot be directly m easured, but m ust be inferred from a quantifiable m ixture of including objective and subjective m easures. He breaks dow n the controller task into m onitoring, vectoring, and conflict solving, and develops a refinem ent of the NAS A-TLX called TLI. Averty's Traffic Load Index is based on num ber of aircraft, but each aircraft is given additional w eight according to processing requirem ents on the controller, including both cognitive and em otional w eight: for exam ple, aircraft w ith path conflicts to resolve are given the highest w eight, w hile isolated flyover traffic is given low w eight. The authors conclude that TLI needs to include physiological inputs as w ell to fully m odel the task-controller interaction. 15 U N CLASSI FI E D / /£DJUOfiW«iM-4#S€-eNtT UNCLASSIFIED//TOR. OFFICIAL USE OHL¥ Figure 5. Information Processing Model for a Human Operator. 37 In m odeling the controllers them selves, one can look to selection efforts and find w hich candidate skills are the best predictors of training success. Pre-strike data (see footnote h on page 9) of ATC training show ed that strong candidates had skills in spatial relations, abstract reasoning, and m ath as w ell as oral decision-m aking.3 8 In designing a m odel, it is tem pting to engage subject m atter experts (S M E) to estim ate the dem and of various resources w ithin a m ultiple resource m odel of ATC . C ohen studied S M E predictions of w orkload w ithin a m odel of 7 channels (visual perception, auditory perception, spatial inform ation processing, analytical inform ation processing, verbal inform ation processing, m anual activity and speech). The authors concluded that using the 7-channel m ultiple resource m odel w ith the S M E approach doesn't w ork for predicting w orkload in the ATC task.3 9 Hancock has w ritten extensively on the m ultiple-resource m odel of cognitive task perform ance under stress. He points out that functional brain im aging studies clearly show resources are separated anatom ically, providing evidence that the m ultiple resource m odel should not be abandoned in future research.40 O ne m odel that w as proposed w as that local visual distractions interfered w ith cockpit ATC tasks. The m odel has attractiveness given the incredible com plexity of cockpit displays and the popular notion that in-vehicle distraction is the root of all evil for autom obile driving. Iona proposed that a tunnel display for in-cockpit ATC inform ation w ould reduce the effect of outside visual distraction and increase prim ary task perform ance m easures. The authors concluded that this type of augm entation has little effect on trained professional pilots in perform ance of their duties.41 16 UNCLASSI FIED/ /BOR OFFICIAL UOE OHL* UNCLASSIFIED//TOR OFFICIAL USE ONLY In an essential foundation study for introducing form s of augm entation to the ATC task, W ickens m odeled the dual task environm ent of pilot traffic avoidance using alarm s to augm ent detection of conflicts. The prim ary task w as m aintaining aircraft flightpath using a sim ulated cockpit display of a crosshair inside a box: the crosshair indicated aircraft direction and drifted tow ard the sides of the box if not corrected by the pilot using a joystick control. The drift rate w as variable and could increase or decrease the difficulty of the prim ary task. The com puter m onitored for potential collisions and w arned pilots if another aircraft w as w ithin 3 m iles. Pilots w ere instructed to m aintain their ow n aircraft flightpath first, and then detect conflicts. Upon alarm , the pilot w as to exam ine an ATC display and recom m end re­ routing of the conflicting aircraft. Pilots w ere inform ed that the autom ated detection system m ay erroneously label som e situations as conflicts; therefore the pilot needed to actually perform several ATC cognitive task functions to confirm the conflict before recom m ending action. Participants in the study w ere 12 student pilots. The results of this experim ent show ed that w hen augm entation w as correct m ore than about 80 % of the tim e, perform ance decreased due to decreased vigilance in confirm ing alarm s properly on the ATC display. Additionally, w ith a high accuracy in the alarm rate, pilots did not regularly check the ATC display to ensure that the augm entation didn't m iss possible conflicts. Auditory and visual binary alarm s w ere presented and the auditory alarm s w ere m ore effective and did not interfere w ith the prim ary task (visual tracking). The authors concluded that a 20 -25 % false alarm rate w as optim al” w hen it is intended for the pilot to w ork alongside the autom ation rather than rely on it.42 The W ickens study w as undertaken w ith the plan of m oving ATC to a shared responsibility of the controller and pilot: the goal being to increase airspace capacity by rem oving som e of the m ore m undane functions like en-route course correction and en­ route conflict detection to prim arily cockpit control. This situation w ould be analogous to a spacecraft pilot operating their prim ary vehicle m anually w hile attending m any sem i­ autom ated ancillary vehicles. Although not exactly the sam e, Landsdow n studied TLX w orkload m easures on drivers perform ing m ultiple in-vehicle tasks. The authors here concluded that secondary tasks significantly increased perceived w orkload in this arrangem ent of task control.43 n This noise in the signal Is analogous to adjusting the squelch level on a C B radio: too high a setting and you m iss traffic; too low a setting and all you hear is random noise. 17 U N CLASSI FI E D / /^QBjQEUGMMSMNHF UNCLASSIFIED// ! UK UI I lUAL USE ONLY Chapter 4: Studies in Command of Multiple Semi­ Automated Vehicles Another analogue to rem otely com m anding several spacecraft is piloting or supervising operation of several unm anned ground or air vehicles (UG Vs and UAVs). The prim ary use of these scenarios is in m ilitary operations w hich im posed additional criteria on the control system . Exam ples of several UAVs and their prim ary m issions are show n in Figure 6. C ontrol of vehicles can be either in-theater, at a range of yards to 10 's of m iles, or from a long-range com m and and control center, such as the Predator reconnaissance in the Iraq or Afghan Theater executed from bases w ithin the continental United S tates. In addition to single vehicle control system s, UAV sw arm s are being developed. In this scenario a rem ote pilot executes a com m and to the sw arm w hich com m unicates am ongst itself to establish, for exam ple, an R F em itter target location.44 In this type of a control system the raw num ber of vehicles under one pilot's control can dram atically increase, but the num ber of sw arm s then takes the place of the num ber of vehicles in developing big picture cognitive lim its. S uch system s are also under developm ent for space exploration.45 In 20 0 5 , the US Arm y w as operating tw o tactical surveillance UAVs: the Hunter and its new er replacem ent, the S hadow . Each UAV requires a team of tw o operators. Dixon studied the w orkload of Hunter/S hadow operators and w ith the help of S M Es designed a sim ulation to determ ine if augm entation system s could increase the num ber of aircraft controlled per pilot from one-half to tw o. Pilots w ere responsible for m ission com pletion (reconnaissance of a com m and target area), locating targets of opportunity (TO O ), and on-board system m onitoring. There w ere three levels of pilot aircraft control: baseline, autoalert, and autopilot. In the first tw o conditions, operators controlled the flight of the aircraft using a joystick to indicate direction; altitude and airspeed w ere help constant, w hile a com puter controlled the rem aining flight param eters (pitch, bank, etc.). O ccasionally pilots needed to com pensate for w ind changes. In the autopilot condition, operators entered the final coordinates of the next com m and target and the aircraft proceeded in a straight line, com pensating autom atically for w ind changes. Pilots flew 10 straight flight legs. At the beginning of each leg, the com m and target w as identified and instructions on w hat to locate w ere given. If the pilot forgot the instructions, they could hit a "repeat" button. At the end of each leg, high-w orkload tasks of loitering and zoom /pan the onboard cam era to visualize the entire com m and target w ere executed. Along each leg betw een com m and targets, pilots w ere instructed to search for TO O s. Prim ary task com pletion included locating all relevant inform ation about the com m and target. S econdary task com pletion included TO O identification and m onitoring for an on-board system failure. The autoalert condition detected system failures and produced an audio alert w hen the com m and target w as reached. R esults show ed that the autoalert augm entation dram atically decreased the tim e to locate system failures, as w ell as significantly decreasing the num ber of requested instruction repeats. Also, the autopilot augm entation dram atically decreased both flightpath deviation and the requested num ber of repeats, and also dram atically increased the num ber of TO O detections. R esults w ere sim ilar for the single and dual aircraft scenarios, though som e perform ance, notably TO O detection (92% to 79% ), did 18 UNCLASSI FIED// FOR OFFICIAL USE ONLY UNCLASSIFIED// HUK OFFICIAL USE ONLY decrease betw een single and dual control cases. The authors attribute this to the UAV interface com plexity of 4 screens per vehicle. The authors conclude that further study is required in augm entation strategies and system developm ent.46 Figure 6. Examples of Unmanned Military vehicles: (left to right, top to bottom ) Predator, G lobal Haw k, Fire S cout, Bell Eagle, BAE M antis, rendering of an airw ing of m ixed UAVs, Tom ahaw k cruise m issile, rem ote ground vehicles, and the R aven. 19 U N CLASSI FI E D //FOR fiKICIAb MCE ONLY- UN CLASSI FI ED//FOR OFFICIAL USE ONLY Lee has previously show n that m ultiple autom ated ground vehicles can be autopiloted successfully in a tw o-stage process w hen nom inal inform ation is available beforehand about the environm ent to be searched. The tw o-stage process includes an offline, perm ission routing table generation w here the prim ary path of each vehicle in planned and dow nloaded, and an online traffic control stage w here sm all changes to the plan are executed to avoid collisions and deal w ith contingent activity.47 Attem pting to increase the lim its of UAV to pilot ratio, R uff exam ined sim ulations using three augm ented control techniques: m anual, m anagem ent by consent, and m anagem ent by exception. The authors concluded that the m iddle level of autom ation perform ed best w hen considering that augm entation algorithm s m ay have associated errors. They also concluded that the absolute m axim um num ber of UAVs a person could control is four. C um m ings addresses the UAV interface issue in a study on retargeting m ultiple in-flight cruise m issiles. C ruise m issiles w ere chosen because they require m inim al active piloting. A dual screen interface w as constructed very sim ilar to the ATC setup on one m ap and one list of objects being tracked (in the case of the ATC system , the list is physical rather than a second com puter display - see Figure 3 b). C um m ings side-steps the issue of a cognitive w orkload m etric based on com plexity and num ber of tracked objects by assum ing that an operator can execute changes to only one vehicle at a tim e, and then counting the ratio of tim e busy m aking changes to total tim e in a scenario. This m easure is called utilization and previous w ork in system s engineering48 and queuing theory has show n that utilization rates around 70 % m ax out the typical hum an operator's ability to hold a big picture. W e note that this scenario involves m inim al interaction betw een the m issiles, such as flight path conflicts. The m issile study is com pared to free flight ATC task, w here en route and conflict resolution is the responsibility of pilots rather than ATC operators. C um m ings develops several perform ance m easures that are w orthy, but for the current treatise an analogue is m ore succinct: the utilization m easure in m ultiple-object tracking and retargeting is sim ilar to a grandm aster playing m any gam es of chess sim ultaneously. They w alk from board to board, think for a bit, m ake a m ove, and then m ove to the next board. M any high-level chess players can look at board position and evaluate w hat the next m ove is w ithout know ledge of previous m oves in the gam e. The question at hand is, how m any sim ultaneous gam es can the grandm aster play before he is forced to revert to cold position analysis w ith every new presentation of a gam e? The conclusion is 16, and it agrees w ith previous w ork on free flight ATC .49 C um m ings additionally takes issue w ith the R uff lim it of four UAVs, noting that this previous experim ent included a far m ore dem anding piloting task. M ore recent w ork by C um m ings added aircraft heterogeneity to the experim ent.5 0 S he concluded again that 70 % utilization is optim al, but notes that the queuing theory concept of wait times w ill significantly affect the m axim um num ber of vehicles that can be attended. In a m ultiple-vehicle control situation, a vehicle that has exhibited som e decrem ent in perform ance has an interaction tim e w ith the operator to bring it back to acceptable perform ance. It w ill then follow its autom ated routine for a period, called the neglect tim e, until it falls again below perform ance threshold and requires the pilot's attention: the tim e betw een the need for attention and the beginning of the next interaction tim e is the w ait tim e. Including w ait tim es generated by m ore com plex 20 UNCLASSI FIED/ /.FOP OFFTc-t"' tffle ?hh UNCLASSIFIED//TOW OFFICIAL UOC ONLY interfaces reduces the m axim um num ber of aircraft controllable to seven.5 1 A final study looks at an abstract m odel of continuous re-planning in different tim e intervals. This study observes that subjects reacted differently, but in three groups, to autom ated suggestions for re-planning. The authors conclude that hum an-autom ation consensus is the prim ary driver of system perform ance.5 2 In other w ords, the hum an-task interaction m ust be taken into account, as suggested above by Averty, Loft, and others outlined above. 21 unclassified//ron OrriCIAL USE ONLY UNCLASSIFIED/ /FOP OFFICIAL USE ONW Chapter 5: Discussion W e have insight into the m axim um num ber of tracked objects in a m ultiple space vehicle piloting experim ent: it depends greatly on the com plexity of the piloting and m ission tasks. Brookings show ed that in the ATC task, m ild com plexity affected perform ance even for a little as six planes being tracked by professional controllers, w hereas these controllers regularly track up to ten. R uff show ed that w hen there is uncertainty in the augm entation system , a m axim um of four craft can be controlled and tasked to com plete m issions. The num ber of four is consistent w ith standard estim ates of hum an w orking m em ory being able to handle three to five disparate objects at a tim e. This im plies that disparate, com plex interfaces require resources from w orking m em ory to prevent loss of the big picture. Augm entation of the hum an capabilities m ainly appears to be helping to m aintain a higher num ber of w orking m em ory registers. W hether it is the handw ritten blocks for the ATC s, the stored instructions for the Dixon study, or the dual displays of C um m ings, the m ost effective augm entations in the studies above hold inform ation for quick visual retrieval that the brain w ould otherw ise keep in w orking m em ory. Any external autom ation system to assist the operator in m aking decisions w ill have an associated error rate. It w as also show n in the ATC and piloting tasks that alerts need to contain a level of noise (false alarm s) of 20 -25 % to avoid autom ation bias. R egarding w here the future of this w ork is headed, it is certain that the field is just getting started. Apollo spacecraft required dozens of ground operators to m onitor for system failures, and just a few years ago it required tw o soldiers to operate a sim ple reconnaissance drone (m ost of them still do). It is fortunate that ATC and UAV control appear to be extrem ely applicable to the initial direction of rem ote space vehicle operations. The 5 -year tim efram e should see spacecraft-specific sim ulator studies begin to appear in m ajor peer-review ed journals. The m ajor advance to com e in developing augm ented hum an capability to pilot m ultiple spacecraft w ill be in understanding the cognitive organization of m ultitasking. W ith brain im aging it has been show n that m ultiple resource theory seem s to follow the anatom ical organization of the brain. In the next 40 years w e w ill find out w hy the functional studies in m ultiple task com pletion don't seem to follow the predictions of m ultiple resource theory. 22 UNCLASSI FIED/ /FOA OFFICIAL USE ONLY UNCLASSIFIED// TUK UI 11L1AL USE ONL¥ Chapter 6: Conclusions There is a lack of research in the area of cognitive lim its on the num ber of spacecraft one pilot could control given any m ission scenario. Tw o m odels for exam ining w hat are sim ilar activities are air traffic control and rem ote piloting of m ultiple unm anned vehicles. W e have show n the research progress in both areas, and the cognitive lim its on the num ber of craft that can be sim ultaneously controlled are 16 for sim ple destination selection, 7 for m oderately com plex piloting and/or m ission task com pletion, and 4 for com plex heterogeneous craft. Future research m ay increase the autom ation com ponent of aircraft and m ission control, but there is no evidence to date that a com plete m ental picture can be m aintained, even w ith external w orking m em ory augm entation, for m ore than about 16 objects at one tim e. W e have additionally show n that physiological variables can be objectively em ployed to indicate overload. Nom inal success has been achieved in classifying physiological states near high w orkload and thus able to predict and thus possibly prevent overload. W e expect this classification w ill be achieved w ith near perfect accuracy w ithin five years of specific studies being com m enced. 23 UNCLASSI FIED / /FOP OFFTfT "1 .irrn..!^ UNCLASSIFIED// FeR OFFICIAL U5E ONLY References 1 Hart, S . G . & S taveland, L. E. in Human Mental Workload, eds Peter A. Hancock & N M eshkati) 13 9-183 (North Holland, 1988). 2 Hart, S . G . NASA Task Load Index (TLX): 20 Years Later, < 20 0 6 Paper. pdf> (20 0 6). http://hum ansvstem s.arc.nasa.qov/qroups/TLX /dow nloads/HFES 3 Hart, S . G . & S taveland, L. E. Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research, < > (1987). http://hum ansvstem s.arc.nasa.qQv/qroups/TLX /dow nloads/NAS A- TLX C hapter.pdf 4 R eid, G . B., S hingledecker, C . A. & Eggem eier, F. T. in Human Factors Society 25th annual meeting. 5 22-5 26 (Hum an Factors S ociety). 5 C um m ings, M . L. & M itchell, P. J. O perator scheduling strategies in supervisory control of m ultiple UAVs. Aerospace Science and Technology 11, 3 3 9-3 48, doi:10 .10 16/j.ast.20 0 6.10 .0 0 7 (20 0 7). 6 G raydon, F. X . et al. Visual event detection during sim ulated driving: Identifying the neural correlates w ith functional neuroim aging. Transportation Research Part F-Traffic Psychology and Behaviour 7, 271-286, doi:10 .10 16/j.trf.20 0 4.0 9.0 0 6 (20 0 4). 7 Kandel, E. R ., S chw artz, J. H. & Jessell, T. M . Principles of neural science. 4th edn, (M cG raw -Hill, Health Professions Division, 20 0 0 ). 8 Jennings, J. R ., S tringfellow , J. C . & G raham , M . A com parison of the statistical distributions of beat-by-beat heart rate and heart period. Psychophysiology 11, 20 7-210 (1974). 9 Porges, S . W . & Byrne, E. A. R esearch m ethods for m easurem ent of heart rate and respiration. Biol Psychol 3 4, 93 -13 0 (1992). 10 M ulder, L. J. M easurem ent and analysis m ethods of heart rate and respiration for use in applied environm ents. Biol Psychol 3 4, 20 5 -23 6 (1992). 11 S teptoe, A. & S aw ada, Y. Assessm ent of baroreceptor reflex function during m ental stress and relaxation. Psychophysiology 26, 140 -147 (1989). 12 G enik, R . J., 2nd, G reen, C . C ., G raydon, F. X . & Arm strong, R . E. C ognitive avionics and w atching spaceflight crew s think: generation-after-next research tools in functional neuroim aging. Aviat Space Environ Med 76, B20 8-212 (20 0 5 ). 13 W arm , J. S ., Parasuram an, R . & M atthew s, G . Vigilance requires hard m ental w ork and is stressful. Hum Factors 5 0 , 43 3 -441 (20 0 8). 14 Dussault, C ., Jouanin, J. C ., Philippe, M . & G uezennec, C . Y. EEG and EC G changes during sim ulator operation reflect m ental w orkload and vigilance. Aviat Space Environ Med 76, 3 44-3 5 1 (20 0 5 ). 15 Kram er, A. F., Trejo, L. J. & Hum phrey, D. Assessm ent of m ental w orkload w ith task-irrelevant auditory probes. Biol Psychol A0, 83 -10 0 , doi:0 3 0 1- 0 5 11(95 )0 5 10 8-2 [pii] (1995 ). 16 Backs, R . W . & W alrath, L. C . Eye m ovem ent and pupillary response indices of m ental w orkload during visual search of sym bolic displays. Appl Ergon 23 , 243 ­ 25 4, doi:0 0 0 3 6870 9290 15 2L [pii] (1992). 17 Freedm an, L. W . et al. The relationship of sw eat gland count to electroderm al activity. Psychophysiology 3 1, 196-20 0 (1994). 24 U N CLASSI FI E D/ /TOR OFFICIAL USE ONLY UNCLASSI FIED/ / FOP AEEICIAL UOC ONEY 18 19 20 21 22 23 24 25 26 27 28 29 3 0 3 1 3 2 3 3 3 4 3 5 M itchell, J. S ., Low e, T. E. & Ingram , J. R . R apid ultrasensitive m easurem ent of salivary cortisol using nano-linker chem istry coupled w ith surface plasm on resonance detection. Analyst 13 4, 3 80 -3 86, doi:10 .10 3 9/b8170 83 p (20 0 9). Billings, C . E. & R eynard, W . D. Hum an factors in aircraft incidents: results of a 7-year study. Aviat Space Environ Med 5 5 , 960 -965 (1984). W ickens, C . D., M avor, A. S ., M cG ee, J. & National R esearch C ouncil (U.S .). Panel on Hum an Factors in Air Traffic C ontrol Autom ation. Flight to the future: human factors in air traffic control. (National Academ y Press, 1997). Halford, G . S ., W ilson, W . H. & Phillips, S . Processing capacity defined by relational com plexity: im plications for com parative, developm ental, and cognitive psychology. Behav Brain Sci 21, 80 3 -83 1; discussion 83 1-864 (1998). Boag, C ., Neal, A., Loft, S . & Halford, G . S . An analysis of relational com plexity in an air traffic control conflict detection task. Ergonomics 49, 15 0 8-15 26, doi: 10 .10 80 /0 0 140 13 0 60 0 779744 (20 0 6). Edw ards, E. in Proceedings of British Airline Pilots Association Technical Symposium. 21-3 6 (British Airline Pilots Association). C hang, Y. H. & Yeh, C . H. Hum an perform ance interfaces in air traffic control. Appl Ergon 41, 123 -129, doi:10 .10 16/j.apergo.20 0 9.0 6.0 0 2 (20 10 ). Di Nocera, F., Fabrizi, R ., Terenzi, M . & Ferlazzo, F. Procedural errors in air traffic control: effects of traffic density, expertise, and autom ation. Aviat Space Environ Med 77, 63 9-643 (20 0 6). Lam oureux, T. The influence of aircraft proxim ity data on the subjective m ental w orkload of controllers in the air traffic control task. Ergonomics 42, 1482-1491 (1999). M elton, C . E., S m ith, R . C ., M cKenzie, J. M ., W icks, S . M . & S aldivar, J. T. S tress in air traffic personnel: low -density tow ers and flight service stations. Aviat Space Environ Med 49, 724-728 (1978). Peiris, M . T. et al. Identification of vigilance lapses using EEG /EO G by expert hum an raters. Conf Proc IEEE Eng Med Biol Soc 6, 5 73 5 -5 73 7, doi: 10 .110 9/IEM BS .20 0 5 .1615 790 (20 0 5 ). Donald, F. M . The classification of vigilance tasks in the real w orld. Ergonomics 5 1, 1643 -165 5 , doi:10 .10 80 /0 0 140 13 0 80 23 27219 (20 0 8). Brookings, J. B., W ilson, G . F. & S w ain, C . R . Psychophysiological responses to changes in w orkload during sim ulated air traffic control. Biological Psychology 42, 3 61-3 77 (1996). C ollet, C ., Averty, P. & Dittm ar, A. Autonom ic nervous system and subjective ratings of strain in air-traffic control. Appl Ergon 40 , 23 -3 2, doi:10 .10 16/j.apergo.20 0 8.0 1.0 19 (20 0 9). ~ W ilson, G . F. & Fisher, F. The use of cardiac and eye blink m easures to determ ine flight segm ent in F4 crew s. Aviation Space and Environmental Medicine 62, 95 9-962 (1991). W ilson, G . F. & R ussell, C . A. O perator functional state classification using m ultiple psychophysiological features in an air traffic control task. Hum Factors 45 , 3 81-3 89 (20 0 3 ). Kaber, D. B., W right, M . C ., Prinzel, L. J., 3 rd & C iam ann, M . P. Adaptive autom ation of hum an-m achine system inform ation-processing functions. Hum Factors 47, 73 0 -741 (20 0 5 ). M ohler, S . R . The hum an elem ent in air traffic control: aerom edical aspects, problem s, and prescriptions. Aviat Space Environ Med 5 4, 5 11-5 16 (1983 ). 25 unclassified//ron ornciAL uor only UNCLASSIFIED//TO K OFFICIAL UDE ONLY 3 6 3 7 3 8 3 9 40 41 42 43 44 45 46 47 48 49 5 0 5 1 5 2 26 Loft, S ., S anderson, P., Neal, A. & M ooij, M . M odeling and predicting m ental w orkload in en route air traffic control: critical review and broader im plications. Hum Factors 49, 3 76-3 99 (20 0 7). Hendy, K. C ., Liao, J. & M ilgram , P. C om bining tim e and intensity effects in assessing operator inform ation-processing load. Hum Factors 3 9, 3 0 -47 (1997). Boone, J. O . Tow ard the developm ent of a new aptitude selection test battery for air traffic control specialists. Aviat Space Environ Med 5 1, 694-699 (1980 ). C ohen, D., W herry, R . J., Jr. & G lenn, F. Analysis of w orkload predictions generated by m ultiple resource theory. Aviat Space Environ Med 67, 13 9-145 (1996). Hancock, P. A. & S zalm a, J. L. Performance under stress. (Ashgate Pub., 20 0 8). lani, C . & W ickens, C . D. Factors affecting task m anagem ent in aviation. Hum Factors 49, 16-24 (20 0 7). W ickens, C . & C olcom be, A. Dual-task perform ance consequences of im perfect alerting associated w ith a cockpit display of traffic inform ation. Hum Factors 49, 83 9-85 0 (20 0 7). Lansdow n, T. C ., Brook-C arter, N. & Kersloot, T. Distraction from m ultiple in­ vehicle secondary tasks: vehicle perform ance and m ental w orkload im plications. Ergonomics 47, 91-10 4, doi:10 .10 80 /0 0 140 13 0 3 10 0 0 1629775 (20 0 4). Pack, D. J., Delim a, P., Toussaint, G . J. & York, G . C ooperative control of UAVs for localization of interm ittently em itting m obile targets. IEEE Trans Syst Man Cybern B Cybern 3 9, 95 9-970 , doi: 10 .110 9/TS M C B.20 0 8.20 10 865 (20 0 9). W ang, J., Qu, Z. H., Ihlefeld, C . M . & Hull, R . A. A control-design-based solution to robotic ecology: Autonom y of achieving cooperative behavior from a high- level astronaut com m and. Autonomous Robots 20 , 97-112, doi:10 .10 0 7/sl0 5 14- 0 0 6-5 942-5 (20 0 6). Dixon, S . R ., W ickens, C . D. & C hang, D. M ission control of m ultiple unm anned aerial vehicles: a w orkload analysis. Hum Factors 47, 479-487 (20 0 5 ). Lee, J. H., Lee, B. H. & C hoi, M . H. A real-tim e traffic control schem e of m ultiple AG V system s for collision free m inim um tim e m otion: A routing table approach. leee Transactions on Systems Man and Cybernetics Part a-Systems and Humans 28, 3 47-3 5 8 (1998). R ouse, W . B. Systems engineering models of human-machine interaction. (North Holland, 1980 ). C um m ings, M . L. & G uerlain, S . Developing operator capacity estim ates for supervisory control of autonom ous vehicles. Human Factors 49, 1-15 (20 0 7). Kornguth, S . E., S teinberg, R . & M atthew s, M . D. Neurocognitive and physiological factors during high-tempo operations. (Ashgate, 20 10 ). C um m ings, M . L. & M itchell, P. J. Predicting controller capacity in supervisory control of m ultiple UAVs. leee Transactions on Systems Man and Cybernetics Part a-Systems and Humans 3 8, 45 1-460 , doi: 10 .110 9/tsm ca.20 0 7.91475 7 (20 0 8). C um m ings, M . L., C lare, A. & Hart, C . The R ole of Hum an-Autom ation C onsensus in M ultiple Unm anned Vehicle S cheduling. Human Factors 5 2, 17-27, doi: 10 .1177/0 0 18720 810 3 68674 (20 10 ). UNCLASSI FIED// FOR GPHHAL USE OMIFF