https://hdl.handle.net/1721.1/46717 2026-04-03T17:51:10Z https://hdl.handle.net/1721.1/90368 Mixed-Initiative Strategies for Real-time Scheduling of Multiple Unmanned Vehicles Clare, A. S.; Macbeth, J. C,; Cummings, M. L. Advances in autonomy have made it possible to invert the typical operator-to-unmanned vehicle ratio so that a single operator can now control multiple heterogeneous Unmanned Vehicles (UVs). Real-time scheduling and task assignment for multiple UVs in uncertain environments will require the computational ability of optimization algorithms combined with the judgment and adaptability of human supervisors through mixed-initiative systems. The goal of this paper is to analyze the interactions between operators and scheduling algorithms in two human- in-the-loop multiple UV control experiments. The impact of real-time operator modifications to the objective function of an optimization algorithm for multi-UV scheduling is described. Results from outdoor multiple UV flight tests using a human-computer collaborative scheduling system are presented, which provide valuable insight into the impact of environmental uncertainty and vehicle failures on system effectiveness. 2012-01-01T00:00:00Z https://hdl.handle.net/1721.1/90367 One Work Analysis, Two Domains: A Display Information Requirements Case Study Cummings, M. L.; Tappan, J.; Mikkelsen, C. d observations, among other techniques. Given the time and resources required, we examine how to generalize a work domain analysis technique, namely the hybrid Cognitive Task Analysis (hCTA) method across two domains in order to generate a common set of display information requirements. The two domains of interest are field workers troubleshooting low voltage distribution networks and telecommunication problems. Results show that there is a high degree of similarity between the two domains due to their service call nature, particularly in tasking and decision-making. While the primary differences were due to communication protocols and equipment requirements, the basic overall mission goals, functions, phases of operation, decision processes, and situation requirements were very similar. A final design for both domains is proposed based on the joint requirements. 2012-01-01T00:00:00Z https://hdl.handle.net/1721.1/90366 Teamwork in controlling multiple robots Gao, F.; Cummings, M. L.; Bertuccelli, L. F. Simultaneously controlling increasing numbers of robots requires multiple operators working together as a team. Helping operators allocate attention among different robots and determining how to construct the human-robot team to promote performance and reduce workload are critical questions that must be answered in these settings. To this end, we investigated the effect of team structure and search guidance on operators’ performance, subjective workload, work processes and communication. To investigate team structure in an urban search and rescue setting, we compared a pooled condition, in which team members shared control of 24 robots, with a sector condition, in which each team member control half of all the robots. For search guidance, a notification was given when the operator spent too much time on one robot and either suggested or forced the operator to change to another robot. A total of 48 participants completed the experiment with two persons forming one team. The results demonstrate that automated search guidance neither increased nor decreased performance. However, suggested search guidance decreased average task completion time in Sector teams. Search guidance also influenced operators’ teleoperation behaviors. For team structure, pooled teams experienced lower subjective workload than sector teams. Pooled teams communicated more than sector teams, but sector teams teleoperated more than pool teams. 2012-01-01T00:00:00Z https://hdl.handle.net/1721.1/90365 Interface Design for Unmanned Vehicle Supervision through Hybrid Cognitive Task Analysis Macbeth, J. C.; Cummings, M. L.; Bertuccelli, L. F.; Surana, A. While there is currently significant interest in developing Unmanned Aerial Systems (UASs) that can be supervised by a single operator, the majority of these systems focus on Intelligence, Surveillance, and Reconnaissance (ISR) domains. One domain that has received significantly less attention is the use of multiple UASs to insert or extract supplies or people. To this end, MAVIES (Multi-Autonomous Vehicle Insertion-Extraction System) was developed to allow a single operator the ability to supervise a primary cargo Unmanned Aerial Vehicle (UAV) along with multiple scouting UAVs. This paper will detail the development of the design requirements generated through a Hybrid Cognitive Task Analysis (hCTA) and the display that resulted from these efforts. A major innovation in the hCTA process in this effort was the alteration of the traditional decision ladder process to specifically identify decision-making tasks that must be augmented with automation. 2012-10-01T00:00:00Z https://hdl.handle.net/1721.1/90363 A Model-Based Measure to Assess Operator Adherence to Procedure Stimpson, A. J.; Buinhas, L. S.; Bezek, S.; Boussemart, Y.; Cummings, M. L. Procedures play an important role in domains where humans interact with critical, complex systems. In such environments, the operator’s ability to correctly follow a given set of procedures can directly impact system safety. A quantitative measure of procedural adherence during training for complex system opera-tion would be useful to assess trainee performance and evaluate a training program. This paper presents a novel model-based objective metric for quantifying procedural adherence in training. This metric is sensi-tive to both the number and nature of procedural deviations, and can be used with cluster analysis to classi-fy trainee performance based on adherence. The metric was tested on an experimental data set gathered from volunteers using aircraft maintenance computer-based training (CBT). The properties of the metric are discussed, along with future possibilities. 2012-10-01T00:00:00Z https://hdl.handle.net/1721.1/90362 Operator Performance in Long Duration Control Operations: Switching from Low to High Task Load Thornburg, K. M.; Peterse, H.P.M.; Liu, A.M. Long duration, low task load environments are typical for nuclear power plant control rooms, where operators, after hours of operating under a low task load situation, may have to shift to a high task load situation. The effects of time-on-task and boredom due to low task load will be an important consideration for the design of new nuclear power plant control rooms, which will rely more heavily on automation. This paper describes a research study of performance in a simulated nuclear control room environment, where 36 participants responded to an alarm during a 4 hour long experiment where the alarm onset time and the availability of distractions were varied. The results indicate that operators perform better in a sterile environment and that the duration of non-active time before the alarm influences operator performance. 2012-10-01T00:00:00Z https://hdl.handle.net/1721.1/90360 Using Variable-Rate Alerting to Counter Boredom in Human Supervisory Control Mkrtchyan, A. A.; Macbeth, J. C.; Solovey, E. T.; Ryan, J. C.; Cummings, M. L. A low task load, long duration experiment was conducted to evaluate the impact of cyclical attention switching strategies on operator performance in supervisory domains. The impetus for such a study stems from the lack of prior work to improve human-system performance in low task load supervisory domains through the use of design interventions. In this study, a design intervention in the form of auditory alerts is introduced and the effects of the alerts are examined. The test bed consists of a video game-like simulation environment, which allows a single opera-tor the ability to supervise multiple unmanned vehicles. Each participant in the study completed two different four hour sessions, with and without the alerts. The results suggest that the alerts can be useful for operators who are dis-tracted for a considerable amount of time, but that the alerts may not be appropriate for operators who are able to sustain directed attention for prolonged periods. 2012-10-01T00:00:00Z https://hdl.handle.net/1721.1/90358 Using Discrete Event Simulation to Model Multi-Robot Multi-Operator Teamwork Gao, F.; Cummings, M. L. With the increasing need for teams of operators in controlling multiple robots, it is important to understand how to construct the team and support team processes. While running experiments can be time consuming and expensive, the use of simulation models is an alternative method. In this study, we built a discrete event simulation model that represents multi-robot multi-operator teamwork. Preliminary results show that the model can generate performance measures consistent with experimental results. 2012-10-01T00:00:00Z https://hdl.handle.net/1721.1/90335 Modeling teamwork of multi-human multi-agent teams Gao, F. Teamwork is important when humans work together with automated agents to perform tasks requiring monitoring, coordination, and complex decision making. While human-agent teams can bring many benefits such as higher productivity, adaptability and creativity, they may also fail for various reasons. It is important to understand the tradeoffs in teamwork. The purpose of this research is to investigate the process and outcomes of human-agent teamwork by running experiments and building quantitative simulation models. Preliminary results are discussed as well as future directions. We expect this research to deepen the under-standing of human-agent teamwork and provide recommendations for the design of teams and agents to support teamwork. 2013-06-01T00:00:00Z https://hdl.handle.net/1721.1/90334 Modeling the Impact of Operator Trust on Performance in Multiple Robot Control, Gao, F.; Clare, A. S.; Macbeth, J. C.; Cummings, M. L. We developed a system dynamics model to simulate the impact of operator trust on performance in multiple robot control. Analysis of a simulated urban search and rescue experiment showed that operators decided to manually control the robots when they lost trust in the autonomous planner that was directing the robots. Operators who rarely used manual control performed the worst. However, the operators who most frequently used manual control reported higher workload and did not perform any better than operators with moderate manual control usage. Based on these findings, we implemented a model where trust and performance form a feedback loop, in which operators perceive the performance of the system, calibrate their trust, and adjust their control of the robots. A second feedback loop incorporates the impact of trust on cognitive workload and system performance. The model was able to replicate the quantitative performance of three groups of operators within 2.3%. This model could help us gain a greater understanding of how operators build and lose trust in automation and the impact of those changes in trust on performance and workload, which is crucial to the development of future systems involving humanautomation collaboration. 2013-03-01T00:00:00Z https://hdl.handle.net/1721.1/90333 Investigating the Efficacy of Network Visualizations for Intelligence Tasks Berardi, C.; Solovey, E. T.; Cummings, M. L. There is an increasing requirement for advanced analytical methodologies to help military intelligence analysts cope with the growing amount of data they are saturated with on a daily basis. Specifically, within the context of terror network analysis, one of the largest problems is the transformation of raw tabular data into a visualization that is easily and effectively exploited by intelligence analysts. Currently, the primary method within the intelligence domain is the node-link visualization, which encodes data sets by depicting the ties between nodes as lines between objects in a plane. This method, although useful, has limitations when the size and complexity of data grows. The matrix offers an alternate perspective because the two dimensions of the matrix are arrayed as an actors x actors matrix. This paper describes an experiment investigating node-link and matrix visualization techniques within social network analysis, and their effectiveness for the intelligence tasks of: 1) identifying leaders and 2) identifying clusters. The sixty participants in the experiment were all Air Force intelligence analysts and we provide recommendations for building visualization tools for this specialized group of users. 2013-06-01T00:00:00Z https://hdl.handle.net/1721.1/90332 Comparing the Performance of Expert User Heuristics and an Integer Linear Program in Aircraft Carrier Deck Operations Ryan, J. C.; Banerjee, A. G.; Cummings, M. L.; Roy, N. Planning operations across a number of domains can be considered as resource allocation problems with timing constraints. An unexplored instance of such a problem domain is the aircraft carrier flight deck, where, in current operations, replanning is done without the aid of any computerized decision support. Rather, veteran operators employ a set of experience based heuristics to quickly generate new operating schedules. These expert user heuristics are neither codified nor evaluated by the United States Navy; they have grown solely from the convergent experiences of supervisory staff. As unmanned aerial vehicles (UAVs) are introduced in the aircraft carrier domain, these heuristics may require alterations due to differing capabilities. The inclusion of UAVs also allows for new opportunities for on-line planning and control, providing an alternative to the current heuristic-based replanning methodology. To investigate these issues formally, we have developed a decision support system for flight deck operations that utilizes a conventional integer linear program-based planning algorithm. In this system, a human operator sets both the goals and constraints for the algorithm, which then returns a proposed schedule for operator approval. As a part of validating this system, the performance of this collaborative human–automation planner was compared with that of the expert user heuristics over a set of test scenarios. The resulting analysis shows that human heuristics often outperform the plans produced by an optimization algorithm, but are also often more conservative. 2013-08-01T00:00:00Z https://hdl.handle.net/1721.1/90325 The Need for Command and Control Instant Message Adaptive Interfaces: Lessons Learned from Tactical Tomahawk Human-in-the-Loop Simulations Cummings, M. L. In the recent development of a human-in-the-loop simulation test bed designed to examine human performance issues for supervisory control of the Navy’s new Tactical Tomahawk missile, measurements of operator situation awareness (SA) and workload through secondary tasking were taken through an embedded instant messaging program. Instant message interfaces (otherwise known as “chat”), already a means of communication between Navy ships, allow researchers to query users in real-time in a natural, ecologic setting, and thus provide more realistic and unobtrusive measurements. However, in the course of this testing, results revealed that some subjects fixated on the real-time instant messaging secondary task instead of the primary task of missile control, leading to the overall degradation of mission performance as well as a loss of SA. While this research effort was the first to quantify command and control performance degradation as a result of instant messaging, the military has recognized that in its network centric warfare quest, instant messaging is a critical informal communication tool, but has associated problems. Recently a military spokesman said that managing chat in current military operations was sometimes a “nightmare” because military personnel have difficulty in handling large amounts of information through chat, and then synthesizing knowledge from this information. This research highlights the need for further investigation of the role of instant messaging interfaces both on task performance and situation awareness, and specifies how the associated problems could be ameliorated through adaptive display design. 2004-01-01T00:00:00Z https://hdl.handle.net/1721.1/90324 Partitioning Complexity in Air Traffic Management Task Cummings, M. L.; Tsonis, C. Cognitive complexity is a term that appears frequently in air traffic control (ATC) research literature, yet there is little principled investigation of the potential sources of cognitive complexity. Three distinctly different sources of cognitive complexity are proposed which are environmental, organizational, and display. Two experiments were conducted to explore whether or not these proposed components of complexity could be effectively partitioned, measured, and compared. The findings demonstrate that sources of complexity can be decomposed and measured and furthermore, the use of color in displays, a display design intervention meant to reduce environmental complexity, can actually contribute to it. 2006-01-01T00:00:00Z https://hdl.handle.net/1721.1/90321 Automation and Accountability in Decision Support System Interface Design Cummings, M. L. When the human element is introduced into decision support system design, entirely new layers of social and ethical issues emerge but are not always recognized as such. This paper discusses those ethical and social impact issues specific to decision support systems and highlights areas that interface designers should consider during design with an emphasis on military applications. Because of the inherent complexity of socio-technical systems, decision support systems are particularly vulnerable to certain potential ethical pitfalls that encompass automation and accountability issues. If computer systems diminish a user’s sense of moral agency and responsibility, an erosion of accountability could result. In addition, these problems are exacerbated when an interface is perceived as a legitimate authority. I argue that when developing human computer interfaces for decision support systems that have the ability to harm people, the possibility exists that a moral buffer, a form of psychological distancing, is created which allows people to ethically distance themselves from their actions. 2006-01-01T00:00:00Z https://hdl.handle.net/1721.1/90289 Operator Scheduling Strategies in Supervisory Control of Multiple UAVs Cummings, M. L.; Mitchell, P. J. The application of network centric operations to time-constrained command and control environments will mean that human operators will be increasingly responsible for multiple simultaneous supervisory control tasks. One such futuristic application will be the control of multiple unmanned aerial vehicles (UAVs) by a single operator. To achieve such performance in complex, time critical, and high risk settings, automated systems will be required both to guarantee rapid system response as well as manageable workload for operators. Through the development of a simulation test bed for human supervisory control of multiple independent UAVs by a single operator, this paper presents recent efforts to investigate workload mitigation strategies as a function of increasing automation. A humanin- the-loop experiment revealed that under low workload conditions, operators’ cognitive strategies were relatively robust across increasing levels of automated decision support. However, when provided with explicit automated recommendations and with the ability to negotiate with external agencies for delays in arrival times for targets, operators inappropriately fixated on the need to globally optimize their schedules. In addition, without explicit visual representation of uncertainty, operators tended to treated all probabilities uniformly. This study also revealed that operators that reached cognitive saturation adapted two very distinct management strategies, which led to varying degrees of success. Lastly, operators with management-by-exception decision support exhibited evidence of automation bias. 2007-01-01T00:00:00Z https://hdl.handle.net/1721.1/90288 Effects of Single versus Multiple Warnings on Driver Performance Cummings, M. L.; Kilgore, R. M.; Wang, E.; Tijerina, L.; Kochhar, D. S. Objective: To explore how a single master alarm system affects drivers’ responses when compared to multiple, distinct warnings. Background: Advanced driver warning systems are intended to improve safety, yet inappropriate integration may increase the complexity of driving, especially in high workload situations. This study investigated the effects of auditory alarm scheme, reliability, and collision event-type on driver performance. Method: A 2x2x4 mixed factorial design investigated the impact of two alarm schemes (master vs. individual) and two levels of alarm reliability (high and low) on distracted drivers’ performance across four collision event-types (frontal collision warnings, left and right lane departure warnings, and follow vehicle fast approach). Results: Participants’ reaction times and accuracy rates were significantly affected by the type of collision event and alarm reliability. The use of individual alarms, rather than a single master alarm, did not significantly affect driving performance in terms of reaction time or response accuracy. Conclusion: Even though a master alarm is a relatively uninformative warning, it produced statistically no different reaction times or accuracy results when compared to information-rich auditory icons, some of which were spatially located. In addition, unreliable alarms negatively impacted driver performance, regardless of event type or alarm scheme. Application: These results have important implications for the development and implementation of multiple driver warning systems. 2007-01-01T00:00:00Z https://hdl.handle.net/1721.1/90287 The Impact of Intelligent Aiding for Multiple Unmanned Aerial Vehicle Schedule Management Cummings, M. L.; Brzezinski, A. S.; Lee, J. D. There is increasing interest in designing systems such that the current many-to-one ratio of operators to unmanned vehicles (UVs) can be inverted. Instead of lower-level tasks performed by today’s UV teams, the sole operator would focus on high-level supervisory control tasks. A key challenge in the design of such single-operator systems will be the need to minimize periods of excessive workload that arise when critical tasks for several UVs occur simultaneously. Thus some kind of decision support is needed that facilitates an operator’s ability to evaluate different action alternatives for managing a multiple UV mission schedule in real-time. This paper describes two decision support experiments that attempted to provide UAV operators with multivariate scheduling assistance, with mixed results. Those automated decision support tools that provided more local, as opposed to global, visual recommendations produced superior performance, suggesting that meta-information displays could saturate operators and reduce performance. 2007-01-01T00:00:00Z https://hdl.handle.net/1721.1/90286 Identifying Predictive Metrics for Supervisory Control of Multiple Robots Crandall, J. W.; Cummings, M. L. In recent years, much research has focused on making possible single operator control of multiple robots. In these high workload situations, many questions arise including how many robots should be in the team, which autonomy levels should they employ, and when should these autonomy levels change? To answer these questions, sets of metric classes should be identified that capture these aspects of the human-robot team. Such a set of metric classes should have three properties. First, it should contain the key performance parameters of the system. Second, it should identify the limitations of the agents in the system. Third, it should have predictive power. In this paper, we decompose a human-robot team consisting of a single human and multiple robots in an effort to identify such a set of metric classes. We assess the ability of this set of metric classes to (a) predict the number of robots that should be in the team and (b) predict system effectiveness. We do so by comparing predictions with actual data from a user study, which is also described. 2007-01-01T00:00:00Z https://hdl.handle.net/1721.1/90285 Automation Architecture for Single Operator, Multiple UAV Command and Control, Cummings, M. L.; Bruni, S.; Mercier, S.; Mitchell, P. J. In light of the Office of the Secretary Defense’s Roadmap for unmanned aircraft systems (UASs), there is a critical need for research examining human interaction with heterogeneous unmanned vehicles. The OSD Roadmap clearly delineates the need to investigate the “appropriate conditions and requirements under which a single pilot would be allowed to control multiple airborne UA (unmanned aircraft) simultaneously.” Toward this end, in this paper, we provide a meta-analysis of research studies across unmanned aerial and ground vehicle domains that investigated single operator control of multiple vehicles. As a result, a hierarchical control model for single operator control of multiple unmanned vehicles (UV) is proposed that demonstrates those requirements that will need to be met for operator cognitive support of multiple UV control, with an emphasis on the introduction of higher levels of autonomy. The challenge in achieving effective management of multiple UV systems in the future is not only to determine whether automation can be used to improve human and system performance, but how and to what degree across hierarchical control loops, as well as determining the types of decision support that will be needed by operators given the high-workload environment. We address when and how increasing levels of automation should be incorporated in multiple UV systems and discuss the impact on not only human performance, but more importantly, on system performance. 2007-01-01T00:00:00Z https://hdl.handle.net/1721.1/90284 Developing Lunar Landing Vehicle Display Requirements through Content Analysis of Apollo Lunar Landing Voice Communications Smith, C. A.; Cummings, M. L.; Sim, L. The lengthy period since the Apollo landings limits present-day engineers attempting to draw from the experiences of veteran Apollo engineers and astronauts in the design of a new lunar lander. In order to circumvent these limitations, content analyses were performed on the voice transcripts of the Apollo lunar landing missions. The analyses highlighted numerous inefficiencies in the design of the Apollo Lunar Module displays, particularly in the substantial use of the cognitive resources of the Lunar Module Pilot in the performance of low-level tasks. The results were used to generate functional and information requirements for the next-generation lunar lander cockpit. 2008-01-01T00:00:00Z https://hdl.handle.net/1721.1/90283 Past, Present And Future Implications Of Human Supervisory Control In Space Missions Sim, L.; Cummings, M.L.; Smith, C. A. Achieving the United States’ Vision for future Space Exploration will necessitate far greater collaboration between humans and automated technology than previous space initiatives. However, the development of methodologies to optimize this collaboration currently lags behind development of the technologies themselves, thus potentially decreasing mission safety, efficiency and probability of success. This paper discusses the human supervisory control (HSC) implications for use in space, and outlines several areas of current automated space technology in which the function allocation between humans and machines/automation is sub-optimal or under dispute, including automated spacecraft landings, Mission Control, and wearable extra-vehicular activity computers. Based on these case studies, we show that a more robust HSC research program will be crucial to achieving the Vision for Space Exploration, especially given the limited resources under which it must be accomplished. 2008-01-01T00:00:00Z https://hdl.handle.net/1721.1/90282 The Impact of Multi-layered Data-blocks on Controller Performance Cummings, M.L.; Tsonis, C.; Rader, A. As a consequence of the push to increase National Airspace System capacity, air traffic control displays will not only have to show the increasing number of aircraft, but also all the associated data such as airspeed and altitude. The representation of aircraft data and associated relational information, often superimposed on a map, leads to cluttered displays, which could negatively affect controller performance, especially as aircraft numbers increase. To investigate these issues further, an experiment was conducted that examined the effect of increasing data-block lines on controller performance in an aircraft vectoring task. Data-block design, the primary factor, varied in the number of lines displayed (2-5). In addition a data-block information priority factor was examined that addressed the frequency of information access across data-block lines. Results demonstrated that while task load, measured as an increasing number of planes under control, negatively influenced reaction times and task accuracy, the number of lines in a data block was not statistically significant. However there was a trend towards reduced performance when data-blocks exceeded more than three lines on a base layer. In addition, the data blocks that contained prioritized information across levels promoted faster reaction times, but at a cost of lower situation awareness. This research demonstrated that the design of data-blocks should consider the balance between reduction in data-block interaction time against the need to allow enough interaction time to build situation awareness. 2008-01-01T00:00:00Z https://hdl.handle.net/1721.1/90281 Design and Evaluation of Path Planning Decision Support for Planetary Surface Exploration Marquez, J. J.; Cummings, M.L. Human intent is an integral part of real-time path planning and re-planning, thus any decision aiding system must support human-automation interaction. The appropriate balance between humans and automation for this task has previously not been adequately studied. In order to better understand task allocation and collaboration between humans and automation for geospatial path problem solving, a prototype path planning aid was developed and tested. The focus was human planetary surface exploration, a high risk, time-critical domain, but the scenario is representative of any domain where humans path plan across uncertain terrain. Three visualizations, including elevation contour maps, a novel visualization called levels of equal costs, and a combination of the two were tested along with two levels of automation. When participants received the lower level of automation assistance, their path costs errors were less than 35% of the optimal, and they integrated manual sensitivity analysis strategies. When participants used the higher level of automation assistance, path costs errors were reduced to a few percentages, and they saved on average 1.5 minutes in the task. However, this increased performance came at the price of decreased situation awareness and automation bias. 2008-01-01T00:00:00Z https://hdl.handle.net/1721.1/90280 Predicting Controller Capacity in Remote Supervision of Multiple Unmanned Vehicles Cummings, M.L.; Mitchell, P. J. In the future vision of allowing a single operator to remotely control multiple unmanned vehicles, it is not well understood what cognitive constraints limit how many vehicles and related tasks a single operator can manage. This paper illustrates that when predicting the number of unmanned aerial vehicles (UAVs) a single operator can control, it is important to model the sources of wait times caused by human-vehicle interaction, especially since these times could potentially lead to system failure. Specifically, these sources of vehicle wait times include cognitive reorientation and interaction wait time, queues for multiple vehicle interactions, and loss of situation awareness wait times. When wait times were included, predictions using a multiple homogeneous and independent UAV simulation dropped by up to 67%, with loss of situation awareness as the primary source of wait time delays. Moreover this study demonstrated that even in a highly automated management-by-exception system, which should alleviate queuing and interaction wait times, operator capacity is still affected by situation awareness wait time, causing a 36% decrease over the capacity model with no wait time included. 2008-01-01T00:00:00Z https://hdl.handle.net/1721.1/90279 The Impact of Heterogeneity on Operator Performance in Future Unmanned Vehicle Systems Nehme, C. E.; Meckeci, B.; Crandall, J. W.; Cummings, M.L. Recent studies have shown that with appropriate operator decision support and with sufficient automation, inverting the multiple operators to single-unmanned vehicle control paradigm is possible. These studies, however, have generally focused on homogeneous teams of vehicles, and have not completely addressed either the manifestation of heterogeneity in vehicle teams, or the effects of heterogeneity on operator capacity. An important implication of heterogeneity in unmanned vehicle teams is an increase in the diversity of possible team configurations available for each operator, as well as an increase in the diversity of possible attention allocation schemes that can be utilized by operators. To this end, this paper introduces a discrete event simulation (DES) model as a means to model a single operator supervising multiple heterogeneous unmanned vehicles. The DES model can be used to understand the impact of varying both vehicle team design variables (such as team composition) and operator design variables (including attention allocation strategies). The model also highlights the sub-components of operator attention allocation schemes that can impact overall performance when supervising heterogeneous unmanned vehicle teams. Results from an experimental case study are then used to validate the model, and make predictions about operator performance for various heterogeneous team configurations. 2008-01-01T00:00:00Z https://hdl.handle.net/1721.1/90278 Auditory Decision Aiding in Supervisory Control of Multiple Unmanned Aerial Vehicles Donmez, B.; Cummings, M.L.; Graham, H. D. This paper investigates the effectiveness of sonification, continuous auditory alert mapped to the state of a monitored task, in supporting unmanned aerial vehicle (UAV) supervisory control. Background: UAV supervisory control requires monitoring each UAV across multiple tasks (e.g., course maintenance) via a predominantly visual display, which currently is supported with discrete auditory alerts. Sonification has been shown to enhance monitoring performance in domains such as anesthesiology by allowing an operator to immediately determine an entity's (e.g., patient) current and projected states, and is a promising alternative to discrete alerts in UAV control. However, minimal research compares sonification to discrete alerts, and no research assesses the effectiveness of sonification for monitoring multiple entities (e.g., multiple UAVs). Method: An experiment was conducted with 39 military personnel, using a simulated setup. Participants controlled single and multiple UAVs, and received sonifications or discrete alerts based on UAV course deviations and late target arrivals. Results: Regardless of the number of UAVs supervised, the course deviation sonification resulted in 1.9 s faster reactions to course deviations, a 19% enhancement from discrete alerts. However, course deviation sonification interfered with the effectiveness of discrete late arrival alerts in general, and with operator response to late arrivals when supervising multiple vehicles. Conclusions: Sonifications can outperform discrete alerts when designed to aid operators to predict future states of monitored tasks. However, sonifications may mask other auditory alerts, and interfere with other monitoring tasks that require divided attention. 2009-01-01T00:00:00Z https://hdl.handle.net/1721.1/90277 Paying Attention to the Man behind the Curtain Cummings, M.L.; Thornburg, K. M. In the push to develop smart energy systems, designers have increasingly focused on systems that measure and predict user behavior to effect optimal energy consumption. While such focus is an important component in these systems' success, designers have paid substantially less attention to the people on the other side of the energy system loop-the supervisors of power generation processes. Smart energy systems that leverage pervasive computing could add to these supervisory control operators' workload. They'll have to predict possible power plant load and production changes caused by environmental and plant events, as well as dynamic system adaptation in response to consumer behaviors. Contrary to many assumptions, inserting more automation, including distributed sensors and algorithms to postprocess data, won't necessarily reduce operators' workload or improve system performance. 2011-03-01T00:00:00Z https://hdl.handle.net/1721.1/87062 The Role of Human-Automation Consensus in Multiple Unmanned Vehicle Scheduling Cummings, M.L.; Clare, A.; Hart, C. Objective: This study examined the impact of increasing automation replanning rates on operator performance and workload when supervising a decentralized network of heterogeneous unmanned vehicles. Background: Futuristic unmanned vehicles systems will invert the operator-to-vehicle ratio so that one operator can control multiple dissimilar vehicles, connected through a decentralized network. Significant human-automation collaboration will be needed due to automation brittleness, but such collaboration could cause high workload. Method: Three increasing levels of replanning were tested on an existing, multiple unmanned vehicle simulation environment that leverages decentralized algorithms for vehicle routing and task allocation, in conjunction with human supervision. Results: Rapid replanning can cause high operator workload, ultimately resulting in poorer overall system performance. Poor performance was associated with a lack of operator consensus for when to accept the automation‟s suggested prompts for new plan consideration, as well as negative attitudes towards unmanned aerial vehicles in general. Participants with video game experience tended to collaborate more with the automation, which resulted in better performance. Conclusion: In decentralized unmanned vehicle networks, operators who ignore the automation‟s requests for new plan consideration and impose rapid replans both increase their own workload and reduce the ability of the vehicle network to operate at its maximum capacity. Application: These findings have implications for personnel selection and training for futuristic systems involving human collaboration with decentralized algorithms embedded in networks of autonomous systems. 2010-01-01T00:00:00Z https://hdl.handle.net/1721.1/87061 Global vs. local decision support for multiple independent UAV schedule management Cummings, M.l; Brezezinski, A. As unmanned aerial vehicles (UAVs) become increasingly autonomous, time-critical and complex single-operator systems will require advance prediction and mitigation of schedule conflicts. However, actions that mitigate a current schedule conflict may create future schedule problems. Decision support is needed allowing an operator to evaluate different mission schedule management options in real-time. This paper describes two decision support visualisations for single-operator supervisory control of four independent UAVs performing a time-critical targeting mission. A configural display common to both visualisations, called StarVis, graphically depicts current schedule problems, as well as projections of potential local and global schedule problems. Results from an experiment showed that subjects using the locally optimal StarVis implementation had better performance, higher situational awareness, and no significant increase in workload over a more globally optimal implementation of StarVis. This research effort highlights how the same decision support design applied at different abstraction levels can produce different performance results. 2010-01-01T00:00:00Z https://hdl.handle.net/1721.1/87060 Supporting Intelligent and Trustworthy Maritime Path Planning Decisions Cummings, M.L.; Buchin, M.; Carrigan, G.; Donmez, B. The risk of maritime collisions and groundings has dramatically increased in the past five years despite technological advancements such as GPS-based navigation tools and electronic charts which may add to, instead of reduce, workload. We propose that an automated path planning tool for littoral navigation can reduce workload and improve overall system efficiency, particularly under time pressure. To this end, a Maritime Automated Path Planner (MAPP) was developed, incorporating information requirements developed from a cognitive task analysis, with special emphasis on designing for trust. Human-in-the-loop experimental results showed that MAPP was successful in reducing the time required to generate an optimized path, as well as reducing path lengths. The results also showed that while users gave the tool high acceptance ratings, they rated the MAPP as average for trust, which we propose is the appropriate level of trust for such a system. 2010-01-01T00:00:00Z https://hdl.handle.net/1721.1/87059 Modeling Workload Impact in Multiple Unmanned Vehicle Supervisory Control Donmez, B.D.; Nehme, C.; Cummings, M.L. Discrete event simulations for futuristic unmanned vehicle (UV) systems enable a cost and time effective methodology for evaluating various autonomy and human automation design parameters. Operator mental workload is an important factor to consider in such models. We present that the effects of operator workload on system performance can be modeled in such a simulation environment through a quantitative relation between operator attention and utilization, i.e., operator busy time used as a surrogate real-time workload measure. In order to validate our model, a heterogeneous UV simulation experiment was conducted with 74 participants. Performance based measures of attention switching delays were incorporated in the discrete event simulation model via UV wait times due to operator attention inefficiencies (WTAI). Experimental results showed that WTAI is significantly associated with operator utilization (UT), such that high UT levels correspond to higher wait times. The inclusion of this empirical UT-WTAI relation in the discrete event simulation model of multiple UV supervisory control resulted in more accurate replications of data, as well as more accurate predictions for alternative UV team structures. These results have implications for the design of future human-UV systems, as well as more general multiple task supervisory control models. 2010-01-01T00:00:00Z https://hdl.handle.net/1721.1/87037 Human-Automation Collaboration in Complex Multivariate Resource Allocation Decision Support Systems Cummings, M.L.; Bruni, S. In resource allocation problems for systems with moving planning horizons and significant uncertainty, typical of supervisory control environments, it is critical that some balance of human-automation collaboration be achieved. These systems typically require leveraging the computational power of automation, as well as the experience and judgment of human decision makers. Human-automation collaboration can occur through degrees of collaboration from automation-centric to human-centric, and such collaboration is inherently distinct from previously-discussed levels of automation. In the context of a command and control mission planning task, we show that across a number of metrics, there is no clear dominant human-automation collaboration scheme for resource allocation problems using three distinct instantiations of human-automation collaboration. Rather, the ultimate selection for the best resource allocation decision support system will depend on a cost-benefit approach that could include mitigation of workload, conformance to intended design characteristics, as well as the need to maximize overall mission performance. 2010-01-01T00:00:00Z https://hdl.handle.net/1721.1/87027 Supervised vs. Unsupervised Learning for Operator State Modeling in Unmanned Vehicle Settings Boussemart, Y.; Cummings, M.L.; Las Fargeas, J.; Roy, N. In this paper, we model operator states using hidden Markov models applied to human supervisory control behaviors. More specifically, we model the behavior of an operator of multiple heterogeneous unmanned vehicle systems. The hidden Markov model framework allows the inference of higher operator states from observable operator interaction with a computer interface. For example, a sequence of operator actions can be used to compute a probability distribution of possible operator states. Such models are capable of detecting deviations from expected operator behavior as learned by the model.The difficulty with parametric inference models such as hidden Markov models is that a large number of parameters must either be specified by hand or learned from example data.We compare the behavioral models obtained with two different supervised learning techniques and an unsupervised hidden Markov model training technique. The results suggest that the best models of human supervisory control behavior are obtained through unsupervised learning. We conclude by presenting further extensions to this work. 2011-01-01T00:00:00Z https://hdl.handle.net/1721.1/87020 Operator Choice Modeling for UAV Visual Search Tasks Bertuccelli, L.F.; Cummings, M.L. Unmanned aerial vehicles (UAVs) provide unprecedented access to imagery of possible ground targets of interest in real time. The availability of this imagery is expected to increase with envisaged future missions of one operator controlling multiple UAVs. This research investigates decision models that can be used to develop assistive decision support for UAV operators involved in these complex search missions. Previous human-in-the-loop experiments have shown that operator detection probabilities may decay with increased search time. Providing the operators with the ability to requeue difficult images with the option of relooking at targets later was hypothesized to help operators improve their search accuracy. However, it was not well understood how mission performance could be impacted by operators performing requeues with multiple UAVs. This work extends a queuing model of the human operator by developing a retrial queue model (ReQM) that mathematically describes the use of relooks. We use ReQM to generate performance predictions through discrete event simulation. We validate these predictions through a human-in-the-loop experiment that evaluates the impact of requeuing on a simulated multiple-UAV mission. Our results suggest that, while requeuing can improve detection accuracy and decrease mean search times, operators may need additional decision support to use relooks effectively. 2012-09-01T00:00:00Z https://hdl.handle.net/1721.1/86958 Predictive models of human supervisory control behavioral patterns using hidden semi-Markov models Boussemart, Y.; Cummings, M.L. Behavioral models of human operators engaged in complex,time-critical high-risk domains, such as those typical in Human Supervisory Control (HSC) settings, are of great value because of the high cost of operator failure. We propose that Hidden Semi-Markov Models (HSMMs) can be employed to model behaviors of operators in HSC settings where there is some intermittent human interaction with a system via a set of external controls. While regular Hidden Markov Models (HMMs) can be used to model operator behavior, HSMMs are particularly suited to time-critical supervisory control domains due to their explicit representation of state duration. Using HSMMs,we demonstrate in an unmanned vehicle supervisory control environment that such models can accurately predict future operator behavior both in terms of states and durations. 2011-01-01T00:00:00Z https://hdl.handle.net/1721.1/86954 Human-Automation Path Planning Optimization and Decision Support Cummings, M.L.; Marquez, J.J.; Roy, N. Path planning is a problem encountered in multiple domains, including unmanned vehicle control, air traffic control, and future exploration missions to the Moon and Mars. Due to the voluminous and complex nature of the data, path planning in such demanding environments requires the use of automated planners. In order to better understand how to support human operators in the task of path planning with computer aids, an experiment was conducted with a prototype path planner under various conditions to assess the effect on operator performance. Participants were asked to create and optimize paths based on increasingly complex path cost functions, using different map visualizations including a novel visualization based on a numerical potential field algorithm. They also planned paths under degraded automation conditions. Participants exhibited two types of analysis strategies, which were global path regeneration and local sensitivity analysis. No main effect due to visualization was detected, but results indicated that the type of optimizing cost function affected performance, as measured by metabolic costs, sun position, path distance and task time. Unexpectedly, participants were able to better optimize more complex cost functions as compared to a simple time-based cost function. 2011-01-01T00:00:00Z https://hdl.handle.net/1721.1/86950 ,The Impact of Human-Automation Collaboration in Decentralized Multiple Unmanned Vehicle Control Cummings, M.L.; How, J.; Whitten, A.; Toupet, O. For future systems that require one or a small team of operators to supervise a network of automated agents, automated planners are critical since they are faster than humans for path planning and resource allocation in multivariate, dynamic, time-pressured environments. However, such planners can be brittle and unable to respond to emergent events. Human operators can aid such systems by bringing their knowledge-based reasoning and experience to bear. Given a decentralized task planner and a goal-based operator interface for a network of unmanned vehicles in a search, track, and neutralize mission, we demonstrate with a human-on-the-loop experiment that humans guiding these decentralized planners improved system performance by up to 50%. However, those tasks that required precise and rapid calculations were not significantly improved with human aid. Thus, there is a shared space in such complex missions for human–automation collaboration. 2012-01-01T00:00:00Z https://hdl.handle.net/1721.1/86948 Collaborative Exploration with a Micro Aerial Vehicle: A Novel Interaction Method for Controlling a MAV with a Hand-Held Device Pitman, D.; Cummings, M.L. In order to collaboratively explore an environment with a Micro Aerial Vehicle (MAV), an operator needs a mobile interface, which can support the operator’s divided attention. To this end, we developed the Micro Aerial Vehicle Exploration of an Unknown Environment (MAV-VUE) interface, which allows operators with minimal training the ability to remotely explore their environment with a MAV. MAV-VUE employs a concept we term Perceived First-Order (PFO) control, which allows an operator to effectively “fly” a MAV with no risk to the vehicle. PFO control utilizes a position feedback control loop to fly the MAV while presenting rate feedback to the operator. A usability study was conducted to evaluate MAV-VUE. This interface was connected remotely to an actual MAV to explore a GPS-simulated urban environment. 2012-01-01T00:00:00Z https://hdl.handle.net/1721.1/86946 Assessing Operator Strategies for Real-time Replanning of Multiple Unmanned Vehicles Clare, A.S.; Maere, P.C.P.; Cummings, M.L. Future unmanned vehicles systems will invert the operator-to-vehicle ratio so that one operator controls a decentralized network of heterogeneous unmanned vehicles. This study examines the impact of allowing an operator to adjust the rate of prompts to view automation-generated plans on system performance and operator workload. Results showed that the majority of operators chose to adjust the replan prompting rate. The initial replan prompting rate had a significant framing effect on the replan prompting rates chosen throughout a scenario. Higher initial replan prompting rates led to significantly lower system performance. Operators successfully self-regulated their task-switching behavior to moderate their workload. 2012-01-01T00:00:00Z https://hdl.handle.net/1721.1/86945 Field Testing of a Quad Rotor Smartphone Control System Cummings, M.L.; Jackson, K.; Quimby, P.; Pitman, D. With recent regulatory efforts to reduce restrictions placed on the operation of Micro Air Vehicles (MAVs) in the United States, it is likely that in the next few years, these vehicles will become commonplace in the commercial marketplace as they are in military environments. In order to reduce the barrier to entry for operations of MAVs, customers of these systems will require ease of operation as well as minimal training time in order to reduce costs. To this end, a smartphone application was developed to control a quadrotor remotely in the exploration of an unknown environment, and tested for users with only three minutes of training. Initial motion capture room tests produced encouraging results for localization and target identification tasks, however, such environments are inherently artificial and the extensibility of such results is limited. A follow-on outdoor field study was conducted in order to compare the indoor and outdoor results and to assess operator performance in a realistic environment. Performance on the outdoor localization tasks was comparable to the indoor study, however, participants generally performed slightly worse on the target identification task in the outdoor experiment, attributed to camera image quality and GPS localization issues. Other issues such as wind and flight safety considerations are discussed. 2012-01-01T00:00:00Z https://hdl.handle.net/1721.1/86942 Boredom and Distraction in Multiple Unmanned Vehicle Supervisory Control Cummings, M.L.; Mastracchio, C.; Thornburg, K.M.; Mkrtchyan, A. Operators currently controlling Unmanned Aerial Vehicles report significant boredom, and such systems will likely become more automated in the future. Similar problems are found in process control, commercial aviation, and medical settings. To examine the effect of boredom in such settings, a long duration low task load experiment was conducted. Three low task load levels requiring operator input every 10, 20, or 30 minutes were tested in a our-hour study using a multiple unmanned vehicle simulation environment that leverages decentralized algorithms for sometimes imperfect vehicle scheduling. Reaction times to system-generated events generally decreased across the four hours, as did participants’ ability to maintain directed attention. Overall, participants spent almost half of the time in a distracted state. The top performer spent the majority of time in directed and divided attention states. Unexpectedly, the second-best participant, only 1% worse than the top performer, was distracted almost one third of the experiment, but exhibited a periodic switching strategy, allowing him to pay just enough attention to assist the automation when needed. Indeed, four of the five top performers were distracted more than one-third of the time. These findings suggest that distraction due to boring, low task load environments can be effectively managed through efficient attention switching. Future work is needed to determine optimal frequency and duration of attention state switches given various exogenous attributes, as well as individual variability. These findings have implications for the design of and personnel selection for supervisory control systems where operators monitor highly automated systems for long durations with only occasional or rare input. 2013-01-01T00:00:00Z https://hdl.handle.net/1721.1/86940 Unloved Aerial Vehicles: Gutting its UAV plan, the Air Force sets a course for irrelevance Spinetta, L.; Cummings, M.L. This month's cover article challenges Air Force leader on the subject of unmanned aerial vehicles. Lt. Col. Lawrence Spinetta and Missy Cummings see disaster, or at least a long-term slide into irrelevance, in recent decisions that appear to under value UAVs in caparison with manned fighters and bombers. 2012-11-01T00:00:00Z https://hdl.handle.net/1721.1/86937 Shared Authority Concerns in Automated Driving Applications Cummings, M.L.; Ryan, J.C. Given the move toward driverless cars, which includes the more short-term goal of driving assistance, what the appropriate shared authority and interaction paradigms should be between human drivers and the automation remains an open question until more principled research and testing has occurred. It is unclear at this time how robust driverless cars are to system failures (including human failures) and operations in degraded sensor environments. Automation onboard such vehicles is inherently brittle and can only account for what it is programmed to consider. Communication between what is technically a very complex system to a human population of extreme variability in driving skills and attention management will be difficult, since the driver will need to be appropriately informed of the state of the system, including limitations, and will need to build appropriate trust in the automation’s capabilities (neither too much or too little). Further complicating this problem is the significant body of research demonstrating that automated systems can lead to boredom, which encourages distraction. This leaves operators unaware of the state of the vehicle (aka, mode confusion) and ill-suited to respond quickly and appropriately in case of a potential accident. Over time, operator skill degradation due to automation use can further reduce the human ability to respond to emergent driving demands, and will likely lead to risk homeostasis even in normal operations. Each of these issues are well-known to the human systems engineering community, but it is unclear that these issues are being considered by driverless car designers or that manufactures are conducting human-in-the-loop tests with representative members of the driving population. Until these tests show that the vehicles account for the aforementioned issues, driverless cars will not be safe for unrestricted access and use on U.S. roadways. Moreover, there are significant socio-technical considerations that do not appear to be a concern in the push to introduce this technology on a wide scale. The utilitarian approach quoted by many in the press, i.e., that driverless cars will eventually kill people but that this should be acceptable due to the likely reduction in overall deaths (which is not yet proven) demonstrates an insensitivity to a deontological perspective that causes many people to be uncomfortable with such a significant shift in responsibility and accountability to computers. 2014-05-13T00:00:00Z https://hdl.handle.net/1721.1/86936 Operator Objective Function Guidance for a Real-time Unmanned Vehicle Scheduling Algorithm Clare, A.S.; Cummings, M.L.; How, J.; Whitten, A.; Toupet, O. Advances in autonomy have made it possible to invert the typical operator-to-unmanned-vehicle ratio so that asingle operator can now control multiple heterogeneous unmanned vehicles. Algorithms used in unmanned-vehicle path planning and task allocation typically have an objective function that only takes into account variables initially identified by designers with set weightings. This can make the algorithm seemingly opaque to an operator and brittle under changing mission priorities. To address these issues, it is proposed that allowing operators to dynamically modify objective function weightings of an automated planner during a mission can have performance benefits. A multiple-unmanned-vehicle simulation test bed was modified so that operators could either choose one variable or choose any combination of equally weighted variables for the automated planner to use in evaluating mission plans. Results from a human-participant experiment showed that operators rated their performance and confidence highest when using the dynamic objective function with multiple objectives. Allowing operators to adjust multiple objectives resulted in enhanced situational awareness, increased spare mental capacity, fewer interventions to modify the objective function, and no significant differences in mission performance. Adding this form of flexibility and transparency to automation in future unmanned vehicle systems could improve performance, engender operator trust, and reduce errors. 2012-12-01T00:00:00Z https://hdl.handle.net/1721.1/46752 Integrating Multiple Alarms & Driver Situation Awareness Cummings, M. L.; Wang, Enlie; Ho, Angela W. L. This study addresses this gap in CAS and intelligent alarm research by examining whether or not a single master alarm warning versus multiple warnings for the different collision warning systems conveys adequate information to the drivers. Intelligent driver warning systems signaling impending frontal and rear collisions, as well as unintentional lane departures were used in this experiment, and all the warnings were presented to drivers through the auditory channel only. We investigated two critical research questions in this study: 1. Do multiple intelligent alarms as opposed to a single master alarm affect drivers’ recognition, performance, and action when they experience a likely imminent collision and unintentional lane departure? 2. Is driver performance and overall situation awareness under the two different alarm alerting schemes affected by reliabilities of the warning systems? 2006-01-01T00:00:00Z https://hdl.handle.net/1721.1/46751 Mitigation of Human Supervisory Control Wait Times through Automation Strategies Mitchell, P. J.; Cummings, M. L.; Sheridan, T. B. The application of network centric operations principles to human supervisory control (HSC) domains means that humans are increasingly being asked to manage multiple simultaneous HSC processes. However, increases in the number of available information sources, volume of information and operational tempo, all which place higher cognitive demands on operators, could become constraints limiting the success of network centric processes. In time-pressured scenarios typical of networked command and control scenarios, efficiently allocating attention between a set of dynamic tasks is crucial for mission success. Inefficient attention allocation leads to system wait times, which could eventually lead to critical events such as missed times on targets and degraded overall mission success. One potential solution to mitigating wait times is the introduction of automated decision support in order to relieve operator workload. However, it is not obvious what automated decision support is appropriate, as higher levels of automation may result in a situation awareness decrement and other problems typically associated with excessive automation such as automation bias. To assess the impact of increasing levels of automation on human and system performance in a time-critical HSC multiple task management context, an experiment was run in which an operator simultaneously managed four highly autonomous unmanned aerial vehicles (UAVs) executing an air tasking order, with the overall goal of destroying a pre-determined set of targets within a limited time period. Four increasing levels automated decision support were investigated as well as high and low operational replanning tempos. The highest level of automation, management-byexception, had the best performance across several metrics but had a greater number of catastrophic events during which a UAV erroneously destroyed a friendly target. Contrary to expectations, the collaborative level of decision support, which provided predictions for possible periods of task overload as well as possible courses of action to relieve the high workload, produced the worst performance. This is attributable to an unintended consequence of the automation where the graphical visualization of the computer’s predictions caused users to try to globally optimize the schedules for all UAVs instead of locally optimizing schedules in the immediate future, resulting in them being overwhelmed. Total system wait time across both experimental factors was dominated by wait time caused by lack of situation awareness, which is difficult to eliminate, implying that there will be a clear upper limit on the number of vehicles that any one person can supervise because of the need to stay cognitively aware of unfolding events. 2005-01-01T00:00:00Z https://hdl.handle.net/1721.1/46750 Conceptual Human-System Interface Design for a Lunar Access Vehicle Smith, Cristin; Essama, Stephane; Duppen, Mark; Marquez, Jessica; Cummings, Mary; Wang, Enlie In support of the vision for humans to establish a large scale, economically viable, permanent human settlement on the Moon within the next 25 years (Space Frontier Foundation, 2005), the next generation lunar landing vehicle must be capable of achieving pinpoint, anytime, anywhere safe landing on the lunar surface with high precision (10- 100m). In addition, this vehicle should support both autonomous and manned lunar missions (NASA ASO-1160). Because of advances in technology over the past thirty-five years since the Apollo landings, the role of the human and automated systems in a new lunar lander system must be reevaluated and redesigned. This report details the design approach and resultant preliminary, conceptual design concepts for a Human-System Interface (H-SI) for a Lunar Access Vehicle (LAV). 2005-01-01T00:00:00Z https://hdl.handle.net/1721.1/46749 Decision Support Design for Workload Mitigation in Human Supervisory Control of Multiple Unmanned Aerial Vehicles Brzezinski, A. S.; Cummings, M. L. As UAVs become increasingly autonomous, the multiple personnel currently required to operate a single UAV may eventually be superseded by a single operator concurrently managing multiple UAVs. Instead of lower-level tasks performed by today’s UAV teams, the sole operator would focus on high-level supervisory control tasks such as monitoring mission timelines and reacting to emergent mission events. A key challenge in the design of such single-operator systems will be the need to minimize periods of excessive workload that could arise when critical tasks for several UAVs occur simultaneously. To a certain degree, it is possible to predict and mitigate such periods in advance. However, actions that mitigate a particular period of high workload in the short term may create long term episodes of high workload that were previously non-existent. Thus some kind of decision support is needed that facilitates an operator’s ability to evaluate different options for managing a mission schedule in real-time. This paper describes two decision support visualizations designed for supervisory control of four UAVs performing a time-critical targeting mission. A configural display common to both visualizations, named the StarVis, was designed to highlight potential periods of high workload corresponding to the current mission timeline, as well as “what if” projections of possible high workload periods based upon different operator options. The first visualization design allows an operator to compare different high workload mitigation options for individual UAVs. This is termed the local visualization. The second visualization is indicates the combined effects of multiple high workload mitigation decisions on the timeline. This is termed the global visualization. The main advantage of the local visualization is that options can be compared directly; however, the possible effects of these options on the mission timeline are only indicated for the individual UAV primarily affected by the decision. For the global visualization, different decisions can be combined to show possible effects on the system propagated across all UAVs, but the different alternatives of a single decision option alternative cannot be directly compared. An experiment was conducted testing these visualizations against a control with no visualization. Results showed that subject using the local visualization had better performance, higher situational awareness, and no significant increase in workload over the other two experimental conditions. This occurred despite the fact that the local and global StarVis displays were very similar. Not only did the Global StarVis produce degraded results as compared to the local StarVis, but those participants with no visualization performed as well as those with the global StarVis. This disparity in performance despite strong visual similarities in the StarVis designs is attributed to operators’ inability to process all the information presented in the global StarVis as well as the fact that participants with the local StarVis were able to rapidly develop effective cognitive problem strategies. This research effort highlights a very important design consideration, in that a single decision support design can produce very different performance results when applied at different levels of abstraction. 2006-01-01T00:00:00Z https://hdl.handle.net/1721.1/46748 An Analysis of Information Complexity in Air Traffic Control Human Machine Interaction Tsonis, C. G. This thesis proposes, develops and validates a methodology to quantify the complexity of air traffic control (ATC) human-machine interaction (HMI). Within this context, complexity is defined as the minimum amount of information required to describe the human machine interaction process in some fixed description language and chosen level of detail. The methodology elicits human information processing via cognitive task analysis (CTA) and expresses the HMI process algorithmically as a cognitive interaction algorithm (CIA). The CIA is comprised of multiple functions which formally describe each of the interaction processes required to complete a nominal set of tasks using a certain machine interface. Complexities of competing interface and task configurations are estimated by weighted summations of the compressed information content of the associated CIA functions. This information compression removes descriptive redundancy and approximates the minimum description length (MDL) of the CIA. The methodology is applied to a representative en-route ATC task and interface, and the complexity measures are compared to performance results obtained experimentally by human-in-the-loop simulations. It is found that the proposed complexity analysis methodology and resulting complexity metrics are able to predict trends in operator performance and workload. This methodology would allow designers and evaluators of human supervisory control (HSC) interfaces the ability to conduct complexity analyses and use complexity measures to more objectively select between competing interface and task configurations. Such a method could complement subjective interface evaluations, and reduce the amount of costly experimental testing. 2006-01-01T00:00:00Z https://hdl.handle.net/1721.1/46746 A Framework for an HSI Downselection Tool Cunio, Phillip M.; Cummings, M. L. This technical report describes the concept and development of SITHE, the Systems Integration Tool for HSI Evaluation. SITHE is a framework for selecting tools to be used in evaluating complex technical systems in terms of Human-Systems Integration, or HSI. HSI, or Human-Systems Integration, is the process of integrating people, technology, and an organization at a systems level, with full consideration given to the human requirements of the user (Booher, 2003). HSI focuses on the human aspects of system definition, development, and deployment, and integrates considerations related to personnel, training, human factors, habitability, and other human-related concerns into the overall systems acquisition process (US Department of Defense, 2004). HSI is a field of interest to researchers in academia and industry because, although systems continue to grow more complex, they have not achieved the level of autonomy that would permit them to operate successfully without humans either in or on the loop. Humans are still an essential component of most complex systems, especially when the context of operation for the complex system is subject to uncertainty, as in military applications. However, HSI as a broad field can encompass a large number of types of interaction between humans and systems, including but not necessarily limited to supervisory control, mechanics and ergonomics of control operation, and visualization and decision support. The universe of tools for HSI (including hardware, software, processes, and techniques used to evaluate HSI aspects of complex systems) is already large and growing quickly. Many HSI tools are developed for research purposes only, or in an ad-hoc fashion for specific projects, and as such there is no such thing as a standard catalogue of HSI tools. In addition, the need to consider downstream competencies such as flexibility, robustness, and usability, is increasing as HSI systems become more complex. Thus the HSI cost-benefit trade space is ever increasing, making it difficult for decision makers to determine if and to what degree a system actually meets some pre-specified HSI criteria. 2009-01-01T00:00:00Z https://hdl.handle.net/1721.1/46745 Modified Cooper Harper Scales for Assessing Unmanned Vehicle Displays Graham, Hudson; Cummings, M. L.; Donmez, Birsen; Brzezinski, A. S. In unmanned vehicle (UV) operations, displays are often the only information link between operators and vehicles. It is essential these displays present information clearly and efficiently so that operators can interact with the UVs to achieve mission objectives. While there are a variety of metrics to evaluate displays, there is no current standardized methodology for operators to subjectively assess a display’s support and identify specific deficiencies. Such a methodology could improve current displays and ensure that displays under development support operator processes. This report presents a quasi- subjective display evaluation tool called the Modified Cooper-Harper for Unmanned Vehicle Displays (MCH-UVD) diagnosis tool. This tool, adapted from the Cooper-Harper aircraft handling scale, allows operators to assess a display, translating their judgments on potential display shortcomings into a number corresponding to a particular deficiency in operator support. The General MCH-UVD can be used to diagnose deficiencies for any UV display, while the Specific MCH-UVD is UV and mission specific in its evaluation of displays. This report presents the General MCH-UVD and provides guidance on how to adapt it to create a Specific MCH-UVD through the use of UV mission taxonomies and a questioning method. A UGV search mission case study provides a how-to guide example for generating a Specific MCH-UVD. The report also presents an experiment conducted to validate the MCH-UVD and assess if a mission-specific version is necessary, or if the general form of the MCH-UVD is sufficient for different UV display evaluation. The report concludes with discussion on how to administer the scale, when a Specific scale is necessary, MCH-UVD diagnosis tool limitations, and future work. 2008-01-01T00:00:00Z https://hdl.handle.net/1721.1/46744 A Predictive Model for Human-Unmanned Vehicle Systems : Final Report Crandall, J. W.; Cummings, M. L. Advances in automation are making it possible for a single operator to control multiple unmanned vehicles (UVs). This capability is desirable in order to reduce the operational costs of human-UV systems (HUVS), extend human capabilities, and improve system effectiveness. However, the high complexity of these systems introduces many significant challenges to system designers. To help understand and overcome these challenges, high-fidelity computational models of the HUVS must be developed. These models should have two capabilities. First, they must be able to describe the behavior of the various entities in the team, including both the human operator and the UVs in the team. Second, these models must have the ability to predict how changes in the HUVS and its mission will alter the performance characteristics of the system. In this report, we describe our work toward developing such a model. Via user studies, we show that our model has the ability to describe the behavior of a HUVS consisting of a single human operator and multiple independent UVs with homogeneous capabilities. We also evaluate the model’s ability to predict how changes in the team size, the human-UV interface, the UV’s autonomy levels, and operator strategies affect the system’s performance. 2008-01-01T00:00:00Z https://hdl.handle.net/1721.1/46743 Selecting Metrics to Evaluate Human Supervisory Control Applications Cummings, M. L.; Pina, P. E.; Donmez, B. The goal of this research is to develop a methodology to select supervisory control metrics. This methodology is based on cost-benefit analyses and generic metric classes. In the context of this research, a metric class is defined as the set of metrics that quantify a certain aspect or component of a system. Generic metric classes are developed because metrics are mission-specific, but metric classes are generalizable across different missions. Cost-benefit analyses are utilized because each metric set has advantages, limitations, and costs, thus the added value of different sets for a given context can be calculated to select the set that maximizes value and minimizes costs. This report summarizes the findings of the first part of this research effort that has focused on developing a supervisory control metric taxonomy that defines generic metric classes and categorizes existing metrics. Future research will focus on applying cost benefit analysis methodologies to metric selection. Five main metric classes have been identified that apply to supervisory control teams composed of humans and autonomous platforms: mission effectiveness, autonomous platform behavior efficiency, human behavior efficiency, human behavior precursors, and collaborative metrics. Mission effectiveness measures how well the mission goals are achieved. Autonomous platform and human behavior efficiency measure the actions and decisions made by the humans and the automation that compose the team. Human behavior precursors measure human initial state, including certain attitudes and cognitive constructs that can be the cause of and drive a given behavior. Collaborative metrics address three different aspects of collaboration: collaboration between the human and the autonomous platform he is controlling, collaboration among humans that compose the team, and autonomous collaboration among platforms. These five metric classes have been populated with metrics and measuring techniques from the existing literature. Which specific metrics should be used to evaluate a system will depend on many factors, but as a rule-of-thumb, we propose that at a minimum, one metric from each class should be used to provide a multi-dimensional assessment of the human-automation team. To determine what the impact on our research has been by not following such a principled approach, we evaluated recent large-scale supervisory control experiments conducted in the MIT Humans and Automation Laboratory. The results show that prior to adapting this metric classification approach, we were fairly consistent in measuring mission effectiveness and human behavior through such metrics as reaction times and decision accuracies. However, despite our supervisory control focus, we were remiss in gathering attention allocation metrics and collaboration metrics, and we often gathered too many correlated metrics that were redundant and wasteful. This meta-analysis of our experimental shortcomings reflect those in the general research population in that we tended to gravitate to popular metrics that are relatively easy to gather, without a clear understanding of exactly what aspect of the systems we were measuring and how the various metrics informed an overall research question. 2008-01-01T00:00:00Z https://hdl.handle.net/1721.1/46737 Assessing the Impact of Haptic Peripheral Displays for UAV Operators Cummings, M. L.; Donmez, B.; Graham, H. D. Objectives: A pilot study was conducted to investigate the effectiveness of continuous haptic peripheral displays in supporting multiple UAV supervisory control. Background: Previous research shows that continuous auditory peripheral displays can enhance operator performance in monitoring events that are continuous in nature, such as monitoring how well UAVs stay on their pre-planned courses. This research also shows that auditory alerts can be masked by other auditory information. Command and control operations are generally performed in noisy environments with multiple auditory alerts presented to the operators. In order to avoid this masking problem, another potentially useful sensory channel for providing redundant information to UAV operators is the haptic channel. Method: A pilot experiment was conducted with 13 participants, using a simulated multiple UAV supervisory control task. All participants completed two haptic feedback conditions (continuous and threshold), where they received alerts based on UAV course deviations and late arrivals to targets. Results: Threshold haptic feedback was found to be more effective for late target arrivals, whereas continuous haptic feedback resulted in faster reactions to course deviations. Conclusions: Continuous haptic feedback appears to be more appropriate for monitoring events that are continuous in nature (i.e., how well a UAV keeps its course). In contrast, threshold haptic feedback appears to better support response to discrete events (i.e., late target arrivals). Future research: Because this is a pilot study, more research is needed to validate these preliminary findings. A direct comparison between auditory and haptic feedback is also needed to provide better insights into the potential benefits of multi-modal peripheral displays in command and control of multiple UAVs. 2008-01-01T00:00:00Z https://hdl.handle.net/1721.1/46735 Assessing the Impact of Auditory Peripheral Displays for UAV Operators Graham, H. D.; Cummings, M. L. A future implementation of unmanned aerial vehicle (UAV) operations is having a single operator control multiple UAVs. The research presented here explores possible avenues of enhancing audio cues of UAV interfaces for this futuristic control of multiple UAVs by a single operator. This project specifically evaluates the value of continuous and discrete audio cues as indicators of course deviations or late arrivals to targets for UAV missions. It also looks at the value of the audio cues in single and multiple UAV scenarios. To this end, an experiment was carried out on the Multiple Autonomous Unmanned Vehicle Experimental (MAUVE) test bed developed in the Humans and Automation Laboratory at the Massachusetts Institute of Technology with 44 military participants. Specifically, two continuous audio alerts were mapped to two human supervisory tasks within MAUVE. One of the continuous audio alerts, an oscillating course deviation alert was mapped to UAV course deviations which occurred over a continual scale. The other continuous audio alert tested was a modulated late arrival alert which alerted the operator when a UAV was going to be late to a target. In this case the continuous audio was mapped to a discrete event in that the UAV was either on time or late to a target. The audio was continuous in that it was continually on and alerting the participant to the current state of the UAV. It either was playing a tone indicating the UAV was on time to a target or playing a tone indicating the UAV was late to a target. These continuous alerts were tested against more traditional single beep alerts which acted as discrete alerts. The beeps were discrete in that when they were used for monitoring course deviations a single beep was played when the UAV got to specific threshold off of the course or for late arrivals a single beep was played when the UAV became late. The results show that the use of the continuous audio alerts enhances a single operator’s performance in monitoring single and multiple semi-autonomous vehicles. However, the results also emphasize the necessity to properly integrate the continuous audio with the other auditory alarms and visual representations in a display, as it is possible for discrete audio alerts to be lost in aural saliency of continuous audio, leaving operators reliant on the visual aspects of the display. 2007-01-01T00:00:00Z https://hdl.handle.net/1721.1/46734 Information Requirements for MCM and ISR Missions : PUMA Phase II Cummings, M. L.; Massie, A. E.; Nehme, C. E. This document contains display requirements for Littoral Combat Ship (LCS) control station displays to be used by unmanned vehicle units in support of heterogeneous unmanned vehicle missions (such as Special Operations Force (SOF) insertion). The method used for generating the requirements was that of a Hybrid Cognitive Task Analysis (CTA)1 which entails describing a scenario overview of a representative mission, generating event flow diagrams, and depicting decision ladders for the key decisions identified in the event flow diagrams. These steps are then used together to generate an informational requirements summary which includes the situational awareness requirements that are derived from the event flow and display requirements of the decision ladders. This method was developed in Phase I of the PUMA (Plan Understanding for Mixed-initiative control of Autonomous systems) project2. In PUMA I, the mission scenario primarily consisted of Intelligence, Surveillance and Reconnaissance (ISR) tasks. For PUMA II, the scenario has been expanded to include Mine Counter Measures (MCM), Harbor Bottom Image-Mapping (HBI), and Anti- Terrorism / Force Protection (AT/FP) mission types. There is a specific emphasis on the MCM and ISR missions to highlight the informational requirement differences between the two task types. This document incorporates the expanded vehicle and mission type heterogeneities that are present in PUMA II in order to develop a cohesive set of informational requirements necessary for such a complex mission. 2007-01-01T00:00:00Z https://hdl.handle.net/1721.1/46733 An Analysis of Heterogeneity in Futuristic Unmanned Vehicle Systems Nehme, C. E.; Cummings, M. L. Recent studies have shown that with appropriate operator decision support and with enough automation aboard unmanned vehicles, inverting the multiple operators to single-vehicle control paradigm is possible. These studies, however, have generally focused on homogeneous teams of vehicles, and have not completely addressed either the manifestation of heterogeneity in vehicle teams, or the effects of heterogeneity on operator capacity. An important implication of heterogeneity in unmanned vehicle teams is an increase in the diversity of possible team configurations available for each operator, as well as an increase in the diversity of possible attention allocation schemes that can be utilized by operators. To this end, this paper introduces a resource allocation framework that defines the strategies and processes that lead to alternate team configurations. The framework also highlights the sub-components of operator attention allocation schemes that can impact overall performance when supervising heterogeneous unmanned vehicle teams. A subsequent discrete event simulation model of a single operator supervising multiple heterogeneous vehicles and tasks explores operator performance under different heterogeneous team compositions and varying attention allocation strategies. Results from the discrete event simulation model show that the change in performance when switching from a homogeneous team to a heterogeneous one is highly dependent on the change in operator utilization. Heterogeneous teams that result in lower operator utilization can lead to improved performance under certain operator strategies. 2007-01-01T00:00:00Z https://hdl.handle.net/1721.1/46732 Design Methodology for Unmanned Aerial Vehicle (UAV) Team Coordination Cummings, M. L.; da Silva, F. B.; Scott, S. D. Unmanned Aerial Vehicle (UAV) systems, despite having no onboard human pilots, currently require extensive human involvement to accomplish successful mission operations. Further, successful operations also require extensive colalboration between mission stakeholders, including operators, mission commanders, and information consumers (e.g. ground troops relying on intelligence reports in their area). Existing UAV system interfaces provide little to no support for collaboration between remote operators or for operators to collaborate with information consumers. As reliance on UAVs continues to increase in military and civilian operations, this lack of support for collaboration will likely become a substantial limitation of existing UAV systems. In order to introduce effective collaboration support to UAV system interfaces, it is essential to understand, and be able to derive system design requirements that address, the necessary group interactions that occur in UAV task enviroments. However, few collaborative requirements analysis methods exist, and to our knowledge, no method exists that captures design requirements for collaborative decision making in complex, time-critical environments. This report describes the development of a new design requirements analysis method for deriving information and functional requirements that address the collaboration needs of UAV (and other complex task) operators, and the needs of stakeholders interacting with these operators. More specifically, theis method extends a recently developed requirements analysis method, called the Hybrid Cognitive Task Analysis (CTA) method, which enables the generation of information and functional requirements for futuristic UAV system interfaces. The original Hybrid CTA method focused on deriving single user system interface requirements. This work extends this method by introducing analytic steps to identify task and decision-making dependencies between different UAV operations collaborators. This collaborative extension to the Hybrid CTA utilizes the notion of boundary objects, an analytic construct commonly used in the study of group work. Boundary objects are physical or information artifacts that cross the task boundaries between members of distinct groups. Identifying boundary objects in complex task operations help the analyst to identify task and decision-making dependencies between local and remote collaborators. Understanding these dependencies helps to identify information sharing requirements that the UAV system should support. This report describes the analytic steps of the collaborative extension, and provides background information on the original Hybrid CTA method and the boundary object construct. The report also describes a project in which the new design requirements method was used to revise a proposed set of UAV operator displays. 2007-01-01T00:00:00Z https://hdl.handle.net/1721.1/46731 An Experimental Platform for Investigating Decision and Collaboration Technologies in Time-Sensitive Mission Control Operations Scott, S. D.; Cummings, M. L. This report describes the conceptual design and detailed architecture of an experimental platform developed to support investigations of novel decision and collaboration technologies for complex, time-critical mission control operations, such as military command and control and emergency response. In particular, the experimental platform is designed to enable exploration of novel interface and interaction mechanisms to support both human-human collaboration and human-machine collaboration for mission control operations involving teams of human operators engaged in supervisory control of intelligent systems, such as unmanned aerial vehicles (UAVs). Further, the experimental platform is designed to enable both co-located and distributed collaboration among operations team members, as well as between team members and relevant mission stakeholders. To enable initial investigations of new information visualization, data fusion, and data sharing methods, the experimental platform provides a synthetic task environment for a representative collaborative time-critical mission control task scenario. This task scenario involves a UAV operations team engaged in intelligence, surveillance, and reconnaissance (ISR) activities. In the experimental task scenario, the UAV team consists of one mission commander and three operators controlling multiple, homogeneous, semi-autonomous UAVs. In order to complete its assigned missions, the UAV team must coordinate with a ground convoy, an external strike team, and a local joint surveillance and target attack radar system (JSTARS). This report details this task scenario, including the possible simulation events that can occur and the logic governing the simulation dynamics. In order to perform human-in-the-loop experimentation within the synthetic task environment, the experimental platform also consists of a physical laboratory designed to emulate a miniature command center. The Command Center Laboratory comprises a number of large-screen displays, multi-screen operator stations, and mobile, tablet-style devices. This report details the physical configuration and hardware components of this Command Center Laboratory. Details are also provided of the software architecture used to implement the synthetic task environment and experimental interface technologies to facilitate user experiments in this laboratory. The report also summarizes the process of conducting an experiment in the experimental platform, including details of scenario design, hardware and software instrumentation, and participant training. Finally, the report suggests several improvements that could be made to the experimental platform based on insights gained from initial user experiments that have been conducted in this environment. 2007-01-01T00:00:00Z https://hdl.handle.net/1721.1/46730 Assisting Interruption Recovery in Mission Control Operations Wan, J.; Scott, S. D.; Cummings, M. L. Frequent interruptions are commonplace in modern work environments. The negative impacts of interruptions are well documented and include increased task completion and error rates in individual task activities, as well as interference with team coordination in team-based activities. The ramifications of an interruption in mission control operations, such as military command and control and emergency response, can be particularly costly due to the time and life-critical nature of these operations. The negative impacts of interruptions have motivated recent developments in software tools, called interruption recovery tools, which help mitigate the effects of interruptions in a variety of task environments. However, mission control operations introduce particular challenges for the design of these tools due to the dynamic and highly collaborative nature of these environments. To address this issue, this report investigates methods of reducing the negative consequences of interruptions in complex, mission control operations. In particular, this report focuses on supporting interruption recovery for team supervisors in these environments, as the research has shown that supervisors are particularly susceptible to frequent interruptions. Based on the results of a requirements analysis, which involved a cognitive task analysis of a representative mission control task scenario, a new interruption recovery tool, named the Interruption Recovery Assistance (IRA) tool, was developed. In particular, the IRA tool was designed to support a military mission commander overseeing a team of unmanned aerial vehicle (UAV) operators performing ground force protection operations. The IRA tool provides the mission commander a visual summary of mission changes, in the form of an event bookmark timeline. It also provides interactive capabilities to enable the commander to view additional information on the primary task displays when further detail about a particular mission event is needed. The report also presents the findings from a user study that was conducted to evaluate the effectiveness of the IRA tool on interruption recovery during collaborative UAV mission operations. The study produced mixed results regarding the effectiveness of the IRA tool. The statistical analysis indicated a negative impact on recovery time, while indicating a positive impact on decision accuracy, especially in complex task situations. The study also indicated that the effect of the IRA tool varied across differ user populations. In particular, the IRA tool tended to provide greater benefits to participants without military experience, compared to military participants involved in the study. The qualitative findings from the study provided key insights into the impact and utility of the IRA tool. These insights were used to identify several future research and design directions related to interruption recovery in mission control operations. 2007-01-01T00:00:00Z https://hdl.handle.net/1721.1/46729 Designing Decision and Collaboration Support Technology for Operators in Multi-UAV Operations Teams Scott, S. D.; Cummings, M. L.; Almirao, F. M.; da Silva, F. B. Effective team collaboration and timely decision-making significantly influence the outcome of time-sensitive military operations. The increasing complexity introduced by the recent move towards network centric operations (NCO) in U.S. military operations provides additional challenges for efficient decision-making. Future operations will include co-located and distributed teams composed of operators from difference services, often at different global locations. Military operations which require extremely quick decisions, such as operations dealing with time-sensitive targets (TST) like improvised explosive devices (IEDs), are particularly challenging in NCO teaming environments. Operators in TST environments not only have to manage overwhelming amounts of target-related information, but also have the overhead of communicating and coordinating with co-located and distributed team members. Given the increasing trend for modern hostile forces to employ unconventional weapons such as IEDs and suicide bombs, the success of TST operations are becoming critical to current and future military operations. Providing TST teams with effective tools for communicating and coordinating their efforts is key to enabling their success. 2007-01-01T00:00:00Z https://hdl.handle.net/1721.1/46728 Shadow TUAV Single Operator Consolidation : Display Assessment Visser, Mark; Cummings, M. L.; Marquez, Jessica J. Currently, Shadow UAV operations require two people: the Air Vehicle Operator (AVO) and the Mission Payload Operator (MPO). A previous workload study demonstrated that it is possible to combine these two positions such that one person can assume both roles (Appendix A). However, to achieve this consolidation, improved displays in terms of usability and increased automated functionality will be necessary to keep the workload of the single operator to acceptable levels. To demonstrate the types of changes that will need to occur for successful AVO and MPO consolidation, this report focuses on display and automation improvements in the following three areas: systems management, vehicle situation awareness, and payload operations. For each of these areas, a previous display has either been designed or improved upon, always applying human factors design principles. Each of these display redesigns exemplifies how operator workload can be decreased, as well as improve overall mission capability. 2007-01-01T00:00:00Z https://hdl.handle.net/1721.1/46727 A UAV Mission Hierarchy Crandall, J. W.; Nehme, C. E.; Cummings, M. L. In the following sections, each of the primary missions are decomposed into mission planning, management, and replanning segments in order to identify what the primary functions a human operator will need to perform. The goal is to understand what tasks/functions are common across different UAV missions and platforms in order to map the generalizability of any particular research project. 2006-01-01T00:00:00Z https://hdl.handle.net/1721.1/46726 Integrating Automobile Multiple Intelligent Warning Systems : Performance and Policy Implications Ho, Angela Wei Ling Intelligent driver warning systems can be found in many high-end vehicles on the road today, which will likely rapidly increase as they become standard equipment. However, introducing multiple warning systems into vehicles could potentially add to the complexity of the driving task, and there are many critical human factors issues that should be considered, such as how the interaction between alarm alerting schemes, system reliabilities, and distractions combine to affect driving performance and situation awareness. In addition, there are also questions with respect to whether there should be any minimum safety standards set to ensure both functional and usage safety of these systems, and what these standards should be. An experiment was conducted to study how a single master alert versus multiple individual alerts of different reliabilities affected drivers’ responses to different imminent collision situations while distracted. A master alert may have advantages since it reduces the total number of alerts, which could be advantageous especially with the proliferation of intelligent warning systems. However, a master alert may also confuse drivers, since it does not warn of a specific hazard, unlike a specific alert for each warning systems. Auditory alerts were used to warn of imminent frontal and rear collisions, as well as unintentional left and right lane departures. Low and high warning reliabilities were also tested. The different warning systems and reliability factors produced significantly different reaction times and response accuracies. The warning systems with low reliability caused accuracy rates to fall more than 40% across the four warning systems. In addition, low reliability systems also induced negative emotions in participants. Thus, reliability is one of the most crucial determinants of driving performance and the safety outcome, and it is imperative that warning systems are reliable. For the master versus distinct alarms factor, drivers responded statistically no different to the various collision warnings for both reaction times and accuracy of responses. However, in a subjective postexperiment assessment, participants preferred distinct alarms for different driver warning systems, even though their objective performance showed no difference to the different alerting schemes. This study showed that it was essential to design robust and reliable intelligent warning systems. However, there are no existing safety standards today to ensure that these systems are safe before they are introduced into vehicles, even though such systems are already available in high-end cars. Even though there are tradeoffs in having standards, such as increased time-to-market and possible loss of innovation, I recommend that safety standards be set nonetheless, since standards will ensure the safety performance of warning systems, to an extent. In terms of functional safety, safety standards should be performance-based, and should specify a minimum level of reliability. In terms of usage safety, the standards should also be performance-based, where driving performance can be indicated by measures such as reaction time, lane position, heading distance and accuracy of responses. In addition, multiple threat scenarios should also be tested. In terms of design guidelines, the various human factors guidelines from different countries should be harmonized internationally to ensure that manufacturers have access to a consistent set of guidelines. Finally, it is also important that these standards, especially for usage safety, specify tests with not just the average driver, but also with peripheral driving populations including novice and elderly drivers. 2006-01-01T00:00:00Z https://hdl.handle.net/1721.1/46725 Audio Decision Support for Supervisory Control of Unmanned Vehicles : Literature Review Nehme, C. E.; Cummings, M. L. Purpose of this literature review: To survey scholarly articles, books and other sources (dissertations, conference proceedings) relevant to the use of the audio supervisory control of unmanned vehicles. 2006-01-01T00:00:00Z https://hdl.handle.net/1721.1/46724 Multiple Alarms and Driving Situational Awareness Ho, Angela W. L.; Cummings, Mary L. There is increasing interest in actively mitigating safety in vehicles beyond that of improving crash worthiness. According to the National Highway Transportation Safety Administration (NHTSA), there are more than 40,000 deaths on highways each year. This number may be decreasing with increasing active public concern and awareness for the use of safety restraints, but the numbers are still in excess of 40,000 deaths annually. Focusing on crash-worthiness as a measure of safety in vehicles will eventually reach a point of diminishing return, thus there is a need for automotive manufacturers to shift their safety focus to crash avoidance safety systems (Runge, 2002). In the public domain, significant progress and advancements have been made under the Intelligent Vehicles Initiative (IVI) set up by U.S. Department of Transportation to prevent motor vehicle crashes by assisting drivers in avoiding hazardous mistakes (U.S DOT, 1998). One IVI focus area is facilitating the rapid deployment of Collision Avoidance Systems (CAS) in vehicles. Collision Avoidance Systems are a subset of Advanced Vehicle Control Safety Systems (AVCSS) which come under the umbrella of Intelligent Transportation Systems (ITS). These Collision Avoidance Systems warn drivers of imminent collisions and can potentially help to save lives. Primary directions of research in CAS are determining implementation strategies and technologies in vehicles and roadway infrastructure, as well as optimizing the driving performance of different populations of drivers when using CAS. In CAS implementation, vehicles will communicate with other vehicles as well as with the roadway infrastructure via sensors and telecommunication networks. The data obtained can then be used in Collision Avoidance Systems. Vehicle-to-vehicle CAS include warnings that trigger when a vehicle is about to collide with another vehicle. Examples include Frontal Warning, Rear Warning and Blind Spot Detection Warnings. Vehicle-to-infrastructure CAS include warnings that trigger when a vehicle is about to have a collision with the roadway infrastructure. Examples include Intersection Warnings, Lane Departure Warnings, Curve Speed Warnings and Road-condition Warnings. Driving in a dynamic environment has become increasingly complex, such that drivers must visually track objects, monitor a constantly changing system, manage system information, to include the explosion of telematics, and make decisions in this dynamic and potentially high mental workload environment. Introducing Collision Avoidance Systems into vehicles could add to the complexity of this dynamic environment as different drivers will respond differently to Collision Avoidance Systems and there are many critical human factors issues that require investigation. 2005-01-01T00:00:00Z https://hdl.handle.net/1721.1/46723 Cognitive Task Analysis for the LCS Operator Scott, S. D.; Cummings, M. L. The following Tables and Figures detail the cognitive task analysis (CTA) performed to determine the information requirements needed to support a single operator located aboard the futuristic Littoral Combat Ship (LCS). This operator is responsible for controlling four underwater unmanned vehicles in conjunction with a UAV operating on a shared network. • Table 1 is a scenario task overview that breaks the overall mission into 3 phases (planning, execution, and recovery) and then details the subtasks for each of the 3 mission phases. • Figure 1 is an event flow diagram that demonstrates what events must occur in a temporal order for each of the 3 phases. There are three basic event types in Figure 1: 1) a loop (L) that represents a process that occurs in a looping fashion until some predetermined event occurs, 2) a decision (D) that represents some decision that is required from the LCS operator, and 3) a process (P) which requires some human-computer interaction to support the required tasks. In each event block, an alphanumeric code is included which labels that particular event type (L#, D#, P#). This label is important because later information requirements will be mapped to one of these events. • Table 2, which details the situation awareness (SA) requirements for the LCS Operator for each of the 3 mission phases and associated subtasks. Each of these SA requirements is mapped directly to one or more events in Figure 1. Because the decisions in Figure 1 represent critical events that require detailed understanding of what information and knowledge is needed to support the operator’s decision-making process, decision ladders were constructed for the diamonds and loops in Figure 1 that correspond to an involved decision process to resolve the question being posed at that stage in the event flow (Figures 2-4). Decision ladders are modeling tools that capture the states of knowledge and information-processing activities necessary to reach a decision. Decision ladders can help identify the information that either the automation and/or the human will need to perform or monitor a task. Decision Ladders, illustrate the need not only for the same information identified by the cognitive task analysis, but the need for several other pieces of information such as the need for visual or aural alerts in contingency situations. In Figures 2-4, three versions are included that detail (a) the basic decision ladder, (b) the decision ladder with corresponding display requirements, and (c) the decision ladder with possible levels of automation. • Figure 2 represents the automated target recognition (ATR) decision ladder (D3 from Event Flow): (a) decision ladder, (b) decision ladder with corresponding display requirements, and (c) decision ladder with possible levels of automation. • Figure 3 shows the decision ladder information and knowledge requirements for the sentry handoff (L3 from Event Flow). • Figure 4, the UUV Recovery Decision Ladder (D7 from Event Flow), illustrates what information is nominally needed. Since this phase was not a major focus, the decision ladder is not as detailed as it could be. This should be a point of focus in Phase II. Lastly Figure 5 demonstrates the coordination loop that must occur in the case where a handoff failure occurs (for a number of reasons to include equipment failure, communication delays, etc.) Again, because the multi-player coordination issues are not a primary focus in Phase I but are a significant consideration for any follow-on phases. In support of Plan Understanding for Mixed-initiative control of Autonomous systems (PUMA) 2006-01-01T00:00:00Z https://hdl.handle.net/1721.1/46719 Human Supervisory Control Issues in Network Centric Warfare Cummings, M. L.; Mitchell, P. J.; Sheridan, T. B. Network centric warfare (NCW) is a concept of operations that seeks to increase combat power by linking battlespace entities to effectively leverage information superiority. A network centric force must be supported by sophisticated automated systems, so human-computer interactions are an important aspect of overall performance. These interactions are examples of human supervisory control (HSC), in which a human operator intermittently interacts with a computer, receiving feedback from and providing commands to a controlled process or task environment, which is connected to that computer. The Department of Defense (DoD) has recognized that a lack of understanding of HSC issues relevant to NCW is a significant barrier limiting NCW’s potential benefits. This report identifies eight central HSC issues that could significantly impact operator performance in NCW: Appropriate levels of automation, information overload, adaptive automation, distributed decision-making through team coordination, complexity measures, decision biases, attention allocation, and supervisory monitoring of operators. The adoption of NCW principles is often misunderstood as requiring increased levels of automation, which makes this a particularly acute problem as NCW is implemented. For the average operator, implementation of NCW will exponentially add to the number of available information sources as well as the volume of information flow. Without measures to mediate this volume, information overload will occur much more often than in the past, as it will be far easier for operators to obtain or be given more information than they can adequately handle. One way to alleviate this problem is through adaptive automation, which has been shown in certain cases to lower workload. There will also be a corresponding increase in information complexity, quantified by complexity measures, which can cause a loss of situation awareness or an unmanageable increase in mental workload. It is therefore essential that the interfaces with which NCW operators interact help to reduce and manage this increased level of data complexity. A more fundamental issue associated with the increase in the number of available information sources, volume of information, and operational tempo under NCW are operator attention allocation strategies. NCW hinges on successful information sharing, so knowledge of the relationship between perceived and actual high priority tasks and associated time management strategies, as well as the impact of task disruptions is critical. As a result of NCW information sharing, command and control (C2) structures will change significantly. Traditional methods where commands are passed down from higher levels in a command hierarchy will, at least, be partially replaced by distributed decisionmaking and low-level team coordination. Therefore, understanding how to make effective, timepressured decisions within these organizational structures takes on greater importance in NCW. These redefined C2 structures will drive an increase in information-sharing tempo and rapid decision-making. Under these time pressures, the use of heuristics and other naturalistic decisionmaking methods may be subject to undesirable decision biases, both for individuals and groups. Lastly, how automated technology can be leveraged in order to observe and diagnose HSC issues during supervisory monitoring of operators is another significant area of concern since NCW will contain embedded HSC systems. 2004-01-01T00:00:00Z