At present, researchers tend to adopt one of two approaches to the workload management problem: the arousal approach and the demand approach. The arousal approach measures the energy level of the autonomic nervous system in attempt to determine the amount of driver workload. The level of demand approach measures external environmental information or uses the method of task analysis to determine the demand placed on the driver. This section will describe these two approaches in more detail and then explain how the SAVE-IT program approach differs from them.
2.2.1 The Arousal Approach
The “Yerkes-Dodson law” has frequently been applied to the workload management problem (De Waard, 1996). This law states that human performance is optimal at an intermediate level of arousal and may suffer if arousal is too low or too high. Arousal is measured from physiological metrics such as heart rate, respiration rate, and pupil diameter. The arousal approach proposes that a workload management system should increase arousal level when it is low and decrease the arousal level when it is high so that the arousal level is maintained within the optimal range. One major problem with this approach is that a high level of arousal could result from a wide range of factors, such as physical or mental workload, emotion, anxiety, or stress. Arousal and distraction, for example, may be not be correlated. Therefore, the assumption that driver distraction, workload, substance impairment, and drowsiness fall onto a uni-dimensional continuum (i.e., arousal) may be overly simplistic. Because of this lack of specificity, arousal is not diagnostic of driver distraction, workload, or driver safety. Even if a high level of arousal is detected, it is not clear what action should be taken to reduce the arousal level. If a driver is emotional or stressed, technology could offer little assistance in calming the driver. Changes in many of the nervous system measures may not be related to arousal, for example, pupil diameter, varies with ambient illumination, and therefore it is of limited application in the automotive industry. Despite the shortcomings of this approach, the concept of arousal is useful in the study of driver impairment because impairment is accompanied with a low level of arousal and has clear and immediate consequences for driver safety.
2.2.2 The Demand Approach
Some researchers have proposed a demand-based, “driver-out-of-the-loop” workload management system that considers driving and non-driving task demands to the exclusion of driver state assessment (e.g., the GIDS approach, Michon, 1993). It is assumed that environmental factors such as road, weather, and traffic conditions can determine driving demand, so that more non-driving tasks (distraction) are permitted under low driving demand situations and fewer distraction tasks are permitted under high driving demand situations. Inspired by the domain of aviation human factors, many researchers have approached the workload management problem through task analysis, drawing parallels between pilots using a declutter button during high workload events and the automotive environment (Mykityshyn & Hansman, 1993). This approach identifies information prioritization and scheduling as an important focus of workload management. This is a reasonable approach because it focuses on task demands that have implications for safety, but it has several limitations.
One limitation is that the aviation human factors research and methodologies may not be directly applicable to the automotive domain. Aviation missions tend to be more predictable at a macroscopic level, with take-off at the beginning, attaining desired altitude, navigation through waypoints, weapons deployment (in military missions), return to final approach, final approach, and landing. The irregularity of the driving task at the macroscopic level makes it less amenable to task analysis. In addition, military missions tend to encompass much higher rates of information flow than the automotive environment. Military pilots are bombarded by multiple screens of information detailing navigation, aircraft state, and threat information, and an analogy has been made between controlling the flight stick and playing a piccolo. The degree of training and personnel selection for pilots compared with drivers is an additional factor that limits the generalizeability of aviation research to the automotive domain. The aviation industry has been researching workload management and adaptive interfaces for several decades, creating a vast amount of data to draw from. The SAVE-IT team will review this literature thoroughly to guide further research, however, we will be careful before generalizing these data to automotive applications. The literature reviews of the relevant tasks will scrutinize information from the aviation domain, and determine which theories and measures are suitable for the SAVE-IT applications.
Another limitation of a “driver-out-of-the-loop” system is that it disregards driver’s individual differences. When given the same weather, road, traffic, and telematics system, different drivers may exhibit different levels of visual and cognitive distraction. Some individual differences may be attributed to driver age and experience. Furthermore, a demand-only approach ignores the fact that when given similar driving and non-driving tasks, the same driver may behave inconsistently across time. Given the same external and internal demands, a driver could be allocating adequate attention to the driving task, or allocating attention to unrelated tasks, such as daydreaming or focusing on an item in the external world that is unrelated to the driving task. Approaches which do not measure the driver would be incapable of making these kinds of distinctions. In short, distraction is inherently an individual- and time-dependent driver state and therefore the driver should be included in the loop of distraction assessment. Although understanding the demands placed upon the driver is crucial for approaching the workload management problem, in isolation, the demand approach is incomplete.
2.2.3 The SAVE-IT Program Approach
The previous approaches are limited because they do not directly measure the allocation of attention. The SAVE-IT system will be comprehensive and consider driver state in addition to driving and non-driving task demands. For driver state, the SAVE-IT approach focuses on the perceptual and cognitive allocation rather than the affective state and personality. To the extent that a low level of arousal is associated with impairment, arousal is studied in that context. Sensor development is paramount to the assessment of driver state. DDE and Seeing Machines, Inc. have co-developed an automotive-grade, non-obtrusive, stereo eye-tracking system (ETS) that tracks driver’s gaze point and will enable the measurement of attention allocation. The nature of the countermeasure subsystems portrayed in Tables 1 and 2 will dictate the types of information that the human factors research investigates, because for this information to be useful, the driver state and environmental assessment must be tailored to the potential countermeasure systems.
A ttention Allocation in Driving.Crashes are frequently caused by drivers paying insufficient attention when an unexpected event occurs, requiring a novel (non-automatic) response. As displayed in Figure 3, attention to the driving task may be depleted from either allocation to non-driving tasks, or from impairment (drowsy or substance) leading to diminished attentional resources. Safe driving requires that attention be commensurate with the driving demand or unpredictability of the environment. Low demand situations (e.g., straight country road with no traffic at daytime) require less attention because the driver can usually predict what will happen in the next few seconds while the driver is attending elsewhere. Conversely, high demand (e.g., multi-lane winding road with erratic traffic) situations require more attention because during any time attention is diverted, there is a high probability that a novel response may be required.
Figure 3. Attention allocation to driving and non-driving tasks
A safety system that mitigates the use of in-vehicle information and entertainment system (telematics) must balance both attention allocated to the driving task and attention demanded by the environment. In low driving demand scenarios, allocation of attention to non-driving tasks may not adversely impact safety. In high driving demand scenarios, the same non-driving tasks could divert much-needed attention away from the driving task. The goal of the distraction mitigation system should be to keep the level of attention allocated to the driving task above the attentional requirements demanded by the current driving environment. For example, as evident in Figure 1, “routine” driving may suffice during low or moderate driving task demand, a distracted driving may suffice during low driving task demand, but high driving task demand requires attentive driving.
Drivers routinely perform three classes of activities: (1) vehicle control such as speed and lane control, (2) tactical maneuvering such as speed and lane choice, navigation, and hazard monitoring, and (3) non-driving tasks. Attentional allocation to these activities is dynamic and may vary with individuals. Both vehicle control and tactical maneuvering are subcomponents of the driving task. Under most situations, vehicle control is well-learned and does not require a high level of attention; however, tactical maneuvering requires a high level of attention because it can require a novel response.
An important component of tactical maneuvering is responding to unpredictable and probabilistic events (e.g., lead vehicle braking, vehicles cutting in front) in a timely fashion. Timely responses are critical for collision avoidance. If a driver is distracted, attention is diverted from tactical maneuvering and vehicle control, and consequently, reaction time (RT) to probabilistic events increases. Because of the tight coupling between reaction time and attention allocation, RT is a useful metric for operationally defining the concept of driver distraction. Furthermore, brake RT can be readily measured and is widely used as input to algorithms, such as the forward collision warning algorithm. Therefore, RT to probabilistic events is chosen as the primary, “ground-truth” dependent variable for this research program. RT may not account for all of the variance in driver behavior, and other measures such as headway, action selection and eye glance behavior may be considered separately.
2.2.4 SAVE-IT Task Structure and Methodology
Task Structure.Given that the SAVE-IT program is a 3-year program of limited budget, we must be selective and scale the proposed research to a level that fits the program objectives. Consequently, impairment- and drowsiness-related research is beyond the purview of the SAVE-IT program. Distraction is usually divided into cognitive distraction, visual distraction, manual distraction, and auditory distraction.1 Cognitive distraction is distinct from visual distraction because drivers may keep their eyes on the road but take their mind off the driving task. To achieve the seven major objectives of the SAVE-IT program (Section 1.2), we adopt a “divide and conquer” approach, dividing the program into fifteen manageable tasks. The mapping from tasks to objectives is discussed below and will be displayed in Table 5. The tasks will be discussed in detail in the statement of work (Section 2.3), and are listed as follows:
Identify which driving scenarios the SAVE-IT technologies are likely to offer the most benefit in reducing crashes.
By targeting the distraction-related problem scenarios for the design of mitigation solutions, this task will contribute to the objectives of advancing the deployment of adaptive interface technology as a countermeasure for distraction-related crashes (Objective 1) and enhancing the effectiveness of collision warning systems by optimizing alarm onset algorithms tailored to the driver’s level of workload and distraction (Objective 2). The identification of scenarios will also contribute to designing experiments to evaluate the SAVE-IT systems (Objective 4), by providing prototypical scenarios to mitigate against.
(2) Driving task demand
Develop algorithms that measure the level of attention that is required by the driving environment as a function of environmental parameters.
Based on the philosophy of maintaining an allocation of attention that is commensurate with the level of driving task demand, this program must develop a reliable and valid means of measuring driving task demand. This will serve the objectives of advancing the deployment of adaptive interface technology (Objective 1) and conducting human factors research to help derive distraction and workload measures (Objective 3). In a vehicle that does not employ an eye-tracking system, driving task demand measures may be of elevated important for determining which telematics functions are appropriate at a given time, therefore this task contributes to the objective of identifying potential scalable system concepts (Objective 7).
Develop algorithms that reliably measure driving performance.
Because poor driving performance may be indicative of inadequate attention allocation, an algorithm that measures driving performance will contribute to the objectives of advancing the deployment of adaptive interface technology (Objective 1) and conducting human factors research to help derive distraction and workload measures (Objective 3). In a vehicle that does not employ an eye-tracking system, performance measures may be of elevated important for determining which telematics functions are appropriate at a given time and therefore this task will also contribute to the objective of identifying potential scalable system concepts (Objective 7).
(4) Distraction mitigation
Develop countermeasures that mitigate against inappropriate levels of distraction, while maintaining high levels of driver acceptance (e.g., screening phone calls).
This task will contribute directly to the objectives of advancing the deployment of adaptive interface technology (Objective 1), and developing performance requirements and standards/guidelines for adaptive interface conventions (Objective 5).
(5) Cognitive distraction
Develop algorithms that can reliably measure the level of cognitive distraction.
This task will contribute directly to the objective of conducting human factors research to help derive distraction and workload measures for use in algorithms for triggering interface adaptation (Objective 3). Because distraction measures provide input to countermeasure systems, this task will also contribute to advancing the deployment of adaptive interface technology (Objective 1), enhancing collision warning systems (Objective 2), and developing performance requirements and standards/guidelines (Objective 5).
(6) Telematics demand
Identify the distraction potential and priorities of telematics functions to support distraction countermeasure systems.
By providing information that will dictate which telematics functions should be blocked or advised against, this task will contribute to the objectives of advancing the deployment of adaptive interface technology (Objective 1). This research will also serve the objectives of conducting human factors research to derive distraction and workload measures (Objective 3), and developing performance requirements and standards/guidelines (Objective 5).
(7) Visual distraction
Develop algorithms that can reliably measure the level of visual distraction.
This task will contribute directly to the objective of conducting human factors research to derive distraction and workload measures (Objective 3). Because distraction measures provide input to countermeasure systems, this task will also contribute to advancing the deployment of adaptive interface technology (Objective 1), enhancing collision warning systems (Objective 2), and developing performance requirements and standards/guidelines (Objective 5).
Develop algorithms that can reliably measure the immediate intention of the driver (e.g., intent to pass)
Providing driver intent information to the safety warning countermeasures will reduce the number of false alarms, therefore, this task contributes to enhancing the effectiveness of collision warning systems (Objective 2).
Improve existing safety warning countermeasures so they can adaptively warn the driver about immediate threats in the environment as a function of driver state information (e.g., forward collision warning).
This task contributes directly to the objectives of enhancing the effectiveness of collision warning systems by optimizing alarm onset algorithms tailored to the driver’s level of workload and distraction (Objective 2), and developing performance requirements and standards/guidelines (Objective 5).
(10) Technology development
Identify and develop technologies for supporting different stages of SAVE-IT system development (e.g., eye tracking). Develop a concept vehicle that can serve as a platform for the system algorithms.
This task will either directly or indirectly serve all seven objectives of this program by providing the necessary technology to support the human factors research and build a viable system. One of the most important objectives that this task serves is identifying scaleable system concepts and sensing technologies for further stages of research and development (Objective 7).
(11) Data fusion
Develop algorithms that coherently fuse all data from the sub-systems into information that can drive the countermeasure systems.
Because this task involves developing the final algorithms that will assess the driver state and govern the behavior of the countermeasures, this task is a requirement for fulfilling the objectives of advancing the deployment of adaptive interface technology (Objective 1), enhancing the effectiveness of collision warning systems (Objective 2), conducting human factors research to help derive distraction and workload measures (Objective 3), and developing performance requirements and standards/guidelines (Objective 5).
(12) Establish Guidelines and Standards
Develop performance requirements for system operation and standards/guidelines for adaptive interface conventions.
This task will contribute directly to the objective of developing performance requirements for system operation and standards/guidelines for adaptive interface conventions (Objective 5). Because these standards/guidelines will be implemented in this program, this task will also contribute to the objectives of advancing the deployment of adaptive interface technology (Objective 1), and enhancing the effectiveness of collision warning systems (Objective 2). These standards/guidelines will also provide an important document to be disseminated to the public (Objective 6).
(13) System integration
Integrate the sensors, countermeasures, and algorithms into a fully functional prototype vehicle.
By providing a prototype vehicle that is composed of the SAVE-IT systems, this task will contribute to the objectives of advancing the deployment of adaptive interface technology (Objective 1), and enhancing the effectiveness of collision warning systems (Objective 2). Because the prototype vehicle will be used in the evaluation, this task will also contribute to the objective of developing and applying evaluation procedures for assessment of SAVE-IT safety benefits (Objective 4).
Evaluate the safety benefits and driver acceptance of the SAVE-IT systems.
This task will directly contribute to the objective of developing and applying evaluation procedures for assessment of SAVE-IT safety benefits (Objective 4). Because this evaluation will reveal which aspects of the SAVE-IT system are effective, this task will contribute to all seven objectives.
(15) Program summary and benefit evaluation
Develop a program summary that describes all of the preceding tasks in detail, evaluates the benefit of the SAVE-IT systems, and disseminates the resultant data of this program to the public.
This task will directly contribute to the objective of providing the public with documentation of the human factors research and with information describing the algorithms for controlling the driver vehicle interface to the extent needed for specifying performance and standardization requirements (Objective 6). Increasing public and industry knowledge about the SAVE-IT systems will contribute to advancing the deployment of both distraction mitigation (Objective 1) and safety warning (Objective 2) countermeasures. An important component of this task will be identifying which concepts and technologies are appropriate for further stages of research and development (Objective 7).
The program is divided into two phases. Phase I includes tasks that can be studied in parallel. Phase I supports the objectives of conducting human factors research to derive distraction and workload measures, and initiates the development of adaptive interface technologies and enhanced collision warning systems. The objectives of Phase I tasks are to identify crash scenarios that SAVE-IT should be designed to prevent, evaluate available technologies and sensors, and conduct initial human factors research, including literature review and the identification of diagnostic measures for respective dimensions, to guide the development of more detailed implementation plan in Phase II. At the end of Phase I, a concept vehicle will be demonstrated and more details will be provided about the plan to be implemented in Phase II. If initial experiments demonstrate the need of adding, removing, or modifying tasks, the task structure can be revised accordingly. The initial human factors research in Phase I will determine which measures are diagnostic for each dimension, however, but stop short of determining and validating algorithms and countermeasures. In Phase II, additional human factors research will be conducted to develop and validate the algorithms and countermeasures.
In addition to the technology demonstration, the SAVE-IT program will collect, summarize, and disseminate data that describe driver behavior. The simulator, on-road, and test track experiments will collect driver behavior data that will be summarized in relational databases for scalability, organization, and documentation. The data will be delivered in the most convenient form for the sponsor, perhaps recordable DVDs, portable hard disks, or CDROM. These data provide an important resource that will enable other researchers to examine algorithm performance and explore alternate interpretations and hypotheses. The SAVE-IT team will provide a small set of representative conditions. These data will include all conditions for several drivers. Depending on the demand for these data, NHTSA has the option to expand this effort to provide a more comprehensive archive. Although UMTRI now employs Microsoft’s SQL Server 2000, scripts will be provided to unpack the data into the sponsor’s particular database application.
Beyond the dissemination of empirical data, the program will publish experiment results through professional conferences and peer-review journals. The SAVE-IT team has a strong history of publication. Each task will also contribute guidelines that will be synthesized in Task 12. These guidelines will be organized in a two page format that has been developed and used successfully in previous projects. These guidelines will capture common findings and principles that should govern adaptive interface technology for cars. The SAVE-IT team will also work to incorporate these findings into SAE and other national and international standards. Several team members are currently contributing to SAE standards on in-vehicle system evaluation, speech system conventions, and collision warning design.
Variables.An important objective of this research program is to identify signature measures and variables for dimensions such as distraction, impairment, and intent. Previous research has revealed that gaze variability (Recartes & Nunes, 2000) and steering entropy (Boer, 2001) are indicative of cognitive distraction, but more research is required to determine combinations of measures that are diagnostic of the given dimensions. Decades of human factors research have identified several signature measures such as glance duration and frequency for visual distraction. Research is required to determine performance and physiological measures that are diagnostic of driver intent. Driving task demand can be determined as a function of road, traffic, and weather variables. Telematics demand can be determined for various in-vehicle devices and features.
Methods and Resources.There is a tradeoff between control and realism in the different research environments. At the most realistic extreme is the on-road environment, which affords no control over environmental parameters such as traffic and road geometry. At the other extreme is the driving simulator, which offers less realism but almost total control over environmental parameters. Between the two extremes of realism and control exists the test track which offers moderate control and realism. Human factors researchers possess differing perspectives on the ecological validity of behavioral research conducted in driving simulators. If drivers do not consider the simulation environment to be “realistic”, drivers may not behave naturally. It is feared that some drivers may treat the task like a video game and test the system to see how it behaves under extreme conditions. Many researchers believe that the test track offers a viable alternative to the driving simulator. The test track, however, also presents departures from reality. Drivers are still confronted with the unusual task of “trying to drive normally” in an unfamiliar vehicle, and may imagine experimenters scrutinizing their every move. Drivers on a test track are also stripped of natural motivating factors such as the need to travel from one location to another in an acceptable time, and may become burdened with unnatural motivating factors such as not damaging the unfamiliar property and not appearing to behave in an abnormal manner. If an experimenter is seated in the passenger seat, the driver may become overwhelmed with feelings of self-consciousness and may drive in a manner that is far more cautious than “normal driving”.
The driving simulator, test track, and on-road environments offer different advantages and disadvantages and consequently no one environment is appropriate for all research questions. In some situations, the level of risk may rule out experimentation with real vehicles. In other situations, where vehicle dynamics such as braking and turning are an integral aspect of the problem focus, the driving simulator may not be appropriate. Skillful researchers can select the most appropriate environment and minimize the disadvantages contained therein. For example, researchers can allow drivers to become accustomed with the new environment so that the novelty wears off, and create a surrogate set of motivational constraints that effectively mimic the natural constraints of the real environment. If a study is not overly dependent on the dynamics and feel of the vehicle, and the driver is able to achieve temporary suspension of disbelief about the environment, valid data may be obtained in a driving simulator.
Given that the scope of the SAVE-IT program requires research to be conducted in simulator, test track, and on-road environments, the SAVE-IT team have secured driving simulators at DDE, the University of Iowa (including National Advanced Driving Simulator, or NADS), UMTRI, and Ford Motor Company (including Virtual Test Track Experiment/VIRTTEX), and test tracks at Transportation Research Center (TRC) and Dana Corporation. Approximately 20 subjects will be used for each experimental condition, sampled from Kokomo (IN), Iowa City (IA), Ann Arbor (MI), Detroit (MI), and Columbus (OH). A ruse and reward system will be used to surrogate the natural motivational constraints of driving, such as productivity, arriving at the location in a reasonable time, and safety.
The first task of this program will be identifying scenarios that are strongly affected by driver distraction. These tasks will be used in experimental designs of other tasks. Vehicle following scenarios have been used in several studies, including the driver-vehicle interface studies of the ACAS FOT program and Lee et al.’s email studies (NHTSA DTNH22-99-H-07019, 2001; Lee et al., 2000), and crash data suggest that driver distraction is a common cause of rear-end collisions (Wang, Knipling, & Goodman, 1996). If the scenario identification task reveals that vehicle following tasks should be used, headway could be constrained to vary within a small range (e.g., 1-2 seconds). The lead vehicle may apply non-imminent braking at approximately 0.2 g erratically several times within a condition, and brake reaction time (BRT) could be analyzed as the primary dependent variable (see Lee et al., 2000). Erratic, non-imminent situations frequently occur in real-world driving and therefore this scenario will have high face validity. An additional reason for using non-imminent braking events is that they are more suitable than imminent events for a within-subjects design. Within-subjects comparisons are crucial for the removal of individual differences among conditions. To control order effects, conditions will be counterbalanced with Latin squares and sufficient practice will be provided before the experimental conditions begin. Results will be analyzed using appropriate statistical techniques (e.g., regression, analysis of variance, and analysis of covariance, and time series analysis or entropy analyses) to develop driver models and algorithms.