När man som bilist ser ett blåljusfordon i trafiken är det inte alltid helt enkelt att tolka situationen och veta hur man förväntas bete sig, och kunskapen hos bilister om betydelsen av de olika ljussignaler som utryckningsfordon använder (t.ex. fast blått sken, blinkande blåljus eller ett blinkande blått-rött ljus) är varierande. Bilisters osäkerhet och stress när de möter blåljusfordon kan leda till en ökad risk för trafikolyckor. Det är därför viktigt att undersöka hur förare av utryckningsfordon kan kommunicera sina intentioner till omgivande trafikanter.

I andra länder, exempelvis Tyskland och Irland, finns möjlighet att kombinera ljussignalering med textskyltar, vilket skulle kunna vara ett bra komplement till ljussignaler. I Sverige har den typen av textskyltar tidigare använts i begränsad utsträckning av Tullverket, men det är numera inte tillåtet enligt Trafikförordningen. Det finns dock anledning att tro att det skulle kunna underlätta i kommunikationen mellan blåljusfordon och övriga bilister.

Incorrect human behavior is a significant contributor to maritime accidents. Navigation skills therefore represent a critical factor for safety at sea. Integration, digitalization, and intelligent navigation technologies impact Maritime Education and Training (MET). The study aims at understanding how maritime experts and maritime trainees allocate their visual attention to avoid collision during intense maritime traffic in a full-mission bridge simulator. A sample of two experienced active navigators and seven maritime students were fitted with a wearable eye-tracker and placed in different navigational watchkeeping simulation contexts. Individual visual attention was quantified through the analysis of areas of interest (AOIs) and gaze shifts between these AOIs. Experts and novices differ in their gaze patterns. The most prominent difference is that experts make less use of instruments and look out more for information gathering. As ships are complex socio-technical systems, the results of this study may provide Integrated Bridge Systems (IBS) designers and MET professionals with useful insights on the interaction between humans and navigation instruments.

Initiating communication and conveying critical information to the human operator is a key problem in human-automation collaboration. This problem is particularly pronounced in time-constrained safety critical domains such as in Air Traffic Management. A visual representation should aid operators understanding why the system initiates the communication, when the operator must act, and the consequences of not responding to the cue. Data glyphs can be used to present multidimensional data, including temporal data in a compact format to facilitate this type of communication. In this paper, we propose a glyph design for communication initialization for highly automated systems in Air Traffic Management, Vessel Traffic Service, and Train Traffic Management. The design was assessed by experts in these domains in three workshop sessions. The results showed that the number of glyphs to be presented simultaneously and the type of situation were domain-specific glyph design aspects that need to be adjusted for each work domain. The results also showed that the core of the glyph design could be reused between domains, and that the operators could successfully interpret the temporal data representations. We discuss similarities and differences in the applicability of the glyph design between the different domains, and finally, we provide some suggestions for future work based on the results from this study.

Traffic management in aviation, shipping, and rail transport shows similarities and dissimilarities in the work process. For example, they share the temporal aspect, but different levels of urgency in the control work set different requirements on monitoring, decisions, and actions. However, few studies have been presented that model and compare the different domains in terms of temporal decision-making. The Joint Control Framework (JCF) is an approach to analyse and temporally model operators’ control processes from a cognitive systems engineering perspective. In this study, we have used JCF to map, and compare, cognitive joints, such as perceptions, decisions, and actions, in temporally challenging control situations in air traffic control, maritime vessel traffic service, and train traffic management. Data was collected collaboratively with traffic operators, focusing on (1) identifying challenging traffic situations and (2) jointly modelling the temporal decision-making patterns of these situations using simplified JCF. Post-analysis was done by breaking down the results into different processes and comparing domains to ascertain how operators maintain control. An intermediate level of activity—between general monitoring and work with specific vehicles—was identified: processes-in-focus. A shared problem arises in the shift between general monitoring and the processes-in-focus. All processes-in-focus comprise cognitive joint cycles of perceptions, decisions, and actions. However, depending on the framing of processes-in-focus, the patterns of joints, such as temporal extension and complexity, differ. In the remainder of the article, implications for the interaction design, in particular the potential for human–AI/automation teaming with higher levels of automation and cognitive autonomy, are discussed. 

The approach for the dissertation has been to use the known problems and issues with automation as a starting point, and then decompose that complex of problems and explore them to find possible solutions. Initiation of communication between automation and human, how and when it is performed, were identified as one critical point in the human-automation collaboration. This insight formed the basis for the two main concepts presented in this dissertation. The first concept is the Reduced Autonomy Workspace (RAW), a design approach that describes the process of when a highly autonomous automation needs to consult the human working together with the automation. Focus is on the temporal aspects, but it also includes transformation of the information content to reduce the cognitive effort on the human receiving the information. A visualization for future automation using glyphs based on the same principles was developed to support the RAW design approach. The glyphs were evaluated in workshops with operators from air traffic control, vessel traffic service, and train traffic management, respectively. The second main concept is a new type of attention support, Soft Visual Cues (SVC). The SVC complements other types of attention support which may, due to the automation’s lack of knowledge of the overall situation, draw the operator’s attention away from something potentially more important. Different designs were evaluated in operator workshops and tested by other air traffic controllers in a series of real-time simulations. In addition to these concepts, studies have also been performed to explore and gain a deeper knowledge of how the involved control processes can be understood, the effect of entangled control processes, and how domain specific prerequisites are reflected in the control processes.

Utgångspunkten i avhandlingen är de redan kända utmaningarna med ökad automation och från det har problematiken brutits ner för att hitta förslag på lösningar. Ett avgörande moment i samarbetet mellan människa och automation är när automationen initierar kommunikation med människan. Den insikten har resulterat i två centrala koncept som presenteras i avhandlingen. Det första konceptet är designansatsen Reduced Autonomy Workspace (RAW) som beskriver hur anpassning av kommunikation mellan automation och människa kan utformas för situationer när automationen måste konsultera människan. De temporala aspekterna är mycket viktiga och designansatsen inkluderar också hur automationen ska transformera informationen till ett format som kräver så lite kognitiv ansträngning som möjligt för människan. En glyfbaserad visualisering för användning i framtida högautomatiserade system baserad på designansatsen RAW togs fram och utvärderades i ett an-tal workshops med operativa domänexperter från flygtrafikledning, sjötrafikinformationstjänst och tågtrafikledning. Det andra konceptet är en ny typ av visualiseringskoncept för uppmärksamhetsstöd, Soft Visual Cues (SVC), eller mjuka visuella påminnelser. Mjuka visuella påminnelser kompletterar andra typer av uppmärksamhetsstöd som aktivt drar uppmärksamheten till sig, vilket kan vara ett problem om användaren är upptagen med något viktigare som automationen på grund av en bristande helhetsbild inte känner till. Olika designalternativ utvärderades i workshopform tillsammans med flygledare och den föredragna designen testades sedan i realtidssimuleringar med andra flygledare. Utöver de framarbetade koncepten har också studier genomförts för att skapa en djupare förståelse för de ingående kontrollprocesserna, hur de påverkar varandra och hur domänspecifika förutsättningar avspeglas i dem.

Trajectory based operations will be an important concept in tomorrow’s Air Traffic Management. The concept is a complex puzzle that includes both strategical and tactical aspects to achieve increased predictability and efficiency. However, research related to trajectory based operations has been focusing on strategical or technical perspectives, thus not addressing how it effects, and is affected by, the air traffic controllers work situation. In this study, we conducted real-time simulations with air traffic controllers to explore how the introduction of a tactical trajectory management tool affects the controllers’ work strategies for conflict resolution. A tool for tactical trajectory management was developed and integrated in an Air Traffic Control simulator, and tested with realistic traffic scenarios. The results showed that by using tactical trajectory management, it was possible to reduce the need for the air traffic controllers to monitor implemented conflict solutions. The tool was incorporated by the controllers in their current work strategies for conflict resolution rather than changing their strategies. However, it also highlighted the need for more advanced support tools for trajectory management in solving vertical conflicts and using temporal constraints.

Lack of support for handling a reduction of autonomy in a highly autonomous automation may lead to a stressful situation for a human when forced to take over. We present a design approach, the Reduced Autonomy Workspace, to address this. The starting point is that the human and the automation work together in parallel control processes, but at different levels of autonomy cognitive control, such as setting goals or implementing plans, which is different from levels of automation. When autonomy is reduced, the automation should consult the human by providing information that has been aligned to the level at which the human is working, and the timing of the provision should be adapted to suit the human’s work situation. This is made possible by allowing the automation to monitor the human in a separate process. The combination of these processes, information level alignment and timing of the presentation, are the key characteristics of the Reduced Autonomy Workspace. The Reduced Autonomy Workspace consists of four phases: Identification of the need; evaluation of whether, and, if so, when, and how to present information; perception and response by the human; implementation of a solution by the automation. The timing of the information presentation should be adapted in real-time to provide flexibility, while the level of the information provided should be tuned offline and kept constant to provide predictability. Use of the Reduced Autonomy Workspace can reduce the risk for surprising, stressful hand-over situations, and the need to monitor the automation to avoid them.

In safety critical control domains, such as Air Traffic Management (ATM), a detailed understanding of the complex interactions and workflow of users is essential for improving the design of supporting tools. However, few frameworks account for temporal sequences of actions and levels of cognitive control. In this paper, we discuss how interaction in the control room can be seen as processes formed by a framed sequence of directly and indirectly related events, external events as well as user actions. To untangle this complexity, we propose to combine different analysis approaches, the Joint Control Framework and visual sequence mining, which can complement each other to reveal patterns of interactions within and between the processes and assign meaning to them.

In visually demanding environments like working positions in air traffic control, there is a risk of missing important information. Attention support might help to reduce this, but there is a risk that it introduces new issues in the form of distractions or increased visual load. The work presented in this paper explores the concept of Soft Visual Cues, with the aim to make the operator aware of missing information, but doing it discreetly enough only to be noticed when the operator is scanning the area where the information is present, thereby complementing the concepts of attention guidance, alarms, and subtle gaze direction. The qualitative evaluation performed aimed primarily at catching the operators' (air traffic controllers) first impressions. Focus was on air traffic control, but was complemented with vessel traffic service and train control environments and operators. The results from air traffic control showed a preference for symbol property changes related to geometry rather than for colour or opacity changes as soft visual cues. The comparison to the other domains highlighted the contextual dependence, visual context and process context, of the concept.

My main research question is how adaptive automation through interactive visualization can be used to improve human-automation collaboration. The research is performed in a cross-domain project including air traffic control, train control, and vessel traffic service, where I mainly focus at air traffic control. All domains are safety critical, include highly complex control room environments, and are facing increasing automation with thereby associated challenges. Adaptive automation by sensing the operator is one approach to mitigate potential automation problems. The first studies will be performed using a prototype with a graphical representation of an automated conflict detection tool where the operator gets visual attention support based on eye tracker data. Different concepts of information visualization feedback will be explored. A literature review on adaptive automation indicated that the projects approach is novel in its combination of adaptive automation, information visualization, and by its cross-domain nature. Finding the best ways of taking advantage of the cross-domain approach, how to get access to operative environments and operators, and how to find effective, yet unobtrusive, sensors are some of the identified challenges.