WorksComputer Science
Towards Transparent Computational Agents for Air Traffic Control: Explainability, Interpretability and Trust
Abstract
We report preliminary findings from an ongoing investigation into transparency techniques for computational Air Traffic Control agents. Air Traffic Controller Officers (ATCOs) observed two types of agent in simulated air traffic controlling exercises. ATCOs found the chosen transparency techniques helpful to some extent: revealing how agents were making controlling decisions was an essential foundation for trust. However, the ATCOs' trust depended on whether they could be made sense of in a live, operational context and the agents' actions where judged to be professionally competent.
Introduction
Project Bluebird is developing agent-based tools and exploring how they may assist ATCOs by proposing clearances1 for aircraft. The safety-critical demands of air traffic control (ATC) make it important that ATCOs are able to assess an agent’s trustworthiness: i.e., is the clearance suggested appropriate in the current context[8].
Transparency is a key requirement for users’ trust in AI systems [10]. We report findings from studies examining how ATCOs evaluate agent transparency techniques for two agents: Hawk, a rule-based agent, and Falcon, an optimisation-based agent.
Explainability and Interpretability
An agent possesses the property of transparency if it supports explainability and interpretability [2]: i.e., how it makes a decision and why it makes a decision [1].
We follow a socio-technical systems approach [3] to understanding how to operationalise and evaluate these properties. In particular, we are interested in the use ATCOs, as members of a specific community of practice [11], make of members’ methods – shared understandings people use to achieve the mundane orderliness of their everyday activities – to make sense of agent behaviour [5].
Visualising Relevant Aircraft and Plans
Following discussions with ATCOs, relevant aircraft and agent plans were chosen as the explainability and interpretability techniques to be evaluated.
In ATC, relevant aircraft is the subset of traffic that materially constrains a selected (subject) aircraft’s safe and efficient plan2. It was chosen because it invites the observer to attend to what is driving the agent’s plans for the subject aircraft.
Plans were chosen as the interpretability technique because they enable the observer to assess the agent’s actions in context. Both these techniques are ATCO- rather than agent-centric, reflecting the methods that are constitutive of ATCOs’ working practices [5].
For Hawk, any aircraft posing a risk to the subject aircraft of a loss of separation is identified as relevant. Hawk does not plan beyond the next clearance, however, ATCOs can infer from its history of clearances whether Hawk’s behaviour is consistent with having a plan [4].
Falcon optimises plans simultaneously for all aircraft. First it attempts to find the most efficient plan for an aircraft entering the sector and relevant aircraft are identified. Additional relevant aircraft are identified transitively with respect to these. Finally, subject and all relevant aircraft are jointly optimised [6] and these plans are displayed (Figure 1).

Figure 1. Falcon display showing relevant aircraft (orange, blue, green circles) and agent plan (magenta lines) for subject aircraft (green bounded box) and time ordered, scrolling list of current and recent clearances (top right).
For both agents, a time ordered, scrolling list of agent clearances was displayed in the top right corner.
Methodology
In a series of formative evaluation sessions, ATCOs observed Hawk and Falcon controlling a simulated sector equipped with the transparency techniques described above. A ’think-aloud’ methodology was followed, with ATCOs encouraged to comment on how well the techniques enabled them to understand agent behaviour [7].
Findings
Analysis centred around three themes: explainability (do agents identify the right relevant aircraft), interpretability (can agent plans be made sense of given the relevant aircraft), and the relationship between transparency and trust in agent performance.
Explainability: is the agent identifying the right aircraft?
ATCOs commented on whether the agents identified relevant aircraft correctly and, if not, what kind of mistakes did the agents make, and where ATCOs able to diagnose the reasons. In this extract, the ATCOs’ question Hawk’s identification of relevant aircraft:
”It’s this longitudinal catch up situation... the catch up speed is 12 knots, which is basically nothing... TomJet was never really relevant traffic.”
The ATCOs’ assessed Hawk was overly cautious about the risk that a faster aircraft will catch it up a slower one (TomJet), because the rate of catch-up was too slow. In such ways, visualising relevant aircraft enabled ATCOs to diagnose the agents’ controlling behaviour.
Interpretability: is the agent issuing the right clearances?
Hawk’s relative simplicity meant its behaviour was generally easy to interpret. In this extract, however, the ATCOs were baffled by its failure to put AKK8 back on its own navigation, i.e., what a competent ATCO would have done:
”AKK8 could be back on its own navigation... we don’t understand why it’s still being vectored... it’s more over controlling.”
The ATCOs acknowledged Falcon’s benefit in presenting complete plans for each aircraft, but its complexity made its behaviour more difficult to interpret:
”Hawk is showing you immediately what’s relevant to these planes. Falcon is showing me not only what’s relevant, but what I intend to do about it with all of the other flights... it is a lot to interpret... there’s pluses and minuses to both.”
Transparency and trust
The study reveals how explainability and interpretability contribute to agent transparency. Relevant aircraft shows promise in satisfying explainability and relevant aircraft and plans together helped the ATCOs to interpret agent behaviour. Transparency doesn’t guarantee trust, however, it enables deciding if the agent’s behaviour can be trusted and this trust is calibrated through experience:
”When it [the agent] fails to implement an early plan for a pair of aircraft, you become a lot more negative... it’s also doing it this time. So, you build up a sort of increasing amount of negativity towards it.”
An agent’s behaviour can be transparent and safe, yet not be trusted if its controlling does not align with the ATCOs’ standards.
Discussion
Our findings suggest agent explainability can be satisfied by visualising relevant aircraft, while agent interpretability resides in the relationship between what relevant aircraft reveal about the agent’s reasoning and the clearances it issues.
ATCOs made sense of an agent’s behaviour in terms of whether relevant aircraft were correctly identified and if the plans were appropriate give those relevant aircraft. Importantly, the ATCOs frequently judged the agents to fall short against the standards of good controlling and their trust in the agents was calibrated on this basis.
ATCOs found Hawk’s behaviour easier to interpret than that Falcon’s. More work is needed to devise visualisation techniques that are better matched to agent complexity [6].
Conclusions and Future Work
ATCOs experienced agent transparency as a situated, ATCO-centred, achievement. Members’ methods featured prominently in the ways ATCOs queried choices of relevant aircraft and the quality of agent plans. ATCOs made sense of what they observed by moment-by-moment assessment of the state of the sector: which aircraft they judged to be relevant and what would be a good plan. In these ways, ATCOs calibrated their trust in the agent: their confidence in the agent’s reasoning (relevant aircraft), together with scrutiny of the quality of the agent’s behaviour (plans).
People’s trust in the tools they use is a volatile relationship; it is calibrated and sustained through practices distributed across the tool life cycle: development and testing, the training people receive in their use and their lived experience of working with them.
The members’ methods ATCOs drew on in this study to calibrate the trustworthiness of the agents are a central part of ATCOs’ community of practice [9]. If agents are to be embedded successfully in ATC, and in High Reliability Organisations in general, then more research is needed so that members’ methods can evolve in ways that ensure human-agent teaming is safe and effective.
Acknowledgments
Project Bluebird is a Prosperity Partnership between NATS, the University of Exeter and the Alan Turing Institute, funded by EPSRC (Grant Number EP/V056522/1).
References
References
- [1]NIST AI. 2023. Artificial intelligence risk management framework (AI RMF 1.0). URL: https://nvlpubs.nist.gov/nistpubs/ai/nist.ai (2023), 100–1.
- [2]Alejandro Barredo Arrieta, Natalia Díaz-Rodríguez, Javier Del Ser, Adrien Bennetot, Shiham Tabik, Alberto Barbado, Salvador García, Sergio Gil-López, Daniel Molina, Richard Benjamins, et al. 2020. Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information fusion 58 (2020), 82–115.
- [3]Gordon Baxter and Ian Sommerville. 2011. Socio-technical systems: From design methods to systems engineering. Interacting with computers 23, 1 (2011), 4–17.DOI
- [4]Benjamin J Carvell, Marc Thomas, Andrew Pace, Christopher Dorney, George De Ath, Richard Everson, Nick Pepper, Adam Keane, Samuel Tomlinson, and Richard Cannon. 2026. Human-In-the-Loop Testing of AI Agents for Air Traffic Control with a Regulated Assessment Framework. In AIAA SCITECH 2026 Forum. 2558.DOI
- [5]Harold Garfinkel. 2023. Studies in ethnomethodology. In Social Theory Re-Wired. Routledge, 58–66.
- [6]Elhassan Mohamed, Benjamin J Carvell, Rob Procter, Eseöghene Benjamin, George De Ath, and Richard Everson. 2026. Towards Transparent AI Agents for Air Traffic Control. In AIAA SCITECH 2026 Forum. 2869.
- [7]Andrew Monk, Lora Draper, Jeanne Haber, and Peter Wright. 1993. Improving your human-computer interface: A practical technique. Vol. 1. Prentice Hall New York.
- [8]Rob Procter and Mark Rouncefield. 2025. Trustworthy AI: UK Air Traffic Control Revised. arXiv preprint arXiv:2507.21169 (2025).arxiv.org/abs/2507.21169
- [9]Rob Procter and Mark Rouncefield. In Preparation. Interpersonal trust, institutional trust and trust in technology in Air Traffic Control. (In preparation).
- [10]Natividad Valle, M Florencia Lema, José Manuel Cordero, Enrique Iglesias, R Rodriguez, Gennady Andrenko, Natalia Andrienko, George A Vourous, Theocharis Kravaris, George Papadopoulos, et al. 2022. Transparency & explainability in higher levels of automation in the atm domain. SESAR Innovation Days 2022 (2022).
- [11]Etienne Wenger. 2000. Communities of practice and social learning systems. Organization 7, 2 (2000), 225–246.