WorksComputer Science
GNN-Based Deep Reinforcement Learning for Coastal Emergency Response under Extreme Water Levels
Abstract
When a storm surge approaches a coastal town, the emergency plan must be put into action quickly using water-level forecasts, flood extent, road access, vulnerable residents and shelters. Bringing this information together under time pressure is slow and depends on the duty officer's expertise. This paper is one step in a wider research line from data to safe decisions: machine learning to understand the hazard, reinforcement learning and planning to choose actions, large language models (LLMs) to explain and cross-check them, and an assurance layer that keeps people in charge. Here we address the action step on a real place. We ask whether a policy can learn the timing of emergency actions from decades of real events. Lido di Venezia is represented as a heterogeneous graph built from public elevation, road, building and census data and driven by hourly tide-gauge records from 1983 to 2024, with seven actions taken from the municipal emergency plan. A PPO policy is trained on this graph and compared, on storm windows unseen in training, with a rule-based system simulating the officer's discretionary decisions. On 18 historical test scenarios, PPO achieves a similar reduction in population exposure to the rule. With uncertain forecasts and limited protection it again performs similarly but does not outperform the tuned rule, and removing graph message passing does not substantially reduce performance. PPO therefore learns an effective protective response from an outcome-based reward alone: it is given the plan's actions and when each is allowed, but not the thresholds that tell the comparator when to act. Next steps test harder edge cases, move closer to a digital twin of the territory, and replace the simulated officer with assessment by emergency-management stakeholders, supported by LLM-based interpretation of each recommendation.
Introduction
When a storm surge approaches Lido di Venezia, the duty officers are responsible to implement an Emergency Plan [1] by observing data feeds, including water level forecasts, streets elevation, where vulnerable residents live, which shelters are open, among other information. This plan has two layers. Two actions are mandatory and executed directly on numeric triggers, an alert at cm and area evacuation at cm. Five actions are discretionary, including sandbags “at exposed points”, assisted evacuation where a route is passable, shelters, road closures and continued surveillance (Table 1 for details), whose choice and timing are left to human judgment. Bringing the data feed and the plan together under time pressure can be slow and depends on local expertise limited by the operator’s experience.
| Action | Plan § | Type | Preconditions | Effect | Lat. | Cost |
| monitor | 4.1.4.1, 4.1.5.5 | default | always legal (no-op) | — | 0 | 0 |
| issue_alert | 4.1.4.9 | mandate | 1-h forecast cm; not yet alerted | preparedness flag | 0 | 0 |
| deploy_sandbags | 4.1.4.8 | discr. | exposed zone; resources | +40 cm protection | 1 | 1 |
| close_road | 4.1.4.9 | discr. | junction water level flood threshold | junction blocked | 0 | 0 |
| open_shelter | 5.3.2, 4.1.4.8 | discr. | accessible; capacity ; not active; resources | shelter active | 1 | 1 |
| assisted_evacuation | 4.2.3.1, 4.2.3.4 | discr. | zone not evacuated; passable path to a shelter; resources | zone evacuated | 1 | 1 |
| full_evacuation | 4.2.3.1, 4.1.5.7 | mandate | 1-h forecast cm | all zones evacuated | 1 | 3 |
Table 1. The seven plan-derived actions [1]: type, legality preconditions, effect, latency (h) and cost (budget 5).
In our research, we explore if a policy can learn the timing of the discretionary actions from the record of real events, given a representation of the territory that carries its geography. We compare the performance of the policy with a simple, rule based simulated human decision based on the idea “if the -hour forecast maximum reaches , do action X, at most times”, which is based on the ideal world where an officer knows exactly what to do and when based on the rules given, which is often not the case. On the other hand, the learned policy is given only the objective, save lives, minimise displacement and cost, and never the rules. Importantly, the mandatory actions in the plan are not used for training, which focuses on the 5 actions that need human judgement.
Contributions. (i) An event-scale environment on real geography: the Lido graph from public data, seven plan-derived actions, and hourly replay of the 1983–2024 tide-gauge record. (ii) An evaluation protocol: rules represent the simulated officer, with tuning and test events and a pass/fail criterion fixed in advance. (iii) Three tests: in Test 1, with accurate forecasts and flexible protection, PPO reduces population exposure to a similar level as the tuned rule. In Test 2, with simulated changing and uncertain forecasts, PPO performs similarly but does not outperform the rule. In Test 3, removing graph message passing does not consistently reduce performance. The result is not that learning beats rules, but that PPO can uncover an effective response from outcomes alone, providing a basis for testing harder decisions where rules may be insufficient. Code and trained checkpoints are available at https://github.com/marcellosano/coastal-emergency-drl.
Related work
Reinforcement learning has already been applied to coastal and flood risk, mainly in two ways: to support long-term investment decisions under sea-level-rise uncertainty [2, 3, 4], and to optimise evacuation during individual or synthetic hazard events [5, 6]. Graph neural networks have also been used for flood modelling and infrastructure control [7, 8, 9], while action masking is commonly used to prevent an agent from selecting actions that are not allowed [10, 11]. In practice, however, emergency preparedness still relies heavily on threshold-based rules and triggers [12, 13]. Our work brings these elements together in a different way: hourly emergency decisions over a representation of real geography, actions and constraints derived from an operational plan, four decades of replayed water-level events, and a rule-based version of the plan’s judgement layer as the main comparator.
Environment and policy
Figure 1 shows the environment and the two deciders that act on it.
Figure 1. The environment and the two deciders, connected by state, actions and reward.
Hazard, forecasts and inundation
For the hazard, we replay the ISPRA Diga Sud Lido hourly record 1983–2024. The forecast is the gauge value 1 h ahead and its maxima over 6 and 12 h. In Test 1 the forecast is a hindcast (perfect) and protection can stack. In Test 2 an emulator adds a forecast error growing with lead time (2–6 cm over 1–12 h) and protection is capped at two sandbag deployments. For inundation, we use a bathtub rule: the gauge level, shifted to the elevation datum, is compared with ground height plus local protection;
a zone is exposed at a modelled depth cm and a junction impassable at cm. For simplicity, the rule ignores the MOSE barrier, so post-2020 lagoon-side inundation is counterfactual [14].
The territory as a graph
For the territory, we build a graph from the gauge record, the ISTAT 2021 census, the Tinitaly elevation model [15], OpenStreetMap [16] and the plan: 15 residential zones, 8 junctions, 3 shelters, 2 depots and 2 hazard nodes. We chose a graph because it represents the geography and is the path toward a digital twin: messages along edges give each zone the state of its junctions and shelters.
Plan-derived actions and outcomes
Actions: The plan gives the decider seven possible actions (Table 1). Each one is allowed only under certain conditions, takes up to an hour to have an effect, and uses part of a small crew budget. Two of the seven are mandatory: the alert and the area evacuation. The environment triggers these itself when the water-level thresholds are reached, for every decider alike, so they are never learned. The comparison between the learned policy and the rules therefore concerns only the five discretionary actions. A decider chooses what to do and when, but not where: if it chooses to deploy sandbags, the environment sends them to the zone with the highest forecast flood depth. At this stage, no decider makes a spatial choice.
Outcomes: Every hour we count three things: people living in flooded zones, people who were evacuated although their zone did not flood, and the crew cost of the action taken. Adding the first two up over a storm gives the two headline numbers of the paper, exposure person-hours and needless-displacement person-hours.
Learning the policy
A graph attention encoder [17] with two message-passing layers yields node representations, mean-pooled per type and concatenated with the scalar state; a policy output scores the seven actions under the legality mask [10] and a value output estimates the return. The reward at hour is
where residents, weighs needless displacement (0.1 and 0.4 were run as scenarios) and weighs crew cost. We train with proximal policy optimisation (PPO) [18] for 500,000 steps per seed on 8 parallel environments, undiscounted over the window.
Experiments and results
We evaluate on 24-hour storm windows, split by storm so that no tuning or test storm appears in training: 16 windows tune the rule-based comparator and 18 held-out windows test every decider. We compare four ways of completing the plan. Mandates only applies the two mandatory actions only. Plan-extended adds the plan’s own heuristic triggers for the discretionary actions. The static trigger is the simulated officer of the Introduction: the best of 300 candidate rules, chosen on the tuning windows. The learned policy is PPO, reported as mean std over 5 seeds; Test 3 repeats it with graph message passing switched off. The headline measure is exposure person-hours, compared window by window with the trigger. The criterion, fixed before any result was seen: the learned policy beats the rule only if its exposure is lower by more than the 95% bootstrap interval on most seeds.
In Table 2 (Test 1), the mandates alone leave 2,424 exposure person-hours per window. The tuned rule, one sandbag deployment when the 12-hour forecast reaches 120 cm, brings this down to 308; the learned policy reaches . The difference is within the uncertainty (95% interval [, ]) and no seed meets the criterion. The learned policy matches the tuned rule and does not beat it. It found the same move as the simulated officer: on 16 of 18 windows the outcomes are identical. In Test 3, removing message passing changes little (). In Test 2 (uncertain forecasts, capped protection), the rule tuned for those conditions does slightly better than the learned policy (645 versus ), and no seed passes.
| Policy | Exposure (person-h) | Displacement (person-h) | Peak exposed (persons) | Zero-exposure windows |
| Mandates only (literal plan) | 2,424 | 40,920 | 805 | 0.39 |
| Plan-extended | 1,439 | 47,269 | 722 | 0.39 |
| Static trigger (tune-selected) | 308 | 41,721 | 233 | 0.72 |
| No message passing | 42,007 | 210 | 0.74 | |
| Learned policy | 42,007 | 187 | 0.77 |
Table 2. Test 1 ladder, 18 test windows (per-window means; learned rows mean std, 5 seeds).
This geography cannot produce cases beyond the rules: exposure is concentrated in three zones, one carrying 90% of it, and the environment, not the decider, chooses where an action is applied, so each storm admits one good discretionary move and both deciders find it.
Discussion and conclusions
Sea-level rise will make extreme water-level events more frequent, increasing the need for emergency decisions that are timely, consistent and able to respond as conditions change. In this study, PPO broadly matches a rule-based system that represents a simplified version of the officer’s discretionary decisions, despite not being given the decision thresholds used by that system. This suggests that the policy can learn an effective protective response directly from the outcomes defined in its reward function. However, the current environment remains relatively simple, and removing graph message passing does not substantially reduce performance because neither PPO nor the rule-based system currently chooses where actions are applied.
The next stage of the research will move beyond comparison with a simulated officer, testing the recommendations of the deep reinforcement learning (DRL) policy directly with emergency-management stakeholders across a wider range of scenarios, including edge cases with competing priorities, limited resources, changing forecasts and disrupted access. Practitioners may assess whether the recommendations are appropriate, useful and operationally credible [19, 20]. Operationally, we may run two heads side by side: a rule engine, i.e., the plan’s judgement layer written as rules and auditable clause by clause (a rule base that does not yet exist), and the learned policy. When the two agree, the recommendation is consistent and explainable; when they disagree, either the rule has reached its limit or the policy is wrong, possibly informing future plan reviews. A large language model (LLM) based layer [21] can then help explain the recommendations and the disagreements, connect them to the emergency plan and available information, and support a human-in-the-loop decision process.
Acknowledgments and Disclosure of Funding
This work was supported by the European Union through the Marie Skłodowska-Curie Fellowship EXPEDITE, “Exploring Opportunities for Developing a Risk and Resilience Climate Service Based on Big Data and Machine Learning” (Grant Agreement No. 101067784). This work took place within the framework of the DoE 2023–2027 (MUR, AIS.DIP.ECCELLENZA2023_27.FF project). The authors also acknowledge the Horizon 2020 MYRIAD-EU project (Grant Agreement No. 101003276) for its contribution to the broader development of multi-hazard and multi-risk research relevant to this study.
References
- [1]Comune di Venezia. Piano comunale di protezione civile. Settore Protezione Civile, Deliberazione di Consiglio Comunale n. 15, 18 February 2009, 2009. URL https://www.comune.venezia.it/it/content/piano-comunale-protezione-civile.
- [2]Kairui Feng, Ning Lin, Robert E. Kopp, Siyuan Xian, and Michael Oppenheimer. Reinforcement learning–based adaptive strategies for climate change adaptation: An application for coastal flood risk management. Proceedings of the National Academy of Sciences, 122(12): e2402826122, 2025. doi: 10.1073/pnas.2402826122.DOI
- [3]Salman Sadiq Shuvo, Yasin Yilmaz, Alan Bush, and Mark Hafen. Modeling and simulating adaptation strategies against sea-level rise using multiagent deep reinforcement learning. IEEE Transactions on Computational Social Systems, 9(4):1185–1196, 2022. doi: 10.1109/TCSS.2021.3122282.DOI
- [4]Miguel Costa, Morten W. Petersen, Arthur Vandervoort, Martin Drews, Karyn Morrissey, and Francisco C. Pereira. Climate adaptation with reinforcement learning: Experiments with flooding and transportation in Copenhagen. In Tackling Climate Change with Machine Learning Workshop at NeurIPS 2024, 2024. URL https://www.climatechange.ai/papers/neurips2024/39.arxiv.org/abs/2409.18574
- [5]Fang Tang, Han Wang, and Maria Laura Delle Monache. Strategizing equitable transit evacuations: A data-driven reinforcement learning approach. Transportation Research Part C: Emerging Technologies, 180:105342, 2025. doi: 10.1016/j.trc.2025.105342.
- [6]Nicolas Pablo Viola. A reinforcement learning approach to real-time emergency evacuation. Master’s thesis, Uppsala University, Department of Information Technology, 2024. URL https://urn.kb.se/resolve?urn=urn:nbn:se:uu:diva-536728. Degree project 30 credits, Master Programme in Data Science, report IT mDA 24 012.
- [7]Roberto Bentivoglio, Elvin Isufi, Sebastiaan Nicolas Jonkman, and Riccardo Taormina. Rapid spatio-temporal flood modelling via hydraulics-based graph neural networks. Hydrology and Earth System Sciences, 27(23):4227–4246, 2023. doi: 10.5194/hess-27-4227-2023. URL https://doi.org/10.5194/hess-27-4227-2023.
- [8]Eli A. Meirom, Haggai Maron, Shie Mannor, and Gal Chechik. Controlling graph dynamics with reinforcement learning and graph neural networks. In Proceedings of the 38th International Conference on Machine Learning (ICML), volume 139 of Proceedings of Machine Learning Research, pages 7565–7577. PMLR, 2021. URL https://proceedings.mlr.press/v139/meirom21a.html.arxiv.org/abs/2010.05313
- [9]Deunsol Yoon, Sunghoon Hong, Byung-Jun Lee, and Kee-Eung Kim. Winning the L2RPN challenge: Power grid management via semi-Markov afterstate actor-critic. In International Conference on Learning Representations (ICLR), 2021. URL https://openreview.net/forum?id=LmUJqB1Cz8.
- [10]Shengyi Huang and Santiago Ontañón. A closer look at invalid action masking in policy gradient algorithms. In Proceedings of the 35th International Florida Artificial Intelligence Research Society Conference (FLAIRS-35), 2022. doi: 10.32473/flairs.v35i.130584. URL https://doi.org/10.32473/flairs.v35i.130584.
- [11]Anssi Kanervisto, Christian Scheller, and Ville Hautamäki. Action space shaping in deep reinforcement learning. In 2020 IEEE Conference on Games (CoG), pages 479–486. IEEE, 2020. doi: 10.1109/CoG47356.2020.9231687.DOI
- [12]Dina Vanessa Gomez Rave, Anna Scolobig, and Manuel del Jesus. Review article: Rethinking preparedness for coastal compound flooding: insights from a systematic review. Natural Hazards and Earth System Sciences, 25(10):3977–4002, 2025. doi: 10.5194/nhess-25-3977-2025. URL https://doi.org/10.5194/nhess-25-3977-2025.
- [13]Christian Ferrarin, Andrea Valentini, Martin Vodopivec, Dijana Klaric, Giovanni Massaro, Marco Bajo, Francesca De Pascalis, Amedeo Fadini, Michol Ghezzo, Stefano Menegon, Lidia Bressan, Silvia Unguendoli, Anja Fettich, Jure Jerman, Matjaž Ličer, Lidija Fustar, Alvise Papa, and Enrico Carraro. Integrated sea storm management strategy: the 29 October 2018 event in the Adriatic Sea. Natural Hazards and Earth System Sciences, 20(1):73–93, 2020. doi: 10.5194/nhess-20-73-2020. URL https://doi.org/10.5194/nhess-20-73-2020.
- [14]Piero Lionello, Robert J. Nicholls, Georg Umgiesser, and Davide Zanchettin. Venice flooding and sea level: Past evolution, present issues, and future projections (introduction to the special issue). Natural Hazards and Earth System Sciences, 21(8):2633–2641, 2021. doi: 10.5194/nhess-21-2633-2021. URL https://doi.org/10.5194/nhess-21-2633-2021.
- [15]Simone Tarquini, Ilaria Isola, Massimiliano Favalli, Francesco Mazzarini, Marina Bisson, Maria Teresa Pareschi, and Enzo Boschi. TINITALY/01: a new triangular irregular network of Italy. Annals of Geophysics, 50(3):407–425, 2007. doi: 10.4401/ag-4424.DOI
- [16]OpenStreetMap contributors. OpenStreetMap planet data. https://www.openstreetmap.org, 2024. Planet dump retrieved from https://planet.openstreetmap.org; licensed under the Open Database License (ODbL) 1.0, https://www.openstreetmap.org/copyright.
- [17]Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. In International Conference on Learning Representations (ICLR), 2018. URL https://openreview.net/forum?id=rJXMpikCZ.arxiv.org/abs/1710.10903
- [18]John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms, 2017. URL https://arxiv.org/abs/1707.06347. arXiv preprint arXiv:1707.06347.
- [19]Maria Katherina Dal Barco, Veronica Casartelli, Marcello Sanò, Sebastiano Vascon, Silvia Torresan, and Andrea Critto. Prioritise risks and improve adaptation strategies in the Veneto coast through the application of a custom AI tool. International Journal of Disaster Risk Reduction, 130:105818, 2025. doi: 10.1016/j.ijdrr.2025.105818.
- [20]Gabriel Dulac-Arnold, Nir Levine, Daniel J. Mankowitz, Jerry Li, Cosmin Paduraru, Sven Gowal, and Todd Hester. Challenges of real-world reinforcement learning: Definitions, benchmarks and analysis. Machine Learning, 110(9):2419–2468, 2021. doi: 10.1007/s10994-021-05961-4.
- [21]Marcello Sano, Davide Ferrario, Samuele Casagrande, Sebastiano Vascon, Silvia Torresan, and Andrea Critto. Deep reinforcement learning for operational coastal emergency response with AI agent orchestration and human oversight. In EGU General Assembly 2026, Vienna, Austria, May 2026. doi: 10.5194/egusphere-egu26-17369. URL https://meetingorganizer.copernicus.org/EGU26/EGU26-17369.html. Abstract EGU26-17369, session VPS14.DOI
