betaFeedback

WorksComputer Science

LLMs as World Models: Data-Driven and Human-Centered Pre-Event Simulation for Disaster Impact Assessment

Abstract

Efficient simulation is essential for enhancing proactive preparedness for sudden-onset disasters such as earthquakes. Recent advancements in large language models (LLMs) as world models show promise in simulating complex scenarios. This study examines multiple LLMs to proactively estimate perceived earthquake impacts. Leveraging multimodal datasets including geospatial, socioeconomic, building, and street-level imagery data, our framework generates Modified Mercalli Intensity (MMI) predictions at zip code and county scales. Evaluations on the 2014 Napa and 2019 Ridgecrest earthquakes using USGS ''Did You Feel It? (DYFI)'' reports demonstrate significant alignment, as evidenced by a high correlation of 0.88 and a low RMSE of 0.77 as compared to real reports at the zip code level. Techniques such as RAG and ICL can improve simulation performance, while visual inputs notably enhance accuracy compared to structured numerical data alone. These findings show the promise of LLMs in simulating disaster impacts that can help strengthen pre-event planning.

Introduction

Natural disasters often disrupt infrastructure, causing significant human and economic losses (Jones et al., 2022). Efficient impact assessment is critical for emergency response and evaluating community resilience (Ma et al., 2024). However, most existing methods are designed for post-event assessment, including expert inspections, ground sensors, and remote sensing (Li et al., 2021; Kucharczyk and Hugenholtz, 2021; Sarkar et al., 2023). While effective for characterizing observed damage, these approaches are reactive by nature and offer limited utility in pre-event planning, especially for sudden-onset events like earthquakes, where early awareness is crucial (Li et al., 2023). Traditional methods for pre-event simulation like scenario-based planning are useful (Ma et al., 2024; Deierlein et al., 2020), but they need extensive domain expertise for region-specific models and often lack empirical validation by addressing human-centered factors.

Advances in large language models (LLMs) have shown promise in contextual simulation and complex reasoning across various domains (Li et al., 2024a; Wang et al., 2024b). They are increasingly being viewed as potential world models—systems capable of learning to simulate and predict real-world scenarios [12] (Wong et al. 2023). Through training on large-scale datasets that encode spatial, temporal, and causal relationships, LLMs have shown potential in learning representations of how the world works. For example, current research has demonstrated their ability to understand environment status [12], plan household activities (Xiang et al., 2023), and predict time-series events (Lee et al., 2025).

In disaster management, while LLMs are not yet widely applied as “world models,” researchers have explored their utility in tasks such as damage detection from satellite imagery (Zhang and Wang, 2024) or social media (Wang et al., 2024a), and emergency identification (Otal et al., 2024). However, key research gaps still remain. First, existing studies mainly use LLMs to analyze available textual or visual data for post-event assessment, not to simulate pre-event situations. Second, while LLMs have well-demonstrated reasoning abilities, effective disaster assessment requires integrating domain-specific knowledge and data fusion so that these models can accurately reason about potential disaster scenarios. To address these limitations, this study poses the fundamental question in the context of sudden-onset disasters: Can LLMs simulate how humans perceive seismic risks before an event occurs?

To answer this, we develop an LLM-based framework to simulate how humans perceive seismic risks, as illustrated in Figure 1. By integrating rich pre-event contextual information, the LLMs are tasked with “reasoning” the likely severity of damage across spatial scales. Importantly, our study moves beyond theoretical simulations, grounding model evaluations in real-world events by testing on two actual earthquakes and comparing outputs against authoritative USGS reports. Our simulation demonstrates strong alignment with real reports at the zip code level, highlighting the potential of leveraging LLMs to improve pre-event planning.

Illustration of LLM-simulated human-centric sensors

Figure 1. An illustration of LLM-simulated human-centric sensors.

Related Work

Seismic Hazard Simulation

Researchers have employed empirical, physics-based, and data-driven methods for simulating seismic disasters. Traditional approaches characterize earthquakes, such as the moment magnitude MwM_w (Moschetti et al., 2024), and then utilize empirical ground-motion prediction equations (GMPEs) (Moschetti et al., 2024; Iervolino, 2023) or physics-based simulations (Deierlein et al., 2020) to estimate site-specific shaking. While these simulations can capture complex local effects and rupture dynamics, they demand extensive data and computational resources. Recent data-driven methods have used machine learning to learn damage patterns from historical events, simulations, and remote sensing data [6] (Yu et al. 2020). These AI-driven techniques offer advantages in scalability and flexible feature integration but are dependent on high-quality labeled data, may struggle with generalization, and often present challenges in interpretability.

A major gap across these models, whether physics-based or data-driven, is the limited integration of human-perceived shaking into predictive frameworks. The USGS has developed the “Did You Feel It” (DYFI) system [4], a crowdsourced platform where individuals report the shaking intensity they experience after an earthquake. These reports are aggregated and converted into Modified Mercalli Intensity (MMI) (U.S. Geological Survey, 1989), which provide a human-centric, ground-truth reference for how seismic shaking is felt. While DYFI has been widely used to validate models or interpolate intensities after earthquakes, simulating human-centric perceived risks for pre-event could be important for developing predictive frameworks that anticipate not only physical shaking but also tangible impacts on communities.

LLM as World Models

Recent advance in LLMs such as GPT-4 [2] and deepseek-R1 [11] has motivated researchers to leverage LLMs to solve complex tasks, such as reasoning (Li et al., 2025; Yu et al., 2025) and domain-specific exploration (?Yang et al., 2024; Zhang et al., 2024). In the context of disaster, researchers have leveraged LLMs to process multimodal information for vulnerability evaluation (Martelo et al., 2024), impact assessment [3] (Taghian Dinani et al. 2023), information coordination (Yu and Wang, 2024), and recovery planning (White and Liptak, 2025).

Beyond conventional question-answering and analytical tasks (Yang et al., 2025), LLMs are being used as world models [12] (Zhao et al. 2023) for complex simulation. While definitions of world models vary, their central concept involves leveraging forward reasoning to predict future states and dynamics in real-world scenarios. For example, they can produce large-scale simulations—an LLM-driven city simulation platform, OpenCity, runs tens of thousands of agents to emulate daily urban activities, successfully reproducing emergent patterns like mobility flows and residential segregation (Yan et al., 2024). In disaster scenarios, LLM-based world models have been explored in flooding forecasting (Wang et al., 2025) and evacuation planning [13].

Building on these advances, we propose leveraging LLMs as simulation tools to estimate how humans might perceive and report seismic risks before an event occurs. Our work addresses two critical gaps: first, the limited pre-event simulation of seismic risk; and second, the underexplored application of LLMs as world models for deriving human-centric insights.

Data and Methods

Framework Design

To simulate disaster impacts before events, we propose a novel framework that treats LLM as synthetic observers. This framework conceptualizes LLMs as “virtual sensors” capable of “perceiving” multimodal inputs and “reasoning” about disaster risks based on the MMI levels (see appendix § 9) that approximate human perception of shaking. As illustrated in Figure 1, we associate each sampled spatial location with a bundle of features, including seismic distance, site conditions, local building characteristics, and socioeconomic factors. In addition, Google Street View provides a first-person view of the built environment. Collectively, this feature set closely replicates the perceptual input available to disaster responders during an event.

Formally, let each sample ii be associated with a fused feature representation. We specifically select the following features that prior research has illustrated as critical predictors of seismic risk. [8] (Kassem et al. 2020; Riedel et al. 2015; Mori et al. 2020).

Xi={Ei,Gi,Li,Bi,Si,Vi}\mathcal{X}_i = \{E_i, G_i, L_i, B_i, S_i, V_i\}
  • EiE_i: Earthquake parameters (e.g., magnitude, epicenter distance, depth),

  • GiG_i: Geospatial features (e.g., VS30),

  • LiL_i: Location metadata (e.g., state, city, zip code),

  • BiB_i: Building attributes (e.g., number, type, height, material),

  • SiS_i: Socioeconomic indicators (e.g., population density, income),

  • ViV_i: Street-level view (Google Street image).

The LLM acts as a reasoning function fθf_\theta with parameters θ\theta, generating both a reasoning trace and an MMI rating:

y^i,ei=fθ(Xi),y^i∈{I,II,…,XII}\hat{y}_i, e_i = f_\theta(\mathcal{X}_i), \qquad\hat{y}_i \in\{\mathrm{I}, \mathrm{II}, \ldots, \mathrm{XII}\}

The full pipeline consists of five components, as shown in Figure 2: (1) spatial sampling, (2) data fusion, (3) prompt engineering, (4) experiment design, and (5) result analysis, which we specifically explain in the following sections.

Figure 2. An illustration of the framework design.

Data Sampling

In Step 1 (Figure 2), we use a polygon-based GIS shapefile to define administrative zones (e.g., zip codes) and apply within-polygon stratified random sampling to ensure spatial representativeness across the study area. Let: Z={z1,z2,…,zM}\mathcal{Z} = \{z_1, z_2, \ldots, z_M\} represent the set of all zip code polygons. For each polygon zjz_j, we sample:

Pj={pj1,pj2,…,pjnj}∼Uniform⁡(zj)\mathcal{P}_j = \{p_{j1}, p_{j2}, \ldots, p_{jn_j}\} \sim\operatorname{Uniform}(z_j)

ensuring that points are uniformly drawn from within each polygon’s spatial boundary. We then sample 50 data points per zip code. This stratified random sampling strategy can help ensure spatial representativeness and mitigate biases associated with population density or urban–rural areas.

Data Fusion

In Step 2 (Figure 2), for each sampled point pjip_{ji}, we collect and assemble the feature set Xji\mathcal{X}_{ji} from open-source and public datasets including USGS earthquake data, OpenStreetMap building data, American Community Survey (ACS) data, and Google Maps street views.

Earthquake data, site conditions, and location information. We source earthquake parameters, including moment magnitude, epicentral coordinates, and focal depth, from the USGS ShakeMap product (Wald et al., 2006) as EE. To account for local site amplification and deamplification, we incorporate the USGS VS30 dataset (McPhillips et al., 2020) as the geospatial features GG, a widely used proxy for near-surface geological effects.

We also incorporate location information, including latitude, longitude, state, county, zip code, and the distance from the epicenter for each sampled point. Additionally, we utilize MMI measurements from the USGS DYFI program as ground-truth labels, represented by yjy_j for each zip code.

Building description. We use OpenStreetMap ((Ramm and Topf, 2010)) (OSM) data to extract building features BB, including the total number of buildings, type distribution, height range, and average height within a 100-meter radius of each sampled point. Meanwhile, we summarize the prevalence of major construction materials (e.g., concrete, masonry, timber, steel). These features can help characterize the distribution, physical attributes, and seismic-design status of buildings within the sensor’s surroundings.

Socioeconomic and demographic factors. We collect socioeconomic and demographic factors SS at the Census Block Group (CBG) level from the American Community Survey (ACS) ((USC, 2022)). We spatially join the coordinates of each randomly selected sample to their corresponding CBG polygon and extract relevant ACS key indicators, including population, population density, urbanization ratio, 65- and over-age proportion, median household income, and higher education attainment rate. These variables imply a quantitative evaluation of population vulnerability in a disaster event.

Google Maps street imagery. We further incorporate Google Street View imagery VV to enrich the environmental context at each sampled location. These street-level images capture fine-grained visual cues of the surrounding built environment—such as building, vegetation, curb conditions, and street density—that are difficult to numerically encode but essential for human-like visual reasoning. Using the Google Maps API ([10]), we query the available image for each point in our sampling grid. This visual context allows the LLM to “see” the landscape as if it conducts a field visit.

Prompt Design

To guide the reasoning process of the LLM and ensure interpretability and consistency, we design a prompt template that mirrors the workflow of a seismic domain expert. The prompt follows a role-based instruction format in which the model is assigned the role of a seismic specialist responsible for evaluating earthquake damage using the MMI scale. The MMI scale provides a human-centric interpretation that consists of twelve levels describing the severity of earthquake shaking. The detailed descriptors of MMI are attached in Appendix § 9.

The prompt includes six distinct sections corresponding to the multimodal feature set Xi\mathcal{X}_i introduced earlier: earthquake parameters EiE_i, geospatial features GiG_i, location metadata LiL_i, building descriptions BiB_i, socioeconomic context SiS_i, and streetlevel view ViV_i. The model’s response is constrained to a JSON format with two fields: a free-text reasoning explanation and the predicted MMI category (see prompt template in Appendix § 10). We implement Chain-of-Thought (CoT) to instruct LLM output a detailed reasoning justifying its assessment before final MMI estimate.

Experiment Design

LLM Selection: We select both open- and closed-source LLMs to conduct the simulation. For open-source LLMs, we use models from Llama-3.2 and Qwen-2.5 family with various sizes, as listed in Table 1.

ModelOpen SourceNapaRidgecrest
RMSEZ_Z ↓CorrZ_Z ↑RMSEC_C ↓CorrC_C ↑RMSEZ_Z ↓CorrZ_Z ↑RMSEC_C ↓CorrC_C ↑
GPT-4o-2024-08-06✗2.430.772.370.881.970.751.910.77
GPT-4.1-mini✗2.560.612.480.670.920.640.770.76
Claude-3.5-haiku✗2.110.582.050.701.350.591.380.71
Llama-3.2-11B-VI✓3.190.443.050.863.220.333.220.27
Llama-3.2-90B-VI✓2.620.572.550.662.060.622.190.59
Qwen2.5-VL-3B✓3.630.293.590.153.880.014.08-0.20
Qwen2.5-VL-7B✓1.790.431.680.701.530.051.59-0.18
Qwen2.5-VL-32B✓1.590.701.560.790.990.710.960.80
Qwen2.5-VL-72B✓2.170.462.120.441.390.641.280.86

Table 1. Main experiment results, the best result for each metric is in bold.

Prompting Techniques: We incorporate the following prompting techniques in our experiment:

  • In-Context Learning (ICL): This helps LLM adapt to tasks by incorporating examples directly within the input prompt. ICL is applied by embedding a detailed MMI reference guide directly within the prompt.

  • Retrieval-Augmented Generation (RAG): It combines information retrieval with text generation that allows LLMs to access external data sources at inference. We provide LLM with a set of multimodal features and the reported MMI within the prompt as the retrieved context to ground their MMI estimates.

Evaluation: The LLM generates a MMI prediction, y^ji=fθ(Xji)\hat{y}_{ji}=f_\theta(X_{ji}), for each sampled location ii within a predefined administrative area jj (e.g., zip code, county). These point-level predictions are then aggregated to compute an average predicted MMI for each area jj. If area jj contains njn_j samples, its average predicted MMI y^ˉj\bar{\hat{y}}_j is calculated as:

y^ˉj=1nj∑i=1njy^ji\bar{\hat{y}}_j = \frac{1}{n_j}\sum_{i=1}^{n_j}\hat{y}_{ji}

Similarly, a corresponding ground-truth MMI value for area jj, denoted as yˉj\bar{y}_j, is derived from USGS DYFI reports. To quantify the model’s predictive accuracy using these aggregated area-level values, we compute two metrics. First, the Root Mean Square Error (RMSE) is used:

RMSE=1N∑j=1N(y^ˉj−yˉj)2\mathrm{RMSE} = \sqrt{\frac{1}{N}\sum_{j=1}^{N}(\bar{\hat{y}}_j-\bar{y}_j)^2}

where NN is the total number of administrative areas being evaluated (indexed by jj). Second, we calculate Pearson’s correlation coefficients rr to assess the strength and direction of the association between the LLM-predicted MMI (y^ˉj\bar{\hat{y}}_j) and the ground-truth MMI (yˉj\bar{y}_j). These evaluations are performed at both zip code and county levels.

Experimental Results

We select two cases to demonstrate our proposed framework: (1) the 2014 Napa earthquake (magnitude 6.0) and (2) the 2019 Ridgecrest earthquake (magnitude 7.1), both of which occurred in California, U.S. (see details in Appendix § 11). For each case, we use the USGS DYFI reports as the ground-truth dataset (U.S. Geological Survey, 2014a, 2019a). Additionally, for each case, we compile feature sets for 50 sample points from each of the top 100 zip codes with the highest number of responses, resulting in 5,000 samples per event. Due to limitations in Google image availability for the Napa case, only 4,920 samples are retrieved. Comparisons with DYFI data are first conducted at the zip code level, where each of the 100 aggregated values represents the average of 50 simulated samples.

Figure 7 presents the spatial distribution of predicted MMI at the zip code level for the Napa and Ridgecrest earthquakes. This visualization highlights variations in simulated seismic impacts across geographical areas and among different LLMs. Based on the best-performing models (lowest RMSE: GPT-4.1-mini for the 2019 Ridgecrest earthquake and Qwen-2.5-32B for the 2014 Napa earthquake), we observe consistent geospatial patterns in both cases. Specifically, the simulations indicate elevated perceived risk near the epicenter (a red star mark in Figure 7), with diminishing simulated impact as distance increases. Moreover, the LLM-based predictions align well with DYFI reports from these two events: it is important to note that the Napa earthquake, despite its lower magnitude, led to more significant impacts.

Spatial distribution of predicted MMI for the 2014 Napa earthquake and the 2019 Ridgecrest earthquake

Figure 7. Spatial distribution of predicted MMI at zip code level: (A) The 2014 Napa earthquake, and (B) the 2019 Ridgecrest earthquake based on GPT4o, GPT-4.1-mini, Claude-3.5-haiku, Llama-3.2-90B, Qwen-2.5-32B, and Qwen-2.5-72B. The red star sign shows the epicenter of the earthquake. These maps compare outputs from different LLMs, showing consistent geospatial patterns with high-intensity predictions concentrated near epicenters

The following sections are organized below. First, we evaluate the performance of the selected LLMs, comparing their accuracy using quantitative metrics, and examining the influence of model scaling and prompting strategies. Next, we conduct an input feature analysis to evaluate how different data modalities can impact predictive performance. Lastly, the output reasoning analysis explores the internal decision-making processes of the models, which identifies linguistic nuances that illustrate how LLMs interpret the inputs.

Model Performance

Before the main experiment, we perform a data leakage test using two close-source models—Claude-3.5-haiku and GPT-4.1-mini—to show that our simulation is free of data leakage issues (see Appendix § 12). The main experiment results are shown in Table 1, from which we draw the following findings:

LLMs deliver promising simulation results. As shown in Table 1, all of the best correlation scores across datasets and area levels exceed 0.7, indicating a strong alignment between the predicted outcomes and the ground truth impact labels. These high correlation values suggest that LLMs hold significant potential for effectively simulating human-perceived risks in disaster scenarios.

Closed-source LLMs generally outperform open-source models. Table 1 shows a clear trend that closed-source LLMs consistently outperform open-source counterparts, achieving the best results in 6 out of 8 cases. This suggests that commercial closed-source models possess stronger geospatial reasoning capabilities and align more closely with human judgment in disaster sensing tasks (Li et al., 2024b). Among the open-source models, Qwen-2.5-32B obtains the top results with lowest RMSE for the Napa case.

RMSE and correlation can be inconsistent. Another noteworthy observation from Table 1 is that the two evaluation metrics—correlation and RMSE—do not always align. For instance, Llama-3.2-11B shows a high correlation but a poor RMSE at the county level for the Napa case. This discrepancy arises because correlation captures the model’s ability to predict relative ordering of seismic impacts, whereas RMSE reflects the absolute prediction errors. Thus, even when models effectively estimate the relevant severity (lower RMSE), they may not correctly distinguish between higher- or lower-impact areas (lower correlation).

We then evaluate the influence of incorporating demonstration methods with RAG and ICL on the simulation, as shown in Figure 3. These demonstration techniques can enhance the capacity of LLMs to generalize from provided information. In particular, the enhanced models consistently show higher performance, illustrated by decreasing RMSEZRMSE_Z. It is also well-noted that even limited demonstrations can greatly improve model predictions.

Figure 3. Demonstration analysis on RAG and ICL.

Input Feature Analysis

In this section, we examine how input features beyond earthquake and location information affect LLMs’ simulation performance. We conduct experiments using Claude-3.5-haiku and Qwen-2.5-7B, with the results presented in Figure 4. Interestingly, we find that only street view information contributes to improved simulation performance. In contrast, removing any of the other three features alone—geospatial, building, or socioeconomic data—can decrease the zip code-level RMSE.

Figure 4. Input information feature analysis results.

We attribute this performance degradation to several potential factors: (1) limitations of LLMs in processing numerical information as they could complicate LLM’s reasoning process [5] (Yin et al. 2024); (2) the lack of domain-specific knowledge required to interpret geospatial, building, and community-related data [9]; (3) the inherent limitations of the self-attention mechanism in capturing spatial adjacency and performing geometric reasoning (Requeima et al., 2024).

Output Reasoning Analysis

To complement our quantitative evaluations, we examine how GPT-4.1-mini and Qwen-2.5-32B (lowest RMSE) reason when predicting MMI values. We summarize our findings as below:

LLMs capture seismic attenuation but underutilize local site conditions. As shown in Figure 5, both models display a clear negative correlation between epicentral distance and predicted MMI, most notably in Qwen2.5-32B’s Napa earthquake predictions, which indicates that LLMs have internalized the concept of seismic attenuation. However, the relationship between VS30 values (a proxy for local ground conditions) and MMI is weak across both models. High MMI values occur almost exclusively near the epicenter, suggesting limited sensitivity to local site effects.

Scatter plots showing output reasoning analysis in terms of distance and VS30 for Qwen-2.5-32B Napa and GPT-4.1-mini Ridgecrest

Figure 5. Output reasoning analysis in terms of (A) distance (where the x-axis is the predicted MMI, and y-axis is the distance from the epicenter (km)) and (B) VS30 (where the x-axis is the predicted MMI, and the y-axis the local site condition represented by VS30 (m/s)).

LLMs use distinct lexical cues for MMI reasoning across multimodal inputs. Figure 6 presents a taxonomy of language used by both models across three perspectives: buildings, socioeconomic context, and street-level imagery. Unigram analysis reveals that GPT-4.1-mini and Qwen2.5-32B employ different vocabularies and reasoning styles. For the building assessment, GPT-4.1-mini adopts a descriptive and hedged reasoning style. At low MMI levels, it uses terms like “enhance,” “robust,” and “improve,” while shifting to “suffer” and “detached” at higher levels. Qwen2.5-32B relies on more technical terms, such as “compliance” and “stringent” at low levels, and “crack” and “susceptibility” at higher ones.

Figure 6. Output reasoning analysis in terms of different input features with (A) Qwen-2.5-32B for the Napa case, and (B) GPT-4.1-mini for the Ridgecrest case. The x-axis is the log(TF-IDF), while the y-axis is the predicted MMI.

Socioeconomic reasoning diverges in focus and tone. GPT-4.1-mini links lower MMI levels to terms like “urbanized” and “welleducated,” and higher levels to “industrial” and “heavy,” occasionally incorporating cautious language such as “possibly” or “suggest.” Qwen2.5-32B emphasizes systemic vulnerability, shifting from “limited” and “stable” to “cascade” and “amplification” as predicted MMI increases.

Visual reasoning contrasts environmental vs. structural emphasis. GPT-4.1-mini references broader environmental cues, from “forest” and “agricultural” at lower levels to “catastrophic” at higher ones. In contrast, Qwen2.5-32B focuses on structural compliance, mentioning “firm,” “code,” and “reinforced” in a progression of seismic risk.

Discussion

Based on our experimental result analysis, we conclude the following insights and opportunities for future study:

  • LLMs can effectively simulate human-centric seismic risks, showing strong alignment with real-world USGS DYFI reports. This can help develop pre-event impact assessment. One promising direction is to explore broader application in LLM-based simulations for disaster responses.

  • Simulation performance depends on model type, size, and input information: closed-source and larger models generally perform better; techniques like RAG and ICL help; street-level imagery boosts accuracy, while structured data may hinder it due to modality alignment limitations. Future works can explore more effective prompt-ing strategies and reasoning structures to further improve the simulation performance.

  • LLMs show diverse reasoning styles and strong practical value, as model-specific rhetorical patterns reflect architectural and data differences. These differences suggest the potential impact of training data and model architecture on LLMs’ reasoning styles. It would be meaningful to further explore the reasoning mechanisms of LLMs when used as world models.

Our study has significant practical implications. Leveraging LLMs and open-source data to simulate seismic risk enables authorities to proactively assess potential disaster impacts. Integrating our framework into early-warning systems can also help identify vulnerable communities and enhance disaster response.

Conclusions

Our study demonstrates the potential of using LLMs to simulate seismic risk before an earthquake occurs. The alignment between model predictions and real-world reports highlights the importance of multimodal inputs and advanced LLM techniques like RAG and ICL in simulation settings. Moreover, our findings reveal that simulation performance can vary across LLMs and input features. These results make a meaningful contribution to advancing data-driven, human-centric simulation with LLMs for real-world scenarios.

Limitations

Several limitations warrant further consideration.

First, our experiments focus only on two earthquake cases—the 2014 Napa and 2019 Ridgecrest events—which may not fully represent global variations in seismic hazards, urban densities, and construction practices. Therefore, generalizability requires additional validation.

Second, despite employing stratified random sampling within zip codes, potential biases remain due to gaps in data availability, such as incomplete Google Street View coverage or inconsistencies in socioeconomic and building datasets. This may lead to underrepresentation of certain neighborhoods or misalignment with high-risk zones.

Third, although our framework integrates diverse heterogeneous data, we do not conduct detailed feature selection or examine individual parameters (e.g., housing age, infrastructure proximity). Consequently, interpreting model behavior at a granular level requires further investigation.

Ethics

Responsible data use. Our research relies exclusively on publicly available and anonymized datasets, including geospatial, demographic, and imagery data from USGS, OpenStreetMap, and Google Street View. All analyses are conducted at aggregated administrative levels (zip code and county), minimizing risks to individual privacy.

Deployment guidance. It is important to recognize that LLM-based simulations cannot fully capture the complexity and diversity of individual experiences in real disaster scenarios. Therefore, model predictions should be viewed as supportive tools rather than replacements for expert judgment, and used responsibly with validated empirical assessments in practical decision-making contexts.

Modified Mercalli Intensity (MMI) Scale

Table 2 shows the MMI scale used to support the classification of seismic risks in our study.

MMI LevelDescription
INot felt except by a very few under especially favorable conditions.
IIFelt only by a few persons at rest, especially on upper floors of buildings. Delicately suspended objects may swing.
IIIFelt quite noticeably by persons indoors, especially on upper floors of buildings. Many people do not recognize it as an earthquake. Standing motor cars may rock slightly. Vibration similar to the passing of a truck. Duration estimated.
IVFelt indoors by many, outdoors by few during the day. At night, some awakened. Dishes, windows, doors disturbed; walls make cracking sound. Sensation like heavy truck striking building. Standing motor cars rocked noticeably.
VFelt by nearly everyone; many awakened. Some dishes, windows broken. Unstable objects overturned. Pendulum clocks may stop.
VIFelt by all, many frightened. Some heavy furniture moved; a few instances of fallen plaster. Damage slight.
VIIDamage negligible in buildings of good design and construction; slight to moderate in well-built ordinary structures; considerable damage in poorly built or badly designed structures; some chimneys broken.
VIIIDamage slight in specially designed structures; considerable damage in ordinary substantial buildings with partial collapse. Damage great in poorly built structures. Fall of chimneys, factory stacks, columns, monuments, walls. Heavy furniture overturned.
IXDamage considerable in specially designed structures; well-designed frame structures thrown out of plumb. Damage great in substantial buildings, with partial collapse. Buildings shifted off foundations.
XSome well-built wooden structures destroyed; most masonry and frame structures destroyed with foundations. Rail bent.
XIFew, if any (masonry) structures remain standing. Bridges destroyed. Rails bent greatly.
XIIDamage total. Lines of sight and level are distorted. Objects thrown into the air.

Table 2. Description of the Modified Mercalli Intensity (MMI) scale (U.S. Geological Survey, 1989).

Prompt Template Design

SYSTEM_PROMPT = """
You are a seismic expert specialized in earthquake damage assessment and
disaster response. You analyze earthquake data, local conditions, and
building characteristics to provide damage assessments using the Modified
Mercalli Intensity (MMI) scale.
"""
EARTHQUAKE_PROMPT = """
The earthquake happened date is 2025-06-01.

Here is the EARTHQUAKE information.
- Epicenter: {eq_place}
- Coordinates: {eq_lat}, {eq_lng}
- Magnitude: {eq_magnitude} mw
- Depth: {eq_depth} km

YOUR LOCATION information is listed below.
- State: {state}
- City: {city}
- Zipcode: {zipcode}
- Coordinates: {lat}, {lng}
- Distance from epicenter: {distance} km

## Geospatial features in YOUR LOCATION
- VS30 at your location: {vs30} m/s
(VS30 represents the time-averaged shear-wave velocity (VS) to a depth of 30
 meters, which is a key index to account for seismic site conditions)

## Building Description in YOUR LOCATION (within a 100-meter radius)
- Building description: {building}
## Community Socioeconomics and Demographics in YOUR LOCATION (at Census Block
Group level)
- Population density: {population_density} people per square km
- Urban population percentage: {urban_population_pct}%
- Over 65 percentage: {over_65_rate}%
- Median household income: ${median_household_income}/year
- Education (bachelor's or higher): {education}%

## Visual Context in YOUR LOCATION
The image provided shows your surrounding environment and infrastructure.

Based on the information provided, ASSESS the potential earthquake damage level
using the Modified Mercalli Intensity (MMI) scale.
1. Identify the damage level.
2. Explain your reasoning by addressing the following factors and considering
   the visual context.
- Distance to the epicenter and earthquake magnitude
- Geospatial features
- Infrastructure quality and building characteristics
- Population density and socioeconomic vulnerabilities
- Visual image of surroundings

The following is an abbreviated description of the 12 levels of Modified
Mercalli intensity. {MMI Scale}

Output the result in JSON format:
{{
  "Reasoning": "<Provide reasoning>"
  "MMI": "<Respond MMI level>",
}}
"""

Earthquake Scenarios

2014 Napa Earthquake ((U.S. Geological Survey, 2014b)). On August 24, 2014, a magnitude 6.0 earthquake struck near Napa, California, causing significant structural damage despite its moderate magnitude. Approximately 613 buildings were tagged for various degrees of structural integrity concerns, including fractures, road cracks, and damage to wine storage facilities. The earthquake resulted in one death and nearly 200 injuries.

2019 Ridgecrest Earthquake ((U.S. Geological Survey, 2019b)). The Ridgecrest earthquake occurred on July 6, 2019, with a magnitude of 7.1, significantly larger than the Napa event but with fewer human casualties. The quake damaged around 50 homes, caused gas leaks and road cracks, and triggered fires in residential properties. Significant infrastructural damage occurred at the Naval Air Station, and widespread power outages were reported.

Data Leakage Test

To further assess the potential for data leakage in the LLMs used in our experiments, we conduct a leakage test on Claude-3.5-haiku and GPT-4.1-mini, the two best-performing models in our main study. Specifically, we remove city and state names from the prompt—two elements most likely to serve as shortcuts for the models to associate with MMI levels and potentially memorize. As shown in Table 3, the removal of location information does not significantly affect the models’ simulation performance. It is fair to rule out the possibility of data leakage in our main results.

ModelEarthquake prompt RMSE_ZEarthquake prompt Corr_ZEarthquake prompt RMSE_CEarthquake prompt Corr_CEarthquake prompt w/o location RMSE_ZEarthquake prompt w/o location Corr_ZEarthquake prompt w/o location RMSE_CEarthquake prompt w/o location Corr_C
claude-3-5-haiku2.110.582.050.702.350.382.260.62
gpt-4.1-mini2.560.612.480.672.670.622.580.73

Table 3. Experiment results on data leakage test.

Spatial Distribution of Predicted MMI by LLMs

Figure 7 shows the spatial distribution of predicted MMI for the 2014 Napa earthquake and the 2019 Ridgecrest earthquake at zip code level from different LLMs.

Scaling Law Analysis

To investigate the impact of LLM scaling laws on simulation performance, we conduct a scaling analysis using the Llama-3.2 and Qwen-2.5 model families. As shown in Table 4, we observe that performance in simulation generally improves with model size, excepting Qwen-2.5-72B. This indicates that the simulation performance tend to be strengthened with a larger size of LLMs.

Model FamilyModel SizeNapa RMSERidgecrest RMSE
Llama-3.211B3.193.22
Llama-3.290B2.622.06
Qwen-2.53B3.633.88
Qwen-2.57B1.791.53
Qwen-2.532B1.590.99
Qwen-2.572B2.171.39

Table 4. Scaling law analysis: Zipcode-level RMSE across model sizes (in billions of parameters).

Samples of LLM Output

Samples of LLM Reasoning Outputs with Different MMI Predictions
GPT-4.1-mini | 2019 Ridgecrest | Predicted MMI: III. The earthquake has a high magnitude of 7.1 Mw, which is capable of causing significant damage near the epicenter. However, the location in question, San Diego, CA, is approximately 339 km away from the epicenter in Ridgecrest, CA. At this distance, seismic waves would have attenuated considerably, reducing the intensity of shaking experienced. The VS30 value of 417 m/s indicates a site with relatively soft soil conditions, which can amplify shaking somewhat, but given the large distance, this effect is limited. The building stock consists mostly of general buildings and schools, likely built to modern California seismic codes, which enhances resilience. The population density is moderate, with a well-educated and relatively affluent community, reducing socioeconomic vulnerabilities. The visual context shows a well-maintained urban street with no visible signs of damage or distress, supporting the assessment of low shaking impact. Considering all these factors, the expected Modified Mercalli Intensity (MMI) level in San Diego would be around III, where shaking is noticeable indoors but causes no damage.
GPT-4.1-mini | 2019 Ridgecrest | Predicted MMI: IV. The earthquake has a high magnitude of 7.1 Mw, which is capable of causing significant damage near the epicenter. However, the location in Ventura, CA is approximately 221 km away from the epicenter in Ridgecrest, CA, which greatly reduces the intensity of shaking experienced. The VS30 value of 391 m/s indicates a site with relatively soft soil conditions, which can amplify shaking somewhat, but the distance still plays a dominant role in reducing impact. The building information is not available, but the visual context shows modern infrastructure such as a highway bridge and vehicles, suggesting reasonably good construction standards typical of California. The population density is low (65 people/km²) with a moderate urban population percentage (25.43 %), and socioeconomic indicators such as median income and education level are moderate, implying some resilience but also potential vulnerabilities in older or poorly maintained structures. Given the distance, moderate site conditions, and infrastructure, the expected shaking intensity would be moderate with little to no structural damage. This corresponds to an MMI level around IV to V, where shaking is felt by many but is minimal or limited to minor non-structural effects.
GPT-4.1-mini | 2019 Ridgecrest | Predicted MMI: V. The earthquake had a high magnitude of 7.1 Mw, which is capable of causing severe damage near the epicenter. However, the location in question, Apple Valley, CA, is approximately 152.25 km away from the epicenter, which significantly reduces the intensity of shaking experienced. The VS30 value of 607 m/s indicates relatively firm ground conditions, which typically reduce amplification of seismic waves. The visual context shows a rural, sparsely populated area with no visible dense infrastructure or tall buildings, suggesting limited exposure to structural damage. The population density is low (18.73 people per square km), and the urban population is only about 32.65 %, further reducing potential damage and casualties. Socioeconomic factors such as median income and education level are moderate, but given the lack of detailed building information and the rural setting, the risk of significant structural damage is low. Considering these factors, the shaking would likely be felt by many but cause only slight damage, consistent with MMI level VI.
Qwen-2.5-32B | 2014 Napa | Predicted MMI: VI. The location is 19.27 km from the epicenter of a 6.0 Mw earthquake, indicating strong shaking. The VS30 value of 344 m/s suggests stiff soil, reducing amplification but not eliminating damage due to proximity. The presence of general buildings and houses, along with a rural setting and a high percentage of older residents, points to moderate to strong damage. The visual context shows a well-maintained residential area, which may indicate better-quality buildings, but this is not definitive.
Qwen-2.5-32B | 2014 Napa | Predicted MMI: VII. The earthquake has a magnitude of 6.0 Mw, and the location is 14.59 km from the epicenter, indicating strong shaking. The VS30 value of 354 m/s suggests moderate amplification. The area consists of general buildings, and the visual context shows industrial/commercial structures that appear relatively robust. High population density (9,318.61 people per sq km) increases risk, but affluence and education suggest better infrastructure. Considering these factors, the damage is expected to be noticeable but not severe.
Qwen-2.5-32B | 2014 Napa | Predicted MMI: VIII. The earthquake has a magnitude of 6.0 Mw and occurred 10.24 km from the epicenter. The firm ground conditions (VS30 = 508 m/s) reduce shaking intensity. The area is sparsely populated (23.04 people per sq km), with low urbanization (2.97 %) and high affluence ($93,750 median income). The visual context shows a rural landscape with minimal infrastructure, indicating low vulnerability to structural damage. Based on these factors, the expected damage level is moderate.

Table 5. Samples of LLM outputs with predicted MMI reasoning for selected earthquake scenarios.

References

  1. [2]Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. Gpt-4 technical report. ArXiv preprint, abs/2303.08774.arxiv.org/abs/2303.08774
  2. [3]Temitope Akinboyewa, Huan Ning, M Naser Lessani, and Zhenlong Li. 2024. Automated floodwater depth estimation using large multimodal model for rapid flood mapping. Computational Urban Science, 4(1):12.arxiv.org/abs/2402.16684
  3. [4]Gail M. Atkinson and David J. Wald. 2007. “did you feel it?” intensity data: A surprisingly good measure of earthquake ground motion. Seismological Research Letters, 78(3):362–368.DOI
  4. [5]Jan-Micha Bodensohn, Ulf Brackmann, Liane Vogel, Anupam Sanghi, and Carsten Binnig. 2025. Unveiling challenges for llms in enterprise data engineering. arXiv preprint arXiv:2504.10950.DOI
  5. [6]Angelo Cardellicchio, Sergio Ruggieri, Valeria Leggieri, and Giuseppina Uva. 2023. A machine learning framework to estimate a simple seismic vulnerability index from a photograph: the vulma project. Procedia Structural Integrity, 44:1956–1963. XIX ANIDIS Conference, Seismic Engineering in Italy.DOI
  6. [8]Ivan Frigerio, Stefania Ventura, Daniele Strigaro, Matteo Mattavelli, Mattia De Amicis, Silvia Mugnano, and Mario Boffi. 2016. A gis-based approach to identify the spatial variability of social vulnerability to seismic hazard in italy. Applied geography, 74:12–22.DOI
  7. [9]Yanjun Gao, Skatje Myers, Shan Chen, Dmitriy Diglach, Timothy A Miller, Danielle Bitterman, Matthew Churpek, and Majid Afshar. 2024. When raw data prevails: Are large language model embeddings effective in numerical data representation for medical machine learning applications? arXiv preprint arXiv:2408.11854.arxiv.org/abs/2408.11854
  8. [10]Google. 2025. Google Maps Platform. https://developers.google.com/maps. Accessed: 2025-05-19.
  9. [11]Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948.DOI
  10. [12]Shibo Hao, Yi Gu, Haodi Ma, Joshua Jiahua Hong, Zhen Wang, Daisy Zhe Wang, and Zhiting Hu. 2023. Reasoning with language model is planning with world model. arXiv preprint arXiv:2305.14992.arxiv.org/abs/2305.14992
  11. [13]Haley Hostetter, MZ Naser, Xinyan Huang, and John Gales. 2024. Large language models in fire engineering: An examination of technical questions against domain knowledge. arXiv preprint arXiv:2403.04795.arxiv.org/abs/2403.04795

Paper details

vlnc:W_r5trnaba
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents