TimelineAgent: An LLM-based Framework for Query-Centric Event Timeline Extraction Across Multiple Documents

Introduction

Events are fundamental units representing occurrences or changes within a given context and a central target of information extraction [16]. While individual events convey isolated facts, organizing them chronologically reveals their evolution and interactions over time (Pustejovsky et al., 2003). An event timeline is a structured representation that orders events temporally to provide a coherent view of their progression [10]. Across domains, professionals rely on event timelines to access reliable information, interpret complex narratives, and make informed decisions. Such timelines are widely used in fields such as news analysis, crisis monitoring, and fact checking [5] (Wang et al. 2015; Rajaby Faghihi et al. 2022). For example, crisis response teams use event timelines to monitor the evolution of emergencies and coordinate timely responses.

In many real-world applications, users require query-centric timelines aggregated from multiple documents to satisfy diverse information needs. As illustrated in Figure 1, within news articles published between 2004 and 2023, users may focus on events involving specific entities, such as Steve Jobs, to track his evolving role in the company. Constructing such a timeline requires a comprehensive event-centric information extraction pipeline. Starting with event detection (Yue et al., 2023) to identify candidate events, relevance matching selects query-relevant events by first extracting event arguments (Yang et al., 2023), and then leveraging entity coreference (Khosla et al., 2021) and bridging anaphora (Hou et al., 2014) to capture both direct and indirect associations between query entities and event arguments.

In addition, timeline events must satisfy general eligibility constraints, such as factuality (i.e., events not actually occurring should not be included in the timeline) (Saurí and Pustejovsky, 2009) and temporality (i.e., events must possess a temporal attribute) (Ning et al., 2018). Constructing coherent cross-document timelines also requires event temporal anchoring (Wang and Zhao, 2024) and cross-document event coreference resolution (Yu et al., 2022). Temporal anchoring demands both temporal understanding (Wang et al., 2022) and reasoning capabilities [6], while event coreference requires compatibility in both event semantics and arguments (Huang et al., 2019).

In particular, the relevance matching requirements vary across query focuses. As shown in Figure 1, aspect-focused queries, such as Human impacts of the earthquake, require identifying fine-grained event semantics (Liu et al., 2021) related to casualties or fatalities, as well as reasoning over preceding events that led to these impacts [15]. Seed-event-focused queries, such as The Labour Party pledged to reduce street crime, involve tracking downstream events triggered by the seed, such as policy changes or subsequent actions. These events must be semantically and causally related to the seed and constrained by event arguments, including participants or affected targets [5].

Building on the timeline construction processes for different queries described above, we formulate the task of Query-centric Cross-document Event Timeline Extraction (QUEST). To address the complexity of timeline construction and query diversity, QUEST requires modeling heterogeneous event features, including syntactic, semantic, argument, temporal, causal, and factuality. For example, semantic features include predefined event types for aspect-focused queries and contextual semantic similarity for seed-event-focused queries. Argument features for entity-focused queries rely on PropBank-based semantic frames, while those for seed-event-focused queries require both FrameNet-based and predefined semantic schemas.

We propose TimelineAgent, an LLM-based multi-agent framework that separates task-agnostic event modeling from the query- and stage-specific reasoning required for timeline construction. Specifically, we establish a multi-stage workflow (as shown in Table 1), where each stage is handled by a dedicated agent performing reasoning toward a clear sub-objective. For example, relevance matching combines semantic and argument features for query-aware event matching, while temporal anchoring first extracts temporal signals and then performs temporal inference to derive normalized timestamps. Heterogeneous event features are systematically categorized and decoupled

Workflow Stage	Skill Categories (No. of Skills)
Event Detection	Syntactic(Parsing)3+Resources1
Relevance Matching	Semantic(2)2,3, Argument(4)1,3, Causal(1)2,3
Temporal Anchoring	Temporal(Extract)4+Inference3
Eligibility Filtering	Factuality(4)1,2,3, Semantic(1)3, Temporal(1)3
Event Coreference	Semantic(1), Argument(1)

into a skill library of reusable atomic operations, reducing interference among different reasoning processes. For example, temporal signals are decomposed into atomic skills such as tense analysis, aspect analysis, and temporal expression extraction, while argument modeling supports multiple schema configurations including predefined schemas, PropBank, and FrameNet. To coordinate the agent’s invocation of skills, we design a goal-oriented orchestration protocol that composes and configures appropriate skills according to specific reasoning substeps and query requirements. For example, the protocol may combine multiple temporal skills to support temporal signal collection, or configure the argument extraction skill with different schemas under different query settings. Table 1 provides an overview of representative skills required at each stage.

To facilitate evaluation, we construct a new benchmark consisting of existing datasets and newly extended annotated data. For entity-focused queries, we use the SemEval-2015 Task 4 dataset (Minard et al., 2015). For aspect- and seed-event-focused queries, we adapt existing datasets—EventStoryLine [2] and PledgeTracker [5]—through manual annotation and reformulation to align with QUEST. Experimental results show that TimelineAgent consistently outperforms both SFT and ICL LLM baselines, as well as the task-agnostic multi-agent framework TDAG (Wang et al., 2025). Further analysis demonstrates that each component of the framework contributes substantially to overall performance.

To the best of our knowledge, we are the first to investigate cross-document event timeline extraction through a unified LLM-based agent framework across multiple query formulations.²

Related Work

Event Timeline Extraction Document-level tasks primarily focus on modeling temporal relations among events [4, 1] (Ning et al. 2018). In contrast, constructing timelines across documents often requires query-specific information aggregation. Prior work has proposed building timelines centered on entities (Ji et al., 2009; Minard et al., 2015; Laparra et al., 2017; Yao et al., 2024). Fang and Glass [12] and Jiao et al. (2023) use loosely defined criteria, such as event similarity or event salience. Methodologically, early work relied on hand-crafted modules to build pipelines, such as NER and temporal clustering, which lack robust semantic understanding (Moulahi et al., 2015; Laparra et al., 2015a; Navarro-Colorado and Boró, 2016; Piskorski et al., 2020). More recently, end-to-end supervised fine-tuning approaches have been applied in certain domains, such as cancer treatment (Zhang et al., 2025; Jain et al., 2025).

LLM-based Agent Frameworks for Complex IE Recent advances in LLMs have promoted the development of agent-based frameworks for complex IE tasks through task decomposition, rich feature modeling and reasoning. Gangi Reddy et al. [13] proposed a framework consisting of a navigator, an extractor, and an aggregator for information extraction across complex web environments. Jiao et al. (2024) introduced frameworks with components such as observers, planners, and analyzers to support extraction over complex database schemas. Li et al. (2025) proposed a framework consisting of an executor, a tool pool, and a validator for information extraction and integration from documents with complex layouts and cross-references.

Our work differs from existing studies in two main aspects. First, prior work on event timeline extraction typically focuses on a single query type, whereas QUEST requires handling diverse query focuses for more realistic applications. Second, existing methods mainly rely on either hand-crafted pipelines or supervised end-to-end modeling. In contrast, our framework leverages LLM-based agents without task-specific fine-tuning, enabling richer event modeling and better generalization across query settings. More broadly, following recent efforts on designing task-specific architectures for complex information extraction, we develop a multi-agent framework tailored to complex cross-document event reasoning in QUEST.

Task Definition

Given a document collection $D = \{D_1, D_2, \ldots, D_n\}$ and a target query $q$, the task of QUEST is to construct a temporally ordered timeline consisting of query-relevant events:

where each tuple $(\tau_i, E_i)$ denotes a query-relevant timeline event entry. $\tau_i$ denotes the normalized event date (YYYY-MM-DD), following the TimeML specification (Pustejovsky et al., 2010), where unknown temporal components are represented using “X”. $E_i$ denotes a set of coreferential event mentions consolidated across documents.

Method

As illustrated in Figure 2, TimelineAgent decomposes QUEST into multiple sequential stages, where each agent performs stage-specific reasoning. During reasoning, LLM agents extract event features by invoking skills through the goal-oriented orchestration protocol. The protocol combines different reasoning sub-objectives and query settings into predefined goals, each associated with specific skill compositions and configurations, enabling flexible adaptation to diverse scenarios.

Multi-stage Workflow

We design a unified workflow in which agents progressively leverage different event features and reasoning strategies to identify, filter, temporally anchor, and aggregate events. This staged design decouples complex reasoning processes while progressively refining intermediate event representations throughout timeline construction.

Sentence Filtering In this stage, the agent identifies potentially relevant sentences from the input document collection: $\mathcal{S} = \{s_1, s_2, \ldots, s_m\}$, $s_i \in D$. The agent primarily relies on coarse-grained query-aware screening to preserve high-recall evidence for downstream timeline construction while reducing irrelevant information. For example, under entity-focused queries, the filtering process mainly retains sentences mentioning the target entity or its coreferential expressions, without requiring explicit event-level reasoning.

Event Candidate Detection Given the filtered sentence set $\mathcal{S}$, the agent identifies candidate event mentions $\mathcal{E} = \{e_1, e_2, \ldots, e_n\}$, which serve as a shared event space across different query focuses. The agent primarily analyzes syntactic structures, including POS tags and constituency structures, and then applies predefined span-level constraints together with lexical knowledge (e.g., Collins English Dictionary) to maintain event boundary consistency. Finally, dependency parsing is applied to filter out light verbs and copula verbs.

Query-relevant Event Matching The agent leverages different forms of correspondence to distinguish query-relevant events from the candidate event set, resulting in $\mathcal{E}_q \subseteq\mathcal{E}$.

For entity-focused queries, the agent first performs open-domain argument extraction based on semantic frame knowledge (e.g., PropBank (Palmer et al., 2005)), and then models associations between query entities and event participants through entity coreference and bridging anaphora.

For aspect-focused queries, the agent first classifies events based on predefined aspect-oriented semantic types, and then identifies causally preceding events based on external causal knowledge (e.g., ConceptNet (Speer et al., 2017)). Incorporating causal dependencies enables the timeline to provide more informative narratives, as illustrated by the aspect-focused example in Figure 1, which links the initial earthquake to downstream human-impact events such as deaths and injuries.

For seed-event-focused queries, the agent first determines event semantic types and then identifies related events through commonsense reasoning and external causal knowledge (e.g., ConceptNet). It further extracts arguments using predefined argument schemas and external frame knowledge (e.g., FrameNet), and evaluates argument consistency based on semantic roles.

Event Temporal Anchoring The agent grounds query-relevant events into a temporally anchored event set $\mathcal{E}_t = \{(e_i, \tau_i)\}$, where $\tau_i$ denotes the normalized timestamp of event $e_i$. The agent first collects diverse temporal signals, including tense, aspect, temporal expressions, and ordering cues, and then performs temporal inference to determine event timestamps through date calculation and temporal normalization. Document creation time (DCT) serves as a critical temporal reference point for integrating fragmented information across multiple documents (Wang et al., 2022). During inference, reasoning over the temporal relation between events and DCT plays an important role, with different decision strategies required under different query types.

Event Eligibility Filtering This stage determines whether a query-relevant event should be included in the final timeline. It takes the temporally anchored event set $\mathcal{E}_t$ as input and produces the eligible event set $\mathcal{E}_{\mathrm{eligible}} \subseteq\mathcal{E}_t$.³ Event eligibility is separated from query relevance because it depends not on specific query content, but on query types and general timeline requirements. These include factuality-related dimensions, such as polarity, certainty, and factual status, as well as temporality constraints, which are typically reflected through semantic features. In addition, some query types impose explicit temporal range constraints on eligible events.

Cross-document Event Coreference Resolution Given $\mathcal{E}_{\mathrm{eligible}}$, the agent groups cross-document event mentions into clusters $\mathcal{C}$, where events in the same cluster refer to the same real-world event. The agent first determines semantic type compatibility between events, and then further evaluates argument consistency (Barhom et al., 2019). Basic event arguments are extracted through semantic frame knowledge, such as PropBank and FrameNet, while previously inferred timestamps are also incorporated for temporal compatibility assessment. An illustrative example of the workflow is provided in Appendix § 7.§ 1.

Feature-oriented Skill Library

We design an LLM-based feature-oriented skill library to support more comprehensive and robust event modeling across diverse reasoning stages and query settings. We decompose heterogeneous event features into reusable atomic skills organized by event attributes, while allowing flexible configurations such as external knowledge sources.

Syntactic In addition to basic syntactic analysis capabilities, such as part-of-speech tagging, constituency parsing, and dependency parsing (Yan et al., 2019; Lai et al., 2020; Wan et al., 2024), we further design a skill based on lexical knowledge (e.g., Collins English Dictionary) to determine the extent of phrase-level constructions.

Semantic Semantic skills mainly support two settings: predefined event semantic type classification [11] (Wang et al. 2020) and open-domain semantic similarity modeling between events [14].

Argument Argument skills mainly support argument extraction under different schemas, including predefined schemas (Ren et al., 2023; Shuang et al., 2024), FrameNet-based (Parekh et al., 2023), and PropBank-based schemas (Zhang et al., 2022). They also provide alignment mechanisms between arguments or entities, including entity coreference resolution (Khosla et al., 2021), and bridging anaphora [17]. The first two are formulated as binary matching tasks, while bridging anaphora is modeled as a multi-class relation classification task covering indirect relations, such as part–whole, set–member, and functional or attributive relations.

Factuality Factuality skills include event polarity, certainty, factuality, and conditionality, all formulated as binary classification tasks (Saurí and Pustejovsky, 2009, 2012; Lee et al., 2015; Rovera et al., 2025). Among them, polarity is relatively objective, while the latter categories are more subjective. These skills can be flexibly combined to support more complex factuality judgments that vary across query settings. For example, in entity-focused queries, future events modified by will are treated as factual events for timeline construction, whereas events expressed in conditional constructions are marked as conditional and excluded from the timeline.

Temporal Temporal skills are divided into signal-oriented and reasoning-oriented subcategories. Signal-oriented skills include temporal expressions, tense, aspect, and ordering cues [9] (Pustejovsky et al. 2010). Reasoning-oriented skills include temporal normalization, date calculation, and temporal relation classification (Wang et al., 2022; Xiong et al., 2024).

Causal Causal category consists of causal relation classification based on causal knowledge from ConceptNet (Speer et al., 2017; Su et al., 2025).

Goal-oriented Orchestration Protocol

Instead of exposing the entire skill library to agents, which may introduce skill-level attention distraction and context pollution, we design an explicit orchestration protocol for controlled skill routing. The protocol is formalized as a structured coordination mechanism that maps reasoning goals to predefined skill compositions and configurations.

Formally, a reasoning goal is defined as: $g=(r,q)$ where $r$ denotes a reasoning substep and $q$ denotes the query type. A protocol entry defines a mapping: $P_i:g\to C_i$, where $C_i$ denotes the orchestration configuration for the target reasoning goal. Each configuration is represented as: $C_i=(O,\Phi)$, where $O$ specifies the skill composition, such as combining tense, aspect, and temporal expressions for temporal reasoning, and $\Phi$ denotes skill-level configurations, such as selecting PropBank or FrameNet as the argument schema.

Detailed implementation of TimelineAgent is provided in Appendix § 7.§ 2.

Experiments

We construct a benchmark covering three query settings for evaluating QUEST: entity, aspect, and seed-event queries. We compare against ICL and SFT baselines and a standard multi-agent framework, and further analyze the effects of each component and of staged reasoning.

Experimental Setup

Benchmark Construction Existing public datasets for query-centric event timeline extraction are very limited.⁴ Due to the high annotation complexity of timeline construction, we construct our benchmark based on existing event datasets.

For entity-focused queries, we use SemEval-2015 Task 4 (Minard et al., 2015) dataset, which contains manually annotated entity-based event timelines from news articles. Entities include persons, organizations, products and financial entities.

For aspect-focused queries, we construct a seven-category crisis-oriented aspect schema based on humanitarian information categories from social media scenarios ([3] (Imran et al. 2016; Vieweg et al. 2010)). We extend the EventStoryLine Corpus ([2]), which models the development of news stories at the document level. We select crisis-related topics, including earthquakes, prisoner escapes, bank blast fires, and cable outages, manually classify events into aspect types, and transform existing temporal and causal relations into document-level timelines. Finally, we incorporate cross-document event coreference annotations from ECB+ ([7]) to construct cross-document timelines. The event classification process was conducted by two linguistics students, achieving a Cohen’s kappa coefficient of 0.85. Details of the aspect types and the document-level timeline transformation process are provided in Appendix § 9.§ 1.

For seed-event-focused queries, we further extend PledgeTracker ([5]), which annotates political pledges and their associated timelines for pledge fulfillment tracking. While their task setting is closely related to ours, timeline events are represented as sentence-level summarized descriptions. We instead use political pledges as seed events and further annotate action-level event mentions grounded in the original source documents based on their event descriptions. The action-level event annotation is based on SemEval-2015 Task 4 annotation guidelines.⁵ Two linguistics students

participated in the annotation, achieving 81% span-level F1 agreement.

Dataset statistics are summarized in Table 2. Illustrative annotation examples for the seed-event- and aspect-focused query settings are provided in Appendix § 9.§ 2.

	Entity	Aspect†	Seed-event†
# documents	120	46	1475
# events	1102	647	406
# timelines	42	28	26
events / timeline	26.3	23.1	15.6
event chains / timeline	19.9	13.9	11.1
docs / timeline	6.9	8.6	6.0

Evaluation Metric We follow the official SemEval-2015 TimeLine evaluation protocol (Minard et al., 2015). Rather than scoring isolated temporal decisions, the protocol transforms each predicted and gold timeline into a temporal graph whose nodes are event mentions and normalized time anchors and whose edges encode event–time anchoring and event–event relations. Graph-based precision, recall, and F1 jointly evaluate event selection, temporal grounding, and cross-document event coreference decisions. Further details of the metric are provided in Appendix § 9.§ 3.

Baseline Methods We compare TimelineAgent with a traditional hand-crafted pipeline system (DLT (Laparra et al., 2015b)), end-to-end LLM settings using in-context learning (ICL) and supervised fine-tuning (SFT), and the task-agnostic multi-agent framework TDAG (Wang et al., 2025). TDAG is a strong task-agnostic multi-agent framework that has demonstrated superior performance over existing agent paradigms such as ReAct (Yao et al., 2022) and Plan-and-Solve (Wang et al., 2023) on complex reasoning tasks. Detailed baseline implementations are provided in Appendix § 7.§ 3.

Implementation Details We employ two backbone LLMs in our experiments: DeepSeek-V3 ([8]) and Qwen3-32B (Yang et al., 2025). DeepSeek-V3 is a 671B-parameter language model accessed through its API for inference. Qwen3-32B is deployed on 4 NVIDIA A100 GPUs with 80GB memory for both fine-tuning and inference. To reduce randomness and ensure stable performance, we set the decoding temperature to 0. For SFT, we use the original trial set (6 timeline instances) for entity-focused queries. To maintain a comparable training scale, we construct five data splits for aspect-focused and seed-event-focused queries, with 6 and 5 timeline instances used for training in each split, respectively, while the remaining instances are used for test. Detailed configurations are provided in Appendix § 10.§ 1.

Main Results

The overall performance comparison is shown in Table 3. TimelineAgent consistently outperforms all baselines across different query settings and backbone model scales, demonstrating the effectiveness of our framework. In particular, our Qwen3-32B model surpasses the general-purpose agent framework TDAG built on DeepSeek-V3, indicating the importance of task-specific reasoning organization and event modeling for QUEST. However, overall performance remains around 30–40% F1, highlighting the inherent difficulty of QUEST.

Method	Entity			Aspect			Seed-event
	P	R	F1	P	R	F1	P	R	F1
DLT	20.7	11.0	14.3	-	-	-	-	-	-
Qwen3-32B
ICL	17.8	12.1	14.4	28.1	23.8	25.8	8.3	12.9	10.1
TDAG	15.3	20.8	17.6	33.5	27.5	30.2	11.3	19.6	14.3
SFT	24.0	27.9	25.8	40.2	31.8	35.5	17.6	25.8	20.9
TimelineAgent	29.2	35.7	32.1	41.3	36.8	38.9	22.9	33.8	27.3
DeepSeek-V3
ICL	21.2	15.0	17.5	33.7	24.2	28.2	9.8	17.4	12.5
TDAG	22.2	44.3	29.6	40.3	33.2	36.4	19.5	30.2	23.7
TimelineAgent	37.2	38.6	37.9	45.8	40.3	42.9	27.8	40.9	33.1

Among different query settings, seed-event-focused queries are the most challenging, followed by entity-focused queries, while aspect-focused queries are relatively easier. This trend is roughly proportional to the sparsity of relevant events in documents, with an average of 0.3, 9.2, and 14.1 relevant events per document, respectively, highlighting the inherent challenges of cross-document event reasoning in QUEST.

Compared with ICL, SFT substantially improves performance, indicating the benefit of task-specific adaptation. However, its effectiveness is still limited by the low-resource training data. TDAG, as a general-purpose multi-agent framework, further demonstrates the effectiveness of task decomposition and coordinated agent reasoning. Nevertheless, both approaches still show a considerable gap from TimelineAgent, highlighting the effectiveness of our task-specific understanding and structured modeling of the complex requirements in QUEST.

Larger backbone models consistently outperform

smaller ones across all settings, suggesting the importance of strong general reasoning capabilities.

Ablation Study

We first conduct architectural ablations by removing different framework components to analyze their contributions. We further perform stage-wise ablations by replacing specialized reasoning with simple single-step prompting. In this setting, each agent is not equipped with explicit skills or multi-step reasoning processes. Finally, we conduct error propagation analysis by replacing stage outputs with gold annotations to investigate the downstream impact of errors.

Architectural Ablation. As shown in Figure 3, removing both skills and orchestration protocols (w/o Skills+Protocol) leads to substantial performance degradation across all query settings, demonstrating that merely providing high-level reasoning objectives without specialized event feature modeling is insufficient for event-centric reasoning. When only orchestration protocol is removed (w/o Protocol), the model can still utilize useful skills to some extent, but performance remains consistently inferior to the full model, indicating that directly exposing all skills as flat context is less effective than our structured skill routing strategy. We observe that preserving both skills and protocols while executing the entire workflow in a single unified stage (Single-stage Execution) results in particularly severe performance degradation. This finding highlights the importance of staged execution for handling the overall reasoning complexity.

Stage-wise Ablation. As shown in Figure 4, event detection consistently contributes to all query settings, demonstrating the effectiveness of our fine-grained syntactic modeling. Relevance matching yields the largest overall contribution across all settings, highlighting the importance of our structured reasoning and decomposed feature modeling. Temporal anchoring is particularly important for entity-focused and seed-event-focused queries, reflecting the challenges of reasoning over long temporal spans and temporally evolving events. Eligibility filtering also provides consistent improvements, indicating that modeling factuality and semantic temporality is beneficial for identifying temporally eligible events. Event coreference contributes most substantially to aspect-focused queries, demonstrating the effectiveness of our modeling for distinguishing highly similar events occurring within relatively compact temporal contexts.

Error Propagation Analysis. Since the datasets do not separately provide annotations for general events, event relevance, and event eligibility, we combine them into a unified stage referred to as event detection, matching, and filtering. As shown in Table 4, event detection, matching, and filtering constitute the most critical stage across all query settings, indicating that accurately identifying and selecting relevant events is fundamental for timeline construction. Temporal anchoring is particularly important for entity-focused and seed-event-focused queries, which typically involve broader temporal spans and require more complex temporal reasoning. In contrast, aspect-focused queries are more sensitive to event coreference, since they are centered around different aspects of a relatively compact seminal event with many repeated event mentions across documents. We provide additional qualitative error analysis in Appendix § 10.§ 2.

Oracle Setting	Entity	Aspect	Seed-event
TimelineAgent	37.9	42.9	33.1
Gold Detection, Matching, & Filtering	67.6	69.8	71.4
Gold Temporal Anchoring	55.6	52.4	41.0
Gold Event Coreference	42.1	56.1	39.4

We provide case studies of key stages in Appendix § 10.§ 3.

Impact of LLM Backbones

We further compare different backbone model scales to analyze the relationship between model capacity and performance, as shown in Figure 5. Overall, larger models consistently achieve better results. However, the performance gains gradually diminish when scaling from Qwen3-32B to DeepSeek-V3 (671B), suggesting that model scale alone is no longer the dominant factor for QUEST, while task-specific reasoning and modeling become increasingly important. Compared with SFT, our framework achieves slightly better performance on smaller models such as 8B and 14B, but demonstrates larger gains as model size further increases to 32B. This trend suggests that stronger LLM reasoning capabilities are more effective for handling decomposed reasoning subproblems in our framework, while directly fine-tuning models to perform complex end-to-end reasoning remains difficult under extremely low-resource conditions with only 5–6 training instances. Moreover, unlike SFT, our framework does not require training separate task-specific models for different query settings, enabling stronger generalization across diverse timeline construction scenarios.

Impact of SFT Training Scale

To assess whether the low-resource setting of the SFT baseline accounts for its performance gap, we increase the amount of supervision at medium and large scales. Table 5 shows that increasing supervision consistently improves SFT performance across all three query settings, although the gains diminish as the training set grows. At the large scale, SFT remains below Qwen3-32B-based TimelineAgent on Entity and Seed-event queries, but exceeds it by 0.6 F1 on Aspect queries. TimelineAgent with DeepSeek-V3 outperforms large-scale SFT in every setting. Thus, while end-to-end SFT bene-

caption: F1 scores at different supervised training scales for end-to-end SFT. The standard, medium, and large SFT settings use 6/6/5, 18/12/10, and 30/18/15 training timelines for Entity, Aspect, and Seed-event queries, respectively.

:::

fits from additional supervision, TimelineAgent remains competitive in limited-supervision settings and can benefit from larger foundation models without task-specific fine-tuning.

Computational Efficiency

To improve computational efficiency, we first perform sentence filtering to remove irrelevant context before event reasoning. This step filters out approximately 50%, 30%, and 90% of sentences for three query settings, respectively. To further reduce inference overhead, we jointly process all events within the same sentence at each step rather than reasoning over them individually. Compared with standard multi-stage ICL approaches that typically execute a single reasoning step per stage, our framework requires only $\frac{2+2+2+3}{3}+\frac{2+2+2}{5} \approx2.1\times$ more reasoning executions on average. Compared with the automatic multi-agent decomposition framework TDAG, which incurs approximately $4\times$ more executions, our framework remains more efficient. In practice, a full inference run takes around ten hours depending on the batching configuration, demonstrating the practical feasibility of our framework.

Conclusion

We introduced the practical yet challenging task of QUEST, which requires handling complex processing pipelines and diverse event features under different query settings. We propose TimelineAgent, a multi-agent framework that decomposes timeline construction into coordinated reasoning stages supported by reusable atomic skills and goal-oriented orchestration protocols. Experimental results across multiple query settings demonstrate the effectiveness of our framework over strong baselines. We hope our framework can provide a practical direction for applying structured agent systems to other complex information extraction tasks.

Limitations

Our work mainly focuses on establishing a principled framework for the complex QUEST task. Based on our atomic feature-oriented skill design and goal-oriented orchestration principles, future work could explore finer-grained orchestration strategies, such as independently invoking atomic skills for individual event mentions to reduce the complexity of each reasoning step and enable more precise reasoning and feature modeling. To address the substantially higher computational overhead introduced by such designs, future research may investigate hybrid architectures that combine large and small language models, where stronger models handle more demanding reasoning processes while lightweight models perform relatively simpler feature extraction tasks such as syntactic analysis and semantic categorization. Such designs may provide stronger reasoning capabilities while maintaining better computational efficiency.

Constructing high-quality structured event timelines is an annotation-intensive process. To scale the benchmark while maintaining annotation quality, human-agent collaboration offers a practical direction. Under different query requirements, human annotators can edit the reasoning logic for timeline construction and customize the corresponding skills and orchestration protocols, enabling more flexible extension of the benchmark. Meanwhile, the agent’s interpretable reasoning process allows annotators to provide supervision and corrections at intermediate steps, helping ensure annotation quality while scaling the benchmark.

Acknowledgments

We would like to thank anonymous reviewers for their insightful comments to help improve the paper. This work has been supported by the National Natural Science Foundation of China (NSFC) Key Project under Grant Number 62336006. Andreas Vlachos and Yulong Chen were supported by the DARPA program SciFy.

Method Details

Example of Multi-stage Timeline Construction

Consider an entity-focused query centered on Steve Jobs. The workflow first performs query-relevant sentence filtering to identify potentially useful context from the document collection. Sentences mentioning the target entity or its coreferential expressions are retained as high-recall evidence for downstream reasoning.

The agent then performs query-relevant event matching to identify events associated with the query entity. Candidate events such as announced, fought, resigned, and unveiled are extracted from the filtered sentences. Using semantic frame knowledge and entity coreference reasoning, the agent determines whether event participants are associated with the target entity. For example, the event unveiled is retained because the pronoun “he” in its argument structure is resolved to “Steve Jobs.”

Next, the agent performs event temporal anchoring. Temporal signals including tense, aspect, temporal expressions, and temporal ordering cues are first extracted from context. For example, given the sentence “Jobs announced the product on Monday” with document creation time 2011-01-17, the agent extracts the temporal expression “Monday” and normalizes the event timestamp to 2011-01-17. Similarly, the sentence “Jobs unveiled the device in 2004” leads to the normalized timestamp 2004-XX-XX through temporal expression normalization.

After temporal grounding, the agent performs cross-document event coreference resolution. Event mentions are first compared based on semantic type compatibility and semantic similarity. The agent then evaluates argument consistency and temporal compatibility using extracted participants and normalized timestamps. For example, two event mentions containing the trigger announced may initially appear semantically compatible. However, if one event corresponds to “Steve Jobs announced Lion OS X” on 2010-10-23, while another corresponds to “Steve Jobs announced medical leave” on 2011-01-17, the agent determines that the arguments and timestamps are inconsistent and therefore assigns them to different event clusters.

Finally, the clustered and temporally grounded events are aggregated into a chronologically ordered timeline. Through this staged workflow, TimelineAgent progressively transforms noisy cross-document evidence into structured event timelines with explicit temporal organization.

Implementation Details of TimelineAgent

Multi-stage Workflow

Sentence Filtering Stage

Objective: Perform coarse-grained query-aware evidence screening to preserve high-recall candidate evidence for downstream timeline construction.

Input:

Query: $\{\text{query}\}$

Document: $\{\text{document}\}$

Query-specific Focus:

• Entity-focused: entity mentions and aliases

• Aspect-focused: semantic topical consistency

• Seed-event-focused: triggered by the seed event

Output: Filtered sentence set $S^{\prime} \subseteq S$.

Event Candidate Detection Stage

Objective: Identify structurally valid event candidates from filtered sentences while preserving high event recall for downstream reasoning.

Input: Sentence set $S^{\prime} = \{s_1, s_2, \ldots, s_m\}$ from the sentence filtering stage.

Step 1: Syntactic Event Analysis

Input Placeholder:

Sentence: $\{\text{sentence}\}$

The first prompt performs coarse-grained syntactic analysis to identify potential event candidates based on part-of-speech categories, constituency structures, and dependency relations.

• Identify candidate verbal, nominal, and predicative event mentions

• Analyze phrase structures and dependency heads

• Detect possible multi-token event constructions

Activated Skills:

• POS tagging

• Constituency parsing

• Dependency parsing

Intermediate Output: Candidate event set $\mathcal{E}' = \{e_1,e_2,\ldots,e_k\}$ with syntactic spans and structural attributes.

Step 2: Rule-guided Event Filtering

Input Placeholder:

Sentence: {sentence}

Candidate Events: {candidate_events}

Syntactic Analysis: {syntactic_outputs}

The second prompt applies rule-based filtering and lexical constraints to refine event candidates according to annotation-oriented syntactic criteria.

• Remove non-eventive syntactic heads

• Filter auxiliary verbs, copular verbs, and light verb constructions

• Verify lexicalized multi-word expressions using Collins English Dictionary

• Resolve uncertain phrase-level event boundaries

Activated Skills:

• Lexical phrase validation

Output:

Structured event candidate set: $\mathcal{E} = \{e_1,e_2,\ldots,e_n\}$ with event spans and syntactic attributes.

Query-relevant Event Matching Stage

Objective: Identify query-relevant events from the candidate event set $\mathcal{E}$ through semantic, argument, and causal correspondence reasoning.

Input: Candidate event set $\mathcal{E} = \{e_1,e_2,\ldots,e_n\}$.

Input Placeholder:

Query: {query}

Query Type: {query_type}

Sentence: {sentence}

Candidate Events: {event_triggers}

Context: {context}

Optional Knowledge Source: {knowledge_source}

General Reasoning Procedure:

• Event semantic correspondence reasoning

• Argument and participant consistency reasoning

• Commonsense and causal dependency reasoning

Output: Query-relevant event set $\mathcal{E}_q \subseteq\mathcal{E}$.

Entity-focused Query

Goal: Identify events semantically or argumentatively associated with the target entity.

Reasoning Procedure:

• Extract event arguments based on open-domain semantic frames

• Identify associations between query entities and event participants. Resolve entity coreference and bridging relations

Skills:

• Argument extraction (PropBank)

• Entity coreference resolution

• Bridging anaphora reasoning

Aspect-focused Query

Goal: Identify events relevant to predefined crisis-oriented aspects.

Reasoning Procedure:

• Classify events into predefined aspect semantic categories

• Identify causally related preceding events. Construct informative aspect-centered narratives

Skills:

• Aspect semantic type classification

• Causal relation reasoning. External causal knowledge (ConceptNet)

Seed-event-focused Orchestration

Goal: Identify events semantically and causally related to the seed event.

Reasoning Procedure:

• Determine event semantic types. Identify commonsense and causal event relations

• Extract arguments

• Evaluate semantic role consistency

Skills:

• Event semantic typing

• Commonsense and causal reasoning

• Argument extraction (FrameNet)

Event Temporal Anchoring Stage

Objective: Ground query-relevant events into normalized timestamps to construct the temporally anchored event set $E_t = \{(e_i, \tau_i)\}$.

Input: Query-relevant event set $E_q$.

Input Placeholder:

Query:

{query}

Sentence:

{sentence}

Events:

{event_triggers}

Temporal Context:

{context}

Step 1: Temporal Signal Extraction

• Extract tense and aspect information

• Identify temporal expressions and DCT

• Identify temporal ordering cues

Skills:

• Tense analysis

• Aspect analysis

• Temporal expression extraction

• Temporal ordering cue extraction

Step 2: Temporal Inference

• Resolve relative temporal expressions

• Perform date calculation and temporal normalization

• Derive normalized timestamps under document-level temporal contexts

Skills:

• Temporal relation classification

• Date calculation

• Temporal normalization

Output: Temporally anchored event set $E_t = \{(e_i, \tau_i)\}$.

Cross-document Event Coreference Resolution Stage

Objective: Group eligible cross-document event mentions into event clusters $C$, where events within the same cluster refer to the same real-world event.

Input: Eligible event set $E_{\mathrm{eligible}}$.

Input Placeholder:

Sentence: {sentence}

Event: {event_trigger}

Event Type: {event_type}

Arguments: {arguments}

Timestamp: {timestamp}

Context: {context}

Step 1: Semantic Compatibility Modeling

• Determine semantic type compatibility between events

• Evaluate semantic similarity between event mentions

• Filter semantically incompatible event pairs

Skills:

• Semantic similarity reasoning

Step 2: Argument and Temporal Consistency Reasoning

• Extract event arguments through semantic frame knowledge

• Evaluate participant and semantic role consistency

• Incorporate temporal compatibility based on inferred timestamps

Skills:

• Argument extraction (PropBank / FrameNet)

• Semantic role consistency modeling

• Temporal compatibility reasoning

Output:

Cross-document event clusters $C$.

Feature-oriented Atomic Skills

Table 6 shows skill formulations and configurations. All atomic skills are implemented as constrained prompting modules with structured inputs, configurations, and standardized outputs. To improve controllability and reduce reasoning interference, each skill operates under restricted objectives and predefined output schemas.

Category	Atomic Skills	Configurations
Syntactic	Part-of-speech tagging, constituency parsing, dependency parsing, phrase span detection	Collins English Dictionary for phrase span detection
Semantic	Event type classification, semantic similarity modeling	predefined semantic categories for classification
Argument	Argument extraction, entity coreference resolution, bridging anaphora	PropBank schemas, FrameNet schemas, predefined schemas for argument extraction
Factuality	Polarity classification, certainty classification, factuality classification, conditionality classification	labels for certainty and factuality classification
Temporal	Temporal expression extraction, tense analysis, aspect analysis, ordering cue extraction, temporal normalization, date calculation, temporal relation classification	Temporal relation label sets for classification
Causal	Causal relation classification	ConceptNet causal knowledge

All skills return structured intermediate outputs in unified JSON-style formats, enabling flexible composition across workflow stages. The following table presents a representative prompt template used for atomic skills. The orchestration protocol dynamically injects task objectives, configuration constraints, and external knowledge specifications according to different reasoning goals.

Atomic Skill Prompt Template

System Instruction:

You are an atomic event modeling skill expert.

Task Objective:

{Task_description}

Input Event:

{event_context}

Configuration:

{Skill_specific_configuration}

Constraints:

• Follow the predefined label space.

• Only perform the assigned task.

• Return structured JSON outputs.

Output Format:

{
  "event": "...",
  "prediction": "...",
  "confidence": "..."
}

Different skills instantiate this template with task-specific objectives and configurations. For example, temporal skills inject temporal normalization rules and relation labels, while argument extraction skills configure different semantic role schemas such as PropBank or FrameNet.

Goal-oriented Orchestration Protocol

The orchestration protocol selects and configures atomic skills according to reasoning substeps and query requirements. In practice, each protocol entry specifies two components: (1) the target reasoning objective and (2) the required skill composition. The protocol configurations are shown in Table 7, which illustrates how different workflow stages activate heterogeneous skills under different query settings.

Pipeline Stage	Skills
Event Candidate Detection	Syntactic (POS tagging, constituency parsing, dependency parsing, lexical resource (Collins English Dictionary))
Query-relevant Event Matching	Semantic (predefined event semantic typing2, semantic similarity3) Argument (argument extraction (predefined3, PropBank1, FrameNet3), entity coreference resolution1, bridging anaphora1) Causal (causal relation (ConceptNet2,3))
Event Temporal Anchoring	Temporal (tense, aspect, time expression, ordering cues, date calculation, temporal normalization, temporal relation)
Event Eligibility Filtering	Factuality (polarity, factuality, certainty, condition construction) Temporal (date calculation3) Semantic (predefined semantic typing)
Cross-document Event Coreference Resolution	Semantic (semantic similarity) Argument extraction (PropBank)

Implementation Details for Baselines

Implementation Details for End-to-End SFT and ICL

Query-focused Event Timeline Construction Template

Task Objective

Given a document collection and a query, construct a query-focused event timeline by identifying, temporally anchoring, ordering, and aggregating relevant events across documents.

Input Placeholder

Documents:

{documents}

Query:

{query}

Query Type:

{query_type}

Timeline Output Format

ordering    time_anchor
docid-sentid-event
0            YYYY-MM-DD
docid-sentid-event
1            YYYY-MM-DD
docid-sentid-event
...

Core Requirements

• Detect query-relevant events

• Resolve cross-document event coreference

• Infer normalized timestamps

• Construct temporally ordered timelines

Temporal Constraints

• Use ISO-8601 date format

• Maximum granularity: DAY

• Use XXXX-XX-XX if timestamps are unknown

Output Placeholder

Timeline:
1   YYYY-MM-DD   docid-sენტid-event
2   YYYY-MM-DD   docid-sentid-event
...

Entity-focused Configuration

Query Objective: Construct timelines centered on events involving a target entity.

Key Reasoning Focus:

• Entity mention and alias matching

• Argument extraction and participant consistency

• Entity coreference and bridging reasoning

Representative Query Placeholder

Target Entity: entity

Aspect-focused Configuration

Query Objective: Construct timelines describing different aspects of a crisis or complex event development.

Key Reasoning Focus:

• Aspect semantic type classification

• Causal dependency reasoning

• Narrative-oriented event aggregation

Representative Query Placeholder

Aspect Query: aspect

Seed-event-focused Configuration

Query Objective: Construct timelines around events related to a given seed event.

Key Reasoning Focus:

• Event semantic similarity

• Commonsense and causal relation reasoning

• Argument and temporal consistency modeling

Representative Query Placeholder

Seed Event: seed_event

Implementation Details of TDAG

Autonomous Agent Prompt Template for QUEST

Task Objective

Given a set of documents and a query, construct a query-focused event timeline by detecting relevant events, resolving cross-document event coreference, anchoring events in time, and ordering them chronologically.

Input Placeholder

Documents: {documents}

Query: {query}

Query Type: {query_type}

Timeline Format

ordering    time_anchor
docid-sentid-event
0           YYYY-MM-DD
docid-sentid-event
1           YYYY-MM-DD
docid-sentid-event
...

Agent Reasoning Protocol

You are an autonomous intelligent agent responsible for constructing the timeline.

• Perform step-by-step reasoning.

• Issue only one action at a time.

• Use subtasks when complex reasoning is required.

Each response must follow:

THOUGHT: reasoning process
ACTION: selected action

Available Actions

• <action>subagent_handle (subtask_name)</action>

• <action>over()</action>

• <action>over()</action>

Subtask Format

<subtask>{
  "subtask_name": string,
  "goal": string,
  "criticism": string,
  "milestones": list[string],
  "result_format": optional
}</subtask>

General Timeline Constraints

• Resolve cross-document event coreference conservatively

• Use ISO-8601 DATE format for timestamps

• Use XXXX-XX-XX if timestamps are unknown

• Order events using temporal relations

• Place temporally uncertain events at ordering index 0

Final Output

Output the final timeline within:

<timeline>
...
</timeline>

Then call:

<action>over()</action>

Entity-focused Configuration Query Placeholder

Target Entity: {entity}

Additional Constraints

• Include only events involving the target entity or its coreferential mentions

• Resolve entity-event participant associations

• Merge coreferential event mentions across documents

Aspect-focused Configuration Query Placeholder

Aspect Query: {aspect}

Additional Constraints

• Focus on aspect-relevant crisis events

• Include causally related supporting events

• Construct coherent narrative-oriented timelines

Seed-event-focused Configuration Query

Placeholder

Seed Event: {seed_event}

Additional Constraints

• Identify semantically or causally related events

• Model argument and temporal consistency

• Preserve temporally associated event developments

Benchmark Details

Aspect-focused Dataset Details

Our crisis-oriented aspect schema consists of seven categories: (1) Situation and Hazard, (2) People Impact, (3) Infrastructure and Lifeline, (4) Needs and Help, (5) Response and Recovery, (6) Risk Communication, and (7) Source and Credibility.

Crisis timeline construction. Starting from the EventStoryLine Corpus v1.5 annotations, we construct aspect-specific crisis timelines for each selected topic. We first group documents by topic and extract event mentions together with their sentence and document identifiers. For each crisis aspect, we select aspect-relevant seed events using aspect keyword rules, and then expand the seed set with causal predecessors annotated by PLOTLINK. This gives each aspect timeline not only the target events of interest, but also the causal context leading to them.

We then anchor events to time using annotated TLINK relations whenever the linked time expression can be normalized to a day-level ISO date. Events that refer to the same action are merged using ACT cross-document coreference, while singleton events are kept as independent timeline nodes. After temporal anchoring and ACT-based merging, events are sorted chronologically within each topic-aspect timeline. Finally, we export the resulting timelines into a simple text format following prior timeline datasets: each line contains a timeline index, a normalized time value, and a compact event identifier.

Illustrative Annotation Examples

We present two representative annotation traces, one for each newly constructed query setting. Each trace follows the annotation stages from source evidence to the final cross-document timeline.

Seed-event Query

Input:

Seed event: Labour: We will introduce a Football Governance Bill, which will establish an independent regulator to ensure financial sustainability of football clubs in England.

Source documents: 42 news articles.

Existing annotations: PledgeTracker relevant summarized event descriptions.

Step 1: Identify supporting sentences and event triggers. For each PledgeTracker event description, we additionally annotate its supporting sentence(s) in the source documents and the corresponding event trigger(s). Representative annotations are shown below.

Step 2: Temporal annotation. Annotators normalize the time of each event trigger by combining explicit temporal expressions, document publication dates, and contextual temporal cues such as tense, aspect, and voice.

Step 3: Cross-document event coreference. After extracting events from multiple documents, annotators determine whether different event triggers refer to the same real-world event based on the trigger type and event arguments.

Step 4: Timeline construction. Finally, event mentions are merged based on temporal information and cross-document event relations to construct the final timeline.

Aspect Query

Input:

Aspect schema: (1) Situation and Hazard, (2) People Impact, (3) Infrastructure and Lifeline, (4) Needs and Help, (5) Response and Recovery, (6) Risk Communication, and (7) Source and Credibility.

Topic: The Oeshm Island earthquake.

Source documents: 11 news articles.

Existing annotations: ECB+/ESC event mentions, event–time relations, intra-document causal relations, and cross-document event-coreference relations.

Step 1: Event type annotation. The source corpus provides the event triggers. We assign the existing event mentions to the seven analytical aspects used to construct aspect-specific timelines. In the examples below, italicized spans are source-annotated events, whereas the aspect in parentheses is the derived category in our benchmark.

Step 2: Normalizing event time from temporal links. The source annotation links events to temporal expressions. We normalize the value already attached to the linked temporal expression; we do not infer a date from world knowledge.

Event	Existing temporal link	Time
2-3-aftershock	11/28/05 CONTAINS aftershock	2005-11-28
2-3-shook	11/28/05 CONTAINS shook	2005-11-28
9-1-hits	No usable event–time relation	XXXX-XX-XX (unknown)

Step 3: Document-level timeline construction. For each article, we first select the events belonging to the requested aspect. We then add only predecessors licensed by an existing intra-document causal relation and consolidate triggers belonging to the same event. Representative causal relations are shown below.

Doc.	Predecessor	Aspect event	Causal relation
2	earthquake	aftershocks	FALLING_ACTION
11	earthquake	aftershocks	PRECONDITION
11	earthquake	hit	PRECONDITION
1	quake	struck	PRECONDITION
1	tremor	aftershocks	PRECONDITION

Document-level timeline for Doc. 2.

Order	Time	Event
1	2005-11-27	2-3-earthquake; 2-4-quake
1	2005-11-27	2-17-quake; 2-18-earthquake
1	2005-11-27	2-4-hit; 2-16-hit
2	2005-11-28	2-3-aftershock
. . .	. . .	. . .

Document-level timeline for Doc. 11.

Order	Time	Event
1	2005-11-27	11-1-Earthquake; 11-5-earthquake
2	2005-11-27	11-11-aftershocks
2	2005-11-27	11-16-hit
. . .	. . .	. . .

Step 4: Global timeline construction. We next merge document-level nodes only when the source annotation places their mentions in the same cross-document event-coreference group.

Source coreference group	Coreferential event triggers
ACT17821472137518203	2-3-earthquake; 11-5-earthquake; 1-3-earthquake
ACT17822303993286740	2-4-hit; 11-16-hit; 1-3-hit
ACT17821815048905207	2-2-Aftershocks; 11-11-aftershocks; 1-15-aftershocks

Resulting global timeline.

Order	Time	Event
0	XXXX-XX-XX	9-1-hits
. . .	. . .	. . .
1	2005-11-27	2-3-earthquake; 11-5-earthquake; 1-3-earthquake
1	2005-11-27	2-4-hit; 11-16-hit; 1-3-hit; 1-4-struck
1	2005-11-27	2-2-Aftershocks; 11-11-aftershocks; 1-15-aftershocks
. . .	. . .	. . .
2	2005-11-28	2-3-aftershock
2	2005-11-28	2-3-shook
. . .	. . .	. . .

Step 5: Topic-level quality review. After constructing the candidate aspect timelines, we conduct a topic-level quality review to ensure the reliability of the resulting benchmark. We mainly remove topics whose aspect timelines are too sparse to support meaningful timeline construction, as well as topics where cross-document coreference introduces many conflicts with the temporal and causal relations.

Evaluation Metric

We adopt the official evaluation protocol of the SemEval-2015 TimeLine task5, which is based on the TempEval-3 metric (UzZaman et al., 2013). This metric comprehensively measures system performance by assessing the accuracy of events, temporal expressions, and their interrelations. Final scores are averaged across all timelines in the dataset.

Event correctness. Event correctness is directly reflected through the graph matching process. Since event mentions constitute the nodes of the temporal graph, incorrect or spurious events cannot be aligned with gold event nodes, causing their associated temporal links to reduce precision. Conversely, missing relevant events leads to missing graph structures and decreases recall.

Temporal grounding. The time component of each (time, event) entry determines both the event’s normalized anchor and its relative position in the timeline. An incorrect anchor changes the event–time relation and the event–event relations induced by temporal closure. The graph comparison therefore evaluates whether selected events are grounded and ordered consistently with the gold timeline.

Cross-document event coreference. Cross-document event organization is also reflected in the temporal graph structure. Events originating from different documents are evaluated based on their positions within the timeline. Incorrectly merging, separating, or aligning cross-document event mentions changes the resulting graph structure, leading to lower graph precision and recall.

Experimental Details

SFT Hyperparameter Settings

To alleviate the difficulty of directly learning long-range cross-document reasoning under extremely limited supervision, we first decompose gold timelines into document-level timelines for single-document training, and then further train on the original cross-document timelines. Detailed training configurations are shown in Table 8.

Parameter	Value
Train Type	LoRA
LoRA Rank ($r$)	16
LoRA Alpha	32
Target Modules	all-linear
Learning Rate	$2 \times10^{-4}$
Warmup Ratio	0.05
Drop	0.05
Batch Size	4
Epochs	10
Random Seeds	3 (results averaged)

Qualitative Error Analysis

As shown in Table 4, Detection, Matching, & Filtering plays the most fundamental role across all query types. This stage is also the starting point of the pipeline: if the relevant events are incorrectly detected or matched at the beginning, later temporal anchoring and cross-document coreference may still be correct locally, but the final timeline will remain incorrect because it is built on the wrong event set. We give qualitative error analysis to make this failure mode clearer, organized by query type:

Entity query. Examples illustrate errors in matching target entities to the correct events, particularly when event arguments are related but not strictly coreferential.

• Sentence 1: The 16GB version is priced at US dollar 199 and the 32GB version at US dollar 299.

• Analysis: “The 16GB version” is an argument of the priced event and is related to iPhone 4 through a product-variant relation, but it is not coreferential with the target entity iPhone 4.

• Sentence 2: On September 21, 2004 the parties agreed to have the case heard by the UK court.

• Analysis: “The parties” is an argument of the agreed event and collectively refers to “Apple Inc.” and “Apple Corps”, but it does not individually corefer with either company.

Aspect query. The query requires identifying events matching the requested aspect (human impact) and connecting them to earlier causal or background events in the document.

• Aspect-relevant sentence: Comeaux also injured a man who was visiting another inmate at the time and tried to stop the attack.

• Classification: injured and attack are classified as People Impact events under the seven-aspect schema.

• Earlier causal/background event: He was serving a life sentence after being convicted in June 1998 of aggravated sexual assault out of Brazos County, located northwest of Houston.

• Analysis: Beyond correctly classifying injured and attack, the query requires linking them to the earlier assault event, which provides the relevant causal or background context.

Seed-event query. The query involves two complementary judgments: (1) filtering out events that share topical or lexical overlap with the seed event but do not correspond to the pledge, and (2) identifying events expressed with different wording that nevertheless provide evidence about the pledge status.

1. Filtering out lexical overlap without pledge relevance

   • Seed event: Labour: We will support families with children by introducing free breakfast clubs in every primary school.

   • Sentence event: Twenty five schools in Devon were among the first in the country to offer free, daily breakfast clubs.

   • Analysis: Although the sentence mentions breakfast clubs, it describes a local implementation in a limited set of schools, whereas the seed event concerns a national pledge to introduce free breakfast clubs in every primary school. This requires distinguishing local evidence from the global policy commitment.

2. Identifying implicit evidence with different wording

   • Seed event: Labour: Labour will not increase taxes on working people, which is why we will not increase National Insurance.

   • Sentence event: Direct taxes as a percentage of gross income for all UK households remained relatively stable at 22.7%, between FYE 2023 and FYE 2024.

   • Analysis: The sentence does not explicitly mention “National Insurance” or “working people”, but the share of direct taxes in household income provides evidence about whether the tax burden increased, making the event relevant to the pledge.

Case Study

We first examine a factuality driven example: “Neither company made any official statement about the technical issues.” In this case, the light verb made serves as syntactic support and does not independently introduce an event. Instead, the nominal predicate statement$_{\text{NegatedFactual}}$ acts as the event trigger, whose factuality is determined by the scope of syntactic negation and filtered out.

We further present a representative case where the event mention offers in “A primary school offers a Breakfast Club running from 7:45am every day” should not be considered related to the seed event “Labour: We will support families with children by introducing free breakfast clubs in every primary school,” because their event participants and affected scopes are fundamentally different. The former describes a routine activity conducted by an individual school, while the latter refers to a nationwide policy initiative proposed by a political organization.