betaFeedback

WorksComputer Science

CREDO: Evidence-Conditioned Credit Assignment for Temporal Reasoning Agents

Abstract

Temporal Knowledge Graph Question Answering (TKGQA) requires language model agents to retrieve evidence, select relevant facts, and perform multi-step temporal reasoning. However, the trajectory-level outcome signal in GRPO does not explicitly distinguish the quality of intermediate decisions: downstream failures can penalize useful evidence acquisition, while identical rewards in all-failed groups eliminate the relative outcome signal despite differences in intermediate progress. We propose Credit Redistribution with Evidence-Driven Opportunities (CREDO), which uses intermediate evidence to recover local supervision, including in all-failed groups, and determine where outcome supervision acts. CREDO assigns positive stage-level credit through comparisons among rollouts of the same question and uses retained answer overlap as an opportunity proxy to shift part of the outcome-policy weight toward reasoning and answering. The framework preserves terminal rewards and trajectory-level advantages, while redistribution conserves total outcome-policy weight per trajectory. Following the public evaluation protocol of prior work, CREDO raises Overall accuracy from 78.0% to 82.4% and Multiple accuracy from 55.0% to 65.6% on MultiTQ relative to the strongest published baseline in our comparison. On TimelineKGQA, it raises cross-KG accuracy from 30.9% to 36.4% relative to our reproduced baseline via direct transfer to ICEWS Actor under matched training and evaluation settings.

Introduction

Temporal Knowledge Graph Question Answering (TKGQA) requires reasoning over time-dependent facts under temporal constraints [6] (Saxena et al. 2021). Recent advances in large language models have improved temporal reasoning through structured retrieval, evidence construction, and experience reuse [14] (Liu et al. 2026; Tan et al. 2026). Agent-based approaches further integrate retrieval and reasoning into dynamic interaction trajectories, allowing models to adapt decisions to new evidence [13, 18, 15] (Lv et al. 2026). However, an agent may retrieve useful facts but fail to retain or reason over them correctly. The final answer therefore only partially reflects intermediate decision quality, motivating closer examination of how temporal agents learn from their interaction trajectories.

Reinforcement learning enables language models to learn multi-step reasoning and search strategies from outcome supervision [12, 19, 1] (Shao et al. 2024; Song et al. 2025), with recent extensions to temporal agents [11, 15]. However, outcome-supervised GRPO assigns the same terminal-reward-based advantage to all trainable tokens within a trajectory, without explicitly distinguishing intermediate decision quality (Shao et al., 2024). Process supervision and fine-grained credit assignment provide more localized learning signals (Lightman et al., 2024; Wang et al., 2024; Xie et al., 2026; Zong et al., 2026; Zhang et al., 2026). For temporal agents, credit assignment must also account for decision dependencies: evidence selection relies on what retrieval exposes, while answer derivation relies on what selection retains. This raises a central question: how can intermediate evidence guide both what agents learn from unsuccessful trajectories and where terminal supervision should act?

Failed trajectories can still contain useful evidence. An agent may retrieve and retain relevant temporal facts yet answer incorrectly (Figure 1). In mixed-outcome groups, useful retrieval and selection decisions can receive negative outcome credit. When all rollouts fail with identical rewards, outcome advantages vanish, leaving differences in intermediate evidence quality unexploited. These cases motivate distinguishing local behavior quality from evidence opportunity: how well a stage performs and how much answer-related evidence upstream decisions provide.

Figure 1. Motivation of CREDO. In a mixed-outcome group, useful intermediate decisions can receive negative outcome credit when the final answer is wrong. CREDO uses intermediate evidence for within-question comparisons and evidence-conditioned outcome redistribution.

Credit redistribution with evidence-driven opportunities. Motivated by this analysis, we propose Credit Redistribution with Evidence-Driven Opportunities (CREDO), which uses intermediate evidence to guide both local supervision and outcome credit. CREDO compares retrieval coverage, evidence retention, and answer quality across rollouts of the same question, rewarding locally superior behavior while conditioning downstream credit on upstream evidence. These comparisons can recover local learning signals even in all-failed groups. For eligible trajectories with retained answer-related evidence, CREDO also shifts a controlled fraction of outcome-policy weight toward reasoning and answering tokens. This redistribution applies to both positive and negative advantages while preserving total outcome-policy weight per trajectory. Both components retain the original terminal reward and trajectory-level GRPO advantage without requiring an additional learned evaluator. Answer overlap with retrieved and retained facts serves as a proxy for evidence opportunity.

Contributions. Our main contributions are threefold. (1) We analyze credit assignment in temporal agents through dependencies among retrieval, evidence selection, and answering, distinguishing local behavior quality from upstream evidence opportunity when learning from failed trajectories. (2) We propose CREDO, which distinguishes local quality from upstream evidence opportunity to recover stage-level supervision through within-question comparisons, including in all-failed groups. (3) We develop evidence-conditioned outcome redistribution that shifts policy weight toward reasoning and answering for both positive and negative trajectory advantages while preserving total outcome-policy weight per trajectory, terminal rewards, and GRPO advantages.

Related Work

Temporal Knowledge Graph Question Answering

Research on factual knowledge spans its acquisition, completion, and updating. KnowPrompt incorporates semantic knowledge from relation labels into prompt tuning for relation extraction [3], while MKGformer integrates visual and textual representations through multi-level fusion for multimodal knowledge graph completion [2]. MultiMedBench examines knowledge updating in medical visual question answering by evaluating the reliability, generality, and locality of knowledge editing (Wen et al., 2026). Complementing these studies on acquiring and maintaining knowledge, TKGQA investigates how to use time-dependent facts to answer questions under temporal constraints. Early TKGQA approaches include embedding-based methods that score candidate answers [6, 5] (Saxena et al. 2021; Mavromatis et al. 2022), semantic parsing methods that generate executable queries [16, 4, 8] (Neelam et al. 2021), and graph-based methods that model structural dependencies [17 a] (Sharma et al. 2023; Liu et al. 2023b). Recent LLM-based methods improve question interpretation through rewriting in TimeR4 (Qian et al., 2024) and decomposition in RTQA [10], organize reasoning through temporal tools in TempAgent [13] and planning in PoK (Qian et al., 2025), and reuse successful solution paths in MemoTime (Tan et al., 2026). Temp-R1 further explores end-to-end reinforcement learning for autonomous temporal reasoning [11]. CREDO thus studies how intermediate evidence guides credit assignment during temporal-agent training, considering dependencies among retrieval, selection, and answering.

Search Agents and Reinforcement Learning

PPO (Schulman et al., 2017) and GRPO (Shao et al., 2024) optimize policies from outcome rewards without token-level process annotations. OpenAI-o1 and DeepSeek-R1 demonstrate effective reinforcement learning for reasoning [12] (OpenAI et al. 2024), while Search-R1, R1-Searcher, and ReSearch extend it to agents interacting with external knowledge [19, 1] (Song et al. 2025). Recent methods supplement terminal feedback with finer-grained credit for intermediate decisions [9] (Ma et al. 2026). TIPS derives turn-level rewards from changes in answer likelihood under a teacher model (Xie et al., 2026), while StepSearch uses intermediate search rewards and token-level supervision using information gain and redundancy penalties (Wang et al., 2025). CREDO instead uses answer-overlap proxies for within-question stage comparisons and evidence-conditioned outcome-weight redistribution, preserving terminal rewards and trajectory-level advantages without an additional learned evaluator.

Method

Credit Redistribution with Evidence-Driven Opportunities (CREDO) uses acquired and retained evidence to reward useful intermediate behavior through within-question comparisons and redistribute outcome supervision (Figure 2). Both components jointly optimize one agent while preserving terminal rewards and GRPO trajectory advantages. Algorithm 1 summarizes the update.

Figure 2. Framework of CREDO. CREDO evaluates retrieved and retained evidence to recover local credit through within-question comparisons and redistribute outcome-policy weight toward reasoning and answering. The illustration uses ρ=0.4\rho= 0.4; the experimental default is ρ=0.04\rho= 0.04.

Problem Formulation

Given a temporal knowledge graph K\mathcal{K} and a training pair (x,G)∈D(x,G) \in\mathcal{D}, an agent πθ\pi_{\theta} interacts with K\mathcal{K} to answer xx, where GG contains acceptable answers. The behavior policy πold\pi_{\mathrm{old}} samples KK trajectories {τi}i=1K\{\tau_i\}_{i=1}^{K} per question. Their original terminal rewards determine the GRPO advantages A^i\hat{A}_i.

We distinguish retrieval QQ (Search), selection EE (Filter and Rank), and reasoning/answering RR (post-retrieval Think and Answer), which may recur across rounds. Let Ait∈{0,1}A_{it} \in\{0,1\} indicate trainable model tokens and Mits≤AitM_{it}^{s} \le A_{it} their membership in stage s∈{Q,E,R}s \in\{Q,E,R\}. Stage masks are disjoint, exclude environment observations (Information), and locate credit; evidence states determine its assignment. Retrieval and selection masks can aggregate repeated actions; reasoning uses the final post-retrieval Think region and Answer contents. Appendix A.1 specifies localization.

Evidence-Driven Local Credit

Evaluating acquired and retained evidence. IiI_i contains all retrieved Information lines. Final selection SiS_i takes non-empty lines from the last complete Rank output, or the last complete Filter output if Rank is empty or absent. This answer-independent preference reflects Rank’s organizational role. Line-level provenance checks yield ordered retained evidence S~i=(l0,l1,…)\widetilde{S}_i=(l_0,l_1,\ldots).

To estimate answer-related evidence, let cov⁡(l,g)\operatorname{cov}(l,g) denote the fraction of normalized answer tokens in gg covered by fact line ll, using multiset overlap. Retrieved and retained coverage are

qiQ=max⁡l∈Ii,g∈Gcov⁡(l,g),cisel=max⁡lj∈S~i,g∈Gcov⁡(lj,g)log⁡2(j+2).(1)q_i^{Q}=\max_{\substack{l\in I_i,\,g\in G}}\operatorname{cov}(l,g),\qquad c_i^{\mathrm{sel}}=\max_{\substack{l_j\in\widetilde{S}_i,\,g\in G}}\frac{\operatorname{cov}(l_j,g)}{\log_2(j+2)}. \tag*{(1)}

The positional discount gives greater credit to evidence placed earlier in the selection. Empty or unreliable evidence receives zero coverage. We then evaluate selection relative to its retrieved input and answering relative to the references:

qiE={min⁡{cisel/qiQ,1},qiQ>δ,0,otherwise,qiR=max⁡g∈GF1⁡token(y^i,g).(2)q_i^{E}= \begin{cases} \min\{c_i^{\mathrm{sel}}/q_i^{Q},1\}, & q_i^{Q}>\delta,\\ 0, & \text{otherwise}, \end{cases} \qquad q_i^{R}=\max_{g\in G}\operatorname{F1}_{\mathrm{token}}(\hat{y}_i,g). \tag*{(2)}

where δ>0\delta>0 is a numerical threshold and y^i\hat{y}_i is the locally extracted answer. Missing or unreliable answers receive zero quality. These coverage and overlap scores serve as local quality proxies; normalization, provenance, and extraction details are provided in Appendix A.1.

Accounting for evidence dependencies. We estimate evidence opportunity through upstream answer overlap:

ωiE=qiQ,ωiR=cisel.(3)\omega_i^{E}=q_i^{Q},\qquad\omega_i^{R}=c_i^{\mathrm{sel}}. \tag*{(3)}

Here ωiE\omega_i^{E} measures retrieved overlap and ωiR\omega_i^{R} retained overlap. High retention quality qiEq_i^{E} can coexist with low ωiR\omega_i^{R} when little answer-related evidence was retrieved. Retrieval has no upstream dependency. These proxies do not verify relations, temporal constraints, or multi-fact completeness: unrelated facts may contain answer tokens, while evidence for computed answers may lack them. Provenance verifies the retrieved source rather than semantic sufficiency; controlled evidence perturbations illustrate this boundary (Appendix A.4).

Recovering useful local behavior. We compare stage quality among trajectories of the same question, allowing intermediate differences to remain informative when terminal rewards coincide. Let

Algorithm 1 CREDO training update

Input: Mini-batch BB, behavior policy πold\pi_{\mathrm{old}}, rollout count KK, and CREDO hyperparameters.

  1. for each (x,G)∈B(x,G) \in B do

  2. Sample KK trajectories with πold\pi_{\mathrm{old}}; compute original rewards and GRPO advantages A^i\hat{A}_i.

  3. Extract evidence and stage masks; determine token availability tist_i^s.

  4. Compute qiQq_i^Q, ciselc_i^{\mathrm{sel}}, qiEq_i^E, and qiRq_i^R using Eqs. (1)–(2).

  5. Determine comparison validity visv_i^s; set ωiE=qiE\omega_i^E=q_i^E and ωiR=cisel\omega_i^R=c_i^{\mathrm{sel}}.

  6. for each stage s∈{Q,E,R}s \in\{Q,E,R\} do

  7. Compute peer comparisons FisF_i^s and local advantages aisa_i^s (Eqs. (4)–(5)).

  8. Compute the unnormalized stage budget; set it to zero if no positives exist (Appendix § 6.§ 1).

  9. end for

  10. Normalize budgets across stages to obtain αs\alpha_s; allocate uitsu_{it}^s by Eq. (7).

  11. Identify eligible trajectories (§ 3.3); assign witw_{it} by Eq. (6).

  12. end for

  13. Optimize Joutcomew+λlocalJlocalJ_{\mathrm{outcome}}^w+\lambda_{\mathrm{local}}J_{\mathrm{local}} using separately normalized clipped surrogates; omit zero-weight terms.

vis∈{0,1}v_i^s \in\{0,1\} indicate comparison validity and Vs={i:vis=1}V_s=\{i:v_i^s=1\}. For i∈Vsi\in V_s and ∣Vs∣>1|V_s|>1, define

Fis=1∣Vs∣−1∑j∈Vs∖{i}sgn⁡(qis−qjs).(4)F_i^s=\frac{1}{|V_s|-1}\sum_{j\in V_s\setminus\{i\}}\operatorname{sgn}(q_i^s-q_j^s). \tag*{(4)}

Ties contribute zero, so a positive score indicates more wins than losses against valid peers. We set Fis=0F_i^s=0 when comparison is unavailable. With [z]+=max⁡(z,0)[z]_+=\max(z,0) and tis∈{0,1}t_i^s\in\{0,1\} indicating reliable stage-token localization, the auxiliary advantages are

aiQ=viQtiQqiQ[FiQ]+,ais=vistisωisqis[Fis]+,s∈{E,R}.(5)a_i^Q=v_i^Qt_i^Qq_i^Q[F_i^Q]_+,\qquad a_i^s=v_i^st_i^s\omega_i^sq_i^s[F_i^s]_+,\quad s\in\{E,R\}. \tag*{(5)}

Locally superior behavior receives positive credit, scaled by upstream opportunity for dependent stages. Evaluable failures can be zero-quality peers without trainable tokens. Local quality differences can thus supply positive supervision even in all-failed groups.

Empirical illustration. Figure 3 shows recurring all-wrong groups during Stage 1. Of these, 38.29% retain active local credit despite zero terminal GRPO advantages. Activity requires positive local advantages, trainable tokens, and allocated weights. The trace demonstrates signal availability, without isolating its contribution to final accuracy. Counts and reward settings appear in Appendix § 6.§ 3.

Line chart showing all-wrong groups and all-wrong groups with local credit over Stage 1 training steps

Figure 3. All-wrong groups and those with active local credit in CREDO Stage 1 (8B). Both denominators count all sampled groups.

Evidence-Conditioned Outcome Redistribution

Locating outcome supervision. The original negative GRPO advantage can still penalize useful evidence acquisition. When retained answer overlap indicates an answering opportunity, we therefore shift part of the outcome-policy weight toward reasoning and answering.

Let ni=∑tAitn_i=\sum_t A_{it} and mi=∑tMitRm_i=\sum_t M_{it}^R be the numbers of trainable and reasoning/answering tokens. Redistribution requires viR=1v_i^R=1, ωiR>0\omega_i^R>0, ni,mi>0n_i,m_i>0, and mi/ni≥ηm_i/n_i\ge\eta, where η∈(0,1]\eta\in(0,1] prevents excessive concentration on a short token span. For eligible trajectories E\mathcal{E}, we assign

wit={(1−ρ)Ait+ρnimiMitR,i∈E,Ait,i∉E,∑twit=ni.(6)w_{it}= \begin{cases} (1-\rho)A_{it}+\rho\frac{n_i}{m_i}M_{it}^R, & i\in\mathcal{E},\\ A_{it}, & i\notin\mathcal{E}, \end{cases} \qquad \sum_t w_{it}=n_i. \tag*{(6)}

Here ρ∈[0,1)\rho\in[0,1) mixes the original token weights with a reasoning/answering-only allocation; the net weight moved into this region is ρ(ni−mi)\rho(n_i-m_i). The factor ni/min_i/m_i preserves total policy weight. Evidence opportunity determines eligibility rather than continuously scaling ρ\rho. The conserved quantity is the sum of policy weights, not the gradient magnitude.

Redistribution applies to either sign of A^i\hat{A}_i. When 0<ρ0 < \rho and mi<nim_i < n_i, it increases reasoning/answering weights and decreases others; if mi=nim_i = n_i, weights are unchanged. If A^i=0\hat{A}_i = 0, redistribution creates no outcome signal, but local credit can exploit intermediate quality differences.

Policy Optimization

Normalizing local supervision. To keep auxiliary learning from scaling with trajectory length or the number of positive examples, we assign each stage a normalized budget αs\alpha_s within each question group (the group index is omitted here). This budget combines valid comparison support, upstream opportunity for dependent stages, within-group quality diversity, and remaining improvement; its complete definition appears in Appendix § 6.§ 1. Let Hs={i:ais>0}\mathcal{H}_s = \{i : a_i^s > 0\}, hs=∣Hs∣h_s = |\mathcal{H}_s|, and nis=∑tMitsn_i^s = \sum_t M_{it}^s. For each positive example, define

uits=αshsnisMits,i∈Hs,∑i,tuits=αs.(7)u_{it}^s = \frac{\alpha_s}{h_s n_i^s}M_{it}^s,\qquad i \in\mathcal{H}_s,\qquad\sum_{i,t}u_{it}^s = \alpha_s. \tag*{(7)}

Other token weights are zero; stages without valid positive examples receive zero budget. The allocation controls total stage weight, while advantage magnitudes still determine the weighted loss.

Joint optimization. We combine the redistributed outcome signal with the recovered local credit:

J(θ)=Joutcomew(θ)+λlocalJlocal(θ).(8)J(\theta) = J_{\mathrm{outcome}}^w(\theta) + \lambda_{\mathrm{local}}J_{\mathrm{local}}(\theta). \tag*{(8)}

where JoutcomewJ_{\mathrm{outcome}}^w uses the original GRPO advantage A^i\hat{A}_i with token weights witw_{it}, and JlocalJ_{\mathrm{local}} uses stage advantages aisa_i^s with weights uitsu_{it}^s. The outcome term uses token-mean aggregation within each static microbatch, with redistributed token weights in both the numerator and denominator. The local term uses PPO-minibatch weight normalization; Appendix § 6.§ 1 specifies clipping, accumulation, and regularization. Zero-weight terms are omitted. The coefficient λlocal\lambda_{\mathrm{local}} controls auxiliary supervision. Local credit determines which stage behavior to reinforce; redistribution determines where outcome supervision acts. Inference requires neither reference answers nor credit computations.

Experiments

Setup

Datasets. We train CREDO on two datasets: MultiTQ (Chen et al., [6]) and TimelineKGQA-CRON (Sun et al., 2025). We evaluate three settings: MultiTQ, CronQuestion KG, and ICEWS Actor. Following Temp-R1’s evaluation layout (Gong et al., [11]), we report MultiTQ results by question and answer type, and evaluate both in-domain question answering and cross-KG transfer on TimelineKGQA. Table 1 and Table 2 present these comparisons. § 4.2 describes the tasks and evaluation protocol; dataset sizes, category distributions, and data-split audits appear in Appendix § 6.§ 2.

ModelOverallQuestion TypeAnswer Type
MultipleSingleEntityTime
TKG Embedding-based methods
EmbedKGQA0.2060.1340.2350.2900.001
CronKGQA0.2790.1340.3370.3280.156
MultiQA0.2930.1590.3470.3490.157
Prompt-based LLMs
ARI0.3800.2100.6800.3940.344
TempAgent0.7020.3160.8570.6240.870
MemoTime0.7300.4590.8290.6770.846
RTQA0.7650.4240.9020.6920.942
Fine-tuning-based LLMs
Search-R10.3520.0940.4740.2300.705
TimeR40.7280.3350.8870.6390.945
PoK0.7790.4090.9290.6960.962
Temp-R10.7800.5500.8880.7140.969
CREDO0.8240.6560.9030.7710.977

Table 1. Performance comparison on MultiTQ. CREDO follows the normalized substring-matching protocol in Temp-R1’s public evaluation code. Baseline scores are adopted from Table 3 of Temp-R1 [11]. The best and second-best scores under each metric are highlighted in colors, respectively; ties receive the same highlight.

ModelCronQuestion KG (In Domain)ICEWS Actor (Out of Domain)
OverallSimpleMediumComplexOverallSimpleMediumComplex
RAG Baseline0.2350.7040.0920.0090.2650.6600.1280.011
LLaMA2-7B0.1690.0490.1430.2820.1110.0350.0660.322
GPT-4o0.2060.0690.1300.3760.1130.0510.0350.353
RTQA0.2980.6080.2180.135––––
Temp-R1*0.5080.8440.2410.4460.3090.4470.1230.324
CREDO0.5230.8440.2780.4560.3640.5010.1700.393

Table 2. Performance comparison on TimelineKGQA across in-domain (CronQuestion KG) and out-of-domain (ICEWS Actor) settings. *Our reproduction, using the same base training configuration as CREDO and Temp-R1’s public scoring protocol (Appendix § 6.§ 2). Other baseline results are adopted from Temp-R1 (Gong et al., [11]). The best and second-best scores under each metric are highlighted in colors, respectively; ties receive the same highlight.

Baselines. On MultiTQ, we compare CREDO with three baseline groups: (1) TKG embedding-based methods, including EmbedKGQA (Saxena et al., 2020), CronKGQA (Saxena et al., 2021), and MultiQA (Chen et al., [6]); (2) prompt-based LLM methods, including ARI (Chen et al., [7]), TempAgent (Hu et al., [13]), MemoTime (Tan et al., 2026), and RTQA (Gong et al., [10]); and (3) fine-tuning-based LLM methods, including Search-R1 (Jin et al., [19]), TimeR4 (Qian et al., 2024), PoK (Qian et al., 2025), and Temp-R1 (Gong et al., [11]). TimelineKGQA baselines include RAG Baseline, LLaMA2-7B, GPT-4o, RTQA, and Temp-R1. For TimelineKGQA, we train Temp-R1 and CREDO under identical base configurations and evaluate them on identical inputs using Temp-R1’s public scoring protocol (Appendix § 6.§ 2), reducing confounding from protocol differences and making the comparison more directly reflect differences between methods. Other baseline scores come from Table 3 and Table 2 of Temp-R1 for MultiTQ and TimelineKGQA, respectively.

λlocal\lambda_{\mathrm{local}}ρ\rhoMultiple (%)
0.030.0445.00
0.010.0443.75
0.050.0445.00
0.030.0245.00
0.030.0645.62

Table 3. Peak Multiple accuracy on MultiTQ (3B), using the same evaluation settings as the 8B experiments. The default configuration is shown in bold.

Implementation Details. We use Meta-Llama-3.1-8B-Instruct with E5-base-v2 retrieval and a FAISS Flat index, following Jin et al. [19]. Training follows Temp-R1’s two-stage procedure (Gong et al., [11]). CREDO-family runs share initial assets and base training settings; selected models use a common evaluation configuration. Checkpoints are selected by monitoring accuracy (Appendix § 6.§ 2). Defaults are λlocal=0.03\lambda_{\text{local}} = 0.03, ρ=0.04\rho= 0.04, and η=0.05\eta= 0.05. All 8B reinforcement learning experiments run on one machine with 8 Ascend 910B NPUs. Appendix § 6.§ 2 gives the configurations and distinguishes these ablations from subsequent frozen-model diagnostics.

Benchmarks and Metrics

Temporal KGQA. Given a question and a temporal knowledge graph, the agent returns the requested entity, time, or other value under the question’s temporal constraints. Evaluation concerns final answers without requiring a predefined tool-use trajectory, allowing different retrieval and reasoning paths across one or more interaction rounds to produce acceptable answers.

MultiTQ. MultiTQ (Chen et al., [6]) evaluates temporal reasoning at year, month, and day granularities. The official Single categories are Equal, Before/After, and First/Last; Multiple includes Equal Multi, Before Last, and After First. We use these official labels without treating them as measured constraint counts. Our fixed evaluation set contains 500 questions, comprising 340 Single and 160 Multiple questions. Table 1 reports results by question type and answer type.

TimelineKGQA. TimelineKGQA (Sun et al., 2025) categorizes questions as Simple, Medium, or Complex according to whether they require one, two, or three contextual facts. Tasks include temporally constrained retrieval, comparison, composition, and ranking; answers include entities, time points, intervals, durations, and temporal relations. Following Temp-R1, we evaluate on CronQuestion KG and directly transfer the same checkpoint to ICEWS Actor without further training, assessing cross-KG transfer. Table 2 reports Overall and complexity-level results.

Metrics. Following the evaluation protocol in the public Temp-R1 repository [11], we evaluate final answers using normalized substring matching. A prediction is correct if any acceptable answer occurs contiguously within the normalized final answer. Let y^i\hat{y}_i denote the predicted answer, GiG_i the set of acceptable answers, and ν(⋅)\nu(\cdot) the normalization function. Accuracy over NN questions is defined as:

Acc⁡sub=1N∑i=1N1[∃ g∈Gi:substr⁡(ν(g),ν(y^i))],(9)\operatorname{Acc}_{\mathrm{sub}} = \frac{1}{N}\sum_{i=1}^{N} \mathbb{1}[\exists\ g \in G_i : \operatorname{substr}(\nu(g), \nu(\hat{y}_i))], \tag*{(9)}

where substr⁡(a,b)\operatorname{substr}(a,b) indicates contiguous occurrence. Answers are extracted from <answer> tags; normalization lowercases text, removes punctuation and English articles, and collapses whitespace. Missing answers and execution failures count as incorrect. This metric follows the baseline protocol and measures answer containment; it does not require the output to contain only a single unambiguous answer. Appendix § 6.§ 1 provides extraction details.

Experimental Results

Results on MultiTQ. CREDO achieves the best Overall and Multiple accuracy in Table 1, exceeding the strongest published baseline by 4.4 and 10.6 percentage points, respectively. The larger gain on Multiple questions highlights its effectiveness on compositional temporal tasks, where agents must coordinate evidence acquisition, selection, and answer derivation. This pattern aligns with the design of CREDO: local comparisons recover supervision from intermediate progress, while evidence-conditioned redistribution directs outcome-policy weight toward reasoning and answering. These mechanisms address the dependencies that make useful retrieval alone insufficient for solving complex questions. CREDO also ranks first on both entity and time answers, showing that its performance advantage extends across answer formats.

Results on TimelineKGQA. In Table 2, CREDO improves Overall accuracy over reproduced Temp-R1 on CronQuestion KG and direct ICEWS Actor transfer by 1.5 and 5.5 percentage points, respectively. On CronQuestion KG, the gains come from Medium and Complex questions while Simple accuracy is maintained, highlighting the benefit on tasks requiring multiple contextual facts to be integrated under the question’s temporal constraints. The larger improvement on ICEWS Actor, with gains across all three complexity levels, further supports transfer beyond the training knowledge graph. The same Cron-trained checkpoint is applied directly without additional training, demonstrating that the learned policy remains effective in a new graph environment. Under matched training and evaluation settings, these results support evidence-conditioned credit assignment for both compositional temporal question answering and direct cross-KG transfer.

Analysis

Performance across training stages. Figure 5(a) summarizes the stage-wise evaluation records. CREDO achieves the highest final-stage accuracy, followed by CWO and our reproduced GRPO baseline. The matched final-stage comparison is consistent with a benefit from local credit and an additional improvement from outcome redistribution. Earlier intermediate-stage records are retained as reported; evaluation details appear in Appendix § 6.§ 2.

Figure 5. CREDO diagnostics. (a) Stage-wise accuracy on 500 MultiTQ inputs; the GRPO Stage 2 score is from our reproduction under the same training and evaluation settings as CREDO. (b) Additional substring-score loss from temporal rather than generic distraction. (c) Substring accuracy with temporal distractors, averaged over three decoding seeds on 67 questions. (d) Official Multiple types from the paired evaluation, with group sizes below the axis. Sources: Appendix A.2.

Contribution of local credit and outcome redistribution. CWO (CREDO without outcome redistribution) retains local credit, allowing Figure 4 to examine the additional value of redistribution under the same two-stage schedule, selection criterion, and evaluation settings. CREDO improves Overall and Multiple accuracy by 2.2 and 3.75 percentage points, respectively, corresponding to net gains of 11 and six correct answers. Because both methods already recover local supervision, these observed differences suggest that controlling where outcome supervision acts can provide further value beyond local credit alone. The larger gain on Multiple questions is consistent with the motivation for emphasizing reasoning and answering in evidence-dependent tasks. Appendix § 6.§ 3 reports the paired estimates and their uncertainty to contextualize the observed performance differences.

Figure 4. MultiTQ component comparisons: (a) Overall; (b) Multiple. GRPO uses the Temp-R1 reported results [11]. CWO disables redistribution; Random changes its target tokens; No-gate adds ungated answer-region redistribution to CWO. See Appendix § 6.§ 2 for selection details.

Redistribution variants. Random preserves eligibility, transferred mass, and target-token count; No-gate retains local credit and increases answer-region weights for trajectories satisfying structural and length conditions, without checking evidence overlap. CREDO exceeds both on Overall and Multiple under the common selection and evaluation protocol (Figure 4), favoring its evidence-conditioned configuration for determining where outcome supervision should act.

Official Multiple question types. In Figure 5(d), CREDO exceeds CWO on Before Last and After First, matches it on Equal Multi, and exceeds in-house GRPO throughout. These labels denote question types, not constraint counts (Appendix § 6.§ 3).

Comparison under matched settings. Under identical base training and evaluation settings, CREDO exceeds our Temp-R1 reproduction by 5.4 percentage points in Overall accuracy and 13.75 points on Multiple questions. The improvement also holds against a baseline evaluated within our setup. The larger Multiple gain supports evidence-conditioned training’s effectiveness on compositional temporal tasks. Appendix § 6.§ 3 reports paired results and confidence intervals.

Answer containment under temporal distraction. Retaining supporting facts without retrieval, CREDO shows near-constant substring accuracy on 67 Single entity questions (Figure 5(c)). On 60 Multiple questions, it leads in three evidence conditions despite CWO’s smaller temporal-distractor drop. Both use three decoding seeds (construction and results: Appendix § 6.§ 3).

Temporal-specific penalty. On the Single subset, Figure 5(b) compares generic and temporal distractors at matched counts and lengths, retaining supporting facts. CREDO combines a near-zero penalty with high substring accuracy (panel (c)), indicating stable target-answer containment with little additional loss from temporal distraction. GRPO’s larger penalty highlights CREDO’s advantage under fixed evidence. Negative penalties indicate higher accuracy with temporal distractors.

Hyperparameter sensitivity. Table 3 examines λlocal\lambda_{\mathrm{local}} and ρ\rho using Llama-3.2-3B-Instruct, varying one coefficient at a time while keeping all other training and evaluation settings fixed. For every setting, we evaluate the saved candidates on the same official 500-question MultiTQ set and report the highest accuracy on its 160 Multiple questions. The reported peaks are close across the tested values, indicating comparable attainable performance around the default configuration within this training budget. This exploratory comparison concerns peak performance rather than performance under the monitoring-based selection procedure used for the main 8B experiments.

Conclusion

In this paper, we introduce CREDO for evidence-conditioned credit assignment in temporal reasoning agents. CREDO recovers positive local supervision through same-question comparisons and reallocates outcome-policy weight toward reasoning and answering based on retained answer overlap. It preserves terminal rewards, GRPO advantages, and total outcome-policy weight without an additional learned evaluator or inference-time credit computation. Results on MultiTQ and TimelineKGQA show gains in compositional question answering and cross-KG transfer, while component comparisons support the combined training procedure under a common evaluation protocol.

Additional Details and Results

Implementation Details

Evidence and answer processing. Local scoring lowercases text, replaces ASCII punctuation with spaces, removes English articles, and collapses whitespace. Coverage is the fraction of reference-answer tokens found in a fact line, using multiset overlap; local answer F1 uses the same normalization. Retrieved evidence pools all complete Information blocks. The retained selection uses the last complete Rank block, falling back to the last complete Filter when that Rank is absent or empty. Document prefixes and title wrappers are removed before checking normalized selected lines against the retrieved-line set. Untraceable lines are discarded individually, and surviving lines are reindexed before positional discounting. Duplicate selected lines are retained. This response-wide provenance check verifies the retrieved source, not semantic or temporal sufficiency.

The local answer extractor accepts complete Answer tags case-insensitively and selects the last match shorter than 200 characters, or the shortest match if all are longer. The archived terminal extractor accepts lowercase tags and, when any short match exists, returns the last match regardless of length; otherwise it returns the shortest. Terminal normalization deletes punctuation rather than replacing it with spaces. The two branches can therefore evaluate different extracted or normalized text when outputs contain multiple Answer blocks. Missing answers receive zero quality or score.

Validity and token localization. Let TiT_i indicate truncation, CiQC_i^Q reliable Search/Information structure with an Information response before the next Search, CiEC_i^E reliable Filter/Rank structure, and NiEN_i^E non-empty selection content. The stage-local rules are

viQ=CiQ∨¬Ti,v_i^Q = C_i^Q \lor\neg T_i,
viE=viQ∧(qiQ>ϵ0)∧[¬Ti∨(CiE∧NiE)],v_i^E = v_i^Q \land(q_i^Q > \epsilon_0) \land[\neg T_i \lor(C_i^E \land N_i^E)],
viR=viE∧(cisel>ϵ0)∧¬Ti,ϵ0=10−8.(10)v_i^R = v_i^E \land(c_i^{\mathrm{sel}} > \epsilon_0) \land\neg T_i,\qquad\epsilon_0 = 10^{-8}. \tag*{(10)}

Here ϵ0\epsilon_0 is the coverage threshold δ\delta in the main text. These validity rules govern local comparisons and the evidence-conditioned CREDO route. No-gate retains them for local credit but uses structural and token-length eligibility for redistribution, without the overlap requirements in viRv_i^R (Appendix A.2). Separately, tist_i^s requires a non-empty reliable token mask. Thus, an evaluable failure may contribute a zero-quality comparison without supplying teacher tokens.

The retrieval mask contains complete Search contents, and the selection mask aggregates complete Filter and Rank contents. The reasoning/answering mask contains complete Think contents starting after the last Information block (or the last Search when Information is absent), together with complete Answer contents, subject to structural validity. Tokens must lie entirely inside their content spans. Tokenizer offsets, with prefix-decoding checks when needed, align text spans to tokens. Global alignment failures, overlapping stage masks, or out-of-bounds masks disable local credit. Plan, tags, and unclassified model text receive no local mask but may retain outcome supervision. Information and padding are excluded from actor updates. These masks aggregate actions across rounds; the credit is stage-level rather than a separate estimate for each action.

Stage budgets. Budgets are computed per complete question group gg before PPO minibatch partitioning. Write Vgs=∣Vgs∣V_{gs} = |V_{gs}| for the valid comparison count and KgK_g for the sampled group size. Let (oiQ,oiE,oiR)=(1,qiQ,cisel)(o_i^Q,o_i^E,o_i^R) = (1,q_i^Q,c_i^{\mathrm{sel}}), and let Hgs\mathcal{H}_{gs} contain positive local teachers with non-empty masks. For Vgs≥2V_{gs} \ge2, define

bgs=Vgs/Kg,oˉgs=1Vgs∑i∈Vgsois,b_{gs} = V_{gs}/K_g,\qquad\bar{o}_{gs} = \frac{1}{V_{gs}}\sum_{i\in V_{gs}} o_i^s,
dgs=∑i,j∈Vgs∣qis−qjs∣Vgs(Vgs−1),egs=[1−1Vgs∑i∈Vgsqis]+.(11)d_{gs} = \frac{\sum_{i,j\in V_{gs}} |q_i^s-q_j^s|}{V_{gs}(V_{gs}-1)},\qquad e_{gs} = \left[1-\frac{1}{V_{gs}}\sum_{i\in V_{gs}}q_i^s\right]_+. \tag*{(11)}

These factors represent comparison support, opportunity, quality dispersion, and remaining improvement. The budget is

ℓgs=bgsoˉgsdgsegs1[∣Hgs∣>0],αgs={ℓgs/∑s′ℓgs′,∑s′ℓgs′>ϵ0,0,otherwise.(12)\ell_{gs}=b_{gs}\bar{o}_{gs}d_{gs}e_{gs}\mathbf{1}[|\mathcal{H}_{gs}|>0],\qquad \alpha_{gs}= \begin{cases} \ell_{gs}/\sum_{s'}\ell_{gs'}, & \sum_{s'}\ell_{gs'}>\epsilon_0,\\ 0, & \text{otherwise}. \end{cases} \tag*{(12)}

Incomplete groups, unavailable comparisons, non-finite values, and stages without positive teachers receive zero corresponding credit. Frontier ties contribute zero. (7) then divides each stage budget equally among its teachers and their stage tokens. Budgets are normalized across stages within a question, without additional question-difficulty weighting.

Policy loss and regularization. GRPO standardizes each trajectory reward within its question group using the sample standard deviation and denominator offset 10−610^{-6}. Local credit does not enter this reward. Outcome redistribution uses (6), with a 10−610^{-6} tolerance in the minimum token-fraction check and implementation range 0≤ρ<10 \le\rho< 1. The net weight transferred from outside the reasoning/answering region is ρ(ni−mi)\rho(n_i-m_i).

In minimization form, the PPO implementation uses the clipped ratio r=exp⁡(clip⁡(log⁡πθ−log⁡πold,−20,20))r = \exp(\operatorname{clip}(\log\pi_\theta- \log\pi_{\mathrm{old}},-20,20)) and dual clipping:

ϕ(a,r)=max⁡{−ar,−aclip⁡(r,0.8,1.2)},\phi(a,r)=\max\{-ar,-a\operatorname{clip}(r,0.8,1.2)\},
ϕ~(a,r)={min⁡{ϕ(a,r),−3a},a<0,ϕ(a,r),a≥0.(13)\widetilde{\phi}(a,r)= \begin{cases} \min\{\phi(a,r),-3a\}, & a<0,\\ \phi(a,r), & a\ge0. \end{cases} \tag*{(13)}

The local advantages are non-negative and do not activate the negative-advantage branch. Saved old-policy log probabilities are used for both terms.

Disjoint stage masks define token-level local advantages aitloc=∑sMitsaisa_{it}^{\mathrm{loc}}=\sum_s M_{it}^s a_i^s and weights uit=∑suitsu_{it}=\sum_s u_{it}^s. For microbatch BμB_\mu, the policy losses are

Lout,μ=∑(i,t)∈Bμwitϕ~(A^i,rit)∑(i,t)∈Bμwit,L_{\mathrm{out},\mu}= \frac{\sum_{(i,t)\in B_\mu}w_{it}\widetilde{\phi}(\widehat{A}_{i},r_{it})} {\sum_{(i,t)\in B_\mu}w_{it}},
Lloc,μ=∑(i,t)∈Bμuitϕ~(aitloc,rit)max⁡(Uμ,10−6),Uμ=∑(i,t)∈Bμuit.(14)L_{\mathrm{loc},\mu}= \frac{\sum_{(i,t)\in B_\mu}u_{it}\widetilde{\phi}(a_{it}^{\mathrm{loc}},r_{it})} {\max(U_\mu,10^{-6})}, \qquad U_\mu=\sum_{(i,t)\in B_\mu}u_{it}. \tag*{(14)}

Both training stages use the legacy actor implementation with static microbatches and token-mean loss aggregation (use_legacy_worker_impl=enable, use_dynamic_bsz=False, and loss_agg_mode=token-mean). The executed path receives no global token-count metadata, so the outcome denominator is computed within the current microbatch. Since redistribution preserves each trajectory’s total weight, this denominator equals the number of valid actor tokens in that microbatch, excluding tool observations and padding. Redistribution therefore changes relative token weighting without changing the denominator for a fixed microbatch.

For a PPO minibatch, local weight is summed across the default distributed process group to obtain U∗U_*, with world size WW. After accumulation scaling, the loss contributions are

∑μ[γμ(Lout,μ+βLKL,μ−ζHμ)+λlocalWUμU∗Lloc,μ].(15)\sum_\mu\left[ \gamma_\mu\left(L_{\mathrm{out},\mu}+\beta L_{\mathrm{KL},\mu}-\zeta H_\mu\right) +\lambda_{\mathrm{local}}\frac{W U_\mu}{U_*}L_{\mathrm{loc},\mu} \right]. \tag*{(15)}

The local term is omitted if U∗=0U_*=0. Static microbatches use the reciprocal accumulation count for γμ\gamma_\mu; the local branch compensates for that scaling before backward. Stage budgets therefore retain their relative weights inside the local loss. Each PPO minibatch ends with an optimizer update. Outcome losses are averaged through static microbatch accumulation, rather than normalized by a single token count over the entire PPO minibatch.

KL is an independent actor loss and is not added to the checked training rewards. The low-variance estimator is clip⁡(exp⁡(z)−z−1,−10,10)\operatorname{clip}(\exp(z)-z-1,-10,10) for z=clip⁡(log⁡πref−log⁡πθ,−20,20)z=\operatorname{clip}(\log\pi_{\mathrm{ref}}-\log\pi_\theta,-20,20). KL and entropy use the original actor response mask, not redistributed or local weights. Weight conservation thus does not imply invariant gradient magnitude.

Experimental Setup

Data and training. The fixed MultiTQ evaluation contains 340 Single and 160 Multiple questions, with 371 entity and 129 time answers. The year/month/day counts are 69/105/326. These are official categories, not inferred constraint counts. The processed data records contain 57,979 MultiTQ training-pool entries and 54,584 test entries, and 24,427 Cron training entries and 8,146 test entries; these describe the local processed files rather than the original benchmark release sizes. Normalized-question overlap between the processed MultiTQ training and test files is zero. The monitoring sets are seed-42 samples: 24 questions from test_multiple for Stage 1 and 30 questions from test_all for Stage 2. Both monitoring sets are disjoint from the fixed 500-question evaluation set. Checkpoints for the main comparisons are selected using monitoring accuracy; the 500-question evaluation scores are not used for this selection.

The policy is Meta-Llama-3.1-8B-Instruct, with E5-base-v2 retrieval and a FAISS Flat index. Table 4 gives the common MultiTQ settings for the in-house GRPO, CREDO, CWO, Random, and No-gate runs. Each starts from the same initial SFT weights and continues its own Stage 1 step-80 checkpoint into Stage 2 with a reinitialized optimizer. The main CREDO result uses Stage 2 step 400. Defaults are λlocal=0.03\lambda_{\mathrm{local}} = 0.03, ρ=0.04\rho= 0.04, and η=0.05\eta= 0.05. CWO is the configuration named CF in archived files: local credit remains active, while outcome redistribution is disabled.

SettingStage 1Stage 2
Scheduled updates80500
Questions per update ×\times rollouts60 ×\times 52 ×\times 8
PPO minibatch trajectories302
Data / sequence parallelism1 / 82 / 4
Learning rate5×10−75 \times10^{-7}5×10−75 \times10^{-7}
Gradient clipping51
KL / entropy coefficients0.01 / 0.0010.001 / 0.001
Sampling temperature / top-pp1 / 0.951 / 0.95
Retrieval top-kk3020
Maximum searches / model calls3 / 43 / 4
Per-call generation limit1,0001,024
Configured generation / response limit4,000 / 5,2004,096 / 8,192
Per-call Information limit1,4001,500
Monitoring interval / questions10 / 2420 / 30

Table 4. Common base training configuration for the five in-house 8B methods on MultiTQ. Generation limits and response-container limits measure different quantities.

Controlled ablations and model selection. Both training stages use normalized substring rewards in GRPO, CREDO, CWO, Random, and No-gate. The archived configurations agree on batch sizes, rollout counts, optimizer settings, KL and entropy coefficients, token budgets, retrieval settings, and scheduled updates. The CREDO-family Stage 1 input manifests record identical hashes for 12 assets, including all four SFT weight shards, processed data files, the knowledge graph, and the retrieval index. GRPO uses outcome-only supervision; CWO removes outcome redistribution; Random changes its target positions while preserving eligibility, target count, and transferred mass. No-gate retains CWO’s local-credit update. Its redistribution route requires reliable reasoning/answering-token localization, ni,mi>0n_i, m_i > 0, and mi/ni≥ηm_i/n_i \ge\eta, together with the structural validity checks, but does not require retrieved/retained answer overlap. Eligible trajectories use the same ρ\rho and weight transformation in Eq. (6); local-credit validity is unchanged. This compares structurally eligible answer-region redistribution with CREDO’s evidence-conditioned configuration.

For the main 8B comparisons, GRPO, CREDO, CWO, Random, and No-gate use monitoring-set accuracy to select among their available saved checkpoints, followed by evaluation under identical settings. All methods share the selection criterion, while candidate sets depend on checkpoint availability. In Stage 1, checkpoints for CREDO, CWO, Random, and No-gate were saved every 10 steps; in Stage 2, checkpoints were saved every 100 steps. All methods initialize Stage 2 from their own Stage 1 step-80 checkpoint. The final 500-question evaluation scores are not used for model selection in these main comparisons.

Evaluation protocol. We adopt normalized substring matching to retain compatibility with Temp-R1’s public evaluation implementation [11]. We verified the scoring behavior at commit 4a4b58f, with validation format and retrieval bonuses set to zero. Its evaluation entry point routes MultiTQ and TimelineKGQA to qa_em_format.compute_score_em; despite the function name, the underlying em_check tests substring containment rather than string equality. Extracted answers are lowercased, stripped of punctuation and English articles, and whitespace-normalized. A prediction receives one point if it contains any normalized acceptable answer; missing answers and execution failures receive zero. The earlier Search-R1 code also provides a normalized substring scorer [19], alongside an exact-match scorer. Our choice follows the scoring branch used by the direct temporal-agent baseline, rather than assuming that all prior implementations use identical metrics. The same rule is applied to all in-house comparisons. This establishes protocol consistency while retaining the metric’s answer-containment interpretation; additional candidates can still receive credit.

The main MultiTQ comparison uses one evaluation configuration for CREDO, CWO, Random, and No-gate, giving Overall scores of 82.4% (412/500), 80.2% (401/500), 79.0% (395/500), and 78.4% (392/500), respectively. Evaluation uses seed 42, temperature 1, top-p 0.95, generation/container limits of 5,000/6,200, Information limit 2,400, retrieval top-k 30, and up to three searches plus a final round.

MultiTQ baseline sources. Our in-house outcome-only GRPO baseline shares CREDO’s SFT initialization, training data, terminal reward, answer-extraction rules, and base training and evaluation settings; local credit and outcome redistribution are disabled. It scores 385/500 (77.0%) and provides the GRPO Stage 2 endpoint in Figure 5(a), the predictions for Table 5–Table 6, and the frozen-model diagnostics. Table 1 and Figure 4 retain the published Temp-R1 reference of 78.0% [11]. The literature comparison and our matched reproduction therefore use distinct result sources: CREDO’s Overall improvements are 4.4 and 5.4 percentage points, respectively. The final-stage GRPO, CWO, and CREDO scores use the common evaluation configuration. Earlier intermediate-stage values in Figure 5(a) remain the reported records; the curve does not constitute a controlled re-evaluation of every intermediate checkpoint.

CategoryNNGRPOCWOCREDO
Equal174156 (89.66)151 (86.78)152 (87.36)
Before/After9481 (86.17)89 (94.68)87 (92.55)
First/Last7265 (90.28)62 (86.11)68 (94.44)
Equal Multi3322 (66.67)28 (84.85)28 (84.85)
Before Last6934 (49.28)38 (55.07)41 (59.42)
After First5827 (46.55)33 (56.90)36 (62.07)
Single340302 (88.82)302 (88.82)307 (90.29)
Multiple16083 (51.88)99 (61.88)105 (65.63)
Entity371263 (70.89)282 (76.01)286 (77.09)
Time129122 (94.57)119 (92.25)126 (97.67)
Overall500385 (77.00)401 (80.20)412 (82.40)

Table 5.

Comparator / subsetNNCorrectedRegressedDifference95% CI
GRPO / Overall5005023+5.40[2.20, 8.80]
CWO / Overall5003423+2.20[−0.80, 5.20]
GRPO / Multiple1603311+13.75[6.25, 21.88]
CWO / Multiple1601913+3.75[−3.13, 10.63]

Table 6.

TimelineKGQA reproduction. The public materials available for our experiments did not provide the complete TimelineKGQA training pipeline needed to reproduce the published Temp-R1 models. We therefore train Temp-R1 and CREDO in our implementation with identical base configurations, including initialization, training data, optimizer settings, training budget, and retrieval environment. Evaluation follows the public normalized substring protocol, with identical inputs, decoding settings, token limits, interaction rules, and extraction routines. This provides a controlled comparison within our training setup. The reproduced Temp-R1 scores (50.8% Cron, 30.9% ICEWS) differ from the published 70.5%/64.2% [11]; we do not claim numerical reproduction of those models or superiority over their reported scores.

Each model is evaluated on 800 Cron and 800 ICEWS rows, with correct counts of 406 versus 418 and 247 versus 291. Cron-trained checkpoints transfer directly to ICEWS without further training. Simple/Medium/Complex counts are 250/245/305 for Cron and 351/276/173 for ICEWS. The sets contain 797 and 793 unique question/answer keys; duplicate rows remain in the averages.

PoK is not included in our TimelineKGQA reproduction because a complete public implementation was unavailable to us. Its published Cron/ICEWS scores of 65.1%/60.2%, reported in Temp-R1, remain external reference results. Other adopted rows in Table 2 provide context; the controlled comparison is reproduced Temp-R1 versus CREDO, rather than an exhaustive ranking of published systems.

Table 3 uses Llama-3.2-3B-Instruct. All 3B runs use 80 training steps with a training batch size of 32; the corresponding 8B Stage 1 runs use 80 steps with a batch size of 60. Within the 3B study, all settings share the same SFT initialization, training data, base training configuration, and evaluation protocol. Only λlocal\lambda_{\mathrm{local}} or ρ\rho is varied relative to the default. Evaluation uses the same settings and official 500-question MultiTQ set as the main 8B experiments, comprising 340 Single and 160 Multiple questions. The table reports only Multiple accuracy.

Supplementary Results

Formal component comparisons. CREDO’s observed Overall gain is 2.2 percentage points over CWO and 3.4 points over the selected Random model. The CREDO/CWO Multiple comparison is 105/160 versus 99/160, a 3.75-point difference. CREDO, CWO, Random, and No-gate use the same best-monitoring-checkpoint criterion and final evaluation settings. No-gate obtains 78.4% Overall and 57.5% Multiple (92/160); CREDO’s point estimates are higher by 4.0 and 8.13 percentage points, respectively. The component figure also includes Random’s reported Multiple accuracy of 57.5% and the literature GRPO reference (78.0% Overall, 55.0% Multiple) from Temp-R1. The latter is a reported baseline rather than a new independent run.

Question-type results. Table 5 gives the complete supplementary breakdown. Each model has 500 unique question identifiers with matching question/answer hashes. Missing extracted answers number 3, 3, and 6 for GRPO, CWO, and CREDO and remain in the denominator. CREDO’s gains over GRPO cover all three Multiple categories, while Single-category gains are not uniform.

Table 5: Paired MultiTQ results: correct answers (accuracy, %). CREDO and CWO also provide the main results; GRPO is our matched in-house baseline. Aggregate rows overlap the individual categories.

Paired comparisons. Table 6 reports corrections, regressions, and net differences on the same inputs. Intervals use 100,000 paired question bootstrap samples, seed 20260925, and percentile end-points at 2.5% and 97.5%, without multiple-comparison correction. Against the matched in-house GRPO checkpoint, corrections exceed regressions by 27 questions Overall and 22 on Multiple, with both paired intervals above zero. The CWO intervals include zero and quantify uncertainty in the main CREDO–CWO difference. These intervals are conditional on the selected checkpoints and quantify question-sampling uncertainty, not training-seed variation or checkpoint-selection uncertainty.

Table 6: Paired supplementary comparisons. “Corrected” means only CREDO is correct; “Regressed” means only the comparator is correct. Differences and intervals are in percentage points.

All-wrong groups. Across 80 Stage 1 updates, 4,800 question-group exposures yield 24,000 trajectories (K=5K = 5). Of these groups, 1,473 are all-wrong (30.69%); 564 have active local credit (11.75% of all groups; 38.29% of all-wrong groups). Both curves use all groups as the denominator. Rewards are binary substring correctness, with format/retrieval bonuses disabled and KL excluded, so all-wrong groups have zero outcome advantages. Within active all-wrong groups, retrieval, selection, and reasoning each occur in 351, 350, and 59 groups, respectively (overlapping counts). These observations establish local-signal availability at all three stages, not gradient magnitude or an isolated performance contribution.

Single-question distractors. Among the fixed 500 inputs, 84 questions have parseable explicit dates, 72 pass rule-based KG answer checks and finite evidence construction, and 67 admit matched distractor conditions. All 67 are Single questions with entity answers: 60 Equal and 7 Before/After, with day/month/year counts 51/13/3. They are not an independently human-labeled evidence-sufficiency set.

The prompt instructs the model to answer from supplied evidence without tools and end with an Answer block. The user input contains the question and numbered Information lines. Retrieval is disabled; decoding uses temperature 1, top-pp 0.95, generation limit 1,024, context limit 11,200, and seeds 42/43/44 with question offsets. The first complete Answer block stops generation.

Over-budget inputs are rejected rather than truncating support. Generic and temporal distractors are matched in count, length, format, and insertion position. The former are unrelated facts; the latter are topically related but violate the question’s temporal condition. Supporting facts remain fixed for k∈{0,2,4,8}k \in\{0, 2, 4, 8\}; an additional condition contains unrelated evidence only.

The temporal penalty subtracts temporal-condition accuracy from generic-condition accuracy at matched kk; negative values favor temporal distractors. Table 7 pools three decoding repetitions per question, with equal numbers of repetitions for every question. These are 67 questions, not 201 independent samples or training runs. This Single-question result concerns target containment with retained support; it does not directly test unique-answer selection, retrieval quality, or Multiple-question composition.

Evidence conditionGRPOCWOCREDO
Supporting facts only178 (88.56)195 (97.01)194 (96.52)
Generic, k=2k = 2164 (81.59)184 (91.54)196 (97.51)
Generic, k=4k = 4170 (84.58)184 (91.54)198 (98.51)
Generic, k=8k = 8154 (76.62)181 (90.05)196 (97.51)
Temporal, k=2k = 2157 (78.11)180 (89.55)196 (97.51)
Temporal, k=4k = 4152 (75.62)178 (88.56)197 (98.01)
Temporal, k=8k = 8142 (70.65)183 (91.04)196 (97.51)
Unrelated evidence only4 (1.99)3 (1.49)15 (7.46)

Table 7. Answer containment under fixed evidence. Entries are correct generations out of 201 (substring accuracy, %), from 67 questions and three decoding seeds.

Multiple-question distractors. We additionally evaluate 60 Multiple questions, with 20 from each official type: Equal Multi, Before Last, and After First. The 8B models are GRPO at step 200 and CWO/CREDO at step 400. Each model answers every question under three decoding seeds in each condition, yielding 180 generations per model and condition. Complete supporting facts and sorting candidates are retained. Generic and temporal conditions each add four facts; inputs are identical across models, retrieval is disabled, and no input is truncated. Scoring follows the same official normalized substring protocol.

CREDO achieves the highest absolute accuracy in every condition (Table 8). Under temporal distraction, its advantage over CWO narrows to three correct generations (1.67 percentage points). Relative to complete evidence, CREDO loses 13.89 percentage points and CWO loses 9.44; relative to generic distraction, CREDO loses 10.56 points. Thus, the advantage in absolute accuracy coexists with greater sensitivity to temporal distraction in this Multiple setting. These results extend the evaluation to compositional question types, but do not establish uniformly stronger temporal robustness or isolate correct reasoning from answer containment. The 180 generations represent repeated decoding on 60 questions, not independent questions or training runs.

Evidence conditionGRPOCWOCREDO
Complete evidence60 (33.33)114 (63.33)125 (69.44)
Generic, k=4k = 458 (32.22)96 (53.33)119 (66.11)
Temporal, k=4k = 462 (34.44)97 (53.89)100 (55.56)

Table 8. Multiple-question answer containment with complete support retained. Entries are correct generations out of 180 (substring accuracy, %), from 60 questions and three decoding seeds. Bold indicates the highest accuracy in each condition.

Hyperparameter sensitivity. Starting from the default (λlocal,ρ)=(0.03,0.04)(\lambda_{\mathrm{local}}, \rho) = (0.03, 0.04), we vary λlocal∈{0.01,0.03,0.05}\lambda_{\mathrm{local}} \in\{0.01, 0.03, 0.05\} with ρ\rho fixed, and vary ρ∈{0.02,0.04,0.06}\rho\in\{0.02, 0.04, 0.06\} with λlocal\lambda_{\mathrm{local}} fixed. All runs share the same 3B backbone, SFT initialization, training data, base training configuration, and evaluation protocol. Every row follows the same reporting rule: evaluate the available candidate checkpoints on the official 500-question MultiTQ set using normalized substring matching, and report the highest accuracy on the same 160 Multiple questions. This rule applies to the default configuration and all coefficient variants. The table therefore compares observed peak performance across parameter settings; unlike the main 8B comparisons, checkpoint selection in this exploratory analysis uses the reported evaluation subset. The results do not estimate performance under independent monitoring-based selection or variability across training seeds.

Discussion and Limitations

Evidence proxies. Answer overlap does not test semantic sufficiency. With answer tokens preserved, all 40 rule-checked Single examples retain positive coverage in each of the original, wrong-time, and wrong-relation conditions. This diagnoses coverage alone, not the full provenance/eligibility pipeline. Natural-trajectory gate precision and recall remain unmeasured; aggregated masks provide stage-level, not per-action, credit.

Mechanism and stability. Component comparisons do not isolate all-wrong updates, component interactions, or individual local-credit ingredients. Best-checkpoint comparisons describe the selection protocol rather than fixed-update effects. The training trace and 3B study use single runs; decoding repetitions and paired question intervals do not measure training-seed stability.

References

  1. [1]Mingyang Chen, Linzhuang Sun, Tianpeng Li, Haoze Sun, Yijie Zhou, Chenzheng Zhu, Haofen Wang, Jeff Z. Pan, Wen Zhang, Huajun Chen, Fan Yang, Zenan Zhou, and Weipeng Chen. Re-Search: Learning to reason with search for LLMs via reinforcement learning. arXiv preprint arXiv:2503.19470, 2025. doi: 10.48550/arXiv.2503.19470. URL https://arxiv.org/abs/2503.19470.
  2. [2]Xiang Chen, Ningyu Zhang, Lei Li, Shumin Deng, Chuanqi Tan, Changliang Xu, Fei Huang, Luo Si, and Huajun Chen. Hybrid transformer with multi-level fusion for multimodal knowledge graph completion. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 904–915. Association for Computing Machinery, 2022a. doi: 10.1145/3477495.3531992.DOI
  3. [3]Xiang Chen, Ningyu Zhang, Xin Xie, Shumin Deng, Yunzhi Yao, Chuanqi Tan, Fei Huang, Luo Si, and Huajun Chen. KnowPrompt: Knowledge-aware prompt-tuning with synergistic optimization for relation extraction. In Proceedings of the ACM Web Conference 2022, pp. 2778–2788. Association for Computing Machinery, 2022b. doi: 10.1145/3485447.3511998.DOI
  4. [4]Zhuo Chen, Zhao Zhang, Zixuan Li, Fei Wang, Yutao Zeng, Xiaolong Jin, and Yongjun Xu. Self-improvement programming for temporal knowledge graph question answering. In LREC/COLING, pp. 14579–14594. ELRA and ICCL, 2024a.arxiv.org/abs/2404.01720
  5. [5]Ziyang Chen, Xiang Zhao, Jinzhi Liao, Xinyi Li, and Evangelos Kanoulas. Temporal knowledge graph question answering via subgraph reasoning. Knowl. Based Syst., 251:109134, 2022c.DOI
  6. [6]Ziyang Chen, Jinzhi Liao, and Xiang Zhao. Multi-granularity temporal question answering over knowledge graphs. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 11378–11392. Association for Computational Linguistics, 2023. doi: 10.18653/v1/2023.acl-long.637. URL https://aclanthology.org/2023.acl-long.637/.DOI
  7. [7]Ziyang Chen, Dongfang Li, and Xiang Zhao. Temporal knowledge question answering via abstract reasoning induction. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, pp. 4872–4889, 2024b.arxiv.org/abs/2311.09149
  8. [8]Wengtao Ding, Hao Chen, Huayu Li, and Yuzhong Qu. Semantic framework based query generation for temporal question answering over knowledge graphs. In EMNLP, pp. 1867–1877. Association for Computational Linguistics, 2022.arxiv.org/abs/2210.04490
  9. [9]Lang Feng, Zhenghai Xue, Tingcong Liu, and Bo An. Group-in-group policy optimization for LLM agent training. In Advances in Neural Information Processing Systems, volume 38, 2025. doi: 10.52202/085713-1544. URL https://proceedings.neurips.cc/paper_files/paper/2025/hash/420c9f777c0b4f78d515e53cf74d58b2-Abstract-Conference.html.DOI
  10. [10]Zhaoyan Gong, Juan Li, Zhiqiang Liu, Lei Liang, Huajun Chen, and Wen Zhang. RTQA : Recursive thinking for complex temporal knowledge graph question answering with large language models. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (eds.), Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 9853–9870, Suzhou, China, November 2025. Association for Computational Linguistics. ISBN 979-8-89176-332-6. doi: 10.18653/v1/2025.emnlp-main.499. URL https://aclanthology.org/2025.emnlp-main.499/.
  11. [11]Zhaoyan Gong, Zhiqiang Liu, Songze Li, Xiaoke Guo, Yuanxiang Liu, Xinle Deng, Zhizhen Liu, Lei Liang, Huajun Chen, and Wen Zhang. Temp-R1: A unified autonomous agent for complex temporal KGQA via reverse curriculum reinforcement learning, 2026. URL https://arxiv.org/abs/2601.18296.
  12. [12]Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, et al. Deepseek-r1 incentivizes reasoning in llms through reinforcement learning. Nature, 645(8081):633–638, 2025.DOI
  13. [13]Qianyi Hu, Xinhui Tu, Cong Guo, and Shunping Zhang. Time-aware ReAct agent for temporal knowledge graph question answering. In Luis Chiruzzo, Alan Ritter, and Lu Wang (eds.), Findings of the Association for Computational Linguistics: NAACL 2025, pp. 6013–6024, Albuquerque, New Mexico, April 2025. Association for Computational Linguistics. ISBN 979-8-89176-195-7. doi: 10.18653/v1/2025.findings-naacl.334. URL https://aclanthology.org/2025.findings-naacl.334/.
  14. [14]Qianyi Hu, Jiaxue Liu, Xinhui Tu, and Shoujin Wang. SAR: A structure-aligned reasoning framework for temporal knowledge graph question answering. Proceedings of the AAAI Conference on Artificial Intelligence, 40(37):31086–31094, 2026. doi: 10.1609/aaai.v40i37.40369. URL https://ojs.aaai.org/index.php/AAAI/article/view/40369.DOI
  15. [15]Zicheng Huang, Yajuan Tong, Xinhui Tu, and Tingting He. TempTool-R1: Tool-augmented reinforcement learning for temporal knowledge graph question answering. In Findings of the Association for Computational Linguistics: ACL 2026, pp. 16224–16241, San Diego, California, United States, July 2026. Association for Computational Linguistics. doi: 10.18653/v1/2026.findings-acl.797. URL https://aclanthology.org/2026.findings-acl.797/.DOI
  16. [16]Zhen Jia, Abdalghani Abujabal, Rishiraj Saha Roy, Jannik Strötgen, and Gerhard Weikum. TEQUILA: temporal question answering over knowledge bases. In CIKM, pp. 1807–1810. ACM, 2018.DOI
  17. [17]Zhen Jia, Soumajit Pramanik, Rishiraj Saha Roy, and Gerhard Weikum. Complex temporal question answering on knowledge graphs. In CIKM, pp. 792–802. ACM, 2021.DOI
  18. [18]Zihao Jiang, Miao Peng, Zhenyan Shan, Wenjie Xu, Ben Liu, Gong Chen, Ziqi Gao, and Min Peng. TKG-Thinker: Towards dynamic reasoning over temporal knowledge graphs via agentic reinforcement learning, 2026. URL https://arxiv.org/abs/2602.05818.
  19. [19]Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. Search-R1: Training LLMs to reason and leverage search engines with reinforcement learning. arXiv preprint arXiv:2503.09516, 2025. doi: 10.48550/arXiv.2503.09516. URL https://arxiv.org/abs/2503.09516.

Paper details

vlnc:W_neashykz
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents