WorksComputer Science
BioMem: Agentic Memory for Biomedical LLM Agents
Abstract
Large language models (LLMs) have demonstrated considerable potential in biomedical applications, while agentic memory systems have emerged as a promising approach to preserving and utilizing information across tasks and interactions. However, a fundamental question remains: how effectively and reliably do these systems retain, retrieve, update, and utilize biomedical information? In this study, we systematically investigate agentic memory in the biomedical domain, focusing on task effectiveness, retrieval quality, information fidelity, and long-term robustness. We evaluate five memory approaches across four LLMs in three representative settings: biomedical knowledge reasoning, clinical prediction, and long-term conversational memory. Our analyses reveal how memory construction, evidence relevance, and conversational history shape downstream biomedical performance, highlighting the distinct challenges of retrieving relevant information, preserving its fidelity, and effectively utilizing it for reasoning. Building on these findings, we introduce Hybrid Memory Retrieval and Utilization (HMRU), a training-free framework that jointly enhances evidence accessibility and task-aware utilization to improve biomedical reasoning and long-term conversational memory. HMRU achieves an average relative improvement of 8.6% across biomedical question-answering and conversational memory benchmarks, with additional gains in selected clinical prediction tasks. It further reduces conversational answer errors by 24.9% and closes 48.0% of the performance gap to full-context inference. Our findings provide a systematic characterization of biomedical agentic memory and practical insights into developing more effective and reliable memory-augmented LLM systems.
Introduction
Large language models (LLMs) have demonstrated considerable potential in biomedical applications, including clinical question answering, medical information extraction, and decision support [1, 2, 3]. Advances in medical-domain pretraining and instruction tuning have further strengthened their ability to interpret biomedical terminology and respond to specialized clinical questions [4, 5, 6]. However, biomedical applications often depend on information that is patient-specific, evolving, or distributed across multiple sources, which cannot be fully captured by knowledge encoded in model parameters alone. These requirements motivate approaches that provide language models with access to external information beyond their parametric knowledge. Retrieval-augmented generation (RAG) addresses this need by incorporating retrieved evidence at inference time without updating the underlying model parameters [7].
Beyond accessing external evidence, biomedical agents must also manage information accumulated through repeated interactions. Patient histories, evolving observations, and previous exchanges may remain relevant to subsequent tasks, requiring information to be preserved and updated over time. Persistent memory systems have emerged to address this need by organizing and reusing information across interactions [8, 9]. Recent approaches have further introduced specialized mechanisms for memory construction, retrieval, and updating. A-MEM [10], for example, constructs interconnected memory notes and updates contextual representations as new information is incorporated. Mem0 [9] extracts and consolidates salient conversational information for persistent, query-dependent retrieval, while LightMem [11] separates lightweight online processing from longer-term consolidation to improve memory efficiency. These approaches illustrate different strategies for storing, structuring, and updating information that may be useful across tasks and interactions. Meanwhile, benchmarks such as RealMem [12] have begun to examine memory capabilities involving temporal reasoning, information updates, and interactions spanning multiple sessions. However, the effectiveness of external memory systems depends on several interconnected processes. Information may be transformed during storage, selected or omitted during retrieval, revised through subsequent updates, and interpreted by the language model at inference time. This raises a fundamental question: how effectively and reliably do agentic memory systems preserve, retrieve, update, and utilize biomedical information?
In this study, we systematically investigate agentic memory in biomedical domain, examining four dimensions: task effectiveness, retrieval and utilization quality, information fidelity, and long-term robustness. We evaluate five memory strategies with four language models across three representative settings: biomedical knowledge reasoning, clinical prediction, and long-term conversational memory. The evaluation covers six biomedical tasks, including reasoning over medical textual knowledge graphs, mortality and readmission prediction, cancer classification using, and opioid overdose prediction. We additionally evaluate long-term conversational memory tasks, examining incremental information, dynamic updating, proactive alignment, temporal reasoning, and cross-session interactions. Beyond overall task performance, we study the effects of memory construction data volume, retrieval depth, and class distribution. We also trace retrieved evidence to original records, assess its relevance to the target question, analyze source-grounded fidelity during iterative memory evolution, and evaluate robustness to conversation length and evidence distance.
Our analyses reveal four key findings. First, successful biomedical knowledge reasoning depends not only on retrieving relevant evidence but also on its appropriate utilization, as answer-bearing memories do not consistently lead to correct predictions. Second, semantic similarity between patient records does not necessarily correspond to similarity in future clinical outcomes, highlighting task-dependent limitations of memory retrieval. Third, iterative memory evolution is associated with the emergence and persistence of source-unsupported information, raising concerns about the fidelity of stored representations. Finally, long-term conversational memory performance depends on more than history length alone, with evidence accessibility and integration emerging as important factors. Together, these findings identify evidence retrieval, information fidelity, and memory utilization as distinct but interconnected challenges in biomedical agentic memory. Motivated by these observations, we introduce Hybrid Memory Retrieval and Utilization (HMRU), a training-free framework designed to improve access to relevant evidence and its effective utilization in downstream biomedical tasks. HMRU consistently improves performance on biomedical knowledge reasoning benchmarks and enhances long-term conversational memory by reducing answering errors and narrowing the performance gap with full-context inference. Our main contributions are summarized as follows:
We systematically evaluate five memory methods using four LLMs across six biomedical tasks and long-term conversational memory settings, examining the effects of memory construction, retrieval depth, and class distribution on downstream performance.
We investigate memory retrieval, evidence utilization, and information fidelity, identifying key limitations in biomedical reasoning and memory evolution. We further examine the effects of evidence completeness, conversation length, and evidence distance on long-term memory performance.
We propose Hybrid Memory Retrieval and Utilization (HMRU), a training-free framework that enhances evidence retrieval and task-aware utilization, improving biomedical knowledge reasoning and long-term conversational memory performance.
Results
Memory Performance and Analysis on Biomedical Tasks
Table 1 presents the performance of five memory methods across six biomedical benchmarks using four language models. The evaluation covers two biomedical knowledge reasoning tasks (RiTeK-PharmKG and RiTeK-ADInt) and four clinical prediction and classification tasks (MIMIC-III Mortality, MIMIC-III Readmission, TCGA-Reports, and MIMIC-IV OOD). For knowledge reasoning tasks, we report precision, recall, and F1 scores using exact-match and ROUGE-1 measures. Clinical prediction tasks are evaluated using AUPRC, AUROC. We compare five memory management methods (A-MEM, MemZero, and LightMem, NaiveRAG and FullContext) to systematically examine how different memory mechanisms affect downstream biomedical task performance across model architectures and task settings. All results are reported as percentages.
| Method | RiTeK-PharmKG | RiTeK-ADInt | Mort.-MIMIC-III AUPRC | AUROC | Readm.-MIMIC-III AUPRC | AUROC | TCGA-Reports AUPRC | AUROC | MIMIC-IV AUPRC | AUROC | ||||||||||
| Qwen3-8B | ||||||||||||||||||||
| A-MEM | 26.97 | 34.88 | 28.53 | 29.25 | 37.76 | 31.00 | 37.59 | 48.86 | 40.17 | 41.94 | 55.81 | 45.24 | 9.61 | 56.76 | 59.25 | 53.61 | 83.08 | 97.88 | 4.15 | 84.21 |
| MemZero | 16.56 | 26.28 | 18.92 | 18.70 | 29.00 | 21.19 | 23.81 | 38.76 | 27.59 | 27.42 | 45.41 | 31.98 | 8.98 | 53.07 | 58.23 | 53.22 | 78.61 | 96.61 | 3.13 | 76.67 |
| FullContext | 14.81 | 20.10 | 15.83 | 18.06 | 24.80 | 19.51 | 21.19 | 28.98 | 22.87 | 27.08 | 38.55 | 29.84 | 9.13 | 54.93 | 54.37 | 47.87 | 20.87 | 65.51 | 4.33 | 76.31 |
| NaiveRAG | 28.34 | 35.08 | 29.57 | 30.84 | 37.75 | 32.16 | 41.10 | 48.20 | 42.74 | 45.94 | 54.56 | 48.15 | 10.22 | 59.60 | 57.79 | 53.75 | 81.29 | 97.79 | 5.76 | 81.64 |
| LightMem | 20.89 | 31.22 | 22.83 | 22.64 | 33.33 | 24.80 | 30.05 | 46.54 | 34.06 | 33.97 | 52.24 | 38.62 | 10.14 | 56.16 | 55.40 | 50.57 | 88.64 | 98.67 | 4.84 | 84.71 |
| Meditron3-Qwen2.5-14B | ||||||||||||||||||||
| A-MEM | 28.47 | 32.13 | 29.07 | 33.35 | 37.55 | 34.09 | 44.16 | 47.99 | 45.08 | 51.19 | 56.29 | 52.59 | 8.98 | 51.74 | 59.25 | 54.27 | 77.58 | 97.19 | 2.49 | 69.53 |
| MemZero | 21.36 | 24.00 | 21.87 | 27.93 | 31.17 | 28.56 | 30.69 | 33.82 | 31.51 | 38.83 | 43.86 | 40.18 | 13.53 | 60.22 | 56.70 | 50.61 | 86.15 | 98.29 | 2.20 | 72.13 |
| FullContext | 19.77 | 21.77 | 19.90 | 23.32 | 26.24 | 23.75 | 28.82 | 31.19 | 29.22 | 37.77 | 41.61 | 38.61 | 7.22 | 48.60 | 54.65 | 49.25 | 24.13 | 73.13 | 2.86 | 70.60 |
| NaiveRAG | 30.69 | 34.15 | 31.23 | 35.47 | 39.15 | 36.01 | 44.23 | 47.48 | 44.90 | 51.91 | 55.82 | 52.89 | 8.55 | 55.75 | 57.55 | 53.65 | 76.25 | 97.39 | 3.58 | 73.69 |
| LightMem | 26.52 | 28.92 | 26.93 | 31.50 | 34.17 | 32.00 | 38.12 | 41.67 | 38.94 | 45.06 | 50.01 | 46.42 | 10.51 | 56.18 | 54.98 | 48.42 | 89.05 | 98.68 | 2.64 | 68.07 |
| HuatuoGPT-3-27B | ||||||||||||||||||||
| A-MEM | 39.95 | 39.23 | 39.27 | 42.18 | 41.40 | 41.51 | 53.32 | 53.08 | 52.94 | 58.84 | 58.62 | 58.53 | 9.81 | 59.88 | 61.91 | 56.69 | 99.06 | 99.93 | 4.94 | 82.16 |
| MemZero | 31.22 | 31.07 | 30.84 | 33.96 | 33.76 | 33.57 | 43.57 | 43.43 | 43.31 | 50.05 | 49.99 | 49.84 | 8.29 | 54.21 | 57.98 | 52.71 | 98.64 | 99.87 | 6.95 | 78.94 |
| FullContext | 27.55 | 26.65 | 26.80 | 30.25 | 29.51 | 29.64 | 37.34 | 36.58 | 36.83 | 45.30 | 44.68 | 44.90 | 9.68 | 59.19 | 57.93 | 52.55 | 98.88 | 99.88 | 8.23 | 82.76 |
| NaiveRAG | 39.42 | 39.33 | 39.04 | 41.48 | 41.36 | 41.07 | 52.07 | 51.63 | 51.63 | 57.69 | 57.34 | 57.36 | 9.81 | 61.98 | 60.76 | 55.79 | 98.97 | 99.93 | 15.61 | 85.47 |
| LightMem | 37.05 | 36.84 | 36.62 | 39.25 | 38.89 | 38.78 | 50.21 | 49.76 | 49.76 | 55.06 | 54.88 | 54.77 | 9.62 | 57.80 | 59.17 | 53.91 | 98.47 | 99.81 | 3.42 | 79.11 |
| Qwen3-30B-A3B-Instruct-2507 | ||||||||||||||||||||
| A-MEM | 31.79 | 36.23 | 33.00 | 33.95 | 38.64 | 35.28 | 44.35 | 50.80 | 46.20 | 50.69 | 58.69 | 53.18 | 9.37 | 51.17 | 55.01 | 49.76 | 98.39 | 99.81 | 4.13 | 81.66 |
| MemZero | 23.76 | 29.20 | 25.29 | 25.26 | 31.31 | 27.01 | 28.75 | 35.79 | 30.80 | 34.82 | 44.95 | 37.86 | 7.54 | 51.35 | 57.13 | 52.33 | 96.76 | 99.75 | 2.67 | 74.60 |
| FullContext | 24.11 | 26.16 | 24.42 | 26.53 | 29.12 | 27.07 | 29.96 | 34.09 | 31.04 | 37.46 | 44.43 | 39.57 | 10.27 | 52.90 | 55.50 | 49.78 | 95.41 | 99.43 | 3.52 | 73.96 |
| LightMem | 29.67 | 33.77 | 30.73 | 31.31 | 35.57 | 32.47 | 39.64 | 45.50 | 41.16 | 45.29 | 52.65 | 47.49 | 8.88 | 52.19 | 54.76 | 49.76 | 97.76 | 99.81 | 3.25 | 77.61 |
| NaiveRAG | 35.24 | 40.29 | 36.53 | 36.87 | 41.87 | 38.18 | 46.14 | 50.17 | 47.29 | 52.68 | 57.93 | 54.26 | 9.02 | 56.57 | 57.89 | 54.70 | 98.36 | 99.90 | 3.92 | 78.59 |
Table 1. Comparative evaluation of agentic memory approaches on biomedical benchmarks. Bold and underlined values indicate the best and second-best distinct results within each language model, respectively. For RiTeK datasets, * denotes exact match and † denotes ROUGE-1.
Comparisons with memory baselines
We evaluated five memory approaches across six biomedical benchmarks using four language models (Table 1). Performance varied substantially across tasks, memory strategies, and model backbones. In biomedical knowledge reasoning, A-MEM and NaiveRAG generally achieved stronger performance than MemZero, LightMem, and FullContext. With HuatuoGPT-3-27B, A-MEM achieved exact-match F1 scores of 39.27 and 52.94 on RiTeK-PharmKG and RiTeK-ADInt, respectively, compared with 39.04 and 51.63 for NaiveRAG and 26.80 and 36.83 for FullContext. However, this advantage was not consistent across model backbones. With Qwen3-30B, NaiveRAG outperformed A-MEM on both datasets, achieving F1 scores of 36.53 versus 33.00 on PharmKG and 47.29 versus 46.20 on ADInt. Clinical prediction and pathology classification exhibited different performance patterns. On TCGA-Reports, LightMem achieved the highest
AUPRC of 88.64 with Qwen3-8B and 89.05 with Meditron3-Qwen2.5-14B, whereas A-MEM achieved an AUPRC of 99.06 and an AUROC of 99.93 with HuatuoGPT-3-27B. For MIMIC-III Mortality, MemZero achieved the highest AUPRC of 13.53 with Meditron3-Qwen2.5-14B, outperforming A-MEM and NaiveRAG, which achieved AUPRC values of 8.98 and 8.55, respectively. In contrast, NaiveRAG showed a pronounced advantage on MIMIC-IV opioid overdose prediction with HuatuoGPT-3-27B, achieving an AUPRC of 15.61 and an AUROC of 85.47, compared with an AUPRC of 4.94 and an AUROC of 82.16 for A-MEM. On MIMIC-III Readmission, AUPRC values were relatively high despite AUROC values remaining close to chance level across several configurations, indicating limited discrimination and emphasizing the importance of interpreting precision–recall performance in relation to outcome prevalence.
Factors Influencing Memory Performance
To better understand the factors influencing memory performance in biomedical tasks, we systematically investigate three key aspects: training data volume, retrieval depth, and class distribution. First, we examine how varying the number of training samples used for memory construction affects downstream performance. Second, we evaluate the impact of retrieval depth by varying the number of memory items retrieved during inference. Third, we investigate how different positive-to-negative class distributions in memory construction influence clinical prediction performance. Across these experiments, we aim to characterize the sensitivity of different memory methods to data availability, retrieval configurations, and class imbalance, providing insights into their effectiveness and robustness across biomedical tasks.
Effect of training data volume. Figure 1(a) illustrates the effects of memory construction data volume across six biomedical tasks, using 250, 500, and 1,000 samples. Increasing data volume generally improves performance on biomedical knowledge reasoning tasks, whereas clinical prediction and pathology classification exhibit more heterogeneous patterns. On RiTeK-PharmKG, A-MEM with HuatuoGPT-3-27B achieves exact-match F1 scores of 27.66, 33.23, and 39.27 as data volume increases, while corresponding scores on RiTeK-ADint rise from 37.83 to 43.08 and 52.94. Similar improvements are observed with Qwen3-30B, where A-MEM increases from 23.17 to 33.00 on RiTeK-PharmKG. In contrast, performance on clinical prediction tasks does not consistently improve with additional data. On MIMIC-IV OOD, NaiveRAG with HuatuoGPT-3-27B achieves AUPRC scores of 7.70, 5.22, and 15.61, whereas LightMem decreases from 5.21 to 4.28 and 3.42. Performance on TCGA-Reports remains comparatively stable across data volumes, particularly for configurations with already high AUPRC values.
Figure 1. Combined Sensitivity (test). a, Effect of training data volume. b, Effect of retrieval depth (Top-k) on biomedical memory performance. c, Effect of class distribution on biomedical memory performance.
Effect of retrieval depth. Figure 1(b) illustrates the effects of retrieval depth across six biomedical tasks, with the number of retrieved memory items varying from 1 to 10. Increasing retrieval depth generally improves performance on biomedical knowledge reasoning tasks, whereas clinical prediction and pathology classification exhibit more heterogeneous patterns. On RiTeK-PharmKG, A-MEM with Qwen3-30B achieves exact-match F1 scores of 27.33, 30.32, 31.81, and 33.00 at Top-1, Top-3, Top-5, and Top-10, respectively. Similar improvements are observed on RiTeK-ADint, where exact-match F1 increases from 36.71 to 46.20. In contrast, deeper retrieval does not consistently improve clinical prediction. On MIMIC-IV OOD, NaiveRAG with HuatuoGPT-3-27B achieves an AUPRC of 3.99 at Top-1 and 15.61 at Top-10, while other configurations show smaller gains or fluctuating performance. On MIMIC-III Readmission, LightMem reaches its highest AUPRC of 61.32 at Top-5 before declining to 59.17 at Top-10. Performance on TCGA-Reports remains relatively stable at higher retrieval depths, particularly for configurations with already high AUPRC values.
Effect of class distribution. Figure 1 (c) illustrates the effects of varying positive-to-negative class ratios in memory across two clinical prediction tasks. Changes in class distribution produce heterogeneous effects on predictive performance, with no single ratio consistently outperforming the natural distribution across memory methods and language models. On MIMIC-III Readmission, A-MEM with Qwen3-30B achieves an AUPRC of 57.92 under the 25:75 ratio, compared with 55.01 under the natural distribution. Similarly, LightMem with Qwen3-30B improves mortality AUPRC from 8.88 to 10.77 under the 10:90 ratio. However, increasing the proportion of positive samples does not consistently improve performance. With HuatuoGPT-3-27B, A-MEM achieves its highest readmission AUPRC of 61.91 under the natural distribution, declining to 60.37 under the 50:50 ratio. NaiveRAG also exhibits substantial performance reductions across several configurations, particularly for readmission prediction.
Memory Retrieval Quality and Utilization
To investigate how memory retrieval and utilization influence downstream biomedical performance, we systematically analyzed retrieved evidence and model predictions across six biomedical benchmarks (Figure 2a–Figure 2f). For biomedical question answering, we examined the factual relevance of retrieved sources, the association between answer-source availability and prediction accuracy, the extent to which model outputs matched retrieved source answers, and the relationship between evidence similarity and answering performance. For clinical prediction and classification, we assessed whether retrieved examples were enriched for the ground-truth class relative to its prevalence in the training data.
Figure 2. Memory retrieval quality and utilization across biomedical tasks. a, Evidence-to-outcome flows for NaiveRAG on the pooled RiTeK test questions. b, Distribution of the strongest retrieved evidence category across five memory methods. c, Answer accuracy stratified by whether retrieved memory contains an answer matching the ground truth. d, Proportions of predictions matching a retrieved source answer, separated by correct and incorrect predictions. e, Answer accuracy by BGE-M3 similarity between the test question and the strongest answer-matching source question; bins contain at least ten questions. f, Retrieval enrichment for the ground-truth class relative to its training-set prior in four classification tasks; error bars show 95 per cent confidence intervals.
Relationship between retrieved evidence and answer correctness. Figure 2a illustrates the relationship between retrieved evidence categories and answer correctness across the two RiTeK benchmarks using NaiveRAG. Retrieved evidence is categorized as containing the same fact, related information, unrelated information with a matching answer, or no answer-matching source. Among the 1,505 questions, 363 contain a same-fact source, 175 a related source, and 46 an unrelated source with a matching answer, accounting for 24%, 12%, and 3% of the sample, respectively. The remaining 921 questions, comprising 61% of the sample, lack an answer-matching source. Overall, NaiveRAG correctly answers 614 questions and incorrectly answers 891, yielding an accuracy of 41%. Correct answers are more strongly associated with factually relevant evidence, although errors persist even when answer-bearing sources are retrieved. Conversely, some questions are answered correctly without an identifiable answer-matching source. These findings highlight the relationship between evidence availability, factual relevance, and answering performance, while indicating that retrieving relevant evidence alone does not guarantee correct utilization.
Coverage of answer-bearing and factually relevant sources. Figure 2b compares the distribution of retrieved evidence categories across memory methods and RiTeK datasets. For each method and dataset, the proportion assigned to evidence category is defined as , where , where is the highest evidence category assigned to test question . On RiTeK-ADInt, A-MEM, LightMem, and NaiveRAG retrieve a same-fact source for approximately 35 per cent of questions, compared with approximately 30 per cent for MemZero and 15 per cent for FullContext. On RiTeK-PharmKG, the corresponding values decrease to approximately 18 per cent for the first three systems, 13 per cent for MemZero, and 5 per cent for FullContext. The larger no-answer-source segments on PharmKG indicate weaker answer-source coverage. Importantly, more elaborate memory construction does not exhibit a clear coverage advantage over NaiveRAG in these traces.
Accuracy conditional on answer-source retrieval. Figure 2c compares answer accuracy across memory methods according to whether an answer-matching source is retrieved. For , conditional accuracy is defined as
where indicates whether an answer-matching source is retrieved for question . Solid and hatched bars represent accuracy with and without an answer-matching source, respectively. Across the four retrieval-based methods, accuracy ranges from 77% to 88% when an answer-matching source is available, compared with only 12% to 16% when no such source is retrieved. On RiTeK-ADInt, A-MEM achieves an accuracy of 88% with an answer-matching source, compared with 16% without one. Similarly, NaiveRAG achieves 88% and 15%, respectively. FullContext exhibits a smaller separation, with conditional accuracies of 75% and 23% on RiTeK-ADInt, and 59% and 20% on RiTeK-PharmKG. Although accuracy is substantially higher when answer-matching evidence is available, incorrect predictions remain even when such evidence is retrieved. These findings demonstrate a strong association between answer-source availability and prediction accuracy, while indicating that successful retrieval does not guarantee correct evidence utilization.
Correct and incorrect matching of retrieved source answers. Figure 2d examines the extent to which model predictions match answers contained in retrieved memory sources. For each test question, we define , where indicates that the predicted answer matches at least one retrieved source answer. We further distinguish correct and incorrect predictions using the proportion , calculated over all test questions. Across the four retrieval-based methods, incorrect source-answer matches account for approximately one-fifth of questions on RiTeK-ADInt and one-quarter on RiTeK-PharmKG. In contrast, the corresponding proportions for FullContext are 9% and 11%, respectively. These findings indicate that incorrect predictions frequently coincide with answers present in retrieved memory, suggesting potential interference from irrelevant or mismatched evidence.
Evidence similarity and answer accuracy. Figure 2e examines the association between source-question similarity and answer accuracy. For questions with at least one answer-matching source (), we calculated the maximum cosine similarity , where denotes the retrieved source records whose answers match the reference answer, is the BGE-M3 text encoder, and denotes cosine similarity. Questions from both RiTeK datasets were pooled and stratified by similarity, excluding bins containing fewer than ten observations. Answer accuracy increased progressively with source-question similarity, approaching 100 per cent among questions with the highest similarity scores. This relationship was consistent across all five memory methods, with largely overlapping accuracy trajectories. In contrast, questions without an answer-matching source exhibited substantially lower accuracy.
Retrieval enrichment for classification outcomes. Figure 2f examines whether retrieved training examples are enriched for the ground-truth class of each test instance relative to its prevalence in the training data. For each test instance , we calculate the fraction of retrieved examples sharing its label as , where denotes the number of retrieved source labels. For a group of test instances , retrieval lift is defined as , where is the training-set prevalence of the ground-truth class. A lift of one corresponds to the expected same-label retrieval rate under random sampling from the training distribution. On TCGA-Reports, the four retrieval-based methods exhibited substantial enrichment of same-class pathology reports, with retrieval lift exceeding 15, whereas FullContext remained close to the class-prior baseline. In contrast, enrichment was considerably weaker and more heterogeneous across clinical outcome prediction tasks. On MIMIC-III Mortality, MemZero achieved a lift of 2.6 for positive cases, while the remaining methods showed limited enrichment. For MIMIC-III Readmission, retrieval lift for the minority negative class was generally below one, indicating that retrieved examples did not consistently reflect the test instances’ outcome labels. Estimates for MIMIC-IV opioid overdose prediction were highly uncertain because only nine positive test instances were available.
Memory evolution and fidelity.
We further investigated information fidelity during A-MEM construction and evolution across six biomedical tasks (Figure 3). Using an LLM-based evaluator, we assessed whether generated memory contexts contained information unsupported by their original training records. During sequential memory construction, the proportion of memories containing unsupported context increased in several tasks, reaching 44% in MIMIC-IV OOD and 35% in RiTeK-PharmKG, while remaining below 2% in both MIMIC-III tasks (Figure 3a). Higher rewrite frequency was associated with a greater prevalence of unsupported information, particularly in MIMIC-IV OOD, where the proportion increased from approximately 10% among memories without rewrites to 80% among those rewritten at least five times (Figure 3b). Comparisons between initial memory contexts and their final states after evolution revealed substantial task-dependent differences (Figure 3c). In MIMIC-IV OOD, the proportion of memories containing unsupported context increased from 12% to 44%, representing a 32-percentage-point increase. RiTeK-PharmKG showed a similar pattern, increasing from 16% to 35%. In contrast, smaller changes were observed in RiTeK-ADInt and TCGA-Reports, while both MIMIC-III tasks remained close to zero throughout memory evolution. Longitudinal inspection of the ten most frequently rewritten memories in MIMIC-IV OOD further revealed extended sequences of unsupported context, although some memories alternated between supported and unsupported states across successive revisions (Figure 3d).
Figure 3. Memory evolution in A-MEM. a, Proportion of memory nodes with unsupported context during sequential memory construction across six biomedical tasks. b, Association between context rewrite frequency and unsupported context rate, with 95 per cent Wilson confidence intervals. c, Proportion of memory notes containing source-unsupported context in their initial and final states after memory evolution. d, Evolution trajectories of the ten most frequently rewritten nodes in MIMIC-IV OOD. Gray and red indicate supported and unsupported context, respectively. Unsupported content is determined by an LLM evaluator against the original training record.
Biomedical Conversational Memory
Overall Performance of Biomedical Conversational Memory
Table 2 presents the performance of five memory methods across four language models on the RealMem medical subset, evaluating long-term conversational memory across four memory categories and two session types. Overall, FullContext achieves the strongest performance in most settings, particularly in dynamic updating and multi-session interactions. With Qwen3-30B, FullContext achieves QA and memory recall scores of 88.90 and 92.30 on dynamic updating, compared with 73.70 and 74.80 for A-MEM and 69.70 and 66.40 for NaiveRAG. This advantage is also observed with HuatuoGPT-3-27B, where FullContext achieves 97.98 in QA and 98.23 in recall. Nevertheless, structured memory methods demonstrate selective advantages. In proactive alignment, A-MEM with Qwen3-30B achieves QA and recall scores of 78.90 and 82.00, outperforming FullContext by 4.90 and 8.00 points, respectively. LightMem also achieves a higher proactive alignment QA score than FullContext with Qwen3-8B, despite substantially lower memory recall. Notably, A-MEM exhibits reduced performance across multi-session interactions, with its QA score decreasing from 81.20 in single-session questions to 72.30 in multi-session questions under Qwen3-30B, whereas FullContext maintains comparable performance across both settings.
| Method | Dyn. Inc. QA | Dyn. Inc. Rec. | Dyn. Upd. QA | Dyn. Upd. Rec. | Proac. Align. QA | Proac. Align. Rec. | Temp. Reas. QA | Temp. Reas. Rec. | Single Sess. QA | Single Sess. Rec. | Multi Sess. QA | Multi Sess. Rec. |
| Qwen3-8B | ||||||||||||
| A-MEM | <u>74.60</u> | <u>76.60</u> | 68.70 | <u>65.80</u> | <u>71.50</u> | <u>68.20</u> | 58.30 | <u>62.50</u> | <u>76.50</u> | <u>74.60</u> | 68.50 | <u>73.60</u> |
| MemZero | 58.00 | 39.70 | 55.60 | 30.60 | 65.90 | 34.30 | 58.30 | 50.00 | 59.70 | 37.20 | 57.30 | 39.90 |
| FullContext | 78.70 | 82.30 | 85.90 | 89.10 | 70.70 | 72.80 | 100.00 | 100.00 | 77.70 | 81.30 | 80.30 | 83.10 |
| NaiveRAG | 71.30 | 70.60 | <u>69.70</u> | 63.10 | <u>71.50</u> | 61.40 | 50.00 | 50.00 | 71.70 | 68.10 | <u>69.60</u> | 69.00 |
| LightMem | 61.30 | 42.40 | 59.60 | 41.90 | 76.40 | 47.90 | <u>66.70</u> | 25.00 | 62.50 | 39.90 | 64.00 | 47.40 |
| Meditron3-Qwen2.5-14B | ||||||||||||
| A-MEM | 38.70 | 45.90 | 36.40 | 40.80 | 49.60 | <u>60.10</u> | <u>66.70</u> | <u>62.50</u> | 41.30 | 44.70 | 38.40 | 51.60 |
| MemZero | 43.20 | 28.60 | 47.50 | 23.50 | 52.00 | 36.90 | 25.00 | 25.00 | 43.00 | 24.70 | 46.90 | 36.10 |
| FullContext | 61.90 | 65.80 | 78.80 | 84.10 | 63.40 | 64.30 | 83.30 | 100.00 | 64.00 | 66.50 | 64.30 | 70.10 |
| NaiveRAG | 50.10 | <u>54.20</u> | 39.40 | <u>41.80</u> | 50.40 | 56.20 | 41.70 | 41.70 | 49.70 | <u>51.60</u> | 47.70 | <u>55.20</u> |
| LightMem | <u>50.70</u> | 39.70 | <u>50.50</u> | 30.40 | <u>54.50</u> | 40.20 | 58.30 | <u>62.50</u> | <u>51.80</u> | 37.30 | <u>50.40</u> | 42.00 |
| HuatuoGPT-3-27B | ||||||||||||
| A-MEM | <u>91.90</u> | <u>90.30</u> | <u>91.90</u> | <u>86.00</u> | <u>95.90</u> | <u>90.20</u> | <u>91.70</u> | 62.50 | <u>93.80</u> | <u>92.20</u> | <u>90.10</u> | <u>85.20</u> |
| MemZero | 83.10 | 58.30 | 85.90 | 53.90 | 91.10 | 59.30 | 100.00 | 66.70 | 84.50 | 58.60 | 84.80 | 57.40 |
| FullContext | 95.68 | 95.98 | 97.98 | 98.23 | 95.93 | 95.99 | 91.67 | <u>87.50</u> | 94.67 | 96.88 | 97.87 | 94.88 |
| NaiveRAG | 84.08 | 81.78 | 81.82 | 79.08 | 88.62 | 83.04 | 75.00 | 100.00 | 85.00 | 82.00 | 83.20 | 81.71 |
| LightMem | 81.40 | 57.20 | 84.80 | 50.10 | 93.50 | 51.60 | 66.70 | 50.00 | 83.00 | 55.30 | 83.20 | 56.20 |
| Qwen3-30B-A3B-Instruct-2507 | ||||||||||||
| A-MEM | <u>77.90</u> | <u>81.40</u> | <u>73.70</u> | <u>74.80</u> | 78.90 | 82.00 | <u>91.70</u> | 62.50 | 81.20 | 83.90 | 72.30 | <u>75.10</u> |
| MemZero | 64.10 | 41.90 | 64.60 | 28.00 | 71.50 | 36.60 | 41.70 | 50.00 | 65.00 | 37.50 | 64.50 | 43.90 |
| FullContext | 80.30 | 84.10 | 88.90 | 92.30 | 74.00 | 74.00 | 100.00 | 100.00 | <u>80.00</u> | <u>83.40</u> | 81.60 | 84.60 |
| NaiveRAG | 76.10 | 72.40 | 69.70 | 66.40 | <u>78.00</u> | 67.90 | 50.00 | 50.00 | 77.00 | 72.40 | <u>72.80</u> | 68.60 |
| LightMem | 65.20 | 45.20 | 67.70 | 36.20 | <u>78.00</u> | 39.80 | 58.30 | <u>66.70</u> | 68.00 | 42.80 | 65.30 | 45.50 |
Table 2. Memory performance on the RealMem medical subset. Performance of five memory methods across four language-model backbones, four memory categories, and two session types. QA and Rec. denote QA Score and Memory Recall, respectively. Scores are percentages. Bold and underlined values indicate the best and second-best distinct scores within each backbone and metric. Dyn. Inc., dynamic incremental; Dyn. Upd., dynamic updating; Proac. Align., proactive alignment; Temp. Reas., temporal reasoning.
Effects of Conversation Length and Memory Distance on Memory Performance
We first investigated the sensitivity of long-term conversational memory to increasing history length using 100 questions from the RealMem medical subset (Figure 4a). Three experimental conditions were constructed by selectively removing irrelevant sessions preceding the earliest evidence-bearing session while preserving all required evidence and subsequent conversational content. This yielded Short, Medium, and Long histories with median lengths of 23,000, 36,000, and 48,000 tokens, respectively. Memory was independently reconstructed for each condition to evaluate the effects of history length on both retrieval and response generation. Contrary to the expectation that longer histories would impair memory performance, increasing conversation length produced no consistent degradation across either memory system. With Qwen3-30B-A3B-Instruct-2507, A-MEM exhibited a modest increase in QA performance from 78.3 to 81.7 and 83.0 across the three conditions, accompanied by improvements in memory recall from 85.2 to 87.4 and 89.1. In contrast, LightMem maintained relatively stable QA scores of 74.0, 74.7, and 73.0, while memory recall remained substantially lower, ranging from 53.6 to 55.2. These observations suggest that, within the evaluated context lengths, additional irrelevant conversational history preceding the evidence does not necessarily compromise memory retrieval or downstream answering. The contrasting recall levels further indicate that differences in memory construction and retrieval mechanisms may be more influential than history length alone.
Figure 4. Conversational memory robustness. a, Effects of conversation history length on QA performance and memory recall under Short, Medium, and Long conditions. b, Relationship between evidence distance and memory performance, measured by the number of tokens separating the last relevant evidence from the question. c, QA performance conditioned on whether all required evidence was retrieved, comparing A-MEM and LightMem. Filled and open circles indicate complete and incomplete evidence retrieval, respectively.
Effects of Evidence Distance on Memory Retrieval and Utilization
We examined whether the distance between relevant evidence and the current question affects memory performance (Figure 4b). Evidence distance was manipulated by repositioning irrelevant sessions while preserving the complete session set and total context length. Across five distance bins spanning up to 42,000 tokens, A-MEM maintained QA scores above 81% at distances up to 26,000 tokens, declining to 76.7% at the greatest distance. Memory recall showed a similar decline, whereas LightMem exhibited no consistent distance-dependent trend. Log-distance regression indicated modest negative associations with QA performance for both methods. We further examined the relationship between evidence retrieval completeness and answering performance (Figure 4c). Across 600 question-condition observations, A-MEM retrieved all required evidence in 82% of cases, achieving a QA score of 89.7 compared with 43.2 when retrieval was incomplete. LightMem achieved complete retrieval in 41% of cases, with corresponding QA scores of 81.8 and 66.2. These results indicate a stronger association between evidence completeness and answering performance for A-MEM than for LightMem. These findings suggest that conversation length and evidence distance do not consistently impair memory performance within the evaluated ranges. In contrast, complete evidence retrieval was strongly associated with successful answering, highlighting the importance of distinguishing memory retrieval quality from downstream evidence utilization.
Association Between Conversation History Length and Memory Performance
We examined the relationship between conversation history length and question-answering performance across all RealMem-Med test questions (Figure 5c). Questions were divided into five approximately equal-sized groups according to the number of historical conversation sessions, ranging from 2 to 238 sessions. A-MEM showed a decline in perfect-answer accuracy from approximately 95% in the shortest-history group to 74% in the longest-history group. MemZero exhibited a similar downward trend, whereas LightMem and NaiveRAG maintained lower accuracy with more variable patterns. In contrast, FullContext consistently achieved high accuracy across history-length groups. Regression analysis further identified significant negative associations between history length and answering accuracy for A-MEM (adjusted odds ratio , ) and MemZero (, ). No statistically significant associations were observed for LightMem, NaiveRAG, or FullContext. These results suggest that longer conversation histories are associated with reduced answering performance for certain memory systems. However, because the analysis compares different questions with naturally varying history lengths, the observed relationships may also reflect differences in question complexity and information requirements rather than a direct effect of history length.
Figure 5. a, Paired comparison of perfect-answer outcomes between four memory methods and FullContext (exact McNemar test; ***$P < 0.001). b, Perfect-answer rates conditional on retrieving none, some, or all gold-standard memories. c, Perfect-answer rates across five conversation history-length groups, with adjusted odds ratios for history length. Error bars indicate 95% confidence intervals.
Evaluating Retrieval and Answering in Long-Term Conversational Memory
We further evaluated the effectiveness of long-term conversational memory using 325 questions from the medical subset of RealMem. We compared four memory-based approaches, A-MEM, MemZero, LightMem, and NaiveRAG, against FullContext, which provides the complete conversation history to the language model. Beyond overall answer accuracy, we examined three complementary aspects of memory performance: question-level differences relative to FullContext, the relationship between evidence retrieval completeness and answer correctness, and the association between conversation history length and answering performance (Figure 5).
Performance gap relative to full conversation history. FullContext consistently outperformed all four memory-based approaches (Figure 5a). In paired question-level comparisons, FullContext answered 55 questions correctly that A-MEM failed to answer, whereas A-MEM uniquely answered 22 questions correctly, corresponding to a 10-percentage-point difference in overall perfect-answer rate. Larger gaps were observed for MemZero, LightMem, and NaiveRAG, with differences of 32, 37, and 39 percentage points, respectively. All four paired comparisons were significant under exact McNemar tests, with . These results indicate that existing memory-based approaches do not fully preserve the answering performance achievable when the complete conversation history remains accessible.
Evidence retrieval completeness and answer correctness. To examine the relationship between memory retrieval and downstream answering, we stratified questions according to whether none, some, or all gold-standard memories were retrieved (Figure 5b). Across memory methods, perfect-answer rates generally increased with retrieval completeness, although substantial differences remained in how effectively retrieved evidence was utilized. A-MEM showed a pronounced increase in accuracy, from 5% when no gold-standard memories were retrieved to 57% with partial retrieval and 89% with complete retrieval. Similar patterns were observed for MemZero and LightMem, which achieved perfect-answer rates of 78% and 76%, respectively, when all required memories were retrieved. In contrast, NaiveRAG achieved only 70% despite complete retrieval, highlighting limitations in downstream evidence utilization. FullContext achieved higher accuracy, reaching 94% when all required evidence was available. Notably, MemZero and LightMem correctly answered 33% and 27% of questions, respectively, even when no gold-standard memories were retrieved. Together, these findings demonstrate that retrieval completeness is strongly associated with answering performance, but access to relevant evidence alone does not guarantee correct utilization. Because the retrieval categories contain different questions, these associations do not establish a causal effect of retrieval completeness on answer accuracy.
Association between history length and memory performance. We additionally examined performance across five groups defined by the number of historical conversation sessions, ranging from 2 to 238 sessions (Figure 5c). A-MEM achieved a perfect-answer rate of approximately 95 per cent among questions with the shortest histories, compared with 67–74 per cent among questions with longer histories. MemZero, LightMem, and NaiveRAG exhibited similar descriptive declines, whereas FullContext maintained comparatively stable performance across history-length groups. After adjusting for question category, evidence count, and session type, history length remained negatively associated with perfect-answer outcomes for A-MEM and MemZero, with adjusted odds ratios of 0.42 and 0.68, respectively. No statistically significant associations were observed for LightMem, NaiveRAG, or FullContext.
Performance Evaluation of HMRU
Improving Biomedical Memory Retrieval and Utilization
We evaluated the proposed HMRU framework across six biomedical tasks (Table 3). We selected A-MEM as the baseline memory method and HuatuoGPT-3-27B as the frozen language model backbone, enabling a controlled comparison of retrieval and memory utilization strategies without modifying the underlying model parameters. Among the evaluated configurations, HMRU-I, which integrates hybrid retrieval, cross-encoder reranking, and task-aware memory utilization instructions, achieved the strongest overall performance, attaining the highest score in 15 of 24 evaluation metrics, including two ties. The largest improvements were observed in biomedical question answering. Compared with the A-MEM baseline, HMRU-I increased exact-match F1 from 39.27 to 43.48 on RiTeK-PharmKG and from 52.94 to 57.64 on RiTeK-ADint, corresponding to gains of 4.21 and 4.70 percentage points, respectively. HMRU-I also outperformed retrieval enhancement alone, increasing exact-match F1 from 42.95 to 43.48 on PharmKG and from 54.60 to 57.64 on ADint. These results suggest that task-aware memory utilization provides complementary benefits beyond hybrid retrieval and reranking. The effects were more heterogeneous across clinical prediction tasks. HMRU-C, which additionally incorporates compact memory representations, achieved the highest mortality prediction AUPRC of 10.03 and AUROC of 61.87, whereas HMRU-I obtained the highest readmission AUPRC of 62.17. In contrast, neither configuration consistently improved opioid overdose prediction, for which the A-MEM baseline retained the highest AUPRC of 4.94. Collectively, these findings demonstrate that hybrid retrieval and task-aware memory utilization improve biomedical question-answering performance, whereas their effectiveness varies across clinical prediction tasks.
| Method | RiTeK-PharmKG | RiTeK-PharmKG | RiTeK-PharmKG | RiTeK-PharmKG | RiTeK-PharmKG | RiTeK-PharmKG | RiTeK-ADint | RiTeK-ADint | RiTeK-ADint | RiTeK-ADint | RiTeK-ADint | RiTeK-ADint | Mort.-MIMIC-III AUPRC | Mort.-MIMIC-III AUROC | Readm.-MIMIC-III AUPRC | Readm.-MIMIC-III AUROC | TCGA-Reports AUPRC | TCGA-Reports AUROC | MIMIC-IV OOD AUPRC | MIMIC-IV OOD AUROC |
| Base method | 39.95 | 39.23 | 39.27 | 42.18 | 41.40 | 41.51 | 53.32 | 53.08 | 52.94 | 58.84 | 58.62 | 58.53 | 9.81 | 59.88 | 61.91 | 56.69 | 99.06 | 99.93 | 4.94 | 82.16 |
| HMRU-R | 43.47 | 43.04 | 42.95 | 45.53 | 45.05 | 45.01 | 55.08 | 54.53 | 54.60 | 60.28 | 59.86 | 59.90 | 9.66 | 60.43 | 61.62 | 57.68 | 98.84 | 99.94 | 4.28 | 84.08 |
| HMRU-I | 44.31 | 43.29 | 43.48 | 46.01 | 45.02 | 45.23 | 58.30 | 57.33 | 57.64 | 63.34 | 62.57 | 62.81 | 9.67 | 61.16 | 62.17 | 57.89 | 98.84 | 99.94 | 4.17 | 78.43 |
| HMRU-C | 44.17 | 43.26 | 43.35 | 45.92 | 45.05 | 45.16 | 56.54 | 55.67 | 55.91 | 62.00 | 61.29 | 61.50 | 10.03 | 61.87 | 61.72 | 57.92 | 98.84 | 99.94 | 3.91 | 76.96 |
Table 3. Comparison of HMRU variants and baseline memory performance across six biomedical tasks. All scores are percentages. Bold and underlined values indicate the best and second-best distinct scores for each metric, respectively.
Mechanistic Analysis of HMRU Improvements.
To investigate the sources of performance gains, we analyzed evidence availability and utilization across successive HMRU components on RiTeK-PharmKG and RiTeK-ADint (Figure 6). Hybrid retrieval increased the proportion of questions with answer-bearing memories, reducing the absence of answer sources from 65.8 to 61.8 per cent on PharmKG and from 51.7 to 50.0 per cent on ADint (Figure 6a). However, the contributions of retrieval enhancement and task-aware utilization differed between tasks (Figure 6b). On PharmKG, BM25 augmentation reduced retrieval failures from 55.3 to 50.0 per cent and increased exact-match F1 from 39.3 to 43.0. In contrast, on ADInt, the largest incremental improvement followed the introduction of the Task-Aware Memory Utilization Adapter (TAMUA), which increased exact-match F1 from 54.6 to 57.6 while reducing utilization failures from 4.5 to 2.8 per cent. Further analysis showed that TAMUA reduced the combined number of utilization failures across both tasks from 99 to 76, eliminating observed entity rephrasing and inappropriate yes/no responses while reducing errors involving multiple entities from 14 to 7 (Figure 6c). Nevertheless, incorrect selection of answers from competing memories remained a frequent source of error. Together, these findings suggest that hybrid retrieval and task-aware memory utilization address complementary limitations, with their relative contributions depending on the characteristics of the biomedical question-answering task.
Figure 6. Mechanistic analysis of Hybrid Memory Retrieval and Utilization on biomedical question answering. a, Distribution of retrieved evidence categories before and after BM25 augmentation. b, Decomposition of answer performance and failure modes across successive retrieval and utilization enhancements. c, Distribution of evidence utilization errors before and after task-aware adaptation. Results are reported for RiTeK-PharmKG and RiTeK-ADInt using HuatuoGPT-3-27B.
Improving Long-Term Conversational Memory Utilization
We further evaluated HMRU on the RealMem medical subset using A-MEM as the baseline memory system and HuatuoGPT-3-27B as the frozen language model (Table 4). We investigated three configurations: HMRU-R, which combines hybrid retrieval with cross-encoder reranking; HMRU-I, which additionally incorporates task-aware instructions for conversational continuity, preference preservation, and temporal consistency; and HMRU-T, which reorganizes the same retrieved memories into chronological conversation sequences and introduces structured state tracking of the ongoing dialogue, user profile, and temporal updates. HMRU-R achieved the strongest overall performance, improving QA scores from 91.90 to 93.25 on Dynamic Incremental, from 91.92 to 94.95 on Dynamic Updating, and from 90.13 to 93.33 on multi-session questions. Corresponding memory recall also improved, including an increase from 86.01 to 91.31 on Dynamic Updating. Adding task-aware utilization instructions yielded further improvements in specific memory capabilities. HMRU-I increased Proactive Alignment recall from 90.16 to 96.86 and multi-session recall from 85.18 to 90.39, exceeding both the baseline and HMRU-R. However, chronological memory organization and explicit state tracking did not provide consistent additional benefits. HMRU-T showed substantial reductions in Dynamic Updating QA, from 94.95 under HMRU-R to 79.80, and Temporal Reasoning QA, from 91.67 to 66.67. These findings suggest that hybrid retrieval provides broad improvements in conversational memory performance, whereas task-aware utilization instructions offer selective benefits for preference-sensitive and multi-session tasks. More extensive temporal restructuring does not necessarily improve memory utilization, highlighting the importance of preserving retrieval relevance alongside conversational context.
| Method | Dyn. Inc. QA | Dyn. Inc. Rec. | Dyn. Upd. QA | Dyn. Upd. Rec. | Proac. Align. QA | Proac. Align. Rec. | Temp. Reas. QA | Temp. Reas. Rec. | Single Sess. QA | Single Sess. Rec. | Multi Sess. QA | Multi Sess. Rec. |
| Base Method | 91.90 | 90.34 | 91.92 | 86.01 | 95.93 | 90.16 | 91.67 | 62.50 | 93.83 | 92.25 | 90.13 | 85.18 |
| HMRU-R | 93.25 | 92.63 | 94.95 | 91.31 | 95.12 | 91.99 | 91.67 | 75.00 | 93.83 | 93.88 | 93.33 | 89.51 |
| HMRU-I | 93.12 | 92.73 | 91.92 | 86.26 | 95.12 | 96.86 | 91.67 | 75.00 | 93.50 | 93.62 | 92.80 | 90.39 |
| HMRU-T | 90.96 | 92.87 | 79.80 | 83.33 | 88.62 | 91.38 | 66.67 | 75.00 | 90.33 | 93.38 | 87.47 | 88.49 |
Table 4. Comparison of HMRU variants and baseline memory performance on RealMem using HuatuoGPT-3-27B. QA and Rec. denote QA Score and Memory Recall, respectively. All scores are percentages. Bold and underlined values indicate the best and second-best distinct scores for each metric, respectively.
Improving Long-Term Conversational Memory through Hybrid Retrieval
We further examined the effectiveness of HMRU-R on RealMem-mem, using A-MEM as the baseline memory system and HuatuoGPT-3-27B as the frozen language model (Figure 7). HMRU-R increased the overall perfect-answer rate from 80.3 to 85.2 per cent, reducing the performance gap relative to FullContext from 10.2 to 5.2 percentage points, corresponding to a 48 per cent reduction in the original gap (Figure 7a). Paired question-level analysis identified 26 questions corrected by HMRU-R and 10 previously correct questions that became incorrect, yielding a statistically significant improvement under the exact McNemar test, with . We next assessed whether these improvements extended to longer conversation histories (Figure 7b). HMRU-R generally outperformed the baseline across history-length groups, with particularly pronounced gains in longer conversations. Nevertheless, after adjustment for question category, evidence count, and session type, history length remained negatively associated with perfect-answer outcomes for both A-MEM and HMRU-R, with adjusted odds ratios of 0.42 and 0.49, respectively. Thus, hybrid retrieval improved overall answering performance but did not eliminate the association between longer histories and reduced accuracy. To further characterize the sources of improvement, we examined changes in gold-memory recall among questions with altered answering outcomes (Figure 7c). Among the 26 questions corrected by HMRU-R, 11 exhibited increased recall of required evidence, whereas 15 were corrected without a change in gold-memory recall. Conversely, among the 10 questions that became incorrect, 8 showed unchanged recall and 2 showed reduced recall. Overall, the proportion of questions for which all required memories were retrieved increased from 82 to 87 per cent. These findings indicate that HMRU-R improves both evidence accessibility and downstream answering, while the substantial number of corrections without recall changes suggests that retrieval ranking and evidence presentation may also contribute to its effectiveness.
Figure 7. Performance improvements of HMRU-R on long-term conversational memory using HuatuoGPT-3-27B. a, Overall perfect-answer rates compared with the baseline and FullContext. b, Performance across five conversation history-length groups. c, Question-level improvements and regressions stratified by changes in gold-memory recall.
Discussion
Biomedical memory performance depends on task characteristics This study examined how agentic memory systems construct, retrieve, update and use information across biomedical knowledge reasoning, clinical prediction and long-term conversational tasks. Across six biomedical benchmarks, four language models and five memory approaches, no method consistently outperformed the others. The analyses instead identified three recurring limitations: relevant evidence may not be retrieved, retrieved evidence may be used incorrectly, and evolving memory representations may diverge from their source records. We further investigated these issues in long-term conversation and developed Hybrid Memory Retrieval and Utilization (HMRU), a training-free framework that improves retrieval and evidence use without updating language-model parameters. These findings suggest that memory quality cannot be characterized by storage capacity or retrieval volume alone. The relative advantages of memory methods changed with both the task and the language-model backbone (Table 1). With HuatuoGPT-3-27B, A-MEM achieved exact-match F1 scores of 39.27 on RiTeK-PharmKG and 52.94 on RiTeK-ADInt, compared with 39.04 and 51.63 for NaiveRAG. With Qwen3-30B, however, NaiveRAG outperformed A-MEM on both tasks. The differences were more substantial in clinical prediction: with HuatuoGPT-3-27B, NaiveRAG achieved an opioid overdose AUPRC of 15.61, compared with 4.94 for A-MEM, whereas MemZero performed best on mortality prediction with Meditron3-Qwen2.5-14B. Thus, more elaborate memory organization does not necessarily produce better downstream results. Increasing the memory construction set from 250 to 1,000 records improved A-MEM F1 on PharmKG from 27.66 to 39.27, but larger memory collections, deeper retrieval and class rebalancing did not uniformly benefit clinical prediction. These task differences may reflect distinct relationships between semantic similarity and the information needed for prediction. In TCGA-Reports, retrieved examples were strongly enriched for the same cancer category, with retrieval lift exceeding 15 (Figure 2f). By contrast, label enrichment was weaker and less consistent for mortality and readmission. Two patients with similar diagnoses or clinical histories may nevertheless experience different future outcomes. Consequently, retrieval objectives that work well for identifying explicit biomedical facts or diagnostic categories may be insufficient for prospective risk prediction, which requires attention to longitudinal patterns and clinically meaningful differences between patients. Outcome prediction, however, requires more than identifying a superficially similar patient. The relevance of historical diagnoses depends on their timing, combinations, severity and relation to the target event. A memory store assembled from examples with matching labels could also overrepresent prevalent outcomes without providing meaningful discrimination. This is particularly important when positive cases are uncommon, as in opioid overdose prediction, where AUPRC and retrieval enrichment should be interpreted relative to event prevalence. Our findings therefore argue against using a single retrieval objective across biomedical applications. A useful next step would compare semantically nearest neighbors with temporally aligned, clinically matched and outcome-aware candidates under the same retrieval budget, while preserving a separate held-out evaluation cohort.
Evidence availability and evidence utilization are distinct limitations Our evidence-level analyses help explain why retrieving additional memories does not always improve question answering. Across the RiTeK datasets, the presence of an answer-matching source was associated with substantially greater accuracy: retrieval-based methods generally answered 80–88% of questions correctly when such a source was available, compared with 12–16% when it was absent (Figure 2c). Yet an answer-matching record did not necessarily support the same biomedical fact. On RiTeK-ADInt, A-MEM, LightMem and NaiveRAG each retrieved same-fact sources for approximately 35% of questions, without a consistent coverage advantage for the more structured methods (Figure 2b). Accuracy also increased with the semantic similarity between the question and its answer-matching source, approaching 100% in the highest similarity bin (Figure 2e). These observations underscore the difference between recovering an answer entity and retrieving evidence that addresses the correct biomedical relation. Evidence utilization introduced a separate challenge. Incorrect predictions exactly matched answers contained in retrieved records for 16–21% of RiTeK-ADInt questions and 24–29% of RiTeK-PharmKG questions across the retrieval-based approaches (Figure 2d). Such overlap is consistent with interference from competing answers, although it does not prove that the model copied a specific memory. Similarly, even complete retrieval of annotated evidence did not always produce correct answers in the long-term conversational evaluation. Thus, memory systems must both recover appropriate evidence and distinguish among potentially conflicting sources during generation. These analyses are observational: questions with retrievable evidence may be inherently easier, and causal effects require interventions that remove, reorder or replace retrieved memories. Nevertheless, they support evaluating factual coverage and downstream evidence use as separate properties of a memory system.
Iterative memory evolution does not guarantee source fidelity A further concern is whether generated memory representations faithfully preserve their original information. Agentic memory mechanisms may summarize incoming records and rewrite existing notes as new information arrives. Although these operations can improve organization, they can also introduce unsupported contextual statements. In our A-MEM analysis, the proportion of memories with context not supported by their original training records increased from 12% to 44% on MIMIC-IV opioid overdose prediction and from 16% to 35% on RiTeK-PharmKG (Figure 3c). In contrast, unsupported-context prevalence remained below 2% in both MIMIC-III tasks, indicating substantial task dependence. Repeated rewriting was associated with the same pattern. In MIMIC-IV OOD, approximately 10% of memories without rewrites contained unsupported context, compared with 80% among those rewritten at least five times (Figure 3b). Longitudinal examination of frequently rewritten memories revealed extended sequences of unsupported versions, but some notes also returned to source-supported states (Figure 3d). These findings do not establish that rewriting causes factual errors: complex records may be rewritten more frequently, and information unsupported by a particular source is not necessarily false. They nevertheless challenge the assumption that iterative refinement invariably improves memory quality. Source-linked provenance, verification against original records and explicit separation of source facts from generated interpretations merit investigation, especially when updated notes are reused in subsequent reasoning.
Long-term conversational memory requires accessible and complete evidence The RealMem experiments revealed a substantial gap between structured memory and access to the full conversation history. On 325 medical-subset questions, FullContext outperformed all four memory-based approaches in overall perfect-answer accuracy (Figure 5a). Relative to A-MEM, it uniquely answered 55 questions correctly, whereas A-MEM uniquely answered 22; the paired difference was significant under the exact McNemar test (). This is not evidence that FullContext is correct on every question, but it indicates that existing memory systems lose information or fail to exploit information that remains accessible in the complete history. Evidence completeness also mattered: A-MEM’s perfect-answer rate increased from approximately 5% when no gold memories were retrieved to 89% when all were retrieved (Figure 5b). However, some errors persisted despite complete evidence retrieval, reinforcing the distinction between access and utilization. Conversation length produced a more nuanced picture. Across naturally occurring histories containing 2–238 sessions, longer histories were negatively associated with answering performance for A-MEM (adjusted odds ratio 0.42) and MemZero (0.68), but not significantly for the other methods (Figure 5c). In contrast, when the same 100 questions were evaluated under controlled histories with median lengths of 23,000, 36,000 and 48,000 tokens, neither A-MEM nor LightMem exhibited a consistent performance decline (Figure 4a). Relocating relevant evidence within a fixed history also produced limited overall deterioration, although A-MEM declined at the greatest tested distance (Figure 4b). Across 600 question-condition observations, A-MEM achieved a QA score of 89.7 with complete evidence retrieval versus 43.2 with incomplete retrieval (Figure 4c). These results suggest that natural conversation length is entangled with evidence distribution, updating requirements and question complexity; token count alone is not an adequate explanation for long-term memory failures. The contrast between observational and controlled results is methodologically important. Longer natural histories may contain more factual revisions, references distributed across sessions, or multiple related entities that must be reconciled. Removing irrelevant material from a fixed question, by comparison, does not necessarily reproduce these challenges. Thus, the absence of deterioration in a controlled length experiment should not be interpreted as proof that memory remains robust in arbitrarily long real-world interactions. Benchmarks should independently vary history length, number of required evidence items, temporal update frequency and the placement of contradictory or superseded statements.
Hybrid retrieval and task-aware utilization offer complementary benefits HMRU was designed to address the retrieval and utilization limitations identified above. It combines lexical and semantic retrieval with cross-encoder reranking and task-aware instructions, while retaining the underlying memory bank and frozen language model. On the two biomedical question-answering tasks, HMRU-I improved exact-match F1 from 39.27 to 43.48 on PharmKG and from 52.94 to 57.64 on ADInt relative to A-MEM Table 3. The component analysis indicated different sources of improvement. On PharmKG, BM25 augmentation reduced classified retrieval failures from 55.3% to 50.0%; on ADInt, task-aware instructions improved F1 from 54.6 to 57.6 after retrieval enhancement Figure 6. Across both tasks, identified utilization failures decreased from 99 to 76 following prompt adaptation. These results support combining evidence acquisition with better interpretation rather than treating retrieval alone as sufficient. The benefits were not universal. HMRU-C modestly improved mortality AUPRC from 9.81 to 10.03, and HMRU-I increased readmission AUPRC from 61.91 to 62.17, but neither configuration improved opioid overdose AUPRC over the A-MEM baseline. This limitation is consistent with the weaker same-label enrichment observed in clinical prediction: identifying relevant medical information and discriminating future outcomes are not equivalent objectives. Clinical prediction may require outcome-aware retrieval and uncertainty-sensitive utilization strategies beyond those effective for factual question answering. In long-term conversation, HMRU-R increased the RealMem perfect-answer rate from 80.3% to 85.2%, reducing the gap to FullContext (90.5%) by nearly half Figure 7. Paired comparison identified 26 corrected questions and 10 newly incorrect answers (exact McNemar ). Gold-memory recall was unchanged in 15 of the 26 corrected cases, suggesting that evidence ranking or presentation may contribute beyond retrieving additional annotated memories. More extensive restructuring was not consistently beneficial: chronological HMRU-T reduced accuracy on some dynamic updating and temporal reasoning questions. Thus, utilization strategies should reflect the specific information requirements of the task, rather than assuming that additional structure always helps.
Methods
Datasets, Memory Baselines, and Language Models
Datasets. We evaluated biomedical agentic memory on six tasks covering knowledge reasoning, clinical outcome prediction, and pathology classification. RiTeK-PharmKG [13] and RiTeK-ADInt [13] assess biomedical question answering over relational knowledge. MIMIC-III Mortality and MIMIC-III Readmission are clinical outcome prediction tasks derived from MIMIC-III [14]. TCGA-Reports supports cancer classification from pathology report text [15], whereas MIMIC-IV OOD [16] evaluates opioid overdose prediction using electronic health records from MIMIC-IV. We further evaluated long-term conversational memory from the medical subset of RealMem [12], which includes multi-session interactions requiring information retention, updating, and temporal reasoning. We report exact-match and ROUGE-1 precision, recall, and F1 for the RiTeK tasks; AUPRC and AUROC are used for the four classification and prediction tasks. For the RealMem evaluation, we used GPT-4o-mini as an LLM-based judge to assess the correctness of generated answers.
Memory baselines with Language model backbones. We compared five memory configurations spanning structured memory management, direct retrieval, and full-context access. A-MEM [10] constructs linked memory notes and supports dynamic indexing and iterative memory evolution. LightMem [11] uses lightweight filtering and memory consolidation to manage persistent information. MemZero was evaluated as an additional persistent-memory approach, with Mem0 [9] providing the related framework for memory extraction, retrieval, and updating. NaiveRAG [7] serves as an embedding-based retrieval baseline that selects relevant source records without evolving a structured memory bank, following the general retrieval-augmented generation paradigm. FullContext provides the available source records or conversation history directly to the language model without selective retrieval. The same underlying training records and evaluation questions were used for method comparisons within each task. We evaluated four open-weight language models: Qwen3-8B1, Qwen3-30B-A3B-Instruct-250722, Meditron3-Qwen2.5-14B3, and HuatuoGPT-3-27B4.
Memory Retrieval Quality and Utilization
Analysis design. To examine why different memory systems yield different downstream outcomes, we analyzed test-set retrieval and prediction traces from A-MEM, MemZero, LightMem, NaiveRAG, and FullContext across six biomedical benchmarks. The analyzed predictions were generated by HuatuoGPT-3-27B, with each method drawing on the same 1,000 training examples per dataset for memory construction. Only memory items that actually entered the final inference prompt were included. We organized the analysis into three layers: answer or label exposure, answer retention following memory construction, and relevance of the retrieved evidence. The full three-layer analysis was applied to RiTeK-ADInt and RiTeK-PharmKG. For TCGA-Reports, MIMIC-IV OOD, MIMIC-III Mortality, and MIMIC-III Readmission, we assessed label exposure and source relevance, but did not apply literal answer-retention checks because class labels were not consistently preserved as explicit text in the memory representations.
For the two RiTeK datasets, let , , and denote the test question, reference answer, and generated answer. Let contain the memory entries included in its final prompt, and let denote the original training records associated with memory entry . When one memory entry merged several records, all recorded source identifiers were used for provenance tracing. We defined answer-source exposure as
where is the original training answer and norm denotes case, punctuation, and whitespace normalization. This criterion identifies sources sharing the answer, not necessarily sources supporting the same biomedical fact. Conditional on , we further checked whether the normalized reference answer appeared as a whole-word sequence in the corresponding stored memory text. This second layer assesses explicit answer retention in retrieved memory, rather than information retention across the entire memory store.
To identify factual relevance, an independent Qwen3-30B-A3B-Instruct-2507 evaluator classified each answer-matching test–source pair as same fact, related, or unrelated, using the test question and answer alongside the source question and answer. Evaluation used temperature zero; source pairs were deduplicated across methods. Each test question was assigned its strongest retrieved evidence category, in the order same fact, related, and unrelated, or no answer source when . Separately, BAAI/bge-m3 embeddings with a maximum input length of 512 tokens were used to compute cosine similarities between test and source questions. For TCGA-Reports, the same embedding model measured pathology-report similarity. For the three binary clinical tasks, similarity was computed as Jaccard overlap between normalized sets of diagnosis codes without using an additional LLM judge.
Layer 1: Answer exposure and label consistency. For RiTeK-ADInt and RiTeK-PharmKG, we traced retrieved memories to their original training records using source identifiers and compared the corresponding reference answers with the test ground truth after text normalization. We measured the proportion of questions with at least one answer-matching source and the prediction accuracy conditioned on its presence or absence. For TCGA-Reports, we calculated the proportion of retrieved training reports sharing the correct cancer label, nearest-neighbor classification accuracy, and majority-voting accuracy. For MIMIC-IV OOD, MIMIC-III Mortality, and MIMIC-III Readmission, we performed the same label-based analysis separately for positive and negative cases. We additionally calculated retrieval enrichment relative to the corresponding training-set class priors.
Layer 2: Information retention. For RiTeK-ADInt and RiTeK-PharmKG, we examined whether answers associated with matching source records remained explicitly present in the retrieved memory text following memory construction, compression, rewriting, or evolution. We applied case normalization, punctuation removal, whitespace normalization, and whole-word matching to compare stored memory content with reference answers. Information retention was measured conditionally on retrieving at least one answer-matching source. This analysis was not applied to TCGA-Reports or the three binary clinical prediction tasks because their compressed memory representations do not consistently preserve explicit class labels.
Layer 3: Evidence relevance. For RiTeK-ADInt and RiTeK-PharmKG, we employed Qwen3-30B-A3B-Instruct-2507 as an independent evaluator to classify test questions and retrieved source questions into three categories: same fact, related, and unrelated. The evaluator received the test question, reference answer, source question, and source answer, using deterministic decoding with temperature zero. We additionally computed cosine similarity between question embeddings generated by BAAI/bge-m3 with a maximum sequence length of 512 tokens. For TCGA-Reports, BAAI/bge-m3 was used to assess semantic similarity between test pathology reports and retrieved source reports, distinguishing matching and nonmatching cancer labels. For MIMIC-IV OOD, MIMIC-III Mortality, and MIMIC-III Readmission, we instead measured patient similarity using Jaccard similarity over normalized diagnosis-code sets, separately evaluating retrieved cases with matching and nonmatching outcome labels.
Memory utilization. Finally, we compared prediction correctness across evidence-availability and relevance categories. For biomedical question answering, we further measured the frequency with which generated answers matched labels associated with retrieved memory sources, distinguishing correct and incorrect predictions. Together, these analyses characterize the relationships among answer availability, information retention, evidence relevance, and downstream model performance.
Construction of Controlled Conversational Memory
To systematically evaluate the robustness of memory systems to conversation length and evidence distance, we constructed two controlled experiments from the ReaMem benchmark. Each question is associated with a sequence of historical conversation sessions, categorized as relevant sessions containing information needed to answer the question or unrelated sessions that do not provide the required evidence. All transformations were performed at the session level, preserving the original content of retained sessions and the evidence required for answering each question.
Conversation history length. We constructed three conditions, Short, Medium, and Long, by selectively removing unrelated sessions preceding the earliest evidence-bearing session. Relevant sessions and all sessions occurring after the earliest evidence were preserved across conditions. In the Short condition, all preceding unrelated sessions were removed. In the Medium condition, unrelated sessions were removed sequentially from the beginning of the conversation until the remaining unrelated content was closest to half of its original token count. The Long condition retained the original conversation history without modification. This procedure produced three levels of historical context while preserving the evidence required for each question, allowing us to examine how preceding contextual information influences memory retrieval and answer generation.
Evidence distance. To isolate the effect of evidence position, we constructed Near, Medium, and Far conditions by rearranging unrelated sessions without removing or modifying their contents. All three conditions contained the same set of sessions and therefore maintained the same overall context length. In the Near condition, unrelated sessions were placed before the relevant evidence, positioning the evidence closer to the final question. In the Far condition, unrelated sessions were placed between the relevant evidence and the question, increasing the distance between them. The Medium condition distributed unrelated sessions approximately equally by token count before and after the evidence. This design isolates evidence position from context length and information availability, enabling a controlled assessment of memory sensitivity to retrieval distance.
Hybrid Memory Retrieval and Utilization (HMRU)
We propose Hybrid Memory Retrieval and Utilization (HMRU), a training-free framework that integrates complementary evidence retrieval and task-aware memory utilization for biomedical applications. HMRU combines dense retrieval with BM25-based lexical retrieval to capture complementary semantic and lexical evidence, followed by cross-encoder reranking to refine evidence selection. Given a query , the retrieved memories are formulated as
where and denote dense and lexical retrieval, respectively, and represents the cross-encoder relevance score. To improve downstream evidence utilization, HMRU constructs a task-conditioned context that integrates retrieved memories with task-specific instructions:
where describes the task, specifies evidence utilization instructions, and determines the memory representation. For biomedical knowledge reasoning, HMRU emphasizes entity–relation consistency and evidence-supported answers. For clinical prediction, it prioritizes task-relevant patient information and clinical risk factors while accounting for outcome prevalence. We examine complementary configurations to distinguish the contributions of retrieval and utilization. HMRU-R enhances evidence selection, while HMRU-I further incorporates task-aware instructions. HMRU-C additionally adopts a compact memory representation:
For long-term conversational memory, HMRU-T examines chronological evidence organization. All configurations operate without modifying the underlying language model parameters.
Conclusion
In this study, we systematically evaluate agentic memory systems across biomedical knowledge reasoning, clinical prediction, and long-term conversational memory tasks. We examine how memory construction, retrieval, utilization, and evolution influence downstream performance, identifying task-dependent limitations in evidence relevance, information fidelity, and long-term memory reliability. Our analyses show that retrieving relevant information does not guarantee correct utilization, semantic similarity does not necessarily correspond to clinical outcome similarity, and iterative memory updates may introduce source-unsupported content. We further demonstrate that long-term conversational memory performance cannot be explained by history length alone. To improve evidence retrieval and utilization, we introduce HMRU, a training-free framework that enhances biomedical knowledge reasoning and long-term conversational memory performance. These findings highlight the importance of evaluating memory systems beyond downstream accuracy, with particular attention to evidence accessibility, source fidelity, and effective utilization.
Supplementary Sensitivity Analyses
This section provides the complete results of the sensitivity analyses for biomedical memory systems. We examine changes in memory construction data volume, retrieval depth, and memory class composition while retaining the corresponding fixed test sets.
Memory Construction Data Volume
Table 5 reports performance across memory collections constructed from different numbers of training examples, covering biomedical knowledge reasoning, clinical prediction, and pathology classification.
| Method | Memory data | RiTeK-PharmKG | RiTeK-ADint | Mortality AUPRC | AUROC | Readmission AUPRC | AUROC | TCGA-Reports AUPRC | AUROC | MIMIC-IV AUPRC | OOD AUROC | ||||||||||
| HuatuoGPT-3-27B | A-MEM 25% | 28.14 | 27.78 | 27.66 | 31.38 | 31.00 | 30.89 | 38.28 | 37.66 | 37.83 | 46.67 | 46.25 | 46.34 | 9.11 | 57.47 | 58.45 | 53.00 | 98.76 | 99.80 | 4.91 | 73.08 |
| A-MEM 50% | 33.76 | 33.48 | 33.23 | 36.52 | 36.28 | 36.05 | 43.36 | 43.15 | 43.08 | 50.23 | 50.13 | 50.00 | 9.67 | 59.66 | 61.25 | 56.24 | 99.01 | 99.85 | 3.87 | 82.96 | |
| A-MEM 100% | 39.95 | 39.23 | 39.27 | 42.18 | 41.40 | 41.51 | 53.32 | 53.08 | 52.94 | 58.84 | 58.62 | 58.53 | 9.81 | 59.88 | 61.91 | 56.69 | 99.06 | 99.93 | 4.94 | 82.16 | |
| MemZero 25% | 26.25 | 25.43 | 25.64 | 29.22 | 28.39 | 28.60 | 30.19 | 29.88 | 29.94 | 38.98 | 38.78 | 38.78 | 9.33 | 53.16 | 55.62 | 51.64 | 98.99 | 99.85 | 3.01 | 72.07 | |
| MemZero 50% | 28.53 | 28.35 | 28.13 | 30.90 | 30.74 | 30.50 | 34.54 | 34.30 | 34.27 | 42.39 | 42.28 | 42.18 | 7.95 | 52.54 | 57.94 | 51.74 | 98.58 | 99.88 | 5.29 | 76.53 | |
| MemZero 100% | 31.22 | 31.07 | 30.84 | 33.96 | 33.76 | 33.57 | 43.57 | 43.43 | 43.31 | 50.05 | 49.99 | 49.84 | 8.29 | 54.21 | 57.98 | 52.71 | 98.64 | 99.87 | 6.95 | 78.94 | |
| FullContext 25% | 26.98 | 25.76 | 26.10 | 29.72 | 28.63 | 28.94 | 36.51 | 35.96 | 36.11 | 44.25 | 43.96 | 43.99 | 10.85 | 60.21 | 55.13 | 50.60 | 98.50 | 99.86 | 8.83 | 85.28 | |
| FullContext 50% | 26.84 | 26.30 | 26.20 | 30.99 | 30.48 | 30.42 | 32.05 | 31.71 | 31.71 | 38.86 | 38.65 | 38.63 | 9.03 | 57.59 | 58.03 | 52.34 | 98.83 | 99.89 | 7.26 | 83.64 | |
| FullContext 100% | 27.55 | 26.65 | 26.80 | 30.25 | 29.51 | 29.64 | 37.34 | 36.58 | 36.83 | 45.30 | 44.68 | 44.90 | 9.68 | 59.19 | 57.93 | 52.55 | 98.88 | 99.88 | 8.23 | 82.76 | |
| NaiveRAG 25% | 28.71 | 28.35 | 28.13 | 31.77 | 31.40 | 31.24 | 38.07 | 37.66 | 37.76 | 46.63 | 46.21 | 46.32 | 9.30 | 58.46 | 59.55 | 53.20 | 99.10 | 99.88 | 7.70 | 82.80 | |
| NaiveRAG 50% | 32.32 | 32.42 | 31.93 | 35.10 | 35.29 | 34.76 | 44.50 | 44.05 | 44.09 | 51.04 | 50.78 | 50.74 | 9.44 | 60.49 | 59.28 | 54.98 | 98.75 | 99.82 | 5.22 | 81.65 | |
| NaiveRAG 100% | 39.42 | 39.33 | 39.04 | 41.48 | 41.36 | 41.07 | 52.07 | 51.63 | 51.63 | 57.69 | 57.34 | 57.36 | 9.81 | 61.98 | 60.76 | 55.79 | 98.97 | 99.93 | 15.61 | 85.47 | |
| LightMem 25% | 27.75 | 27.06 | 27.13 | 30.82 | 30.15 | 30.24 | 33.61 | 33.40 | 33.40 | 41.12 | 40.99 | 40.95 | 8.96 | 54.25 | 54.88 | 49.01 | 98.74 | 99.85 | 5.21 | 83.54 | |
| LightMem 50% | 31.48 | 31.31 | 31.05 | 34.05 | 33.96 | 33.68 | 38.80 | 38.66 | 38.55 | 45.58 | 45.45 | 45.37 | 8.79 | 55.38 | 56.87 | 51.34 | 98.70 | 99.89 | 4.28 | 78.41 | |
| LightMem 100% | 37.05 | 36.84 | 36.62 | 39.25 | 38.89 | 38.78 | 50.21 | 49.76 | 49.76 | 55.06 | 54.88 | 54.77 | 9.62 | 57.80 | 59.17 | 53.91 | 98.47 | 99.81 | 3.42 | 79.11 | |
| Qwen3-30B-A3B-Instruct-2507 | A-MEM 25% | 22.31 | 25.64 | 23.17 | 25.20 | 29.25 | 26.36 | 28.27 | 34.34 | 30.11 | 35.05 | 44.02 | 37.84 | 9.59 | 54.25 | 56.72 | 48.58 | 97.77 | 99.77 | 3.74 | 73.95 |
| A-MEM 50% | 27.43 | 32.10 | 28.72 | 29.83 | 34.98 | 31.29 | 36.45 | 43.02 | 38.41 | 42.22 | 51.13 | 44.92 | 9.20 | 53.36 | 56.98 | 51.70 | 97.68 | 99.85 | 4.06 | 75.72 | |
| A-MEM 100% | 31.79 | 36.23 | 33.00 | 33.95 | 38.64 | 35.28 | 44.35 | 50.80 | 46.20 | 50.69 | 58.69 | 53.18 | 9.37 | 51.17 | 55.01 | 49.76 | 98.39 | 99.81 | 4.13 | 81.66 | |
| MemZero 25% | 17.76 | 25.50 | 19.11 | 20.40 | 25.89 | 22.00 | 19.37 | 24.86 | 21.04 | 26.21 | 35.70 | 29.12 | 8.06 | 50.34 | 56.50 | 50.52 | 96.57 | 99.44 | 3.34 | 75.42 | |
| MemZero 50% | 19.06 | 24.14 | 20.45 | 20.94 | 26.64 | 22.62 | 25.11 | 32.12 | 27.14 | 30.60 | 40.22 | 33.54 | 8.11 | 50.64 | 56.08 | 50.76 | 96.53 | 99.65 | 2.16 | 72.24 | |
| MemZero 100% | 23.76 | 29.20 | 25.29 | 25.26 | 31.31 | 27.01 | 28.75 | 35.79 | 30.80 | 34.82 | 44.95 | 37.86 | 7.54 | 51.35 | 57.13 | 52.33 | 96.76 | 99.75 | 2.67 | 74.60 | |
| FullContext 25% | 17.84 | 25.29 | 23.47 | 25.32 | 28.08 | 25.93 | 29.82 | 32.03 | 26.88 | 37.44 | 42.76 | 38.95 | 10.00 | 54.83 | 56.13 | 49.30 | 95.08 | 99.59 | 7.23 | 76.58 | |
| FullContext 50% | 21.48 | 23.67 | 21.90 | 24.82 | 27.49 | 25.40 | 25.17 | 30.57 | 26.74 | 32.34 | 38.43 | 34.64 | 8.42 | 53.08 | 56.71 | 50.65 | 97.70 | 99.87 | 3.43 | 79.02 | |
| FullContext 100% | 24.11 | 26.16 | 24.42 | 26.53 | 29.12 | 27.07 | 29.96 | 34.09 | 31.04 | 37.16 | 44.43 | 39.57 | 10.27 | 52.90 | 55.50 | 49.78 | 94.51 | 99.43 | 3.52 | 73.96 | |
| NaiveRAG 25% | 24.15 | 27.57 | 25.01 | 26.93 | 31.08 | 28.07 | 31.92 | 36.17 | 33.12 | 39.94 | 46.88 | 42.04 | 9.79 | 54.16 | 54.82 | 50.72 | 97.85 | 99.65 | 3.88 | 77.82 | |
| NaiveRAG 50% | 28.88 | 33.72 | 30.16 | 31.32 | 36.54 | 32.76 | 36.26 | 42.15 | 37.98 | 43.27 | 50.98 | 45.66 | 9.71 | 61.01 | 57.68 | 52.45 | 98.29 | 99.75 | 2.61 | 74.94 | |
| NaiveRAG 100% | 35.24 | 40.29 | 36.53 | 36.87 | 41.87 | 38.18 | 46.14 | 50.17 | 47.29 | 52.68 | 57.93 | 54.26 | 9.02 | 56.57 | 57.89 | 54.70 | 98.36 | 99.90 | 3.92 | 78.59 | |
| LightMem 25% | 21.91 | 25.59 | 22.91 | 24.28 | 28.53 | 25.46 | 27.58 | 32.88 | 29.21 | 34.52 | 42.75 | 37.16 | 8.98 | 53.85 | 55.08 | 49.54 | 97.10 | 99.81 | 2.50 | 72.03 | |
| LightMem 50% | 25.40 | 29.52 | 26.43 | 27.65 | 32.29 | 28.88 | 32.64 | 38.66 | 34.39 | 38.10 | 46.43 | 40.66 | 7.80 | 50.44 | 54.91 | 50.12 | 97.27 | 99.72 | 6.63 | 80.54 | |
| LightMem 100% | 29.67 | 33.77 | 30.73 | 31.31 | 35.57 | 32.47 | 39.64 | 45.30 | 41.16 | 45.29 | 52.65 | 47.49 | 8.88 | 52.19 | 54.76 | 49.76 | 97.76 | 99.81 | 3.25 | 77.61 |
Table 5. Effect of memory construction data volume across six biomedical tasks. Nested memory-construction subsets contain 250, 500, and 1,000 training samples, corresponding to 25%, 50%, and 100% of the full subset. The test sets and other experimental settings are fixed. RiTeK tasks report precision (), recall (), and F1 () using exact match () and ROUGE-1 (). Mortality and readmission use MIMIC-III. TCGA-Reports results are macro-averaged. Clinical prediction and pathology classification report AUPRC and AUROC. All results are percentages. Within each model and memory data volume, bold and <u>underlined</u> values indicate the best and second-best scores among memory approaches, respectively.
Retrieval Depth
Table 6 presents the results obtained with different numbers of retrieved memory items for each model and memory method.
| Method | Top- | RiTeK-PharmKG | RiTeK-PharmKG | RiTeK-PharmKG | RiTeK-PharmKG | RiTeK-PharmKG | RiTeK-PharmKG | RiTeK-ADint | RiTeK-ADint | RiTeK-ADint | RiTeK-ADint | RiTeK-ADint | RiTeK-ADint | Mortality AUPRC | Mortality AUROC | Readmission AUPRC | Readmission AUROC | TCGA-Reports AUPRC | TCGA-Reports AUROC | MIMIC-IV OOD AUPRC | MIMIC-IV OOD AUROC |
| Qwen3-30B-A3B-Instruct-2507 / A-MEM | 1 | 26.03 | 31.02 | 27.33 | 28.72 | 34.46 | 30.28 | 34.98 | 41.94 | 36.71 | 40.28 | 50.02 | 42.94 | 8.99 | 56.72 | 56.17 | 52.02 | 96.02 | 99.56 | 6.42 | 78.18 |
| 3 | _29.14_ | _33.46_ | _30.32_ | _31.29_ | _36.14_ | _32.68_ | _41.13_ | _46.23_ | _42.57_ | _46.92_ | _54.37_ | _49.09_ | _8.43_ | 54.76 | _55.54_ | _50.89_ | _97.03_ | _99.63_ | 4.10 | 73.39 | |
| 5 | _30.45_ | _35.57_ | _31.81_ | _32.63_ | _38.03_ | _34.12_ | _43.25_ | _49.20_ | _44.95_ | _49.01_ | _57.27_ | _51.50_ | _8.86_ | 54.00 | _56.98_ | _52.28_ | _97.97_ | _99.77_ | 3.08 | _74.98_ | |
| 10 | 31.79 | 36.23 | 33.00 | 33.95 | 38.64 | 35.28 | 44.35 | 50.80 | 46.20 | 50.69 | 58.69 | 53.18 | 9.37 | 51.17 | 55.01 | 49.76 | 98.39 | 99.81 | 4.13 | 81.66 | |
| MemZero | 1 | 18.99 | 22.97 | 20.13 | 21.81 | 26.43 | 23.16 | 27.40 | 32.54 | 28.82 | 32.35 | 39.98 | 34.53 | 8.85 | 53.43 | 57.33 | 53.15 | 94.06 | 99.39 | 6.27 | 74.18 |
| 3 | 21.42 | 26.44 | 22.82 | 23.40 | 29.02 | 25.02 | 28.01 | 34.41 | 29.87 | 34.02 | 43.65 | 36.94 | 7.75 | 51.29 | 57.37 | 52.17 | 95.49 | 99.59 | 2.22 | 72.77 | |
| 5 | 22.33 | 27.31 | 23.78 | 24.21 | 29.88 | 25.91 | 28.55 | 34.20 | 30.21 | 34.19 | 42.78 | 36.80 | 8.03 | 49.78 | 55.83 | 51.03 | 95.62 | 99.60 | 2.12 | 72.61 | |
| 10 | 23.76 | 29.20 | 25.29 | 25.26 | 31.31 | 27.01 | 28.75 | 35.79 | 30.80 | 34.82 | 44.95 | 37.86 | 7.54 | 51.35 | 57.13 | 52.33 | 96.76 | 99.75 | 2.67 | 74.60 | |
| LightMem | 1 | 23.09 | 27.45 | 24.25 | 25.97 | 31.09 | 27.35 | 31.66 | 36.27 | 32.83 | 36.81 | 44.25 | 38.92 | 7.28 | 49.87 | 54.97 | 49.35 | 94.74 | 99.28 | 3.56 | 74.16 |
| 3 | 26.81 | 31.02 | 28.08 | 28.88 | 33.56 | 30.30 | 35.50 | 41.56 | 37.19 | 40.99 | 49.37 | 43.54 | 7.86 | 50.37 | 54.63 | 49.39 | 96.70 | 99.37 | _3.41_ | 77.47 | |
| 5 | 28.54 | 33.02 | 29.82 | 30.43 | 35.16 | 31.82 | 37.54 | 42.60 | 38.91 | 43.62 | 51.02 | 45.83 | 8.66 | 53.20 | 54.54 | 49.57 | 97.10 | 99.59 | 4.46 | 74.65 | |
| 10 | 29.67 | 33.77 | 30.73 | 31.31 | 35.57 | 32.47 | 39.64 | 45.30 | 41.16 | 45.29 | 52.65 | 47.49 | 8.88 | _52.19_ | 54.76 | 49.76 | _97.76_ | _99.81_ | 3.25 | 77.61 | |
| NaiveRAG | 1 | 27.30 | 30.97 | 28.23 | 30.47 | 34.91 | 31.65 | 36.02 | 41.04 | 37.34 | 41.67 | 49.10 | 43.80 | 9.33 | 55.87 | 55.76 | 51.55 | 96.31 | 99.65 | 3.68 | 76.46 |
| 3 | 31.19 | 35.73 | 32.36 | 33.04 | 37.87 | 34.36 | 42.48 | 47.89 | 43.98 | 48.73 | 55.75 | 50.82 | 10.16 | 57.59 | _56.79_ | 53.00 | 98.33 | 99.85 | 2.55 | 73.53 | |
| 5 | 33.31 | 37.90 | 34.47 | 34.86 | 39.68 | 36.12 | 45.07 | 49.55 | 46.24 | 51.58 | 58.04 | 53.52 | 9.21 | 57.90 | 58.08 | 53.61 | 98.13 | 99.85 | _3.23_ | _76.71_ | |
| 10 | 35.24 | 40.29 | 36.53 | 36.87 | 41.87 | 38.18 | 46.14 | 50.17 | 47.29 | 52.68 | 57.93 | 54.26 | _9.02_ | _56.57_ | 57.89 | 54.70 | _98.36_ | _99.90_ | _3.92_ | 78.59 | |
| HuatuoGPT-3-27B / A-MEM | 1 | 29.49 | 28.82 | 28.93 | 32.69 | 32.07 | 32.16 | 43.27 | 42.95 | 42.84 | 48.64 | 48.45 | 48.32 | 9.97 | 58.47 | 59.43 | 55.94 | 98.86 | 99.81 | 4.67 | 77.97 |
| 3 | _35.83_ | _35.42_ | _35.34_ | _38.33_ | _37.88_ | _37.84_ | _50.93_ | _50.48_ | _50.45_ | _56.20_ | _55.90_ | _55.87_ | 9.70 | 60.07 | _59.61_ | _55.41_ | _98.66_ | _99.83_ | 3.57 | _79.35_ | |
| 5 | 37.65 | 37.23 | 37.15 | 40.18 | 39.76 | 39.70 | 53.01 | 52.56 | 52.52 | 58.66 | 58.36 | 58.33 | 9.49 | _58.10_ | 59.63 | 55.74 | 98.94 | 99.93 | 5.31 | 83.75 | |
| 10 | 39.95 | 39.23 | 39.27 | 42.18 | 41.40 | 41.51 | 53.32 | 53.08 | 52.94 | 58.84 | 58.62 | 58.53 | 9.81 | 58.89 | 61.91 | 56.69 | 99.06 | 99.93 | 4.94 | 82.16 | |
| MemZero | 1 | 23.64 | 23.20 | 23.22 | 27.59 | 27.19 | 27.17 | 35.37 | 35.06 | 35.06 | 41.49 | 41.52 | 41.29 | _9.82_ | 56.25 | 58.86 | 53.95 | _98.93_ | _99.89_ | 3.92 | 70.89 |
| 3 | 27.24 | 26.87 | 26.78 | 30.50 | 30.14 | 30.06 | 39.21 | 39.00 | 38.93 | 46.18 | 46.11 | 46.00 | _9.71_ | 57.70 | 57.93 | 50.96 | _98.54_ | _99.89_ | 6.34 | 78.81 | |
| 5 | 29.20 | 29.21 | 28.89 | 32.01 | 31.97 | 31.68 | 42.84 | 42.74 | 42.60 | 49.62 | 49.69 | 49.44 | 9.12 | 53.67 | 57.36 | 51.24 | 98.50 | _99.87_ | 7.43 | 79.50 | |
| 10 | 31.22 | 31.07 | 30.84 | 33.96 | 33.76 | 33.57 | 43.57 | 43.43 | 43.31 | 50.05 | 49.99 | 49.84 | 8.29 | 54.21 | 57.98 | 52.71 | 98.64 | _99.87_ | _6.95_ | 78.94 | |
| LightMem | 1 | 29.14 | 28.48 | 28.57 | 32.12 | 31.55 | 31.59 | 41.08 | 40.46 | 40.66 | 46.54 | 46.16 | 46.25 | 8.78 | 57.70 | 60.23 | 55.20 | 99.17 | 99.92 | 4.90 | 71.69 |
| 3 | 33.04 | 32.73 | 32.67 | 35.76 | 35.29 | 35.33 | 47.61 | 46.96 | 47.06 | 52.38 | 52.06 | 52.02 | 7.99 | 54.47 | _60.97_ | _55.25_ | _98.87_ | 99.88 | 4.02 | 78.57 | |
| 5 | 34.41 | 33.85 | 33.89 | 36.48 | 35.89 | 35.96 | 47.93 | 47.58 | 47.54 | 53.82 | 53.52 | 53.52 | 7.53 | 52.99 | 61.32 | 56.09 | 98.73 | _99.87_ | 3.28 | 76.99 | |
| 10 | 37.05 | 36.84 | 36.62 | 39.25 | 38.89 | 38.78 | 50.21 | 49.76 | 49.76 | 55.06 | 54.88 | 54.77 | _9.62_ | 57.80 | 59.17 | 53.91 | 98.47 | 99.81 | 3.42 | 79.11 | |
| NaiveRAG | 1 | 29.80 | 29.57 | 29.52 | 33.56 | 33.16 | 33.17 | 50.56 | 50.15 | 40.18 | 45.57 | 45.35 | 45.30 | 8.63 | 55.56 | 59.75 | _56.10_ | 98.64 | 99.87 | 3.99 | 74.58 |
| 3 | 33.92 | 33.69 | 33.52 | 36.60 | 36.29 | 36.18 | 48.76 | 48.31 | 48.31 | 54.39 | 54.16 | 54.12 | 10.41 | 62.99 | 61.06 | 55.72 | 98.90 | 99.92 | _5.26_ | 87.62 | |
| 5 | 36.58 | 36.59 | 36.22 | 38.86 | 38.87 | 38.52 | 50.00 | 49.65 | 49.62 | 55.46 | 55.28 | 55.20 | _9.25_ | 60.59 | _60.94_ | _56.71_ | 99.03 | 99.93 | 5.84 | 88.67 | |
| 10 | 39.42 | 39.33 | 39.04 | 41.48 | 41.36 | 41.07 | 52.07 | 51.63 | 51.63 | 57.69 | 57.34 | 57.36 | 9.81 | 61.98 | 60.76 | 55.79 | 98.97 | 99.93 | 15.61 | 85.47 |
Table 6. Effect of retrieval depth on biomedical memory performance. We evaluate four memory methods (A-MEM, MemZero, LightMem, and NaiveRAG) with two language-model backbones across six biomedical tasks. For each benchmark, the same 1,000 samples are used to construct the memory or retrieval corpus, and the number of retrieved memory items varies over . The test sets and other experimental settings are held fixed. RiTeK tasks report precision (), recall (), and F1 () using exact match (*) and ROUGE-1 (†). Mortality and readmission are evaluated on MIMIC-III; TCGA-Reports results are macro-averaged. Clinical prediction and pathology classification report AUPRC and AUROC. All results are percentages. Within each model and retrieval depth, bold and <u>underlined</u> values indicate the best and second-best scores among the four memory approaches, respectively. The Top-10 results reproduce the corresponding full-data baseline results.
Memory Class Distribution
Table 7 reports the effects of varying positive-to-negative class composition in the memory collection for mortality and readmission prediction.
| Method | Pos:Neg | Mortality–MIMIC-III AUPRC | Mortality–MIMIC-III AUROC | Readmission–MIMIC-III AUPRC | Readmission–MIMIC-III AUROC |
| Qwen3-30B-A3B-Instruct-2507 | |||||
| A-MEM | 10:90 | 8.57 | 54.49 | 55.06 | 49.29 |
| A-MEM | 25:75 | 9.67 | 54.66 | 57.92 | 52.23 |
| A-MEM | 50:50 | 7.11 | 49.38 | 56.82 | 51.30 |
| A-MEM | Natural | 9.37 | 51.17 | 55.01 | 49.76 |
| MemZero | 10:90 | 7.37 | 49.38 | 54.51 | 48.59 |
| MemZero | 25:75 | 7.16 | 50.46 | 53.65 | 47.97 |
| MemZero | 50:50 | 7.46 | 50.73 | 56.88 | 52.46 |
| MemZero | Natural | 7.54 | 51.35 | 57.13 | 52.33 |
| FullContext | 10:90 | 7.68 | 51.34 | 54.84 | 48.37 |
| FullContext | 25:75 | 7.72 | 50.02 | 55.15 | 48.52 |
| FullContext | 50:50 | 11.17 | 56.01 | 54.37 | 47.02 |
| FullContext | Natural | 10.27 | 52.90 | 55.50 | 49.78 |
| NaiveRAG | 10:90 | 7.77 | 50.69 | 53.72 | 47.86 |
| NaiveRAG | 25:75 | 8.44 | 55.28 | 55.29 | 50.09 |
| NaiveRAG | 50:50 | 7.93 | 52.45 | 56.44 | 49.67 |
| NaiveRAG | Natural | 9.02 | 56.57 | 57.89 | 54.70 |
| LightMem | 10:90 | 10.77 | 52.72 | 56.14 | 50.75 |
| LightMem | 25:75 | 9.16 | 53.00 | 55.65 | 50.88 |
| LightMem | 50:50 | 8.13 | 51.98 | 54.71 | 50.33 |
| LightMem | Natural | 8.88 | 52.19 | 54.76 | 49.76 |
| HuatuoGPT-3-27B | |||||
| A-MEM | 10:90 | 10.17 | 60.79 | 56.09 | 50.02 |
| A-MEM | 25:75 | 10.25 | 56.24 | 58.42 | 54.48 |
| A-MEM | 50:50 | 10.07 | 58.34 | 60.37 | 57.16 |
| A-MEM | Natural | 9.81 | 59.88 | 61.91 | 56.69 |
| MemZero | 10:90 | 8.40 | 52.58 | 58.88 | 54.62 |
| MemZero | 25:75 | 9.25 | 57.77 | 56.12 | 53.13 |
| MemZero | 50:50 | 8.99 | 56.67 | 53.26 | 48.78 |
| MemZero | Natural | 8.29 | 54.21 | 57.98 | 52.71 |
| FullContext | 10:90 | 9.88 | 56.05 | 56.05 | 50.85 |
| FullContext | 25:75 | 9.35 | 57.77 | 57.06 | 51.50 |
| FullContext | 50:50 | 10.58 | 61.17 | 55.35 | 50.40 |
| FullContext | Natural | 9.68 | 59.19 | 57.93 | 52.55 |
| NaiveRAG | 10:90 | 8.48 | 57.53 | 56.01 | 50.33 |
| NaiveRAG | 25:75 | 9.87 | 59.04 | 58.75 | 54.96 |
| NaiveRAG | 50:50 | 9.28 | 57.96 | 55.93 | 51.90 |
| NaiveRAG | Natural | 9.81 | 61.98 | 60.76 | 55.79 |
| LightMem | 10:90 | 9.72 | 58.80 | 58.32 | 52.34 |
| LightMem | 25:75 | 9.78 | 54.90 | 54.68 | 47.82 |
| LightMem | 50:50 | 10.20 | 57.32 | 58.58 | 52.89 |
| LightMem | Natural | 9.62 | 57.80 | 59.17 | 53.91 |
Table 7. Effect of class distribution on biomedical memory performance. Five memory methods are evaluated using two language-model backbones on MIMIC-III mortality and readmission prediction. For each task, 1,000 samples are selected for memory construction or context preparation under positive-to-negative ratios of 10:90, 25:75, and 50:50, or under the natural distribution without explicit class balancing. Test sets retain their original class distributions, and other experimental settings are held fixed. Performance is reported as AUPRC and AUROC in percentages. Within each backbone and class-ratio setting, the best and second-best results across methods are shown in bold and underlined, respectively. Natural rows reproduce the corresponding baseline results.
Supplementary Prompt Templates
This section provides the prompt templates used to examine task-aware evidence utilization and memory representation. The biomedical prompts compare the original and compact memory formats, while the RealMem prompts contrast relevance-ordered memory utilization with chronological state tracking.
Task-Aware Biomedical Memory Utilization
Figures Figure 8 and Figure 9 show the HMRU-I and HMRU-C prompt templates, respectively. The latter retains task-specific instructions while simplifying the presentation of retrieved memories.

Figure 8. Task-aware memory utilization instructions in HMRU-I. The adapter applies task-specific evidence utilization rules while preserving the original memory representation.

Figure 9. Compact memory representation in HMRU-C. The adapter retains the task-aware utilization instructions of HMRU-I while removing auxiliary memory metadata, including keywords, tags, and timestamps.
Long-Term Conversational Memory Prompts
Figure 10 and Figure 11 present the RealMem HMRU-I and RealMem HMRU-T prompt templates, respectively. HMRU-I presents memories by relevance, whereas HMRU-T organizes retrieved turns chronologically to support temporal state tracking.

Figure 10. RealMem HMRU-I: relevance-ordered memory utilization. Retrieved memories retain their original A-MEM representation, with task-aware instructions for personalized, temporally consistent responses.

Figure 11. RealMem HMRU-T: chronological memory organization and state tracking. The same retrieved memories are reorganized by session date and original turn order, with a compact state checklist for dialogue continuity, user preferences and temporal updates.
RealMem HMRU-T: Temporal Memory and State Tracking
System Instruction
You are the user’s long-term personal assistant (fitness, health, mental wellbeing, learning, travel and other personal plans). You talk with the same user over many sessions spread across weeks. You cannot see the earlier sessions directly; instead you receive memories retrieved from them.
Earlier Conversation
Retrieved conversation turns are grouped by session date from oldest to newest. Within each session, the original turn order is preserved.
Session: {earlier session date} [R#] User: {earlier user message} Summary: {memory context}
…
[R#] Assistant: {earlier assistant response} Summary: {memory context}
Session: {latest session date} (Most Recent Session) [R#] User: {recent user message} Summary: {memory context}
Additional retrieved turns may appear.
Current Session Date: {session date} Current User Message: {question}
Instructions
The retrieved conversation is a partial transcript organized chronologically. “User” and “Assistant” identify speakers; “…” marks omitted turns. [R#] indicates retrieval relevance, with R1 being the most relevant memory. Generated summaries provide context; rely on message content for facts.
Before answering, briefly identify three aspects of the user’s conversational state:
THREAD: {most recent discussion and next step} PROFILE: {relevant stable facts, goals, preferences and constraints} UPDATES: {changed information and latest values}
Keep these three statements short and then proceed to the response.
Reply directly to the user as the next message in the ongoing conversation. Do not mention memories, the transcript or the state checklist.
Continue the THREAD rather than restarting the conversation. Respect the PROFILE and use the latest values identified in UPDATES.
Ground the response in the user’s own plans, numbers, schedules, names, constraints and prior agreements, rather than providing generic advice.
For temporal questions, use historical session dates and the current session date.
Do not invent personal facts. If necessary information is missing, acknowledge it briefly and provide the best available assistance.
Answer every part of the current message clearly and concisely.
Your Reply
{assistant response}
Additional Related Work
Retrieval-augmented knowledge integration. Retrieval-augmented generation (RAG) conditions language-model predictions on evidence selected from external collections, providing an alternative to relying exclusively on parametric knowledge [7, 17]. Subsequent work has incorporated retrieval and evidence assessment into the generation process itself [18]. In biomedical applications, retrieved demonstrations have supported few-shot medical entity recognition [19], while external evidence has been used for healthcare prediction and cancer-related language understanding [1, 6]. Evaluations of biomedical RAG have also examined robustness to noisy evidence and the ability of models to recognize their own limitations [3]. Complementing external retrieval, K2K investigates access to knowledge within model representations through activation-guided probing and reranking [20]. These studies establish a range of mechanisms for accessing task-relevant information, but evidence availability alone does not determine whether the information is appropriate for a particular clinical question.
Structured and relational evidence retrieval. Biomedical evidence frequently involves relationships between entities, events, and attributes that cannot be adequately represented by isolated text passages. GraphRAG [21] organizes entity relationships into hierarchical community summaries for corpus-level reasoning, while knowledge-graph retrieval has also been explored for tracking changing entity states [22]. HippoRAG [23] leverages knowledge graphs and graph-based ranking for multi-hop retrieval, whereas condensed relational paths have been used to support zero-shot link prediction [24]. G-Retriever [25] identifies relevant subgraphs for question answering over textual graphs, and KG2RAG [26] connects related passages through knowledge-graph structures to improve evidence coverage. LightRAG-Driven Medical QA [27] further explores graph-based retrieval for medical question answering. RiTeK [13] evaluates complex biomedical reasoning over textual knowledge graphs, while GLEN-Bench [28] extends graph–language evaluation to nutritional-health knowledge. These studies demonstrate the value of relational structures for evidence retrieval, while leaving open how reliably retrieved information is preserved, updated, and utilized within persistent memory systems.
Adaptive retrieval and interaction-aware reasoning. Recent approaches increasingly integrate retrieval into multi-step reasoning rather than treating it as a fixed preprocessing operation. IRCoT [29] interleaves retrieval with chain-of-thought reasoning, while FLARE [30] introduces active retrieval during generation. RAG-GD [31] explores in-context optimization to adapt how frozen generators utilize retrieved evidence, whereas DeepRAG [32] investigates when additional retrieval is warranted during reasoning. RIVO [33] examines lightweight interventions that redirect agent behavior during longer research trajectories, while Adaptive-RAG [34] selects retrieval strategies according to question complexity. RICE-PO [35] develops localized learning signals for interactive retrieval policies by evaluating executable retrieval actions and their counterfactual outcomes, whereas Self-RAG [18] incorporates self-reflection to guide retrieval and generation. Together, these methods address complementary aspects of adaptive reasoning, including evidence acquisition, generator adaptation, trajectory control, and policy learning, motivating the evaluation of retrieval as part of a broader information-use process.
Implications for biomedical agentic memory. Unlike one-off retrieval, persistent memory systems may transform source records into compact representations, revise those representations across interactions, and reuse them for subsequent questions. This setting raises additional questions about whether stored content remains supported by its source, whether retrieval identifies evidence relevant to the specific biomedical fact or clinical outcome, and whether the generator uses that evidence correctly. BioMem examines these dimensions jointly across biomedical reasoning, prediction, and long-term conversational tasks. HMRU further investigates whether complementary retrieval and task-aware evidence utilization can mitigate selected limitations without updating the underlying language-model parameters.
References
- [1]Mingchen Li, Halil Kilicoglu, Hua Xu, and Rui Zhang. Biomedrag: A retrieval augmented large language model for biomedicine. Journal of Biomedical Informatics, 162:104769, 2025.
- [2]Tianyi Ma, Yiyang Li, Yiyue Qian, Jiatan Huang, Zheyuan Zhang, Zehong Wang, Chuxu Zhang, and Yanfang Ye. Openbench: A graph benchmark to combat the opioid crisis. In Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Vol. 2, pages 9534–9545, 2026.arxiv.org/abs/2602.14602
- [3]Mingchen Li, Zaifu Zhan, Han Yang, Yongkang Xiao, Huixue Zhou, Jiatan Huang, and Rui Zhang. Benchmarking retrieval-augmented large language models in biomedical nlp: Application, robustness, and self-awareness. Science Advances, 11(47):eadr1443, 2025.DOI
- [4]Huixue Zhou, Geetanjali Rajamani, Jiatan Huang, Magali Jorand-Fletcher, Yara Mohamed, Kody A DeGollier, Annette Xenopoulos-Oddsson, Erjia Cui, Carla D Zingariello, Rui Zhang, et al. Accelerating medical record data abstraction and analysis in muscular dystrophy: Large language models and international classification of diseases codes. Neurology: Clinical Practice, 15(6):e200542, 2025.
- [5]Jiatan Huang, Zheyuan Zhang, Kaiwen Shi, Yanfang Ye, and Chuxu Zhang. Evolverouter: Co-evolving routing and prompt for multi-agent question answering. arXiv preprint arXiv:2604.05149, 2026.arxiv.org/abs/2604.05149
- [6]Mingchen Li, Zaifu Zhan, Jiatan Huang, Jeremy Yeung, Kai Ding, Anne Blaes, Steven Johnson, Hongfang Liu, Hua Xu, and Rui Zhang. Cancelllm: a large language model in cancer domain. NPJ Digital Medicine, 9(1):266, 2026.DOI
- [7]Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems, 33:9459–9474, 2020.arxiv.org/abs/2005.11401
- [8]Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G Patil, Ion Stoica, and Joseph E Gonzalez. Memgpt: Towards llms as operating systems. arXiv preprint arXiv:2310.08560, 2023.arxiv.org/abs/2310.08560
- [9]Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, and Deshraj Yadav. Mem0: Building production-ready ai agents with scalable long-term memory. arXiv preprint arXiv:2504.19413, 2025.arxiv.org/abs/2504.19413
- [10]Wujiang Xu, Zujie Liang, Kai Mei, Hang Gao, Juntao Tan, and Yongfeng Zhang. A-mem: Agentic memory for llm agents. Advances in Neural Information Processing Systems, 38:17577–17604, 2026.arxiv.org/abs/2502.12110
- [11]Jizhan Fang, Xinle Deng, Haoming Xu, Ziyan Jiang, Yuqi Tang, Ziwen Xu, Shumin Deng, Yunzhi Yao, Mengru Wang, Shuofei Qiao, et al. Lightmem: Lightweight and efficient memory-augmented generation. In International Conference on Learning Representations, volume 2026, pages 98706–98729, 2026.arxiv.org/abs/2510.18866
- [12]Haonan Bian, Zhiyuan Yao, Sen Hu, Zishan Xu, Shaolei Zhang, Yifu Guo, Ziliang Yang, Xueran Han, Huacan Wang, and Ronghao Chen. Realmem: Benchmarking llms in real-world memory-driven interaction. In Findings of the Association for Computational Linguistics: ACL 2026, pages 14349–14365, 2026.
- [13]Jiatan Huang, Mingchen Li, Zonghai Yao, Dawei Li, Yuxin Zhang, Zhichao Yang, Yongkang Xiao, Feiyun Ouyang, Xiaohan Li, Shuo Han, et al. Ritek: A dataset for large language models complex reasoning over textual knowledge graphs in medicine. In Findings of the Association for Computational Linguistics: ACL 2026, pages 35985–36001, 2026.
- [14]Alistair EW Johnson, Tom J Pollard, Lu Shen, Li-wei H Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark. Mimic-iii, a freely accessible critical care database. Scientific data, 3(1):1–9, 2016.DOI
- [15]Jenna Kefeli and Nicholas P. Tatonetti. TCGA-Reports: A machine-readable pathology report resource for benchmarking text-based AI models. Patterns, 5(3):100933, 2024. doi: 10.1016/j.patter.2024.100933.DOI
- [16]Mingchen Li, Rohan Pandey, Junhui Qian, Feiyun Ouyang, Sunjae Kwon, Avijit Mitra, Zonghai Yao, and Hong Yu. Overdoseomoe: A multi-expert framework for opioid overdose risk prediction. arXiv preprint arXiv:2609.40108, 2026.arxiv.org/abs/2609.40108
- [17]Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen tau Yih. Dense passage retrieval for open-domain question answering. In EMNLP, 2020.arxiv.org/abs/2004.04906
- [18]Akari Asai, Zeqiu Wu, Yizhong Wang, Aviprup Sil, and Hannaneh Hajishirzi. Self-rag: Learning to retrieve, generate, and critique through self-reflection. In ICLR, 2024.arxiv.org/abs/2310.11511
- [19]Mingchen Li, Huixue Zhou, Han Yang, and Rui Zhang. Rt: a retrieving and chain-of-thought framework for few-shot medical named entity recognition. Journal of the American Medical Informatics Association, 31(9):1929–1938, 2024.DOI
- [20]Mingchen Li, Jiatan Huang, Zonghai Yao, and Hong Yu. Efficient and effective internal memory retrieval for llm-based healthcare prediction. In Findings of the Association for Computational Linguistics: ACL 2026, pages 35895–35906, 2026.
- [21]Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, Dasha Metropolitan sky, Robert Osazuwa Ness, and Jonathan Larson. From local to global: A graph rag approach to query-focused summarization. arXiv preprint arXiv:2404.16130, 2024.arxiv.org/abs/2404.16130
- [22]Mingchen Li and Lifu Huang. Understand the dynamic world: An end-to-end knowledge informed framework for open domain entity state tracking. In Proceedings of the 46th international ACM SIGIR conference on research and development in information retrieval, pages 842–851, 2023.arxiv.org/abs/2304.13854
- [23]Bernal J Gutiérrez, Yiheng Shu, Yu Gu, Michihiro Yasunaga, and Yu Su. Hipporag: Neurobiologically inspired long-term memory for large language models. Advances in neural information processing systems, 37:59532–59569, 2024.arxiv.org/abs/2405.14831
- [24]Mingchen Li, Chen Ling, Rui Zhang, and Liang Zhao. A condensed transition graph framework for zero-shot link prediction with large language models. arXiv preprint arXiv:2402.10779, 2024.arxiv.org/abs/2402.10779
- [25]Xiaoxin He, Yijun Tian, Yifei Sun, Nitesh V Chawla, Thomas Laurent, Yann LeCun, Xavier Bresson, and Bryan Hooi. G-retriever: Retrieval-augmented generation for textual graph understanding and question answering. Advances in Neural Information Processing Systems, 37:132876–132907, 2024.arxiv.org/abs/2402.07630
- [26]Xiangrong Zhu, Yuexiang Xie, Yi Liu, Yaliang Li, and Wei Hu. Knowledge graph-guided retrieval augmented generation. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 8912–8924, 2025.arxiv.org/abs/2502.06864
- [27]Rishabh Kushwaha, Reshma Swain, and Bal Krishna Saraswat. Lighttrag-driven medical qa: Leveraging domain-specific for efficient llm reasoning. In 2025 5th International Conference on Advancement in Electronics & Communication Engineering (AECE), pages 685–689. IEEE, 2025.DOI
- [28]Jiatian Huang, Zheyuan Zhang, Tianyi Ma, Mingchen Li, Yaning Zheng, Yanfang Ye, and Chuxu Zhang. Glen-bench: A graph-language based benchmark for nutritional health. arXiv preprint arXiv:2601.18106, 2026.arxiv.org/abs/2601.18106
- [29]Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions. In Proceedings of the 61st annual meeting of the association for computational linguistics (volume 1: long papers), pages 10014–10037, 2023.arxiv.org/abs/2212.10509
- [30]Zhengbao Jiang, Frank F Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi-Yu, Yiming Yang, Jamie Callan, and Graham Neubig. Active retrieval augmented generation. In Proceedings of the 2023 conference on empirical methods in natural language processing, pages 7969–7992, 2023.
- [31]Mingchen Li, Jiatian Huang, Chuxu Zhang, Liang Zhao, and Hong Yu. In-context optimization for retrieval-augmented generation: A gradient-descent perspective. arXiv preprint arXiv:2605.26356, 2026.arxiv.org/abs/2605.26356
- [32]Xinyan Guan, Jiali Zeng, Fandong Meng, Chunlei Xin, Yaojie Lu, Hongyu Lin, Xianpei Han, Le Sun, and Jie Zhou. Deeprag: Thinking to retrieve step by step for large language models. In International Conference on Learning Representations, volume 2026, pages 117056–117071, 2026.arxiv.org/abs/2502.01142
- [33]Mingchen Li, Jiatian Huang, and Hong Yu. Rivo: Lightweight adaptive rerouting for deep research agents. arXiv preprint arXiv:2610.15910, 2026.arxiv.org/abs/2610.15910
- [34]Soyeong Jeong, Jinheon Baek, Sukmin Cho, Sung Ju Hwang, and Jong C. Park. Adaptive-rag: Learning to adapt retrieval-augmented large language models through question complexity. In Proceedings of the 2024 conference of the north american chapter of the association for computational linguistics: Human language technologies (volume 1: Long papers), pages 7036–7050, 2024.arxiv.org/abs/2403.14403
- [35]Mingchen Li, Hansi Zeng, Zhuo Qian, Jiatian Huang, Sunjae Kwon, Hamed Zamani, and Hong Yu. Rice-po: Turning retrieval interactions into credit signals for reasoning agents. arXiv preprint arXiv:2605.26352, 2026.arxiv.org/abs/2605.26352






