betaFeedback

WorksComputer Science

Scalaris: A Scalable Enterprise Memory Architecture for Conversational AI Assistants

Abstract

Enterprise conversational systems require memory to personalize interactions, retain information across sessions, and maintain conversational coherence. At enterprise scale, however, memory must not only retrieve relevant information accurately but also provide it efficiently to the answer-generation model. Existing memory systems face a trade-off between retrieval accuracy and context efficiency. Retrieving more context can improve recall but increases inference cost and latency. We present Scalaris, a scalable enterprise memory architecture that addresses this trade-off through a unified long-term and short-term memory design. We propose that user interactions and assistant responses possess distinct characteristics and roles in a conversation: user interactions capture evolving knowledge about the user and provide value across sessions, whereas assistant responses must be stored efficiently so that only information relevant to the current query is retrieved. We therefore treat them separately and support each with specialized representations and ingestion/retrieval pipelines. Scalaris reaches 92.2 % accuracy on LongMemEval-S and 90.77 % on LoCoMo using Qwen3-32B, state-of-the-art results on both benchmarks, while supplying the answer-generation model with approximately 10 times fewer retrieved context tokens than Hindsight, a leading memory system for conversational AI. When integrated with our proprietary closed-source AI assistants, Scalaris improved overall response quality by 36 %. Together, these results establish Scalaris as a cost-effective framework for building scalable, personalized conversational AI systems.

Introduction

The emergence of large language models has enabled organizations to deploy conversational AI systems across user-facing applications such as customer support and sales assistance (Yi et al., 2026). Memory is essential to these systems for maintaining continuity across conversations and personalizing interactions (Wu et al., 2025b). However, simply providing an assistant with prior message history becomes limiting as conversations grow longer and span multiple sessions, since model performance degrades over long contexts (Liu et al., 2024). Existing memory frameworks have therefore explored mechanisms for selectively storing and retrieving information from past conversations (Zhang et al., 2025a), but they often face a trade-off between retrieval accuracy and context efficiency. Zep (Rasmussen et al., 2025), for example, retrieves relatively little context but achieves lower accuracy on long-term memory benchmarks, whereas Hindsight (Latimer et al., 2026) achieves substantially higher accuracy while retrieving considerably more. An effective enterprise memory system must therefore retrieve the information required to answer a query accurately without unnecessarily increasing the context supplied to the answer-generation model [2] (Lewis et al. 2020; Shi et al. 2023).

Conversational memory must also operate across two different timescales. Information such as user preferences, interests, and circumstances may remain useful across sessions and must evolve as new interactions occur (Xu et al., 2025; Zhang et al., 2025b), creating the need for long-term memory. At the same time, an assistant must retain the immediate conversational state required to interpret follow-up questions, resolve references, and maintain coherence within an ongoing session (Packer et al., 2024; Yu et al., 2026), motivating short-term memory. A scalable conversational memory architecture must therefore support both persistent cross-session knowledge and immediate within-session context.

Existing systems typically place all conversational turns under a shared memory representation despite the different roles of user interactions and assistant responses. User interactions contain persistent facts, preferences, and evolving relationships that remain valuable across sessions; assistant responses are largely contextual and often obsolete outside their original dialogue. User interactions therefore benefit from relational representations that capture evolving user knowledge, while assistant responses are better stored as independently retrievable atomic units, so that only the relevant portion of a response is recalled rather than the whole.

Our Proposal: We present Scalaris, a unified long-term and short-term memory architecture designed around this distinction. As illustrated in Figure 1, Scalaris maintains separate memory representations for user interactions and assistant responses. User interactions are organized in a temporal knowledge graph that maintains relevant facts, entities, and evolving user information, together with user and session summaries. Assistant responses are decomposed into independently retrievable atomic units and stored in a vector database, allowing only information relevant to the current query to be retrieved. In parallel, user interactions and assistant responses are preserved verbatim in a shared history store for recent conversational context. By providing specialized ingestion and retrieval pipelines for these different forms of memory, Scalaris aims to preserve the information needed for accurate recall while limiting unnecessary retrieved context.

Figure 1. Overview of Scalaris’s source-aware memory organization. User interactions are processed to maintain relevant facts, relevant entities, and user and session summaries in a temporal knowledge graph. Assistant responses are decomposed into independently retrievable atomic units and stored in a vector store. In parallel, both user interactions and assistant responses are preserved verbatim in a shared history store.

Our main contributions are as follows:

  1. We present Scalaris, a unified long-term and short-term memory framework for building scalable, enterprise-ready conversational AI assistants.

  1. We propose that user interactions and assistant responses exhibit distinct characteristics that call for different memory representations, and we support each with a specialized ingestion and retrieval pipeline.

  1. We achieve state-of-the-art performance on both LongMemEval-S (92.2%) and LoCoMo (90.77%) using the relatively small Qwen3-32B model, while requiring approximately $10 \times$ fewer retrieved context tokens than Hindsight.

Related Work

Long-Term and Knowledge-Graph Memory Representations. Long-term memory frameworks let conversational agents retain information across sessions through varied representations. Knowledge-graph approaches such as Zep ((Rasmussen et al., 2025)) and Supermemory ((Supermemory, 2025)) capture entities and their evolving relationships over time, while Mem0 ([2]) combines graph components with selective consolidation. LangMem ([7]) provides a general-purpose toolkit for extracting and consolidating memories. Unit-based and reflective models such as A-Mem ((Xu et al., 2025)) treat memory as interconnected units that update as new interactions occur; RGMem ((Tian et al., 2026)) derives higher-level user profiles from episodic logs, and RMM ((Tan et al., 2025)) applies reflective management to long-term dialogue. Others target efficiency through different mechanisms: SimpleMem ((Liu et al., 2026)) compresses conversations into compact semantic units, FraCom ([6]) composes fine-grained fragments at retrieval time, RecMem ([5]) optimizes memory construction cost, and Zero-Mem ((Xiao et al., 2026)) eliminates LLM calls during memory operations through deterministic memory management.

Temporal Organization and Hierarchical Architectures. Operating across immediate and multi-session contexts requires tracking temporal evolution and balancing short- against long-term stores. MemGPT ((Packer et al., 2024)) introduced hierarchical, OS-like memory management, while AgeMem ((Yu et al., 2026)) learns actions across long- and short-term tiers. To handle temporal shifts over long interaction spans, TiMem ((Li et al., 2026)), APEX-MEM ([1]), and THEA-NINE ((Ong et al., 2025)) structure information around timelines and state updates. Complementary work explores content-based categorization: Hindsight (Latimer et al., 2026) organizes memory into world knowledge, agent experience, entity observations, and beliefs. This implicitly separates assistant-originated opinions from user observations, but the division follows from epistemic role rather than a deliberate partition by the origin of the information.

Partitioning User Interactions and Assistant Responses. Despite these developments, no existing system uses the functional distinction between user interactions and assistant responses as an explicit organizing principle. User inputs call for relational, graph-based representations that track changing preferences, facts, and context across sessions, whereas assistant responses are better decomposed into independently retrievable atomic units so that downstream models receive only the relevant excerpts. Scalaris closes this gap, unifying short- and long-term stores while applying a representation tailored to each source.

Methodology

System Architecture

Scalaris is a unified long-term and short-term memory architecture designed for conversational AI assistants. As illustrated in Figure 2, the architecture is partitioned into a long-term component tailored for cross-session recall and a short-term component dedicated to intra-session continuity.

Figure 2. An illustrative instance of context retrieval in Scalaris. Long-term memory supplies the user’s preference for fuel economy together with prior Honda Civic interactions, while short-term memory supplies the Relevant Assistant Messages, session summary, and recent turns (only the latest exchange is shown, for visual clarity). Together these resolve the referent, let the agent determine that only Corolla information must be retrieved, and enable personalization by focusing on fuel economy and including trade-offs.

The long-term component is built on Graphiti (Rasmussen et al., 2025; Zep AI, 2024), an open-source temporal knowledge graph framework, which we substantially modify for use as a memory layer. It consists of a user summary and Relevant user interactions. The user summary maintains an evolving, holistic profile of the user’s attributes, behavioral patterns, and focus areas. It is updated incrementally after each interaction and periodically rebuilt from historical logs to prevent semantic drift. Relevant user interactions extract three elements from user messages: Relevant Facts, which capture temporal semantic data; Relevant Entities, which represent previously discussed people, organizations, or concepts alongside their context; and Relevant Past Messages, which preserve the verbatim user interactions.

The short-term component consists of a session summary, the Last-N Exchanges, and Relevant Assistant Messages. The session summary compactly tracks the ongoing dialogue state, while the Last-N Exchanges provide the verbatim recent context required for immediate coreference resolution and intent tracking. For Relevant Assistant Messages, we leverage a vector database to store assistant responses as independently retrievable atomic units. The assistant’s outputs are fragmented into self-contained units, ensuring only informative segments are retrieved rather than full, verbose responses. We intentionally keep Relevant Assistant Messages in short-term memory for two reasons. First, assistant responses are often not useful across sessions, and enabling cross-session retrieval increases retrieved context with limited benefit. Second, in domains with rapidly changing information like finance, retrieving stale assistant responses increases the likelihood of hallucinations. This behavior is nonetheless configurable for applications that require assistant responses to persist across sessions. Figure 3 details the complete ingestion and retrieval pipeline.

Figure 3. Flow diagram of the ingestion and retrieval pipeline of Scalaris. The ingestion phase organizes user interactions as facts, entities, and summaries in the graph store, assistant responses as atomic units in the vector store, and stores each incoming user query and AI response verbatim in the history store. The retrieval pipeline begins by reformulating the incoming query, which is then used to retrieve information from the graph and vector stores. Summaries are fetched directly from the graph store, and the NN most recent exchanges from the history store. All retrieved components are combined during context assembly and passed to the answering system.

Ingestion

Relevant user interactions. Each incoming user message is ingested as an episode from which entities, facts, and temporal information are extracted and linked to the memory graph. The memory uses a bi-temporal model that tracks both a fact’s validity interval and when it was learned or recorded, preserving historical changes. Each episode is processed as follows:

(a) Entity Extraction. Entities are extracted from each episode, and a concise summary of up to 250 characters is maintained for each entity to capture its conversational context.

(b) Entity Resolution. Extracted entities are matched against existing entities using a resolution pipeline that combines exact matching, lexical similarity, and embedding-based similarity. If a match is identified, the entity is linked to its existing graph representation; otherwise, a new representation is created.

(c) Fact Extraction. Atomic facts are extracted from the episode and represented as relationships between resolved entities, capturing the specific information conveyed by the user and its associated temporal context.

(d) Edge De-duplication. Newly extracted relationships are compared against existing edges to identify duplicate information, preventing repeated interactions from unnecessarily creating redundant relationships in the graph.

Temporal Extraction

(e) Temporal Extraction. Temporal expressions associated with extracted facts are identified and normalized to determine the validity interval of each relationship, including explicit dates as well as relative expressions found in the conversation context.

(f) Edge Invalidation. When a newly extracted fact contradicts an existing relationship, the validity of the previous edge is terminated at the point when the new fact becomes valid, rather than deleting the historical edge, preserving the evolution of the user’s information over time.

For each extracted fact, the original user message is retained alongside its corresponding memory representation, preserving the source utterance for subsequent retrieval.

User and Session Summaries. User and session summaries are maintained in the same graph-based memory layer and follow a common update strategy. Both summaries are updated incrementally as new interactions are observed and periodically reconstructed from the underlying conversation history, with interactions provided in temporal order during reconstruction. Let σt\sigma_t denote the summary at time tt, xtx_t the newly observed interaction at time tt, and HtH_t the underlying user interaction history

available at the reconstruction point:

σt={Rebuild⁡(Ht),at reconstruction intervals,Update⁡(σt−1,xt),otherwise.(1)\sigma_t = \begin{cases} \operatorname{Rebuild}(H_t), & \text{at reconstruction intervals},\\ \operatorname{Update}(\sigma_{t-1}, x_t), & \text{otherwise}. \end{cases} \tag*{(1)}

Relevant Assistant Messages. Assistant responses are fragmented into atomic memory units to retain only informative content from each response and enable retrieval of only the relevant portions when needed. This is achieved through the following steps:

(a) Contextual Windowing. The conversation is divided into fixed-size windows of dialogue, with each window providing the surrounding context required to interpret the assistant response being processed.

(b) Information Filtering. Each dialogue window is evaluated for the information content of its assistant response, and windows containing only greetings, acknowledgements, or other low-value interactions are discarded to avoid storing uninformative content.

(c) Atomic Memory Construction. Informative assistant responses are decomposed into self-contained atomic memory units with temporal awareness. During construction, references to preceding dialogue are resolved, and related information is combined when necessary to form a complete representation.

(d) Keyword and Entity Extraction. Relevant keywords and entities are extracted and stored alongside each memory unit as metadata.

Last-NN Exchanges. All user-assistant messages are stored verbatim in a history store, preserving their exact wording and ordering.

Retrieval

Query Reformulation. Queries in ongoing conversations often depend on preceding dialogue through pronouns, ellipsis, or implicit references, making them incomplete retrieval keys in isolation. Each incoming query is therefore reformulated using recent conversation history before retrieval. Given the current query qtq_t and recent conversation history HtrecentH_t^{\mathrm{recent}}, a reformulation function produces a self-contained query

qt∗=R(qt,Htrecent).(2)q_t^* = R(q_t, H_t^{\mathrm{recent}}). \tag*{(2)}

The reformulation resolves references, restores omitted subjects, and makes implicit continuations explicit. If qtq_t is already self-contained, it is retained unchanged, such that qt∗=qtq_t^* = q_t. The resulting query qt∗q_t^* is used for all subsequent retrieval operations.

Relevant user interactions. Relevant user interactions are retrieved through a three-stage search, reranking, and context-construction pipeline.

(a) Search. The reformulated query is used to retrieve candidate facts and entities through semantic, lexical, and graph-traversal search. Semantic search uses dense embedding similarity, computed with Cohere’s embed-english-v3.0 model [3] to identify facts and entities that are semantically related to the query. Lexical search uses BM25 (Robertson and Zaragoza, 2009) over fact text and entity names to recover exact terms, names, and rare expressions. Graph traversal complements these searches by starting from seed entities and exploring a bounded number of graph hops, retrieving related facts that may not be lexically or semantically similar to the query.

(b) Reranking. Candidate results are reranked using Reciprocal Rank Fusion (RRF) [4], which combines independently ranked lists without requiring their scores to be directly comparable. For a candidate dd, the fused scores for facts and entities are computed as:

Rfact(d)=∑r∈{sem,lex,graph}1k+rank⁡r(d),(3)R_{\mathrm{fact}}(d) = \sum_{r \in\{\mathrm{sem},\mathrm{lex},\mathrm{graph}\}} \frac{1}{k + \operatorname{rank}_r(d)}, \tag*{(3)}
Rentity(d)=∑r∈{sem,lex}1k+rank⁡r(d),(4)R_{\mathrm{entity}}(d) = \sum_{r \in\{\mathrm{sem},\mathrm{lex}\}} \frac{1}{k + \operatorname{rank}_r(d)}, \tag*{(4)}

where dd denotes a candidate result, rr denotes a retrieval ranking, rank⁡r(d)\operatorname{rank}_r(d) is the rank of dd in ranking rr, and kk is the RRF smoothing constant, which we set to the default value of k=60k = 60. This stage improves precision by retaining the highest-ranked candidates across complementary retrieval mechanisms.

(c) Context Construction. The reranked results are converted into the context supplied to the answering system. Each relevant fact is represented with its descriptive text, while each relevant entity is represented by its name and concise summary. Relevant Past Messages can also be retrieved from the metadata of relevant edges to provide the assistant with the exact user utterances from which the corresponding knowledge was derived.

Relevant Assistant Messages.

(a) Query Planning and Decomposition. Before retrieval begins, the reformulated query is passed to a single LLM call that performs all query-planning operations required by the subsequent retrieval stages. The LLM analyzes the information need and simultaneously generates one or more self-contained query phrasings for semantic retrieval, extracts keywords for lexical retrieval, and identifies attributes—including persons, locations, entities, and temporal expressions—for symbolic retrieval. Relative temporal expressions are resolved into absolute time ranges using the current date and time as a reference.

(b) Hybrid Retrieval. The three retrieval views execute concurrently, each using its corresponding query representation. Semantic retrieval ranks memory units by embedding similarity, lexical retrieval uses BM25 to identify exact or closely matching terms, and symbolic retrieval filters memory units using stored attributes such as persons, locations, entities, and temporal ranges:

Rsem=TopK⁡nsem{mi∣cos⁡(E(qsem),E(mi))≥τsem},Rlex=TopK⁡nlex{mi∣BM25⁡(qlex,mi)≥τlex},Rsym=TopK⁡nsym{mi∣Meta⁡(mi)⊨qsym}.(5)\begin{aligned} R_{\mathrm{sem}} &= \operatorname{TopK}_{n_{\mathrm{sem}}}\left\{m_i \mid\cos\left(E(q_{\mathrm{sem}}),E(m_i)\right) \ge\tau_{\mathrm{sem}}\right\},\\ R_{\mathrm{lex}} &= \operatorname{TopK}_{n_{\mathrm{lex}}}\left\{m_i \mid\operatorname{BM25}(q_{\mathrm{lex}},m_i) \ge\tau_{\mathrm{lex}}\right\},\\ R_{\mathrm{sym}} &= \operatorname{TopK}_{n_{\mathrm{sym}}}\left\{m_i \mid\operatorname{Meta}(m_i) \models q_{\mathrm{sym}}\right\}. \tag*{(5)} \end{aligned}

Here, mim_i denotes a candidate memory unit; qsemq_{\mathrm{sem}}, qlexq_{\mathrm{lex}}, and qsymq_{\mathrm{sym}} denote the query representations generated for semantic, lexical, and symbolic retrieval, respectively; E(⋅)E(\cdot) denotes the embedding function; nsemn_{\mathrm{sem}}, nlexn_{\mathrm{lex}}, and nsymn_{\mathrm{sym}} specify the number of candidates retained from each retrieval view; and τsem\tau_{\mathrm{sem}} and τlex\tau_{\mathrm{lex}} are the minimum similarity and BM25 score thresholds, respectively. Meta⁡(mi)⊨qsym\operatorname{Meta}(m_i) \models q_{\mathrm{sym}} indicates that the stored metadata of mim_i satisfies the constraints specified by qsymq_{\mathrm{sym}}.

Duplicate memory units returned by multiple views are removed using their unique identifiers, producing a single consolidated set of Relevant Assistant Messages.

Summaries and Last-NN Exchanges. The user and session summaries are directly fetched from their corresponding graph nodes, while the most recent NN user-assistant exchanges are fetched from the history store verbatim.

Context Assembly. Finally, the outputs of all memory components are concatenated and passed to the answering system.

Experiments and Results

Experimental Setup

Datasets

We evaluate Scalaris on two long-term conversational memory benchmarks: LongMemEval-S (Wu et al., 2025a), comprising 500 questions across six evaluation categories with approximately 115K input tokens per question, and LoCoMo (Maharana et al., 2024), comprising 1,986 questions across 10 multi-session conversations and five categories, with approximately 9K tokens per conversation. For LoCoMo, we follow the category labels and question distribution provided by the Hugging Face version of the dataset.

Evaluation Metrics

To assess both the effectiveness and practical viability of the memory systems, we measure accuracy (%), the primary metric for retrieval and reasoning success across all benchmark categories; retrieval efficiency, the average number of context tokens retrieved per query; and latency, the average retrieval time per query in seconds.

Implementation Details

For the end-to-end memory system, we consider Qwen3-32B (Qwen Team, 2025) and OSS-120B (OpenAI, 2025) as the underlying models. Following benchmark protocols, GPT-4o (OpenAI, 2024) is used as the answer generator and evaluator.

Because these benchmarks primarily evaluate long-term memory recall and reasoning rather than conversational continuity, we exclude the session summary and Last-NN Exchanges during evaluation. We configure Relevant Assistant Messages for cross-session retrieval to align with the benchmarks’ requirements. All other components, including the proposed ingestion and retrieval pipelines, remain unchanged from the core architecture. The recommended hyperparameter ranges and sensitivity analysis are presented in the extended analysis in Appendix § 9.§ 2. Baseline results in Table 1 and Table 2 are taken from Latimer et al. [8].

TypeZep GPT-4oSuperm. GPT-5Superm. Gemini-3Hinds. OSS-120BHinds. Gemini-3Scalaris Qwen3-32BScalaris OSS-120B
SSU92.997.198.6100.097.197.1495.71
SSA80.4100.098.298.296.483.9385.71
SSP56.776.770.086.780.093.3380.0
KU83.387.289.792.394.989.7489.74
TR62.481.282.085.791.093.9890.98
MS57.975.276.781.287.292.4892.48
Overall71.284.685.289.091.492.290.6

Table 1. Performance comparison on LongMemEval-S (% Accuracy). SSU: single-session-user; SSA: single-session-assistant; SSP: single-session-preference; KU: knowledge-update; TR: temporal-reasoning; MS: multi-session. Superm.: Supermemory; Hinds.: Hindsight. (Bold: best; underline: second best)

CategoryZepMem0-GraphMem0LangMemHindsight Gemini-3Scalaris Qwen3-32B
S-Hop74.1165.7167.1362.2386.1789.36
M-Hop66.0447.1951.1547.9270.8390.34
O-Domain67.7175.7172.9371.1295.1293.10
Temporal79.7958.1355.5123.4383.8076.04
Overall75.1468.4466.8858.1089.6190.77

Table 2. Performance comparison on LoCoMo (% Accuracy). S-Hop: Single-Hop; M-Hop: Multi-Hop; O-Domain: Open Domain.

Results

Performance on LongMemEval-S

Table 1 compares Scalaris with existing memory systems across six categories of LongMemEval-S.

Scalaris achieves the highest overall accuracy of 92.2%, outperforming all compared systems while utilizing an average of only 4,116 context tokens per query and maintaining a 0.94 s retrieval latency. It achieves state-of-the-art performance in three critical categories: SSP (93.33%), TR (93.98%), and MS (92.48%). Notably, this 92.2% overall accuracy is achieved with Qwen3-32B, a substantially smaller model than those used by the competing baselines. When paired with OSS-120B, Scalaris still outperforms most baseline configurations with 90.6% accuracy, including Hindsight with the same backbone (89.0%), while using 3,866 context tokens per query and 1.04 s latency. Failure cases are analyzed in Appendix § 9.§ 3.

Performance on LoCoMo

As most baseline frameworks do not evaluate adversarial questions, Table 2 reports the four standard LoCoMo categories for direct comparison, with adversarial results detailed in the text.

Scalaris yields the highest overall accuracy of 90.77% among the compared systems and dominates the S-Hop (89.36%) and M-Hop (90.34%) categories. When the adversarial subset is included in the evaluation, Scalaris’s overall accuracy rises to 92.55%, driven by a 98.65% accuracy on adversarial questions. On this benchmark, Scalaris uses an average of 4,145 context tokens per query with a retrieval latency of 0.98 s.

Accuracy–Efficiency Trade-off

Accuracy alone does not fully characterize the practical effectiveness of a memory system, as retrieved context length directly dictates enterprise inference costs. We evaluate this trade-off by comparing Scalaris with Hindsight (Table 3). Because Hindsight publicly reports detailed retrieval metrics (Vectorize, 2026), the figures in Table 3 utilize these publicly reported values for a strictly fair comparison (and thus differ slightly from baseline figures in Table 1 and Table 2, which were taken from Latimer et al. [8]).

MethodLME Acc. (%)LME TokensLoCoMo Acc. (%)LoCoMo Tokens
Hindsight94.643,62592.036,235
Scalaris92.24,11690.774,145

Table 3. Tokens vs. accuracy (LME = LongMemEval).

Scalaris achieves comparable accuracy to Hindsight while utilizing approximately 10× fewer retrieved context tokens on both benchmarks. Neither baseline distinguishes user interactions from assistant responses, and each sits at one end of the trade-off: Zep organizes all conversational content in a single knowledge graph, retrieving exceptionally lightweight contexts (averaging 1.6K tokens) at the cost of severe accuracy degradation (71.2% on LongMemEval) (Rasmussen et al., 2025), whereas Hindsight stores all content in a single PostgreSQL-backed memory bank (Latimer et al., [8]), securing high accuracy but demanding massive context windows. By representing each source according to its role, Scalaris attains comparable accuracy at a fraction of the context, validating our design choice to structurally differentiate user interactions from assistant responses.

Ablation Study

We conduct an ablation study to isolate the impact of our dual-representation memory strategy. All experiments use Qwen3-32B as the underlying model, with retrieval hyperparameters held constant across configurations. We compare four configurations by alternating the memory structures used for user interactions and assistant responses. The Graph configuration utilizes the ingestion and retrieval pipeline designed for Relevant user interactions, while the VectorDB configuration follows the pipeline for Relevant Assistant Messages. Table 4 reports the results across both benchmarks.

Memory organizationLongMemEval-SLoCoMo
User interactionsAssistant responsesAcc.Avg. tokensAcc.Avg. tokens
GraphGraph76.25,53273.55,729
VectorDBGraph79.36,26777.26,183
VectorDBVectorDB86.38,72988.58,325
GraphVectorDB92.24,11690.774,145

Table 4. Ablation study of memory organization on LongMemEval-S and LoCoMo (% Accuracy). The last row is the proposed Scalaris organization.

The results demonstrate that memory organization plays a critical role in retrieval effectiveness and efficiency. Utilizing a graph-based structure for assistant responses yields poor accuracy despite retrieving more context, suggesting that relational graphs are not well suited for assistant responses. Conversely, using a VectorDB for both information streams provides substantially better accuracy, indicating that flat storage serves as a strong baseline. However, this homogeneous approach more than doubles the required context window while still underperforming the proposed architecture.

Ultimately, by organizing user interactions via a temporal graph and assistant responses as atomic units in a VectorDB, Scalaris achieves state-of-the-art performance while requiring substantially fewer context tokens. These findings validate our core hypothesis: user and assistant messages exhibit distinct characteristics and require tailored memory representations. Additional retrieval-component ablations and hyperparameter sensitivity analyses are reported in Appendices § 9.§ 1 and § 9.§ 2.

Internal Evaluation

To validate Scalaris in a live production environment, we integrated the architecture into our proprietary, closed-source conversational AI assistants and evaluated its performance on a sample of 1,200 real user interactions. For each query, we generated two responses: one with memory (WM) utilizing all Scalaris components, and one without memory (WOM) relying solely on the Last-N Exchanges.

Using a blind evaluation protocol, GPT-4o scored each response on a 0–5 scale considering five dimensions: answer quality, personalization, clarity, actionability, and context consistency. Memory-enabled responses were preferred in 91.9% of cases, achieving a mean score of 4.33 compared to 3.18 for the WOM baseline (+36.2%). For the evaluation prompt and extended analysis, refer to the Appendices § 9.§ 4 and § 9.§ 5.

Notably, the performance gap widened significantly as dialogue length increased. For queries in the latter half of extended sessions (defined as sessions exceeding 20 turns, representing 433 queries in our dataset), WM responses averaged 4.62, while WOM responses dropped sharply to 2.40. This confirms Scalaris’s ability to maintain conversational coherence in long-horizon interactions.

Conclusion

We present Scalaris, a unified long-term and short-term memory architecture designed to address the diverse memory requirements of conversational AI systems. Scalaris enables agents to personalize responses based on users’ preferences and interests, recall relevant information with temporal awareness, and maintain coherence across long and complex conversations. By appropriately organizing user interactions and assistant responses according to their characteristics, Scalaris achieves state-of-the-art performance on both LongMemEval-S and LoCoMo. Notably, these results are achieved with a relatively small Qwen3-32B model while requiring approximately 10×10 \times fewer retrieved context tokens than Hindsight with comparable accuracy. Internal evaluation further demonstrates significant improvements in response quality when Scalaris is integrated with conversational AI assistants, highlighting its potential as a practical and scalable framework for enterprise conversational AI systems.

Limitations

The current Scalaris architecture is restricted to declarative and episodic memory—such as user preferences, explicit facts, entities, and past interactions—and lacks dedicated mechanisms for modeling procedural knowledge like learned skills, workflows, or reusable action sequences. Furthermore, Scalaris is designed exclusively for text-based interfaces and has not been evaluated in voice-driven environments, which require strict real-time latency, continuous streaming, and low-overhead retrieval. Future Work: To address these constraints, future research will extend the framework to incorporate procedural memory capabilities while optimizing retrieval pipelines and streaming efficiency to support ultra-low-latency voice interactions.

Ethical Considerations

While Scalaris enhances personalization and continuity in conversational AI, persistent memory architectures introduce potential privacy and security risks. Long-term memory stores sensitive user information across sessions, making data protection critical. To mitigate these risks, production deployments must incorporate robust access controls, data anonymization, and user-driven mechanisms for inspecting, editing, or deleting stored memory entries (aligning with right-to-be-forgotten regulations). Furthermore, because long-term memory retains user preferences and traits, systems must safeguard against amplifying user biases or unintentionally storing personally identifiable information (PII).

Artifacts and Licenses. Scalaris builds on Graphiti (Rasmussen et al., 2025; Zep AI, 2024), released under the Apache-2.0 license. We evaluate on LongMemEval (Wu et al., 2025a) (MIT license) and LoCoMo (Maharana et al., 2024) (CC BY-NC 4.0), both used solely for non-commercial research evaluation in accordance with their licenses. We verified that these licenses permit our use and the planned release of our code, which will retain the original license and copyright notices; benchmark data will not be redistributed. No internal user data will be released.

Use of AI Assistants

AI-based writing tools were used exclusively for grammar correction, proofreading, and language polishing of author-drafted text. All core concepts, methodology, experiments, analyses, and final phrasing remain the original work of the authors, who take full responsibility for the content of this manuscript.

Extended Analysis

To rigorously evaluate the individual contributions of Scalaris’s modules and assess its robustness to hyperparameter variations, we present an extended analysis encompassing fine-grained ablations, sensitivity experiments, and a qualitative error breakdown.

Retrieval Component Analysis

To better understand the contribution of individual retrieval mechanisms, we evaluate the retrieval pipelines of Relevant Assistant Messages and Relevant User Interactions independently on the corresponding LongMemEval categories. Specifically, the assistant-response retrieval pipeline is evaluated only on the single-session-assistant category, while the user-interaction retrieval pipeline is evaluated only on the single-session-user category. These subsets isolate questions whose answers primarily depend on assistant responses or user interactions, respectively, allowing each retrieval component to be analyzed without interference from the other memory stream. Also, these questions require information distributed across multiple human-AI exchanges within one session, where omitting even a single relevant memory unit can lead to an incomplete or incorrect response. During each experiment, only the retrieval mechanism under evaluation is enabled while all other retrieval components within the same pipeline are disabled. Since graph traversal (BFS) requires an initial set of retrieved entities, it is evaluated only in combination with semantic retrieval or BM25 rather than as an independent retrieval mechanism. For reference, the complete retrieval pipeline achieves 83.93% accuracy on single-session-assistant and 97.14% on single-session-user.

The single-session-assistant (SSA) gap relative to some baselines in Table 1 reflects a deliberate design trade-off. The SSA subset contains 56 questions, of which Scalaris answers 47 correctly (83.93%). As noted above, these questions require combining details from several assistant responses within a session, so an answer is correct only if every relevant response is retrieved; missing even one yields an incomplete answer. This setting naturally favors systems that retrieve large portions of the conversation: Hindsight, for instance, retrieves roughly 10×10 \times more context than Scalaris (Table 3). Scalaris instead retrieves compact atomic units under a fixed retrieval budget, and among low-context systems it remains stronger than Zep (83.93% vs. 80.4%).

As shown in Table 5, semantic retrieval substantially outperforms lexical retrieval by capturing paraphrases and semantically related assistant responses that do not necessarily share surface-level vocabulary with the query. Lexical retrieval nevertheless remains valuable for recalling exact phrases, named entities, and numerical information, but its reliance on token overlap limits overall recall. Symbolic retrieval alone performs considerably worse because metadata filtering narrows the search space but cannot identify semantically related assistant responses, making it insufficient as a standalone retrieval strategy. Increasing the retrieval budget consistently improves performance for semantic and lexical retrieval until the optimal value, after which the gains become negligible despite retrieving additional memory units. Symbolic retrieval saturates even earlier because the number of metadata-matching candidates is inherently limited. These observations suggest that semantic, lexical, and symbolic retrieval capture complementary characteristics of assistant responses, motivating their integration in the final retrieval pipeline while avoiding unnecessary retrieval beyond the optimal budget.

Retrieval Methodk=5k=5k=10k=10k=15k=15k=20k=20k=25k=25
Semantic Search52.162.369.269.769.9
Lexical Search44.149.254.755.255.3
Symbolic Search36.737.737.837.837.8

Table 5. Retrieval analysis for Relevant Assistant Messages (Single-Session-Assistant). The optimal retrieval budget for each method (the elbow point beyond which gains become negligible) is shown in bold.

As reported in Table 6, semantic retrieval consistently outperforms BM25 because user queries often refer to previously discussed facts using different wording, allowing dense retrieval to identify semantically related interactions that lexical matching fails to capture. However, BM25 remains valuable for queries involving exact entity names, identifiers, or uncommon user-specific terminology, where lexical overlap provides a stronger retrieval signal than semantic similarity. Incorporating graph traversal improves both retrieval strategies by following the relationships of the initially retrieved entities and bringing together related facts that are distributed across multiple user interactions. The improvement is considerably larger when combined with semantic retrieval, as semantically retrieved entities provide more relevant starting points for graph traversal, whereas BM25 already retrieves highly specific matches that leave less additional information to be discovered through the graph. Similar to the assistant-response pipeline, increasing the retrieval budget consistently improves performance until the optimal value, after which the gains become negligible despite retrieving additional facts. Taken together, these observations suggest that the three retrieval mechanisms capture different aspects of user memory, motivating their integration in the final retrieval pipeline.

Retrieval Methodk=5k=5k=10k=10k=15k=15k=20k=20k=25k=25
Semantic Search63.871.977.279.679.8
BM2559.466.872.572.973.0
Semantic + BFS74.680.984.285.885.9
BM25 + BFS67.574.879.880.380.4

Table 6. Retrieval analysis for Relevant User Interactions (Single-Session-User). The optimal retrieval budget for each method (the elbow point beyond which gains become negligible) is shown in bold.

Hyperparameter Sensitivity

We further evaluate the sensitivity of Scalaris to its primary hyperparameters. During this analysis, the retrieval budget for each retrieval mechanism is fixed to its corresponding optimal value identified in Section A.1, allowing the effect of each hyperparameter to be analyzed independently. Table 7 summarizes the evaluated ranges and the corresponding recommended values.

Hyper-parameterRange EvaluatedOptimal RangeEval. Dataset
Semantic similarity threshold0.3–0.7 (step 0.1)0.4–0.6LME
BM25 threshold0.6–1.1 (step 0.1)0.9–1.0LME
Graph traversal depth1–5 hops3 hopsLME
Last-NN Exchanges1, 3, 5, 7, 105Internal Eval.

Table 7. Evaluated ranges and recommended values for Scalaris’s primary hyperparameters. LME: LongMemEval.

Overall, Scalaris remains stable across a broad range of parameter values, indicating that its performance is not sensitive to minor variations in hyperparameter selection. We vary the semantic similarity threshold from 0.3 to 0.7 in increments of 0.1, observing consistently strong performance within the 0.4–0.6 range. Lower thresholds admit semantically weak matches, whereas higher thresholds filter relevant memories that differ in wording from the query. Similarly, we evaluate BM25 thresholds from 0.6 to 1.1, with performance remaining stable within the 0.9–1.0 range. Lower thresholds admit low-scoring lexical matches, whereas higher thresholds begin filtering lexical matches required for retrieval. Increasing the graph traversal depth improves retrieval up to three hops, after which additional hops primarily retrieve less relevant neighboring entities while increasing retrieval latency with negligible accuracy gains. Finally, evaluation on our internal benchmark shows that retaining the most recent five user-assistant exchanges provides sufficient conversational context for immediate coreference resolution, whereas larger context windows increase prompt length without improving response quality.

Qualitative Error Analysis

To understand the limitations of Scalaris, we conducted a qualitative analysis of the remaining ∼8%\sim8\% of failure cases in LongMemEval. The majority of errors stem from three distinct patterns. First, ambiguous temporal expressions (e.g., “a couple of quarters ago”) occasionally fail to resolve accurately when the surrounding context lacks explicit anchor dates. Second, distributed evidence retrieval remains challenging when the information required to answer a query is scattered across multiple distant interactions, but the initial retrieval stage fails to surface all relevant memory units needed for complete reasoning. Finally, implicit fact negation poses a challenge; when a user implicitly changes a previously stated preference without directly contradicting it, the temporal edge invalidation fails to trigger, causing the system to retrieve outdated context. Addressing these implicit reasoning bottlenecks remains a critical area for future work.

Internal Evaluation: Detailed Analysis

To complement the quantitative results presented in Section 4.4, we performed a qualitative analysis of 1,200 real user queries collected from our proprietary production finance assistants. Unlike benchmark datasets, these conversations represent genuine user interactions and therefore naturally contain incomplete follow-up questions, pronouns, ellipsis, and implicit references. Rather than restating the complete context in every turn, users progressively rely on the assistant’s conversational memory, making later queries increasingly ambiguous when viewed independently. A representative conversation from our evaluation is shown below:

  1. Compare Tata Steel and Hindalco.

  1. Which one looks better for long-term investment?

  1. Compare it with Larsen & Toubro.

  1. Which one is the weakest stock now?

  1. Replace the weakest stock.

  1. Which one looks safer?

  1. What are the downside risks?

  1. What are the upside drivers?

  1. What is the expected timeline?

  1. Summarize the key insights.

Although the first few questions explicitly identify the companies under discussion, the subsequent queries depend almost entirely on the preceding conversation. References such as “it”, “the weakest stock”, “the downside risks”, “the upside drivers”, and “the expected timeline” cannot be interpreted correctly without resolving the conversational context established in earlier turns. Similar interaction patterns were consistently observed throughout the internal evaluation, reflecting how users naturally communicate with conversational assistants rather than how benchmark datasets are typically constructed.

To understand how existing memory frameworks handle such conversations, we additionally evaluated the same production queries using Zep and Hindsight. A common limitation observed across both systems was their inability to consistently resolve conversational coreferences and implicit follow-up queries before retrieval. Retrieval was frequently performed using the surface form of the latest user query, causing relevant memories associated with previously discussed entities to be missed. In contrast, Scalaris first reformulates each conversationally dependent query into a self-contained retrieval query by resolving pronouns, omitted entities, and implicit continuations before initiating memory retrieval.

Despite this shared limitation, the two frameworks exhibited different behaviors. Zep was able to provide limited personalization through its maintained user summary, but often failed to recall previously generated assistant responses. Consequently, the downstream assistant repeatedly invoked external tools to retrieve information that had already been fetched earlier in the conversation, increasing unnecessary computation and response latency. Hindsight recalled prior assistant information more reliably by retrieving substantially larger portions of the conversation, thereby reducing repeated tool invocations. However, this came at the cost of retrieving significantly more context tokens, frequently overwhelming the answering model with redundant information. Furthermore, because Hindsight does not explicitly maintain an evolving user profile, it was generally less effective at personalizing responses according to long-term user preferences.

These observations suggest that an effective conversational memory system must adapt to the way users naturally communicate, rather than expecting them to formulate fully self-contained queries at every turn. Real-world conversations are inherently incremental, with users relying on shared conversational context through pronouns, omitted entities, and implicit references. Furthermore, accurate recall alone is insufficient for a practical memory system. It must retrieve only the information necessary to answer the current query, avoiding redundant or irrelevant context. This is particularly important in enterprise deployments, where excessive retrieval increases latency, inference costs, and can overwhelm the downstream language model, increasing the likelihood of hallucinations.

Internal Evaluation Prompt

The exact scoring prompt supplied to the GPT-4o judge for the blind pairwise evaluation described in § 4.4 is shown in Figure 4. Placeholders in braces are populated per query.

Response Evaluation Prompt

Figure 4. Evaluation prompt used by the GPT-4o judge to compare two AI assistant responses. The prompt provides the original user question, recent conversational history, and available memory context, followed by both candidate responses. The judge assesses answer quality, personalization, clarity, actionability, and consistency with the conversation before assigning a score from 0 to 5 to each response and producing a comparative verdict in a strictly specified JSON format.

References

  1. [1]Pratyay Banerjee, Masud Moshtaghi, Shivashankar Subramanian, Amita Misra, and Ankit Chadha. 2026. APEX-MEM: Agentic semi-structured memory with temporal reasoning for long-term conversational AI. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 16470–16489, San Diego, California, United States. Association for Computational Linguistics.
  2. [2]Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, and Deshraj Yadav. 2025. Mem0: Building production-ready AI agents with scalable long-term memory. In ECAI 2025: 28th European Conference on Artificial Intelligence, volume 413 of Frontiers in Artificial Intelligence and Applications, Bologna, Italy. IOS Press.arxiv.org/abs/2504.19413
  3. [3]Cohere. 2023. Introducing embed v3. https://cohere.com/blog/introducing-embed-v3. Embedding model (embed-english-v3.0).
  4. [4]Gordon V. Cormack, Charles L. A. Clarke, and Stefan Büttcher. 2009. Reciprocal rank fusion outperforms Condorcet and individual rank learning methods. In Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 758–759. ACM.DOI
  5. [5]Zijie Dai, Shiyuan Deng, Sheng Guan, Yizhou Tian, Xin Yao, Xiao Yan, and James Cheng. 2026. RecMem: Recurrence-based memory consolidation for efficient and effective long-running LLM agents. In Findings of the Association for Computational Linguistics: ACL 2026, pages 32533–32376, San Diego, California, United States. Association for Computational Linguistics.
  6. [6]Cai Ke, Yiming Du, Bin Liang, Yifan Xiang, Lin Gui, Zhongyang Li, Baoyun Wang, Yue Yu, Hui Wang, Kam-Fai Wong, and Ruifeng Xu. 2025. Flexibly utilize memory for long-term conversation via a fragment-then-compose framework. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 21119–21136, Suzhou, China. Association for Computational Linguistics.
  7. [7]LangChain. 2025. LangMem: Long-term memory for AI agents. https://github.com/langchain-ai/langmem. Open-source software.
  8. [8]Chris Latimer, Nicolò Boschi, Andrew Neeser, Chris Bartholomew, Gaurav Srivastava, Xuan Wang, and Naren Ramakrishnan. 2025. Hindsight is 20/20: Building agent memory that retains, recalls, and reflects. Preprint, arXiv:2512.12818.arxiv.org/abs/2512.12818

Paper details

vlnc:W_7emq6q8n
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 1

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

AI-based writing tools

View statement
Looking for related papers…
Contents