betaFeedback

WorksComputer Science

CoLink: Global Compression and Knowledge Linking for Multi-Turn Dialogue Modeling

Abstract

Extracting salient information from long-horizon dialogue histories remains a significant challenge for Large Language Models (LLMs) due to the restricted context windows and efficient retrieval challenges. Current methods typically rely on history organization or local token-level optimization, suffering from information loss, retrieval redundancy, and myopic attention. Inspired by the view in cognitive science that dialogue constitutes sense-making and memory association, we propose CoLink, a framework that enhances LLMs' global context awareness in domain-specific multi-turn dialogues through information linking and compression. CoLink first clusters historical contexts into topic graphs to establish cross-semantic associations with external knowledge. Guided by the adaptive control of thought, it then retrieves and activates knowledge via synergizing semantic relevance and graph-structural topology. Crucially, to achieve effective compression of history and knowledge, we ground multi-turn dialogues in information theory and derive a principled optimization objective for extracting the core sense-making information. Coupled with a competitive slot mechanism, CoLink condenses the global context into dedicated memory and knowledge slots, efficiently distilling lengthy, redundant inputs into an extremely compact latent representation. Extensive empirical results on both public and our MMD datasets validate CoLink's superiority and generalizability in knowledge linking and sense modeling.

Introduction

Language Models (LLMs) effectively capture real-world knowledge and dialogue patterns via the long-range dependency modeling of Transformer, achieving remarkable performance in summarization, comprehension, and multi-hop reasoning [11, 13, 49]. However, scaling these capabilities to long-context scenarios remains a formidable challenge. The quadratic complexity of self-attention and restricted context windows constrain full historical perception, while the accumulation of redundant information often obscures critical semantic associations [10, 1]. Consequently, efficiently extracting contextually relevant memory and knowledge from histories is crucial for advancing multi-turn understanding and generation.

Figure 1. Rigid Retrieval & Local Tokens vs. History Compression in CoLink.

Current strategies to mitigate history compression fall into two categories, as illustrated in Figure 1. (1) Structured Organization and Retrieval methods organize segmented history into memory banks [14], trees [4, 15], or graphs [6, 3, 5] for retrieval. Despite enabling long-term storage, these methods often suffer from information loss during construction and retrieval noise during generation. (2) Local Token Modeling methods approximate full-history attention by retaining a sparse subset of tokens, such as attention sinks or recent windows [8, 10, 1]. Although computationally efficient, they tend to be myopic, failing to capture global semantic dependencies and the latent thematic coherence across turns.

In contrast to these existing approaches, which focus on structural organization or local context approximation, insights from human cognition suggest an alternative paradigm for semantic organization that emphasizes abstraction, dynamic memory activation, and global coherence [17, 18]. Rather than storing verbatim history, the human brain distills interactions into abstract representations and dynamically activates relevant long-term memories to ground new inputs through the interplay of the prefrontal cortex and hippocampus. This mechanism ensures global semantic associativity and enables efficient sense-making [16].

Drawing on these insights, we propose CoLink, a novel framework that seamlessly integrates cognitive memory modeling with global information compression. CoLink operates on two key levels: (1) Adaptive associativity and activation. To simulate short-long memory associativity, CoLink organizes history into topic graphs that enable cross-semantic linking with external knowledge. Upon receiving a new dialogue turn, it identifies the relevant historical clusters and dynamically activates context-specific knowledge via an adaptive control mechanism, exploiting both semantic associations and structural linkage importance to retrieve highly pertinent facts. (2) Sense-making via compression. CoLink adopts the Information Bottleneck (IB) principle [20] to compactly represent the expansive history and activated knowledge. Guided by the objective of minimizing the mutual information (MI) between the compressed representations and the raw inputs while maximizing the MI with the target response, we derive a tri-part loss comprising compression, prediction, and reconstruction to effectively balance globality and conciseness. To guarantee extreme compression efficiency, CoLink competitively distills salient information into a limited set of latent slots. This mechanism directly abstracts compact memory and knowledge slots from the full contexts, yielding highly condensed representations optimally tailored for LLM generation.

To evaluate CoLink in complex domains requiring multi-expert reasoning, where existing benchmarks often fall short, we design a knowledge-enhanced, self-reflective multi-agent debate framework and construct MMD, a multi-turn medical dialogue dataset with 23702370 samples. Extensive experiments demonstrate that CoLink significantly enhances information retention and cross-turn consistency, achieving up to +2.59% gains over baselines even under a 32×\times compression constraint. Furthermore, additional evaluations on both open-domain dialogue and specialized disease diagnosis datasets firmly validate the strong domain universality and zero-shot generalizability of our framework. Our paper presents three main innovations:

  • To the best of our knowledge, we are the first to formulate context abstraction in multi-turn dialogues through the lens of the Information Bottleneck principle. By integrating this objective with a competitive latent slot mechanism, our framework achieves efficient and effective compression of global content.

  • We propose a cross-level linking mechanism that bridges dialogue history with external knowledge bases. With both semantic associations and graph-structural reasoning, our method dynamically activates knowledge subgraphs that are strictly relevant to the current conversational context.

  • We conduct extensive evaluations across diverse datasets using a multifaceted validation approach, including main comparisons, component/loss ablations, and in-depth effectiveness analyses. The empirical results demonstrate that CoLink exhibits clear advantages in compression efficiency, information retention, and generation accuracy.

Related Work

History Organization and Utilization

Hierarchical structures and retrieval mechanisms mitigate redundancy by selectively retaining crucial history and enabling localization of relevant context. MemoryBank [14] partitions history into chunks and retrieves relevant blocks via similarity search. RAPTOR [15] and MemTree [4] organize dialogue into hierarchical trees through recursive clustering or semantic aggregation, enabling retrieval at multiple granularities. Graph-based approaches, such as GraphRAG [3], HippoRAG [6], ArchRAG [7], KG2RAG [22], and HippoRAG2 [5], extract entity–relation graphs and leverage graph traversal to identify relevant substructures. While these methods improve scalability and retrieval efficiency, they primarily rely on sparse entity or structural cues, often discarding fine-grained contextual information. Moreover, they lack explicit modeling of inter-turn semantic dependencies, which limits holistic integration and global compression of the dialogue history.

Exploiting Potential Gains from Local Tokens

These Local Token Modeling methods are fundamentally predicated on the phenomenon of attention sparsity. Empirical observations indicate that LLMs consistently allocate disproportionate attention weights to a minimal subset of tokens, particularly the initial and the last few tokens. This phenomenon is known as the "Attention Sink" [44, 42]. Leveraging this insight, methods such as StreamLLM [8], StreamingDialogue [10], and LM-Infinite [9] persistently cache initial tokens, turn delimiters, or recent tokens to approximate full-history attention. Similarly, H2O [45] dynamically retains only a fraction of tokens with the highest accumulated attention scores. Chen et al. [1] introduce core context-aware attention, which partitions long contexts into groups and preserves only a single representative token per group.

However, recent studies [41, 43] reveal that these sinks predominantly serve as structural biases to maintain attention stability rather than as repositories of core semantics. By confining the receptive field to local windows or sparse anchors, these methods inevitably fragment the historical continuity. Consequently, they inherently lack global perception, leading to degraded performance in tasks that necessitate deep, long-horizon understanding.

Cognitive Memory Mechanisms

Cognitive neuroscience has extensively investigated the mechanisms of memory retention, retrieval, and knowledge representation. Advanced cognitive abilities rely not merely on the synergy among various memory systems, but fundamentally on the high-level abstraction of information [18, 46]. Regarding memory architecture, Squire’s taxonomy classically distinguishes declarative memory (comprising semantic facts and episodic events) from non-declarative memory (e.g., procedural skills) [38]. Furthermore, research posits that semantic networks serve as the critical interface between sensory inputs and consolidated long-term knowledge [39]. During dialogue processing, cognitive theories suggest that novel linguistic stimuli trigger a complex complementary interaction [18]: the hippocampus acts as a rapid episodic index, which subsequently activates long-term semantic knowledge in the neocortex via associative retrieval. These activated memories are then integrated within short-term working memory to facilitate coherent language generation.

Motivated by these insights, recent studies have explored brain-inspired architectures for dialogue systems. MGMR simulates the memory-stage theory utilizing memory graphs and rehearsal mechanisms [16], while HippoRAG models dialogue history through graph structures to enhance retrieval [6]. Nevertheless, these approaches predominantly focus on memory storage structures and flow pathways, largely overlooking a fundamental characteristic of semantic memory: its high degree of abstraction. Consequently, they lack principled mechanisms to distill abstract semantic representations from redundant signals to prevent information overload.

In this work, we bridge cognitive science and information theory to model multi-turn dialogues. We conceptualize the external knowledge base as long-term semantic memory. To simulate neural retrieval interactions, we construct a historical topic graph that functions as a hippocampal-like "memory index" to precisely activate context-relevant external knowledge. Crucially, we introduce IB-driven objectives to emulate the brain’s inherent semantic abstraction process. This objective rigorously distills core information from complex, intertwined history and knowledge streams, ensuring that the response generation relies exclusively on the most critical semantic features. The conceptual diagram of CoLink is visually summarized in Figure 2.

Theoretical Framework and Objective Derivation

In this section, we establish the foundational objectives and theoretical formulations of our approach. We first formalize the multi-turn dialogue task for LLMs and introduce the preliminaries of the Information Bottleneck (IB) principle [24]. Building upon this, we extend the IB framework to multi-turn dialogues and rigorously derive the theoretical objectives for the global compression of dialogue history and associated knowledge.

Problem Definition

For the multi-turn dialogue task, the objective of LLMs is to generate a response given the historical dialogue H=[u1,…,ut−1]H = [u_1, \dots, u_{t-1}] and the current query Q=utQ = u_t. Through autoregressive prediction, the model maximizes the joint distribution of the target sequence Y=(y1,y2,…,ym)Y = (y_1, y_2, \dots, y_m). Furthermore, considering that domain-specific conversations (e.g., healthcare) often require auxiliary background knowledge KK, the objective can be formulated as:

Lgen=max∑i=1mlog⁡P(yi∣H,K,ut,y<i)\begin{equation} \mathcal{L}_{gen}=max \displaystyle\sum_{i=1}^m \log P(y_{i}|H,K,u_t,y_{<{i}}) \tag*{(1)} \end{equation}

where y<i=(y1,…,yi−1)y_{<i} = (y_1, \dots, y_{i-1}) denotes the previously generated tokens. While LLMs rely on attention mechanisms to model long-range dependencies, it remains challenging to assign precise attention weights to the most relevant historical and knowledge contexts as the dialogue progresses, amid extensive conversational noise.

Information Bottleneck Preliminary

The Information Bottleneck principle provides a rigorous framework for learning an intermediate representation ZZ of input signals HH that preserves maximal information relevant to the target variable YY. The fundamental objective is shown in:

minLIB=I(Z;H)−βI(Z;Y)\begin{equation} min \mathcal{L}_{IB}=I(Z;H)-\beta I(Z;Y) \tag*{(2)} \end{equation}

where I(⋅)I(\cdot) denotes mutual information and β\beta is a Lagrange multiplier. For variables h∈Hh \in H and z∈Zz \in Z, the mutual information term can be expressed via Kullback-Leibler (KL) divergence:

\begin{equation} \resizebox{0.8\linewidth}{!}{$ \begin{aligned} I(Z;H) &= D_{KL}[p(z,h) \| p(z) \otimes p(h)] \\ &= \int_z \int_h p(z,h) \log \frac{p(z,h)}{p(z)p(h)} \, dh \, dz \end{aligned} $} \tag*{(3)} \end{equation}

Effective modeling of long-horizon dialogue requires compressing extensive context to extract salient semantics while balancing the suppression of redundancy with the retention of long-term dependencies [1]. The IB framework perfectly aligns with our requirement for global information compression during multi-turn interactions. Therefore, we adopt it as our theoretical foundation. We define the two components of the IB objective as compression (minimizing I(Z;H)I(Z;H)) and prediction (maximizing I(Z;Y)I(Z;Y)), which conceptually strive to eliminate input redundancy while retaining the core semantics necessary for generation.

Global Context Sense-Making Strategy

Since historical text and external knowledge are inherently distinct, we design dual compressors—memory and knowledge—based on the same theoretical objective to ensure LLMs can effectively disentangle and utilize these abstract representations. In the following subsections, we detail the derivation of the global compression objectives for both the historical sequences and the related knowledge.

Derivation of Memory Compression Objective

The compression of the history HH must be conditioned on the current query QQ. For instance, in medical dialogues spanning diverse topics such as symptoms, lifestyle, and examinations, a query about a medication plan requires selectively integrating relevant historical context. Therefore, we formulate a conditional objective for history compression:

min⁡P(Z∣H,Q)Lmem=I(Z;H∣Q)−βI(Z;Y∣Q)\begin{equation} \min_{P(Z|H,Q)} \mathcal{L}_{mem} = I(Z;H|Q) - \beta I(Z;Y|Q) \tag*{(4)} \end{equation}

The first term enforces compactness by penalizing redundant information, while the second term ensures the preservation of salient information required for generation. We expand the Compactness term as:

\begin{equation} \resizebox{0.9\linewidth}{!}{$ \begin{aligned} I(Z;H|Q) &= \mathbb{E}_q \left[ \int_h \int_z p(z,h|q) \log \frac{p(z,h|q)}{p(z|q)p(h|q)} \, dz \, dh \right] \\ &= \mathbb{E}_q \left[ \int_h \int_z p(h|q)p(z|h,q) \log \frac{p(z|h,q)}{p(z|q)} \, dz \, dh \right] \\ &= \mathbb{E}_{p(h,q)} \left[ \int_z p(z|h,q) \log \frac{p(z|h,q)}{p(z|q)} \, dz \right] \\ &= \mathbb{E}_{p(h,q)} \left[ \mathbb{D}_{KL}[p(z|h,q) \| p(z|q)] \right] \end{aligned} $} \tag*{(5)} \end{equation}

Minimizing I(Z;H∣Q)I(Z;H|Q) compels the model to compress the history HH as aggressively as possible relative to QQ, ensuring ZZ should not depend on HH.

The Preservation term is derived as:

\begin{equation} \resizebox{0.9\linewidth}{!}{$ \begin{aligned} I(Z;Y|Q) &= \mathbb{E}_q \left[ \int_y \int_z p(z,y|q) \log \frac{p(z,y|q)}{p(z|q)p(y|q)} \, dz \, dy \right] \\ &= \mathbb{E}_q \left[ \int_y \int_z p(z|q)p(y|z,q) \log \frac{p(y|z,q)}{p(y|q)} \, dz \, dy \right] \\ &= \mathbb{E}_{p(y,z,q)} \left[ \log \frac{p(y|z,q)}{p(y|q)} \right] \\ &= \mathbb{E}_{p(y,z,q)} \left[ \log p(y|z,q) \right] - \mathbb{E}_{p(y,q)} \left[ \log p(y|q) \right] \end{aligned} $} \tag*{(6)} \end{equation}

Since the second term in (6) is independent of the compression ZZ, maxI(Z;Y∣Q)maxI(Z;Y|Q) is equivalent to maxEp(y,z,q)[log⁡p(y∣z,q)]max\mathbb{E}_{p(y,z,q)}[\log p(y|z,q)]. This ensures that ZZ alone suffices to generate accurate responses, thereby guaranteeing the retention of the most task-relevant semantics.

We define the global history compression encoder as qϕ(z∣h,q)q_{\phi}(z|h,q) and the LLM generator as pθ(y∣z,q)p_{\theta}(y|z,q). Since the marginal p(z∣q)p(z|q) is intractable, we optimize the variational upper bound using a parametric prior rφ(z∣q)r_\varphi(z|q). The final training objective for memory compression becomes:

Lmem= Ep(h,q)[DKL[qϕ(z∣h,q)∥rφ(z∣q)]]+βEp(y,h,q)Ez∼qϕ(⋅∣h,q)[−log⁡pθ(y∣z,q)]\begin{equation} \begin{aligned} \mathcal{L}_{mem} = & \ \mathbb{E}_{p(h,q)}\Big[ \mathbb{D}_{KL}[q_{\phi}(z|h,q) \| r_\varphi(z|q)] \Big] \\ & + \beta\mathbb{E}_{p(y,h,q)}\mathbb{E}_{z \sim q_{\phi}(\cdot|h,q)}\Big[ -\log p_{\theta}(y|z,q) \Big] \end{aligned} \tag*{(7)} \end{equation}

Derivation of Knowledge Compression Objective

During dialogue, historical information extraction is independent of external knowledge and conditioned solely on the current query. In contrast, global modeling of relevant knowledge needs to additionally consider the dialogue state encoded in the compressed history. Let KK denote the set of retrieved knowledge paths activated by QQ. The goal of the knowledge compressor is to learn a representation LL that captures the minimal sufficient statistics of KK required to generate YY, conditioned on QQ and ZZ. Since the derivation closely follows § 3.3.1, we omit the intermediate steps for brevity. The fundamental objective function is formulated as:

min⁡P(L∣K,Q,Z)Lknow=I(L;K∣Q)−βI(L;Y∣Q,Z)\begin{equation} \min_{P(L|K,Q,Z)} \mathcal{L}_{know} = I(L;K|Q) - \beta I(L;Y|Q, Z) \tag*{(8)} \end{equation}

The Compactness term minimizes the mutual information between the compressed knowledge LL and the raw knowledge KK, filtering out irrelevant noise. Using a variational approximation with a conditional prior rφ(l∣q)r_\varphi(l|q) and a knowledge compression encoder qψ(l∣k,q)q_{\psi}(l|k,q), we obtain the upper bound:

I(L;K∣Q)≤Ep(k,q)[DKL[qψ(l∣k,q)∥rφ(l∣q)]]\begin{equation} I(L;K|Q) \leq \mathbb{E}_{p(k,q)} \left[ \mathbb{D}_{KL}[q_{\psi}(l|k,q) \| r_\varphi(l|q)] \right] \tag*{(9)} \end{equation}

The Preservation term for knowledge is derived as:

\begin{equation} \resizebox{\linewidth}{!}{$ \begin{aligned} I(L;Y|Q, Z) &= H(Y|Q, Z) - H(Y|L, Q, Z) \\ &= H(Y|Q, Z) + \mathbb{E}_{p(k,y,q,z)} \left[ \mathbb{E}_{l \sim q_{\psi}(\cdot|k,q)} [\log p_{\theta}(y|l, z, q)] \right] \end{aligned} $} \tag*{(10)} \end{equation}

Since H(Y∣Q,Z)H(Y|Q, Z) is constant with respect to the compressor parameters ψ\psi, maximizing this mutual information is equivalent to maximizing the expected log-likelihood of the generator pθp_{\theta}. Thus, the final knowledge compression objective is formulated as:

Lknow= Ep(k,q)[DKL[qψ(l∣k,q)∥rφ(l∣q)]]+βEp(k,y,q,z)[El∼qψ(⋅∣k,q)[log⁡pθ(y∣l,z,q)]]\begin{equation} \begin{aligned} \mathcal{L}_{know} = & \ \mathbb{E}_{p(k,q)} \left[ \mathbb{D}_{KL}[q_{\psi}(l|k,q) \| r_\varphi(l|q)] \right] \\ & +\beta \mathbb{E}_{p(k,y,q,z)} \left[ \mathbb{E}_{l \sim q_{\psi}(\cdot|k,q)} [\log p_{\theta}(y|l, z, q)] \right] \end{aligned} \tag*{(11)} \end{equation}

Derived from (7) and (11), we obtain the compressed representations ZZ and LL (Z≪HZ \ll H and L≪KL \ll K). Consequently, the original generative objective in (1) is reformulated as:

Lgen=max∑i=1mlog⁡P(yi∣Z,L,ut,y<i)\begin{equation} \mathcal{L}_{gen}=max \displaystyle\sum_{i=1}^m \log P(y_{i}|Z,L,u_t,y_{<{i}}) \tag*{(12)} \end{equation}

In § 4, we further describe the adaptive activation of external knowledge, alongside the concrete instantiation and training objectives of the compressors.

Method

In this section, we detail the architectural pipeline and implementation of CoLink. Inspired by cognitive sense-making and associativity, CoLink integrates memory modeling with information-theoretic compression. As described in § 4.1, it constructs historical topic graphs to establish cross-semantic links with external knowledge and dynamically activates context-relevant facts by exploiting the semantic and structural associations of structured knowledge. To guarantee efficient global compression and the retention of core semantics from long contexts, CoLink competitively abstracts raw inputs into a small set of latent slots, namely memory slots and knowledge slots (§ 4.2). In § 4.3, we design a tri-part training loss, comprising KL divergence, reconstruction, and prediction, to enforce global information compression and semantic alignment under the theoretical formulations derived in § 3. An overview of the framework is illustrated in Figure 3.

Association and Activation

Cross-Level Semantic Linking

To preserve dialogue coherence and global consistency while mitigating interference across distinct topics, CoLink clusters the dialogue history H=[u1,…,ut−1]H=[u_1,\dots,u_{t-1}] into thematic clusters C∗=[C1,…,CN]C^*=[C_1,\dots,C_N] using BERTopic [23]. For each cluster, we apply OpenIE to extract relational triples and construct a set of topic graphs T∗={T1,…,TN}\mathcal{T}^*=\{\mathcal{T}_1,\dots,\mathcal{T}_N\}. Each graph Ti=(Ei,Ri)\mathcal{T}_i=(E^i,R^i) encodes the entities EiE^i and relations RiR^i associated with the ii-th theme. For knowledge augmentation, these local graphs are then grounded to the external Knowledge Graph (KG) G=(E,R)G=(E,R) via entity alignment to construct a multi-layered semantic bridge. The alignment is achieved by maximizing the cosine similarity of the entities from the topic graph and GG:

max⁡Sim(eT,eG)=eT⋅eG∣∣eT∣∣⋅∣∣eG∣∣\begin{equation} \max Sim(e^{\mathcal{T}},e^G)=\frac {\mathbf{e}^{\mathcal{T}} \cdot \mathbf{e}^G}{||\mathbf{e}^{\mathcal{T}}||\cdot||\mathbf{e}^G||} \tag*{(13)} \end{equation}

where eTe^{\mathcal{T}} and eGe^G are the entities from the topic graph T\mathcal{T} and the knowledge graph GG, respectively, while eT\mathbf{e}^{\mathcal{T}} and eG\mathbf{e}^G represent their corresponding semantic embeddings.

As the dialogue progresses, both the thematic clusters and their corresponding topic graphs expand dynamically. This structure functions as a "semantic episodic memory index," providing the foundational support for subsequent knowledge activation and memory compression.

Adaptive Control of Knowledge Activation

When a new user utterance utu_t arrives and is assigned to a historical topic cluster CjC_j, CoLink employs adaptive control of thought to activate context-relevant knowledge paths in the external KG. Given a set of 22-hop candidate paths K=[k1,…,kN]K=[k_1,\dots,k_N], where the paths are constructed from entities in the external KG GG aligned with the historical topic graph Tj\mathcal{T}_j, the activation score of path kkk_k is determined by its semantic relevance to the dialogue entities and its structural linkage importance, as defined in (14):

Ak=Bk+∑e∈ETj∩EkkwjSjk\begin{equation} A_k=B_k+\sum_{e \in E^{\mathcal{T}_j} \cap E^{k_k}} w_jS_{jk} \tag*{(14)} \end{equation}

where BkB_k denotes the baseline activation derived from the semantic similarity between kkk_k and utu_t. ETj=[e1Tj,…,e∣ETj∣Tj]E^{\mathcal{T}_j}=[e^{\mathcal{T}_j}_1, \dots, e^{\mathcal{T}_j}_{|E^{\mathcal{T}_j}|}] and Ekk=[e1kk,e2kk,e3kk]E^{k_k}=[e^{k_k}_1, e^{k_k}_2, e^{k_k}_3] denote the entity sets of the topic graph Tj\mathcal{T}_j and the path kkk_k, respectively. The term wjw_j is a time-decay factor that downweights entities associated with temporally distant utterances. Assuming EG=[e1,…,e∣ETj∣]E^{G}=[e_1,\dots,e_{|E^{\mathcal{T}_j}|}] represents the set of external knowledge entities aligned with ETjE^{\mathcal{T}_j}, the variable SjkS_{jk} quantifies the associative strength between the aligned entity ej∈EG∩Ekke_j \in E^{G} \cap E^{k_k} and the path kkk_k. Details of the computation are provided below.

For each node ej∈EG∩Ekke_j \in E^{G} \cap E^{k_k}, we isolate the connected subgraph G′=(E′,R′)G^{'}=(E^{'},R^{'}) anchored at node eje_j. We then initialize the graph-structural relevance Sj\mathbf{S}_j between all subgraph nodes and eje_j using a Gaussian kernel, and iteratively update it via Random Walk with Restart (RWR) [47] over the upper-layer KG until convergence. The initialization and the update rule are defined as:

Sji(0)=exp(−dis(j,i)22h2),whereei∈E′Sj′=c⋅A~Sj+(1−c)⋅Oj\begin{equation} \begin{aligned} S^{(0)}_{ji} = exp(- \frac{dis(j, i)^2}{2h^2}), where e_i \in E^{'} \\ \mathbf{S}_j^{'} = c \cdot \tilde{\mathbf{A}} \mathbf{S}_j + (1-c) \cdot \mathbf{O}_j \end{aligned} \tag*{(15)} \end{equation}

where hh is the bandwidth and A~∈R∣E′∣×∣E′∣\tilde{\mathbf{A}} \in \mathbb{R}^{|E^{'}| \times |E^{'}|} is the transition matrix of the knowledge subgraph. In (15), Oj\mathbf{O}_j is the one-hot restart vector centered at the entity eje_j, and cc denotes the restart probability. The resulting vector Sj′\mathbf{S}_j^{'} captures the global graph relevance of entity eje_j to all entities in the subgraph G′G^{'}, from which the specific associativity SjkS_{jk} for the path kkk_k is extracted:

Sjk=max⁡ekk∈EkkSj′(ekk)\begin{equation} S_{jk}=\max_{e^{k_k} \in E^{k_k}} \mathbf{S}^{'}_j(e^{k_k}) \tag*{(16)} \end{equation}

Ultimately, all entities aligned with Tj\mathcal{T}_j jointly determine the activation level AkA_k.

Structurally, this framework mirrors the hippocampus-cortex interplay: the topic graph functions as a dynamic contextual index, while the external KG serves as consolidated long-term storage [18, 6, 16]. Their interaction through context-adaptive activation facilitates efficient semantic integration and retrieval.

Competitive Extraction of Salient Information

To instantiate the theoretical variables ZZ and LL, we propose abstracting the raw, variable-length representations of global history HH and knowledge KK into a fixed, highly condensed set of latent bottleneck slots. This architectural design natively guarantees the strict compactness bounds (Z≪HZ \ll H and L≪KL \ll K) while preserving seamless compatibility with LLM generators. To operationalize this competitive abstraction, we adapt the Slot Attention [25] as our computational backbone, which iteratively distills complex long-range dependencies into these fixed-capacity slots. Since the memory and knowledge compressors use the same training framework, we describe the historical compressor as an example. Specifically, we initialize MM learnable memory slots S0=[s10,…,sM0]∈RM×d\mathbf{S}^0 = [s^0_1, \dots, s^0_M] \in \mathbb{R}^{M \times d} as the concrete instantiation of ZZ to distill the salient information.

For history compression, let ChC_h denote the relevant historical cluster. We encode ChC_h into a sequence representation X=[x1,…,xL]∈RL×d\mathbf{X}=[x_1, \dots, x_L] \in \mathbb{R}^{L \times d} . First, the inputs and slots are projected into key, value, and query spaces via linear transformations and Layer Normalization (LN) [27]:

\begin{equation} \resizebox{0.9\linewidth}{!}{$ \mathbf{K}=\mathbf{W}_k \text{LN}(\mathbf{X}), \quad \mathbf{V}=\mathbf{W}_v \text{LN}(\mathbf{X}), \quad \mathbf{Q}=\mathbf{W}_q \text{LN}(\mathbf{S}^t) $} \tag*{(17)} \end{equation}

where Wk,Wv,Wq\mathbf{W}_k, \mathbf{W}_v, \mathbf{W}_q are learnable projections. The slots then perform a selective aggregation of input features through softmax attention and obtain the updates U~t\widetilde{\mathbf{U}}^t:

Ai,jt=exp⁡(Qi⋅Kj/d)∑k=1Mexp⁡(Qk⋅Kj/d)U~t=Norm(At)⋅V,Norm(At)i,j=Ai,jt∑l=1LAi,lt\begin{equation} \begin{aligned} \mathbf{A}^t_{i,j} &= \frac{\exp(\mathbf{Q}_i \cdot \mathbf{K}j / \sqrt{d})}{\sum_{k=1}^M \exp(\mathbf{Q}_k \cdot \mathbf{K}_j / \sqrt{d})} \\ \widetilde{\mathbf{U}}^t &= \text{Norm}(\mathbf{A}^t) \cdot \mathbf{V}, \quad \text{Norm}(\mathbf{A}^t)_{i,j} = \frac{\mathbf{A}^t_{i,j}}{\sum_{l=1}^L \mathbf{A}^t_{i,l}} \end{aligned} \tag*{(18)} \end{equation}

The aggregated updates are fused into the slots using a Gated Recurrent Unit (GRU) [28], as described in (18). After TT iterations, the resulting slots ST\mathbf{S}^T constitute the compressed latent representation ZZ.

S~t=GRU(U~t,St−1)St=S~t+MLP(LN(S~t))\begin{equation} \begin{aligned} \widetilde{\mathbf{S}}^t &= \text{GRU}(\widetilde{\mathbf{U}}^t, \mathbf{S}^{t-1}) \\ \mathbf{S}^{t} &= \widetilde{\mathbf{S}}^t + \text{MLP}(\text{LN}(\widetilde{\mathbf{S}}^t)) \end{aligned} \tag*{(19)} \end{equation}

Since each slot interacts with all tokens through attention, the module captures globally salient information while achieving an effective compression ratio of ∣H∣/M|H|/M. By prepending the memory slots and knowledge slots as prefix tokens for LLMs, CoLink effectively enhances the models’ perception of critical context.

Training Objectives and Optimization

To effectively optimize the compressors, we operationalize the objective derived in § 3 via a composite loss function comprising three terms, as defined in (20).

KL Divergence Loss (LKL\mathcal{L}_{KL}): Constrains the posterior qϕ(z∣h,q)q_{\phi}(z|h,q) to approximate the prior rφ(z∣q)r_\varphi(z|q), enforcing aggressive compression of the history relative to the query.

Prediction Loss (Lpred\mathcal{L}_{pred}): Maximizes the log-likelihood of the ground-truth response yy, ensuring the compressed slots retain sufficient utility for generation.

Reconstruction Loss (Lrec\mathcal{L}_{rec}): Strictly minimizing mutual information can sometimes lead to representation collapse. To preserve interpretability and recoverability, we incorporate an auxiliary objective that reconstructs distilled context summaries h^\hat{h} (via GPT-4 [11]).

Lmem=αDKL[qϕ(z∣h,q)∥rφ(z∣q)]⏟Compression+βEz[−log⁡pθ(y∣z,q)]⏟Prediction+λEz[−∑log⁡p(h^t∣h^<t,z)]⏟Reconstruction\begin{equation} \begin{aligned} \mathcal{L}_{mem} &= \underbrace{\alpha\mathbb{D}_{KL}[q_{\phi}(z|h,q)\|r_\varphi(z|q)]}_{\text{Compression}} \\ &\quad + \underbrace{\beta\mathbb{E}_{z}[-\log p_{\theta}(y|z,q)]}_{\text{Prediction}} \\ &\quad + \underbrace{\lambda \mathbb{E}_{z}[-\sum \log p(\hat{h}_t|\hat{h}_{<t},z)]}_{\text{Reconstruction}} \end{aligned} \tag*{(20)} \end{equation}

where α,β,λ\alpha, \beta, \lambda are hyperparameters balancing each term. The objective for the knowledge compressor follows an analogous formulation as (21).

Lknowledge=αEp(k,q)[DKL[qψ(l∣k,q)∥rφ(l∣q)]]⏟Compression+βEp(y,k,q,z)El∼qψ[−log⁡pθ(y∣l,z,q)]⏟Prediction+λEl∼qψ[−∑tlog⁡p(k^t∣k^<t,l)]⏟Reconstruction\begin{equation} \begin{aligned} \mathcal{L}_{knowledge} &= \underbrace{\alpha \mathbb{E}_{p(k,q)} \left[ \mathbb{D}_{KL}[q_{\psi}(l|k,q) \| r_\varphi(l|q)] \right]}_{\text{Compression}} \\ &\quad + \underbrace{\beta \mathbb{E}_{p(y,k,q,z)} \mathbb{E}_{l \sim q_{\psi}} \left[ -\log p_{\theta}(y|l,z,q) \right]}_{\text{Prediction}} \\ &\quad + \underbrace{\lambda \mathbb{E}_{l \sim q_{\psi}} \left[ -\sum_{t} \log p(\hat{k}_t|\hat{k}_{<t}, l) \right]}_{\text{Reconstruction}} \end{aligned} \tag*{(21)} \end{equation}

where k^\hat{k} represents the target knowledge summary.

To ensure training stability and semantic consistency with the backbone LLM, we adopt a two-stage training strategy. In stage 1, we freeze the LLM parameters and optimize only the compression modules. In stage 2, we employ LoRA [29] to jointly fine-tune both the compressors and the LLM.

Experiment

Datasets

Existing medical dialogue datasets often suffer from knowledge leakage, role confusion, poor coherence, and factual hallucinations [21, 30, 12]. To mitigate these issues and enhance granularity, we construct the MMD dataset using a novel knowledge-augmented multi-agent debate framework. This framework simulates expert consultations involving three roles: User, Domain Experts (N≥2N \ge 2), and Judge.

  • User Agent: Initiates queries related to specific diseases.

  • Expert Agent Group: Each expert performs reasoning based on a shared domain knowledge base, retrieves relevant facts, reflects on retrieved content, and filters out irrelevant knowledge.

  • Judge Agent: Evaluates and aggregates expert opinions, and outputs a final diagnosis or recommendation only when a consensus is reached.

Dialogue scenarios follow real medical settings (e.g., diagnosis, examination recommendation, medication). Each conversation is grounded in an anonymized electronic health record and a corresponding knowledge base. For each user query, experts, augmented by a private, expert-only knowledge base, engage in iterative debate to correct errors. The final response is emitted only upon consensus verified by the Judge agent (see Figure 4).

Figure 4. Example of the data construction process.

Our primary dataset, MMD, encompasses 23702370 samples across 3737 diseases, with an average of 2.772.77 debate rounds per response. The train/test split is 9:19:1. Three general practitioners, each with over three years of clinical experience, evaluate knowledge relevance, debate logic, and response accuracy using a 5-point scale. Among 800800 samples, 94%94\% are rated as high quality, and inter-annotator agreement reaches a Cohen’s Kappa of 0.860.86. To assess generalization, we further evaluate on the public benchmark CMtMedQA [31]. Statistics for both MMD and CMtMedQA are summarized in Table tab:dataset_statistics.

DatasetDomainLang.# Samples# DiseasesAvg. TurnsDebate Turns
MMDMedicalChinese2,370378.092.77
CMtMedQAMedicalChinese5,993148.85--

Baselines

We perform extensive comparisons against standard context strategies and two primary paradigms.

Standard Baselines: We report performance using full history concatenation and summaries of all history.

Structured Organization & Retrieval: We evaluate MemoryBank [14] , MemTree [4] and RAPTOR [15], and graph-based methods including GraphRAG [3], HippoRAG [6], and HippoRAG2 [5]. They all use OpenAI encoders for semantic-similarity-based retrieval. We also include NV-Embed [2] as a dense retrieval baseline.

Local Token Attention: We compare against StreamLLM [8] and CCA [1].

All baselines are implemented following their original papers, including the information organization schemes and retrieval depths.

HyperparameterDescriptionValue
E1E_1Stage 1 Epochs1010
E2E_2Stage 2 Epochs1010
η1\eta_1Stage 1 Learning Rate5e−55e-5
η2\eta_2Stage 2 Learning Rate1e−51e-5
wdw_dWeight Decay0.010.01
BBBatch Size44
X\mathbf{X}Input RepresentationLast Hidden State
ddHidden Dimension15361536
TTSlot Iterations33
α\alphaCompression Coeff.11
β\betaPrediction Coeff.11
λ\lambdaReconstruction Coeff.11

Experimental Details

We implement CoLink using PyTorch and optimize it via the Adam optimizer. All experiments are conducted on a single NVIDIA A800 (80GB) GPU. To ensure semantic alignment between the compressor and the generator, we initialize the input X\mathbf{X} using the final-layer hidden states of the backbone LLM. During inference, we set the temperature to 00 and the maximum sequence length to 20482048. For external knowledge integration, since the MMD and CMtMedQA datasets lack natively aligned external knowledge bases, we employ WIMedKG [33] and CMeKG [36] as their respective supporting KGs. Detailed hyperparameters are listed in Table tab:hyperparameters. All reported results are averaged over three independent runs. Regarding evaluation, we employ BLEU, ROUGE, and BERTScore as automated metrics. Furthermore, to explicitly assess dialogue consistency and correctness, we sample 200200 instances for both LLM-as-a-Judge and human evaluation.

MethodMMD (%)CMtMedQA (%)
B1B2B3B4R1R2RLBSB1B2B3B4R1R2RLBS
MemoryBank19.418.554.472.3223.315.2414.9868.8327.2014.208.344.9029.699.0621.6773.75
NV-embed17.778.104.222.3123.345.6015.3568.2223.8212.187.144.1927.578.1720.0972.10
StreamLLM0.380.060.030.021.750.021.6945.970.600.110.060.043.700.210.9946.08
GraphRAG16.456.643.241.6521.574.3714.1567.9124.1012.026.903.9427.257.6419.4572.41
CCA8.622.991.460.7813.612.078.7564.144.141.320.560.2711.391.497.9858.67
MemTree12.424.662.291.2618.433.2813.5164.3613.974.752.331.2416.102.5911.9564.31
RAPTOR14.917.444.182.5023.876.7716.8668.7422.3712.007.424.6128.469.1422.1072.40
HippoRAG8.143.131.510.7818.423.0513.1464.4316.147.123.641.9724.045.4517.9468.65
HippoRAG216.317.894.422.4919.595.1813.6564.81----------------
Base17.626.823.341.7921.653.8013.4267.8918.739.165.202.9424.956.4516.0971.45
+ Full History22.4711.046.323.6727.196.9516.5370.2629.4216.5210.456.5533.8611.4523.1075.90
+ Summary History13.976.413.592.1626.746.7517.2069.1625.2113.988.845.5133.1911.1124.3174.24
CoLink(Ours)27.5213.877.664.1031.638.6419.2672.8431.6218.0211.747.6234.2812.3125.5075.36

Table 1. The comparative results. We highlight the best performance. B, R, and BS stand for BLEU, Rouge, and Bertscore, respectively.

Main Analysis

Comparison Results

Table 1 presents the comparative performance across all datasets. All experiments utilize Qwen2.5-1.5B [32] as the backbone model. Based on these results, we summarize the key findings as follows:

CoLink attains effective global information compression. CoLink consistently outperforms both structured retrieval and local token attention baselines, achieving an average gain of 2.59%2.59\% even under a 32×\times compression constraint across two datasets. This indicates its superior ability to compress long-context information. Notably, CoLink matches or even exceeds the performance of Full History. This validates that, governed by the Information Bottleneck principle, CoLink effectively distills sufficient statistics from the global context, filtering out redundancy and noise that may distract the full-context baseline while retaining decision-critical semantics.

Rigid structuring compromises contextual integrity. Although chunk-based retrieval methods, such as MemoryBank and NV-Embed, offer moderate improvements over the base model, rigid graph and tree-based approaches (e.g., GraphRAG, MemTree) often degrade performance. Our analysis reveals that while structured extraction preserves discrete entities, it fails to capture the holistic semantic flow of the text. Furthermore, entity retention is highly sensitive to extraction quality and provides limited benefits for long-range dependencies (see § 5.6.2). Similarly, Summary History, despite reducing redundancy, suffers from inevitable information loss during condensation and obstructs performance. In contrast, CoLink resolves this dilemma via gradient-optimized slots, proving that soft, learnable compression is a more robust strategy for retaining holistic semantics than rigid structuring.

MethodMMDCMtMedQA
History/Knowledge Processing Time (ms)GPU Memory (MB)Compression RatioHistory/Knowledge Processing Time (ms)GPU Memory (MB)Compression Ratio
MemoryBank2113.57/0.63884521.04/0.4884
NV-embed218.4127674.820.8095202.6927612.200.7732
StreamLLM/3050.950.9665/3064.591.0000
GraphRAG41511.27/2.894647679.49/2.4866
CCA/3679.530.8903/3724.541.0000
MemTree51725.50/1.72333911.10/1.5834
RAPTOR763.50/0.9900823.20/0.9900
HippoRAG29149.56/0.832810195.56/0.9005
CoLink(Ours)3536.074840.540.03132836.894843.310.0156

Table 2. Comparison of online resource consumption and compression ratios.

CoLink achieves superior compression efficiency. Beyond generation quality, we evaluate the compression ratio of various approaches, as summarized in Table 2. CoLink reaches optimal performance utilizing only 3.1%3.1\% of the original context length. This evidence reaffirms the high efficiency and effectiveness of our cognitive-inspired cross-semantic linking and IB-driven abstraction. Unlike traditional structured approaches, CoLink directly compresses the dialogue history in the latent space, circumventing costly entity extraction and graph construction to achieve O(1)O(1) retrieval complexity. Furthermore, let TT denote the number of dialogue turns and LL the average length per turn, yielding a total historical context length of N=T⋅LN = T \cdot L. Standard dense attention mechanisms suffer from quadratic time complexity O((TL)2)O((TL)^2) and linear space complexity O(TL)O(TL) due to the continuous growth of the KV cache. In contrast, CoLink compresses the unbounded history into MM fixed-size slots (where M≪TLM \ll TL). This reduces the storage complexity for history representation to O(M)O(M) and the attention complexity during generation to O(M2)O(M^2), ensuring constant-time scalability regardless of dialogue duration.

Input EncodeBLEU (%)ROUGE (%)BS (%)
B1B2B3B4R1R2RL
Base(Full History)18.2111.027.844.6630.0310.1315.3169.38
Base(Summary History)21.8913.117.655.5732.3611.2117.0970.46
Memory Slot(Ours)24.2114.048.605.0231.8210.4417.7070.25

Generalizability of CoLink

To further investigate the robustness of our framework, we evaluate CoLink across three critical dimensions: architectural transferability, cross-domain adaptability, and cross-dataset generalizability.

Architectural Transferability. To assess whether CoLink is model-agnostic, we apply the framework to Llama-3.2-3B [35] by training the memory slots on the MMD dataset. As presented in Table tab:model-transfer, CoLink yields consistent performance gains over the vanilla Llama-3.2 architecture. This result underscores its strong architectural transferability, indicating that our cognitive-inspired compression mechanism can effectively augment various LLM backbones.

Input EncodeBLEU (%)ROUGE (%)BS (%)
B1B2B3B4R1R2RL
Base(Full History)23.2812.677.975.1228.728.6225.6074.96
Base(Summary History)18.189.786.033.7530.389.0227.0374.16
Memory Slot(Ours)38.4923.1615.5910.4839.4214.5235.3378.68

Cross-Domain Adaptability. We further evaluate CoLink on UltraData [40], an English open-domain multi-turn dialogue dataset, to verify its performance beyond the medical domain. Notably, since external knowledge is not natively required for this task, we deploy only the memory slots for history compression. As shown in Table tab:domain-adaption, CoLink outperforms the full-history concatenation baseline by 8.16%8.16\%, demonstrating that its IB-driven abstraction maintains high efficacy even in knowledge-scarce, open-domain scenarios.

Cross-Dataset Generalizability. Lastly, we evaluate CoLink’s inference capacity by directly transferring the MMD-trained model to the CMtMedQA dataset without further fine-tuning. This cross-dataset setting, summarized in Table tab:method-gener, reveals that CoLink consistently outperforms standard baselines. Such results highlight its robust capacity for in-domain transfer and its ability to handle distribution shifts between different clinical dialogue contexts.

MethodBLEU (%)ROUGE (%)BS (%)
B1B2B3B4R1R2RL
CoLink (stage1)26.4013.407.884.5429.098.6320.5273.30

Effectiveness Analysis

We theoretically analyze the superiority of CoLink over sequential summarization from both forward (attention) and backward (gradient) perspectives.

Attention Competition. To compress a long text of length LL into MM memory slots, the information that an input token jj contributes to a slot ii is formulated as:

Ai,j=exp⁡(qi⋅kj)∑l=1Mexp⁡(ql⋅kj)\begin{equation} A_{i,j} = \frac{\exp(q_i \cdot k_j)}{\sum_{l=1}^M \exp(q_l \cdot k_j)} \tag*{(22)} \end{equation}

This indicates that the same input token jj must vote and allocate its weight across all MM slots, facilitating a competitive absorption of the contextual information. In contrast, transformer-based summarization typically relies on hidden states or generates a short text directly. The attention between a summary token ii and an input token jj is calculated as:

Ai,j=exp⁡(qi⋅kj)∑l=1Lexp⁡(qi⋅kl)\begin{equation} A_{i,j} = \frac{\exp(q_i \cdot k_j)}{\sum_{l=1}^{L}\exp(q_i \cdot k_l)} \tag*{(23)} \end{equation}

Compared to the competitive allocation of slots, this standard aggregation acts merely as a weighted average over the LL input tokens. As the input sequence length LL grows, this Softmax distribution inevitably flattens, inducing severe attention dilution and feature blurring.

Gradient Saliency. Furthermore, we analyze the optimization dynamics from a backward propagation perspective. Let C=[slots][query]C = [\text{slots}][\text{query}] and C=[summary][query]C = [\text{summary}][\text{query}] denote the corresponding LLM input formats during inference.

The loss function L\mathcal{L} inherently depends on the attention matrix AA (across any layer and head). By applying a first-order Taylor expansion [48], we obtain the loss perturbation:

L(A+ΔA)≈L(A)+⟨∂L∂A,ΔA⟩ΔL≈⟨∂L∂A,A⟩=∑i,jAi,j⋅∂L∂Ai,j\begin{equation} \begin{aligned} \mathcal{L}(A + \Delta A) \approx \mathcal{L}(A) + \left\langle \frac{\partial \mathcal{L}}{\partial A}, \Delta A \right\rangle \\ \Delta \mathcal{L} \approx \left\langle \frac{\partial \mathcal{L}}{\partial A}, A \right\rangle = \sum_{i,j} A_{i,j} \cdot \frac{\partial \mathcal{L}}{\partial A_{i,j}} \end{aligned} \tag*{(24)} \end{equation}

Compared to the relatively sparse representations in standard summarization, CoLink encapsulates denser semantics, thereby contributing higher information saliency (ΔL\Delta \mathcal{L}) to the optimization objective. Specifically, in sequential summarization, all gradients from a monotonic summary are forced through a shared representation channel, inevitably causing semantic interference. Conversely, in our proposed method, since the matrix Ai,jA_{i,j} strictly routes tokens via the aforementioned competition mechanism, the loss gradients ∂L∂Ai,j\frac{\partial \mathcal{L}}{\partial A_{i,j}} can backpropagate through highly orthogonal and decoupled paths, fundamentally preserving distinct and fine-grained features.

We explicitly visualize this in Figure 5. The figure illustrates the attention and saliency distributions of a dialogue sample, comparing the standard summary history against memory slot compression. Excluding structural attention sinks (e.g., the initial token and the terminal query tokens), the compact slot representations (red lines) consistently exhibit significantly higher attention mass and gradient saliency towards both the query and the next-token prediction. This empirical observation perfectly corroborates our theoretical analysis: compared to the diluted and flattened distribution of the sequential summary (blue lines), CoLink inherently guarantees a highly condensed and salient feature space.

Figure 5. Attention and loss saliency of summary history and memory slots towards the query and next-token prediction.

Human and LLM-as-a-Judge Evaluation

MethodHumanLLM-as-Judge
Accuracy & ProfessionalismInformativenessContextual CoherenceOverallAccuracy & ProfessionalismInformativenessContextual CoherenceOverall
MemoryBank3.773.084.283.713.943.494.463.96
NV-embed3.562.724.143.473.823.244.313.79
GraphRAG3.542.444.653.543.212.564.423.40
CCA1.001.001.001.001.001.001.001.00
MemTree2.801.844.403.013.783.034.703.84
RAPTOR3.131.913.983.013.542.644.383.52
HippoRAG23.032.003.973.003.142.414.453.33
CoLink(Ours)3.713.004.983.903.903.334.944.06

Table 3. Human and LLM-as-a-Judge evaluation on Accuracy & Professionalism, Informativeness, and Contextual Coherence.

We employ a dual-track evaluation strategy involving human experts and GPT-4o-mini [11] to rate responses on a 5-point scale across three specific dimensions: Accuracy & Professionalism: Assesses whether the response is factually correct, free from hallucinations, and adheres to clinical standards. Informativeness: Measures the richness and utility of the content, penalizing vague or generic responses that fail to address specific queries. Contextual Coherence: Evaluates the model’s ability to maintain logical consistency across turns and correctly reference historical details without contradiction.

Table 3 summarizes the results. CoLink consistently achieves superior ratings across all metrics. Notably, it surpasses the strongest baseline by a substantial margin in Contextual Coherence, highlighting its robustness in modeling long-horizon dependencies.

Ablation Studies

We design ablation studies to dissect the contributions of CoLink’s components, including component analysis, history clustering strategies, loss sensitivity, and the impact of the input encoding space.

History Len.MMD (%)CMtMedQA (%)
B1B2B3B4R1R2RLBSB1B2B3B4R1R2RLBS
CoLink(Ours)27.5213.877.664.1031.638.6419.2672.8431.6218.0211.747.6234.2812.3125.5075.36
w/o Memory Slots24.1311.426.263.3528.357.0817.8770.9428.7315.839.966.1831.8910.8123.7474.33
w/o Knowledge Slots27.6113.827.713.9631.888.5419.3372.7830.0817.0010.916.9732.7311.6224.0174.81

Table 4. Performance after ablating memory slots and knowledge slots.

Ablation of Model Components

Table 4 details the impact of removing the memory and knowledge slots. Ablating memory slots induces severe degradation. This strongly validates our IB-driven objective, which effectively distills critical state information from redundant history while filtering noise. It underscores that accurate context state maintenance is a prerequisite for reliable decision-making in multi-turn medical consultations. In contrast, the gains from knowledge activation and compression are relatively marginal. We attribute this to two factors: 1. Knowledge Misalignment: Since neither MMD nor CMtMedQA possesses a natively aligned KG, we rely on general medical KGs. These external KGs often contain irrelevant entities and lack scenario-specific details (e.g., irrelevant diseases and lifestyle advice). This misalignment imposes an upper bound on the potential performance gain. 2. Semantic Sparsity: Unlike dense textual history, KGs rely on discrete symbolic triples. This structural representation inherently suffers from semantic information loss, making it more challenging for the model to extract nuanced cues than from the rich natural language context.

To investigate this, we conduct experiments on the GMD dataset, which features a precisely aligned KG for symptom-based disease diagnosis [19]. Table tab:gmd presents the diagnostic accuracy of various methods. Notably, the IB-driven knowledge slots yield a positive impact, demonstrating their effectiveness when accurate external knowledge is available.

MethodAccuracy (%)
Base79.08
+Full Knowledge82.00
+Summary Knowledge59.83
CoLink (Knowledge Slots)90.37

Sensitivity to Loss Configuration

MethodBLEU (%)ROUGE (%)BS (%)
B1B2B3B4R1R2RL
CoLink (stage1)23.2110.965.663.0127.906.8417.6070.88
w/oLrec\mathcal{L}_{rec}16.746.603.241.7321.363.8814.3667.84
w/oLKL\mathcal{L}_{KL}25.6511.815.993.1529.276.8117.3371.36
w/oLpred\mathcal{L}_{pred}17.136.743.231.6022.544.1915.2567.73

To analyze the effects of the tripartite objective defined in § 4.3, we conduct loss ablations on the MMD dataset during first-stage memory slot training. The results are reported in Table tab:loss-ablation. Removing either Lpred\mathcal{L}_{pred} or Lrec\mathcal{L}_{rec} results in significant degradation, confirming their roles in preventing hallucination and semantic drift, respectively. Intriguingly, omitting the KL constraint (LKL\mathcal{L}_{KL}) yields marginal metric improvements. We attribute this to the misalignment between the isotropic Gaussian prior and the pre-trained LLM’s highly anisotropic embedding space [34]. This suggests a trade-off between strict regularization (for OOD robustness, see Table tab:method-gener) and surface-level n-gram matching. We validate this trade-off through two auxiliary analyses: Factor Scaling: Varying the LKL\mathcal{L}_{KL} weight (Table tab:scale-kl) confirms that weaker regularization correlates with higher surface metrics. Training Dynamics: As shown in Figure 6(a), removing LKL\mathcal{L}_{KL} leads to a slower convergence rate, indicating that although LKL\mathcal{L}_{KL} incurs a minor fitting penalty, it is essential for constraining representational dispersion and guiding stable convergence.

α\alphaBLEU (%)ROUGE (%)BS (%)
B1B2B3B4R1R2RL
10.010.020.308.744.402.1824.305.1115.7469.28
0.10.121.329.404.962.6326.465.9516.7369.57

Figure 6. Training loss trajectories under different ablation settings.

Analysis of Input Encoding Space

Input EncodeBLEU (%)ROUGE (%)BS (%)
B1B2B3B4R1R2RL
Layer 0 Hidden State15.866.002.871.4121.073.6814.7967.42
nn.embedding17.146.743.211.6021.643.6714.4367.93
Last Layer Hidden State23.2110.965.663.0127.906.8417.6070.88

Figure 7. Comparison of generations for a case across different models.

Since CoLink encodes compressed slots as prefix prompts for the LLM, alignment between the compressor’s semantic space and the LLM’s native representation is critical. To mitigate semantic mismatch, our default implementation utilizes the LLM’s own encoder to encode the context and extract the last hidden state for global abstraction. To quantify the impact of semantic drift introduced by different encoders, we compare this approach against two variants on dataset MMD: (1) a standard embedding layer (independent initialization), and (2) the LLM’s first hidden layer. To strictly isolate the encoder’s impact, we restrict these experiments to first-stage memory slot training and use a simplified input sequence: [Memory Slot] [Query]. Results are presented in Table tab:input-encode. We observe that maintaining alignment between the compression input and the LLM’s internal representation yields noticeable performance gains. We hypothesize that deriving representations from the LLM’s final layers ensures that the compressed slots reside within the same semantic manifold as the generative model. This alignment minimizes the adaptation gap, enabling the model to rapidly and accurately assimilate the core semantic signals encapsulated in the slots.

Impact of History Clustering Strategies

MethodBLEU (%)ROUGE (%)BS (%)
B1B2B3B4R1R2RL
History Cluster25.1811.565.863.0928.986.7617.7571.18
Full History25.5811.866.153.2028.886.8617.6171.34

To evaluate the necessity of pre-retrieval clustering, we compare compressing retrieved topic clusters against compressing the full history. The comparative results for first-stage memory slot training are presented in Table tab:history-culster. We observe negligible performance variance between the two settings. We attribute this parity to two primary factors: 1. Robust Information Extraction: It demonstrates CoLink’s intrinsic capability for noise filtration and semantic distillation. Even when exposed to the full unpruned history, the Information Bottleneck-guided compression effectively identifies and retains salient information while discarding redundancy. 2. High Information Density: Unlike sparse document retrieval tasks, domain-specific multi-turn dialogues (e.g., medical consultations) are characterized by high information density, where a larger portion of the history remains contextually relevant. This density likely diminishes the marginal gain of explicit clustering in this specific setting, though clustering remains computationally advantageous.

Further Analysis

Impact of Slot Number

The number of memory slots (MM) controls the granularity of global history compression. To investigate the trade-off between compression ratio and generation fidelity, we conduct sensitivity analysis by varying MM on MMD, as visualized in Figure 6(b). Performance exhibits a non-monotonic trend: both overly small and excessively large MM lead to degradation. This behavior is theoretically intuitive. With too few slots, long dialogue histories are over-compressed, forcing each slot to encode excessive information and causing semantic loss. Conversely, too many slots dilute information density, leaving many slots underutilized or dominated by noise. This observation aligns with the local vs. global trade-offs discussed in § 5.4.

MethodBLEU (%)ROUGE (%)BS (%)
B1B2B3B4R1R2RL
AdaptSlot (w/o Sparse Loss)6.652.511.430.8815.452.3610.7260.53
AdaptSlot17.096.493.281.7022.383.7614.1967.33
CoLink23.2110.965.663.0127.906.8417.6070.88

We further evaluate a dynamic-slot strategy adapted from AdaptSlot [26], which learns a masking function to prune redundant slots during training. The results are reported in Table tab:adaptslot. We find that dynamic slots often collapse into trivial behaviors: either masking the majority of slots or retaining the entire set, both of which substantially harm performance. Consequently, achieving stable, self-adaptive slot selection remains an open challenge.

Case Study

Figure 7 presents a qualitative comparison of representative model outputs. MemTree offers a partially relevant response (e.g., the pain may be related to breathing), but it relies primarily on the immediate query and fails to incorporate valuable historical context. GraphRAG exhibits similar deficiencies. Visualizing their underlying structures reveals significant information loss, attributable to both extraction errors and the constraints of rigid structured representations (discussed in § 5.4). In contrast, CoLink demonstrates superior alignment with the Full History baseline. By leveraging soft memory slots, it avoids structural fragmentation and successfully synthesizes global context to deliver precise and actionable medical advice.

Conclusion and Future Works

We revisit multi-turn dialogue through the lens of cognitive neuroscience and propose CoLink, a framework that innovatively integrates the IB principle of global abstraction with adaptive control of thought to activate associative knowledge. To address the scarcity of multi-turn dialogue data, we design a knowledge-augmented multi-agent debate framework and construct a dataset, MMD. Comprehensive evaluations demonstrate CoLink’s significant superiority over strong baselines while utilizing only 3.125%3.125\% of the original context length. Driven by the IB principle, CoLink matches or even exceeds the performance of the full visible history. Furthermore, its soft, learnable slot mechanism preserves continuous semantic flow, avoiding the structural fragmentation inherent in rigid approaches. Rigorous validation through ablations and human evaluation confirms the robustness, model-agnostic nature, and generalization capability of our approach.

While theoretical derivations and extensive experiments validate CoLink’s efficiency in history compression, our analysis reveals a trade-off: the method excels at capturing abstract semantic representations but struggles with tasks that require precise lexical retrieval. We evaluated this using the MemDaily dataset [37], which features concise histories but demands high-precision recall of specific entities (e.g., phone numbers, job titles). On this benchmark, CoLink underperforms compared to text-based prompt baselines (such as Full History and Summary History). This suggests that the compression process may filter out fine-grained details necessary for exact factual reconstruction. Consequently, enhancing fidelity in precise information retrieval remains a pivotal direction for future work.

References

  1. [1]Chen, Y., You, Z., Zhang, S., Li, H., Li, Y., Wang, Y., & Tan, M. (2024). Core context aware transformers for long context language modeling. https://arxiv.org/abs/2412.12465
  2. [2]Lee, C., Roy, R., Xu, M., Raiman, J., Shoeybi, M., Catanzaro, B., & Ping, W. (2024). Nv-embed: Improved techniques for training llms as generalist embedding models. https://arxiv.org/abs/2405.17428
  3. [3]Edge, D., Trinh, H., Cheng, N., Bradley, J., Chao, A., Mody, A., Truitt, S., Metropolitansky, D., Ness, R. O., & Larson, J. (2024). From local to global: A graph rag approach to query-focused summarization. https://arxiv.org/abs/2404.16130
  4. [4]Rezazadeh, A., Li, Z., Wei, W., & Bao, Y. (2024). From isolated conversations to hierarchical schemas: Dynamic tree memory representation for llms. https://arxiv.org/abs/2410.14052
  5. [5]Gutiérrez, B. J., Shu, Y., Qi, W., Zhou, S., & Su, Y. (2025). From rag to memory: Non-parametric continual learning for large language models. https://arxiv.org/abs/2502.14802
  6. [6]Jimenez Gutierrez, B., Shu, Y., Gu, Y., Yasunaga, M., & Su, Y. (2024). Hipporag: Neurobiologically inspired long-term memory for large language models. Advances in Neural Information Processing Systems, 37, 59532–59569.
  7. [7]Wang, S., Fang, Y., Zhou, Y., Liu, X., & Ma, Y. (2025). ArchRAG: Attributed Community-based Hierarchical Retrieval-Augmented Generation. https://arxiv.org/abs/2502.09891
  8. [8]Xiao, G., Tian, Y., Chen, B., Han, S., & Lewis, M. (2023). Efficient streaming language models with attention sinks. https://arxiv.org/abs/2309.17453
  9. [9]Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P. (2024). Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12, 157–173.
  10. [10]Li, J.-N., Tu, Q., Mao, C., Yu, Z., Wen, J.-R., & Yan, R. (2024). Streamingdialogue: Prolonged dialogue learning via long context compression with minimal losses. Advances in Neural Information Processing Systems, 37, 86074–86101.
  11. [11]Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. (2023). Gpt-4 technical report. https://arxiv.org/abs/2303.08774
  12. [12]Wang, H., Du, X., Yu, W., Chen, Q., Zhu, K., Chu, Z., Yan, L., & Guan, Y. (2025). Learning to break: Knowledge-enhanced reasoning in multi-agent debate system. Neurocomputing, 618, 129063.
  13. [13]Jiang, J., Yan, L., Liu, H., Xia, Z., Wang, H., Yang, Y., & Guan, Y. (2025). Knowledge assimilation: Implementing knowledge-guided agricultural large language model. Knowledge-based Systems, 314, 113197.
  14. [14]Zhong, W., Guo, L., Gao, Q., Ye, H., & Wang, Y. (2024). Memorybank: Enhancing large language models with long-term memory. Proceedings of the AAAI Conference on Artificial Intelligence, 38(17), 19724–19731.
  15. [15]Sarthi, P., Abdullah, S., Tuli, A., Khanna, S., Goldie, A., & Manning, C. D. (2024). Raptor: Recursive abstractive processing for tree-organized retrieval. The Twelfth International Conference on Learning Representations.
  16. [16]Cai, X., Fu, Y., Zhao, H., Jiang, W., & Pu, S. (2022). Memory graph with message rehearsal for multi-turn dialogue generation. Proceedings of the 31st ACM International Conference on Information & Knowledge Management, 108–117.
  17. [17]Huitt, W. (2003). The information processing approach to cognition. Educational Psychology Interactive, 3(2), 53.
  18. [18]Atkinson, R. C., & Shiffrin, R. M. (1968). Human memory: A proposed system and its control processes. In Psychology of learning and motivation (Vol. 2, pp. 89–195). Elsevier.
  19. [19]Yan, L., Guan, Y., Wang, H., Lin, Y., Yang, Y., Wang, B., & Jiang, J. (2024). EIRAD: An evidence-based dialogue system with highly interpretable reasoning path for automatic diagnosis. IEEE Journal of Biomedical and Health Informatics.
  20. [20]Zhu, K., Feng, X., Du, X., Gu, Y., Yu, W., Wang, H., Chen, Q., Chu, Z., Chen, J., & Qin, B. (2024). An Information Bottleneck Perspective for Effective Noise Filtering on Retrieval-Augmented Generation. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 1044–1069.
  21. [21]Sun, Y., Liu, C., Zhou, K., Huang, J., Song, R., Zhao, W. X., Zhang, F., Zhang, D., & Gai, K. (2024). Parrot: Enhancing multi-turn instruction following for large language models. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 9729–9750.
  22. [22]Zhu, X., Xie, Y., Liu, Y., Li, Y., & Hu, W. (2025). Knowledge Graph-Guided Retrieval Augmented Generation. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), 8912–8924.
  23. [23]Grootendorst, M. (2022). BERTopic: Neural topic modeling with a class-based TF-IDF procedure. https://arxiv.org/abs/2203.05794
  24. [24]Tishby, N., Pereira, F. C., & Bialek, W. (2000). The information bottleneck method. In arXiv preprint physics/0004057.
  25. [25]Locatello, F., Weissenborn, D., Unterthiner, T., Mahendran, A., Heigold, G., Uszkoreit, J., Dosovitskiy, A., & Kipf, T. (2020). Object-centric learning with slot attention. Advances in Neural Information Processing Systems, 33, 11525–11538.
  26. [26]Fan, K., Bai, Z., Xiao, T., He, T., Horn, M., Fu, Y., Locatello, F., & Zhang, Z. (2024). Adaptive slot attention: Object discovery with dynamic slot number. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 23062–23071.
  27. [27]Ba, J. L., Kiros, J. R., & Hinton, G. E. (2016). Layer normalization. https://arxiv.org/abs/1607.06450
  28. [28]Dey, R., & Salem, F. M. (2017). Gate-variants of gated recurrent unit (GRU) neural networks. 2017 IEEE 60th International Midwest Symposium on Circuits and Systems (MWSCAS), 1597–1600.
  29. [29]Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W., et al. (2022). Lora: Low-rank adaptation of large language models.. ICLR, 1(2), 3.
  30. [30]Kong, C., Fan, Y., Wan, X., Jiang, F., & Wang, B. (2024). PlatoLM: Teaching LLMs in multi-round dialogue via a user simulator. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 7841–7863.
  31. [31]Yang, S., Zhao, H., Zhu, S., Zhou, G., Xu, H., Jia, Y., & Zan, H. (2024). Zhongjing: Enhancing the chinese medical capabilities of large language model through expert feedback and real-world multi-turn dialogue. Proceedings of the AAAI Conference on Artificial Intelligence, 38(17), 19368–19376.
  32. [32]Bai, J., Bai, S., Chu, Y., Cui, Z., Dang, K., Deng, X., Fan, Y., Ge, W., Han, Y., Huang, F., et al. (2023). Qwen technical report. https://arxiv.org/abs/2309.16609
  33. [33]Li, X., Yuan, Y., Yang, Y., Guan, Y., Wang, H., Jiang, J., Shi, H., & Liu, X. (2025). Quality-Controllable automatic construction method of Chinese knowledge graph for medical decision-making applications. Information Processing & Management, 62(4), 104148.
  34. [34]Ethayarajh, K. (2019). How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 55–65.
  35. [35]Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023). Llama: Open and efficient foundation language models. https://arxiv.org/abs/2302.13971
  36. [36]Byambasuren, O., Yang, Y., Sui, Z., Dai, D., Chang, B., Li, S., & Zan, H. (2019). Preliminary study on the construction of Chinese medical knowledge graph. Journal of Chinese Information Processing, 33(10), 1–9.
  37. [37]Zhang, Z., Dai, Q., Chen, L., Jiang, Z., Li, R., Zhu, J., Chen, X., Xie, Y., Dong, Z., & Wen, J.-R. (2024). Memsim: A bayesian simulator for evaluating memory of llm-based personal assistants. https://arxiv.org/abs/2409.20163
  38. [38]Squire, L. R. (2004). Memory systems of the brain: a brief history and current perspective. Neurobiology of Learning and Memory, 82(3), 171–177.
  39. [39]Popham, S. F., Huth, A. G., Bilenko, N. Y., Deniz, F., Gao, J. S., Nunez-Elizalde, A. O., & Gallant, J. L. (2021). Visual and linguistic semantic representations are aligned at the border of human visual cortex. Nature Neuroscience, 24(11), 1628–1636.
  40. [40]Ding, N., Chen, Y., Xu, B., Qin, Y., Hu, S., Liu, Z., Sun, M., & Zhou, B. (2023). Enhancing chat language models by scaling high-quality instructional conversations. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 3029–3051.
  41. [41]Sun, M., Chen, X., Kolter, J. Z., & Liu, Z. (2024). Massive activations in large language models. First Conference on Language Modeling (COLM).
  42. [42]Zhang, Q., Singh, C., Liu, L., Liu, X., Yu, B., Gao, J., & Zhao, T. (2024). Tell your model where to attend: Post-hoc attention steering for llms. International Conference on Learning Representations, 2024, 42411–42430.
  43. [43]Gan, C., Tu, Y., Chen, X., Chen, T., Li, Y., Harandi, M., & Lin, W. (2026). Unleashing diffusion transformers for visual correspondence by modulating massive activations. Advances in Neural Information Processing Systems, 38, 114432–114462.
  44. [44]Kang, S., Kim, J., Kim, J., & Hwang, S. J. (2025). SEE WHAT YOU ARE TOLD: VISUAL ATTENTION SINK IN LARGE MULTIMODAL MODELS. 13th International Conference on Learning Representations, ICLR 2025, 13547–13574.
  45. [45]Zhang, Z., Sheng, Y., Zhou, T., Chen, T., Zheng, L., Cai, R., Song, Z., Tian, Y., Ré, C., Barrett, C., et al. (2023). H2o: Heavy-hitter oracle for efficient generative inference of large language models. Advances in Neural Information Processing Systems, 36, 34661–34710.
  46. [46]Liang, J., Li, H., Li, C., Zhou, J., Jiang, S., Wang, Z., Ji, C., Zhu, Z., Liu, R., Ren, T., et al. (2025). Ai meets brain: Memory systems from cognitive neuroscience to autonomous agents. https://arxiv.org/abs/2512.23343
  47. [47]Liu, C., Xie, Z., Zehmakan, A. N., & Zhang, Z. (2025). Efficient Algorithms for Computing Random Walk Centrality. IEEE Transactions on Knowledge and Data Engineering.
  48. [48]Yin, H., Si, G., & Wang, Z. (2025). Clearsight: Visual signal enhancement for object hallucination mitigation in multimodal large language models. Proceedings of the Computer Vision and Pattern Recognition Conference, 14625–14634.
  49. [49]Qiang, M., Wang, Z., Li, S., & Zhou, G. (2026). Exploring Knowledge Filtering for Retrieval-Augmented Question Answering. IEEE Transactions on Audio, Speech and Language Processing.

Paper details

vlnc:W_chc3u798
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 1

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents