QProtFuse: Query-Guided Layer Fusion for Retrieval-Augmented Protein Function Prediction

Introduction

Proteins are essential to biological processes. Predicting their functions from amino acid sequences is therefore a central task in computational biology. Traditional sequence analysis uses BLAST [1] to find homologs and InterPro [5] to identify family and domain signatures. These methods depend on informative matches and cannot directly answer specific biological questions.

More recently, broader large language models (LLMs) for life-science research have begun to tackle protein-related tasks [23, 2]. Protein-oriented LLMs incorporate amino acid sequences through direct tokenization [3, 25] (Wang et al. 2024b) or dedicated protein encoders [10] (Zhuo et al. 2024; Zhou et al. 2026). Together, these models offer the potential to answer complex biological questions in free-form by leveraging their internal pretrained knowledge and reasoning capabilities.

Despite these advances, the inherent gap between proteins and natural language remains a major obstacle to reliable protein understanding by LLMs (Zhang et al., 2026). For protein function description, Wu et al. (2025) show that retrieval-based methods can outperform direct LLM generation. This highlights the value of external biological knowledge, motivating retrieval-augmented generation (RAG) approaches (Wu et al., 2025; Ma et al., [22]; Ding et al. [7]. As illustrated in Figure 1(a), these approaches retrieve relevant proteins and annotations to enhance LLM-based protein function description.

However, current protein RAG methods focus on incorporating external retrieval signals, such as those from sequence alignment tools [1, 29] and additional protein language models (PLMs) [19, 11] (Wang et al. 2024a), while leaving the knowledge within a single PLM insufficiently explored. Previous studies reveal that PLMs encode rich and distinct biological features across different layers [27, 19, 12] (Vig et al. 2021). These findings raise two questions:

To answer these questions, we first probe the layer-wise retrieval performance of ESM-2 [19] on Prot-Inst-OOD (Wu et al., 2025). As shown in Figure 1(b), the four subtasks achieve their best retrieval performance using features from different intermediate layers. These observations answer Q1 and show that relying solely on final-layer features is suboptimal. As a first step toward Q2, we test a simple fusion strategy by uniformly averaging features across all layers. Even this simple strategy outperforms those incorporating MMseqs2 sequence similarity or ProstT5 features (Figure 1(c)).² However, uniform averaging still generally falls short of task-specific best-performing layers (see Appendix § 7). This gap calls for a more adaptive fusion strategy that exploits the distinct strengths of different layers.

Inspired by multi-layer visual feature fusion in multimodal LLMs [18, 16] (Yao et al. 2024), we propose QProtFuse (Query-guided multi-layer Protein Feature Fusion) for retrieval-augmented protein function prediction. QProtFuse leverages the query to adaptively assign layer-wise weights and fuse multi-layer features. To optimize this fusion, we propose a dual-objective contrastive learning framework that couples an intra-modal protein-to-protein objective

with an inter-modal protein-to-annotation objective. These two objectives are mutually reinforcing and jointly enable both protein and annotation retrieval during inference, thereby broadening the applicability of QProtFuse. On Prot-Inst-OOD, QProtFuse achieves superior protein and annotation retrieval performance compared with all baselines, reaching a Recall@1 of 16.4% and Recall@10 of 25.9% with our reranking configuration. These retrieval gains improve downstream function descriptions, yielding the highest LLM-as-a-judge scores among the evaluated methods, with 6.22/10 on Qwen3-8B and 6.68/10 on DeepSeek-V4-Pro.

Our contributions are summarized as follows:

• We present the first systematic study of multi-layer PLM features for protein retrieval, showing that the best layer varies across tasks and using only final-layer features is suboptimal.

• We propose QProtFuse, which combines query-guided layer fusion with joint protein-to-protein and protein-to-annotation alignment to exploit the internal knowledge of PLMs.

• We achieve SOTA retrieval and RAG-based protein function description performance, demonstrating the value of adaptive layer fusion for protein search and annotation.

Related Work

LLMs for Protein Understanding

Protein-oriented LLMs mainly integrate proteins through direct tokenization or encoder-based alignment. Direct tokenization approaches model amino-acid tokens alongside natural language (Wang et al., 2024b; Xia et al., 2025), with further developments in both tokenization and training objectives, where Intern-S1 [3] introduces modality-specific tokenization and AMix-2 [25] adopts block-wise diffusion for joint protein–text modeling. Encoder-based approaches instead use projectors or Q-Formers to align protein features with an LLM [21] (Zhuo et al. 2024; Zhou et al. 2026; Wang et al. 2025). Within this paradigm, Prot2Text-V2 [10] strengthens protein–text alignment through contrastive learning, while BioReason-Pro [8] emphasizes biological reasoning through reinforcement learning. Despite these advances, Zhuang et al. (2026) identify limitations in both approaches, motivating an alternative that supplies LLMs with structured biological context extracted by external tools.

Reliable protein function description remains challenging for LLMs due to the inherent modality gap. Wu et al. (2025) show that simple retrieval can outperform protein-oriented LLMs and propose a dual-index retrieval method combining MMseqs2 (Steinegger and Söding, [29]) and ESM (Lin et al., [19]). ProtQueSt (Ma et al., [22]) ensembles ESM and ProstT5 (Heinzinger et al., [11]) features with query-conditioned contrastive learning to retrieve annotations suited to the biological question. These methods improve retrieval through more external similarity signals. Our work focuses on the knowledge within a single PLM, learning to combine its layer features according to the query.

Layerwise Probing in PLMs

Recent studies have probed the biological knowledge encoded within PLMs. Rao et al. [27] and Lin et al. [19] show that PLMs learn structural information from sequences alone that supports residue contact prediction and atomic-level folding. More specifically, Vig et al. (2021) find that attention captures residue contacts and binding sites, with deeper layers attending to increasingly complex biophysical properties. Kumar and Jha [12] probe ESM-2 layers for kinase functional prediction and show that selected intermediate layers out-perform the final layer. From a geometric perspective, Beshkov and Malthe-Sorenssen [4] trace changes in feature geometry across depth and find that structural fidelity is highest near, but before, the final layer. Together, these studies establish that the most useful biological features need not occur at the encoder output. We investigate how these differences affect protein retrieval and how query-guided fusion can turn them into more effective retrieval features.

Multi-Layer Feature Fusion

Motivated by the observation that earlier visual layers preserve visual details that are not fully captured by final layers, recent work has studied multi-layer fusion in multimodal LLMs. Dense Connector (Yao et al. (2024)) explores integrating features from multiple visual encoder layers to enrich visual input. Chen et al. [6] show that shallow and middle layers are particularly useful for fine-grained visual tasks and develop a lightweight layer-concat fusion strategy. Li et al. (2026) and Lin et al. [16] use textual queries to adaptively determine the weights of visual features from different layers. Lin et al. [18] conduct a systematic comparison and find that combining features from different encoder layers improves generalization and directly fusing visual features at the LLM input, rather than injecting them into its intermediate layers, provides consistently strong performance. We extend layer fusion to PLMs and adapt it to retrieval through joint protein-to-protein and protein-to-description contrastive objectives.

Method

We study retrieval-augmented protein function description given natural-language queries. Each sample is a tuple $(p, q, d)$ comprising a protein sequence $p$, a natural-language query $q$, and its functional description $d$. Given a reference database $\mathcal{D} = \{(p_i, q_i, d_i)\}_{i=1}^{M}$, retrieval-augmented prediction selects $K$ reference samples $S_K$ to enhance LLM generation of $d$. For a fixed LLM generator $P_{\phi}$, the ideal evidence set satisfies

Our goal is to train a retriever that selects such evidence. However, directly optimizing Equation (1) is expensive, as the $\binom{M}{K}$ possible combinations form a prohibitively large search space, and the evaluation requires costly LLM inference. Prior work shows that improving retrieved evidence can enhance generation quality [14, 28]. We therefore focus on retrieval effectiveness as a proxy for evidence quality:

where $\mathcal{T}$ is the training set, $\pi_{\theta}(p, q)$ ranks the samples in $\mathcal{D}$ by $(p, q)$, and $\mathcal{R}(d)$ is the set of oracle samples in $\mathcal{D}$ that match description $d$. $\mathcal{U}$ measures ranking quality, such as Recall@$K$.

Figure 2 illustrates the QProtFuse framework. We evenly divide consecutive PLM layers into groups and mean-pool their features, then use the query to predict group weights for adaptive fusion (§ 3.1). We train QProtFuse with two complementary objectives: intra-modal protein-to-protein (PP) alignment and cross-modal protein-to-annotation (PA) alignment (§ 3.2). The same model supports protein retrieval, annotation retrieval, and two-stage retrieval with reranking (§ 3.3).

Query-Guided Layer Fusion

Grouping layer features. For a protein $p$ with $n$ residues, a frozen ESM2-650M encoder [19] produces residue features $\mathbf{H}^{(\ell)}(p) \in\mathbb{R}^{n \times1280}$ at each of its $L = 33$ transformer layers. We average over residues to obtain a protein feature

for each layer. As shown in Figure 3, neighboring ESM layers typically yield highly similar protein features, suggesting redundancy in routing over every layer independently. We therefore follow the consecutive-layer grouping strategy of Li et al. (2026), and partition the layers into $G = 7$ groups $\{\mathcal{G}_g\}_{g=1}^{G}$ and average within each group:

Grouping reduces the number of fusion weights while retaining features from different depths.

Query-guided fusion. As shown in prior studies [4] (Vig et al. 2021), different PLM layers capture distinct biological properties with varying relevance to downstream tasks. We therefore condition layer fusion on the query, adapting each group's contribution to the requested functional aspect. A frozen text encoder $T$ maps $q$ to $\mathbf{e}_q = T(q)$, from which an MLP predicts group weights:

Using these weights, we fuse the group features to obtain the final query-conditioned protein feature:

A high-level view. QProtFuse makes the features of a protein conditional on the biological question. From a broader perspective, this follows the principle of conditional representation learning, which adapts features to a specified task [20], and is consistent with the query-conditioned protein retrieval formulation of ProtQueSt [22]. Our conditioning acts on the mixture of PLM layers, where the same sequence can receive different combinations of features from a frozen PLM under different queries.

Joint Contrastive Objectives

A natural way to construct $\pi_\theta(p,q)$ in Equation (2) is to use $z(p \mid q)$ from Equation (5) to rank candidates according to their descriptions. This leads to protein–text cross-modal contrastive learning, as used by Ma et al. [22]. However, cross-modal alignment alone can neglect within-modality relationships and degrade feature quality (Yang et al., 2022). We therefore combine intra-modal PP and cross-modal PA alignment objectives to encourage query-conditioned similarity between proteins while aligning them with their annotations.

Protein-to-protein alignment. Given positive and negative pairs with labels $y_{ij} \in\{+1,-1\}$, the PP objective learns to distinguish functionally related proteins under the query. We encode the query protein and each candidate with the same instruction $q_i$ and compute their similarity:

We optimize the expected SigLIP loss (Zhai et al., 2023) over training queries $s_i \in\mathcal{T}$ and candidates $s_j \in\mathcal{D}$:

Here $\sigma$ is the sigmoid function, and both the temperature $\tau$ and bias $b$ are learnable. During training, $\mathcal{D}=\mathcal{T}$, with candidate selection specified below. This objective increases query-conditioned similarity for positive protein pairs and decreases it for negative pairs.

Protein-to-annotation alignment. The PA objective aligns the same query-conditioned protein features with candidate annotation features $\mathbf{u}(d_j)$, obtained from the same frozen text encoder $T$ used for queries and a separate annotation projector. We define the similarity and expected loss as

This objective draws protein features toward matching annotations and pushes them from unrelated descriptions. We combine the two objectives into the final objective:

Both objectives update the fusion network. The ESM and text encoder remain frozen.

Positive and negative pairs. For a query sample $s_i=(p_i,q_i,d_i)$, positives share its target description, while eligible negatives have similar instructions but dissimilar descriptions. We adopt the setting of ProtQueSt [22] and define

$\tau_{\mathrm{query}}$ restricts both positive and negative pairs to samples with similar instructions, while $\tau_{\mathrm{fn}}$ filters out potential false negatives with similar annotations. From $\mathcal{N}_i$, we select up to $H$ hard negatives with the highest similarity under all-layer-mean ESM features and unweight them during training. Positive pairs receive $y_{ij}=+1$ and negative pairs receive $y_{ij}=-1$.

Inference

Joint training enables QProtFuse to retrieve evidence for RAG through three complementary modes: protein retrieval, annotation retrieval, and protein retrieval with annotation reranking. All three modes use the same checkpoint.

Protein retrieval. The PP branch performs sequence-based protein retrieval using Equation (6). The same protein can yield different rankings when queried with respect to distinct biological perspectives, such as catalytic activity or domain organization. This makes protein retrieval responsive to the biological question, while retaining its applicability to candidates with missing, incomplete, or unreliable annotations.

Annotation retrieval. The PA branch supports cross-modal search from a protein and its query to functional descriptions using $r_{ij}^{\mathrm{PA}}$ in Equation (8). Existing methods [30] (Zhou et al. 2025) align text with proteins whose features are independent of functional queries. QProtFuse, in contrast, conditions the protein features on $q$, allowing annotation relevance to vary with the requested functional aspect. This enables direct retrieval of textual evidence tailored to the query.

Protein retrieval with annotation reranking. For databases containing paired proteins and annotations, QProtFuse enables us to retrieve a shortlist by using PP to identify proteins from query-conditioned features, and then rerank its entries using PA scores, which refine the order based on how well annotations match the query protein and question. This reranking step can improve RAG by prioritizing relevant evidence within an initial candidate set (Yu et al., 2024), thereby surfacing informative annotations for downstream generation.

Experiments

In this section, we investigate the value of exploiting a single PLM’s multi-layer features for retrieval-augmented protein function prediction. Through baseline comparisons and controlled ablations, we address three questions: (1) Can QProtFuse improve protein retrieval? (2) How do query-guided fusion and the PP and PA objectives contribute? (3) Do retrieval improvements translate into better RAG-generated function descriptions?

Experimental Setup

Dataset. We use Prot-Inst-OOD (Wu et al., 2025), an instruction-based protein understanding benchmark covering protein function, catalytic activity, domain/motif, and general function. It uses an out-of-distribution split to reduce annotation leakage between training and test sets. We train QProtFuse on the training split. Evaluation follows official test-to-train retrieval: all 14,503 test samples serve as queries, and the 284,526 training records from all four tasks form a single shared reference database. Appendix § 8 provides task descriptions, split construction, and dataset statistics.

Metrics. We report Recall@$K$ (R@$K$) and Mean Reciprocal Rank (MRR) for retrieval. For generated descriptions, we report the standard NLP metrics ROUGE-L [17] for lexical overlap and Entity-BLEU (Wu et al., 2025), a biological entity-focused metric which computes the standard BLEU score [24] on extracted biological entities. We also adopt ProtQueSt’s LLM-as-a-judge metric [22], assessing biological recall, precision, specificity, and plausibility. Scores are computed per task and macro-averaged. Definitions and calculations of these metrics are provided in Appendix § 9.

Baselines. We compare QProtFuse with two representative RAG-based function description methods: RAPM (Wu et al., 2025) and ProtQueSt [22]. For retrieval, we additionally include ESM all-layer mean and the sequence-based retriever MMseqs2 for protein-to-protein retrieval, and ProTrek [30] for protein-to-text retrieval. For generation, we also compare general-purpose LLMs without RAG and protein-oriented LLMs. All RAG-based methods use Top-10 retrieved references per query for generation. Generation details are provided in Appendix § 10.

Implementation details. QProtFuse uses OpenAI’s text-embedding-3-large (3072-D) for query and annotation embedding. We set $\tau_{\mathrm{query}} = 0.65$, $\tau_{\mathrm{fn}} = 0.70$, and $H = 128$, and initialize the learnable SigLIP temperature $\tau$ and bias $b$ to 0.07 and $-10$, respectively. Further training hyperparameters are provided in Appendix § 11. We apply instruction-based candidate filtering with a similarity threshold of 0.65 across retrieval methods.³ We apply this filter because each record is annotated only for the functional aspect specified by its instruction. Otherwise, a relevant protein could be treated as a false negative simply because its recorded answer addresses a different question.

Results

Retrieval Results

Table 1 shows that QProtFuse achieves the highest scores on all three metrics within both PP and PA retrieval. Compared with ESM all-layer averaging, QProtFuse improves MRR from 14.0 to 14.7 and $R@10$ from 24.6 to 25.9, demonstrating gains beyond uniform fusion of the PLM features. QProtFuse also consistently outperforms RAPM and ProtQueSt in PP retrieval, as well as ProtQueSt in PA retrieval. These gains over methods integrating additional external signals highlight the value of exploiting the rich inner knowledge within a single PLM. Adding annotation reranking to PP retrieval further improves performance for both ProtQueSt and QProtFuse, indicating that relevant samples are ranked higher, which can benefit downstream RAG (Yu et al., 2024). QProtFuse outperforms reranked ProtQueSt on all three metrics and achieves the highest MRR and $R@1$ among the Hybrid methods. We use this reranking configuration for downstream RAG. Note that PA ranks only 59,065 candidate annotations, whereas PP and Hybrid retrieve from the full training set ($\sim284\mathrm{K}$). Their scores are therefore not directly comparable.

Method	MRR@10	R@1	R@10
Protein to protein
MMseqs2	9.5	5.5	19.0
ESM-650M$_{\mathrm{all-layer\ avg.}}$	14.0	9.3	24.6
RAPM	11.0	6.0	20.7
ProtQueSt$_{\mathrm{struct-only}}$	12.5	8.3	21.7
QProtFuse$_{\mathrm{pp}}$	14.7	9.7	25.9
Protein to annotation
ProTrek	12.3	7.9	22.7
ProtQueSt$_{\mathrm{FiLM-only}}$	17.5	12.4	28.2
QProtFuse$_{\mathrm{pa}}$	20.4	15.5	29.2
Hybrid
ProtQueSt*	8.2	3.0	26.8
ProtQueSt$_{\mathrm{rerank}}$	17.1	15.0	21.7
QProtFuse$_{\mathrm{rerank}}$	18.6	16.4	25.9

RAG Generation Results

Table 2 shows that QProtFuse’s retrieval improvements benefit downstream function description. On

Method	E-BLEU	RG-L	Judge-F
Prot2Text-V2
SFT	28.05	48.18	6.61
Qwen3-8B
LLM-only	0.62	21.66	4.26
RAPM	22.37	28.46	5.63
ProtQueSt	26.69	21.00	6.05
QProtFuse	32.29	31.53	6.22
DeepSeek-V4-Pro
LLM-only	0.79	26.00	5.05
RAPM	20.66	30.93	6.26
ProtQueSt	35.43	21.20	6.51
QProtFuse	28.73	30.95	6.68

Qwen3-8B (Yang et al., 2025), QProtFuse outperforms RAPM and ProtQueSt on all three metrics. Pairing QProtFuse with DeepSeek-V4-Pro (Xu et al., 2026) yields the highest Judge-F among all compared methods, although ProtQueSt retains the highest Entity-BLEU. The task-level results and judge scores in Appendix § 12 further show that QProtFuse improves judge precision over ProtQueSt across all four tasks with both LLMs, suggesting that its retrieved evidence supports more focused, biologically justified descriptions. These results highlight the value of query-guided retrieval for generation quality beyond lexical entity overlap.

Ablation and Analysis

We conduct controlled ablations to isolate the contributions of query-guided fusion and joint training. We further analyze the learned group weights to examine how QProtFuse combines features from different PLM depths.

Query-guided fusion improves retrieval effectiveness. We compare QProtFuse with fixed mean and global weights variants in Figure 4 (left) to isolate the value of query guidance. QProtFuse substantially outperforms both in PA and Hybrid retrieval, indicating better cross-modal alignment between protein and text. While PP retrieval performance remains comparable across the three strategies, we speculate that the pooled PLM features already retain sufficient information for protein-to-protein matching, which the final MLP projection can extract under the joint objectives. Furthermore, learning global weights offers little improvement over fixed mean, highlighting the importance of query-aware layer fusion for effective retrieval.

The PP and PA objectives reinforce each other. As shown in Figure 4 (right), joint training improves both retrieval modes over their respective single-objective models, with a larger gain in PA. This suggests that intra-modal contrastive learning improves protein feature quality and facilitates cross-modal alignment, while cross-modal supervision enriches the functional information captured by these features, benefiting protein retrieval. These findings support the complementary roles of intra-modal and cross-modal contrastive learning (Yang et al., 2022).

QProtFuse learns adaptive fusion weights. Figure 5 shows that joint training produces distinct mixtures of early, middle, and late layer features across tasks. Catalytic queries combine early (1-5) and late (26-29) layers, general-function queries favor the final group, and domain/motif queries draw more broadly on different depths. Single-objective training often exhibits weight collapse toward a single group. We speculate that this limits the expressiveness of the fused features and contributes to weaker retrieval performance. Global weights, in contrast, remain nearly uniform.

Figure 6 further reveals distinct fusion patterns for three queries. Subcellular localization emphasizes early-to-middle layers, biological-process prediction combines early and late features, and general property description relies predominantly on the final group. The contrast between localization and broader property description is qualitatively consistent with prior findings that lower PLM layers capture local structural information, whereas deeper layers encode more complex structural and functional properties [27] (Vig et al. 2021). More examples of query-specific group weights are provided in Appendix Figure 8. Together, these weights may offer insight into how features from different PLM depths are used to address distinct biological queries.

Conclusion

In this work, we investigated multi-layer PLM features for protein retrieval and showed that the best-performing layers vary across biological tasks, making conventional final-layer selection and naive all-layer averaging suboptimal. Building on these findings, we proposed QProtFuse, which adaptively fuses layer features according to the query and jointly optimizes protein and annotation retrieval through intra-modal and cross-modal contrastive learning. On Prot-Inst-OOD, QProtFuse achieves the strongest protein and annotation retrieval performance and improves downstream function description. Ablations confirm the value of query guidance and the complementary training objectives, while weight analysis reveals distinct fusion patterns for different biological queries. These findings highlight adaptive use of a single PLM’s internal knowledge as an effective approach to retrieval-augmented protein function prediction.

Limitations

QProtFuse is currently limited by its evaluation scale and its decoupled retrieval-generation design. Our current evaluation focuses on Prot-Inst-OOD, which provides a controlled setting for studying query-guided protein retrieval and its effect on function description. Scaling to larger protein resources, such as UniProt [31], could provide more diverse biological knowledge and support further performance gains. In addition, our method focuses on improving retrieval while keeping the LLM frozen. Generated responses are not subsequently refined or verified against the retrieved evidence. This design isolates the contribution of retrieval and leaves room for further improvements through evidence-aware generation, response verification, and joint optimization of retrieval and generation.

Ethical Considerations

Potential Misuse. Query-guided feature fusion can support protein research, but it also carries potential risks of misuse. Malicious queries could direct the system to identify proteins relevant to harmful biological applications. Responsible deployment should therefore incorporate query screening, appropriate access controls, and human review of potentially sensitive requests.

AI Assistance. We used OpenAI Codex to assist with code development and manuscript drafting. For each manuscript section, the authors provided an outline specifying the logical structure, from which the AI produced an initial draft. The authors then revised and polished these drafts. The authors take responsibility for the final manuscript, implementation, and reported results.

Single-Layer Features versus Uniform Averaging

Table 3 compares uniform averaging with the best single layer selected post hoc by MRR@10 for each task. The selected layers vary across tasks and generally outperform uniform averaging, motivating adaptive fusion over a fixed average.

Task	Layer	MRR@10	R@1	R@10
Best single layer
Catalytic activity	17	17.46	13.39	25.97
Domain/motif	15	21.40	13.87	38.51
General function	29	5.75	3.26	11.61
Protein function	30	12.66	8.00	23.62
All-layer mean
Catalytic activity	1–33	16.89	12.78	25.87
Domain/motif	1–33	20.45	12.88	37.85
General function	1–33	5.65	3.16	11.43
Protein function	1–33	12.82	8.26	23.36

Prot-Inst-OOD Dataset

Prot-Inst-OOD is constructed by repartitioning the protein understanding tasks in Mol-Instructions (Fang et al., [9]). Its four tasks cover protein functions, including molecular roles, biological processes, and localization; catalytic activities and reactions; domains and motifs; and general functional descriptions. Protein sequences are clustered with MMseqs2, and clusters are randomly assigned to training and test sets at an initial ratio of 80:20. To reduce annotation leakage, each test protein is matched to its nearest training protein with MMseqs2. Test samples sharing the same label as their retrieved counterpart are moved to the training set, and this filtering step is repeated twice. Table 4 summarizes the sample counts and the number of distinct instructions for each task.

Task	Train	Test	Instructions
Protein function	108,696	5,487	38
Catalytic activity	51,187	1,987	5
Domain/motif	42,368	2,732	10
General function	82,275	4,297	7
Total	284,526	14,503	60

Evaluation Metrics

Retrieval metrics. A retrieved record counts as a hit only when its bio-entity annotation exactly matches the query protein’s ground-truth annotation. We report Recall@1 and Recall@10, defined as the fractions of queries with at least one hit among the top 1 and top 10 results, respectively. MRR@10 additionally measures how highly the first hit is ranked:

where $N$ is the number of evaluated queries and $r_i$ is the rank of the first hit for query $i$. All three metrics are computed per task and macro-averaged across the four tasks.

ROUGE-L. ROUGE-L (Lin, [17]) measures overlap through the longest common subsequence (LCS) between a prediction $\hat{d}$ and reference $d$. We report its F1 score:

where lengths and LCS are computed over tokens. We use the Python package rouge-score version 0.1.2 with RougeScorer(["rougeL"], use_stemmer=False).

Entity-BLEU. Entity-BLEU (Wu et al., 2025) measures biological content overlap rather than overlap in generic response wording. Biological entities are first extracted from predictions and references using RAPM’s vocabulary derived from InterPro, EC-GO, and Mol-Instructions metadata. Standard BLEU (Papineni et al., [24]) is then computed on the extracted sequences:

where $p_n$ is the corpus-level clipped $n$-gram precision of the extracted sequences, and $\operatorname{BP}=\min\{1,\exp(1-L_{\mathrm{ref}}/L_{\mathrm{pred}})\}$ penalizes predictions shorter than their references. Here, $L_{\mathrm{pred}}$ and $L_{\mathrm{ref}}$ are the total extracted sequence lengths. We use $k=2$, combining unigram and bigram precision, and report scores as percentages.

LLM-as-a-judge. Given the instruction, reference description, reference bio-entities, and prediction, the judge assesses four axes: recall of reference entities, precision of supported or reasonably inferred claims, specificity of biological content, and plausibility without contradictions or fabrications. Each active axis receives a score in $\{0, 3, 5, 7, 10\}$. The final per-sample score and the macro-average are

where $g_{i,a}$ is the score for axis $a$, $A_i$ excludes axes marked inactive ($-1$), and $I_t$ contains the $N_t$ evaluated samples for task $t$. The final score ranges from 0 to 10. We refer readers to ProtQueSt for the complete prompt and rubric.

We adopt ProtQueSt’s prompt and scoring rubric [22], using Qwen3.8-27B [26] in place of Claude as the judge. We set the temperature to 0.0 and the total generation budget to 5,024 tokens, including a 4,000-token thinking budget. Following ProtQueSt, we randomly sample 500 predictions per task, yielding 2,000 evaluated samples per method.

Generation Details

Prompt. Figure 7 shows our generation template. All settings use the same two randomly selected examples per task, each formatted as “{Instruction: ..., Protein Sequence: ..., Description: ...}”. Example sequences longer than 80 amino acids are truncated to their first 80 residues followed by ‘...’. The placeholder {retrieved_records} is filled with a list of ten retrieved samples organized as “[{’db_label’: ..., ’protein_similarity’: ..., ’protein_text_alignment’: ...}, ...]”. For RAPM and ProtQueSt, we use their respective published prompts.

Inference and sampling. We run Qwen3-8B locally with vLLM [13] and access DeepSeek-V4-Pro through its online API, using the deepseek-v4-pro-0813 version. Both models use a temperature of 1.0, a maximum output length of 512 tokens, and disabled thinking.

Training Configuration

QProtFuse. Our model has 5.13M trainable parameters. We train it for 30 epochs with three random seeds (20260929, 20260930, and 20261001) using mixed bfloat16 precision on four NVIDIA H200 GPUs. The batch size is 256 per GPU, yielding a global batch size of 1,024. Each training run takes approximately 0.36 GPU hours, excluding the precomputation of ESM features, which are cached locally. We use AdamW with learning rates of $10^{-4}$ for the router and $10^{-3}$ for the remaining modules, and a weight decay of $10^{-4}$. All reported results use the last checkpoint.

Baselines. For Prot2Text-V2, only the projection MLP and the LLM LoRA adapters are trained, with LoRA rank 32, scaling factor 64, and dropout 0.1. Training runs for up to 24 epochs on 16 NVIDIA H200 GPUs using mixed bfloat16 precision. A per-GPU batch size of 4 and four gradient-accumulation steps yield an effective batch size of 256. We use AdamW with a learning rate of $2 \times10^{-4}$, weight decay of 0.01, cosine learning-rate decay, and a random seed of 42. For ProtQueSt, whose code was unavailable for our experiments, we independently implement and train the model following the configuration disclosed in its paper.

Detailed Generation Results

Table 5 reports generation metrics for each task, and Table 6 breaks down the four judge dimensions. QProtFuse achieves higher judge precision than ProtQueSt in every task with both generators.

Method	Protein function E-BLEU	Protein function RG-L	Protein function Judge-F	General function E-BLEU	General function RG-L	General function Judge-F	Domain/motif E-BLEU	Domain/motif RG-L	Domain/motif Judge-F	Catalytic activity E-BLEU	Catalytic activity RG-L	Catalytic activity Judge-F
Prot2Text-V2
SFT	42.39	50.31	7.22	16.82	42.52	5.75	26.27	45.98	7.23	26.74	53.92	6.25
Qwen3-8B
LLM-only	0.04	18.29	4.00	0.02	18.30	3.57	2.40	16.36	4.48	0.00	33.68	4.98
RAPM	29.10	25.32	6.44	7.39	25.69	5.03	38.24	23.79	5.42	14.73	39.05	5.62
ProtQueSt	36.25	22.29	6.63	18.79	24.01	5.43	19.34	8.66	5.73	32.37	29.05	6.38
QProtFuse	34.99	27.18	6.87	10.28	29.33	5.30	48.61	29.25	6.62	35.27	40.36	6.09
DeepSeek-V4-Pro
LLM-only	1.86	25.47	5.02	0.12	26.62	4.23	0.01	13.30	5.76	1.17	38.63	5.20
RAPM	33.92	30.40	6.93	5.29	32.56	5.58	27.23	23.10	6.75	16.21	37.65	5.77
ProtQueSt	48.25	22.11	7.00	18.66	24.58	5.72	37.56	11.53	6.65	37.26	26.60	6.68
QProtFuse	36.34	30.77	7.17	6.05	33.45	5.60	40.57	21.74	7.63	31.97	37.83	6.31

Method	Protein function Rec.	Protein function Prec.	Protein function Spec.	Protein function Plaus.	General function Rec.	General function Prec.	General function Spec.	General function Plaus.	Domain/motif Rec.	Domain/motif Prec.	Domain/motif Spec.	Domain/motif Plaus.	Catalytic activity Rec.	Catalytic activity Prec.	Catalytic activity Spec.	Catalytic activity Plaus.
Prot2Text-V2
SFT	5.26	7.37	8.63	7.36	3.04	6.70	8.17	4.93	4.04	7.90	8.80	7.87	2.30	7.57	9.69	4.82
Qwen3-8B
LLM-only	0.51	4.69	7.45	2.91	0.04	4.95	4.31	4.60	0.56	2.70	7.43	6.71	0.00	6.99	9.81	2.60
RAPM	5.32	5.31	8.74	6.07	3.49	4.54	6.97	4.82	4.08	2.22	8.41	6.64	1.28	7.14	9.69	3.83
ProtQueSt	5.63	5.13	9.17	6.22	4.13	4.29	8.76	4.42	3.76	3.46	8.86	6.43	3.92	6.67	9.44	4.97
QProtFuse	5.92	5.74	8.75	6.66	3.61	5.05	7.27	4.97	5.88	4.52	8.45	7.07	2.44	7.27	9.62	4.46
DeepSeek-V4-Pro
LLM-only	2.00	5.71	6.79	5.30	0.70	5.41	5.76	4.81	0.96	6.84	7.90	6.90	0.58	7.18	9.24	3.37
RAPM	5.67	6.23	8.91	6.59	3.72	5.69	7.24	5.37	3.96	6.59	8.65	7.28	1.80	7.37	9.56	4.02
ProtQueSt	6.65	5.17	9.34	6.44	4.32	4.63	8.58	5.00	4.98	5.20	9.01	6.80	4.76	6.68	9.27	5.57
QProtFuse	6.04	6.53	8.89	6.83	3.49	5.96	7.37	5.26	5.58	7.59	8.96	7.86	2.70	7.67	9.46	4.97