WorksComputer Science
MECI: Multi-Element Collaborative Interaction for Multimodal Entity Linking
Abstract
Multimodal Entity Linking (MEL) aims to disambiguate mentions in multimodal contexts by grounding them to specific entities in a knowledge base. A pivotal challenge in MEL is capturing multi-level correspondences: the semantic consistency between mention-entity pairs and the complementary correlations across modalities. However, existing methods often suffer from element dominance due to their reliance on coupled interactions or coarse global aggregations. In response, we propose the Multi-Element Collaborative Interaction (MECI) framework. First, to capture multi-element mention-entity correspondences, we develop a Multi-view Experts Network that leverages a "divide-and-conquer" strategy for decoupled feature learning to mitigate element dominance, supported by a KL-guided routing mechanism that governs expert specialization and collaboration. Furthermore, to model cross-modal complementary correlations, we propose a Hierarchical Multimodal Interaction Module, where a dynamic modality-aware weighting network refines interactions across hierarchical semantic levels, thereby integrating multi-granular evidence to counteract element dominance. Finally, we incorporate a generative semantic refinement stage that utilizes large language models for zero-shot re-ranking. Extensive experiments on WikiDiverse, RichpediaMEL, and WikiMEL show that MECI consistently outperforms state-of-the-art baselines, improving Hits@1 by 1.95%, 7.30%, and 2.31%, respectively.
CCS Concepts
• Information systems → Specialized information retrieval; Learning to rank.
ACM Reference Format:
Jie Peng, Yongxue Shan, Yongfu Zha, and Xiaodong Wang. 2026. MECI: Multi-Element Collaborative Interaction for Multimodal Entity Linking. In Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’26), July 20–24, 2026, Melbourne, VIC, Australia. ACM, New York, NY, USA, 11 pages. https://doi.org/10.1145/3805712.3809583
Introduction
Entity Linking (EL) serves as a fundamental bridge connecting real-world information to knowledge bases. Over the past years, large efforts focused on text-based EL [3, 35]. However, images and text have become the most widely seen medium for publishing and understanding web information. Therefore, Multimodal Entity Linking (MEL) [7, 16] has emerged as a pivotal evolution, attracting attention from both academia and industry. By leveraging textual and visual cues, MEL plays a crucial role in knowledge-intensive applications such as semantic search [12, 40], question answering [21, 26, 37], and dialog systems [1, 13, 31]. As illustrated in Figure 1, given a multimodal mention, the MEL model aims to link it to the entity Canberra United in a multimodal knowledge base.

Figure 1. An illustration of the MEL task: multimodal mentions (left) and candidate entities (right).
To effectively leverage multimodal information in MEL, a pivotal challenge is to identify and utilize the multi-element correspondences inherent in entity and mention contexts: 1) Multi-element correspondences between mentions and entities: Both the mention context and the entity representation often contain multiple consistent cues that facilitate semantic matching. As shown in Figure 1, the mention and the entity share textual elements such as Canberra, football, and University of Canberra. These correspondences highlight semantic similarities and provide explicit evidence for accurate entity linking, thereby reducing ambiguity and enhancing matching confidence. 2) Multi-element correspondences between modalities: The textual content and visual patches may exhibit correlative correspondences that are pivotal for comprehensive understanding. For instance, in Figure 1, the presence of a football in the image, combined with the textual context, helps definitively identify the entity as a football team—rather than a rugby or basketball team. Such cross-modal interactions allow textual and visual information to complement each other, enriching the model’s discriminative capacity for entity disambiguation.
This work is licensed under a Creative Commons Attribution 4.0 International License.
SIGIR ’26, Melbourne, VIC, Australia
© 2026 Copyright held by the owner/author(s).
ACM ISBN 979-8-4007-2599-9/2026/07
https://doi.org/10.1145/3805712.3809583 To facilitate these multi-element correspondences, existing studies propose sophisticated correlative mechanisms, such as hierarchical gating mechanisms [20, 30, 33], attention-based paradigms [19, 22, 29], graph-based aggregation schemes [2, 4, 36], and adaptive fusion strategies [8, 28, 38]. Despite their progress, these methods face inherent challenges in handling the complexity of heterogeneous inputs. Specifically, hierarchical gating and attention-based approaches often employ coupled interaction designs across element levels (e.g., coupling coarse- and fine-grained semantics), which hinders adaptive adjustment to the characteristics of input semantics and readily leads to element dominance issues in the competition among heterogeneous features. Furthermore, graph-based and fusion approaches typically compress textual and visual features into a coarse global representation. This representation-level bottleneck makes it difficult to capture fine-grained, context-dependent associations, such as fine-grained visual correlations between entity and mention. In summary, existing designs tend to over-rely on prominent elements, leading to element dominance, where stronger modalities often overshadow weaker ones (modality dominance) or coarse global representations suppress fine-grained details (granularity bias). Such dependencies severely limit their capacity to model multi-element correspondences comprehensively, ultimately compromising entity linking accuracy in complex multimodal scenarios.
To mitigate these limitations and comprehensively capture multi-element correspondences, we propose the Multi-Element Collaborative Interaction (MECI) framework for multimodal entity linking. First, targeting the multi-element correspondences between mentions and entities, we introduce the Multi-view Experts Network (MVEN), which employs a “divide-and-conquer” strategy to capture both universal and view-specific characteristics. To regulate expert specialization and knowledge sharing, we propose a KL-guided routing mechanism that utilizes structural priors to adaptively coordinate information flow. Then, incorporating with a hierarchical matching strategy, MVEN provides specialized processing capacities for multi-element correlation analysis, effectively mitigating the element dominance problem. However, while MVEN effectively refines view-specific representations, these features remain relatively isolated. To bridge this gap and fully exploit the multi-element correspondences between modalities, we develop the Hierarchical Multimodal Interaction Module (HMIM).
This module employs a dynamic cross-modal weighting network to refine interactions at distinct granularity levels—avoiding the granularity bias inherent in unified interaction paradigms—and utilizes a granularity-aware gating network to dynamically integrate multi-granular evidence for comprehensive cross-modal understanding. Finally, to further enhance linking precision, we introduce a finetuning-free generative semantic refinement stage, which reformulates candidate re-ranking as a zero-shot multiple-choice task to leverage the parametric knowledge of frozen Large Language Models (LLMs). In summary, our main contributions are threefold:
We propose the Multi-view Experts Network for mention-entity correspondences. It employs a “divide-and-conquer” strategy with specialized routing and matching to decouple heterogeneous features, mitigating element dominance.
We develop a Hierarchical Multimodal Interaction Module for cross-modal correspondences. It employs dynamic weighting to avoid granularity bias and a gating network to incorporate multi-granular evidence.
Experiments on WikiMEL, RichpediaMEL, and WikiDiverse demonstrate MECI’s superiority over state-of-the-art baselines, while ablation studies validate the efficacy of each proposed component.
Related Work
Text-based Entity Linking
Text-based Entity Linking (EL) aims to ground ambiguous textual mentions to their corresponding entities in a knowledge base. Generally, existing methodologies fall into two paradigms: local-level and global-level models. Local-level models [3, 23, 35, 42] focus on utilizing the immediate context, such as surrounding words or sentences, to aid disambiguation. In contrast, global-level models [6, 10, 25, 39] incorporate entity or topic coherence to ensure semantic consistency between mentions and entities. Nevertheless, these unimodal methods often struggle to effectively handle multimodal data that combines both textual and visual information.
Multimodal Entity Linking
Multimodal Entity Linking (MEL) extends traditional entity linking by leveraging visual context for disambiguation. Its main goal is to map mentions from multimodal content to specific entities in a multimodal knowledge base. To address this, existing studies have proposed various sophisticated mechanisms, which can be broadly categorized into distinct paradigms. First, hierarchical gating mechanisms [8, 20, 30, 33] and attention-based paradigms [7, 22, 29] aim to refine feature matching through multi-level structures or cross-modal attention. However, these methods often employ coupled interaction designs, limiting their adaptability to varying input semantics and leading to element dominance. Meanwhile, graph-based aggregation schemes [2, 4, 36] explicitly model cross-modal relations as graphs, whereas adaptive fusion strategies [28, 38] aggregate features into unified representations. Despite their utility, these approaches tend to fuse information into global representations, making it difficult to capture fine-grained, context-dependent visual-textual associations. Recently, LLM-based methods [16, 17, 27] have achieved remarkable progress by leveraging the generative capabilities of large models, yet remain significantly constrained by excessive computational resource consumption. In summary, existing methods exhibit limitations in comprehensively modeling multi-element collaboration across modalities and granularities. Current designs tend to over-rely on prominent elements, causing modality dominance and granularity bias, which ultimately undermines linking performance in complex multimodal scenarios.
Methodology
Problem Formulation
MEL aims to resolve a mention to its target entity from a knowledge base . Formally, an entity is defined as a tuple , consisting of textual attributes and a visual representation . Similarly, a mention is modeled as , containing the textual context and an associated image . To align mentions with entities, the model parameters are optimized to maximize the likelihood of the ground-truth on the training set :
Here, serves as a similarity metric to evaluate the relevance of a candidate entity to the query mention .
Feature Encoding
To initialize the representations for the entity and mention , the model encodes their associated textual and visual information, as illustrated in Figure 2 (step 1). Specifically, the framework leverages the pre-trained CLIP model (specifically CLIP–ViT–Base–Patch32). We utilize its text encoder () for textual feature extraction and its image encoder () for visual feature encoding:
Figure 2. The Overall Framework of MECI, illustrating a sequential workflow: (1) Feature Encoding → (2) Multi-view Experts Network → (3) Hierarchical Multimodal Interaction → (4) Generative Semantic Refinement.
where and represent the token-level textual sequences (lengths ), while and denote the patch-level visual sequences (patch counts ). Beyond these fine-grained representations, we extract coarse-grained global vectors to capture comprehensive semantic context. These are obtained via the output of the special [EOS] token from and the [CLS] token from , respectively.
Multi-view Experts Network
To effectively model multi-element correspondences between mentions and entities, we recognize that inputs of varying modalities and granularities offer diverse and complementary evidence from distinct perspectives. However, existing strategies, such as feature-coupled interactions [8, 22] or global alignment [28, 38], often suffer from element dominance, thereby struggling to comprehensively capture these dependencies.
To address these challenges, we propose the Multi-view Experts Network (MVEN). By adopting a decoupled representation learning paradigm, MVEN ensures specialized processing capabilities for heterogeneous features, thereby effectively mitigating element dominance. Specifically, MVEN comprises three key components: (1) a View-Specific Experts Module employing a “Divide-and-Conquer” strategy to capture universal patterns and view-specific characteristics; (2) a KL-Guided Routing Mechanism introducing structural priors to balance expert specialization and knowledge sharing; and (3) a Gated Aggregation and Matching Layer that dynamically fuses expert signals and performs specialized correlations computation across diverse perspectives.
View-Specific Experts Module
To capture semantic information, we adopt a “Divide-and-Conquer” strategy to model both common patterns and view-specific characteristics.
Universal Semantic Projection. Formally, we formulate each expert unit as a learnable transformation comprising a linear projection, normalization, and activation. Let denote the multimodal input features for either a mention or an entity. Based on this, we employ a shared expert mechanism to extract view-agnostic semantic structures. Accordingly, for any arbitrary input token , the universal expert projects it into a unified semantic space:
where and denote the learnable parameters, and represents a non-linear activation function (such as ReLU).
Collaborative Intra-Expert Fusion. To capture the inherent characteristics unique to each perspective, we establish two distinct expert pools: a modality expert set and a granularity expert set . Despite the greater number of fine-grained tokens, we maintain an equal number of experts for both granularities. This symmetric design ensures balanced interaction and prevents potential fusion bias from imbalanced expert scales. To enrich the representation of a specific view without compromising its distinctiveness, we propose a Collaborative Intra-Expert Fusion mechanism. For a given sample belonging to category , let denote its ground-truth view index (e.g., a specific modality ID). The fused representation is computed as a dynamic weighted aggregation:
where represents the native expert corresponding to the sample’s identity, and the summation term incorporates complementary knowledge from peer experts within the same pool. The coefficient acts as a dynamic trade-off parameter, governing the balance between preserving native specificity and incorporating external knowledge.
KL-Guided Routing Mechanism
The efficacy of the above collaborative fusion depends significantly on the routing mechanism that determines the trade-off coefficient . Thus, we introduce a KL-Guided Routing Mechanism, a novel approach that leverages structural priors to regularize the routing distribution. Specifically, for each input token , we first identify its categorical identity:
let and denote its ground-truth modality and granularity identities, respectively. Instead of employing these as rigid one-hot targets, we construct “Soft Target” distributions to serve as regularizers. This is achieved by applying temperature scaling to the one-hot vectors :
where denotes the one-hot vector for index . The temperature modulates the prior’s focus: a lower sharpens the distribution to enforce expert specialization, while a higher flattens it to encourage inter-expert collaboration and knowledge sharing.
Simultaneously, for each input , the router predicts the routing distributions via learnable projections:
where and are learnable parameters of the routing network.
To align the learned policy with our structural priors while permitting data-driven adaptation, we minimize the Kullback-Leibler (KL) divergence between the predicted distribution and the soft target :
where denotes the number of experts in category (i.e., and ).
This formulation aligns the router with semantic priors while preserving input context-driven flexibility. By softening via the temperature mechanism, the model assigns non-zero probabilities to peer experts, thus avoiding the knowledge isolation typical of hard routing. Finally, the dynamic trade-off coefficient is instantiated as the predicted probability of the native expert: , where .
Gated Inter-Expert Aggregation
Building upon the universal (), modality-specific (), and granularity-specific () representations, we introduce a Gated Inter-Expert Aggregation strategy to integrate these heterogeneous signals. Specifically, for an input token , we determine the salience of each expert category by computing gating coefficients:
These coefficients adaptively modulate expert contributions to produce the fused representation:
This residual-like formulation maintains a stable universal foundation while flexibly incorporating evidence tailored to the specific context. Consequently, the final representations for a mention and a candidate entity are constructed as the concatenation of all view-specific features: .
Hierarchical Matching Strategy
To accommodate the distinct matching requirements of heterogeneous perspectives, we categorize the full view set into global views and local views , where and denote textual and visual modalities, and subscripts and represent global and local granularities, respectively. For global views , we prioritize Global Semantic Coherence. Given the global feature vectors and , we compute their correlation via cosine similarity:
Conversely, for fine-grained views , we employ a Local Evidence Accumulation layer. Given local feature sequences and , we first construct context-aware entity prototypes via attention-based aggregation: . This mechanism dynamically aligns entity features with the mention context, effectively highlighting relevant evidence segments while suppressing noise. The local score is then obtained by averaging the cosine similarity at each local position:
Finally, the comprehensive similarity is derived by averaging the matching scores across all global and local views:
Hierarchical Multimodal Interaction
While the Multi-view Experts Network effectively models mention-entity correlations through decoupled representations, these view-specific features remain relatively isolated, limiting the model’s ability to capture multi-element correspondences between modalities. To bridge this gap, we develop the Hierarchical Multimodal Interaction Module (HMIM). HMIM first employs a Dynamic Cross-modal Weighting Network (DCWN) to refine interactions at distinct granularities. Subsequently, a Granularity-aware Gating Network dynamically fuses the multi-granular evidence into a robust joint representation for precise matching.
Dynamic Cross-modal Weighting Network
The DCWN processes fine-grained and coarse-grained streams in parallel to model intricate cross-modal dependencies. This hierarchical strategy effectively avoids the granularity bias inherent in unified interaction paradigms, where multi-level features are concatenated without structural distinction. We describe the operation using the fine-grained token stream as an illustrative example. Let denote the input sequence combining textual and visual local tokens derived from the previous module. We employ a multi-head cross-modal attention mechanism with heads to model pairwise correspondences. For the -th head (), the input is projected into query, key, and value subspaces via learnable matrices: , , and , where .
The attention matrix captures the pairwise correspondences between tokens and is computed as:
The context-enriched output is derived by aggregating the values weighted by , followed by concatenation and a linear projection :
Crucially, to explicitly quantify the reliability of cross-modal correspondences and refine the features, we introduce a Token Confidence Estimation (TCE) mechanism. Rather than treating all tokens equally, TCE computes a scalar confidence score for each token , reflecting its saliency within the multimodal context. This is formulated by aggregating the attention scores across all heads and tokens:
This mechanism acts as a soft attention filter, adaptively emphasizing tokens that exhibit strong semantic consistency. The final granularity-specific representation is obtained by re-weighting the enhanced features:
where and denote the aggregated fine-grained (local) and coarse-grained (global) representations, respectively (with denoting the length of the global stream).
Granularity-aware Gating Network
To effectively integrate the refined multi-granular evidence from and , the Granularity-aware Gating Network dynamically fuses them into a robust joint representation. We compute granularity-aware gating scores for via:
where and are learnable parameters. The unified representation is then derived through a weighted aggregation:
This mechanism enables the model to selectively emphasize the granularity that is most informative for the current context. Finally, the matching score is computed as the cosine similarity between the unified mention and entity representations:
Optimization and Generative Refinement
3.5.1 Holistic Training Objectives. We employ a multi-perspective contrastive learning paradigm [20] to enforce discriminative alignment across both individual and fused perspectives. Specifically, we apply contrastive constraints to three distinct scoring components: the expert-based score , the interaction-based score , and the overall consensus score . The InfoNCE loss for a given scoring function (where ) is defined as:
where denotes the set of negative samples corresponding to the anchor . The total objective aggregates these matching losses with the routing regularization term:
where governs the weight of the KL-divergence constraint. This formulation ensures that the model learns robust representations while strictly adhering to the semantic structural priors imposed on the router.
3.5.2 LLM-based Generative Reasoning. Inspired by [14, 16], we incorporate a Generative Semantic Refinement stage to further enhance entity linking precision. Initially, a candidate set is retrieved based on the top- scores :
Subsequently, the refinement stage is reformulated as a multiple-choice problem. By constructing a prompt that integrates the mention context and candidate descriptions —as illustrated in the prompt design in Figure 2 (step 4)—the optimal entity is determined by maximizing the LLM’s generation probability:
where denotes frozen LLM parameters. This way effectively leverages parametric knowledge to pinpoint the ground-truth entity.
Evaluation
In this section, we evaluate our proposed MECI through comprehensive experiments on three public MEL datasets, accompanied by extensive analyses to provide deeper insights into our framework.
Experimental Setup
4.1.1 Datasets. To evaluate our approach, we utilize three established datasets: WikiDiverse [34], WikiMEL [33], and RichpediaMEL [33]. Derived from Wikipedia entity pages, WikiMEL targets Wikidata as its knowledge base and consists of over 22,000 multimodal sentences, focusing predominantly on Person entities. In contrast, WikiDiverse is sourced from Wikinews and maps to Wikipedia; it spans seven distinct entity categories, including Person, Organization, Location, Country, Event, Works, and Misc. RichpediaMEL integrates Wikidata attributes for entities found in Richpedia [32] with multimodal content from Wikipedia. Regarding data partitioning, we adopt the standard split configurations: an
8:1:1 ratio for WikiDiverse, and a 7:1:2 ratio for both WikiMEL and RichMEL. The statistics of these datasets are shown in Table 1.
| Statistics | WikiDiverse | WikiMEL | RichpediaMEL |
| # number of sentences | 7,405 | 22,070 | 17,724 |
| # number of entities | 132,460 | 109,976 | 160,935 |
| # entities with image | 67,309 | 67,195 | 86,769 |
| # number of mentions | 15,093 | 25,846 | 17,805 |
| # image of mentions | 6,697 | 22,136 | 15,853 |
| # mentions in train | 11,351 | 18,092 | 12,463 |
| # mentions in valid | 1,664 | 2,585 | 1,780 |
| # mentions in test | 2,078 | 5,169 | 3,562 |
Table 1. Statistics of three MEL datasets.
4.1.2 Baselines and Evaluation Metrics. We evaluate our method against three groups of representative baselines: (1) Text-based EL methods, such as BERT [3], RoBERTa [15], and BLINK [35]; (2) Discriminative MEL methods, covering CLIP [24], ViLT [9], ALBEF [11], METER [5], GHMF [33], MIMIC [20], OT-MEL [41], MELOV [30], MEL [7], and MMOE [8]; and (3) LLM-based methods, including GEMEL [27], UniMEL [14], IIER [22], and I2CR [16]. To assess model performance, we utilize two standard metrics: Mean Reciprocal Rank (MRR) and Hits@N. MRR calculates the average of the reciprocal ranks of the first correct answer. Hits@N reports the percentage of test cases where the correct entity appears within the top predictions, where .
4.1.3 Implementations. We conducted the experiments on a workstation equipped with an Intel Core i9-12900K CPU (64GB RAM) and two NVIDIA A100 GPUs (2x40GB). we employ the pre-trained CLIP-ViT-Base-Patch32 [24] model as the initialization of the textual and visual encoder. For input preprocessing, visual inputs were resized to a resolution of 224 × 224. Subsequently, the maximum sequence length was set to 50 tokens for both visual and textual inputs. A unified training configuration was maintained across all datasets. The models were optimized using AdamW [18] with a batch size of 64 for 20 epochs. The learning rate was initialized at for WikiMEL and for both the WikiDiverse and RichpediaMEL datasets. In the re-ranking phase, we utilized DeepSeek-v3 to re-rank the retrieved candidates, setting the candidate pool size to . Our code is available at MEL-MECI-main.
Main Results
The comparative results on three MEL datasets are summarized in Table 2. Overall, our proposed MECI framework demonstrates superior performance across all benchmarks. Specifically, our MECI consistently outperforms previous strong discriminative models, while our zero-shot generative refinement model MECI+ achieves new state-of-the-art results. The detailed analysis is as follows:
| Methods | Venue | WikiDiverse | RichpediaMEL | WikiMEL | |||||||||
| MRR | Hits@1 | Hits@3 | Hits@5 | MRR | Hits@1 | Hits@3 | Hits@5 | MRR | Hits@1 | Hits@3 | Hits@5 | ||
| BERT | NAACL’19 | 67.38 | 55.77 | 75.73 | 83.11 | 71.67 | 59.55 | 81.12 | 87.16 | 81.78 | 74.82 | 86.79 | 90.47 |
| RoBERTa | arXiv’19 | 70.52 | 59.46 | 78.54 | 85.08 | 72.80 | 61.34 | 81.56 | 87.15 | 80.86 | 73.75 | 85.85 | 89.80 |
| BLINK | EMNLP’20 | 69.15 | 57.14 | 78.04 | 85.32 | 71.39 | 58.47 | 81.51 | 88.09 | 81.72 | 74.66 | 86.63 | 90.57 |
| CLIP | ICML’21 | 71.69 | 61.21 | 79.63 | 85.18 | 77.57 | 67.78 | 85.22 | 90.04 | 88.23 | 83.23 | 92.10 | 94.51 |
| ViLT | ICML’21 | 45.22 | 34.39 | 51.07 | 57.83 | 56.63 | 45.85 | 62.96 | 69.80 | 79.46 | 72.64 | 84.51 | 87.86 |
| ALBEF | NeurIPS’21 | 69.93 | 60.59 | 75.59 | 81.30 | 75.29 | 65.17 | 82.84 | 88.28 | 84.56 | 78.64 | 88.93 | 91.75 |
| METER | CVPR’22 | 63.71 | 53.14 | 70.93 | 77.59 | 74.15 | 63.96 | 82.24 | 87.08 | 79.49 | 72.46 | 84.41 | 88.17 |
| GHMFC | SIGIR’22 | 70.99 | 60.27 | 79.40 | 84.74 | 80.76 | 72.92 | 86.85 | 90.60 | 83.36 | 76.55 | 88.40 | 92.01 |
| MIMIC | KDD’23 | 73.44 | 63.51 | 81.04 | 86.43 | 86.95 | 81.02 | 91.77 | 94.38 | 91.82 | 87.98 | 95.07 | 96.37 |
| OT-MEL | ACL’24 | 75.43 | 66.07 | 82.82 | 87.39 | 88.27 | 83.30 | 92.39 | 94.83 | 92.59 | 88.97 | 95.63 | 96.96 |
| MELOV | ACL’24 | 76.57 | 67.32 | 83.69 | 87.54 | 88.80 | 84.14 | 92.81 | 94.89 | 92.32 | 88.91 | 95.61 | 96.58 |
| M3EL | KDD’25 | 81.29 | 74.06 | 86.57 | 90.04 | 88.26 | 82.82 | 92.73 | 95.34 | 92.30 | 88.84 | 95.20 | 96.71 |
| MMOE | KDD’25 | 81.72 | 74.59 | 87.25 | 90.33 | 89.04 | 84.36 | 92.93 | 94.64 | 92.53 | 89.07 | 95.36 | 97.02 |
| GEMEL | COLING’24 | - | 86.30 | - | - | - | - | - | - | - | 82.60 | - | - |
| UniMEL | CIKM’24 | - | 86.50 | 89.80 | 90.40 | - | 85.20 | 92.60 | 96.80 | - | 88.70 | 95.20 | 96.90 |
| IIER | ACL’25 | - | 69.47 | 84.43 | 88.79 | - | 84.63 | 93.27 | 95.30 | - | 88.93 | 95.69 | 96.73 |
| I2CR | MM’25 | - | 91.60 | 94.70 | 95.60 | - | 86.80 | 92.90 | 97.20 | - | 92.20 | 96.10 | 97.50 |
| MECI | - | 86.71 | 79.93 | 90.23 | 92.93 | 89.94 | 85.63 | 93.43 | 95.65 | 93.48 | 90.38 | 96.05 | 97.21 |
| MECI+ | - | 95.00 | 93.55 | 96.01 | 96.97 | 95.49 | 94.10 | 96.38 | 97.39 | 96.25 | 94.51 | 97.72 | 98.34 |
Table 2. Experimental results (%) on three MEL benchmark datasets: WikiDiverse, RichpediaMEL, and WikiMEL. The bold scores indicate the best results, and the underlined scores represent the second-best results.
▹ Comparison with Text-based Methods. Text-based EL consistently underperforms multimodal approaches. The best text-only model achieves only 74.82% Hits@1 on WikiMEL compared to over 90% for our method. This suggests these text-based EL baselines struggle to capture deep semantics. Our approach addresses this by learning richer representations from both textual context and auxiliary visual signals.
▹ Comparison with Discriminative MEL Methods. Compared to state-of-the-art discriminative baselines like MMOE and M{sup}3`EL, our MECI achieves consistent superiority, surpassing the strongest baseline MMOE by 5.34%, 1.27%, and 1.31% in Hits@1 on WikiDiverse, RichpediaMEL, and WikiMEL, respectively. This comes from our collaborative design: MVEN employs a “divide-and-conquer” strategy to decouple subspaces and mitigate element dominance, while HMIM utilizes a granularity-dependent cross-modal weighting network to capture intricate cross-modal dependencies. In contrast to baselines limited by coupled interactions or coarse global fusion, MECI effectively alleviates element dominance, capturing comprehensive multi-element correspondences.
▹ Comparison with Generative Methods. Finally, we benchmark MECI+ (MECI augmented with a zero-shot generative refinement stage) against LLM-based paradigms. While I2CR exhibits competitive performance, our zero-shot MECI+ surpasses the strongest baseline by large margins of 1.95%, 7.30%, and 2.31% in Hits@1 across the three datasets, respectively. Crucially, our model realizes these gains without the need for resource-intensive fine-tuning. By leveraging a frozen DeepSeek V3 to re-rank candidates via a multiple-choice prompting strategy, we demonstrate that a robust retrieval foundation is sufficient to effectively elicit the reasoning capabilities of LLMs.
▹ Low Resource Setting. We further examine model performance under data-constrained settings (10% and 20% training data) to address the challenge of costly data annotation. As shown in Figure 3, our method consistently surpasses baselines M{sup}3`EL and MMOE. Note that the MMOE baseline was reproduced by omitting the DME mention text augmentation to ensure comparable input configurations. Even in the challenging 10% setting, our framework maintains high performance: it achieves 74.1% accuracy on WikiDiverse (vs. 69.7% for MMOE) and **81.5% Hits@1 on RichpediaMEL
Figure 3. Performance comparison of low resource settings on RichpediaMEL and WikiDiverse datasets.
(vs. 80.1% for MMOE and 69.5% for M{sup}3`EL). These significant margins attest to the data efficiency of our approach, highlighting its potential for applications where labeled data is scarce.
Ablation Study
We examine the contribution of specific components through ablation studies on all datasets (Table 3), covering three main aspects:
▹ Impact of Multi-view Experts Network. As shown in Table 3, removing the MVEN module leads to the most significant performance degradation, with a sharp drop of 7.46% in Hits@1 on the WikiDiverse dataset. This empirical evidence strongly supports our “Divide-and-Conquer” strategy, indicating that aggregating diverse views is critical for capturing comprehensive evidence. Specifically, ablating the grain expert, which is designed to capture distinctive granularity-specific semantics, causes a Hits@1 decline of 3.32%.
| Methods | WikiDiverse | RichpediaMEL | WikiMEL | |||||||||
| MRR | Hits@1 | Hits@3 | Hits@5 | MRR | Hits@1 | Hits@3 | Hits@5 | MRR | Hits@1 | Hits@3 | Hits@5 | |
| MECI | 86.71 | 79.93 | 90.23 | 92.93 | 89.94 | 85.63 | 93.43 | 95.65 | 93.48 | 90.38 | 96.05 | 97.21 |
| w/o multi-view expert network | 81.34 | 72.47 | 86.28 | 90.52 | 85.16 | 78.95 | 90.15 | 92.62 | 90.40 | 85.97 | 93.98 | 96.09 |
| w/o shared expert | 85.89 | 78.77 | 90.08 | 92.10 | 89.55 | 85.49 | 92.64 | 94.55 | 93.07 | 89.70 | 95.83 | 96.95 |
| w/o modal expert | 84.95 | 77.24 | 89.12 | 91.72 | 89.01 | 84.42 | 92.76 | 94.58 | 92.26 | 88.58 | 95.25 | 96.88 |
| w/o grain expert | 83.27 | 76.61 | 88.59 | 91.48 | 89.07 | 84.76 | 92.61 | 94.27 | 92.56 | 89.06 | 95.43 | 97.09 |
| w/o hierarchical multimodal interaction | 85.76 | 78.58 | 89.93 | 92.78 | 89.32 | 85.13 | 92.86 | 94.50 | 91.94 | 88.52 | 94.49 | 96.26 |
| repl. granularity unified interaction | 85.18 | 77.57 | 89.57 | 91.73 | 88.41 | 83.47 | 92.44 | 94.30 | 91.75 | 88.23 | 94.40 | 96.15 |
| w/o expert-based matching | 83.28 | 75.17 | 87.73 | 91.00 | 82.13 | 75.16 | 87.34 | 90.79 | 89.00 | 84.13 | 92.78 | 95.09 |
| w/o interaction-based matching | 85.76 | 78.58 | 89.93 | 92.78 | 89.32 | 85.13 | 92.86 | 94.50 | 91.94 | 88.52 | 94.49 | 96.26 |
| w/o overall consensus matching | 83.99 | 75.84 | 88.26 | 91.53 | 89.03 | 84.12 | 93.01 | 95.19 | 92.03 | 88.23 | 95.16 | 96.73 |
| w/o router loss | 83.39 | 75.17 | 87.92 | 91.38 | 89.40 | 85.02 | 92.90 | 94.75 | 92.68 | 89.27 | 95.51 | 96.92 |
| repl. hard router | 83.43 | 76.75 | 88.48 | 91.92 | 88.78 | 84.22 | 92.63 | 94.55 | 93.03 | 89.62 | 95.95 | 97.06 |
| w/o visual modality | 85.02 | 77.33 | 89.60 | 91.92 | 89.05 | 84.96 | 92.22 | 93.87 | 89.72 | 85.11 | 93.32 | 95.53 |
Table 3. Ablation studies on three MEL datasets. “w/o” denotes the removal of a specific component, and “repl.” stands for replacing the component with a variant. The bold scores indicate the results of the complete model.
Similarly, the modal expert, responsible for extracting distinctive modality-specific semantics, contributes a notable gain of 2.69%. Even the shared expert, aimed at modeling common underlying patterns across views to bridge semantic gaps, brings a consistent improvement of 1.16%. These results confirm that each expert plays an indispensable and complementary role in robust entity linking.
Effectiveness of Hierarchical Interaction. Removing the HMIM module (w/o hierarchical multimodal interaction) results in an observable performance drop, e.g., decreasing Hits@1 by 1.86% on WikiMEL. This indicates that the HMIM module is essential for modeling deep cross-modal dependencies. Crucially, replacing our hierarchical design with a flat interaction mechanism (repl. granularity unified interaction)—where fine-grained and coarse-grained features are concatenated for unified DCWN attention processing—yields suboptimal results with a 2.15% deficit on WikiMEL. This validates the effectiveness of our granularity-specific interaction module, avoiding the granularity bias caused by unified processing.
Analysis of Optimization and Visual Modality. Regarding optimization, for instance, the sharp 10.47% Hits@1 drop on RichpediaMEL without (w/o expert-based matching) emphasizes the need for direct supervision on expert-based matching, while declines without and validate our interaction-based matching and overall consensus matching objective. Notably, removing and removing the HMIM module correspond to the same ablation setting. Crucially, omitting (w/o router loss) confirms that KL-guided priors are essential for regularization. These priors enforce a critical balance between specialization and collaboration that unconstrained adaptation lacks. Furthermore, the performance drop with a hard router (repl. hard router) validates our approach: its discrete nature hinders effective expert combination, whereas our differentiable gating smoothly synthesizes diverse ca-features. Finally, the consistent performance drop without visual features (w/o visual modality)—exemplified by a 5.27% decrease on WikiMEL—highlights the necessity of complementary visual evidence.
Parameter Sensitivity Analysis
Effect of Loss Weight . Figure 4 presents the impact of the routing loss weight on model performance. We observe that the performance improves as increases from small values, confirming the effectiveness of our KL-guided routing mechanism in regulating expert behaviors. The model achieves optimal results at for WikiDiverse and for RichpediaEML. However, further increasing leads to performance degradation. This is likely because an excessive weight on over-constrains the routing network, hindering the model’s ability to learn flexible representations for entity matching.

Figure 4. Sensitivity analysis of the routing loss weight .
Effect of Temperature and . We further investigate the joint impact of temperature hyperparameters and on model performance across the WikiDiverse and WikiMEL datasets, as illustrated in Figure 5. From the figure, the overall trend indicates that the model achieves optimal or near-optimal performance when both temperatures are set to moderate values (e.g., 1.0, 0.5). This pattern is particularly pronounced in the WikiDiverse dataset. Deviating from this range results in performance degradation, which corroborates our design rationale. Specifically, extremely low temperatures (e.g., 0.05) induce rigid, deterministic routing that hinders necessary cross-expert knowledge sharing. Conversely, excessively high temperatures (e.g., 5.0) lead to an overly uniform distribution—similar to a standard soft router—which weakens expert specialization and dilutes domain-specific information. Based on these empirical findings, we observe a peak Hit@1 of 79.93% on WikiDiverse when both temperatures are set to 1.0, and 90.38% on WikiMEL with and .
Figure 5. Sensitivity analysis of the temperature and on the WikiDiverse and WikiMEL datasets.
Further Discussion
Analysis of Generative Semantic Refinement. To assess the efficacy and universality of the generative semantic refinement stage, we conducted evaluations using three distinct LLMs: DeepSeek v3 (text-only LLM), GPT-4o (multimodal LLM), and the locally deployable Qwen3-8B. As illustrated in Figure 6, integrating
Figure 6. Results of generative semantic refinement stage with different LLMs.
this refinement strategy yields consistent performance improvements on all baselines, and our proposed framework robustly outperforms two baseline methods (M3EL and MMOE) across all LLM backbones. Among the tested models, DeepSeek v3 demonstrates superior performance, achieving a Hit@1 of 93.55% on WikiDiverse and 94.10% on RichpediaMEL. Remarkably, the smaller-scale Qwen3-8B exhibits competitive performance, particularly on the RichpediaMEL dataset, where it achieves a Hit@1 of 94.08%.
Embedding Visualization. To verify the View-Specific Experts Module, we visualize expert outputs via t-SNE in Figure 7, corroborating our motivations from three perspectives: 1) As shown in Figure 7(a) and Figure 7(b), the fused outputs generated by the Modality and Granularity expert pools form distinct clusters aligned with their respective input categories (Text vs. Image; Coarse vs. Fine). Despite integrating complementary knowledge from peer experts, the aggregated representations preserve the intrinsic discriminability of the heterogeneous signals. 2) Figure 7(c) demonstrates that experts operate with distinct roles. When provided with identical coarse input, the Coarse (Native) and Fine (Peer) Experts still produce separable output clusters. This indicates that each expert has learned to extract unique semantic features tailored to its specific function, rather than simply replicating the input. 3) Figure 7(d) visualizes the outputs from the Modality, Granularity, and Universal experts given identical input. The formation of three non-overlapping clusters confirms that the Universal Expert successfully captures shared semantics, distinct from the view-specific details captured by the other experts. This disentanglement ensures a comprehensive representation that merges a stable common foundation with context-specific nuances.

Figure 7. Different types of embedding t-SNE results on WikiDiverse dataset.
Complexity Analysis. To evaluate the trade-off between effectiveness and efficiency, we compare the model performance (H@1) against the computational cost (GFLOPs) and model size (parameters) under the WikiDiverse dataset. As illustrated in Figure 8, the proposed method is located in the top-left region, which represents the better balance of high accuracy and low complexity. Existing LLM-based fine-tuning methods, such as IIER, suffer from excessive parameter counts and prohibitive computational overhead. Notably, by incorporating a training-free LLM reranking mechanism, our approach significantly outperforms the IIER method. Additionally, our model outperforms the competitive MMOE and M3EL methods in H@1 while maintaining a comparable or lower computational overhead. Furthermore, the moderate parameter size (indicated by the bubble radius) demonstrates that our approach does not rely on simply scaling up parameters to improve results. Overall, our method achieves a superior trade-off, offering a highly efficient solution for deployment.
Figure 8. Comparison of model complexity and performance. The y-axis denotes the H@1 on the WikiDiverse dataset, while the x-axis represents the GFLOPs per sample. Circle size corresponds to the number of parameters.
Conclusion
We proposed the MECI framework to capture multi-level multi-element correspondences for multimodal entity linking. Specifically, to address element dominance in mention-entity correlations, we design a Multi-view Experts Network that employs a “divide-and-conquer” strategy and KL-Guided Routing to decouple feature processing. Meanwhile, to mitigate granularity-level element dominance in cross-modal correlations, we introduce a Hierarchical Multimodal Interaction Module, utilizing dynamic weighting and gating mechanisms to refine interactions. Furthermore, a generative refinement stage leverages LLMs for precise re-ranking. Experiments on WikiMEL, RichpediaMEL, and WikiDiverse confirm MECI’s superiority over state-of-the-art baselines. Notably, the visual information in these datasets is sparse; thus, exploring mechanisms for missing or noisy modalities is an interesting research direction in the future.
References
References
- [1]Ali Ahmadvand, Harshita Sahijwani, Jason Ingyu Choi, and Eugene Agichtein. 2019. ConCET: Entity-Aware Topic Classification for Open-Domain Conversational Agents. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management (Beijing, China) (CIKM ’19). Association for Computing Machinery, New York, NY, USA, 1371–1380. doi:10.1145/3357384.3358048DOI
- [2]Yixin Cao, Lei Hou, Juanzhi Li, and Zhiyuan Liu. 2018. Neural Collective Entity Linking. In Proceedings of the 27th International Conference on Computational Linguistics, Emily M. Bender, Leon Derczynski, and Pierre Isabelle (Eds.). Association for Computational Linguistics, Santa Fe, New Mexico, USA, 675–686. https://aclanthology.org/C18-1057/arxiv.org/abs/1811.08603
- [3]Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Jill Burstein, Christy Doran, and Thamar Solorio (Eds.). Association for Computational Linguistics, Minneapolis, Minnesota, 4171–4186. doi:10.18653/v1/N19-1423DOI
- [4]Zhang Dongjie and Longtao Huang. 2022. Multimodal Knowledge Learning for Named Entity Disambiguation. In Findings of the Association for Computational Linguistics: EMNLP 2022, Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (Eds.). Association for Computational Linguistics, Abu Dhabi, United Arab Emirates, 3160–3169. doi:10.18653/v1/2022.findings-emnlp.230
- [5]Zi-Yi Dou, Yichong Xu, Zhe Gan, Jianfeng Wang, Shuo-hang Wang, Lijuan Wang, Chienguang Zhu, Pengchuan Zhang, Lu Yuan, Nanyun Peng, Zicheng Liu, and Michael Zeng. 2022. An Empirical Study of Training End-to-End Vision-and-Language Transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 18166–18176.arxiv.org/abs/2111.02387
- [6]Zheng Fang, Yanan Cao, Qian Li, Dongjie Zhang, Zhenyu Zhang, and Yanbing Liu. 2019. Joint Entity Linking with Deep Reinforcement Learning. In The World Wide Web Conference (San Francisco, CA, USA) (WWW ’19). Association for Computing Machinery, New York, NY, USA, 438–447. doi:10.1145/3308558.3313517DOI
- [7]Zhiwei Hu, Víctor Gutiérrez-Basulto, Rui Li, and Jeff Z. Pan. 2025. Multi-level Matching Network for Multimodal Entity Linking. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1 (Toronto ON, Canada) (KDD ’25). Association for Computing Machinery, New York, NY, USA, 508–519. doi:10.1145/3690624.3709306
- [8]Zhiwei Hu, Víctor Gutiérrez-Basulto, Zhiliang Xiang, Rui Li, and Jeff Z. Pan. 2025. Multi-level Mixture of Experts for Multimodal Entity Linking. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2 (Toronto ON, Canada) (KDD ’25). Association for Computing Machinery, New York, NY, USA, 979–990. doi:10.1145/3711896.3737060
- [9]Wonjae Kim, Bokyoung Son, and Ildoo Kim. 2021. ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision. In Proceedings of the 38th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 139), Marina Meila and Tong Zhang (Eds.). PMLR, 5583–5594. https://proceedings.mlr.press/v139/kim21m.html
- [10]Phong Le and Ivan Titov. 2018. Improving Entity Linking by Modeling Latent Relations between Mentions. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Iryna Gurevych and Yusuke Miyao (Eds.). Association for Computational Linguistics, Melbourne, Australia, 1595–1604. doi:10.18653/v1/P18-1148
- [11]Junnan Li, Ramprasaath Selvaraju, Akhilesh Gotmare, Shafiq Joty, Caiming Xiong, and Steven Chu Hong Hoi. 2021. Align before Fuse: Vision and Language Representation Learning with Momentum Distillation. In Advances in Neural Information Processing Systems, M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (Eds.), Vol. 34. Curran Associates, Inc., 9694–9705. https://proceedings.neurips.cc/paper_files/paper/2021/file/50525975624493872b7709a8b01b536-Paper.pdfarxiv.org/abs/2107.07651
- [12]Ke Liang, Lingyuan Meng, Meng Shi, Yue Liu, Wenyu Tan, Siwei Wang, Sihang Zhou, Xinwang Liu, Fuchun Sun, and Kunlun He. 2024. A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multi-Modal. IEEE Transactions on Pattern Analysis and Machine Intelligence 46, 12 (2024), 9456–9478. doi:10.1109/TPAMI.2024.3417451
- [13]Lizi Liao, Yunshan Ma, Xiangnan He, Richang Hong, and Tat-Seng Chua. 2018. Knowledge-aware Multimodal Dialogue Systems. In Proceedings of the 26th ACM International Conference on Multimedia (Seoul, Republic of Korea) (MM ’18). Association for Computing Machinery, New York, NY, USA, 801–809. doi:10.1145/3240508.3240605DOI
- [14]Yinghan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv preprint arXiv:1907.11692 (2019).
- [15]Yinghan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv preprint arXiv:1907.11692 (2019).arxiv.org/abs/1907.11692
- [16]Ziyan Liu, Junwen Li, Kaiwen Li, Tong Ruan, Chao Wang, Xinyan He, Zongyu Wang, Xuezhi Cao, and Jingping Liu. 2025. I2CR: Intra- and Inter-modal Collaborative Reflections for Multimodal Entity Linking. In Proceedings of the 33rd ACM International Conference on Multimedia (Dublin, Ireland) (MM ’25). Association for Computing Machinery, New York, NY, USA, 4942–4951. doi:10.1145/3746027.3755674
- [17]Xinwei Long, Jiali Zeng, Fandong Meng, Jie Zhou, and Bowen Zhou. 2024. Trust in Internal or External Knowledge? Generative Multi-Modal Entity Linking with Knowledge Retriever. In Findings of the Association for Computational Linguistics: ACL 2024, Lun-Wei Ku, Andre Martins, and Verik Sakayan (Eds.). Association for Computational Linguistics, Bangkok, Thailand, 7559–7569. doi:10.18653/v1/2024.findings-acl.450DOI
- [18]Ilya Loshchilov and Frank Hutter. 2019. Decoupled Weight Decay Regularization. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net. https://openreview.net/forum?id=Bkg6RiCqY7
- [19]Pengfei Luo, Tong Xu, Che Liu, Suojian Zhang, Linli Xu, Minglei Li, and Enhong Chen. 2024. Bridging Gaps in Content and Knowledge for Multimodal Entity Linking. In Proceedings of the 32nd ACM International Conference on Multimedia (Melbourne VIC, Australia) (MM ’24). Association for Computing Machinery, New York, NY, USA, 9311–9320. doi:10.1145/3664647.3681661
- [20]Pengfei Luo, Tong Xu, Shiwei Wu, Chen Zhu, Linli Xu, and Enhong Chen. 2023. Multi-Grained Multimodal Interaction Network for Entity Linking. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (Long Beach, CA, USA) (KDD ’23). Association for Computing Machinery, New York, NY, USA, 1583–1594. doi:10.1145/3580305.3599439DOI
- [21]Kenneth Marino, Jingwan Lu, Xinyu Zhang, Haonan Cheng, Jiahao Wang, Di Sun, and Gang Pan. 2025. VP-MEL: Visual Prompts Guided Multimodal Entity Linking. In Findings of the Association for Computational Linguistics: ACL 2025, Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (Eds.). Association for Computational Linguistics, Vienna, Austria, 17122–17137. doi:10.18653/v1/2025.findings-acl.880
- [22]Kenneth Marino, Jingwan Lu, Xinyu Zhang, Haonan Cheng, Jiahao Wang, Di Sun, and Gang Pan. 2025. VP-MEL: Visual Prompts Guided Multimodal Entity Linking. In Findings of the Association for Computational Linguistics: ACL 2025, Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (Eds.). Association for Computational Linguistics, Vienna, Austria, 17122–17137. doi:10.18653/v1/2025.findings-acl.880DOI
- [23]Matthew E. Peters, Mark Neumann, Robert Logan, Roy Schwartz, Vidur Joshi, Sameer Singh, and Noah A. Smith. 2019. Knowledge Enhanced Contextual Word Representations. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan (Eds.). Association for Computational Linguistics, Hong Kong, China, 43–54. doi:10.18653/v1/D19-1005DOI
- [24]Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. In Proceedings of the 38th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 139), Marina Meila and Tong Zhang (Eds.). PMLR, 8748–8763. https://proceedings.mlr.press/v139/radford21a.html
- [25]Chenwei Ran, Wei Shen, and Jianyong Wang. 2018. An Attention Factor Graph Model for Tweet Entity Linking. In Proceedings of the 2018 World Wide Web Conference (Lyon, France) (WWW ’18). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 1135–1144. doi:10.1145/3178876.3186012DOI
- [26]Sanket Shah, Anand Mishra, Naganand Yadati, and Partha Pratim Talukdar. 2019. VQKA: knowledge-aware visual question answering. In Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence (AAAI’19/IAAI’19/EAAI’19). AAAI Press, Article 1089, 9 pages. doi:10.1609/aaai.v33i01.33018876DOI
- [27]Shenbao Shi, Zhenran Xu, Baotian Hu, and Min Zhang. 2024. Generative Multimodal Entity Linking. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), Nicoletta Calzolari, Min-Yen Kan, Veronique Hoste, Alessandro Lenci, Sakriani Sakti, and Nanwen Xue (Eds.). ELRA and ICCL, Torino, Italia, 7654–7665. https://aclanthology.org/2024.lrec-main.676/arxiv.org/abs/2306.12725
- [28]Shengzong Sheng, Shasha Li, Jie Yu, Shan Zhao, Xiaopeng Li, Jun Ma, Xiaodong Liu, Zhuo Li, and Xiaoguang Mao. 2025. DIM: Dynamic Integration of Multimodal Entity Linking with a Large Language Model. In Pattern Recognition and Computer Vision, Zhouchhen Lin, Ming-Ming Cheng, Ran He, Kurban Ubul, Wushouer Slamu, Hongbin Zha, Jie Zhou, and Cheng-Lin Liu (Eds.). Springer Nature Singapore, Singapore, Singapore, 187–200.
- [29]Shengzong Song, Shan Zhao, Chengyu Wang, Tianwei Yan, Shasha Li, Xiaoguang Mao, and Meng Wang. 2024. A dual-way enhanced framework from text matching point of view for multimodal entity linking. In Proceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances in Artificial Intelligence (AAAI’24/IAAI’24/EAAI’24). AAAI Press, Article 2119, 9 pages. doi:10.1609/aaai.v38i17.29867
- [30]Xuhui Sui, Ying Zhang, Yu Zhao, Kehui Song, Baohang Zhou, and Xiaojie Yuan. 2024. MELOV: Multimodal Entity Linking with Optimized Visual Features in Latent Space. In Findings of the Association for Computational Linguistics: ACL 2024, Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.). Association for Computational Linguistics, Bangkok, Thailand, 816–826. doi:10.18653/v1/2024.findings-acl.46DOI
- [31]Hongru Wang, Wenyu Huang, Yufei Wang, Yuanhao Xi, Jianqiao Lu, Huan Zhang, Nan Hu, Zeming Liu, Jeff Z. Pan, and Kam-Fai Wong. 2025. Rethinking Stateful Tool Use in Multi-Turn Dialogues: Benchmarks and Challenges. In Findings of the Association for Computational Linguistics: ACL 2025, Wanxiang Che, Joyce Nabende, Akshita Shutova, and Mohammad Taher Pilehvar (Eds.). Association for Computational Linguistics, Vienna, Austria, 5433–5453. doi:10.18653/v1/2025.findings-acl.284DOI
- [32]Meng Wang, Guilin Qi, HaoFen Wang, and Qiushuo Zheng. 2019. Rchipedia: A Comprehensive Multi-modal Knowledge Graph. In Semantic Technology: 9th Joint International Conference, JIST 2019, Hangzhou, China, November 25–27, 2019, Proceedings (Hangzhou, China). Springer-Verlag, Berlin, Heidelberg, 130–145. doi:10.1007/978-3-030-41407-8_9DOI
- [33]Peng Wang, Jiangheng Wu, and Xiaohang Chen. 2022. Multimodal Entity Linking with Gated Hierarchical Fusion and Contrastive Training. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval (Madrid, Spain) (SIGIR ’22). Association for Computing Machinery, New York, NY, USA, 938–948. doi:10.1145/3477495.3513867DOI
- [34]Xuwu Wang, Junfeng Tian, Min Gui, Zhixu Li, Rui Wang, Ming Yan, Lihan Chen, and Yangxiao Xiao. 2022. WikiDiverse: A Multimodal Entity Linking Dataset with Diversified Contextual Topics and Entity Types. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (Eds.). Association for Computational Linguistics, Dublin, Ireland, 4785–4797. doi:10.18653/v1/2022.acl-long.328DOI
- [35]Ledell Wu, Fabio Petroni, Martin Josifoski, Sebastian Riedel, and Luke Zettlemoyer. 2020. Scalable Zero-shot Entity Linking with Dense Entity Retrieval. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (Eds.). Association for Computational Linguistics, Online, 6397–6407. doi:10.18653/v1/2020.emnlp-main.519
- [36]Shangxing Xing, Fei Zhao, Zhen Wu, Chunhui Li, Jianbing Zhang, and Xinyu Dai. 2023. DRIN: Dynamic Relation Interactive Network for Multimodal Entity Linking. In Proceedings of the 31st ACM International Conference on Multimedia (Ottawa ON, Canada) (MM ’23). Association for Computing Machinery, New York, NY, USA, 3599–3608. doi:10.1145/3581783.3612575DOI
- [37]Wenhan Xiong, Mo Yu, Shiyu Chang, Xiaoxiao Guo, and William Yang Wang. 2019. Improving Question Answering over Incomplete KBs with Knowledge-Aware Reader. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Anna Korhonen, David Traum, and Lluís Màrquez (Eds.). Association for Computational Linguistics, Florence, Italy, 4258–4264. doi:10.18653/v1/P19-1417DOI
- [38]Chengmei Yang, Bowei He, Yimeng Wu, Chao Xing, Lianghua He, and Chen Ma. 2023. MMEL: a joint learning framework for multi-mention entity linking. In Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence (Pittsburgh, PA, USA) (UAI ’23). JMLR.org, Article 225, 11 pages.
- [39]Ziyuan Yang, Xiaotao Gu, Sheng Lin, Silian Tang, Yueting Zhuang, Fei Wu, Zhigang Chen, Guoping Hu, and Xiang Ren. 2019. Learning Dynamic Context Augmentation for Global Entity Linking. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan (Eds.). Association for Computational Linguistics, Hong Kong, China, 3271–3281. doi:10.18653/v1/D19-1026
- [40]Xihong Yang, Yiqi Wang, Jin Chen, Wenqi Fan, Xiangyu Zhao, En Zhu, Xinwang Liu, and Defu Lian. 2025. Dual Test-Time Training for Out-of-Distribution Recommendation System. IEEE Transactions on Knowledge and Data Engineering 37, 6 (2025), 3312–3326. doi:10.1109/TKDE.2025.3548160DOI
- [41]Zefeng Zhang, Jiawei Sheng, Chuang Zhang, Yunzhi Liang, Wenyuan Zhang, Siqi Wang, and Tingwen Liu. 2024. Optimal Transport Guided Correlation Assignment for Multimodal Entity Linking. In Findings of the Association for Computational Linguistics: ACL 2024, Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.). Association for Computational Linguistics, Bangkok, Thailand, 4103–4117. doi:10.18653/v1/2024.findings-acl.243DOI
- [42]Liu Zhiyuan, Lin Yawen, Shi Ya, and Zhao Jun. 2021. A Robustly Optimized BERT Pre-training Approach with Post-training. In Proceedings of the 20th Chinese National Conference on Computational Linguistics, Sheng Li, Maosong Sun, Yang Liu, Hua Wu, Kang Liu, Wanxiang Che, Shizhu He, and Gaoqi Rao (Eds.). Chinese Information Processing Society of China, Huhhot, China, 1218–1227. https://aclanthology.org/2021.ccl-1.108/DOI




