betaFeedback

WorksComputer Science

AcrossPlace: Do Vision–Language Models Recognize Places Beyond Their Source Views? A Comprehensive Benchmark

Abstract

Language-guided place recognition requires recovering geographic identity across visual observations beyond the image from which a description originates. However, aggregate retrieval scores often entangle observation transfer, gallery competition, and linguistic evidence, obscuring the underlying recognition capability. We introduce AcrossPlace, a provenance-aware benchmark establishing a systematic evaluation standard for bidirectional language–vision place recognition. Built upon GSV-Cities, AcrossPlace comprises 56,726 places across 23 cities, 156,800 automatically generated, rule-filtered descriptions, and 113,404 target images. Each description retains its source-image provenance, while retrieval operates on disjoint target observations. Text-to-image (T2I) and image-to-text (I2T) retrieval share a unified place-identity relevance criterion supporting multiple valid matches. Two geographic protocols evaluate unseen places within represented cities (P0) and generalization to held-out cities (P1). Paired observation comparisons, gallery-controlled evaluations, fixed-query scaling, and blinded annotation auditing systematically characterize retrieval performance. Across three training seeds, place-supervised SigLIP2 T2I Recall@1 decreases from 19.69% on source images to 12.31% on independent targets under P0. For fixed P1 queries, expanding the gallery from 500 to 8,425 places reduces Recall@1 from 14.67% to 2.87%. A 240-description audit further reveals discrepancies between automated acceptance and factual evidence. These findings establish AcrossPlace as a reproducible evaluation standard for assessing transferable place identity across visual observations and geographic environments.

observations and geographic environments.

Introduction

Geographic places retain their identities across visual observations, even when their appearances change [6, 21]. However, a language description derived from one image may accurately characterize that observation without identifying another view of the same place [9, 15, 16]. This distinction exposes a fundamental challenge in language–vision place recognition: matching a description to its source image does not necessarily imply recognition of the underlying place (Figure 1). Reliable place recognition therefore requires representations that transfer across observations while remaining discriminative among geographically distinct but visually similar locations.

Figure 1. Does recognizing visual evidence suffice to recognize a place? Verbatim captions and unmodified GSV-Cities images illustrate a successful (left) and failed (right) T2I query. Top-1 predictions are archived coarse results from place-supervised SigLIP2. Source support and target-set visibility are single-reviewer human labels; both are positive in these cases. Examples are illustrative, not frequency estimates.

Recent advances in multi-view text–vision matching [17], bidirectional multimodal representations [23], and large-scale language supervision [4] have substantially expanded the scope of language-based place retrieval [14]. These developments motivate a more fundamental question: Do language–vision models capture transferable place identity beyond the observations from which descriptions originate? Addressing this question requires disentangling three factors that jointly determine retrieval performance: observation transfer, which reflects the consistency of place recognition across independent views; gallery competition, which governs discrimination among candidate locations; and language-evidence reliability, which determines whether descriptions provide visually grounded and geographically distinctive information. A rigorous benchmark must characterize these factors to establish what retrieval performance actually measures.

To this end, we introduce AcrossPlace, a comprehensive benchmark for provenance-aware, bidirectional language–vision place recognition. Built upon GSV-Cities [1], AcrossPlace encompasses 56,726 places across 23 cities, 156,800 automatically generated, rule-filtered descriptions, and 113,404 target images. Its central design principle is source–target independence: each description preserves its generating-image provenance, while retrieval operates on disjoint target observations. AcrossPlace supports both text-to-image (T2I) and image-to-text (I2T) retrieval under a unified place-identity relevance criterion, allowing multiple valid matches without requiring exact image–caption correspondence. This formulation shifts evaluation from image-specific semantic agreement toward recognition of geographic identity across distinct observations.

AcrossPlace establishes two geographic evaluation protocols: P0, which assesses unseen places within represented cities, and P1, which evaluates generalization to held-out cities. Beyond these principal protocols, three controlled diagnostics systematically characterize retrieval behavior.

First, paired source–target comparisons measure observation transfer under fixed descriptions, place identities, and gallery cardinality. Second, gallery-matched comparisons and fixed-query scaling examine the influence of geographic composition and candidate competition. Third, a blinded annotation audit independently assesses source-image support, target-view visibility, and place specificity. Together, these protocols establish a structured evaluation framework connecting recognition performance with its underlying visual, geographic, and linguistic conditions.

Our experiments reveal three key findings. First, replacing caption-source images with independent target observations reduces supervised SigLIP2 T2I Recall@1 from 19.69% to 12.31% on P0, revealing a substantial observation-transfer gap. Second, enlarging the P1 gallery from 500 to 8,425 places reduces Recall@1 from 14.67% to 2.87% under fixed queries and model parameters, demonstrating the influence of candidate competition on geographic evaluation. Third, a blinded audit of 240 descriptions identifies 166 with full source support and 148 with sufficient target evidence, revealing discrepancies between automated screening and independently assessed linguistic evidence. Collectively, these findings demonstrate that observation consistency, gallery composition, and description reliability are essential dimensions for interpreting language–vision place recognition performance.

Our contributions are threefold:

  • We establish AcrossPlace, a large-scale language–vision resource that explicitly separates description sources from retrieval targets and defines unified place-identity relevance for bidirectional cross-modal retrieval.

  • We introduce fixed within-city and held-out-city protocols, complemented by paired observation comparisons, gallery-controlled evaluations, and fixed-query scaling to characterize place recognition across visual observations and geographic environments.

  • Through pretrained and place-supervised baselines, controlled retrieval experiments, and independent language-evidence auditing, we identify three critical factors governing benchmark outcomes: observation transfer, candidate competition, and description reliability.

Related Work

Visual Place Recognition

Visual place recognition (VPR) [7, 8] identifies geographic places across observations affected by viewpoint and appearance changes and perceptual aliasing. GSV-Cities provides large-scale place-level supervision for learning discriminative visual representations [1]. Recent approaches improve recognition through cross-image correlations, multi-observation aggregation, and semantic supervision. CricaVPR models inter-image correlations to capture consistent place features [10], while HOPS aggregates reference observations into compact place signatures [11]. SemVPR transfers semantic knowledge from vision–language models to visual place representations [29]. These advances motivate evaluating whether place identity remains recoverable when a linguistic description and its retrieval target arise from distinct observations.

Language-Guided Place Recognition

Natural language supports place retrieval through descriptions of landmarks, scene attributes, and spatial relations. Text4VPR associates multi-directional street views with language descriptions [17], while UniLoc investigates bidirectional retrieval across images, text, and point clouds [23]. LaVPR provides large-scale language annotations and evaluates descriptions against reference images distinct from their sources [4]. DialogueVPR extends place retrieval to interactive, multi-turn queries [18]. Complementary localization tasks include language-to-point-cloud localization in Text2Loc [24], partial text–submap correspondence [3], and text-based satellite/map retrieval in CVG-Text [27].

These works establish diverse language-grounded geographic retrieval settings. AcrossPlace focuses on bidirectional retrieval between descriptions and street-level images under unified place-identity relevance. It combines explicit description provenance with paired source–target evaluations, geographic partitions, and controlled gallery scaling to distinguish observation transfer from candidate competition. Table 1 compares the documented evaluation protocols of closely related resources.

BenchmarkScale (unit)T2II2TProv.Paired Obs.Geo. Ctrl.Text AuditEval
Street360Loc [17]7k locations✓×××✓✓Direct
UniLoc [23]29,438 triplets✓✓××✓×Direct
LaVPR [4]651,865 pairs✓×✓×✓✓Direct
DlgQuest-Cities [18]106,880 images✓×✓×–✓Direct
AcrossPlace56,726 places✓✓✓✓✓✓Direct/Diagnostic

Table 1. Comparison of evaluation protocols across language–vision place recognition benchmarks. A dash denotes information not established by the cited source, rather than the absence of a capability. AcrossPlace integrates provenance-aware bidirectional retrieval, paired source–target observation controls, geographic controls, and text auditing.

Fine-Grained Vision–Language Understanding

Language-guided recognition also depends on fine-grained visual grounding and geographic distinctiveness. FLAIR learns localized representations from detailed descriptions for fine-grained retrieval [25]. Compositional grounding investigates object attributes and relations [28], while SpatialVLM and SpatiaLQA address spatial understanding and reasoning [2, 26]. For place recognition, a description may be correct for its source image yet contain evidence absent from a target observation or shared by other locations. AcrossPlace evaluates these distinct properties through a three-axis audit of source correctness, target visibility, and place specificity, linking language evidence to retrieval outcomes.

Retrieval Reference Models

Pretrained vision–language models provide general-purpose image–text representations. CLIP [12] and SigLIP2 [20] learn aligned cross-modal embeddings, while BLIP combines contrastive features with image–text matching [5, 13, 19]. AcrossPlace evaluates these references alongside place-supervised adaptations, parameter-efficient fine-tuning, and validation-selected reranking under fixed retrieval protocols. The resulting comparisons characterize model behavior across observation transfer, gallery competition, and language-evidence conditions.

AcrossPlace Benchmark

AcrossPlace evaluates transferable place identity by linking descriptions to their source observations while separating those sources from retrieval targets under a unified place-level relevance criterion. Fixed geographic protocols and controlled diagnostics assess observation transfer, gallery competition, and language-evidence reliability (Figure 2); Figure 3 summarizes the benchmark scope.

Figure 2. AcrossPlace overview using original images and verbatim annotations. (a) Two places with source/target dates and traceable text. (b) Bidirectional place relevance, illustrated with unranked candidates. (c) Archived paired-P0 and fixed-query-P1 diagnostics, distinct from principal full-gallery scores; supervised results average three seeds. Human counts describe a stratified, single-reviewer audit.

Figure 3. Scope of AcrossPlace. Eight themes and 32 facets connect resources/protocols (warm/green) with models, adaptation, metrics, and diagnostics (cool/purple). Equal-angle leaves denote categories, not sample frequencies.

Resource Construction and Observation Provenance

Source and target observations. We begin with 62,279 GSV-Cities places, each containing four images (249,116 images). Qwen3-VL-4B-Instruct generates descriptions from designated source observations; distinct images of the same place form the target pool. Each description retains its source-image link. Image/text identifiers, canonical paths, capture timestamps, GPS metadata, panorama identifiers, and SHA256 hashes are preserved in manifests to verify source–target separation.

Eligible resource. Rule-based screening retains 56,726 places across 23 cities, 156,800 accepted descriptions, and 113,404 target images. Accepted descriptions form principal text queries and galleries; rule-uncertain descriptions support quality diagnostics. Screening assesses rule compliance, while factual support is examined through human auditing. All benchmark totals refer to this eligible resource.

Independent visual observations. Median source–target heading differences are 10∘10^\circ on P0 and 9∘9^\circ on P1, with capture-time gaps of 46 and 51 months, respectively. Controlled source–target pairs share neither identical image bytes nor panorama IDs. Thus, cross-view denotes independent observations of a place, potentially with similar viewing directions, rather than necessarily large angular changes.

Geographic Evaluation Protocols

AcrossPlace defines two fixed, place-disjoint geographic protocols with predetermined query/gallery manifests (Table 2).

ProtocolSplitPlacesTextsTargets
P0Train54,009149,126107,970
P0Validation1,3553,8762,710
P0Test1,3623,7982,724
P1Train39,706109,80979,390
P1Validation8,51123,62217,000
P1Test8,50923,36917,014

Table 2. Fixed geographic partitions. P0 and P1 use the same 56,726 places, 156,800 valid descriptions, and 113,404 target images, but assign places to different splits. Source observations are kept separate.

P0: Unseen places within represented cities. P0 tests unseen places in cities represented during training, with 54,009 training, 1,355 validation, and 1,362 test places.

P1: Generalization to held-out cities. P1 separates cities across splits, assigning four to validation and four distinct cities to testing. Its training, validation, and test sets contain 39,706, 8,511, and 8,509 places. City assignments follow geographic metadata and a fixed seed.

P0 and P1 characterize within-city and held-out-city evaluation, respectively. Because training cohorts, queries, and gallery galleries also differ, their raw score gap reflects combined protocol differences. Cardinality-matched and fixed-query controls further examine gallery effects.

Bidirectional Retrieval and Place-Level Relevance

In text-to-image (T2I) retrieval, descriptions query independent target images; in image-to-text (I2T) retrieval, target images query stored descriptions. Let p(x)p(x) denote the place identity of image or description xx. Candidate gg is relevant to query qq when p(g)=p(q)p(g)=p(q), regardless of their original image–caption pairing. Multiple gallery samples may therefore represent the same relevant place.

We rank individual samples, without place deduplication,

and report Recall@KK for K∈{1,5,10}K \in\{1, 5, 10\}:

R⁡@K=1∣Q∣∑q∈Q1[∃g∈TopK⁡(q):pg=pq].(1)\operatorname{R}@K = \frac{1}{|Q|} \sum_{q \in Q} \mathbf{1}[\exists g \in\operatorname{TopK}(q) : p_g = p_q]. \tag*{(1)}

Repeated same-place items retain separate ranks. Principal galleries use the corresponding held-out split; target-image galleries exclude caption-source images. GPS and source observations support construction and auditing, not model inference; I2T retrieves rather than generates descriptions. The separate language-evidence audit (§ 5.3) examines visual support and place specificity without altering place-ID relevance labels.

Controlled Diagnostic Evaluations

Three diagnostics complement principal Recall@KK by characterizing observation transfer (D1), gallery competition (D2), and language evidence (D3).

D1: Observation transfer. We select one accepted, single-source description and one source–target pair per eligible place. For T2I, fixed descriptions query equal-sized source or independent-target galleries; for I2T, the corresponding source or target image queries a fixed caption gallery. Place cohorts and candidate cardinality remain unchanged. The diagnostic includes 1,349 P0 and 8,425 P1 places; 13 and 84 test places, respectively, fail the single-source criterion. Paired differences capture the combined effect of replacing observations, including temporal, directional, and visibility changes.

D2: Geographic composition and gallery competition. We sample five 1,000-place galleries per protocol, each with one image and one description per place, to match candidate counts across P0/P1. Separately, 500 fixed query places are evaluated against nested galleries of 500, 1,000, 2,000, 4,000, and the full eligible test pool where available. Query identities, positive-place membership, and model weights remain fixed; galleries and relevance labels are predetermined. Gallery-draw and training-seed variability are reported separately.

D3: Language-evidence reliability. On P1, 7,735 test places have both rule-accepted and rule-uncertain descriptions. A matched 4,291-place subset controls description type, source observation, and word-count bin while fixing query places, candidate counts, and background captions as focal description status varies. P0 has no corresponding uncertain-caption test pool. A blinded, single-reviewer audit examines 240 descriptions (120 per stratum) along three axes: source correctness, target visibility, and place specificity. Together, matched retrieval and human audit distinguish screening status from visual support and geographic discriminability.

Evaluation Setup

Reference Systems and Adaptation

We evaluate four pretrained references: CLIP ViT-B/16, CLIP ViT-B/32, BLIP contrastive features, and SigLIP2. Four place-supervised configurations include CLIP with place loss, LaVPR-inspired BLIP with Multi-Similarity (MS) loss, and SigLIP2 with MS or bidirectional place loss. Four additional SigLIP2 variants examine residual heads and modality-specific adaptation. Each supervised configuration uses three seeds (42, 43, 44) and a common three-epoch budget.

The standard setup samples 16 places per batch, with up to two target images and four descriptions per place. It employs AdamW and rank-eight LoRA on query/value projections in the final four blocks of both encoders, with identity-initialized residual heads. BLIP+MS instead uses all-linear rank-64 rank-stabilized LoRA, an image-to-text objective, and scaled SGD. These settings provide reproducible comparisons under a fixed adaptation budget.

Checkpoint Selection and Statistical Reporting

Checkpoints maximize the weaker-direction validation Recall@100, with weaker-direction Recall@1 breaking ties. A frozen BLIP image–text matching (ITM) model reranks coarse Top-100 candidates using validation-selected score fusion. Checkpoints, fusion weights, and evaluation settings are fixed before testing; coarse and reranked results are reported separately. Supervised results aggregate three training seeds, while pretrained references use fixed checkpoints. Repeated gallery draws quantify candidate-sampling variability, and paired place-level bootstrap intervals assess source–target retrieval differences. A stratified, single-reviewer audit of 240 descriptions assesses source correctness, target visibility, and place specificity. These analyses support reproducible evaluation of observation transfer, gallery competition, and language-evidence reliability.

Results

We assess transferable place identity through P0/P1 reference retrieval and three diagnostics: observation transfer (D1), gallery competition (D2), and language-evidence reliability (D3). Training-seed variation, gallery draws, and paired-place confidence intervals quantify distinct sources of uncertainty.

Reference performance. Tables Table 3 and Table 4 compare pretrained and place-supervised references. On P0, SigLIP2 T2I/I2T R@1 improves from 4.16%/5.29% to 12.95%/13.64% with place supervision, reaching 15.24%/17.35% with ITM reranking. BLIP+MS achieves the highest P0 coarse R@1 (17.54%/23.62%), compared with 16.87%/21.28% after reranking. On P1, its coarse and reranked scores are 6.09%/8.36% and 6.23%/8.18%, respectively. Place supervision improves the tested references, while reranking effects vary by representation and direction.

Table 3. P0 test Recall@K (%). Supervised R@1: mean ±\pm SD across three training seeds; R@5/10: seed means. Pretrained controls use one frozen checkpoint. ITM denotes frozen BLIP [5] matching of coarse Top-100 results. MS models are common-budget adaptations, not reproductions of native training schedules.

ReferenceScoringT2I R@1T2I R@5T2I R@10I2T R@1I2T R@5I2T R@10
CLIP B/32 (zero-shot) [12]Coarse1.765.879.352.688.4812.74
CLIP B/16 (zero-shot) [12]Coarse2.135.618.773.639.5813.62
BLIP (zero-shot) [5]Coarse4.0810.0113.724.4111.0116.04
SigLIP2 (zero-shot) [20]Coarse4.169.0612.595.2911.3415.93
SigLIP2 (zero-shot)+ ITM7.6116.0620.3010.0219.4224.01
CLIP B/16 + place loss [12]Coarse10.59 ±\pm 0.1223.9833.3911.17 ±\pm 0.1126.3536.81
CLIP B/16 + place loss+ ITM13.54 ±\pm 0.2927.0836.4015.38 ±\pm 0.0431.6440.79
BLIP + MS adaptation [4, 22]Coarse17.54 ±\pm 0.3035.1344.8923.62 ±\pm 0.2742.7352.80
BLIP + MS adaptation+ ITM16.87 ±\pm 0.2831.6041.1121.28 ±\pm 0.2040.4449.47
SigLIP2 + MS [20, 22]Coarse14.25 ±\pm 0.1129.3138.0517.35 ±\pm 0.0936.2345.69
SigLIP2 + MS+ ITM14.56 ±\pm 0.0329.4737.3419.20 ±\pm 0.1036.2845.33
SigLIP2 + place loss [20]Coarse12.95 ±\pm 0.2227.6836.6913.64 ±\pm 0.2730.8741.45
SigLIP2 + place loss+ ITM15.24 ±\pm 0.3029.7338.1217.35 ±\pm 0.1534.8844.77

Table 3. P0 test Recall@K (%). Supervised R@1: mean ±\pm SD across three training seeds; R@5/10: seed means. Pretrained controls use one frozen checkpoint. ITM denotes frozen BLIP matching of coarse Top-100 results. MS models are common-budget adaptations, not reproductions of native training schedules.

Table 4. P1 full-gallery test Recall@K (%). Supervised R@1: mean ±\pm SD over three seeds; R@5/10: means. Reranking weights are validation-selected. P1 differs from P0 in training and query cohorts as well as gallery composition and size.

ReferenceScoringT2I R@1T2I R@5T2I R@10I2T R@1I2T R@5I2T R@10
CLIP B/32Coarse0.571.702.660.812.423.79
CLIP B/16Coarse0.421.512.460.922.664.08
BLIPCoarse1.143.304.971.103.274.91
SigLIP2Coarse1.203.545.231.423.725.30
SigLIP2+ ITM2.756.358.602.666.498.80
CLIP + placeCoarse2.70 ±\pm 0.087.8011.602.82 ±\pm 0.058.0211.90
CLIP + place+ ITM4.31 ±\pm 0.0510.3414.234.70 ±\pm 0.0311.5415.82
BLIP + MSCoarse6.09 ±\pm 0.1214.3019.668.36 ±\pm 0.1018.2324.38
BLIP + MS+ ITM6.23 ±\pm 0.0213.8719.008.18 ±\pm 0.0917.6323.68
SigLIP2 + MSCoarse4.11 ±\pm 0.0410.3114.845.18 ±\pm 0.0412.9118.10
SigLIP2 + MS+ ITM5.25 ±\pm 0.0411.9216.476.40 ±\pm 0.0414.7920.07
SigLIP2 + placeCoarse3.68 ±\pm 0.059.5213.743.46 ±\pm 0.049.9714.59
SigLIP2 + place+ ITM5.21 ±\pm 0.0512.0316.475.72 ±\pm 0.0813.3818.25

Table 4. P1 full-gallery test Recall@K (%). Supervised R@1: mean ±\pm SD over three seeds; R@5/10: means. Reranking weights are validation-selected. P1 differs from P0 in training and query cohorts as well as gallery composition and size.

Adaptation behavior. Within the common three-epoch budget, BLIP+MS validation T2I/I2T R@1 on P0 increases from 13.97%/20.26% to 16.94%/23.75%. These results characterize reference performance at a fixed adaptation stage, with comparisons based on retrieval metrics rather than objective-dependent loss values.

Finding I: Recognition degrades across independent observations

D1 tests observation transfer by fixing place cohorts, descriptions, and gallery cardinality while replacing source images with independent target observations for T2I, or switching the image query for I2T (Figure 4, Table 5). Each place contributes one image and one description.

Observation-transfer diagnostic

Figure 4. Observation-transfer diagnostic. Identical captions and place cohorts are retrieved against equal-size source or independent-target galleries. Bars average three supervised SigLIP2 seeds; this one-sample-per-place setting differs from principal retrieval.

Table 5. Source-to-target observation transfer (%; Δ\Delta in percentage points). Same captions and equal-size galleries for 1,349 P0 and 8,425 P1 places. 95% CIs use 2,000 paired place bootstraps after averaging seed outcomes; intervals exclude training-seed uncertainty.

ProtocolReferenceT2I R@1 SourceT2I R@1 TargetT2I R@1 Δ\Delta [95% CI]I2T R@1 SourceI2T R@1 TargetI2T R@1 Δ\Delta [95% CI]
P0BLIP (pretrained)7.783.48-4.30 [-5.78, -2.82]6.893.63-3.26 [-4.52, -2.00]
P0SigLIP2 (pretrained)5.714.89-0.82 [-2.15, +0.52]6.084.60-1.48 [-2.82, -0.22]
P0SigLIP2 (place-supervised)19.6912.31-7.39 [-9.24, -5.49]18.7310.30-8.43 [-10.16, -6.67]
P1BLIP (pretrained)2.851.12-1.73 [-2.12, -1.35]2.491.04-1.45 [-1.82, -1.09]
P1SigLIP2 (pretrained)2.991.27-1.72 [-2.10, -1.36]2.561.13-1.44 [-1.76, -1.10]
P1SigLIP2 (place-supervised)7.493.11-4.37 [-4.92, -3.84]6.352.85-3.50 [-4.01, -3.01]

Table 5. Source-to-target observation transfer (%; Δ\Delta in percentage points). Same captions and equal-size galleries for 1,349 P0 and 8,425 P1 places. 95% CIs use 2,000 paired place bootstraps after averaging seed outcomes; intervals exclude training-seed uncertainty.

For place-supervised SigLIP2 on P0, T2I R@1 falls from 19.69% to 12.31% and I2T from 18.73% to 10.30%, with paired-bootstrap 95% intervals of [−9.24, −5.49] and [−10.16, −6.67] percentage points for target-minus-source differences. On P1, T2I declines from 7.49% to 3.11% and I2T from 6.35% to 2.85%, with intervals of [−4.92, −3.84] and [−4.01, −3.01]. Pretrained BLIP shows comparable declines, whereas the P0 pretrained SigLIP2 T2I interval

[−2.15, +0.52] includes zero. These results identify an observation-transfer gap: agreement with caption-source images does not reliably predict recognition from independent evidence. The paired comparison measures the combined effects of observation changes, and its intervals reflect place resampling rather than training-seed variation.

Finding II: Gallery competition shapes geographic comparisons

D2 distinguishes candidate competition from broader P0/P1 protocol differences through cardinality-matched galleries and fixed-query gallery scaling. At 1,000 places (Table 6), supervised SigLIP2 achieves T2I/I2T R@1 of 13.58%/11.69% on P0 versus 10.61%/9.37% on P1. Pretrained BLIP instead scores 3.88% on P0 versus 4.12% on P1 for T2I, showing that the direction of the P0–P1 difference varies by reference model. Matched cardinality controls gallery size but retains differences in training cohorts, queries, and candidate semantics.

ReferenceP0P1
T2II2TT2II2T
Pretrained BLIP3.884.004.123.40
Pretrained SigLIP25.365.083.643.14
Supervised SigLIP213.5811.6910.619.37

Table 6. Matched 1,000-place gallery Recall@1 (%), averaged over five gallery draws and, when applicable, three training seeds. Other P0/P1 differences remain.

With 500 P1 query places fixed (Figure 5), expanding the gallery from 500 to 8,425 places reduces supervised SigLIP2 T2I R@1 from 14.67% to 2.87% and I2T from 14.73% to 3.13%. Intermediate T2I scores are 10.03%, 6.87%, and 4.85% at 1,000, 2,000, and 4,000 places. Because queries, positive identities, and model weights remain unchanged, the decline directly characterizes gallery-size sensitivity. Candidate competition is therefore essential to interpreting geographic retrieval comparisons.

Figure 5. Fixed-query gallery-scale diagnostics across protocols, directions, and ranks. Each panel uses 500 query places and five nested galleries; supervised bands show seed SD. Full-pool repeats are identical; P0 and P1 end at 1,349 and 8,425 places.

Finding III: Automated screening and language evidence capture distinct properties

D3 distinguishes place-ID retrieval from the visual support and specificity of language descriptions. A blinded audit covers 240 descriptions from 237 P1 places, equally divided between rule-valid and rule-uncertain strata (Table 7). Of these, 166 are fully source-supported, 148 have sufficient target evidence, and 133 are place-distinctive. Full source support occurs in 81/120 valid and 85/120 uncertain descriptions, indicating that rule status does not consistently rank factual support in this sample.

AxisJudgmentAllValidUncertain
Source correctnessSupported1668185
Partial826
Contradicted23149
Unknown432320
Target visibilityVisible1487573
Partial927
Not visible371720
Unknown462620
SpecificityDistinctive1336370
Generic422319
Unknown653431

Table 7. Blinded audit of 240 captions (120 per automatic stratum) by one reviewer. Unknown is distinct from contradiction or nonvisibility; counts are sample-specific.

In a matched 4,291-place P1 cohort, valid descriptions outperform uncertain ones in supervised SigLIP2 R@1: 3.16% versus 2.19% for T2I and 3.25% versus 1.78% for controlled I2T. Audited T2I R@10 is 17.79% for target-visible versus 0.00% for not-visible descriptions, and 17.07% for source-supported versus 4.35% for contradicted descriptions.

These descriptive subgroup comparisons reflect overlapping samples and a single reviewer; partial and unknown judgments remain distinct from negative evidence.

Audited T2I searches 17,014 target images, whereas controlled I2T replaces one caption per focal place in an 8,509-caption gallery. These units differ from principal I2T retrieval. Errors involving similar facades, vegetation, and entrances illustrate semantic aliasing across distinct places. Overall, D3 separates automated screening, factual support, target visibility, and place specificity as complementary properties of language-guided recognition.

Conclusion

AcrossPlace investigates whether language–vision models recognize geographic places beyond the visual observations used to generate their descriptions. By integrating provenance-aware bidirectional retrieval with controlled diagnostics, it systematically examines three factors governing place-recognition performance: observation transfer, gallery competition, and language-evidence reliability. Paired evaluations reveal substantial retrieval degradation when caption-source images are replaced by independent observations, highlighting the distinction between image–text alignment and transferable place identity. Fixed-query gallery scaling demonstrates the influence of candidate competition, while geographic comparisons underscore the importance of controlling gallery composition. Human auditing further distinguishes automated caption acceptance from factual support, target visibility, and place specificity. Together, these findings establish AcrossPlace as a reproducible framework for interpreting cross-modal place-retrieval performance under well-defined evaluation conditions. Future work will extend geographic coverage, diversify observation conditions, and strengthen independent annotation and external validation. Ultimately, AcrossPlace provides a systematic foundation for advancing language-guided place recognition beyond individual visual observations.

Evaluation Units, Evidence Scope, and Reproducibility

Purpose and evidence scope. These analyses decompose the recorded benchmark results without changing a split, checkpoint, fusion weight, or relevance label. Principal scores, validation histories, paired diagnostic outcomes, gallery-draw summaries, and human-review records are read from archived artifacts. The newly produced city, capture-metadata, caption-length, and complementarity breakdowns are post-hoc descriptive analyses. They are not additional independently held-out experiments, and no new training or GPU inference is performed. Three epochs equalize the nominal epoch budget, not wall-clock duration, parameter count, or optimization objective.

Different estimands require different denominators. Principal galleries rank individual target images or descriptions, allowing several same-place positives. Paired diagnostics instead select one caption and one observation per place. Gallery-scaling experiments further retain only 500 query places while adding negatives. Human T2I queries the full P1 image gallery, whereas human I2T uses a focal-caption replacement against one caption per place. These settings answer related questions, but their percentages cannot be substituted for one another.

AnalysisUnit and scope
Principal P01,362 test places; 2,724 target images; 3,798 descriptions.
Principal P18,509 test places; 17,014 target images; 23,369 descriptions.
Paired transfer1,349 P0 / 8,425 P1 places; one caption and one source/target observation per place.
Matched geographyFive 1,000-place gallery draws within each protocol and checkpoint.
Fixed-query scaling500 query places; nested candidate pools up to 1,349 P0 / 8,425 P1 places.
Rule-status matching7,735 raw paired places; 4,291 after type/source/length matching.
Human retrieval240 descriptions at 237 places; 120 valid and 120 uncertain; one reviewer.
Qualitative outcomesThree cached model configurations, supervised seed 42; first target view for I2T.

Table A1. Evaluation-unit ledger. Counts specify what is actually ranked or averaged.

Uncertainty and reproducibility. Training-seed SD describes three independently fitted checkpoints. Gallery-draw SD describes candidate sampling conditional on a checkpoint. Paired place bootstraps hold model weights fixed and preserve source/target pairing. None of these becomes fifteen independent training runs by multiplying three seeds and five draws. The released analysis inputs are SHA256-indexed, and machine-readable tables accompany the figures. Human labels were not used to choose model weights or checkpoints. Because the audit is balanced by rule status and includes repeated places, its unweighted percentages describe the reviewed sample rather than dataset-wide error prevalence.

Supervision Gains Across Ranks and Protocols

A backbone-matched comparison. We subtract each frozen backbone’s coarse score from its corresponding supervised recipe. CLIP uses the B/16 control; both SigLIP2 recipes use the same pretrained SigLIP2 control. These differences describe the combined effect of the recorded adaptation recipe and training exposure, not an isolated causal effect of the loss function. Encoder updates, heads, optimizer choices, and objectives differ across recipes.

Figure 6. Principal full-gallery coarse recall at measured ranks 1, 5, and 10. Lines are visual guides; no intermediate ranks are inferred.

ProtocolAdaptationT2I gain R@1T2I gain R@10I2T gain R@1I2T gain R@10
P0CLIP + place+8.46+24.63+7.54+23.19
P0BLIP + MS+13.46+31.17+19.21+36.76
P0SigLIP2 + MS+10.09+25.46+12.07+29.76
P0SigLIP2 + place+8.79+24.10+8.36+25.51
P1CLIP + place+2.29+9.14+1.90+7.81
P1BLIP + MS+4.95+14.69+7.26+19.47
P1SigLIP2 + MS+2.90+9.61+3.75+12.81
P1SigLIP2 + place+2.48+8.51+2.04+9.29

Table A2. Absolute gain over the matching pretrained backbone, in percentage points. Three-seed supervised means are compared with one frozen pretrained reference.

Interpretation. All four recipes improve the tested coarse references, but the gain varies with direction and retrieval cutoff. BLIP+MS yields a particularly strong coarse operating point in both protocols; this is evidence about the tested common-budget recipe, not a native-schedule reproduction. An increase at R@10R@10 reflects broader candidate coverage and need not imply the same increase at R@1R@1. The full-gallery P0–P1 comparison changes training cohorts, query identities, and gallery composition simultaneously; gains within a protocol are more directly interpretable than comparing raw percentages across protocols.

Reranking Responses and Directional Asymmetry

Frozen systems can respond differently to the same reranker. The reranker is the already validation-selected BLIP ITM fusion stage. We compute selected-system minus coarse recall without selecting a different branch on test. Positive and negative cells therefore characterize fixed outputs; they are not a new tuning search.

Figure 7. Left/middle: reranking changes in percentage points (T/I denote direction, 1/10 rank). Right: coarse T2I versus I2T R@1; the diagonal denotes equal directional recall.

ProtocolReferenceT2I R@1T2I R@10I2T R@1I2T R@10
P0SigLIP2 pre.+3.45+7.71+4.74+8.08
P0CLIP + place+2.95+3.00+4.21+3.98
P0BLIP + MS-0.68-3.78-2.34-3.33
P0SigLIP2 + MS+0.31-0.70+1.85-0.37
P0SigLIP2 + place+2.29+1.43+3.71+3.33
P1SigLIP2 pre.+1.54+3.37+1.23+3.50
P1CLIP + place+1.61+2.63+1.88+3.92
P1BLIP + MS+0.14-0.66-0.19-0.69
P1SigLIP2 + MS+1.14+1.63+1.22+1.96
P1SigLIP2 + place+1.53+2.73+2.26+3.67

Table A3. Reranking changes, in percentage points. A single validation-selected system can gain at one direction or cutoff and decline at another.

Representation-dependent effects. On P0, BLIP+MS changes by −0.68 points at T2I R@1 and −2.34 at I2T R@1; its R@10 also decreases in both directions. Other coarse references benefit more. The scatter illustrates that directional symmetry is not automatic even though both directions share the same place-ID relevance rule. Their query distributions and numbers of same-place gallery samples differ. Neither asymmetric recall nor a reranking decline identifies a unique failure mechanism: candidate coverage, similarity calibration, and cross-modal scoring can all contribute.

Adaptation Components Under the Same Protocol

What this comparison controls. The SigLIP2 component variants retain the fixed protocol and the recorded common training budget. They train only residual heads, omit residual heads, adapt the vision tower alone, adapt the text tower alone, or adapt both towers with heads. This tests sensitivity to the chosen adaptation components. It does not introduce a new architecture, and changing the trainable parameter subset also changes optimization capacity.

Figure 8. Coarse bidirectional R@1 across adaptation variants. The x-axis lists categorical configurations; connecting segments aid comparison rather than imply a continuous parameter sweep.

ProtocolVariantCoarse R@1 T2ICoarse R@1 I2TReranked R@1 T2IReranked R@1 I2T
P0Heads only9.36±0.109.36 \pm0.1010.40±0.1810.40 \pm0.1812.31±0.6712.31 \pm0.6715.06±0.5315.06 \pm0.53
P0No residual heads10.14±0.1610.14 \pm0.1611.27±0.1011.27 \pm0.1013.09±0.0713.09 \pm0.0715.47±0.2115.47 \pm0.21
P0Vision only12.59±0.2112.59 \pm0.2113.33±0.1913.33 \pm0.1915.01±0.0715.01 \pm0.0717.47±0.1017.47 \pm0.10
P0Text only9.21±0.189.21 \pm0.1810.22±0.3410.22 \pm0.3413.15±0.7613.15 \pm0.7615.39±0.4915.39 \pm0.49
P0Both + heads12.95±0.2212.95 \pm0.2213.64±0.2713.64 \pm0.2715.24±0.3015.24 \pm0.3017.35±0.1517.35 \pm0.15
P1Heads only2.46±0.042.46 \pm0.042.02±0.112.02 \pm0.114.08±0.024.08 \pm0.024.16±0.024.16 \pm0.02
P1No residual heads2.76±0.032.76 \pm0.033.43±0.063.43 \pm0.064.53±0.044.53 \pm0.045.00±0.045.00 \pm0.04
P1Vision only3.52±0.053.52 \pm0.053.31±0.043.31 \pm0.045.13±0.035.13 \pm0.035.41±0.015.41 \pm0.01
P1Text only2.53±0.042.53 \pm0.042.24±0.112.24 \pm0.114.16±0.004.16 \pm0.004.51±0.064.51 \pm0.06
P1Both + heads3.68±0.053.68 \pm0.053.46±0.043.46 \pm0.045.21±0.055.21 \pm0.055.72±0.085.72 \pm0.08

Table A4. Adaptation sensitivity in percent; mean ±\pm sample SD across three training seeds.

Why the scoring stage matters. The component ranking should be assessed separately for coarse retrieval and the frozen reranked system. An adaptation that alters candidate coverage can interact with the reranker even when its coarse R@1 changes little. The table preserves both outputs instead of selecting the better one after testing. All variants are finite-budget references; a weaker result here is not evidence that a component can never help under a different training schedule. The benchmark interpretation remains the same: place supervision can improve representations while leaving substantial dependence on observation and gallery conditions.

Training-Seed Dispersion and Checkpoint Selection

Reported variation is training variation. Each supervised recipe uses seeds 42, 43, and 44. Error bars are sample SD over these three fitted systems, not confidence intervals over test queries. A small bar describes repeatability under this specific recipe and fixed dataset; it does not cover data resampling, annotation uncertainty, or an unseen geographic distribution.

Figure 9. Coarse R@1 means and training-seed SD. Circle/square markers distinguish T2I and I2T.

ProtocolRecipeT2I R@1 MeanT2I R@1 SDI2T R@1 MeanI2T R@1 SDSelected epochs Seeds 42/43/44
P0CLIP + place10.590.1211.170.113,3,3
P0BLIP + MS17.540.3023.620.273,3,3
P0SigLIP2 + MS14.250.1117.350.093,3,3
P0SigLIP2 + place12.950.2213.640.273,3,3
P1CLIP + place2.700.082.820.053,3,3
P1BLIP + MS6.090.128.360.103,3,3
P1SigLIP2 + MS4.110.045.180.043,3,3
P1SigLIP2 + place3.680.053.460.043,3,3

Table A5. Seed stability of the four supervised references. Epochs are the archived validation-selected coarse checkpoints.

Selection is distinct from convergence. All listed coarse checkpoints were selected at epoch 3 under the recorded weaker-direction validation R@100 criterion. This shows that the final recorded epoch was preferred under that criterion; it does not establish that further training would be beneficial, harmful, or converged. Several between-method differences exceed the displayed seed SD, but formal claims about model superiority would require an appropriate paired evaluation and a broader uncertainty design. No new seed is generated here, and no test score changes checkpoint selection.

Epoch-Wise Optimization and Diminishing Increments

Validation trajectories, not test trajectories. The eight panels expose the logged three-epoch budget. Loss values belong to different objectives and are not comparable as a quality ranking. Validation R@1R@1 characterizes early candidate ranking; weaker-direction R@100R@100 is the selection criterion and can improve without a matching change in every R@1R@1 direction.

Figure 10. Three-seed means and SD across the logged training epochs; no epoch-zero measurements are invented.

ProtocolRecipeT2I R@1R@1 increment 1 to 2T2I R@1R@1 increment 2 to 3I2T R@1R@1 increment 1 to 2I2T R@1R@1 increment 2 to 3
P0CLIP + place+0.64+0.32+1.46+0.14
P0BLIP + MS+2.67+0.31+2.63+0.86
P0SigLIP2 + MS+1.69+0.53+1.41+0.31
P0SigLIP2 + place+1.09+0.13+1.12+0.34
P1CLIP + place+0.37+0.06+0.30+0.06
P1BLIP + MS+1.05+0.20+1.14+0.26
P1SigLIP2 + MS+0.44+0.09+0.50+0.07
P1SigLIP2 + place+0.25+0.06+0.14+0.16

Table A6. Changes in mean validation recall, in percentage points. These are not selected-checkpoint test gains.

What the plateau does and does not show. Every listed recipe improves mean bidirectional validation R@1R@1 between epochs 1 and 3, while the later increment is often smaller. This supports reporting the training budget explicitly rather than describing these references as converged. It also explains why low training loss alone is not a stopping criterion for the benchmark. Longer schedules, different learning rates, or additional data remain untested hypotheses; the archived test set is not used to choose such changes.

Paired Observation Transfer as Hit Transitions

Aggregate recall hides different transitions. For each paired place, a source-view hit can remain a hit, be lost, or be replaced by a new target-view hit that did not occur for the source. We average these transition fractions over the frozen training seeds. Pretrained controls have one frozen model. The same place is not counted as three independent observations merely because three checkpoints evaluate it.

Figure 11. Percent of paired queries that lose a source-view R@1R@1 hit or gain a new target-view hit. Each protocol has its own fixed singleton cohort.

ProtocolReferenceT2I queries (%)I2T queries (%)
Both hitLostNewBoth hitLostNew
P0BLIP pretrained1.855.931.632.304.601.33
P0SigLIP2 pretrained2.223.482.672.523.562.08
P0SigLIP2 supervised8.8010.903.517.8610.872.45
P1BLIP pretrained0.382.470.740.392.100.65
P1SigLIP2 pretrained0.522.470.750.641.920.49
P1SigLIP2 supervised1.645.841.471.634.721.22

Table A7. R@1R@1 transitions. Shared misses complete each directional distribution to 100%; counts are averaged over seeds where applicable.

A net decrease is not universal per-place failure. For supervised SigLIP2 on P0, 10.90% of T2I queries lose a source hit while 3.51% gain a target hit, yielding the approximately 7.39-point net decline. On P1, the corresponding fractions are 5.84% and 1.47%. Some independent observations are therefore easier for a fixed representation even though aggregate transfer is worse. The experiment replaces visual observations; it does not identify whether a particular lost hit is caused by time, orientation, occlusion, or description visibility. Transition analysis complements the paired bootstrap intervals reported in the principal account.

City-Level Heterogeneity in Independent-Observation Retrieval

The controlled unit remains a place. We group the paired diagnostic queries by city without changing any gallery or model. Each heatmap cell is mean target-view R@10R@10 for that city’s query places; negatives remain the complete diagnostic pool for the protocol. B, S, and A denote pretrained BLIP, pretrained SigLIP2, and adapted SigLIP2; T/I denote retrieval direction. City labels include query counts.

Figure 12. Per-city diagnostic R@10R@10 (%). P0 has 23 city groups; P1 has four held-out test cities. The common color scale is clipped at 45% for visual contrast; machine-readable values retain values above this limit.

ProtocolDirectionCitiesCity-macro R@10R@10Query-micro R@10R@10
P0T2I2335.7636.84
P0I2T2334.3035.53
P1T2I413.1913.44
P1I2T412.2612.27

Table A8. Adapted SigLIP2 macro/micro summaries. Macro gives each city equal weight; micro weights cities by their query counts.

Composition changes the summary. In P0, adapted T2I R@10R@10 ranges from 15.60% in OSL to 51.57% in Osaka; these groups contain 47 and 53 queries. P1 ranges from 11.05% in WashingtonDC to 15.79% in Rome, with much larger groups. Such heterogeneity motivates reporting sample counts and aggregation rules. The city differences combine visual content, descriptions, available captures, and competition from other places. They are not causal estimates of geographic difficulty, nor are small P0 groups reliable rankings of cities. No city-specific tuning is performed.

Capture-Time and Heading Metadata

Independent observations are not uniformly extreme viewpoints. We derive circular heading gaps in [0,180][0, 180] degrees and absolute month gaps from the selected source/target metadata. The empirical distributions describe the same paired diagnostic cohort used for observation transfer. Heading is a capture attribute, not a measurement of visible overlap or landmark visibility.

Figure 13. Top: metadata distributions. Bottom: adapted target T2I R@10 by metadata bin; bins are post-hoc descriptive groups, not controlled interventions.

ProtocolMetadata25thMedian75th90th
P0Heading (degrees)4.0010.0023.00172.00
P0Capture gap (months)23.0046.0074.00109.00
P1Heading (degrees)4.009.0021.00173.00
P1Capture gap (months)25.0051.0083.00116.00

Table A9. Capture-gap quantiles for selected paired observations. Heading is in degrees; capture gap is in months.

Temporal association is visible, but confounded. For P1, target T2I R@10 is 16.35% in the 0–12 month group and 11.00% for gaps of at least 73 months; P0 shows 40.55% and 33.05%. The heading relation is not monotonic across all bins: P0 obtains 38.27%, 33.18%, and 35.39% for gaps below 15, 15–44, and at least 45 degrees. These associations are consistent with several explanations, including scene persistence and place composition. They do not isolate temporal robustness or angular invariance. Bin counts and exact scores are preserved in the released capture-diagnostic JSON; no pair is removed because of its retrieval outcome.

Caption Length and Generation-Type Associations

A selected-caption analysis. This analysis uses only the valid, single-source descriptions selected for the paired diagnostic. Length is the number of whitespace-delimited tokens, not a tokenizer-specific sequence length. We retain the original description text and generation type; no caption is edited or lengthened. The four length bins are post-hoc groups, and some groups are much smaller than others.

Figure 14. Target T2I R@10 by caption length (top) and categorical generation type (bottom), within each fixed diagnostic protocol. Type and length are associated and are not independent controls.

ProtocolToken binPlacesAdapted T2I R@10Adapted I2T R@10
P01–863329.4430.96
P09–1624037.2234.31
P017–3241542.3339.76
P033+6174.8659.02
P11–836988.527.71
P19–16148613.7512.83
P117–32281617.9017.00
P133+42525.6518.59

Table A10. Selected-description length groups, in percent. Each place contributes one paired-diagnostic caption.

Additional words are not a causal treatment. P1 adapted T2I R@10 increases from 8.52% for 1–8 tokens to 25.65% for 33 or more; P0 increases from 29.44% to 74.86%. The latter long-caption group has only 61 places, compared with 633 short-caption places. Longer captions can encode more distinguishing facts, but length also co-varies with generation type, source evidence, and scene content. These observations cannot justify automatically maximizing caption length, and they are not a length-controlled benchmark ranking. A future intervention would need paired edits on the same places while preserving correctness and target visibility.

Fixed-Query Gallery Scaling and Endpoint Retention

Negative competition changes while queries remain fixed. The nested-gallery experiment retains 500 query places and their positives in each protocol. The final P0 and P1 pools have different sizes, so endpoint-retention ratios summarize their respective recorded sweeps rather than a common geographic treatment. The one-image/one-caption candidate pools also differ from the principal full-sample galleries.

Figure 15. Gallery responses across protocol, direction, and cutoff. Points average nested-gallery samples within each checkpoint; adapted bands show training-seed SD. Full-pool repetitions are identical, not independent samples.

ProtocolReferenceT2I R@1 500T2I R@1 FinalT2I R@1 Retain%I2T R@1 500I2T R@1 FinalI2T R@1 Retain%
P0BLIP pretrained6.002.6043.334.803.2066.67
P0SigLIP2 pretrained7.204.6063.896.804.4064.71
P0SigLIP2 supervised18.0711.1361.6218.8711.0058.30
P1BLIP pretrained6.401.0015.624.600.8017.39
P1SigLIP2 pretrained7.601.0013.164.800.6012.50
P1SigLIP2 supervised14.672.8719.5514.733.1321.27

Table A11. Endpoint recall and final/initial retention. Final means 1,349 places for P0 and 8,425 for P1.

Interpretation. Adapted P1 T2I recall retains about one fifth of its 500-place level at the full diagnostic pool. Rank-10 curves also decline, demonstrating that the effect is not confined to the first result. This design isolates candidate-pool expansion for fixed queries and weights; it does not show how a newly trained model would respond to a larger training dataset. The final fixed-query score must not be equated with recall over every eligible query place.

Gallery-Draw Variation at Matched Cardinality

Five candidate draws are not five training runs. At 1,000 places, the diagnostic changes sampled candidates and associated queries according to the predefined gallery draws. Each pretrained checkpoint evaluates all five draws. For adapted SigLIP2, each of three training checkpoints evaluates the same five draws. The displayed points average training seeds within each draw; the table separately summarizes candidate-draw and training-seed variation.

Figure 16. R@1 for five fixed 1,000-place draws. The x-axis is a categorical draw identifier, not a time or optimization variable.

ProtocolReferenceDirectionMean R@1Draw SDSeed SD
P0BLIP pretrainedT2I3.880.24–
P0SigLIP2 pretrainedT2I5.360.40–
P0SigLIP2 supervisedT2I13.580.700.04
P0BLIP pretrainedI2T4.000.41–
P0SigLIP2 pretrainedI2T5.080.54–
P0SigLIP2 supervisedI2T11.690.530.28
P1BLIP pretrainedT2I4.120.54–
P1SigLIP2 pretrainedT2I3.640.69–
P1SigLIP2 supervisedT2I10.611.070.18
P1BLIP pretrainedI2T3.400.27–
P1SigLIP2 pretrainedI2T3.140.34–
P1SigLIP2 supervisedI2T9.371.130.06

Table A12. Draw SD is the five-draw sample SD per checkpoint, averaged over checkpoints. Seed SD uses each checkpoint's five-draw mean. Values are percentage points.

A concrete separation of uncertainties. For adapted P0 T2I, mean R@1 is 13.58%, mean within-checkpoint gallery-draw SD is 0.70 points, and the SD of checkpoint means is 0.04 points. Thus, candidate sampling can matter even when training-seed means are stable. Equalizing candidate count removes the large raw gallery-size difference, but does not equalize place content or training coverage. The residual protocol difference is therefore not a pure estimate of city shift. The descriptive SDs should not be pooled into a nominal sample of fifteen independent experiments.

Rule-Status Differences Before and After Matching

Retrieval association and factual quality are separate. We contrast rule-valid and rule-uncertain descriptions first on 7,735 paired places, then on 4,291 places matched by generation type, source observation, and word-count bin. The query-place cohort and gallery construction are fixed within each setting. Matching changes which places qualify; comparing the two settings is not a randomized correction of all confounders.

Figure 17. Valid-minus-uncertain differences at R@1 and R@10 for raw and matched P1 cohorts. Positive values favor valid captions in retrieval, without certifying their factual correctness.

ReferenceCohortT2I gap R@1T2I gap R@10Controlled I2T gap R@1Controlled I2T gap R@10
BLIP pretrainedRaw (7,735)-0.16+0.18+0.18+0.65
BLIP pretrainedMatched (4,291)+0.14+0.40+0.30+1.56
SigLIP2 pretrainedRaw (7,735)+0.21+0.31+0.22+0.23
SigLIP2 pretrainedMatched (4,291)+0.49+1.00+0.30+0.82
SigLIP2 supervisedRaw (7,735)-0.09+1.89+0.50+2.44
SigLIP2 supervisedMatched (4,291)+0.97+3.46+1.48+4.82

Table A13. Absolute valid-minus-uncertain differences in percentage points; adapted results average three training seeds.

ReferenceStatusT2I (%) R@1T2I (%) R@10Controlled I2T (%) R@1Controlled I2T (%) R@10
BLIP (pretrained)Valid1.194.470.864.19
BLIP (pretrained)Uncertain1.054.080.562.63
SigLIP2 (pretrained)Valid1.514.851.103.89
SigLIP2 (pretrained)Uncertain1.033.850.793.08
SigLIP2 (place-supervised)Valid3.16 ± 0.1412.27 ± 0.163.25 ± 0.2112.39 ± 0.36
SigLIP2 (place-supervised)Uncertain2.19 ± 0.088.81 ± 0.001.78 ± 0.087.57 ± 0.17

Table A14. Absolute scores on the matched 4,291-place cohort; adapted values retain their archived seed SD.

Matching changes the observed contrast. For adapted T2I R@1, the raw valid-minus-uncertain difference is approximately −0.09 points, whereas the matched difference is +0.97 points. At R@10 the matched T2I/I2T differences are +3.46 / +4.82 points. This sensitivity demonstrates why an automatic status comparison must specify its cohort and controls. It does not reverse the human audit: rule status still lacks a certified factual-quality ordering. Residual content differences and screening rules remain possible explanations. P0 does not supply the corresponding uncertain-caption pool, so no unsupported P0 comparison is manufactured.

Joint Human Judgments and Rule-Stratum Composition

The annotation axes are related measurements. The 240 audited descriptions receive source-correctness, target-visibility, and specificity labels from one blinded reviewer. Cross-tabulating the first two axes reveals how a correct source description can fail to transfer visually. Unknown labels are retained as separate outcomes. These counts neither estimate population prevalence nor provide inter-rater reliability.

Figure 18. Joint source/target judgments and source-correctness counts by automatic rule stratum. Each rule stratum contains 120 descriptions.

Source judgmentVisiblePartialNot visibleUnknownTotal
supported148693166
partial03238
contradicted0015823
unknown00113243

Table A15. Observed joint human labels; the target columns concern the displayed target-image set, not every individual view.

What the joint pattern adds. All 148 target-visible descriptions in this reviewed sample are also fully source-supported. Of 166 source-supported descriptions, 18 are not fully target-visible: six are partial, nine not visible, and three unknown. The axes should therefore not be assumed statistically independent; the pattern may also reflect the related evidence and the single reviewer's annotation logic. Source contradiction and target nonvisibility remain distinct judgments.

Screening labels do not replace this audit. Supported counts are 81 in the valid stratum and 85 in the uncertain stratum, while contradicted counts are 14 and nine. The balanced sampling and small groups preclude claiming that uncertain text is generally superior. The 240 rows cover 237 places, so row-level percentages are descriptive and should not be assigned independent-binomial population intervals without accounting for repeated places and stratified sampling.

Human-Stratified Retrieval Beyond a Single Cutoff

Cutoff and annotation axis change the interpretation. We retain the audited query texts and the original frozen model outputs. The adapted reference averages three training seeds. T2I uses all 17,014 target-image candidates; controlled I2T replaces the focal caption in an 8,509-place, one-caption-per-place gallery and averages its two target-image query outcomes. These are diagnostic scores, not the principal all-description I2T leaderboard.

Figure 19. Adapted retrieval by source support and target visibility at three cutoffs. Groups overlap across annotation axes; bars do not denote independent cohorts.

Human groupnnT2I R@1T2I R@5T2I R@10Controlled I2T R@1Controlled I2T R@5Controlled I2T R@10
source correctness / supported1661.819.8417.071.517.0311.24
source correctness / contradicted234.354.354.352.172.172.90
target visibility / visible1482.039.6817.791.697.6612.16
target visibility / not visible370.000.000.000.000.450.90
target visibility / unknown462.172.172.171.091.812.90
place specificity / distinctive1332.2610.7819.801.888.5213.66
place specificity / generic422.382.382.381.191.191.59
place specificity / unknown650.003.083.080.001.032.31

Table A16. Percent recall for selected human groups; adapted results average three frozen training seeds.

AxisJudgmentnnBLIP pretrained T2IBLIP pretrained I2TSigLIP2 pretrained T2ISigLIP2 pretrained I2TSigLIP2 adapted T2ISigLIP2 adapted I2T
Source Correctnesssupported1668.437.834.823.6117.0711.24
Source Correctnesscontradicted230.002.170.000.004.352.90
Target Visibilityvisible1488.788.115.414.0517.7912.16
Target Visibilitynot visible372.702.700.000.000.000.90
Place Specificitydistinctive1339.778.655.264.1419.8013.66
Place Specificitygeneric420.000.000.000.002.381.59
Target Visibilityunknown460.000.000.001.092.172.90

Table A17. Cross-model human-stratified R@10 (%), retaining the earlier three-family comparison. I2T remains the controlled diagnostic.

Rare hits should not dominate the scientific claim. The contradicted group has 23 items and can produce a higher R@1 percentage than the 166-item supported group from very few hits. At R@10, supported descriptions reach 17.07% T2I versus 4.35% for contradicted descriptions; visible descriptions reach 17.79% versus 0% for not-visible descriptions. This does not establish that correcting a caption would cause a particular retrieval improvement. Semantic content, specificity, and observability co-vary, and pair-level visibility does not certify each individual I2T query. The evidence supports distinguishing retrieval success from factual verification.

Shared Failures and Frozen-Model Complementarity

A query-level overlap analysis. We compare the three model configurations for which the audited feature caches and exact rankings are available: pretrained BLIP, pretrained SigLIP2, and adapted SigLIP2 with seed 42. The supervised seed is fixed, not selected for qualitative performance. A binary pattern records which model obtains a place-ID hit at rank one. For I2T, this page uses the first target view, matching the visualized queries rather than the two-view average in the audit tables.

Figure 20. Observed R@1R@1 success patterns among three frozen configurations on 240 audited descriptions. Most queries are shared misses; the audit is not a representative sample of all principal test queries.

DirectionBLIP hitsSigLIP2 hitsAdapted hitsUnionIntersectionAll miss
T2I824120228
I2T53390231

Table A18. Numbers of audited descriptions with the indicated outcome. Union is a label-aware descriptive upper bound, not a deployed ensemble.

Complementarity exists within a difficult sample. For T2I, BLIP, SigLIP2, and the adapted model obtain eight, two, and four hits; their union contains 12 descriptions, while 228 are missed by all three. I2T has five, three, and three hits, a union of nine, and 231 shared misses. Neither direction contains a description hit by all three configurations. A hypothetical oracle that chooses a successful model after seeing relevance labels is not an inference-time retrieval method and must not be reported as a new benchmark result.

Scope limits. These outcomes do not contradict the full-gallery leaderboard: the human cohort includes uncertain captions and is deliberately balanced by rule status. It uses a different query distribution, and I2T uses the controlled replacement gallery. This analysis also does not include the stronger BLIP+MS adaptation or ITM reranked systems, whose audited feature caches are not available here. The useful conclusion is conditional: the inspected models have both shared failures and distinct successes, motivating analysis of evidence and competition rather than explaining every error through a single architectural weakness.

Qualitative Text-to-Image Retrieval

Figure 21. Six post-hoc T2I cases. Rows contain a caption query, one same-place target, and top-1 results from the full 17,014-image P1 gallery. Blue solid/HIT and red dashed/MISS encode place identity, not factual correctness.

Reading the cases. Q108 and Q143 are adapted-only successes, Q049 is a BLIP-only success, and Q001/Q002 are shared failures. Q041 is hit by the two pretrained models but missed by the adapted model. Similar vegetation, entrances, or facade structure can generate plausible matches at the wrong place; the common miss pattern alone does not reveal which cue was used. The same-place reference shown is only one of multiple valid image positives. Cases were selected deterministically by audit ID within outcome strata, not randomly. They illustrate mechanisms and exceptions without estimating their frequency or selecting a preferred model. Complete captions, identities, and predictions remain available in the ranking records; ellipses denote display truncation.

Qualitative Image-to-Text Retrieval

Figure 22. Six controlled I2T cases from an 8,509-caption replacement gallery. The first target view is shown; no ITM reranking is used. Same-place captions are stored annotations, not certified factual ground truth.

Reading the cases. Q015/Q095 are adapted-only successes, Q055 favors pretrained BLIP, and Q089 favors pretrained SigLIP2; Q001/Q002 are shared failures. The opposite outcomes show that adaptation is not a per-query guarantee. A fluent returned description can remain wrong under place-ID relevance, while a place-ID hit does not establish that every sentence is visible in the query image. This page displays individual target queries; aggregate human I2T results average two views and should not be reconstructed from the six displayed rows. The same deterministic, post-hoc case-selection policy used for T2I is disclosed, and long display quotes are explicitly abbreviated. No model, fusion weight, caption, or gallery is changed to improve these examples.

References

  1. [1]Amar Ali-bey, Brahim Chaib-draa, and Philippe Giguère. GSV-Cities: Toward appropriate supervised visual place recognition. Neurocomputing, 513:194–203, 2022. 2
  2. [2]Boyuan Chen, Zhuo Xu, Sean Kirmani, Brain Ichter, Dorsa Sadigh, Leonidas Guibas, and Fei Xia. SpatialVLM: Endowing vision-language models with spatial reasoning capabilities. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14455–14465, 2024. 3arxiv.org/abs/2401.12168
  3. [3]Mingtao Feng, Longlong Mei, Zijie Wu, Jianqiao Luo, Fenghao Tian, Jie Feng, Weisheng Dong, and Yaonan Wang. Partially matching submap helps: Uncertainty modeling and propagation for text to point cloud localization. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8296–8305, 2025. 3DOI
  4. [4]Ofer Idan, Dan Badur, Yosi Keller, and Yoli Shavit. LaVPR: Benchmarking language and vision for place recognition, 2026. Version 2. 2, 3, 4, 7arxiv.org/abs/2602.03253
  5. [5]Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. BLIP: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In Proceedings of the 39th International Conference on Machine Learning, pages 12888–12900, 2022. 3, 7arxiv.org/abs/2201.12086
  6. [6]Zhenyu Li and Tianyi Shang. A2gc: Asymmetric aggregation with geometric constraints for locally aggregated descriptors. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 19423–19431, 2026. 2arxiv.org/abs/2511.14109
  7. [7]Zhenyu Li, Tianyi Shang, Pengjie Xu, Zhaojun Deng, and Ruirui Zhang. Toward robust visual place recognition for mobile robots with an end-to-end dark-enhanced net. IEEE Transactions on Industrial Informatics, 21(2):1359–1368, 2024. 2DOI
  8. [8]Zhenyu Li, Pengjie Xu, Zhenbiao Dong, Ruirui Zhang, and Zhaojun Deng. Feature-level knowledge distillation for place recognition based on soft-hard labels teaching paradigm. IEEE Transactions on Intelligent Transportation Systems, 26 (2):2091–2101, 2024. 2DOI
  9. [9]Zhenyu Li, Tianyi Shang, Pengjie Xu, and Zhaojun Deng. Place recognition meet multiple modalities: a comprehensive review, current challenges and future development. Artificial Intelligence Review, 58(11):363, 2025. 2arxiv.org/abs/2505.14068
  10. [10]Feng Lu, Xiangyuan Lan, Lijun Zhang, Dongmei Jiang, Yaowei Wang, and Chun Yuan. CricaVPR: Cross-image correlation-aware representation learning for visual place recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16772–16782, 2024. 3arxiv.org/abs/2402.19231
  11. [11]Connor Malone, Somayeh Hussaini, Tobias Fischer, and Michael Milford. A hyperdimensional one place signature to represent them all: Stackable descriptors for visual place recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9822–9833, 2025. 3arxiv.org/abs/2412.06153
  12. [12]Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Proceedings of the 38th International Conference on Machine Learning, pages 8748–8763, 2021. 3, 7arxiv.org/abs/2103.00020
  13. [13]Jiangwei Ren, Xingyu Jiang, Zizhuo Li, Dingkang Liang, Xin Zhou, and Xiang Bai. Minima: Modality invariant image matching. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 23059–23068. IEEE, 2025. 3arxiv.org/abs/2412.19412
  14. [14]Tianyi Shang, Zhenyu Li, Shuaishuai Lu, and Pengjie Xu. Fourierplace: A vision-language localization framework based on frequency domain representations. IEEE Robotics and Automation Letters, 11(1):642–649, 2025. 2
  15. [15]Tianyi Shang, Zhenyu Li, Pengjie Xu, and Jinwei Qiao. Mamplace: Text-to-point-cloud cross-modal place recognition with attention mamba mechanisms. In 2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 16985–16992. IEEE, 2025. 2DOI
  16. [16]Tianyi Shang, Zhenyu Li, Pengjie Xu, Jinwei Qiao, Gang Chen, Zihan Ruan, and Weijun Hu. Bridging text and vision: A multi-view text-vision registration approach for cross-modal place recognition. In 2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 8150–8157. IEEE, 2025. 2arxiv.org/abs/2502.14195
  17. [17]Tianyi Shang, Zhenyu Li, Pengjie Xu, Jinwei Qiao, Gang Chen, Zihan Ruan, and Weijun Hu. Bridging text and vision: A multi-view text-vision registration approach for cross-modal place recognition, 2025. Version 2. 2, 3, 4arxiv.org/abs/2502.14195
  18. [18]Yukun Song, Changwei Wang, Xingtian Pei, Shibiao Xu, Wenhao Xu, Shunpeng Chen, Yu Zhang, Ke Zhang, Rongtao Xu, Xuqiang Feng, and Pengyang Wang. DialogueVPR: Towards conversational visual place recognition, 2026. Version 1. 3, 4arxiv.org/abs/2607.14115
  19. [19]George Stoica, Vivek Ramanujan, Xiang Fan, Ali Farhadi, Ranjay Krishna, and Judy Hoffman. Contrastive flow matching. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV), pages 1185–1194. IEEE, 2025. 3arxiv.org/abs/2506.05350
  20. [20]Michael Tschannen, Alexey Gritsenko, Xiao Wang, Muhammad Ferjad Naeem, Ibrahim Alabdulmohsin, Nikhil Parthasarathy, Talfan Evans, Lucas Beyer, Ye Xia, Basil Mustafa, Olivier Hénaff, Jeremiah Harmsen, Andreas Steiner, and Xiaohua Zhai. SigLIP 2: Multilingual vision-language encoders with improved semantic understanding, localization, and dense features, 2025. 3, 7arxiv.org/abs/2502.14786
  21. [21]Issar Tzachor, Boaz Lerner, Matan Levy, Michael Green, Tal Berkovitz Shalev, Gavriel Habib, Dvir Samuel, Noam Zailer, Or Shimshi, Nir Darshan, et al. Effovpr: Effective foundation model utilization for visual place recognition. In International Conference on Learning Representations, pages 42817–42839, 2025. 2arxiv.org/abs/2405.18065
  22. [22]Xun Wang, Xintong Han, Weilin Huang, Dengke Dong, and Matthew R. Scott. Multi-similarity loss with general pair weighting for deep metric learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5022–5030, 2019. 7arxiv.org/abs/1904.06627
  23. [23]Yan Xia, Zhendong Li, Yun-Jin Li, Letian Shi, Hu Cao, João F. Henriques, and Daniel Cremers. UniLoc: Towards universal place recognition using any single modality, 2024. Version 1. 2, 3, 4arxiv.org/abs/2412.12079
  24. [24]Yan Xia, Letian Shi, Zifeng Ding, Joao F. Henriques, and Daniel Cremers. Text2Loc: 3d point cloud localization from natural language. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14958–14967, 2024. 3arxiv.org/abs/2311.15977
  25. [25]Rui Xiao, Sanghwan Kim, Mariana-Juliana Georgescu, Zeynep Akata, and Stephan Alaniz. FLAIR: VLM with fine-grained language-informed image representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24884–24894, 2025. 3arxiv.org/abs/2412.03561
  26. [26]Yuechen Xie, Xiaoyan Zhang, Yicheng Shan, Zhu Hao, Rui Tang, Rong Wei, Mingli Song, Yuanyu Wan, and Jie Song. SpatialLQA: A benchmark for evaluating spatial logical reasoning in vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2645–2657, 2026. 3arxiv.org/abs/2602.20901
  27. [27]Junyan Ye, Honglin Lin, Leyan Ou, Dairong Chen, Zihao Wang, Qi Zhu, Conghui He, and Weijia Li. Where am i? cross-view geo-localization with natural language descriptions. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5890–5900, 2025. 3arxiv.org/abs/2412.17007
  28. [28]Yunan Zeng, Yan Huang, Jinjin Zhang, Zequn Jie, Zhenhua Chai, and Liang Wang. Investigating compositional challenges in vision-language models for visual grounding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14141–14151, 2024. 3DOI
  29. [29]Sitao Zhang, Hongda Mao, Qingshuang Chen, and Yelin Kim. Efficient visual place recognition through multimodal semantic knowledge integration. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5601–5610, 2025. 3DOI

Paper details

vlnc:W_dwwg5r55
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 14

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents