betaFeedback

WorksComputer Science

VisProve: Learning Typed Visual Provenance for Scientific Document Question Answering

Abstract

Visual citations show where an answer is supported, but not how the cited content contributes to it. In scientific document question answering, a model may read a value directly, compress several observations, or combine evidence to infer a conclusion. We study typed visual provenance, where a system generates an answer together with source regions and explicit support relations. We introduce ReViE, a dataset of 3,941 questions over complete scientific documents, with page, section, and region annotations labeled as Quotation, Compression, or Inference. We further propose VisProve, which augments joint answer and provenance generation with differentiable Spatial Evidence Reading and Evidence-Answer Relation Feedback. VisProve conditions next-token prediction on localized visual evidence and its estimated support relation through a gated norm-bounded residual. On the document-disjoint ReViE test set, VisProve improves Typed-F1 from 3.52 to 5.48 and Section-F1 from 45.99 to 53.24 over joint supervised fine-tuning with the same backbone, while Page-F1 changes only marginally. These results suggest that coupling visual grounding with support relation modeling mainly improves fine-grained provenance, while the low absolute Typed-F1 reveals a substantial gap between answer generation and precise visual evidence attribution.

Introduction

An answer to a scientific question is easier to assess when its evidence is visible. However, locating the correct table, figure, or text region does not fully explain how that evidence supports the answer. A model may read a single entry directly, summarize several observations, or compare multiple regions to derive a conclusion. These cases require different forms of verification. A bounding box therefore identifies where to look, but not how the cited region contributes to the answer. This distinction is especially important in scientific document question answering, where text, figures, and tables may provide complementary evidence across pages [25, 49].

Existing visual grounding benchmarks make supporting pages and regions inspectable [49, 23, 34]. In parallel, textual provenance research distinguishes how a source is used, including direct restatement, compression, and inference (Zhu et al., 2025; Wei et al., 2026b). These two lines of work capture complementary aspects of provenance, but they are usually modeled separately. We ask whether explicitly modeling both visual source locations and support relations can improve provenance beyond learning to serialize answers and citations jointly. Such relation labels make the proposed use of evidence explicit and evaluable, while remaining distinct from a claim that the evidence itself is correct.

Figure 1. Overview of fine-grained typed visual provenance. The model links a complete answer to supporting visual regions and specifies how each region contributes to the answer.

We study fine-grained typed visual provenance. Given a question and the complete ordered sequence of document page images, a model generates an answer and a set of evidence items. Each item specifies a page, a section, a bounding box, and a support relation. We use Quotation, Compression, and Inference to distinguish information that is directly available in a region, information that is faithfully condensed from the evidence, and conclusions that require reasoning over the evidence. Fine granularity refers to the localization of regions rather than to sentence or claim decomposition of the answer. Spatial extent and semantic use are treated separately. For example, a directly stated sentence that spans two pages remains Quotation even though its evidence occupies multiple regions.

To support this task, we construct ReViE from SciEGQA [49]. ReViE combines refined question-answering pairs from SciEGQA with newly constructed, visually verified multi-hop questions and a separate set of cross-page-contiguous questions. The dataset contains 3,941 instances linked to 8,523 visual evidence regions, with page, section, region, and support relation annotations under document-disjoint splits. We restore the complete ordered page sequence for every document, so models must locate sparse supporting evidence within the full document rather than reason only over preselected gold evidence.

We further propose VisProve, which couples evidence reading with support relation modeling during generation. Joint supervised fine-tuning (SFT) learns answer content and structured provenance, but it does not explicitly connect a predicted location to a differentiable representation of the visual content at that location. Spatial Evidence Reading (SER) addresses this gap by constructing an evidence representation from a page distribution and a latent spatial mask, both conditioned on the question and the current decoder state. Evidence–Answer Relation Feedback (ERF) maps this evidence representation to a distribution over support relations. VisProve then integrates the evidence and relation representations through a gated norm-bounded residual before the original language head. The model continues to generate both answers and provenance fields autoregressively, and its internal location and relation estimates guide generation without replacing the emitted citations.

Our main comparison is against joint supervised fine-tuning using the same backbone and training split. On the document-disjoint ReViE test set, VisProve improves Typed-F1 from 3.52 to 5.48 and Section-F1 from 45.99 to 53.24, while Page-F1 changes from 75.00 to 75.73. The larger gains at the section and typed levels suggest that the benefit is concentrated in finer provenance rather than coarse page localization. Feedback ablations also favor the full configuration for typed provenance, although improvements in answer quality are not uniform across metrics. Overall, these results suggest that explicit evidence reading and relation modeling provide complementary signals, while the low absolute Typed-F1 highlights the remaining difficulty of precise typed visual provenance.

Our contributions are summarized as follows.

  • We introduce ReViE, which combines hierarchical visual source localization with region-level support relations under complete document input.

  • We propose VisProve, which connects differentiable spatial evidence reading and support relation feedback to joint answer and provenance generation.

  • We provide shared-backbone comparisons against joint SFT, quantify gains in fine-grained provenance, and expose the substantial remaining gap in precise typed visual provenance.

Related Work

Visual document question answering. Document QA has progressed from OCR- and layout-aware representations ([26, 43, 29]) to direct understanding of document images ([16, 17, 46]). Multi-page models address cross-page aggregation and visual compression ([4, 12]), while long-document benchmarks examine understanding and reasoning over extended visual contexts ([25, 8]). Retrieval, interactive navigation, and multi-step reasoning offer complementary strategies for accessing relevant evidence ([32, 52, 19, 10]). UniDoc-RL ([37]) combines retrieval, reranking, active visual perception, and reasoning within a reinforcement-learning framework, progressively narrowing evidence from documents to images and local regions. Complementing these evidence-acquisition pipelines, SCVER ([5]) enables decoder-state-dependent access to fine-grained visual features during autoregressive document parsing. Our setting takes the complete, ordered page-image sequence of a scientific document as input and requires an answer together with structured visual provenance, making evidence localization part of the output rather than only an intermediate processing step.

Visual evidence grounding and provenance. Visual grounding associates answers with spatial support, including page references and bounding boxes ([24, 53, 23]). SciEGQA provides semantic-region evidence for scientific-document QA ([49]), while M3Grounder formulates multi-span and multi-granular grounding as pixel-level segmentation ([34]). Grounding supervision also supports evidence-aware chart and document reasoning ([35, 42]). Beyond localization alone, LAT ([21]) and Chain of Evidence ([20]) integrate attribution into reasoning or generation. DocSeeker ([45]) organizes document reasoning into analysis, evidence localization, and reasoning, while CoExVQA (Indrehus et al., 2026) explicitly localizes answer regions and restricts answer decoding to the grounded content. VinQA ([15]) interleaves cited visual elements with long-form answers, and DocTrace ([41]) builds hierarchical evidence graphs with node-level provenance. More recently, DocAttriBench ([7]) constructs element-level grounding annotations using changes in answer perplexity after masking candidate document elements. VisProve focuses on jointly predicting hierarchical visual locations and the semantic support relation of each evidence region to the complete answer, while using relation feedback to condition generation.

Textual provenance and evidence–answer relations. Evidence-based text generation studies how generated content can be connected to supporting sources, with diverse choices of attribution granularity and evaluation ([30]). Sentence- and span-level attribution provide localized source links ([50, 11]), while AliCite examines fine-grained citation placement within generated sentences ([44]). Beyond identifying sources, TROVE annotates evidence-use relations, including Quotation, Compression, Inference (Zhu et al., 2025)). GenerationPrograms makes source transformations explicit through executable text operations ([36]), and GenProve jointly generates answers and sentence-level textual provenance with semantic support relations ([40]). Extending this perspective to multimodal agents, TRACER links generated sentences to supporting tool observations and their evidence-use relations ([47]). ReViE adopts relation-aware provenance for native scientific-document images while retaining complete-answer supervision: a complete answer is linked to a set of visual evidence regions rather than decomposed into individually attributed sentences. Each evidence item specifies a page, section, bounding box, and relation. The resulting task couples locating visual support with identifying whether it supplies directly stated information, a faithful condensation, or an inference supporting the answer.

ReViE: Relation-aware Visual Evidence

Task and Annotation Schema

Given a question qq and an ordered sequence of document page images D=(I1,…,IM)D = (I_1, \ldots, I_M), the task is to generate an answer aa and an evidence set E={(pk,sk,bk,rk)}k=1KE = \{(p_k, s_k, b_k, r_k)\}_{k=1}^{K}.

Each item records its 1-based page index pkp_k, section or subsection title sks_k, bounding box bkb_k, and relation rk∈{Quotation,Compression,Inference}r_k \in\{\textit{Quotation}, \textit{Compression}, \textit{Inference}\}. The serialized fields are page, section, rel_bbox, and relation; box coordinates use a 0–1000 page-relative scale, and an unresolved section uses JSON null.

The evidence set supports the complete answer. Each region is labeled according to the answer content it contributes, and multiple regions can jointly support a conclusion. Quotation covers direct reading and minor rephrasing; Compression covers condensation of explicitly available information; and Inference covers conclusions requiring comparison, calculation, or reasoning [40] (Zhu et al. 2025). The schema associates evidence with the answer as a whole, with source granularity ranging from supporting sentences to necessary figure or table components.

Dataset Construction

ReViE combines source QA pairs from SciEGQA [49] with newly constructed multi-hop and cross-page-contiguous QA pairs (Figure 2). GPT-5.6 Sol [28] assists evidence refinement and relation annotation, complemented by human annotator review and second-round expert validation to ensure evidence quality and annotation consistency.

Figure 2. ReViE construction. Source QA pairs and newly generated, verified multi-hop QA pairs undergo fixed-QA evidence refinement. A later branch constructs additional QA pairs from contiguous evidence across adjacent pages. Supporting regions receive relation annotations, and complete ordered document pages are restored for every retained instance.

QA preparation and multi-hop construction. We associate source annotations with page images and screen candidate QA–evidence associations for consistency, correct evidence pages, valid boxes, and sufficient support. In an independent image-direct stage, new questions and reference answers are generated from at least two visual evidence regions. A blind visual judge checks whether the complete answer genuinely requires combining multiple regions. Verified QA pairs are then fixed for subsequent refinement with their question–answer content preserved.

Fixed-QA evidence refinement. This path processes both source QA pairs and previously constructed, verified multi-hop QA pairs. It preserves the question, reference answer, evidence-page identities, and correspondence to the original evidence anchors. Refinement selects complete supporting sentences or necessary figure and table components within the supplied evidence crops. Individual regions contribute support for particular answer content, while the evidence set must collectively support the complete reference answer.

Cross-page-contiguous QA construction. A later branch constructs additional QA pairs from continuous supporting content across adjacent-page boundaries. Valid configurations include a sentence split across pages or semantically continuous adjacent sentences that jointly support the QA pair. New questions and reference answers are grounded in the boundary evidence without outside knowledge. Cross-page continuity does not imply verified multi-hop status.

Annotation and finalization. Both later processing paths record the exact real section or subsection heading when it can be reliably established; otherwise, they use JSON null. Once a QA pair and its evidence are fixed, each region receives a support relation according to its contribution to the complete answer. Source units split across boxes are reconstructed before relation assignment, and uncertain cases enter a second-pass audit. The final quality-control pipeline retains 3,941 of 4,168 candidates after excluding 218 instances requiring box repair and nine with evidence mismatches. We restore complete ordered document pages and partition instances at the document level. Appendix § 7.§ 2 details quality control, Appendix § 7.§ 3 describes the construction protocol, and Figure 6 presents the relation-annotation instructions.

Prompt used for first-pass evidence–answer relation annotation with GPT-5.6 Sol.

Figure 6. Prompt used for first-pass evidence–answer relation annotation with GPT-5.6 Sol. The prompt assigns a typed provenance relation to each verified visual evidence region before subsequent review and validation.

Dataset Analysis

Coverage and spatial structure. ReViE contains 3,941 instances, 1,612 documents, 31,981 unique page images, and 8,523 evidence regions across eight scientific domains. Its document-disjoint training, validation, and test splits contain 3,115, 417, and 409 instances. Single-page single-region (SPSR), single-page multi-region (SPMR), and multi-page multi-region (MPMR) instances account for 35.70%, 46.18%, and 18.12%, respectively. Figures, tables, or mixed evidence occur in 28.88% of instances.

Support relations and sparsity. At the region level, Quotation, Compression, and Inference account for 76.91%, 2.84%, and 20.25%. Figure 3 separates MPMR instances without and with a cross-page-contiguity annotation: the former has the highest within-group Inference proportion, while the latter mainly contains Quotation. On average, each instance has 2.16 evidence regions on 1.18 pages on average, within an 18.28-page input document. The task therefore requires locating a small set of supporting regions within a much longer document. Additional statistics appear in Appendix § 7.§ 1.

Figure 3. Evidence-region distribution by spatial structure and support relation. Percentages are over 8,523 annotated regions.

Benchmark Comparison

Existing benchmarks evaluate multi-page document QA or spatial evidence grounding [25, 49, 34]. ReViE combines full scientific-document input with page, section, and region annotations and a semantic relation for each evidence item. It additionally records multi-region support, multi-page support, cross-page continuity, and multi-hop configurations. Table 7 compares the documented annotation schemes; these dimensions jointly describe task coverage and the granularity of available supervision.

BenchmarkSci.InputLocatorMRMPContig.Hop unitRel.
DocVQA—1——————
MP-DocVQA—CapP—————
SlideVQA—CapP—✓—Slide—
MMLongBench-Doc◦FullP—✓———
XL-DocBench◦FullP+T—✓—Sect.—
GDQA-Bench—1Mh^{h}✓————
SciEGQA✓FullP+B✓✓———
ReViE (Ours)✓FullP+S+B✓✓✓RegionQ/C/I

Table 7. Comparison of ReViE with seven visual document benchmarks. Columns summarize document context, visual evidence, and reasoning and relation supervision.

VisProve

VisProve learns joint answer–provenance generation and augments it with localized visual reading and relation feedback. Figure 4 summarizes the framework. SER forms a spatial evidence representation from document features, ERF encodes its support relation, and a gated residual conditions the language head on both signals.

Figure 4. VisProve. Joint SFT learns answer–provenance generation. SER reads localized visual features, ERF encodes their support relation, and a gated residual combines the two representations before the original language head.

Joint Answer–Provenance Learning

We fine-tune Qwen3-VL-8B on a unified target containing the complete answer and its structured evidence set. The standard causal language-modeling objective supervises the entire target sequence. This SFT model provides the common generation backbone; its parameters are then frozen while the provenance components are trained on the same data. Appendix B.1 specifies the SFT objective and configuration.

Spatial Evidence Reading

SER connects a latent location prediction to differentiable reading from the corresponding spatial feature grid. At generation step tt, the decoder state hth_t depends on the document, question, and generated prefix.

For each page jj, frozen visual features are pooled into a 16×1616 \times16 grid and projected to a d=128d = 128-dimensional visual memory {vjk}\{v_{jk}\} with spatial positional encodings. The memory contains all document pages. Using hth_t and the mean question embedding eˉq\bar{e}_q, SER computes a query and soft page distribution:

ut=WqLN⁡(ht)+Wquestioneˉq,πtj=softmax⁡j(utTvˉjd)(1)u_t = W_q \operatorname{LN}(h_t) + W_{\mathrm{question}}\bar{e}_q,\qquad\pi_{tj} = \operatorname{softmax}_j\left(\frac{u_t^{\mathsf{T}}\bar{v}_j}{\sqrt{d}}\right) \tag*{(1)}

where vˉj\bar{v}_j is the mean visual-memory representation of page jj.

A box predictor maps utu_t to an ordered normalized box b^t\hat{b}_t. Smooth boundary functions convert this box into spatial weights αtjk\alpha_{tjk}, which pool local evidence alongside page-level context:

rtvis=∑jπtj∑kαtjkvjk,ct=∑jπtjvˉj(2)r_t^{\mathrm{vis}} = \sum_j \pi_{tj}\sum_k \alpha_{tjk}v_{jk},\qquad c_t = \sum_j \pi_{tj}\bar{v}_j \tag*{(2)}
st=GELU⁡(Wf[ut;rtvis;ct])+MLP⁡geometry(b^t).s_t = \operatorname{GELU}\left(W_f[u_t; r_t^{\mathrm{vis}}; c_t]\right) + \operatorname{MLP}_{\mathrm{geometry}}(\hat{b}_t).

The spatial representation sts_t combines local appearance, page context, the question-conditioned query, and box geometry. The internal box guides feature reading, while the final output coordinates are separately generated by the language head. Appendix B.2 defines the box parameterization and spatial weights.

Evidence–Answer Relation Feedback

Given the question-conditioned spatial representation sts_t, ERF predicts a distribution over three support relations and maps it to a continuous feedback embedding for next-token prediction:

ztrel=Wrelst+brel,ρt=softmax⁡(ztrel),et=Wtypeρt.(3)z_t^{\mathrm{rel}} = W_{\mathrm{rel}}s_t + b_{\mathrm{rel}}, \quad\rho_t = \operatorname{softmax}(z_t^{\mathrm{rel}}), \quad e_t = W_{\mathrm{type}}\rho_t. \tag*{(3)}

The entries of ρt∈R3\rho_t \in\mathbb{R}^{3} correspond to Quotation, Compression, and Inference. Since sts_t incorporates both visual evidence and hth_t, the relation estimate is conditioned on the question and generated prefix. The embedding ete_t provides semantic feedback to generation; the final relation field is produced by the original language head.

Joint Fusion and Training Objective

Gated residual fusion. The full model combines the spatial and relation representations as

zt=st+et.(4)z_t = s_t + e_t. \tag*{(4)}

It projects this representation to the decoder dimension and applies a bounded residual update:

Δht=gtλtWupzt,ht′=ht+Δht.(5)\Delta h_t = g_t \lambda_t W_{\mathrm{up}} z_t, \quad h'_t = h_t + \Delta h_t. \tag*{(5)}

The learned gate gtg_t scales the update, and λt\lambda_t limits its pre-gated norm to 3% of ∥ht∥2\lVert h_t \rVert_2. The update is applied after the final decoder layer and before the original output head, which predicts the next token from ht′h'_t. Appendix § 8.§ 2 provides the gate and normalization formulas. § 5.1 defines the feedback variants.

Training objective. With the SFT backbone frozen, the added components are optimized using the structured generation objective together with auxiliary provenance supervision:

L=Lgen+λlocISER(Lpage+Lbox)+λrelIERFLrel+λfieldLfield+λoutsideLoutside+λgateISERLgate.(6)\mathcal{L} = \mathcal{L}_{\mathrm{gen}} + \lambda_{\mathrm{loc}} \mathbb{I}_{\mathrm{SER}} \left(\mathcal{L}_{\mathrm{page}} + \mathcal{L}_{\mathrm{box}}\right) + \lambda_{\mathrm{rel}} \mathbb{I}_{\mathrm{ERF}}\mathcal{L}_{\mathrm{rel}} + \lambda_{\mathrm{field}}\mathcal{L}_{\mathrm{field}} + \lambda_{\mathrm{outside}}\mathcal{L}_{\mathrm{outside}} + \lambda_{\mathrm{gate}} \mathbb{I}_{\mathrm{SER}}\mathcal{L}_{\mathrm{gate}}. \tag*{(6)}

The indicators ISER\mathbb{I}_{\mathrm{SER}} and IERF\mathbb{I}_{\mathrm{ERF}} specify whether the corresponding components are optimized. The page and box losses supervise latent localization, the relation loss supervises the three-way relation prediction, and the remaining losses weight provenance fields and regularize residual feedback. Auxiliary supervision pairs serialized evidence items with their gold annotations in order and is applied at configuration-specific decoder positions. Gold annotations provide training targets; inference uses the question and document pages. Appendix § 8.§ 2 specifies the supervision positions, trainable components, loss reductions, and weights.

Experiments

Experimental Setup

Data and models. We train on the 3,115-instance ReViE training split and evaluate on the document-disjoint 409-instance test split. Models receive the question and complete ordered document pages. General-purpose baselines comprise Qwen3-VL (8B and 32B) (Bai et al., [2]), InternVL3.5 (8B and 38B) (Wang et al., [38]), MiniCPM-V4.5-8B (Yu et al., [48]), Qwen2.5-VL-7B (Bai et al., [3]Chen et al., 2025), GPT-4o-mini ({citet}OpenAI, 2024), Claude Haiku 4.5 ({citet}Anthropic, 2025), and Gemini 2.5 Flash-Lite ({citet}Google DeepMind, 2025). These models and the VisProve variants use the prompt in {xref}Appendix B.4`.

Training and feedback variants. VisProve uses LoRA-based SFT (Hu et al., [13]) of Qwen3-VL-8B followed by provenance-component training with the SFT backbone frozen. The base model is included in the main comparison, while Default SFT provides the common backbone for the ablation variants. w/o ERF uses spatial feedback only (zt=stz_t = s_t) and disables ERF. w/o SER uses relation feedback only (zt=etz_t = e_t), retaining a frozen spatial reader to supply ERF’s input but excluding sts_t from direct residual feedback. The full model jointly optimizes and injects both representations. Auxiliary field losses follow the active components, as specified in Appendix § 8.§ 2. All configurations generate their final fields through the language head, without post-hoc prediction repair.

Metrics. Answer quality is measured by ROUGE-L (Lin, [18]) and BERTScore-F1 (Zhang et al., [51]); localization by Page-F1, Section-F1, Evidence-IoU, and Region-F1; and typed provenance by Relation-F1 and Typed-F1. All are reported on a 0–100 scale. Typed-F1 evaluates all test instances using page-, section-, and relation-consistent region matching. Evidence-IoU, Region-F1, and Relation-F1 are conditional averages over each model’s instances with an exactly correct evidence-page set; Section-F1 uses instances with a non-null reference section. Overall averages the answer, localization, and typed-provenance category scores with equal weight. A text-only judge scores answer quality on 50 shared questions using a 0–5 scale; its mean score is reported separately as Judge. Appendix C defines matching and aggregation.

Main Results and Ablation Study

Table 1 compares VisProve with general-purpose open- and closed-source VLMs, while Table 2 reports the comparison with Default SFT and the two feedback variants.

ModelAnswerLocalizationTyped Prov.Summary
R-L↑B-F1↑P-F1↑S-F1↑E-IoU↑R-F1↑Rel-F1↑T-F1↑Overall↑Judge↑
Open-source models
Qwen3-VL-8B48.2786.0866.5531.901.321.600.730.1830.993.04
InternVL3.5-8B26.6384.2621.8114.350.540.760.660.0021.712.66
MiniCPM-V4.5-8B10.7380.991.884.630.000.000.000.0015.831.88
Qwen3-VL-32B33.2484.7066.1637.011.772.200.930.8728.893.34
InternVL3.5-38B4.9879.914.383.771.482.362.360.0015.542.64
Qwen2.5-VL-7B44.3988.0932.2912.040.020.040.000.0425.792.44
Eagle2.5-8B35.8685.6928.1313.890.971.361.300.0824.182.24
Closed-source models
GPT-4o-mini18.8885.6731.4614.350.450.590.130.2321.391.36
Claude Haiku 4.515.7684.8636.3511.542.342.730.970.1721.371.48
Gemini 2.5 Flash-Lite23.4184.5273.8132.106.597.086.683.2129.603.42
VisProve (Ours)67.0894.1875.7353.248.589.689.225.4841.603.46

Table 1. Main results on ReViE. E-IoU, R-F1, and Rel-F1 condition on exact page-set prediction; T-F1 uses all 409 test instances. Overall equally averages the three metric categories. Judge uses 50 shared questions. Light orange and light green mark the best and second-best values, respectively.

ModelAnswerLocalizationTyped Prov.Summary
R-L↑B-F1↑P-F1↑S-F1↑E-IoU↑R-F1↑Rel-F1↑T-F1↑Overall↑Judge↑
VisProve (Ours)67.0894.1875.7353.248.589.689.225.4841.603.46
w/o ERF66.8593.9275.2145.066.667.607.033.7339.803.56
w/o SER67.8293.9674.6948.616.867.947.294.1140.373.54
Default SFT64.2493.8575.0045.995.075.915.223.5238.803.52

Table 2. Ablation study on ReViE. Metric definitions, evaluation conditions, and best/second-best highlighting follow Table 1.

Task adaptation and typed provenance. Tables 1 and 2 show gains from task-specific SFT and provenance feedback. Relative to the Qwen3-VL-8B base, Default SFT raises ROUGE-L from 48.27 to 64.24 and Typed-F1 from 0.18 to 3.52. VisProve further improves Typed-F1 to 5.48 and Section-F1 from 45.99 to 53.24, while Page-F1 increases only modestly from 75.00 to 75.73, indicating larger gains in finer provenance outputs than in page identification. In the main comparison, VisProve achieves the highest Overall score of 41.60 and leads all eight task-specific metrics.

Feedback configurations. Table 2 compares the feedback variants on a shared SFT backbone. Spatial feedback only (w/o ERF) reaches 3.73 Typed-F1, while relation feedback with a frozen spatial reader (w/o SER) reaches 4.11; combining both representations yields 5.48. VisProve also raises Overall from 38.80 for Default SFT to 41.60 and leads seven of the eight task-specific metrics, although w/o SER achieves the highest ROUGE-L and w/o ERF the highest Judge score. These results support the full configuration’s advantage in typed provenance without implying uniform gains across answer-quality measures; the variants differ in feedback and associated training configurations rather than the SFT backbone.

Comparison with Existing Methods

Table 3 compares VisProve with VISA [24], Chain of Evidence (CoE) [20], and UniDoc-RL-7B [37] in their shared answer–page–region output space. Their native inference procedures are evaluated with ROUGE-L, BERTScore-F1, Page-F1, Evidence-IoU, and Region-F1. The comparison-specific score Overallshared_{\mathrm{shared}} equally weights answer quality and localization (Appendix § 9.§ 3).

Results. VisProve has the highest values on all five shared metrics and reaches 55.98 Overallshared_{\mathrm{shared}}. Among the baselines, UniDoc-RL achieves the highest answer-quality and page-level scores, while CoE obtains higher conditional region-overlap scores (Table 3). This comparison measures VisProve after ReViE-specific training against the existing methods under their native inference procedures; the shared-backbone comparison with Default SFT provides the task-trained reference point.

MethodAnswer R-L↑Answer B-F1↑Localization P-F1↑Localization E-IoU↑Localization R-F1↑Summary Overallshared_{\mathrm{shared}} ↑
VISA [24]11.6585.537.500.020.0225.55
CoE [20]10.4584.5730.421.371.5529.31
UniDoc-RL-7B [37]28.3787.1065.310.600.7739.98
VisProve (Ours)67.0894.1875.738.589.6855.98

Table 3. Comparison in the shared answer–page–region output space. Overallshared_{\mathrm{shared}} uses two categories and differs from Overall in Table 1. E-IoU and R-F1 retain the page-exact conditioning. Light orange and light green mark the best and second-best values, respectively.

Qualitative Error Analysis

The examples in Appendix § 10 show several discrepancies between answers and provenance. One prediction gives a correct chart-reading answer but cites adjacent prose. Another omits a supporting page and misidentifies two ion species. A third omits a legend region, gives an underspecified answer, and predicts a relation different from the annotation. These cases show that answer correctness does not necessarily imply accurate visual provenance, and that localization, evidence coverage, and relation typing can fail jointly. A mismatch with the annotated evidence also does not necessarily exclude other valid supporting regions. These observations motivate evaluating source location, evidence coverage, and semantic support alongside answer quality.

Consistency with Human Judgments

Human reviewers assess 220 answers from 11 models on the same 20 test questions, using the question, reference answer, and predicted answer. Reviewers score answers independently without access to model identities or automated scores, and the final human rating for each answer is the median of its individual ratings. The model-level mean ratings have Pearson correlation r=0.9898r = 0.9898 with Judge scores on those same questions (Figure 5). This analysis concerns agreement across all 11 model averages in reference-based answer assessment. Appendix § 9.§ 5 describes the independent ratings and aggregation procedure.

Figure 5. Consistency between LLM-as-Judge scores and human ratings across 11 evaluated models.

Conclusion

Visual provenance should reveal both where supporting evidence appears and how it contributes to an answer. ReViE makes source localization and support relations jointly evaluable, while VisProve couples differentiable spatial evidence reading with support relation feedback during generation. Using the same backbone and training split, VisProve improves typed provenance and section-level localization over joint SFT, although gains in answer quality are not uniform. Meanwhile, the low absolute typed provenance scores highlight a substantial gap between producing a correct answer and precisely attributing its visual support. These results support learning evidence localization and evidence use jointly, while treating provenance as an evaluation target distinct from answer quality.

AI Use Statement

Generative AI tools were used to assist dataset construction, including QA construction, visual-evidence refinement, multi-hop verification, and evidence–answer relation annotation. The authors take responsibility for the final dataset, reported results, and conclusions.

Reproducibility Statement

Appendix A documents dataset construction, statistics, and quality-control procedures; Appendix B provides training and inference configurations; and Appendix C specifies metric definitions and automated and human evaluation protocols.

Additional Dataset Details

Additional Dataset Statistics

ReViE contains 3,941 instances from 1,612 scientific documents, comprising 31,981 unique page images and 8,523 annotated evidence regions. Table 4 reports the split-level statistics. Splits are constructed at the document level, with no document or sample overlap across training, validation, and test sets.

SplitInstancesDocumentsPagesEvidence regions
Train3,1151,29625,4396,738
Validation4171613,106921
Test4091553,436864
Total3,9411,61231,9818,523

Table 4. Split-level statistics of ReViE. Page counts refer to unique document pages within each split.

Domain distribution. ReViE covers eight scientific domains. Table 5 reports both instance- and document-level distributions, showing broad coverage across computer science, economics, engineering, mathematics, physics, quantitative biology, quantitative finance, and statistics.

DomainInstancesInstance %DocumentsDocument %
Computer Science49312.5120012.41
Economics3308.3716910.48
Electrical Engineering and Systems Science66216.8023214.39
Mathematics1072.72593.66
Physics48012.1818411.41
Quantitative Biology55514.0827316.94
Quantitative Finance94023.8534521.40
Statistics3749.491509.31
Total3,941100.001,612100.00

Table 5. Domain distribution of ReViE at the instance and document levels.

Spatial and reasoning structure. At the instance level, ReViE contains 1,407 SPSR instances (35.70%), 1,820 SPMR instances (46.18%), and 714 MPMR instances (18.12%). Among all instances, 604 (15.33%) are labeled as multi-hop and 162 (4.11%) as cross-page contiguous. These attributes describe different properties: multi-hop concerns the necessity of combining evidence regions, whereas cross-page continuity concerns supporting content across an adjacent-page boundary. The two labeled subsets do not overlap in the final dataset. This dataset-specific observation does not imply conceptual incompatibility: multi-hop evidence may span multiple pages without being labeled as cross-page contiguous.

Evidence modality, granularity, and relation distribution. Table 6 summarizes evidence modality at the instance level and evidence–answer relations at the evidence-region level. ReViE is predominantly text-based, while figures, tables, and mixed-modality evidence together account for 28.88% of instances. At the evidence-granularity level, 2,803 instances use sentence-level evidence, 836 use localized visual regions, 24 require complete visual regions, and 278 combine multiple granularities. Quotation is the dominant relation type, while Inference forms a substantial minority and Compression is comparatively rare. A total of 482 instances (12.23%) contain more than one evidence–answer relation type.

Evidence modalityEvidence–answer relation
ModalityInstances%RelationRegions%
Text2,80371.12Quotation6,55576.91
Figure63216.04Compression2422.84
Table2125.38Inference1,72620.25
Mixed2947.46
Total3,941100.00Total8,523100.00

Table 6. Evidence modality and evidence–answer relation distributions in ReViE. Modality statistics are computed over 3,941 instances, whereas relation statistics are computed over 8,523 evidence regions.

Evidence density and document length. Each instance contains 2.16 evidence regions and 1.18 unique evidence pages on average. The complete document input contains 18.28 pages on average (median 14), with a maximum of 234 pages. The median number of evidence regions per instance is 2, and 90% of instances contain at most four evidence regions. No reliable section heading can be identified for 2,383 evidence regions (27.96%); these cases use JSON null in the final annotation schema.

Quality Control and Split Integrity

This section describes qualification, verification, and review procedures. The construction operations and annotation conventions are detailed in Appendix § 7.§ 3.

Task-specific qualification. Candidate QA–evidence associations are screened for question–answer–evidence consistency, evidence-page correctness, bounding-box validity, and evidence sufficiency. Instances that do not provide reliable visual supervision are excluded from subsequent construction.

Visual and structural verification. An independent visual quality check examines answer support, evidence sufficiency, localization precision, section annotations, and structural metadata. For multi-hop candidates, a blind visual judge checks whether the complete answer genuinely requires combining information from multiple evidence regions. This construction-stage verification is distinct from the text-only answer evaluation in Appendix § 9.§ 4. Cross-page-contiguous candidates are accepted only when the adjacent-page boundary content provides genuine continuous support.

Relation review and final exclusions. After the QA pair and its visual evidence are fixed, first-pass relation annotation may return NEEDS_REVIEW when the supplied evidence is unsupported or the relation is genuinely uncertain. Such cases undergo a separate second-pass audit, which may accept the existing evidence, flag a bounding-box repair requirement or evidence mismatch, or defer the instance for human review. The first-pass relation instructions are shown in Figure 6. Across the final quality-control pipeline, 227 of 4,168 candidate instances are excluded, including 218 requiring bounding-box repair and nine with evidence mismatches, yielding the final 3,941-instance dataset.

Human review and split integrity. The model-assisted construction process is complemented by human annotator review and second-round expert validation. We partition ReViE at the document level so that all instances from the same source document remain within a single split. The training, validation, and test sets contain 3,115, 417, and 409 instances, respectively, with no document or sample overlap across splits. Complete document pages are restored in their original order for every retained instance, and gold evidence pages are not used to preselect model inputs.

QA and Visual Evidence Construction Protocol

Image-direct multi-hop construction and QA freezing. An independent image-direct stage constructs new questions and reference answers from at least two visual evidence regions. After passing the multi-hop verification described in Appendix § 7.§ 2, these QA pairs enter fixed-QA evidence refinement together with source QA pairs. Questions and reference answers are frozen for both groups during refinement.

Anchor-preserving evidence refinement. Here, “existing QA” refers to a QA pair established before refinement, not necessarily inherited from the source dataset. Refinement preserves evidence-page identities and correspondence to the original evidence anchors. Each anchor defines a supplied evidence crop within which one or more refined regions are selected. The returned region boundaries may change, but the source anchor is not replaced and the question is not re-answered. Individual regions support the answer content they contribute, while the complete evidence set must collectively support the full reference answer.

Later cross-page-contiguous QA construction. A separate later branch constructs additional QA pairs from supporting content at adjacent page boundaries, without outside knowledge. It covers a sentence split across pages or semantically continuous adjacent sentences that jointly support the QA pair. In NEW_CROSS_PAGE_QA mode, an accepted instance has cross_page_continuity_valid set to true. Candidate preparation sets verified_multihop to false for this branch, so continuity alone does not confer a verified multi-hop label. This setting does not, by itself, establish that combining multiple regions is unnecessary.

Evidence completeness. Textual refinement requests complete supporting sentences. For figures and tables, it selects necessary cells, rows, headers, panels, legends, or other local components, retaining complete visual regions when required for sufficient support. When a supporting sentence spans a page boundary, the selected regions jointly cover the complete sentence across both pages; each page-local fragment need not independently form a complete sentence.

Section annotation. Both fixed-QA refinement and cross-page construction use the exact real section or subsection heading when it can be reliably established from the supplied visual context; otherwise, they use JSON null. An unresolved heading does not by itself invalidate an instance, and headings must not be fabricated to avoid a null value. In fixed-QA refinement, any supplied low-resolution page context is used to identify section headings, not to determine evidence coordinates.

Intermediate localization representations. Fixed-QA refinement returns crop_bboxes on a 0–1000 coordinate scale relative to each supplied evidence crop. The cross-page branch selects image-derived line identifiers for textual evidence, from which code computes bounding boxes, and uses page-relative coordinates for figure and table regions. These intermediate fields are distinct from the final rel_bbox field, which always uses a 0–1000 scale relative to the complete document page.

Relation Annotation Prompt

Figure 6 presents the first-pass evidence–answer relation instructions applied after QA construction and visual evidence refinement. At this stage, the question, complete reference answer, and supporting regions are fixed regardless of whether the QA pair was inherited from the source dataset or newly constructed. GPT-5.6 Sol assigns one of Quotation, Compression, or Inference to each evidence item. Cases marked as NEEDS_REVIEW follow the second-pass audit described in Appendix A.2.

The displayed output-format block specifies alternative field values and value ranges; it is a schema description rather than a literal JSON response. An actual response selects one allowed value for each field.

First-pass Relation Annotation Prompt

You annotate semantic provenance relations for an EXISTING, already verified scientific visual-document QA sample. The Question, complete Reference Answer, and every supplied Evidence region are trusted and FROZEN. Do not rewrite the answer, change evidence, add evidence, delete evidence, or assess bbox geometry.

For EACH supplied evidence image E1, E2, ..., decide how the visual content in that evidence region participates in supporting the complete Reference Answer. The red rectangle marks the exact evidence region; surrounding pixels are only small reading context.

Choose exactly one relation for each evidence region:

  • Quotation: The portion of the Answer supported by this evidence can be read directly from visible text, a table cell/value, a label, or an explicit visual statement, with at most minor grammatical or wording changes.

  • Compression: The supported Answer content faithfully condenses, summarizes, or reorganizes information explicitly present in this evidence (possibly together with other evidence), without introducing a new conclusion.

  • Inference: The supported Answer content is not explicitly stated in this evidence and requires comparison, calculation, logical deduction, visual trend interpretation, or joint reasoning with other evidence regions.

Important rules:

  1. Treat the complete QA pair as one sample. Do NOT split or rewrite it.

  2. An evidence region need not support every clause in the complete Answer. Judge its relation to the particular Answer content that it actually supports.

  3. If multiple evidence boxes are fragments of one continuous source unit because a sentence, table row, or visual element is split by a line, column, or page boundary, reconstruct that unit first and assign the SAME relation to every fragment. A directly stated sentence split across boxes remains Quotation; the physical split alone does not make it Compression or Inference.

  4. If a conclusion can only be obtained by combining multiple evidence regions, label every region essential to that conclusion as Inference.

  5. Multi-hop is not automatically Inference. Combining explicitly stated facts without deriving a new conclusion can be Compression.

  6. If one region participates in more than one way, choose the relation required for the most substantive Answer content it supports, using the priority Inference > Compression > Quotation, and explain the mixed case.

  7. If a supplied region does not actually support any part of the Answer, or the relation is genuinely indeterminate, return status NEEDS_REVIEW.

Output format.

Return JSON only with this exact top-level structure:

{
  "status": "VALID" or "NEEDS_REVIEW",
  "overall_reason": "brief sample-level assessment",
  "relations": [
  {
  "evidence_index": 1,
  "relation": "Quotation" or "Compression" or "Inference",
  "supported_answer_content": "the exact Answer content supported",
  "justification": "brief evidence-based reason",
  "confidence": 0.0 to 1.0
  }
  ]
}

Return exactly one relations item for every supplied evidence image, in index order. Do not include markdown fences or explanations outside the JSON object.

Benchmark Comparison Details

Table 7 compares ReViE with DocVQA [26], MP-DocVQA [33], SlideVQA [31], MMLongBench-Doc [25], XL-DocBench [39], GroundingDocQA-Bench [34], and SciEGQA [49]. We compare documented input settings and answer-linked evidence annotations across eight dimensions. GroundingDocQA-Bench is abbreviated as GDQA-Bench and is not conflated with the larger GroundingDocQA training resource.

Input and spatial criteria. Input distinguishes a single page (1), a capped multi-page context (Cap), and a full-document setting (Full). MP-DocVQA uses up to 20 pages, and SlideVQA retains the first 20 slides [33, 31]. SciEGQA offers a full-document setting alongside evidence-page and evidence-crop settings [49]. MR requires multiple answer-linked visual regions, not general OCR or layout boxes. MP denotes evidence on multiple pages; Contig. additionally requires explicit annotation of continuous support across adjacent page boundaries.

Localization and evidence sufficiency. Locator summarizes the evidence representation and its annotated hierarchy. XL-DocBench provides pages and supporting text snippets [39]; GroundingDocQA-Bench provides nested phrase-, line-, and block-level masks [34]; and SciEGQA provides semantically coherent region boxes [49]. ReViE adds section context and refines evidence to supporting sentences or necessary figure/table regions, retaining full regions when required. Section titles are actual document headings or null. Refinement seeks precise, collectively sufficient support; multiple valid evidence sets may exist for a question.

✓: explicitly provided; ◦: scientific content in a mixed-domain corpus; —: not explicitly specified under our criteria. MR/MP: multi-region/multi-page evidence; Contig.: contiguous cross-page annotation. P/S/B/T/M: page/section/box/text snippet/mask; hh: phrase–line–block hierarchy. Q/C/I: Quotation/Compression/Inference.

Reasoning and relation criteria. Hop unit reports the evidence granularity used in explicit multi-hop construction or verification: slides for SlideVQA bridge questions Tanaka et al., [31], chapter/section branches for XL-DocBench dependency checks Wei et al., [39], and distinct visual regions for ReViE’s image-direct multi-hop construction and blind visual verification. Multi-hop status is not inferred from MR, MP, or cross-page continuity alone. Rel. denotes per-evidence support relations, rather than question-level reasoning categories or answer-verification rules. ReViE associates each relation with an evidence item’s contribution to the complete answer, without sentence- or claim-level answer decomposition.

Implementation Details

SFT Configuration

VisProve is built on Qwen3-VL-8B [2]. We first perform LoRA-based supervised fine-tuning [13] on the 3,115-instance ReViE training split, while keeping the visual backbone frozen. The model is trained to generate a unified serialized target containing the complete answer together with its structured visual provenance.

Let y=(y1,…,yT)y = (y_1, \ldots, y_T) denote the complete assistant target. The SFT stage optimizes the standard causal language-modeling objective:

LSFT=−1T∑t=1Tlog⁡pθ(yt∣D,q,y<t).(7)\mathcal{L}_{\mathrm{SFT}} = -\frac{1}{T}\sum_{t=1}^{T}\log p_{\theta}(y_t \mid D, q, y_{<t}). \tag*{(7)}

We use LoRA with rank 32, scaling factor 64, and dropout 0.05, applied to linear layers of the language model. Optimization uses AdamW [22] with cosine learning-rate scheduling and BF16 precision. The model is trained for three epochs in the main SFT stage with a learning rate of 1×10−41 \times10^{-4}. Training then continues for two epochs in the long-document continuation stage with a learning rate of 1×10−51 \times10^{-5}. The maximum sequence length is 16,384 and gradient clipping is set to 1.0.

The resulting SFT model serves as the common backbone for all provenance-enhancement variants. Its parameters are frozen during subsequent SER and ERF optimization.

SER and ERF Training

SER and ERF are optimized using the same 3,115-instance ReViE training split as SFT, with the SFT backbone kept frozen. All provenance-enhancement configurations are trained for two epochs using AdamW ([22]) with a learning rate of 1×10−41 \times10^{-4} and weight decay of 0.01. The variants use the same SFT backbone and differ in the optimized components, direct feedback, and associated auxiliary field losses.

Spatial visual memory. For each document page, frozen visual features are adaptively average-pooled into a 16×1616 \times16 grid containing 256 spatial locations. Each grid feature is normalized, projected to d=128d = 128 dimensions, and augmented with a two-dimensional positional encoding, yielding the visual-memory vectors vjkv_{jk} used by SER. All document pages participate in the visual memory; no gold evidence page or external retriever is used to preselect the input.

Latent bounding-box parameterization. Given the SER query utu_t, the model predicts

ztb=fbox(ut)∈R4.z_t^b = f_{\mathrm{box}}(u_t) \in\mathbb{R}^4.

To ensure an ordered normalized bounding box, we define

lt=σ(zt,1:2b),dt=σ(zt,3:4b),htb=lt+(1−lt)⊙dt,(8)l_t = \sigma(z_{t,1:2}^b), \qquad d_t = \sigma(z_{t,3:4}^b), \qquad h_t^b = l_t + (1-l_t) \odot d_t, \tag*{(8)}

and form b^t=[lt;htb]\hat{b}_t = [l_t; h_t^b] as two ordered corners on the internal [0,1]2[0,1]^2 page plane. SER converts b^t\hat{b}_t into a differentiable spatial mask over the visual-memory grid. Let cjkc_{jk} denote the center coordinate of location kk on page jj. We compute

α~tjk=∏d∈{x,y}σ(κ(cjk,d−lt,d))σ(κ(ht,db−cjk,d)),(9)\widetilde{\alpha}_{tjk} = \prod_{d \in\{x,y\}} \sigma\bigl(\kappa(c_{jk,d} - l_{t,d})\bigr)\sigma\bigl(\kappa(h_{t,d}^b - c_{jk,d})\bigr), \tag*{(9)}

followed by page-wise normalization:

αtjk=α~tjk∑k′α~tjk′+ϵ.(10)\alpha_{tjk} = \frac{\widetilde{\alpha}_{tjk}}{\sum_{k'} \widetilde{\alpha}_{tjk'} + \epsilon}. \tag*{(10)}

We set κ=30\kappa= 30, which controls the sharpness of the differentiable region boundary.

Norm-bounded residual feedback. For the representation ztz_t defined in Equation (4), we first project it to the decoder dimension:

δt=Wupzt.(11)\delta_t = W_{\mathrm{up}}z_t. \tag*{(11)}

Its magnitude is bounded relative to the original hidden state:

λt=min⁡(1,0.03∥ht∥2max⁡(∥δt∥2,ϵ)).(12)\lambda_t = \min\left(1, \frac{0.03\lVert h_t\rVert_2}{\max(\lVert\delta_t\rVert_2,\epsilon)}\right). \tag*{(12)}

A learned gate further controls the residual contribution:

gt=σ(Wgut+bg),Δht=gtλtδt,ht′=ht+Δht.(13)g_t = \sigma(W_g u_t + b_g), \qquad\Delta h_t = g_t\lambda_t\delta_t, \qquad h'_t = h_t + \Delta h_t. \tag*{(13)}

Thus, before gating, the residual norm is bounded by 3% of ∥ht∥2\lVert h_t\rVert_2, while gtg_t may further reduce its contribution. The modification is applied after the final decoder layer and before the original output head.

Training objective. The objective in Equation (6) uses the following weights:

L=Lgen+0.1[ISER(Lpage+Lbox)+IERFLrel]+0.25Lfield+10Loutside+0.1ISERLgate.(14)\mathcal{L} = \mathcal{L}_{\mathrm{gen}} + 0.1\left[\mathbb{I}_{\mathrm{SER}}(\mathcal{L}_{\mathrm{page}} + \mathcal{L}_{\mathrm{box}}) + \mathbb{I}_{\mathrm{ERF}}\mathcal{L}_{\mathrm{rel}}\right] + 0.25\mathcal{L}_{\mathrm{field}} + 10\mathcal{L}_{\mathrm{outside}} + 0.1\mathbb{I}_{\mathrm{SER}}\mathcal{L}_{\mathrm{gate}}. \tag*{(14)}

Supervision alignment and trainable components. Auxiliary targets are paired with gold evidence items in their serialized order after checking that the field counts agree. Field-value character spans are mapped to tokenizer positions. Let τb\tau_b, τs\tau_s, and τr\tau_r denote the decoder positions immediately preceding the first coordinate token of rel_bbox, the first content token of section, and the first content token of relation, respectively. Table 8 summarizes the supervision positions and trainable components. In w/o SER, the frozen spatial reader still supplies sts_t to ERF, but only ete_t is directly injected. Serialized page values are learned through generation cross-entropy, and there is no separate section-title classification head.

Loss computation and reductions. Lgen\mathcal{L}_{\mathrm{gen}} is causal cross-entropy averaged over the complete answer–provenance target and its end-of-sequence token. Each page or relation classification loss is averaged over evidence items; each box loss combines mean coordinate-level ℓ1\ell_1 error with mean 1−IoU1-\mathrm{IoU} over evidence items. Lpage\mathcal{L}_{\mathrm{page}} and Lbox\mathcal{L}_{\mathrm{box}} sum the corresponding contributions listed in Table 8, without averaging over the number of contributions. In w/o ERF, the section-associated localization call is mapped to τb\tau_b, so both page-and-box contributions are retained even though they use the same position.

ConfigurationTrainable componentsAuxiliary supervisionField CE values
w/o ERFSpatial reader, geometric fusion, gate, and up-projectionPage and box at τb\tau_b, with two loss contributionsrel_bbox
w/o SERRelation classifier and relation embeddings onlyRelation at τr\tau_rsection, relation
VisProveAll provenance-enhancement componentsPage, box, and relation at τr\tau_r; page and box at τs\tau_srel_bbox, section, relation

Table 8. Configuration-specific training of provenance-enhancement components. The SFT backbone remains frozen in all variants.

Field masks and residual regularization. Lfield\mathcal{L}_{\mathrm{field}} averages causal cross-entropy over tokens overlapping the field-value spans listed in Table 8, including JSON null for unresolved sections and punctuation within the selected spans. Both generation and field losses supervise the prediction of each token from its preceding decoder state. The field-token mask also defines positive gate targets, with other output positions assigned negative targets. Lgate\mathcal{L}_{\mathrm{gate}} uses binary cross-entropy with equal weight for the positive and negative groups and is included only for w/o ERF and VisProve. Loutside\mathcal{L}_{\mathrm{outside}} averages the relative residual energy

∥Δht∥22∥ht∥22(15)\frac{\lVert\Delta h_t\rVert_2^2}{\lVert h_t\rVert_2^2} \tag*{(15)}

over decoder positions predicting output tokens outside the selected fields. These positions include answer text and field names, which remain supervised by Lgen\mathcal{L}_{\mathrm{gen}}. Prompt tokens do not contribute to the output losses.

Inference Configuration

At inference time, the model receives the question together with all pages of the corresponding document in their original order. Gold evidence pages, sections, bounding boxes, and relation labels are not provided. We use greedy decoding with a maximum of 768 newly generated tokens, without sampling or beam search. The answer and all provenance fields are generated autoregressively by the original language head.

For SER and ERF, the per-page visual budget is adjusted according to document length to control the total visual context: documents with at most 12 pages use up to 262,144 pixels per page, documents with 13–32 pages use up to 131,072 pixels per page, and longer documents use up to 65,536 pixels per page.

No post-hoc bounding-box calibration, section lookup, relation rewriting, or other prediction repair is applied. The latent bounding box predicted by SER is used only for internal differentiable visual reading and does not overwrite the bounding box generated in the final structured output.

The decoding and visual-budget settings in this subsection apply to VisProve and its two feedback variants. General-purpose baselines use the shared task prompt in Appendix § 8.§ 4 with model-specific image processing, while VISA, CoE, and UniDoc-RL follow their native inference procedures as described in § 5.3.

Unified Inference Prompt

Figure 7 shows the unified task prompt used for general-purpose open- and closed-source VLM baselines and for the SFT/SER/ERF variants. Each instance additionally provides all document pages in their original order together with the question. Reference answers and gold provenance annotations are not included at inference time. Specialized methods in Section 5.3 follow their respective native inference procedures.

Unified inference prompt

Figure 7. Unified inference prompt used for general-purpose VLM baselines and VisProve variants. The prompt specifies complete-document input, structured answer–provenance output, visual localization requirements, and evidence–answer relation types.

Evaluation Details

Metric Definitions

We evaluate models along three complementary dimensions. Answer quality is measured by ROUGE-L and BERTScore-F1. Evidence localization is evaluated using Page-F1, Section-F1, Evidence-IoU, and Region-F1, covering page identification, section localization, and fine-grained visual-region overlap. Typed provenance is measured by Relation-F1 and Typed-F1, which assess relation-consistent evidence prediction and complete typed provenance, respectively. All task-specific metrics are reported on a 0–100 scale, with higher values indicating better performance.

Answer quality. ROUGE-L is computed between the predicted and reference answers after basic Unicode and whitespace normalization. BERTScore-F1 measures their semantic similarity using contextual representations. Both metrics are evaluated over all 409 test instances.

Page and section localization. Page-F1 treats the predicted and reference page indices as sets and computes

Page-F1⁡=2∣Π^∩Π∣∣Π^∣+∣Π∣,(16)\operatorname{Page\text{-}F1} = \frac{2\lvert\hat{\Pi} \cap\Pi\rvert}{\lvert\hat{\Pi} \rvert+ \lvert\Pi\rvert}, \tag*{(16)}

where Π^\hat{\Pi} and Π\Pi denote the predicted and reference evidence-page sets, respectively. Page-F1 is averaged over all test instances.

Section-F1 similarly compares the sets of predicted and reference non-null section names. Section strings are normalized using Unicode normalization, case folding, whitespace normalization, and removal of leading numerical section identifiers. Section-F1 is evaluated only on instances containing at least one non-null reference section; a missing predicted section receives zero credit for such an instance.

Region and typed provenance metrics. Evidence-IoU, Region-F1, Relation-F1, and Typed-F1 use one-to-one matching between predicted and reference evidence regions. Bounding boxes are evaluated directly in the normalized coordinate space, and region similarity is measured by continuous IoU rather than thresholding at a fixed IoU value. The detailed matching procedure is described in Appendix § 9.§ 2.

We additionally report a text-only LLM-as-Judge score on a shared 50-instance subset of the test set. The judge assigns an integer score from 0 to 5 to the predicted answer. This score is reported separately and is not included in the Overall metric.

Evidence Matching and Failure Handling

Let PP and GG denote the numbers of predicted and reference evidence items for an instance. For region-based metrics, we construct pairwise IoU scores between predicted and reference regions and perform maximum-weight one-to-one matching using the Hungarian algorithm. Unmatched predicted or reference evidence remains in the corresponding denominator, penalizing both over-prediction and missing evidence.

Evidence-IoU. Predicted and reference regions are matchable only when they occur on the same page. For instances whose predicted page set exactly matches the reference page set, Evidence-IoU is computed as

Evidence-IoU⁡=∑(i,j)∈MIoU⁡(bi,b^j)max⁡(P,G),(17)\operatorname{Evidence\text{-}IoU} = \frac{\sum_{(i,j)\in\mathcal{M}} \operatorname{IoU}(b_i,\hat{b}_j)}{\max(P,G)}, \tag*{(17)}

where M\mathcal{M} is the maximum-weight one-to-one matching. The final Evidence-IoU is averaged over test instances satisfying the page-set exact-match condition.

Region-F1. Using the same page-consistent IoU matching, Region-F1 is

Region-F1⁡=2∑(i,j)∈MIoU⁡(bi,b^j)P+G.(18)\operatorname{Region\text{-}F1} = \frac{2\sum_{(i,j)\in\mathcal{M}} \operatorname{IoU}(b_i,\hat{b}_j)}{P+G}. \tag*{(18)}

Region-F1 is likewise averaged over instances whose predicted and reference page sets are identical.

Relation-F1. Relation-F1 additionally requires the predicted and reference evidence items to have the same relation label. Only same-page, relation-consistent pairs retain their IoU score before Hungarian matching:

Relation-F1⁡=2∑(i,j)∈MrelIoU⁡(bi,b^j)P+G.(19)\operatorname{Relation\text{-}F1} = \frac{2\sum_{(i,j)\in\mathcal{M}_{\mathrm{rel}}} \operatorname{IoU}(b_i,\hat{b}_j)}{P+G}. \tag*{(19)}

Like Evidence-IoU and Region-F1, Relation-F1 is averaged over page-exact test instances. This metric evaluates relation-consistent evidence matching rather than relation classification in isolation.

Typed-F1. Typed-F1 applies the strongest matching condition. A matched pair must agree on page, section, and evidence–answer relation, in addition to its continuous visual overlap:

Typed-F1⁡=2∑(i,j)∈MtypedIoU⁡(bi,b^j)P+G.(20)\operatorname{Typed\text{-}F1} = \frac{2\sum_{(i,j)\in\mathcal{M}_{\mathrm{typed}}} \operatorname{IoU}(b_i,\hat{b}_j)}{P+G}. \tag*{(20)}

When the reference section is null, the section condition is treated as satisfied; when the reference section is non-null, the normalized predicted and reference section names must match. Unlike Evidence-IoU, Region-F1, and Relation-F1, Typed-F1 is evaluated over all 409 test instances rather than conditioning on exact page-set prediction.

Invalid and incomplete predictions. Predicted page indices outside the valid document range are discarded. A retained evidence item with an invalid or missing bounding box receives zero region overlap while remaining in the prediction count, and an invalid or missing relation label cannot satisfy relation-aware matching. Partially malformed structured outputs are evaluated using recoverable fields only; gold annotations are never used to repair model predictions.

Overall Score

Main evaluation. For the main results, we first average metrics within the three evaluation dimensions:

A=ROUGE-L+BERTScore-F12,(21)A = \frac{\mathrm{ROUGE\text{-}L} + \mathrm{BERTScore\text{-}F1}}{2}, \tag*{(21)}
L=Page-F1+Section-F1+Evidence-IoU+Region-F14,L = \frac{\mathrm{Page\text{-}F1} + \mathrm{Section\text{-}F1} + \mathrm{Evidence\text{-}IoU} + \mathrm{Region\text{-}F1}}{4},

and

T=Relation-F1+Typed-F12,Overall=A+L+T3.(22)T = \frac{\mathrm{Relation\text{-}F1} + \mathrm{Typed\text{-}F1}}{2}, \qquad\mathrm{Overall} = \frac{A + L + T}{3}. \tag*{(22)}

This formulation gives equal weight to answer quality, evidence localization, and typed provenance regardless of the number of individual metrics in each category.

Existing-method comparison. VISA, CoE, and UniDoc-RL do not provide the section and typed-relation outputs required by the full ReViE evaluation. We therefore evaluate their shared answer and localization metrics using the same definitions as in the main evaluation, but compute a comparison-specific Overall score over only these shared dimensions:

A5=ROUGE-L+BERTScore-F12,L5=Page-F1+Evidence-IoU+Region-F13,(23)A_5 = \frac{\mathrm{ROUGE\text{-}L} + \mathrm{BERTScore\text{-}F1}}{2}, \qquad L_5 = \frac{\mathrm{Page\text{-}F1} + \mathrm{Evidence\text{-}IoU} + \mathrm{Region\text{-}F1}}{3}, \tag*{(23)}
Overallshared=A5+L52.(24)\mathrm{Overall}_{\mathrm{shared}} = \frac{A_5 + L_5}{2}. \tag*{(24)}

This comparison-specific score is used only in Table 3 and is not directly comparable to the three-category Overall score used in Table 1.

LLM-as-Judge Protocol

We use GPT-5.6 Luna as a blind, text-only judge on a shared 50-instance subset of the 409-instance ReViE test set. All evaluated systems are scored on the same set of questions. For each instance, the judge receives only the Question, Reference Answer, and Predicted Answer; document pages, evidence regions, section annotations, relation labels, provenance predictions, and model identities are not provided.

The judge evaluates the predicted answer jointly in terms of correctness, completeness, and faithfulness to the reference answer. It assigns a single integer score from 0 to 5 according to the following rubric:

  • 5: fully correct, complete, and faithful, with no material error or omission;

  • 4: the core answer is correct, with only a minor omission, imprecision, or wording issue;

  • 3: substantial correct content is present, but with one major error or omission, or several minor issues;

  • 2: some relevant content is correct, but key information is missing or materially incorrect;

  • 1: only weakly relevant content is present, with little usable answer to the question;

  • 0: the response is completely incorrect, contradictory, off-topic, empty, a refusal, or otherwise unusable.

Faithful paraphrases, equivalent scientific notation or units, and harmless rounding are accepted, and lexical overlap is not required. Each system’s Judge score is the arithmetic mean over the shared 50 evaluated instances. The judge evaluates answer content only and does not inspect visual evidence or provenance. Its score is therefore treated as an auxiliary answer-quality signal rather than a provenance metric and is excluded from Overall.

Judge prompt. The complete system prompt and user template used for LLM-based answer evaluation are shown in Figure 8.

LLM-as-Judge Prompt

Figure 8. Prompt used for blind text-only LLM-as-Judge evaluation. The judge receives only the question, reference answer, and predicted answer and assigns an integer score from 0 to 5.

Human Evaluation Protocol

We conduct human evaluation on a shared subset of 20 test questions across 11 models, yielding 220 model–question answer observations. The evaluated systems are Qwen3-VL-8B Base, InternVL3.5-8B, MiniCPM-V-4.5-8B, Qwen3-VL-32B, InternVL3.5-38B, Qwen2.5-VL-7B, Eagle2.5-8B, GPT-4o-mini, Claude Haiku 4.5, Gemini 2.5 Flash-Lite, and VisProve. Ablation variants are not included in this analysis.

Four expert reviewers participate in the evaluation. Reviewers assess answers independently using only the Question, Reference Answer, and Predicted Answer. Document pages, separately provided evidence or provenance annotations, model identities, and automated evaluation scores are not shown. Although provenance-like content may appear in a model’s raw output, no separate evidence fields are explicitly provided to the reviewers.

Each reviewer assigns an integer score from 0 to 5 based jointly on correctness, completeness, and faithfulness to the reference answer. Here, faithfulness is assessed against the reference answer; reviewers do not inspect the source document. The scoring rubric is:

  • 5: fully correct and complete, with no material error or omission;

  • 4: the core answer is correct, with only minor issues;

  • 3: substantial correct content is present, but with a major omission or error;

  • 2: some relevant content is correct, but key information is missing or materially incorrect;

  • 1: the response is only weakly relevant and contains little usable answer content;

  • 0: the response is incorrect, off-topic, empty, or otherwise unusable.

All 220 answers are independently scored by three reviewers. A fourth reviewer independently re-evaluates 16 answers exhibiting larger disagreement among the initial three reviewers. No discussion-based consensus score is used. For the 16 re-evaluated answers, the final human score is the median of four ratings; for all remaining answers, it is the median of three ratings. Consequently, the final per-answer human score may take half-point values in the re-evaluated cases.

For consistency analysis, we compute each model’s human score as the arithmetic mean over the same 20 questions. The LLM-as-Judge score is computed over these same model–question pairs, and each point in Figure 5 represents one model’s mean human score and mean LLM-as-Judge score. Pearson’s correlation across the 11 model-level pairs is r=0.9898r = 0.9898. The correlation is computed over 11 model-level pairs; instance-level agreement is a separate quantity.

Qualitative Error Analysis

Figure 9 presents three representative failure patterns observed in VisProve predictions.

Figure 9. Representative prediction errors. Left: a correct chart-reading answer paired with a misplaced evidence box. Middle: incomplete cross-page evidence coverage accompanied by an incorrect identification. Right: incomplete evidence coverage, an underspecified answer, and a relation discrepancy.

Region localization mismatch. The left case answers the CMOS ranking question correctly, but places the evidence box over adjacent prose. The reported region differs from the chart region supporting the answer.

Cross-page evidence omission. The middle case requires introductory text and a later figure. The prediction cites only the introductory page and misidentifies two ion species. Its output exhibits both incomplete evidence coverage and an answer error.

Evidence coverage and relation discrepancy. In the right case, a predicted box overlaps an annotated plot region but omits a separate legend region. The answer gives a color-based description with insufficient specificity, and its predicted relation differs from the annotation. This case combines evidence-coverage, answer-completeness, and relation discrepancies.

These examples distinguish several forms of disagreement with annotated support. Their interpretation is relative to the reference evidence sets, which may coexist with other valid support.

References

  1. [2]Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, et al. Qwen3-vl technical report. arXiv preprint arXiv:2511.21631, 2025a.arxiv.org/abs/2511.21631
  2. [3]Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhao hai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report, 2025b. URL https://arxiv.org/abs/2502.13923.
  3. [4]Tsachi Blau, Sharon Fogel, Roi Ronen, Alona Golts, Roy Ganz, Elad Ben Avraham, Aviad Aberdam, Shahar Tsiper, and Ron Litman. Gran: Global reasoning for multi-page vqa. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 15598–15607, 2024.arxiv.org/abs/2401.03411
  4. [5]Mingxu Chai, Chenyu Liu, Ziyu Shen, Jiazheng Zhang, Kaidi Zhang, Ruoyu Chen, Jun Long, Jihua Kang, Tao Gui, and Qi Zhang. State-conditioned visual evidence retrieval for fine-grained perception in document vision-language models. arXiv preprint arXiv:2608.28698, 2026.
  5. [7]Luca De Grandis, Silvia Cappelletti, William Raccagni, Marcella Cornia, Lorenzo Baraldi, and Rita Cucchiara. Docattribute: Benchmarking answer grounding in document visual question answering. arXiv preprint arXiv:2609.20574, 2026.
  6. [8]Chao Deng, Jiale Yuan, Pi Bu, Peijie Wang, Zhong-Zhi Li, Jian Xu, Xiao-Hui Li, Yuan Gao, Jun Song, Bo Zheng, et al. Longdocurl: a comprehensive multimodal long document benchmark integrating understanding, reasoning, and locating. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1135–1159, 2025.
  7. [10]Ligi He, Zuchao Li, Hao Huang, and Ping Wang. Trace: Traversal retrieval-augmented chain of evidence for document understanding. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 9806–9825, 2026.
  8. [11]Eran Hirsch, Aviv Slobodkin, David Wan, Elias Stengel-Eskin, Mohit Bansal, and Ido Dagan. Laquer: Localized attribution queries in content-grounded generation. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 15355–15370, 2025.
  9. [12]Anwen Hu, Haiyang Xu, Liang Zhang, Jiabo Ye, Ming Yan, Ji Zhang, Qin Jin, Fei Huang, and Jingren Zhou. mplug-docowl2: High-resolution compressing for ocr-free multi-page document understanding. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 5817–5834, 2025.
  10. [13]Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021.arxiv.org/abs/2106.09685
  11. [15]Young Rok Jang, Hyesoo Kong, Kyunghwan An, Jae Sub Huh, Gyeonghun KIM, and Stanley Jungkyu Choi. Vinqa: Visual elements interleaved long-form answer generation for real-world multimodal document qa. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 41130–41139, June 2026.arxiv.org/abs/2606.16092
  12. [16]Geewook Kim, Taekgyu Hong, Moonbin Yim, Jeong Yeon Nam, Jinyoung Park, Jinyeong Yim, Wonseok Hwang, Sangdoo Yun, Dongyoon Han, and Seunghyun Park. Ocr-free document understanding transformer. In European Conference on Computer Vision, pp. 498–517. Springer, 2022.arxiv.org/abs/2111.15664
  13. [17]Kenton Lee, Mandar Joshi, Iulia Raluca Turc, Hexiang Hu, Fangyu Liu, Julian Martin Eisenschlos, Urvashi Khandelwal, Peter Shaw, Ming-Wei Chang, and Kristina Toutanova. Pix2struct: Screenshot parsing as pretraining for visual language understanding. In International Conference on Machine Learning, pp. 18893–18912. PMLR, 2023.arxiv.org/abs/2210.03347
  14. [18]Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pp. 74–81, 2004.
  15. [19]Jiaheng Lin, Kai Hu, Binghai Wang, Yuhao Zhou, Zhiheng Xi, Honglin Guo, Shichun Liu, Junzhe Wang, Shihan Dou, Enyu Zhou, et al. Mm-doc-r1: Training agents for long document visual question answering through multi-turn reinforcement learning. In Findings of the Association for Computational Linguistics: ACL 2026, pp. 29770–29783, 2026.
  16. [20]Peiyang Liu, Ziqiang Cui, Xi Wang, Di Liang, and Wei Ye. Chain of evidence: Pixel-level visual attribution for iterative retrieval-augmented generation. arXiv preprint arXiv:2605.01284, 2026a.DOI
  17. [21]Shuochen Liu, Pengfei Luo, Chao Zhang, Yuhao Chen, Haotian Zhang, Qi Liu, Xin Kou, Tong Xu, and Enhong Chen. Look as you think: Unifying reasoning and visual evidence attribution for verifiable document rag via reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pp. 32159–32167, 2026b.
  18. [22]Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017.arxiv.org/abs/1711.05101
  19. [23]Dongsheng Ma, Jiayu Li, Zhengren Wang, Yijie Wang, Jiahao Kong, Weijun Zeng, Jutao Xiao, Jie Yang, Wentao Zhang, Bin Wang, et al. Citevqa: Benchmarking evidence attribution for trustworthy document intelligence. arXiv preprint arXiv:2605.12882, 2026.arxiv.org/abs/2605.12882
  20. [24]Xueguang Ma, Shengyao Zhuang, Bevan Koopman, Guido Zuccon, Wenhu Chen, and Jimmy Lin. Visa: Retrieval augmented generation with visual source attribution. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 30154–30169, 2025.arxiv.org/abs/2412.14457
  21. [25]Yubo Ma, Yuhang Zang, Liyangyu Chen, Meiqi Chen, Yizhu Jiao, Xinze Li, Xinyuan Lu, Ziyu Liu, Yan Ma, Xiaoyi Dong, et al. Mmlongbench-doc: Benchmarking long-context document understanding with visualizations. Advances in Neural Information Processing Systems, 37:95963–96010, 2024.arxiv.org/abs/2407.01523
  22. [26]Minesh Mathew, Dimosthenis Karatzas, and C.V. Jawahar. Docvqa: A dataset for vqa on document images. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 2200–2209, January 2021.arxiv.org/abs/2007.00398
  23. [28]OpenAI. GPT-5.6: Frontier intelligence that scales with your ambition. https://openai.com/index/gpt-5-6/, July 2026.
  24. [29]Rafal Powalski, Lukasz Borchmann, Dawid Jurkiewicz, Tomasz Dwojak, Michal Pietruszka, and Gabriela Pałka. Going full-tilt boogie on document understanding with text-image-layout transformer. In International Conference on Document Analysis and Recognition, pp. 732–747. Springer, 2021.arxiv.org/abs/2102.09550
  25. [30]Tobias Schreieder, Tim Schopf, and Michael Färber. Attribution, citation, and quotation: A survey of evidence-based text generation with large language models. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 30956–31000, 2026.
  26. [31]Ryota Tanaka, Kyosuke Nishida, Kosuke Nishida, Taku Hasegawa, Itsumi Saito, and Kuniko Saito. Slidevqa: A dataset for document visual question answering on multiple images. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 13636–13645, 2023.
  27. [32]Ryota Tanaka, Taichi Iki, Taku Hasegawa, Kyosuke Nishida, Kuniko Saito, and Jun Suzuki. Vdocrag: Retrieval-augmented generation over visually-rich documents. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 24827–24837. IEEE, 2025.arxiv.org/abs/2504.09795
  28. [33]Rubén Tito, Dimosthenis Karatzas, and Ernest Valveny. Hierarchical multimodal transformers for multipage docvqa. Pattern Recognition, 144:109834, 2023.
  29. [34]Venkata Kesav Venna, Sai Madhusudan Gunda, Jyothi Swaroopa Jinka, Hrithik Sagar Rachakonda, Anirudh Srinivasan, and Ravi Kiran Sarvadebhatla. Mgrounder: Mask-based multi-span and multi-granular grounding for document qa. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 23685–23695, 2026.
  30. [35]Alexander Vogel, Omar Moured, Yufan Chen, Jiaming Zhang, and Rainer Stiefelhagen. Refchartqa: Grounding visual answer on chart images through instruction tuning. In International Conference on Document Analysis and Recognition, pp. 523–537. Springer, 2025.arxiv.org/abs/2503.23131
  31. [36]David Wan, Eran Hirsch, Elias Stengel-Eskin, Ido Dagan, and Mohit Bansal. Generation programs: Fine-grained attribution with executable programs. arXiv preprint arXiv:2506.14580, 2025.arxiv.org/abs/2506.14580
  32. [37]Jun Wang, Shuo Tan, Zelong Sun, Tiancheng Gu, Yongle Zhao, Ziyong Feng, Kaicheng Yang, and Zhiwu Lu. Unidoc-rl: Coarse-to-fine visual rag with hierarchical actions and dense rewards. arXiv preprint arXiv:2604.14967, 2026.arxiv.org/abs/2604.14967
  33. [38]Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, et al. Internvl3.5: Advancing open-source multimodal models in versatility, reasoning, and efficiency. arXiv preprint arXiv:2508.18265, 2025.arxiv.org/abs/2508.18265
  34. [39]Hongchen Wei, Yuanzhe Wang, Bei Liu, Yifan Yang, Qi Dai, Ruichun Ma, Kai Qiu, Yunsheng Li, Dongdong Chen, Chong Luo, et al. Xl-docbench: Benchmarking evidence-grounded extra-long document understanding. arXiv preprint arXiv:2608.00036, 2026a.arxiv.org/abs/2608.00036
  35. [40]Jingxuan Wei, Xingyue Wang, Yanghaoyu Liao, Jie Dong, Yuchen Liu, Caijun Jia, Bihui Yu, and Junnan Zhu. Genprove: Learning to generate text with fine-grained provenance. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 5027–5048, 2026b.
  36. [41]Le Xiang, Zhicheng Guan, Hong Chen, Xiaocong Lin, Zhenghua Lei, Teng Hu, Bolei He, and Long Zeng. Doctrace: Towards traceable long document vqa via hierarchical evidence graph reasoning. arXiv preprint arXiv:2608.03292, 2026.arxiv.org/abs/2608.03292
  37. [42]Junyu Xiong, Yonghui Wang, Weichao Zhao, Chenyu Liu, Bing Yin, Wengang Zhou, and Houqiang Li. Docr1: Evidence page-guided qpro for multi-page document understanding. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pp. 11178–11186, 2026.
  38. [43]Yang Xu, Yiheng Xu, Tengchao Lv, Lei Cui, Furu Wei, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, Wanxiang Che, et al. Layoutlmv2: Multi-modal pre-training for visually-rich document understanding. In Proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing (volume 1: Long papers), pp. 2579–2591, 2021.arxiv.org/abs/2012.14740
  39. [44]Yilong Xu, Jinhua Gao, Xiaoming Yu, Baolong Bi, Huawei Shen, and Xueqi Cheng. Aliice: Evaluating positional fine-grained citation generation. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pp. 545–561, 2025.arxiv.org/abs/2406.13375
  40. [45]Hao Yan, Yuliang Liu, Xingchen Liu, Yuyi Zhang, Minghui Liao, Jihao Wu, Wei Chen, and Xiang Bai. Docseeker: Structured visual reasoning with evidence grounding for long document understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 41140–41149, June 2026.arxiv.org/abs/2604.12812
  41. [46]Jiabo Ye, Anwen Hu, Haiyang Xu, Qinghao Ye, Ming Yan, Guohai Xu, Chenliang Li, Junfeng Tian, Qi Qian, Ji Zhang, et al. Ureader: Universal ocr-free visually-situated language understanding with multimodal large language model. In Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 2841–2858, 2023.arxiv.org/abs/2310.05126
  42. [47]Bihui Yu, Caijun Jia, Jing Chi, Xiaohan Liu, Yining Wang, He Bai, Yuchen Liu, Jingxuan Wei, and Junnan Zhu. Tracer: Verifiable generative provenance for multimodal tool-using agents. arXiv preprint arXiv:2605.09934, 2026a.arxiv.org/abs/2605.09934
  43. [48]Tianyu Yu, Zefan Wang, Chongyi Wang, Fuwei Huang, Wenshuo Ma, Zhihui He, Tianchi Cai, Weize Chen, Yuxiang Huang, Ranchi Zhao, et al. Minicpm-v 4.5: Cooking efficient llmms via architecture, data, and training recipe. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11704–11715, 2026b.arxiv.org/abs/2509.18154
  44. [49]Wenhan Yu, Zhaoxi Zhang, Wang Chen, Guangiang Qi, Weikang Li, Lei Sha, Deguо Xia, and Jizhou Huang. Sciegqa: A dataset for scientific evidence-grounded question answering and reasoning, 2026c. URL https://arxiv.org/abs/2511.15090.
  45. [50]Jiajie Zhang, Yushi Bai, Xin Lv, Wanjun Gu, Danqing Liu, Minhao Zou, Shulin Cao, Lei Hou, Yuxiao Dong, Ling Feng, et al. Longcite: Enabling llms to generate fine-grained citations in long-context qa. In Findings of the Association for Computational Linguistics: ACL 2025, pp. 5098–5122, 2025.arxiv.org/abs/2409.02897
  46. [51]Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675, 2019.arxiv.org/abs/1904.09675
  47. [52]Yuanlei Zheng, Pei Fu, Hang Li, Ziyang Wang, Yuyi Zhang, Wenyu Ruan, Xiaojin Zhang, Zhongyu Wei, Zhenbo Luo, Jian Luan, et al. Doc-v*: Coarse-to-fine interactive visual reasoning for multi-page document vqa. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 45901–45923, 2026.
  48. [53]Yinan Zhou, Yuxin Chen, Haokun Lin, Yichen Wu, Shuyu Yang, Zhongang Qi, Chen Ma, and Li Zhu. Dogr: Towards versatile visual document grounding and referring. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 3596–3606. IEEE, 2025.arxiv.org/abs/2411.17125

Paper details

vlnc:W_vewq8rjq
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Generative AI tools

View statement
Looking for related papers…
Contents