betaFeedback

WorksComputer Science

Evidence Diet: When Graph Evidence Helps LLMs Rank Competing Candidates

Abstract

When several candidates have graph support, does it matter which candidate receives which supporter and path counts? We study how LLM rankers use collaborative evidence on Amazon Books and MovieTV. Swapping user-level evidence degrades ranking, while matched item substitutions have little effect. On fixed clean-trained checkpoints, unrelated catalog text lowers Hit@1 from 44.77% to 38.70%, with 99.9% of rankings still valid. Across all three paired seeds, the NDCG@5 difference between count and path prompts was larger after count training than after path training. Degree matching preserves target paths while giving more rivals support. Across the full degree-matched development pools, adding candidate-level graph records to history improves NDCG@5 by 2.28 points on Books and 2.54 on MovieTV. When the target and a rival both have support, path- and count-trained rankers benefit from retaining counts. On these development queries, adding counts numerically to history scores outperforms presenting them as text. For count-trained rankers, exchanging intact count pairs among supported candidates preserves support and token length, yet lowers NDCG@5 by 4.06 points on 141 Books queries and 3.26 on 104 MovieTV queries. A frozen evaluation on 274 previously unused Books users with supported targets and rivals confirms the attribution effect: correct assignment improves pairwise accuracy by 6.07 percentage points (95% CI [4.18, 8.00]).

Paper details

vlnc:W_69f95ugr
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
arXiv nonexclusive license
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…