Big Claims, Small Margins: Do Reported GraphLLM Gains Survive Simple Baselines?

Introduction

Text-attributed graphs (TAGs), such as citation networks, copurchase graphs and hyperlinked web pages, attach natural-language text to every node. A fast-growing line of work combines large language models (LLMs) with graph learning on TAGs: recent methods use LLMs to encode node text, predict from graph inputs, or supply supervision (§ 2). New methods typically report state-of-the-art node-classification accuracy.

Graph learning has seen this pattern before. Careful re-evaluations showed that reported GNN gains often shrink under fair comparison ([11, 28] (Shchur et al. 2019)). They also showed that propagated features with a linear head (Wu et al., 2019), label propagation with a base predictor ([16]), and tuned MLPs ([45, 17]) are competitive on popular benchmarks. Comparable reality checks exist in recommendation ([9]), metric learning ([24]) and tabular learning ([13]). LLM-based methods add two complications: a much higher training and inference cost, and papers that inherit splits, features and sometimes baseline numbers from one another.

Question and scope. We ask a narrow question: when inexpensive baselines are run on the splits a paper used, with a fixed tuning budget and validation-only selection, how do they compare with the best LLM-based result that paper reports in the same setting? We cover three recent studies whose splits we could recover: LLM4HeG (Wu et al., 2025b) and KCoT on the LLaGA split ([40, 7]) ship masks, and for LLMNodeBed (Wu et al., 2025a) we reconstruct the seeded splits from the released code. Together they give 24 dataset–protocol pairs. We also report problems in released code that we verified directly.

Findings. Several reported gains remain substantial and significant, but on many tasks the margin over a cheap baseline-like representative is fixed by name as a few points (§ 4). The released code of several papers selects on the test set, trains on labels of test-pool nodes, or misaligns node texts and labels, and one paper reuses another paper’s baseline numbers without disclosure; we cannot always establish whether the published tables came from this code (§ 5). Finally, papers whose configurations are not documented to match report the same method on the same split up to 12.5 points apart, and KCoT reports no variance, so its margins cannot be tested.

Related Work

LLMs on text-attributed graphs. The LLM can enhance or encode node text (He et al., 2024), or predict from graph tokens or serialized neighborhoods [7, 32, 46, 40, 33]. It can also annotate, teach or judge in co-training loops (Xu & Ding, 2026; Peng et al., 2026; Wei et al., 2025; Wu et al., 2025b; Duan et al., 2025), or provide the text side of graph foundation models (He et al., 2025; Wang et al., 2025b; Zhu et al., 2025; Liu et al., 2025; Wang et al., 2025a).

Benchmarks. CS-TAG [43], GLBench (Li et al., 2024), TSGFM [8] and LLMNodeBed (Wu et al., 2025a) re-run many methods under one protocol each. LLMNodeBed finds that LLMs help most when labels are scarce. These benchmarks do not systematically report the cheap baseline families we use: bag-of-words with a linear model, and label propagation or Correct & Smooth. Unlike them, we evaluate on each paper’s own splits and compare against the numbers that paper reports.

Evaluation critiques and simple baselines. Split choice and unequal tuning distort GNN comparisons (Shchur et al., 2019; Errica et al., 2020; Platonov et al., 2023), and many benchmarks barely need structure [22, 5, 17, 6]. For GraphLLMs, Petkar et al. (2026) and Xu et al. (2025) argue that current benchmarks can be solved largely from one modality, and a review proposes a reporting standard for graph learning [19]. Strong simple baselines include propagated features with simple heads (Wu et al., 2019; Frasca et al., 2020; Rubio-Madrigal & Burkholz, 2026), Correct & Smooth [16], and gradient-boosted trees on neighbor aggregates [4].

Concurrent work. NodeGround [20] and Platonov et al. [29] re-evaluate graph foundation models against tuned GNNs, without LLM-based methods and on their own splits. A concurrent submission finds that most published gaps on standard node-classification benchmarks fall below the benchmarks’ statistical resolution [2]; it does not cover GraphLLM methods or compare numbers across papers.

Evaluation Protocol

We specified the initial protocol before the pilot runs. Later changes are logged with dates. Among them are two reporting rules introduced after we inspected interim results: named family representatives, and a reference rule that rescales reported standard deviations.

Name	Representative
TF-IDF	TF-IDF → logistic regression (no graph)
TF-IDF+Prop	propagated TF-IDF → logistic regression
C&S	Correct & Smooth on the TF-IDF predictions
e5-MLP	frozen e5-large-v2 embedding → MLP (no graph)

Splits, data and metrics

For each paper we use its own train/validation/test split and its own node texts. LLM4HeG and the LLaGA split used by KCoT ship masks; we use them as-is and check that they are disjoint and agree with the node identifiers in the instruction files. For LLMNodeBed we re-implement the split function and seed sequence of the released code. We have not verified that nothing else consumes the NumPy random state before the split in the original pipeline, so these splits are likely, but not proven, to be identical. LLMNodeBed’s semi-supervised WikiCS split cannot be recovered from the released code (its released masks have 20 columns), so that task is excluded. We report the paper’s own metric, accuracy, on its own test nodes. External files are treated as untrusted: released code is read, never executed, and pickled graph objects are loaded with an allow-list of data-container types only.

Baselines and representatives

Every baseline uses only node texts, the graph and training labels (Table 1). TF-IDF statistics and SVD projections are fitted on all nodes without labels. This is the same transductive feature access a GNN has. Each family is reported through one representative fixed by name. TF-IDF applies logistic regression to TF-IDF features, without the graph. TF-IDF+Prop propagates TF-IDF as $[X, \hat{A}X, \ldots, \hat{A}^{K}X]$ before logistic regression, with the input fixed to TF-IDF. C&S applies Correct & Smooth [16] to the TF-IDF predictions, and e5-MLP is an MLP on frozen e5-large-v2 embeddings (Wang et al., 2024), without the graph. We do not report the best of our baselines per dataset as a result.

Tuning and selection

Each representative receives 50 trials of TPE search [1] over a fixed space (Section B). The trial with the best validation accuracy is selected, with ties broken by validation log-loss. Test scores are recorded for every trial for later analysis, but selection uses validation scores only. Each tuning run uses its own sampler and model seed. On LLM4HeG’s fixed split we match its 10 runs; on the LLaGA split, where KCoT reports no run count, we also use 10. LLMNodeBed reports 4 runs: on its stored masks we perform 4 tuning runs, and where it re-draws splits per run we use one tuning run per re-drawn split.

Reference and statistics

The reference for each task is the highest mean that the paper reports for an LLM-based method in the matched evaluation setting: LLM4HeG Table 2, KCoT Table 2 (Single Focus, node classification) and LLMNodeBed Table 2 (semi-supervised and supervised). It is a best-of-$N$ over methods on the paper’s test set, which makes it a demanding comparison for a single fixed baseline; we do not claim that this makes our tests conservative. LLM4HeG and LLMNodeBed compute their reported standard deviations with np.std, i.e. the population standard deviation. We rescale these by $\sqrt{n/(n-1)}$ to sample standard deviations. We then apply Welch’s $t$-test from summary statistics, Holm-corrected over the table. Where a paper reports no standard deviation or run count (KCoT), we give the difference only and mark significance as not assessable. A non-significant difference is not evidence of equivalence. Our runs and the paper’s runs share a split but are not paired, so the test treats them as independent samples. On the 12 tasks with fixed masks, the compared spread reflects training randomness only; on the 12 tasks with re-drawn splits, it also reflects variation in the partition of the observed graph. On fixed masks the test therefore omits test-set sampling variance, which on graphs is inflated by dependence between test nodes ([2]). These summary-statistic tests concern the evaluated runs and do not establish generalization to newly sampled graphs.

Pipeline check

With the same code we reproduced GCN (Kipf & Welling, [18]) and SGC (Wu et al., 2019) on the public Planetoid splits (Yang et al., [44]) (Section C). This validates the training and evaluation code, not every data loader.

Cheap Baselines on Published Splits

Table 2 lists all 24 tasks, and Table 3 summarizes each representative. Propagation narrows the median gap: TF-IDF+Prop and C&S are closer to the reference than TF-IDF alone, which is far behind on Photo and Computer. For C&S, the largest gaps are on Reddit in both LLMNodeBed settings, on Texas and Wisconsin, and on PubMed in all three settings. e5-MLP, which uses no graph, varies strongly by dataset. For C&S, of the 22 tasks where a test is possible, the reference is significantly higher on 12 and significantly lower on 1; a non-significant difference is not a tie. These tests do not establish generalization to newly sampled graphs (§ 3.4).

Dataset	Reference	TF-IDF	TF-IDF+Prop	C&S	e5-MLP
LLM4HeG (NAACL’25)
Cornell	Vicuna-7B $77.6\pm3.1$	$73.6\pm1.4$ (−4.0)	$62.6\pm5.5$ (−15.0)▼	$75.7\pm2.5$ (−1.9)	$80.7\pm2.4$ (+3.1)
Texas	Vicuna-7B $89.1\pm3.5$	$83.2\pm3.1$ (−5.9)▼	$81.1\pm6.1$ (−8.0)	$80.9\pm3.6$ (−8.2)▼	$88.6\pm2.6$ (−0.5)
Wisconsin	7B-to-560M $87.2\pm2.6$	$81.8\pm1.5$ (−5.4)	$79.3\pm2.5$ (−7.9)▼	$80.4\pm2.8$ (−6.8)▼	$83.2\pm3.3$ (−4.0)
Actor	Vicuna-7B $76.8\pm0.5$	$71.9\pm0.4$ (−4.9)▼	$73.1\pm0.7$ (−3.7)▼	$73.5\pm0.8$ (−3.3)▼	$76.0\pm0.3$ (−0.8)▼
Amazon	Bloom-1B $52.3\pm0.6$	$55.2\pm0.8$ (+2.8)▲	$47.4\pm1.1$ (−4.9)▼	$55.9\pm1.9$ (+3.5)▲	$57.7\pm0.9$ (+5.4)▲
LLaGA split (KCoT, ICML’26)
Cora	KCoT $90.6$†	$84.6\pm0.6$ (−6.0)	$88.1\pm0.6$ (−2.6)	$89.2\pm0.6$ (−1.4)	$83.0\pm0.8$ (−7.6)
PubMed	KCoT $95.9$†	$91.0\pm0.1$ (−4.8)	$90.8\pm0.4$ (−5.1)	$91.2\pm0.5$ (−4.7)	$92.6\pm0.2$ (−3.2)
LLMNodeBed semi-sup. (ICML’25)
Cora	ENGINE $84.2\pm0.5$	$71.6\pm0.8$ (−12.6)▼	$81.4\pm0.5$ (−2.8)▼	$82.0\pm0.5$ (−2.3)▼	$73.7\pm1.4$ (−10.5)▼
Citeseer	ENGINE $72.1\pm0.9$	$66.6\pm0.5$ (−5.5)▼	$63.3\pm1.7$ (−8.9)▼	$69.7\pm0.1$ (−2.4)	$70.9\pm0.5$ (−1.2)
PubMed	LLM IT $81.0\pm0.5$	$72.7\pm0.0$ (−8.3)▼	$69.9\pm0.3$ (−11.0)▼	$76.0\pm0.3$ (−4.9)▼	$76.8\pm0.9$ (−4.2)▼
Instagram	GCN LLMEmb $67.5\pm0.5$	$64.5\pm0.2$ (−3.0)▼	$65.1\pm0.4$ (−2.4)▼	$64.4\pm0.3$ (−3.1)▼	$66.5\pm0.1$ (−1.0)
Reddit	LLaGA $70.1\pm0.4$	$57.6\pm0.2$ (−12.5)▼	$60.2\pm0.1$ (−9.9)▼	$59.2\pm0.4$ (−10.9)▼	$60.3\pm0.3$ (−9.8)▼
Books	TAPE $84.9\pm0.3$	$79.7\pm0.3$ (−5.2)▼	$81.7\pm0.2$ (−3.3)▼	$82.3\pm0.4$ (−2.6)▼	$82.8\pm0.2$ (−2.1)▼
Photo	TAPE $86.5\pm0.1$	$68.1\pm0.3$ (−18.3)▼	$84.0\pm0.3$ (−2.4)▼	$84.5\pm0.3$ (−2.0)▼	$74.6\pm0.4$ (−11.9)▼
Computer	TAPE $89.5\pm0.0$	$65.3\pm0.3$ (−24.2)▼	$87.3\pm0.4$ (−2.3)▼	$88.0\pm0.2$ (−1.5)▼	$71.2\pm0.3$ (−18.4)▼
LLMNodeBed supervised (ICML’25)
Cora	GCN LLMEmb $88.2\pm2.1$	$82.0\pm3.0$ (−6.1)	$86.7\pm1.4$ (−1.4)	$86.5\pm2.2$ (−1.7)	$82.2\pm2.0$ (−5.9)
Citeseer	LLaGA $76.7\pm2.0$	$74.6\pm1.8$ (−2.1)	$74.5\pm0.5$ (−2.3)	$76.5\pm1.6$ (−0.2)	$74.9\pm2.2$ (−1.8)
PubMed	LLM IT $94.2\pm0.2$	$91.0\pm0.5$ (−3.2)▼	$89.9\pm0.5$ (−4.3)▼	$90.8\pm0.5$ (−3.4)▼	$92.3\pm0.7$ (−1.8)
WikiCS	TAPE $87.1\pm0.8$	$83.9\pm0.8$ (−3.2)▼	$84.9\pm0.4$ (−2.2)	$86.1\pm0.7$ (−1.0)	$84.8\pm1.6$ (−2.4)
Instagram	LLaGA $69.2\pm0.8$	$67.0\pm0.4$ (−2.1)	$66.9\pm0.9$ (−2.3)	$66.9\pm0.5$ (−2.3)	$68.7\pm0.7$ (−0.5)
Reddit	ENG1NE $71.2\pm0.9$	$58.9\pm0.7$ (−12.3)▼	$62.2\pm0.8$ (−9.0)▼	$60.6\pm1.1$ (−10.6)▼	$61.2\pm0.4$ (−10.0)▼
Books	TAPE $86.0\pm0.7$	$81.9\pm0.4$ (−4.0)▼	$83.7\pm0.5$ (−2.2)	$84.3\pm0.6$ (−1.6)	$85.0\pm0.6$ (−1.0)
Photo	TAPE $87.7\pm0.3$	$73.7\pm0.5$ (−14.0)▼	$86.0\pm0.3$ (−1.7)▼	$87.3\pm0.6$ (−0.4)	$76.9\pm0.1$ (−10.8)▼
Computer	TAPE $90.5\pm0.2$	$69.2\pm0.1$ (−21.3)▼	$88.5\pm0.2$ (−2.0)▼	$90.7\pm0.3$ (+0.2)	$73.7\pm0.6$ (−16.8)▼

Name	Tasks	Median gap	Gap ≤ 3	▲ / ▼	Testable
TF-IDF	24	5.5	3	17 / 1	22
TF-IDF+Prop	24	3.0	12	16 / 0	22
C&S	24	2.3	15	12 / 1	22
e5-MLP	24	2.8	12	10 / 1	22

On the LLaGA split, KCoT reports no variance, so none of its margins can be tested.

Do the gains survive? Several reported gains remain substantial under our comparison. On many of the evaluated tasks, however, a cheap baseline representative fixed by name comes within a few points of the best LLM-based result. The code audit in § 5 identifies separate evaluation risks; it does not establish that published gains arise from them.

What the Released Code Shows

We read the released code of several recent GraphLLM papers next to their tables. Below are problems we verified in the source files, pinned to specific commits in Section D. In some cases we cannot determine whether the published tables were produced with the released code; we say so where it applies.

A text–label misalignment in a released loader. Petkar et al. [27] report 28.7–29.7% accuracy on Cora for text-only (soft-prompted LLM) models and conclude that Cora is “structurally sufficient”. Their released loader takes the text of node $i$ from the paper ID in row $i$ of the original cora.content file. It takes the label of node $i$ from the PyG Planetoid version of Cora. The two files order nodes differently: position by position, the two label sequences agree on only 18.4% of nodes. Matching rows by their bag-of-words vectors yields a consistent mapping between the class labels of the two files. We re-implemented the loading logic without executing their code, which leaves 2,692 nodes after their de-duplication. We then trained TF-IDF + logistic regression on 5 of their seeded 60/20/20 splits. With their alignment it reaches 29.1$\pm$1.5%, close to the majority-class rate of 30.4%; with the correct alignment it reaches 80.9$\pm$1.2%. In this reconstruction, text alone is highly informative on Cora, and the misalignment alone brings TF-IDF to near the majority-class rate. We cannot establish whether this loader produced the published text-only result.

Selection on the test set. We found test-based selection at three levels. LLM4HeG (Wu et al., 2025b) picks its learning-rate $\times$ weight-decay grid point by the highest mean test accuracy over ten seeds. Sweep configurations in GIT’s repository history (Wang et al., [35]) maximize final/test_mean. The LLM–GNN co-teaching README (Peng et al., [26]) reports the “round-best” accuracy over training rounds that log test accuracy. In the released baseline script of Wang et al. [33], GNN checkpoints are selected by test accuracy; we cannot establish whether the published baseline rows used this script. We also reconstructed the LLM4HeG grid-selection procedure.

(same grid, seeds and epoch rule) for FAGCN, GraphSAGE and an MLP on their features. In this reconstruction, test-based selection raises accuracy by 0.30 pp on average and by at most 0.91 pp over 15 dataset–model pairs. We do not transfer this size to the full LLM4HeG pipeline.

Labels of test-pool nodes used in training. GNN-as-Judge Xu & Ding, 2026 retrains its GNN judge on “agreed” nodes, where the GNN and the LLM predict the same class. These nodes are drawn from the test-mask pool, and the re-training uses their ground-truth labels. The retrained GNN’s predictions then become targets for LLM fine-tuning. The same code evaluates LLM rows on 1,000 randomly sampled test nodes but GNN rows on the full test set. We cannot tell whether the published tables used this code path.

Reused baseline numbers. In the node-classification part of the Single Focus block of KCoT’s Table 2 (Xie et al., [40]), 32 of 32 baseline cells are identical to LLaGA’s Table 1 (Chen et al., [7]). The paper states that it adopts LLaGA’s settings; we found no statement that these numbers are taken from LLaGA.

One split, different numbers. On LLaGA’s PubMed split, LLaGA reports 95.03% for itself, while RGLM (Zhang et al., [46]) reports 90.34% for LLaGA. KCoT reports 94.23% for GraphGPT, whereas RGLM reports 81.76%. The split is shared, but the configurations are not documented to match: RGLM’s released evaluation script uses SBERT node features (Section D), which need not match the features behind the other numbers. Without the full configuration, differences of 4.7 and 12.5 points show that such numbers are not comparable across papers, not that either method is unstable.

Baselines far below standard results. UniGraph He et al. (2025) reports a supervised GCN on DeBERTa features at 48.42% on Cora with 20 labels per class. A standard GCN on bag-of-words features with the same label budget reaches 81.5% [18]; the features differ, and we have not established that the rest of the protocol is identical.

Recommendations

1. Report cheap baselines on your split, with the same tuning budget as your method: TF-IDF with a linear model, label propagation or Correct & Smooth on top of it, and frozen embeddings from the LLM your method uses.

2. Select on validation only, at every level: epochs, check-points, grids and rounds.

3. Release the split (masks or seeded code) and the exact text version.

4. Report effect sizes with uncertainty: mean, standard deviation and number of runs. A non-significant difference is not evidence of equivalence.

5. Mark copied numbers, and copy only when splits, features and tuning match.

6. Evaluate all rows on the same test nodes.

Limitations

We do not re-run the proposed methods; we compare against reported numbers, so our conclusions concern the published claims. Results are conditional on the specified search spaces, the 50-trial budget and the chosen representatives. We cover three studies whose splits we could recover. LLMNodeBed’s splits are reconstructed from its released code and are likely, but not proven, to be identical, and studies without recoverable splits are not compared. Finally, the code findings describe the released code at the pinned commits, which need not be the code that produced the published tables.

LLM Usage Statement. This manuscript acknowledges the use of Claude Opus 5.5 [3], developed by Anthropic, and of GPT-6.1 Sol [25], developed by OpenAI, to improve language clarity, refine sentence structure, and enhance overall writing precision. The study design, experiments, analysis, results, and conclusions remain the sole responsibility of the authors.

References

Data, Splits and Node Order

LLM4HeG. • Source: the authors’ dataset/*.npz files (Git LFS) at commit 99c97e3.

• Each file has a single train/val/test mask (48/32/20; Amazon 50/25/25). The “10 splits” in the training script are ten seeds on this split.

• Texts: node_texts.

LLaGA split (KCoT). • Source: LLaGA’s processed_data.pt for Cora and PubMed, from the data release linked in the authors’ README, with stored random 6:2:2 masks.

• We assert that the stored masks equal the node ids in sampled_2_10_{train,val,test}.jsonl.

• Cora node order equals cora.content (all 2,708 feature rows and labels coincide).

LLMNodeBed. • Source: the authors’ Hugging Face dataset xxwu/LLMNodeBed, revision bba716f (recorded in the download metadata).

• Files: cora, citeseer, pubmed, wikics, instagram, reddit, history (Books), photo and computer, each as .pt.

• Semi-supervised: stored masks, except Books/Photo/Computer, whose 10/10/60 splits are regenerated per run seed as in the released code.

• Supervised: np.random.seed(s) followed by a shuffle and 60/20/20 cut points, for $s = 0\ldots3$. ogbn-arxiv uses the stored OGB split.

• Semi-supervised WikiCS is excluded because its 20-column masks cannot be indexed by the released code.

• The Cora/PubMed node order in this release differs from the LLaGA/TAPE order, so node ids are never mixed across releases.

Seeds: 0–9 for LLM4HeG and the LLaGA split; 0–3 for LLMNodeBed.

Search Spaces

TF-IDF. • max_features $\in\{2\mathrm{k}, 5\mathrm{k}, 10\mathrm{k}, 20\mathrm{k}, 50\mathrm{k}\}$;

• min_df $\in\{1, 2, 3, 5\}$;

• $n$-grams $\in\{1, 2\}$;

• sublinear tf $\in\{\text{yes}, \text{no}\}$.

Logistic regression. • $C \in[10^{-3}, 10^{3}]$ (log scale);

• class weights $\in\{\text{none}, \text{balanced}\}$.

Propagation (TF-IDF+Prop). • SVD dimension $\in\{64, 128, 256, 512\}$;

• $K \in\{1, 2, 3\}$;

• normalization $\in\{\text{sym}, \text{rw}\}$;

• self-loops $\in\{\text{yes}, \text{no}\}$;

• each hop standardized and concatenated.

Correct & Smooth. • fixed-scale correction: training-node residuals are kept fixed while residuals propagate; training labels are clamped during smoothing;

• propagation over $\hat{A}$ with self-loops;

• $\alpha_1, \alpha_2 \in[0, 0.99]$;

• scale $\in[0.1, 10]$ (log scale);

• normalization $\in\{\text{sym}, \text{rw}\}$;

• 50 propagation steps;

• tuned jointly with the base model inside the same 50 trials.

MLP (e5-MLP). • embeddings: intfloat/e5-large-v2 via sentence-transformers, prefix “query: ”, at most 512 tokens, L2-normalized;

• hidden size $\in\{64, 128, 256, 512\}$;

• layers $\in\{1, 2, 3\}$;

• dropout $\in[0, 0.8]$;

• learning rate $\in[10^{-3}, 3 \cdot10^{-2}]$ (log scale);

• weight decay $\in[10^{-7}, 10^{-3}]$ (log scale);

• input standardization $\in\{\text{yes}, \text{no}\}$;

• early stopping on validation accuracy (patience 100, at most 1,000 epochs).

Pipeline Validation

Table 4. GCN and SGC on the Planetoid public splits (accuracy, %; GCN: mean$\pm$std over 10 seeds). Literature values: Kipf & Welling [18]; Wu et al. (2019).

Dataset	GCN (ours)	GCN (lit.)	SGC (ours)	SGC (lit.)
Cora	$81.9\pm0.8$	81.5	81.0	81.0
CiteSeer	$70.8\pm1.2$	70.3	71.9	71.9
PubMed	$79.2\pm0.4$	79.0	78.8	78.9

Evidence for Section 5

• Cora alignment.

  • Location: rethinking-graph-language-evals/CLEGR @ 72acb02, src/data/cora.py.

  • Planetoid labels are taken at l. 45 and l. 108; cora.content paper ids are read at l. 54–56.

  • Each node’s label is set from common_y[current_idx] at l. 169, 182 and 196, while its text is indexed by the cora.content row.

• LLM4HeG grid selection.

  • Location: honey0219/LLM4HeG @ 99c97e3.

  • src/GNN/train.py l. 104 and 127–128 print the test accuracy at the best-validation epoch.

  • src/GNN/parse_results.py l. 35–36 keeps the grid point with the highest mean of these test accuracies.

  • l. 41 prints np.std, i.e. the population standard deviation.

• GIT sweeps.

  • Location: Zehong-Wang/GIT @ 207e3a4.

  • sweep/citation/base.py l. 16 and sweep/citation/zero_shot.py l. 16 set "metric": {"goal": "maximize", "name": "final/test mean"}.

• Co-teaching.

  • Location: realjeremybot/LLM-GNN-Coteaching @ 98abd79, README.md l. 91.

  • The README states that round-by-round test accuracy is logged and that the round-best accuracy is reported.

• Pure-LLM benchmark baselines.

  • Location: JensenYX/LLM-benchmarking @ 9c3ac47, baseline/gcn_sage_gat_node.py.

  • Accuracy is computed on test_mask (l. 99–100).

  • The checkpoint is saved when that accuracy improves (l. 119–123) and is reloaded at the end (l. 135).

• GNN-as-Judge.

  • Location: rux001/GNN-as-Judge @ 61fa4c1.

  • The candidate pool is graph_data.test_mask (select_influential_nodes.py l. 421).

• Retraining on agreed nodes uses graph_data.y[train_mask] (node_selection.py l. 152–161), called at create_wsft.py l. 173.

• The retrained predictions define the SFT/DPO data (l. 184).

• LLM test sampling: create_sft.py l. 20 sets --max_test_samples with default 1000, used at l. 109.

• KCoT vs. LLaGA. KCoT arXiv:2605.24867v1, Table 2 (Single Focus, node classification), against LLaGA arXiv:2402.08170v3, Table 1, compared cell by cell.

• Same-split discrepancies and UniGraph. LLaGA Table 1, KCoT Table 2, RGLM arXiv:2603.01385v1 Table 1, and UniGraph arXiv:2402.13630v3 Table 2. RGLM’s released evaluation script sets SBERT node features (zhongjian-zhang/RGLM @ e2e7442, scripts/eval.sh l. 9).

Baselines in Existing Benchmarks

• None of CS-TAG, GLBench, TSGFM or LLMNodeBed reports TF-IDF with a linear model or gradient-boosted trees without a graph, or label propagation / Correct & Smooth, in its main results.

• Parameter-free propagation appears only in narrow settings: CS-TAG on ogbn-papers100M, and TSGFM as a co-trained backbone.