betaFeedback

WorksComputer Science

From SEC Filings to Financial Signals: Evaluating LLMs on Stock Market Reactions and Analyst Forecasts

Abstract

Form 10-Q MD&A sections combine long-form management narrative with accounting and forward-looking information. Benchmarks for financial filings commonly evaluate question answering or numerical reasoning, leaving unclear whether outputs from large language models (LLMs) align with market and analyst outcomes. We evaluate prompted, zero-shot outputs from 13 general-purpose and finance-specialized LLMs on 45,964 Form 10-Q filing events from 2001 to 2024, linking MD&A text to cross-sectional fundamental ranks, pre-filing analyst forecasts, and realized abnormal returns. The evaluation covers return ranking under five controlled information views and next-quarter EPS forecasting under direct and consensus-anchored interfaces. Relative to disclosure alone, adding cross-sectional fundamental ranks raises mean effective-date rank correlation (RankIC) by 0.0089 for effective-event-date returns and 0.0098 for following-day returns, whereas analyst-summary context provides no comparable average gain. Across the four disclosure-containing views, finance-specialized models show an effective-event-date group advantage but no statistically reliable following-day advantage, while general-purpose Qwen models produce the strongest individual return-ranking results. Under the full-information view, direct LLM forecasts average the 43.4th percentile of the analyst distribution and do not consistently beat individual analysts. Consensus anchoring raises the mean inside-range share from 41.1% to 59.5% but increases reported-EPS error.

CCS Concepts

• Computing methodologies → Natural language processing;

• Applied computing → Economics.

Keywords

large language models, financial disclosures, stock returns, analyst forecasting

Introduction

U.S. public firms disclose quarterly operating and financial information through Form 10-Q filings submitted to the SEC’s EDGAR system [28]. Among these filings, the Management’s Discussion and Analysis (MD&A) section is particularly relevant for disclosure research because it combines management narrative, accounting quantities, risk discussion, and forward-looking statements [15]. Their language is also associated with future earnings and returns [5, 15], providing observable outcomes after the model. This makes 10-Q MD&A a natural test of whether language models can turn long-form disclosures into market-aligned signals; Figure 1

Adding cross-sectional fundamental ranks to the same 10-Q MD&A can reverse the return signal inferred from the filing

Figure 1. Adding cross-sectional fundamental ranks to the same 10-Q MD&A can reverse the return signal inferred from the filing.

shows how cross-sectional context can change the signal extracted from the same disclosure.

Existing financial NLP and LLM benchmarks mainly evaluate whether models can read, reason over, or answer questions about financial documents, while the conversion of such document understanding into market-aligned signals remains less studied. FinQA evaluates numerical reasoning over financial reports [4], and TAT-QA evaluates question answering over hybrid table-text financial content [37]. SECQUE moves SEC filing evaluation toward expert-written financial analysis questions [1]. Fin-RATE further studies detail reasoning, cross-entity comparison, and longitudinal tracking over filings [10]. InvestorBench evaluates LLM-based agents in financial decision-making environments [16]. These settings evaluate answer quality, reasoning behavior, or decisions in simulated financial environments. Our study instead asks how to measure whether models can turn 10-Q disclosures into financial signals.

This shift from document answers to financial signals creates a measurement problem. A useful target must be observable outside the model, tied to how financial users act on disclosures, and comparable across information sets and response formats. The problem is difficult in SEC-filing settings, where analysts combine filing text, accounting quantities, prior expectations, and firm context, while tasks may require cross-document, cross-entity, or longitudinal reasoning [10, 13]. Model behavior may also vary with financial specialization [31] and answer format [23, 36].

We turn this measurement problem into a controlled filing-to-signal evaluation with two financial prediction tasks. First, the return-ranking task tests whether model-generated filing scores capture realized post-filing abnormal returns, a market-response target that also relates to asset-pricing evidence on firm characteristics and cross-sectional returns [8, 14]. Second, the analyst-forecasting task tests how LLM-generated EPS forecasts compare with professional analyst forecasts and reported EPS.

We provide comparative evidence on when and how signals generated by LLMs from long 10-Q filings align with market and analyst outcomes. Across matched filing events, three findings emerge. First, cross-sectional fundamental ranks provide the most consistent incremental gains in return alignment. Second, finance-specialized LLMs do not reliably dominate general-purpose models under the same information and evaluation setting. Third, LLM-generated EPS forecasts are consensus-centered and do not consistently beat individual analysts. Consensus anchoring moves forecasts closer to the professional forecast distribution while worsening accuracy against reported EPS. Our contributions are:

  • We provide a controlled comparative filing-to-signal evaluation that links LLM outputs from long 10-Q disclosures to realized abnormal returns, professional analyst forecasts, and reported EPS.

  • We show that adding cross-sectional fundamental ranks to identical MD&A inputs significantly improves return-ranking alignment on both the effective event date and the following trading day, whereas pre-filing analyst expectations provide no comparable average gain.

  • We compare general-purpose and finance-specialized LLMs under matched filing events, and show that financial-domain adaptation does not uniformly translate into stronger post-filing return alignment.

  • We show that LLM-generated EPS forecasts remain close to consensus in aggregate but do not consistently outperform individual pre-filing analyst forecasts; consensus anchoring increases the share of forecasts within the analyst range while worsening accuracy against reported EPS.

Related Work

Financial Text, Firm Characteristics, and Market Prediction

Financial NLP and empirical asset pricing both study whether public information can be converted into predictive market signals. In asset pricing, firm characteristics are widely used to model the cross-section of expected returns, including characteristic-based forecasts and machine-learning models over stock-level predictors [8, 14]. Disclosure-based studies show that SEC filings also contain market-relevant information: forward-looking MD&A statements are associated with future earnings and returns [15], and changes in filing language predict future returns and operations [5].

Recent work also studies LLMs directly as return-prediction systems. Lopez-Lira and Tang [19] derive short-horizon stock signals from news headlines, whereas Wang et al. [29] develop an LLM architecture for stock-price time series. We connect this literature with disclosure-based return prediction by generating prompted, zero-shot scores from long 10-Q MD&A under matched information views that add cross-sectional fundamental ranks or pre-filing analyst expectations. We then evaluate cross-sectional alignment with abnormal returns among events sharing the same effective event date. This design asks not only whether an LLM signal predicts returns, but how information-set construction affects measured return alignment.

Financial LLMs and Analyst-Style Evaluation

A separate line of work adapts language models to finance and evaluates financial reasoning. Domain-specific models such as FinBERT and BloombergGPT use financial corpora or financial-domain adaptation to improve financial signal processing [18, 31], while FinGPT develops an open-source financial LLM framework [34]. Financial LLM benchmarks and datasets evaluate information extraction, numerical reasoning, question answering, forecasting, retrieval, and decision-oriented tasks over financial text, including FinQA, TAT-QA, PIXIU, FinBen, and open-book financial QA over public-company evidence in FinanceBench [4, 9, 32, 33, 37]. SECQUE, SEC-QA, and Fin-RATE focus more directly on SEC-filing analysis [1, 10, 13]. Work on LLM financial-statement analysis also compares model outputs with analyst forecasts using standardized financial statements [12]. Typical evaluation targets are answer correctness, retrieval, reasoning quality, or earnings-direction prediction. We evaluate model signals against realized abnormal returns, the professional analyst forecast distribution, and reported EPS.

Task Formulation

Let E\mathcal{E} be a set of filing events, where each event e=(i,d)e=(i,d) corresponds to firm ii and original filing date dd. The model observes an event-level input xex_e and returns a structured output that is parsed into a return-ranking score or an EPS forecast produced directly or as a revision to analyst consensus. Section 4.1.1 describes the filing samples and linked data sources used to construct xex_e.

Disclosure-to-Return Ranking

In the return-ranking task, let tet_e denote the effective event date assigned from the filing date and SEC acceptance time as described below. Let ri,tr_{i,t} denote firm ii’s daily stock return on trading day tt, and let rtmr_t^m denote the value-weighted market return on the same day. The primary target is the abnormal return on the effective event date,

ye,0=ri,te−rtem.(1)y_{e,0}=r_{i,t_e}-r_{t_e}^m. \tag*{(1)}

We refer to ye,0y_{e,0} as the effective-event-date abnormal return. We also define a following-trading-day companion target:

ye,1=ri,te+1−rte+1m.(2)y_{e,1}=r_{i,t_e+1}-r_{t_e+1}^m. \tag*{(2)}

Following prior studies of disclosure text and market outcomes [5, 15], we use market-adjusted abnormal returns rather than raw returns. This choice removes broad market movements from the outcome and makes the target closer to the firm-specific response associated with each filing event.

The model produces a score ses_e for each filing event. Its cross-sectional alignment with abnormal returns is evaluated contemporaneously using y0y_0 and for persistence using following-day y1y_1. For two comparable events ee and e′e' and target q∈{0,1}q \in\{0,1\}, the desired ordering is

se>se′whenye,q>ye′,q.(3)s_e > s_{e'} \quad\text{when} \quad y_{e,q} > y_{e',q}. \tag*{(3)}

Figure 2 shows that 10-Q filings can be accepted before the market opens, during regular trading hours, after the market closes, or without an observed acceptance time. Because the return targets use daily returns, the mapping from SEC filing time to the effective event date tet_e determines which daily return is assigned to each event. Let ded_e be the official filing date and let τe\tau_e be the SEC acceptance timestamp when observed. If ded_e is a trading day and τe\tau_e is observed before the regular market close, we set te=det_e = d_e; otherwise, we assign the event to the next trading day after ded_e. Thus, every filing is public before the close of its effective-event-date session, and after-close disclosures are not matched to a session that ended before release.

Annual counts of 10-Q filing events by SEC acceptance-time bucket

Figure 2. Annual counts of 10-Q filing events by SEC acceptance-time bucket.

Analyst Forecasting

For analyst forecasting, the model predicts an earnings update from the filing. We use I/B/E/S Detail History forecasts (Institutional Brokers’ Estimate System) for GPS earnings per share (EPS), where GPS is the I/B/E/S measure for generally accepted accounting principles (GAAP), or fully reported, EPS. For each event ee, we select the first quarterly GPS EPS target period whose fiscal-period end follows the filing firm’s fiscal-quarter end. Let Ae\mathcal{A}_e be the set of individual pre-filing analyst forecasts for the matched I/B/E/S ticker and target period, and let cec_e be their mean. The model observes the event-level input xex_e and produces either a direct GPS EPS forecast a^e\hat{a}_e or an adjustment Δe\Delta_e to the pre-filing consensus. For the consensus-anchored interface, the final forecast is

a^e=ce+Δe.(4)\hat{a}_e = c_e + \Delta_e . \tag*{(4)}

Evaluation compares a^e\hat{a}_e with the pre-filing analyst distribution and matched I/B/E/S reported GPS EPS.

Experiments

This section describes the data, model set, prompting inputs, and evaluation metrics used for the two tasks. We first define the 10-Q filing-event sample and linked financial data, then describe the evaluated LLMs, input settings, output formats, and outcome metrics.

Setup

Data

We construct filing-event data by aligning authoritative SEC disclosures [28] with financial databases accessed through Wharton Research Data Services [30]. The SEC 10-Q texts are publicly available and can be retrieved directly from EDGAR, while WRDS provides standardized access to licensed financial databases used for linking firm fundamentals, analyst expectations, and returns. The design goal is to evaluate filing understanding in a market-response setting: each event connects what a firm disclosed, structured firm fundamentals, pre-filing analyst expectations, and the stock-market response observed afterward. The main quarterly-report analysis uses a 10-Q return universe of 45,964 filing events from 2001–2024, built from Management’s Discussion and Analysis (MD&A) sections from SEC EDGAR and linked to Compustat firm fundamentals [27], I/B/E/S analyst expectations [21], and CRSP daily stock returns [3]. Market-adjusted abnormal returns use CRSP’s daily value-weighted market index return including distributions, a market-capitalization-weighted return over CRSP-covered U.S. equities, as the market return. The median 10-Q MD&A span contains 6,494 words.

Input Information Views

We construct controlled information views from three reusable blocks while holding the 10-Q filing events, response interface, and post-filing return targets fixed, so differences in return alignment reflect the information provided to the model. The disclosure block contains the 10-Q MD&A text and represents the narrative information released by management. The fundamentals block contains cross-sectional fundamental ranks computed among firms sharing the same original filing date. The ranks cover current-quarter Compustat variables and their year-over-year changes for revenue, net income, EPS, assets, equity, debt, cash, operating cash flow, net margin, leverage, and cash-to-assets. The block also reports cross-section size. This block tests whether LLMs can ground qualitative management language in the firm’s cross-sectional rank profile. The analyst-expectations block contains pre-filing I/B/E/S fields for the selected target period: estimate count, EPS consensus mean and median, dispersion, and recommendation statistics. It provides a professional anchor to available before the disclosure. Combining these blocks yields five evaluated views: disclosure only; fundamentals only; disclosure plus analyst expectations; disclosure plus fundamentals; and disclosure plus fundamentals plus analyst expectations.

Models

We evaluate 13 LLMs in two descriptive groups:

  • General-purpose LLMs: Qwen3-32B and Qwen3-8B [26]; Qwen2.5-Instruct at 32B, 14B, and 7B [25]; Llama-3.1-8B [7]; Mistral-NeMo-12B [22]; and the Llama-8B version of DeepSeek-R1-Distill [6].

  • Finance-specialized LLMs: Finо1-14B and Fin-o1-8B [24], Fin-R1-7B [17], Llama-Fin-8B [11], and ODA-Fin-RL-8B [2]. These finance-adapted LLMs are built from general backbones: Qwen2.5-14B-Instruct for Fino1-14B, Qwen2.5-7B-Instruct for Fin-R1-7B, Meta-Llama-3-8B-Instruct for Llama-Fin-8B, and Qwen3-8B for Fino1-8B and ODA-Fin-RL-8B.

We select this panel along three dimensions relevant to model choice in financial workflows: domain specialization, model family, and parameter scale. Qwen- and Llama-based models appear in both groups, enabling backbone- and scale-aligned comparisons of finance adaptation, while the broader coverage measures variation across model families and sizes from 7B to 32B.

Prediction Interfaces

Output format is part of the measurement design because LLM behavior is sensitive to answer format

[23, 36]. Return ranking uses se∈[−1,1]s_e \in[-1, 1], where larger values indicate a more positive expected abnormal return; numeric and verbal ordinal formats are evaluated only in the interface ablation. Figure 3 shows that continuous scoring provides the finest within-date resolution, averaging about 5.3 distinct scores per eligible effective event date. For GPS EPS forecasting, the direct interface outputs a^e\hat{a}_e, while the consensus-anchored interface outputs Δe\Delta_e and sets a^e=ce+Δe\hat{a}_e = c_e + \Delta_e.

Score granularity by output interface

Figure 3. Score granularity by response interface: mean distinct parsed scores on RankIC-eligible effective event dates. VkV_k and NkN_k denote kk-level verbal and numeric ordinal interfaces; Cont. denotes continuous scoring.

All interfaces use temperature 0 and require one prediction per event in the specified schema, with no additional prose. Filing text is capped at 12,000 words using head–tail truncation that retains the final 2,000 words. The parser reads JSON fields for return scores and consensus adjustments and a tagged answer block for direct forecasts; missing required fields invalidate an output. Table 3 summarizes the three interfaces.

Condensed prompt templates and output schemas for the three prediction interfaces.
Shared Prompt Structure
Role and task. The model acts as a financial analyst and produces one prediction from the supplied information for the specified return-ranking or next-quarter EPS task.
Conditional input blocks. Only the blocks belonging to the evaluated information view are supplied:
• [FUNDAMENTALS]: cross-sectional ranks of current-quarter variables and year-over-year changes in revenue, earnings, margins, cash flow, leverage, and liquidity;
• [ANALYST SUMMARY]: pre-filing I/B/E/S target period, estimate count, EPS consensus mean and median, forecast dispersion, and recommendation statistics; and
• [DISCLOSURE]: the filed 10-Q MD&A section.
Evidence rule. Use only the supplied blocks, return one final prediction, and tie the requested short evidence fields to those blocks.
Return-Score Interface
Task. Produce a bounded score for expected short-horizon post-filing abnormal return; higher values indicate a more positive relative return signal.
Decision rules. Start from a neutral prior of zero and adjust once when the evidence justifies it. Positive and negative values indicate correspondingly positive and negative abnormal-return signals. Keep the score conservative unless evidence is strong; do not give alternative scenarios or revise the final score.
Output (JSON only). { "score": number in [-1.0, 1.0], "confidence": number in [0.00, 1.00], "evidence": ["short evidence 1", "short evidence 2"] }
Direct GPS Forecast Interface
Task. Forecast next-quarter GPS (GAAP/fullly reported EPS) for the specified fiscal-period end, using the same scale as the I/B/E/S GPS forecast.
Decision rules. Use cross-sectional fundamental ranks as operating context and consensus as a pre-filing expectation anchor. Focus on guidance, demand, pricing, margins, costs, liquidity, and risks affecting next-quarter earnings, and depart from consensus when the MD&A supports a revision. Use the specified I/B/E/S units and return one numeric point forecast, not a trading score; the target date must match exactly.
Required answer block. <answer> MEASURE: GPS TARGET_PERIOD: YYYY-MM-DD FORECAST: float CONFIDENCE: float in [0.00, 1.00] EVIDENCE: — 2–4 concise evidence bullets </answer>
Consensus-Anchored GPS Interface
Task. Begin from the pre-filing analyst-mean GPS anchor and produce a single bounded adjustment. The evaluated point forecast is anchor + delta.
Anchor fields. The prompt supplies the target fiscal-period end and the analyst mean, median, and dispersion. The adjustment should remain close to zero unless the filing provides strong revision evidence.
Decision rules. Choose one delta around the analyst mean; when evidence is weak, keep it close to zero. Do not give alternative scenarios or output the final forecast directly, which is evaluated as anchor + delta.
Output (JSON only). { "delta": GPS EPS adjustment in [-3.0, 3.0], "confidence": number in [0.00, 1.00], "evidence": ["short evidence 1", "short evidence 2"] }

Table 3. Condensed prompt templates and output schemas for the three prediction interfaces.

Identity Masking and Context Matching

To distinguish identifier cues from event-level matching, we evaluate Qwen3-32B with masked DD, masked DD plus matched FF, and masked DD plus shuffled FF. Company names, tickers, dates, and four-digit years are replaced with typed placeholders; all other text and numbers remain. Shuffling pools events by effective event date and original filing date and deranges whole FF blocks across different firms, preserving the RankIC and fundamental-rank cross-sections and each assigned rank profile. Matched inputs are (Di,Fi)(D_i,F_i); shuffled inputs are (Di,Fπ(i))(D_i,F_{\pi(i)}) with yiy_i fixed, breaking the event-level DD–FF correspondence.

Classical and supervised baselines

For classical baselines, we compute Loughran–McDonald net tone and its same-firm prior-quarter change [5, 20]. Over 2013–2024, Ridge predictors are fitted on expanding histories of realized return labels and tested on subsequent years. Structured Ridge uses standardized, median-imputed fundamentals and analyst expectations; Text + Structured Ridge adds MD&A features. Qwen3-32B receives the corresponding full view without label fitting. Headline RankICs use each method’s eligible effective dates; paired LLM–Ridge contrasts use their shared dates.

Evaluation

Return ranking is evaluated within effective event dates; EPS forecasts are compared with pre-filing analyst forecasts and reported GPS EPS.

Return-ranking metric

For target q∈{0,1}q \in\{0,1\} and effective event date tt, let Et,q\mathcal{E}_{t,q} contain events with te=tt_e=t and valid ses_e and ye,qy_{e,q}. We retain dates with at least five events, two distinct scores, and two distinct returns; Dq\mathcal{D}_q denotes this date set.

The effective-date RankIC is the Spearman correlation between score and return ranks within an effective event date:

ρt,q=corr⁡(rank⁡e∈Et,q(se),rank⁡e∈Et,q(ye,q)),(5)\rho_{t,q}=\operatorname{corr}\left(\operatorname{rank}_{e\in\mathcal{E}_{t,q}}(s_e),\operatorname{rank}_{e\in\mathcal{E}_{t,q}}(y_{e,q})\right), \tag*{(5)}

We report its average across effective event dates,

RankIC⁡q=1∣Dq∣∑t∈Dqρt,q,(6)\operatorname{RankIC}_q=\frac{1}{|\mathcal{D}_q|}\sum_{t\in\mathcal{D}_q}\rho_{t,q}, \tag*{(6)}

and its information ratio,

RankICIR⁡q=RankIC⁡qsd⁡t∈Dq(ρt,q).(7)\operatorname{RankICIR}_q=\frac{\operatorname{RankIC}_q}{\operatorname{sd}_{t\in\mathcal{D}_q}(\rho_{t,q})}. \tag*{(7)}

RankIC is standard for cross-sectional return-ranking evaluation [35]. We complement it with a top-minus-bottom score-quintile return spread that measures return magnitude, following the portfolio-sort convention in Gu et al. [8]. Within each effective event date, we sort events into five equal-count score quintiles and compute

LS⁡q=1∣Dq∣∑t∈Dq(yˉt,q+−yˉt,q−),(8)\operatorname{LS}_q=\frac{1}{|\mathcal{D}_q|}\sum_{t\in\mathcal{D}_q}\left(\bar{y}_{t,q}^{+}-\bar{y}_{t,q}^{-}\right), \tag*{(8)}

where yˉt,q+\bar{y}_{t,q}^{+} and yˉt,q−\bar{y}_{t,q}^{-} are the highest- and lowest-score-quintile means. All return-ranking inference treats effective event dates, rather than individual filing events, as the statistical observations. One-sample tt-statistics use the effective-date RankIC series {ρt,q}\{\rho_{t,q}\}. Information-view contrasts average paired within-model RankIC differences across models within each effective event date. Model-group contrasts are the finance-specialized group mean minus the general-purpose group mean within each view and effective event date; reported contrast means and paired tt-statistics use these date-level differences.

Analyst metrics

For each forecast-evaluable event, e^e\hat{e}_e is the parsed model forecast and geg_e is the matched I/B/E/S-reported GPS EPS. We report mean absolute error ∣e^e−ge∣|\hat{e}_e-g_e|, position within the pre-filing analyst distribution, and analyst-relative beat rates.

To locate each model forecast among the contemporaneous analyst distribution, we compute an analyst-equivalent percentile:

pe=100⋅1∣Ae∣∑b∈Ae1{b≤e^e},(9)p_e=100\cdot\frac{1}{|\mathcal{A}_e|}\sum_{b\in\mathcal{A}_e}\mathbf{1}\{b\leq\hat{e}_e\}, \tag*{(9)}

Lower percentiles indicate more pessimistic forecasts. We also report whether e^e\hat{e}_e lies below, inside, or above the analyst range. Analyst-relative beat rates compare model absolute error with the same-event median, 25th-percentile, and minimum individual-analyst errors [12].

Results

We report three findings on information sets, model specialization, and analyst-relative forecasting.

Cross-Sectional Context Improves Return Alignment

Table 1 reports the main 10-Q return-ranking results across five controlled information views while holding the filing events, response interface, and return targets fixed. This design isolates whether disclosure text, cross-sectional fundamental ranks, and pre-filing analyst expectations contribute distinct return-ranking information.

Information ViewDisclosure onlyFundamentals onlyDisclosure + expectationsDisclosure + fundamentalsDisclosure + fundamentals + expectations
ModelICy0_{y_0}ICy1_{y_1}ICIRy0_{y_0}LS0_0ICy0_{y_0}ICy1_{y_1}ICIRy0_{y_0}LS0_0ICy0_{y_0}ICy1_{y_1}ICIRy0_{y_0}LS0_0ICy0_{y_0}ICy1_{y_1}ICIRy0_{y_0}LS0_0ICy0_{y_0}ICy1_{y_1}ICIRy0_{y_0}LS0_0
General-purpose LLMs
Qwen3-32B0.070680.026400.240430.007070.060190.030250.196260.006130.061240.032000.190770.006340.075550.039060.255820.007610.074470.034260.236780.00564
Qwen3-8B0.045130.044020.138850.005330.055070.027640.183120.003880.051980.039720.161000.006010.062790.057420.195730.007050.061460.051790.188900.00670
Qwen2.5-32B0.055570.028750.181140.005730.059140.029950.205320.005380.062040.026480.198420.006650.063840.032170.205960.006680.063310.035070.204530.00645
Qwen2.5-14B0.059350.020480.190280.006770.066180.036520.212190.003940.066040.023550.210160.006960.076680.034490.240740.007620.068590.031440.216080.00716
Qwen2.5-7B0.045870.016790.144510.005200.061320.035270.190630.006570.038040.016370.118830.004330.056070.027900.176120.006210.052260.021400.163550.00542
Llama-3.1-8B0.046240.007090.148760.004430.046970.024000.144560.004410.035420.014800.112540.004190.038260.013300.123730.003990.045330.019610.141970.00370
Mistral-NeMo-12B0.02539-0.005050.082510.003570.040440.016840.129980.002360.02031-0.007590.067320.000740.02476-0.006220.078000.001250.02538-0.004830.083440.00159
DeepSeek-R1-Distill-Llama-8B0.032990.010800.118200.002120.053410.032950.186010.005290.020900.002570.07588-0.001270.025620.011180.09041-0.001170.036790.012210.12766-0.00057
Finance-specialized LLMs
Fino1-14B0.056970.019380.179040.006330.061940.031810.197770.003200.061410.023260.198190.006330.067900.030160.211300.006510.065240.034390.205030.00648
Fin-R1-7B0.057130.021980.183600.005640.040980.022650.130370.001520.041270.022150.131310.004430.057510.029420.179590.006810.056150.034550.185210.00248
Llama-Fin-8B0.046700.005890.146420.005160.043240.004800.139630.002190.053680.018370.170480.005060.055870.01033
Fin-o1-8B0.054900.022110.170720.005800.063970.042760.211530.004610.059890.020560.187470.006830.070320.032630.222940.007170.070040.033890.222940.00684
ODA-Fin-RL-8B0.059010.023170.185350.006530.061380.034360.194000.004220.066000.026610.207190.007110.071530.038280.249660.007220.071990.032990.227250.00638

Table 1. Main 10-Q return-ranking results. IC is mean effective-date RankIC, and ICIR is its information ratio. y0y_0 is the effective-event-date abnormal return; y1y_1 is the following-trading-day abnormal return. Column maxima are blue (general-purpose) or green (finance-specialized); runners-up are underlined, and boldface marks the metric-wide maximum.

Both unstructured disclosure and cross-sectional fundamental ranks contain standalone signal. All 13 models produce positive y0y_0 RankIC from disclosure alone. The fundamentals-only view produces positive y0y_0 and y1y_1 RankIC for all 13 models. Qwen2.5-14B achieves the highest y0y_0 RankIC of 0.06618, whereas Fin-o1-8B achieves the highest y1y_1 RankIC of 0.04276.

The paired effective-date comparisons isolate the source of the incremental gain. Disclosure plus fundamentals improves y0y_0 RankIC for 10 of 13 models and y1y_1 RankIC for 12 of 13 models relative to disclosure alone. Adding cross-sectional fundamental ranks increases y0y_0 RankIC by 0.0089 (t=2.55t = 2.55, p=0.011p = 0.011) and y1y_1 RankIC by 0.0098 (t=2.84t = 2.84, p=0.005p = 0.005). Under this combined view, Qwen2.5-14B attains the highest y0y_0 RankIC of 0.07668, Qwen3-8B the highest y1y_1 RankIC of 0.05742, and Qwen3-32B the highest y0y_0 RankICIR of 0.25582. Cross-sectional fundamental ranks add return-ranking information beyond the filing text alone.

The gain is specific to the type of context rather than to adding more prompt content. Relative to disclosure alone, adding analyst expectations changes y0y_0 RankIC by only −0.0003-0.0003 (t=−0.11t = -0.11, p=0.914p = 0.914) and y1y_1 RankIC by 0.0003 (t=0.09t = 0.09, p=0.924p = 0.924). Adding expectations on top of disclosure and fundamentals changes y0y_0 RankIC by −0.0014-0.0014 (t=−0.61t = -0.61, p=0.542p = 0.542) and y1y_1 RankIC by −0.0050-0.0050 (t=−2.12t = -2.12, p=0.034p = 0.034). Across these comparisons, the reliable incremental gain comes from cross-sectional fundamental ranks. Analyst expectations do not improve average return alignment.

Masked and shuffled results. Table 2 reports the conditions defined in § 4.1.5 and includes the original disclosure-plus-fundamentals view as an unmasked reference.

Input configurationy0y_0 RankICy0y_0 tty1y_1 RankICy1y_1 tt
Original D + F0.075559.3190.039064.963
Masked D0.061637.8520.020732.583
Masked D + matched F0.071708.6180.027763.596
Masked D + shuffled F0.049356.1040.011471.453
Paired contrast
Matched F − shuffled F0.022352.9370.016292.446

Table 2. Qwen3-32B RankIC under original and identity-masked input configurations. D denotes disclosure and F denotes cross-sectional fundamental ranks. Input-configuration tt-statistics test mean effective-date RankIC against zero; the paired contrast reports matched-minus-shuffled Δ\DeltaRankIC and its tt-statistic.

Among the masked inputs, matched fundamentals produce the strongest RankIC: 0.07170 (t=8.618t = 8.618) on y0y_0 and 0.02776 (t=3.596t = 3.596) on y1y_1, versus 0.06163 and 0.02073 for disclosure alone and 0.04935 and 0.01147 with shuffled fundamentals. The matched-minus-shuffled Δ\DeltaIC is 0.02235 on y0y_0 (paired t=2.937t = 2.937) and 0.01629 on y1y_1 (paired t=2.446t = 2.446), making the difference statistically reliable at both horizons. Significant return alignment persists after company names, tickers, dates, and years are masked, indicating that the signal does not depend on recalling firm-specific outcomes from explicit identifiers. The matched-over-shuffled advantage further ties the incremental signal to the correct event-level correspondence between disclosure and fundamentals.

Baseline comparisons. Classical baselines test whether dictionary tone explains the signal. Loughran–McDonald net tone measures the positive-minus-negative word share, while net-tone change subtracts the same firm’s prior-quarter value. Their y0y_0 RankICs are 0.0192 and 0.0263, respectively, well below the LLM result.

The supervised comparison is stricter because the Ridge models learn from realized return labels over expanding historical windows.

On the common 2013–2024 test period, Qwen3-32B reaches a y0y_0 RankIC of 0.1009, versus 0.0714 for Structured Ridge and 0.0735 for Text + Structured Ridge. On shared effective dates, its paired advantages are 0.0340 over Structured Ridge (t=3.13t = 3.13, p=0.002p = 0.002) and 0.0319 over Text + Structured Ridge (p=0.003p = 0.003). The prompted, zero-shot LLM signal therefore outperforms both label-trained Ridge variants on the primary effective-event-date target.

The comparison also concerns the output available to users. In our implementation, Ridge yields a scalar prediction, whereas Qwen3-32B returns each score with short, event-specific textual evidence referring to the supplied filing and structured context. Human analysts can compare that evidence with the source information and assess whether the stated rationale is economically plausible. The resulting interface couples the return-ranking score with evidence for human review.

Finance-Specialized LLMs Do Not Reliably Dominate General-Purpose Models

Group-level effects depend on the return horizon. Paired effective-date contrasts compare finance-specialized and general-purpose models under matched information views, with positive Δ\DeltaIC favoring the finance-specialized group. Across the four disclosure-containing views, this group leads on y0y_0 by 0.0062 under disclosure only (t=2.08t = 2.08, p=0.038p = 0.038), 0.0116 under disclosure plus expectations (t=3.81t = 3.81, p<0.001p < 0.001), 0.0129 under disclosure plus fundamentals (t=4.26t = 4.26, p<0.001p < 0.001), and 0.0096 under the full-information view (t=3.21t = 3.21, p=0.001p = 0.001). On y1y_1, the corresponding differences are −0.0001 (t=−0.04t = -0.04, p=0.970p = 0.970), 0.0041 (t=1.32t = 1.32, p=0.188p = 0.188), 0.0021 (t=0.70t = 0.70, p=0.483p = 0.483), and 0.0039 (t=1.30t = 1.30, p=0.195p = 0.195); none is statistically reliable. Financial specialization therefore provides a group advantage on y0y_0, but not on y1y_1.

Backbone- and family-aligned gains remain model dependent. Fino1-14B trails Qwen2.5-14B in all five y0y_0 views and exceeds it in only one of five y1y_1 views. Fin-o1-8B and ODA-Fin-RL-8B exceed Qwen3-8B in all five y0y_0 views but trail it in four of five y1y_1 views. Fin-R1-7B exceeds Qwen2.5-7B in four of five views at both horizons; Llama-Fin-8B exceeds Llama-3.1-8B in four of five y0y_0 views but only one y1y_1 view. General-purpose Qwen models also attain every Table 1 maximum: y0y_0 RankIC of 0.07668, y1y_1 RankIC of 0.05742, y0y_0 RankICIR of 0.25582, and LS0LS_0 of 0.00762.

Scale and horizon effects remain mixed. Across the five views, Qwen3-32B beats Qwen3-8B on y0y_0 but trails it in four y1y_1 views; Qwen2.5-14B beats Qwen2.5-32B on y0y_0. As a longer-horizon robustness check under the full-information view, we compute RankIC using cumulative abnormal returns over trading days te+1t_e + 1 through te+5t_e + 5. The five leading models are Qwen3-32B, ODA-Fin-RL-8B, Fin-o1-8B, Qwen2.5-7B, and Fino1-14B, with RankICs of 0.03561, 0.03064, 0.02782, 0.02471, and 0.02237, respectively. Neither specialization nor scale dominates across horizons.

LLM Forecasts Are Consensus-Centered but Not Consistently Analyst-Beating

Figure 4 compares alignment with the pre-filing analyst distribution and accuracy against reported GPS EPS.

Analyst-relative forecast position and reported-EPS accuracy under direct and consensus-anchored interfaces

Figure 4. Analyst-relative forecast position and reported-EPS accuracy under direct and consensus-anchored interfaces. Left: shares beating the median, top-quartile, and best individual analyst; center: analyst-equivalent percentile; right: shares below, inside, and above the pre-filing analyst range.

Direct forecasts cluster near the analyst center. Under the full view, the analyst-equivalent percentile averages 43.4 across models and ranges from 36.6 to 49.3, placing direct forecasts near but slightly below the analyst median. Yet only 41.1% fall inside the analyst range, versus 35.0% below and 23.9% above. Mean percentile measures aggregate directional centering, whereas inside-range coverage measures event-level agreement. Direct forecasts exhibit the former but not consistently the latter.

Consensus anchoring improves distributional alignment but worsens reported-EPS error. Anchoring raises the inside-range share from 41.1% to 59.5%, or 18.432 percentage points (t=6.08t = 6.08, $p < 0.001), and the mean analyst-equivalent percentile from 43.4 to 54.3, or 10.886 points (t=3.04t = 3.04, p=0.010p = 0.010). Model-level percentiles nevertheless widen from 36.6–49.3 to 29.5–75.3, and mean absolute error against reported EPS rises by 10.798 (t=9.41t = 9.41, p<0.001p < 0.001). The wider percentile range shows that a common anchor does not homogenize model behavior; models differ in how they translate filing evidence into Δe\Delta e. Combined with the higher error, the increase in inside-range coverage reveals a shrinkage trade-off. Forecasts become more analyst-like at the cost of realized-earnings accuracy.

LLMs do not consistently beat individual analysts. The strongest direct models beat the median individual analyst on roughly 40–43% of events with reported EPS and the best analyst on roughly 22–25%. With anchoring, only Llama-Fin-8B exceeds a 50% beat-median rate, at 52.5%. Across models, anchoring raises the beat-median rate by only 1.163 percentage points (t=0.64t = 0.64, p=0.535p = 0.535) and lowers the beat-best rate by 7.390 points (t=−6.16t = -6.16, p<0.001p < 0.001). LLM-generated EPS forecasts resemble professional forecasts without consistently outperforming them.

Conclusion

We evaluate 13 prompted, zero-shot LLMs across controlled information views and prediction interfaces, linking long 10-Q MD&A to abnormal returns, analyst forecasts, and reported EPS. Adding cross-sectional fundamental ranks to disclosure yields statistically significant RankIC gains for effective-event-date y0y_0 and following-day y1y_1. Matched-versus-shuffled differences are also significant at both horizons, tying the gains to correct event-level context; analyst summaries provide no comparable improvement. Finance-specialized models show a group advantage on y0y_0 but no reliable advantage on y1y_1, while general-purpose Qwen models lead individually and scale effects are non-monotonic. LLM forecasts remain near consensus without consistently beating individual analysts; anchoring improves distributional alignment but worsens reported-EPS accuracy. Information-set construction, return horizon, model choice, and prediction interface therefore remain central to financial LLM evaluation.

References

References

  1. [1]Noga BenYoash, Menachem Brief, Oded Ovadia, Gil Shenderovitz, Moshik Mishaeli, Rachel Remberg, and Eitam Sheetrit. 2025. SEQUE: A Benchmark for Evaluating Real-World Financial Analysis Capabilities. In Proceedings of the Fourth Workshop on Generation, Evaluation and Metrics. Association for Computational Linguistics, Vienna, Austria and virtual meeting, 212–230. https://aclanthology.org/2025.gem-1.16/
  2. [2]Chuxue Cao, Honglin Lin, Zhanping Zhong, Xin Gao, Mengzhang Cai, Conghui He, Sirui Han, and Lijun Wu. 2026. Unlocking Data Value in Finance: A Study on Distillation and Difficulty-Aware Training. arXiv:2603.07223. https://arxiv.org/abs/2603.07223
  3. [3]Center for Research in Security Prices. 2026. CRSP U.S. Stock Databases. https://www.crsp.org/research/crsp-us-stock-databases/. Accessed through WRDS, April 2026.
  4. [4]Zhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting-Hao Huang, Bryan Routledge, and William Yang Wang. 2021. FinQA: A Dataset of Numerical Reasoning over Financial Data. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Online and Punta Cana, Dominican Republic, 3697–3711. doi:10.18653/v1/2021.emnlp-main.300DOI
  5. [5]Lauren Cohen, Christopher J. Malloy, and Quoc Nguyen. 2020. Lazy Prices. The Journal of Finance 75, 3 (2020), 1371–1415. doi:10.1111/jofi.12885DOI
  6. [6]DeepSeek-AI. 2025. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv:2501.12948. https://arxiv.org/abs/2501.12948
  7. [7]Aaron Graffatari, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. 2024. The Llama 3 Herd of Models. arXiv:2407.21783. https://arxiv.org/abs/2407.21783
  8. [8]Shihua Gu, Bryan Kelly, and Dacheng Xiu. 2020. Empirical Asset Pricing via Machine Learning. The Review of Financial Studies 33, 5 (2020), 2223–2273. doi:10.1093/rfs/hhaa009DOI
  9. [9]Pranab Islam, Anand Kannappan, Douwe Kiela, Rebecca Qian, Nino Scherrer, and Bertie Vidgen. 2023. FinanceBench: A New Benchmark for Financial Question Answering. arXiv:2311.11944. doi:10.48550/arXiv.2311.11944DOI
  10. [10]Yidong Jiang, Junrong Chen, Eftycha Makri, Jialin Chen, Peifeng Li, Ali Maatouk, Leandros Tassis, Eliot Brenner, Bing Xiang, and Rex Ying. 2026. Fin-RATE: A Real-world Financial Analytics and Tracking Evaluation Benchmark for LLMs on SEC Filings. arXiv:2602.07294. https://arxiv.org/abs/2602.07294
  11. [11]Zixuan Ke, Yifei Ming, Xuan-Ping Nguyen, Caiming Xiong, and Shafiq Joty. 2025. Demystifying Domain-adaptive Post-training for Financial LLMs. arXiv:2501.04961. https://arxiv.org/abs/2501.04961
  12. [12]Alex G. Kim, Maximilian Muhn, and Valeri V. Nikolaev. 2024. Financial Statement Analysis with Large Language Models. SSRN working paper; arXiv:2407.17866. doi:10.2139/ssrn.4835311DOI
  13. [13]Viet Lai, Michael Krundick, Charles Lovering, Varshini Reddy, Craig Schmidt, and Chris Tanner. 2025. SEC-QA: A Systematic Evaluation Corpus for Financial QA. In Proceedings of the 10th Workshop on Financial Technology and Natural Language Processing. Association for Computational Linguistics, Suzhou, China, 221–236. doi:10.18653/v1/2025.finnlp-2.15DOI
  14. [14]Jonathan Lewellen. 2015. The Cross-Section of Expected Stock Returns. Critical Finance Review 4, 1 (2015), 1–44. doi:10.1561/104.00000024DOI
  15. [15]Feng Li. 2010. The Information Content of Forward-Looking Statements in Corporate Filings—A Naive Bayesian Machine Learning Approach. Journal of Accounting Research 48, 5 (2010), 1049–1102. doi:10.1111/j.1475-679X.2010.00382.xDOI
  16. [16]Haohang Li, Yupeng Cao, Yangyang Yu, Shashidhar Reddy Javvaji, Zhiyang Deng, Yueru He, Yuechen Jiang, Zining Zhu, K.P. Subbalakshmi, Jimin Huang, Lingfei Qian, Xueqing Peng, Jordan W. Suchow, and Qianqian Xie. 2025. INVESTOR-BENCH: A Benchmark for Financial Decision-Making Tasks with LLM-based Agent. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics, Volume 1: Long Papers. Association for Computational Linguistics, Vienna, Austria, 2509–2525. doi:10.18653/v1/2025.acl-long.126DOI
  17. [17]Zhaowei Liu, Xin Guo, Zhi Yang, Fangqi Lou, Lingfeng Zeng, Jinyi Niu, Mengping Li, Qi Qi, Zhiqiang Liu, Yiyang Han, et al. 2025. Fin-R: A Large Language Model for Financial Reasoning through Reinforcement Learning. arXiv:2503.16252. https://arxiv.org/abs/2503.16252
  18. [18]Zhuang Liu, Degen Huang, Kaiyu Huang, Zhuang Li, and Jun Zhao. 2020. FinBERT: A Pre-trained Financial Language Representation Model for Financial Text Mining. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence. International Joint Conferences on Artificial Intelligence Organization, 4513–4519. doi:10.24963/ijcai.2020/622DOI
  19. [19]Alejandro Lopez-Lira and Yuehua Tang. 2023. Can ChatGPT Forecast Stock Price Movements? Return Predictability and Large Language Models. SSRN Electronic journal (2023). doi:10.2139/ssrn.4412788
  20. [20]Tim Loughran and Bill McDonald. 2016. Textual Analysis in Accounting and Finance: A Survey. Journal of Accounting Research 54, 4 (2016), 1187–1230. doi:10.1111/1475-679X.12123DOI
  21. [21]LSEG Data & Analytics. 2026. L/B/E/S Estimates. https://www.lseg.com/en/data-analytics/financial-data/company-data/ibes-estimates. Accessed through WRDS, April 2026.
  22. [22]Mistral AI. 2024. Mistral NeMo 12B Model Card. https://docs.mistral.ai/models/model-cards/mistral-nemo-12b-24-07. Accessed 2026-05-14.
  23. [23]Pouya Pezeshkpour and Estevam Hruschka. 2024. Large Language Models Sensitivity to the Order of Options in Multiple-Choice Questions. In Findings of the Association for Computational Linguistics: NAACL 2024. Association for Computational Linguistics, Mexico City, Mexico, 2006–2017. doi:10.18653/v1/2024.findings-naacl.130DOI
  24. [24]Lingfei Qian, Weipeng Zhou, Yan Wang, Xueqing Peng, Han Yi, Yilun Zhao, Jimin Huang, Qianqian Xie, and Jian-yun Nie. 2025. Fin101: On the Transferability of Reasoning-Enhanced LLMs and Reinforcement Learning to Finance. arXiv:2502.08127. https://arxiv.org/abs/2502.08127
  25. [25]Qwen Team. 2024. Qwen2.5 Technical Report. arXiv:2412.15115. https://arxiv.org/abs/2412.15115
  26. [26]Qwen Team. 2025. Qwen3 Technical Report. arXiv:2505.09388. https://arxiv.org/abs/2505.09388
  27. [27]S&P Global Market Intelligence. 2026. Compustat North America Fundamentals. https://www.spglobal.com/market-intelligence/en/solutions/products/compustat. Accessed through WRDS, April 2026.
  28. [28]U.S. Securities and Exchange Commission. 2026. EDGAR: Electronic Data Gathering, Analysis, and Retrieval System. https://www.sec.gov/edgar. Accessed April 2026.
  29. [29]Shengkun Wang, Taoran Ji, Linhan Wang, Yanshen Sun, Shang-Ching Liu, Amit Kumar, and Chang-Tien Lu. 2024. StockTime: A Time Series Specialized Large Language Model Architecture for Stock Price Prediction. arXiv preprint arXiv:2409.08281 (2024). doi:10.48550/arXiv.2409.08281DOI
  30. [30]Wharton Research Data Services. 2026. Wharton Research Data Services. https://wrds-www.wharton.upenn.edu/. Accessed April 2026.
  31. [31]Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhanjan Kambadur, David Rosenberg, and Gideon Mann. 2023. BloombergGPT: A Large Language Model for Finance. arXiv preprint arXiv:2303.17564 (2023). doi:10.48550/arXiv.2303.17564DOI
  32. [32]Qianqian Xie, Weiguang Han, Zhengyu Han, Ruoyu Xiao, Xiaodan Zhang, Yueru He, Mengxi Xiao, Dong Li, Yongfu Dai, Duanuyu Feng, Yijing Xu, Haoqiang Kang, Ziyan Zhuang, Chenhan Yuan, Kailai Zhang, Zheheng Luo, Tianlin Zhang, Zhiwei Liu, Guojun Xiong, Zhiyang Deng, Yuechen Jiang, Zhiyuan Yao, Haohang Li, Yangyang Yu, Gang Hu, Jiajia Huang, Xiao-Yang Liu, Alejandro Lopez-Lira, Benyou Wang, Yanzhao Lai, Hao Wang, Min Peng, Sophia Ananiadou, and Jimin Huang. 2024. FinBen: A Holistic Financial Benchmark for Large Language Models. In Advances in Neural Information Processing Systems 37: Datasets and Benchmarks Track. https://arxiv.org/abs/2402.12659
  33. [33]Qianqian Xie, Weiguang Han, Zhaoyue Han, Lai Min Peng, Alejandro Lopez-Lira, and Jimin Huang. 2023. PIXIU: A Comprehensive Benchmark, Instruction Dataset and Large Language Model for Finance. In Advances in Neural Information Processing Systems 36: Datasets and Benchmarks Track. https://arxiv.org/abs/2306.05443
  34. [34]Hongyang Yang, Xiao-Yang Liu, and Christina Dan Wang. 2023. FinGPT: Open-Source Financial Large Language Models. FinLLM Workshop at IJCAI 2023; arXiv:2306.06031. doi:10.48550/arXiv.2306.06031arxiv.org/abs/2306.06031
  35. [35]Xiao Yang, Weiqing Liu, Dong Zhou, Jian Zhang, and Tie-Yan Liu. 2020. Qlib: An AI-oriented Quantitative Investment Platform. arXiv preprint arXiv:2009.11189 (2020).
  36. [36]Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021. Calibrate Before Use: Improving Few-Shot Performance of Language Models. In Proceedings of the 38th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 139). PMLR, 12697–12706. https://proceedings.mlr.press/v139/zhao21c.htmlarxiv.org/abs/2102.09690
  37. [37]Fengbin Zhu, Wenqiang Lei, Youcheng Huang, Chao Wang, Shuo Zhang, Jiancheng Lv, Fuli Feng, and Tat-Seng Chua. 2021. TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics, Volume 1: Long Papers. Association for Computational Linguistics, Online, 3277–3287. doi:10.18653/v1/2021.acl-long.254DOI

Paper details

vlnc:W_yvan4wys
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents