betaFeedback

WorksMathematics

Conditional Asset Pricing Transformer

Abstract

In this paper, we propose a conditional asset pricing transformer (CAP-T) that captures nonlinear interactions among firms through attention to forecast excess returns. In the first stage, a transformer estimates conditional expected returns, and kernel-weighted principal component analysis of its residuals estimates the local factor loading space. Projecting the conditional mean onto this space separates factor risk compensation from conditional mispricing. Then, in the second stage, separate transformers learn these two components, whose predictions are combined to forecast returns. As a result, the proposed framework decomposes returns into conditional mispricing, factor risk compensation, a centered factor component, and an unpredictable innovation. We establish approximation and generalization bounds and characterize how first-stage prediction errors propagate through local factor estimation and component learning, providing statistical guaranties for conditional mispricing, factor loadings, and factor risk premia. Finally, simulations and empirical studies demonstrate the predictive and interpretive benefits of the proposed method.

Introduction

Understanding differences in expected returns is a central task of asset pricing. It informs the allocation of capital, the measurement of investment performance, and the assessment of compensation for risk. The foundations laid by Sharpe [39] and Ross [36] connect expected returns to systematic exposures, placing the relationship between an individual asset and the broader market at the center of the pricing problem. As increasingly rich information about firms becomes available, a central challenge is to determine how that information enters expected returns and what it reveals about their economic sources.

Factor models provide a parsimonious framework for addressing this challenge. They summarize common variation in a large collection of returns through a small number of factors, making the estimation of systematic exposures feasible in large panels [1]. Their usefulness for pricing depends on both the information retained by the estimated factors and the evolution of the loading space. Lettau and Pelger [31] show that factors important for expected returns can be difficult to recover from return variation alone and incorporate pricing information into factor estimation. Su and Wang (2017) develop local PCA for models with smoothly varying loadings, while Kong [27] and Kong (2018) independently introduce high-frequency local PCA with four types of general varying loadings. Robustness to heavy tails has also motivated Huber-loss-based estimators for vector and tensor factor models [2] (He et al. 2025). These developments illustrate how factor estimation can accommodate features of financial data while preserving an interpretable representation of common risk.

With the inclusion of observable firm characteristics, we can use conditional factor models to connect these characteristics to systematic risk exposures. Remarkably, Kelly et al. [25] develop a novel instrumented principal component analysis (IPCA) framework, allowing characteristics to determine factor loadings through a linear mapping. To deal with non-linearity, Su et al. (2025) develop sieve estimation for state-varying factor models, allowing loadings to be unknown individual-specific functions of observed state variables. Recently, deep learning has opened new possibilities for capturing richer non-linear relationships between characteristics and returns. For example, Gu et al. (2020) document the predictive value of nonlinear methods for equity returns, while Gu et al. (2021) use conditional autoencoders to model nonlinear dependence of factor exposures on characteristics. A notable advance in this direction is the structural framework of Fan et al. [13], which brings flexible prediction and economic interpretation together. Their DNN-varying approach combines neural network estimation at each date with local PCA to separate conditional mispricing from risk compensation and subsequently learn predictive functions for both components. This framework gives nonlinear return prediction a clear economic structure, distinguishing the sources of expected returns while allowing each component to be learned separately.

However, the characteristic mappings in these pricing models rely primarily on each firm’s own information. This leaves an important dimension of the pricing problem less fully explored: firms operate within a shared economic environment and are connected through production, competition, and financing. The information contained in a firm’s profitability or investment may therefore become more informative when viewed alongside the characteristics of other firms. Such connections have observable pricing implications, for instance, Cohen and Frazzini [9] document return predictability along customer and supplier relationships, highlighting the relevance of information originating outside the firm itself. To exploit structured cross-information in high-dimensional data, Wang et al. [44] develop matrix factor models that preserve the row and column structure of matrix-valued time series. Then, projection-based methods further improve loading-space estimation in matrix and tensor factor models (Yu et al., 2022; Han et al., [18]; Barigozzi et al., [2]). Related work develops CP-decomposition-based representations for matrix time series (Chang et al., [4]) and tests for the presence and dimensions of row and column factor spaces (He et al., 2023). Meanwhile, Chen et al. [5] estimate time-varying Granger-causality and partial-correlation networks for high-dimensional time series, with extensions that account for common factors. Together, these studies motivate pricing models that use firm characteristics jointly and allow the relevance of other firms’ information to vary with economic conditions.

Transformers offer another conceptually natural way to model these cross-firm relationships. Through attention, they combine information across firms using weights that depend on the observed characteristics (Vaswani et al., [43]; Lee et al., [30]). In asset pricing, Kelly et al. [26] incorporate attention into stochastic discount factor estimation to share information across assets. In parallel, our focus is on conditional expected returns and on how information across firms contributes differently to risk compensation and conditional mispricing. To the best of our knowledge, this is the first transformer framework for conditional asset pricing that jointly attends to a broad cross section of assets to learn risk compensation and conditional mispricing separately. Indeed, predicting expected returns alone does not distinguish compensation for systematic risk from conditional mispricing, and the distinction between these components is central to our approach. Note that the relationships among firms that inform risk compensation may differ from those that inform conditional mispricing, and learning only their sum leaves these differences implicit. Allowing each component to draw on different aspects of the characteristic information serves an interpretive purpose as well as an predictive one.

Finally, establishing statistical guaranties for non-linear asset pricing models often requires connecting flexible function approximation with estimation under dependence. Econometric work on sieve estimation and subsequent estimation stages addresses weak dependence and the effects of estimated functions (Chen and Shen, [8]; Chen and Liao, [7]), whereas Wang [45] studies inference in factor-augmented regressions and forecasts when latent factors are estimated from nonlinear or mixed data. For neural networks, Schmidt-Hieber [37] derive regression rates that exploit compositional structure. Then, Yun et al. [48] establish universal approximation results for transformers, and Havrilla and Liao [19] develop approximation and statistical learning theory on manifolds of low intrinsic dimension. Finally, in the context of conditional asset pricing, Fan et al. [13] establish statistical guaranties linking neural network estimation to factor decomposition.

In this work, we develop a Conditional Asset Pricing Transformer, denoted CAP-T, that integrates these features within a joint estimation framework. The paper makes four contributions.

First, we broaden the information used in conditional asset pricing from individual firm characteristics to the joint characteristics of the market cross section. Compared with the firm-specific pricing functions in Fan et al. [13], CAP-T allows other firms’ characteristics to inform an individual firm’s expected return. Attention learns these relationships from the observed characteristics without requiring a prespecified network of economic links. This formulation accommodates relationships that change with market conditions and allows different firms to provide relevant information for different pricing components.

Second, inspired by the integration of economic structure and nonlinear learning in Gu et al. (2021) and Fan et al. [13], we use an identified factor decomposition to organize learning from information across firms. Predicting expected returns alone leaves the sources of predictability unresolved: the same characteristic may convey information about risk compensation, conditional mispricing, or both. Our decomposition provides separate targets for these economic components, allowing their networks to learn different relationships among firms before combining their predictions. The value of this structure is therefore twofold: it gives the predictions an economic interpretation and permits each component to use the joint information according to its own predictive relationships.

Third, we establish approximation and statistical error bounds for this estimation procedure. Building on the manifold approximation analysis of Havrilla and Liao [19], we develop a construction for pricing functions with vector outputs whose approximation complexity is governed by the intrinsic dimension of the joint characteristic state. Our statistical analysis allows geometric β\beta-mixing and changing marginal state distributions. It also addresses the dependence between successive estimation stages, a central concern in Chen and Liao [7], by tracing errors from the initial regression through local PCA and component learning. For fixed dimensions, the initial regression achieves a squared L2L^2 error rate of T−2β/(2β+d)T^{-2\beta/(2\beta+d)} up to logarithmic factors, where β\beta is smoothness and dd is the intrinsic dimension. The component estimators and their sum inherit this rate with an additional loading estimation error term.

Fourth, we document predictive gains alongside economic structure in the learned representations. Complementing the evidence on nonlinear return prediction in Gu et al. (2020), our simulations examine recovery of the underlying pricing components, while the equity application shows improved return prediction relative to IPCA and DNN-varying and useful portfolio signals after transaction costs. With five factors and semiannual updating, the three layer CAP-T (ReLU) mispricing portfolio achieves a net annualized Sharpe ratio of 0.85, compared with at most 0.69 for DNN-varying and 0.67 for IPCA. Interestingly, the attention representations are able to cluster firms along distinct economic dimensions: risk representations emphasize volatility and trading conditions, whereas alpha representations distinguish profitability and financing constraints. Moreover, selected firm associations also find support in corporate disclosures and characteristic similarity. The analysis on characteristic importance further provides complementary evidence, with risk prediction emphasizing market information and alpha placing greater weight on valuation and investment. These findings connect the model’s predictive performance to interpretable differences in the information used for risk compensation and conditional mispricing.

The remainder of the paper is organized as follows. § 2 presents the model and estimation procedure. § 3 develops the theoretical results. § 4.1 reports the simulation evidence, and § 4.2 presents the empirical analysis. § 5 gives further discussion.

The Model and Methodology

In this section, we develop CAP-T within a conditional factor framework that separates expected excess returns into risk compensation and conditional mispricing. We first specify the model and its identifying restrictions. Then we introduce the transformer pricing functions that incorporate information across firms. In the end, we present an estimation procedure that uses local factor estimation to construct the targets for learning the two pricing components.

Conditional Factor Model

Inspired by Fan et al. [13], we consider a conditional factor model that separates risk compensation from conditional mispricing. Let yt∈RN\mathbf{y}_t \in\mathbb{R}^{N} denote excess returns for NN firms at date tt, and let Xt−1∈RN×P\mathbf{X}_{t-1} \in\mathbb{R}^{N \times P} collect their PP characteristics observed before these returns are realized. We specify

yt=gα(Xt−1)+β(Xt−1)λ(Xt−1)+β(Xt−1)ζt+ut,(1)\mathbf{y}_t = g_{\alpha}(\mathbf{X}_{t-1}) + \boldsymbol{\beta}(\mathbf{X}_{t-1})\boldsymbol{\lambda}(\mathbf{X}_{t-1}) + \boldsymbol{\beta}(\mathbf{X}_{t-1})\boldsymbol{\zeta}_t + \mathbf{u}_t, \tag*{(1)}

where gα(Xt−1)∈RNg_{\alpha}(\mathbf{X}_{t-1}) \in\mathbb{R}^{N} denotes conditional mispricing, β(Xt−1)∈RN×K\boldsymbol{\beta}(\mathbf{X}_{t-1}) \in\mathbb{R}^{N \times K} contains exposures to KK latent factors, and λ(Xt−1)∈RK\boldsymbol{\lambda}(\mathbf{X}_{t-1}) \in\mathbb{R}^{K} denotes factor risk premia. The vectors ζt∈RK\boldsymbol{\zeta}_t \in\mathbb{R}^{K} and ut∈RN\mathbf{u}_t \in\mathbb{R}^{N} are factor innovations and idiosyncratic shocks, respectively. We assume that the two innovation sequences are independent and identically distributed over time, mutually independent, and centered. Current innovations are independent of past information, implying

E(ζt∣Xt−1)=0,E(ut∣Xt−1)=0.(2)\mathbb{E}(\zeta_t \mid X_{t-1}) = 0,\qquad\mathbb{E}(u_t \mid X_{t-1}) = 0. \tag*{(2)}

Define risk compensation as grisk(X):=β(X)λ(X)g_{\mathrm{risk}}(X) := \beta(X)\lambda(X). Taking conditional expectations in (1) under (2) decomposes expected excess returns into the sum of two components:

f(Xt−1):=E(yt∣Xt−1)=gα(Xt−1)+grisk(Xt−1).(3)f(X_{t-1}) := \mathbb{E}(y_t \mid X_{t-1}) = g_{\alpha}(X_{t-1}) + g_{\mathrm{risk}}(X_{t-1}). \tag*{(3)}

Equation (3) expresses expected excess returns as the sum of conditional mispricing and risk compensation. Our statistical objective is to recover and separately learn these two components. To identify the two components, we assume that the loading matrix has full column rank and impose

β(X)Tgα(X)=0.(4)\beta(X)^{\mathsf{T}}g_{\alpha}(X) = 0. \tag*{(4)}

Under (4), the decomposition in (3) identifies risk compensation as the projection onto the loading space and mispricing as the orthogonal remainder. Factor risk premia are the corresponding coefficients in the chosen loading coordinates.

Following the structural estimation strategy of Fan et al. [13], we proceed in three steps. First, we estimate the conditional mean ff. Second, we apply local PCA to the resulting residuals and use the estimated loading space to construct targets for the two pricing components. Third, we learn each component separately and combine their predictions to forecast excess returns.

Transformer Pricing Functions

The proposed conditional asset pricing transformer maps the joint characteristic state X∈RN×PX \in\mathbb{R}^{N \times P} to an NN-dimensional vector of conditional return components. That is, we seek a compact representation of firm characteristics, using attention to combine predictive information across firms. Attention allows the pricing functions to incorporate interactions among firms ([43, 30, 26]). For a single attention head, let WQ,WK,WV∈RP×d′W_Q, W_K, W_V \in\mathbb{R}^{P \times d'} denote the learned query, key, and value weight matrices. The attention operation is

S(X)=σ((XWQ)(XWK)Td′),A(X)=S(X)XWV,(5)S(X) = \sigma\left(\frac{(XW_Q)(XW_K)^{\mathsf{T}}}{\sqrt{d'}}\right),\qquad A(X) = S(X)XW_V, \tag*{(5)}

where σ\sigma denotes the attention activation, S(X)∈RN×NS(X) \in\mathbb{R}^{N \times N}, and A(X)∈RN×d′A(X) \in\mathbb{R}^{N \times d'}. The matrix (XWQ)(XWK)T(XW_Q)(XW_K)^{\mathsf{T}} collects cross products of the projected firm characteristics. Its (i,j)(i,j)th entry measures the interaction between the query representation of firm ii and the key representation of firm jj. After activation, Sij(X)S_{ij}(X) determines the contribution of firm jj to the updated representation of firm ii. The resulting pricing functions can therefore use characteristics throughout the cross section, with the strength of these interactions varying with the observed state. Quite often, multiple heads based on (5) capture different patterns of interaction. Their outputs are concatenated and projected to form

H(X)=concat⁡(A1(X),…,Am(X))WO,WO∈Rmd′×d′H(X) = \operatorname{concat}(\mathcal{A}_1(X), \ldots, \mathcal{A}_m(X))W_O,\quad W_O \in\mathbb{R}^{md'\times d'}

where each head uses separate weight matrices. A shared feedforward transformation is applied to each firm’s representation. Writing this representation as a column vector h∈Rd′h \in\mathbb{R}^{d'}, we have

FFN⁡(h)=WLFFNσ(WLFFN−1⋯σ(W1h+b1)⋯+bLFFN−1)+bLFFN,\operatorname{FFN}(h) = W_{L_{\mathrm{FFN}}}\sigma\left(W_{L_{\mathrm{FFN}}-1}\cdots\sigma(W_1h+b_1)\cdots+b_{L_{\mathrm{FFN}}-1}\right)+b_{L_{\mathrm{FFN}}},

where LFFNL_{\mathrm{FFN}} denotes the feedforward depth and the weight matrices and biases have compatible dimensions. Attention and feedforward transformations are combined through residual connections and stacked across layers. An output map then produces the NN conditional return components.

Let F\mathcal{F} denote the class of transformer maps Fθ:RN×P→RNF_\theta:\mathbb{R}^{N\times P}\to\mathbb{R}^{N} over which the pricing functions are estimated, where θ\theta varies across its members2. The proposed CAP-T represents ff, gαg_\alpha, and griskg_{\mathrm{risk}} using a common attention architecture with separate parameters. Each network draws on the joint characteristic matrix, allowing the relationships among firms that inform total returns, mispricing, and risk compensation to differ. The component predictions are then combined into the final return forecast. The following algorithm summarizes the estimation procedure.

The Estimation Algorithm

At a forecast origin TT, the rolling training sample consists of {(Xt−1,yt):t=1,…,T}\{(X_{t-1},y_t):t=1,\ldots,T\}. All model fitting and target construction use the information available through TT, and the fitted models then use XTX_T to forecast returns at T+1T+1. For notational brevity, we suppress the rolling sample index throughout. Here, we set the factor count to KK and the local window length to mPCA=Thm_{\mathrm{PCA}}=Th, where h∈(0,1)h\in(0,1) and mPCA≥2m_{\mathrm{PCA}}\ge2 is an integer.

S1. Fit the conditional mean and construct residuals. Estimate the initial transformer by least squares:

f^=arg min⁡F∈F∑t=1T∥yt−F(Xt−1)∥22.\widehat{f}=\underset{F\in\mathcal{F}}{\operatorname{arg\,min}}\sum_{t=1}^{T}\lVert y_t-F(X_{t-1})\rVert_2^2.

For t=1,…,Tt=1,\ldots,T, we can compute y^t:=f^(Xt−1)\widehat{y}_t:=\widehat{f}(X_{t-1}) and ν^t:=yt−y^t\widehat{\nu}_t:=y_t-\widehat{y}_t.

S2. Estimate the local loading spaces. For each t=mPCA+1,…,Tt=m_{\mathrm{PCA}}+1,\ldots,T, define Wt:={t−mPCA,…,t−1}\mathcal{W}_t:=\{t-m_{\mathrm{PCA}},\ldots,t-1\}. Conceptually, local estimation uses the approximation β(Xs−1)≈β(Xt−1)\beta(X_{s-1})\approx\beta(X_{t-1}) for s∈Wts\in\mathcal{W}_t, as formalized in Assumption 4. The use of temporal localization follows the local-PCA approach of Su and Wang (2017), applied here to residuals from the first-stage return regression. Here, we use the quartic kernel

K(x)={1516(1−x2)2,∣x∣≤1,0,∣x∣>1,ws,t:=K((s−t)/mPCA)∑j∈WtK((j−t)/mPCA).(6)\mathcal{K}(x)= \begin{cases} \frac{15}{16}(1-x^2)^2, & |x|\le1,\\ 0, & |x|>1, \end{cases} \qquad w_{s,t}:=\frac{\mathcal{K}((s-t)/m_{\mathrm{PCA}})}{\sum_{j\in\mathcal{W}_t}\mathcal{K}((j-t)/m_{\mathrm{PCA}})}. \tag*{(6)}

Note that the weights sum to one and vanish at s=t−mPCAs=t-m_{\mathrm{PCA}}. Then, we form the residual second moment matrix

Z^t:=∑s∈Wtws,tν^sν^s⊤.\widehat{Z}_t:=\sum_{s\in\mathcal{W}_t}w_{s,t}\widehat{\nu}_s\widehat{\nu}_s^{\top}.

Let v^1,t,…,v^K,t\widehat{v}_{1,t},\ldots,\widehat{v}_{K,t} be orthonormal eigenvectors associated with the KK largest eigenvalues of Z^t\widehat{Z}_t. Define

β^(Xt−1):=N[v^1,t,…,v^K,t],1Nβ^(Xt−1)⊤β^(Xt−1)=IK.(7)\widehat{\beta}(X_{t-1}):=\sqrt{N}[\widehat{v}_{1,t},\ldots,\widehat{v}_{K,t}],\qquad \frac{1}{N}\widehat{\beta}(X_{t-1})^{\top}\widehat{\beta}(X_{t-1})=I_K. \tag*{(7)}

Here β^(Xt−1)\widehat{\beta}(X_{t-1}) denotes the local loading estimate at date tt. Since the window contains mPCA−1m_{\mathrm{PCA}}-1 observations with positive weights, we have rank⁡(Z^t)≤min⁡{N,mPCA−1}\operatorname{rank}(\widehat{Z}_t)\le\min\{N,m_{\mathrm{PCA}}-1\}. If factor innovation estimates are required, we can compute

ζ^s∣t:=1Nβ^(Xt−1)⊤ν^s,s∈Wt,\widehat{\zeta}_{s\mid t}:=\frac{1}{N}\widehat{\beta}(X_{t-1})^{\top}\widehat{\nu}_s,\qquad s\in\mathcal{W}_t,

where the index tt identifies the local PCA coordinates.

S3. Construct the component targets. For each eligible date t=mPCA+1,…,Tt=m_{\mathrm{PCA}}+1,\ldots,T, we can estimate the factor risk premia by regressing the fitted conditional mean f^(Xt−1)\widehat{f}(X_{t-1}) on the estimated loadings using least squares across firms. The normalization in (7) yields

λ^(Xt−1)=1Nβ^(Xt−1)⊤f^(Xt−1).(8)\widehat{\lambda}(X_{t-1})=\frac{1}{N}\widehat{\beta}(X_{t-1})^{\top}\widehat{f}(X_{t-1}). \tag*{(8)}

The risk compensation target is g~risk(Xt−1):=β^(Xt−1)λ^(Xt−1)\widetilde{g}_{\mathrm{risk}}(X_{t-1}):=\widehat{\beta}(X_{t-1})\widehat{\lambda}(X_{t-1}), and the conditional mispricing target is

g~α(Xt−1):=f^(Xt−1)−g~risk(Xt−1).\widetilde{g}_{\alpha}(X_{t-1}) := \widehat{f}(X_{t-1}) - \widetilde{g}_{\mathrm{risk}}(X_{t-1}).

Thus, the targets satisfy

f^(Xt−1)=g~α(Xt−1)+g~risk(Xt−1),β^(Xt−1)Tg~α(Xt−1)=0.(9)\widehat{f}(X_{t-1}) = \widetilde{g}_{\alpha}(X_{t-1}) + \widetilde{g}_{\mathrm{risk}}(X_{t-1}), \qquad\widehat{\beta}(X_{t-1})^{\mathsf{T}}\widetilde{g}_{\alpha}(X_{t-1}) = 0. \tag*{(9)}

Both targets have dimension NN and are invariant to changes of basis within the selected loading space, whereas the estimated risk premia have dimension KK and depend on the coordinates.

S4. Learn the component pricing functions. Finally, we fit separate transformers using only dates with complete residual windows. We denote the learned mispricing function by g^α\widehat{g}_{\alpha}, so as to distinguish it from the projection estimates g~α\widetilde{g}_{\alpha} used as targets:

g^α=arg min⁡G∈F∑t=mPCA+1T∥g~α(Xt−1)−G(Xt−1)∥22,\widehat{g}_{\alpha} = \underset{G \in\mathcal{F}}{\operatorname{arg\,min}} \sum_{t=m_{\mathrm{PCA}}+1}^{T} \left\lVert\widetilde{g}_{\alpha}(X_{t-1}) - G(X_{t-1}) \right\rVert_{2}^{2},
g^risk=arg min⁡R∈F∑t=mPCA+1T∥g~risk(Xt−1)−R(Xt−1)∥22.\widehat{g}_{\mathrm{risk}} = \underset{R \in\mathcal{F}}{\operatorname{arg\,min}} \sum_{t=m_{\mathrm{PCA}}+1}^{T} \left\lVert\widetilde{g}_{\mathrm{risk}}(X_{t-1}) - R(X_{t-1}) \right\rVert_{2}^{2}.

Note that each network is trained on its component targets using only Xt−1X_{t-1} as input.

S5. Forecast with the rolling sample. For the new market state XTX_T, we can compute

y^T+1∣T:=g^α(XT)+g^risk(XT).\widehat{y}_{T+1 \mid T} := \widehat{g}_{\alpha}(X_T) + \widehat{g}_{\mathrm{risk}}(X_T).

This combined forecast and the direct forecast f^(XT)\widehat{f}(X_T) target the same conditional mean through different estimation procedures. On the other hand, the component forecasts also provide separate investment signals for mispricing and risk compensation.

Theory

Our theoretical analysis first exploits the intrinsic geometry of the cross-sectional state space to establish a vector-valued Transformer approximation result. The approximation theory is then combined with generalization bounds for dependent samples. Finally, we derive recovery bounds for the structural components by tracing how estimation error from the first stage propagates through local factor estimation and component learning. We start by stating some technical conditions3.

Assumption 1 (Compact Low-Dimensional Bounded-Rank Manifold). For Xt∈RN×PX_t \in\mathbb{R}^{N \times P}, assume that there exists rmax⁡∈Z+r_{\max} \in\mathbb{Z}_{+}, with rmax⁡≪min⁡{N,P}r_{\max} \ll\min\{N, P\}, such that sup⁡trank⁡(Xt)≤rmax⁡\sup_t \operatorname{rank}(X_t) \le r_{\max}. That is, the support of {Xt}\{X_t\} is contained in a compact C2C^2 embedded submanifold

M⊆V≤rmax⁡:={X∈RN×P:rank⁡(X)≤rmax⁡}.\mathcal{M} \subseteq\mathcal{V}_{\le r_{\max}} := \left\{X \in\mathbb{R}^{N \times P} : \operatorname{rank}(X) \le r_{\max}\right\}.

Let d:=dim⁡(M)≤dmax⁡:=rmax⁡(N+P−rmax⁡)d := \dim(\mathcal{M}) \le d_{\max} := r_{\max}(N + P - r_{\max}). Furthermore, M\mathcal{M} has positive reach(M)≥τ>0(\mathcal{M}) \ge\tau> 0, where reach⁡(M):=sup⁡{ρ>0:every Z∈RN×P with dist⁡(Z,M)<ρ, has a unique nearest point in M}\operatorname{reach}(\mathcal{M}) := \sup\{\rho> 0 : \text{every } Z \in\mathbb{R}^{N \times P} \text{ with } \operatorname{dist}(Z,\mathcal{M}) < \rho, \text{ has a unique nearest point in } \mathcal{M}\}. Consequently, M\mathcal{M} admits a finite collection of uniformly bi-Lipschitz local coordinate charts at a common geometric scale controlled by τ\tau.

Assumption 1 states that, although each state XtX_t is represented by an N×PN \times P matrix with NPNP entries, these entries do not vary independently. The bounded-rank condition restricts the state matrices to a structured subset whose dimension is at most rmax⁡(N+P−rmax⁡)r_{\max}(N + P - r_{\max}). Intuitively, a manifold is a possibly curved set that can be described locally by only a small number of coordinates. Thus, a state matrix may have many observed entries while its effective variation is driven by only dd underlying directions. The number dd is the intrinsic dimension of the state space, whereas NPNP is its ambient dimension. Meanwhile, compactness ensures that the entire state space can be covered by finitely many local neighborhoods. The positive-reach condition rules out sharp singularities, near self-intersections, and excessively irregular curvature at the scale relevant for the analysis. As a result, each local neighborhood of M\mathcal{M} can be represented stably by dd coordinates, with distances in the local coordinates remaining comparable to distances between the original matrices. This allows the approximation problem for high-dimensional matrix inputs to be reduced to a finite collection of dd-dimensional local problems. Consequently, the approximation rate depends primarily on the intrinsic dimension dd, rather than directly on the ambient dimension NPNP. Similar neural-network approximation results based on intrinsic dimension are developed by Havrilla and Liao [19] and Chen et al. [6]. The exact-support condition could, in principle, be relaxed to allow states in a small neighborhood of M\mathcal{M} under suitable additional regularity conditions, following the approximate-manifold approach of Jiao et al. [24]. In this section, we retain the current exact manifold assumption mainly for theoretical simplicity.

Assumption 2 (Hölder Continuous and Boundedness of the Pricing Components). The functions gα,grisk:M→RNg_{\alpha}, g_{\mathrm{risk}} : \mathcal{M} \to\mathbb{R}^{N} satisfy, for some β∈(0,1]\beta\in(0,1] and Hf,R>0H_f, R > 0,

∥gα(X)−gα(Y)∥2+∥grisk(X)−grisk(Y)∥2≤Hf∥X−Y∥Fβ,X,Y∈M,\lVert g_{\alpha}(X) - g_{\alpha}(Y) \rVert_2 + \lVert g_{\mathrm{risk}}(X) - g_{\mathrm{risk}}(Y) \rVert_2 \le H_f \lVert X - Y \rVert_F^{\beta}, \qquad X,Y \in\mathcal{M},

and sup⁡X∈M{∥gα(X)∥2+∥grisk(X)∥2}≤R\sup_{X \in\mathcal{M}}\{\|g_{\alpha}(X)\|_{2}+\|g_{\mathrm{risk}}(X)\|_{2}\}\leq R. Since f=gα+griskf=g_{\alpha}+g_{\mathrm{risk}}, the triangle inequality gives

∥f(X)−f(Y)∥2≤Hf∥X−Y∥Fβ,sup⁡X∈M∥f(X)∥2≤R.\|f(X)-f(Y)\|_{2}\leq H_f\|X-Y\|_{F}^{\beta},\qquad\sup_{X\in\mathcal{M}}\|f(X)\|_{2}\leq R.

Thus, the two pricing components and their sum share the regularity bounds used in the approximation analysis. Bi-Lipschitz local coordinates preserve their Hölder regularity on each chart.

Assumption 3 (State Dependence and Return Innovations). We assume the following conditions hold.

(i) Weakly dependent states. Let QtQ_t denote the marginal distribution of Xt−1X_{t-1} and define Q‾T:=T−1∑t=1TQt\overline{Q}_T:=T^{-1}\sum_{t=1}^{T}Q_t. The process is uniformly geometrically β\beta-mixing (βmix\beta_{\mathrm{mix}}): there exist constants C,c>0C,c>0, independent of TT, such that

βmix(k):=sup⁡0≤t≤T−1−kβmix(σ(X0,…,Xt),σ(Xt+k,…,XT−1))≤Ce−ck,k≥1.\beta_{\mathrm{mix}}(k):=\sup_{0\leq t\leq T-1-k}\beta_{\mathrm{mix}}\bigl(\sigma(X_0,\ldots,X_t),\sigma(X_{t+k},\ldots,X_{T-1})\bigr)\leq Ce^{-ck},\qquad k\geq1.

(ii) Return innovations. Let Jt−1:=σ(X0,…,Xt−1,y1,…,yt−1)\mathcal{J}_{t-1}:=\sigma(X_0,\ldots,X_{t-1},y_1,\ldots,y_{t-1}). Write Σζt:=E(ζtζt⊤)\Sigma_{\zeta_t}:=\mathbb{E}(\zeta_t\zeta_t^{\top}) and Σu:=E(utut⊤)\Sigma_u:=\mathbb{E}(u_tu_t^{\top}); note that cross-sectional dependence within utu_t is allowed. For all s∈Rs\in\mathbb{R}, b∈RKb\in\mathbb{R}^{K}, a∈RNa\in\mathbb{R}^{N}, and i,ti,t, we have

Eesb⊤ζt≤eC0s2∥b∥22/2,Eesuit≤eC0s2/2,E[esa⊤νt∣Jt−1]≤eσν2s2∥a∥22/2,\mathbb{E}e^{s b^{\top}\zeta_t}\leq e^{C_0s^2\|b\|_2^2/2},\qquad\mathbb{E}e^{s u_{it}}\leq e^{C_0s^2/2},\qquad\mathbb{E}\left[e^{s a^{\top}\nu_t}\mid\mathcal{J}_{t-1}\right]\leq e^{\sigma_{\nu}^{2}s^2\|a\|_2^2/2},

where C0>0C_0>0 is uniform over indices and sample sizes, whereas σν>0\sigma_{\nu}>0 is uniform over time and may depend on NN.

Assumption 4 (Slow Variation of Loadings). Let hh denote the temporal bandwidth used in local PCA. Across sample sizes, the effective window length ThTh is held fixed. There exist ϑ∈(0,1]\vartheta\in(0,1] and a small constant δβ∈(0,1)\delta_{\beta}\in(0,1), independent of NN and TT, such that, almost surely,

∥β(Xs−1)−β(Xt−1)∥FN≤δβ(∣s−t∣Th)ϑ,1≤s,t≤T,0<∣s−t∣≤Th.\frac{\|\beta(X_{s-1})-\beta(X_{t-1})\|_{F}}{\sqrt{N}}\leq\delta_{\beta}\left(\frac{|s-t|}{Th}\right)^{\vartheta},\qquad1\leq s,t\leq T,\quad0<|s-t|\leq Th.

Assumption 5 (Identification, Factor Strength, and Idiosyncratic Covariance). Let KK be fixed. There exist constants 0<c<C<∞0<c<C<\infty such that the following conditions hold.

(i) Identification. Almost surely, for every tt, we have β(Xt−1)⊤gα(Xt−1)=0\beta(X_{t-1})^{\top}g_{\alpha}(X_{t-1})=0. (ii) Uniform bounds. The loading rows and risk premia satisfy

sup⁡X∈Mmax⁡1≤i≤N∥βi(X)∥2≤C,sup⁡t∥λ(Xt−1)∥2≤C.\sup_{X\in\mathcal{M}}\max_{1\le i\le N}\|\beta_i(X)\|_2\le C,\qquad\sup_t\|\lambda(X_{t-1})\|_2\le C.

The bound RR in Assumption 2 satisfies R2≤CNR^2\le CN.

(iii) Factor strength. Almost surely, uniformly over tt,

cIK≤1Nβ(Xt−1)⊤β(Xt−1)≤CIK,cIK≤Σv≤CIK.cI_K\le\frac{1}{N}\beta(X_{t-1})^{\top}\beta(X_{t-1})\le CI_K,\qquad cI_K\le\Sigma_v\le CI_K.

(iv) Weak idiosyncratic covariance. The idiosyncratic covariance satisfies ∥Σu∥op≤C\|\Sigma_u\|_{\mathrm{op}}\le C.

Assumptions 3–5 provide the innovation, slow-variation, identification, and factor strength conditions used by the residual based local PCA. The loading matrix is identified up to a nonsingular transformation of factor coordinates. The idiosyncratic covariance may exhibit cross-sectional dependence, but its operator norm is uniformly bounded, whereas the nonzero eigenvalues of the factor covariance component are of order NN.

Transformer Function Approximation and Generalization Theory

In this section, we give the approximation and generalization theory for the proposed method.

Theorem 1 (NN-Output Manifold Approximation Theorem). Let τ,R,Hf>0\tau, R, H_f>0, 0<β≤10<\beta\le1, d,N,P∈Nd,N,P\in\mathbb{N} and M\mathcal{M} satisfies Assumption 1. For any ϵ∈(0,1)\epsilon\in(0,1), there exists a transformer neural network class F(LT,LFFN,wFFN,l,dembd,m,R,κ)\mathcal{F}(L_T,L_{\mathrm{FFN}},w_{\mathrm{FFN}},l,d_{\mathrm{embd}},m,R,\kappa) with parameters

LT=O(log⁡(d)),LFFN=O(log⁡(ϵ−1)),wFFN=O(1),l=O(dϵ−dβ)L_T=O(\log(d)),\quad L_{\mathrm{FFN}}=O(\log(\epsilon^{-1})),\quad w_{\mathrm{FFN}}=O(1),\quad l=O(d\epsilon^{-\frac{d}{\beta}})
dembd=O(1),m=O(dϵ−dβ),κ=O(d2ϵ−2dβ),Lpara<LT,d_{\mathrm{embd}}=O(1),\quad m=O(d\epsilon^{-\frac{d}{\beta}}),\quad\kappa=O(d^2\epsilon^{-\frac{2d}{\beta}}),\quad L_{\mathrm{para}}<L_T,

where O(⋅)O(\cdot) hides terms in CM,N,P,Hf,R,τC_{\mathcal{M}},N,P,H_f,R,\tau, such that for ff satisfying Assumption 2, we have

∥Fθ−f∥L∞(M;RN):=sup⁡X∈M∥Fθ(X)−f(X)∥2≤ϵ,\|F_\theta-f\|_{L^\infty(\mathcal{M};\mathbb{R}^N)}:=\sup_{X\in\mathcal{M}}\|F_\theta(X)-f(X)\|_2\le\epsilon,

for some Fθ∈F(LT,LFFN,wFFN,l,dembd,m,R,κ)F_\theta\in\mathcal{F}(L_T,L_{\mathrm{FFN}},w_{\mathrm{FFN}},l,d_{\mathrm{embd}},m,R,\kappa). The same class also contains separate approximations to gαg_\alpha and griskg_{\mathrm{risk}}, each with uniform error at most ϵ\epsilon.

Theorem 1 establishes the approximation foundation of the analysis. Although the input state XX is an N×PN\times P matrix, its support is restricted to the dd-dimensional manifold M\mathcal{M}. The Transformer first represents the high-dimensional input through local coordinates on M\mathcal{M} and then uses these low-dimensional representations to approximate the NN-dimensional conditional return function. Consequently, the dependence of the approximation complexity on the target accuracy ϵ\epsilon is governed by the intrinsic dimension dd, while the ambient dimensions NN and PP enter only through the multiplicative constants. Based on the approximation result, we establish the following Theorem 2 by deriving the corresponding statistical error rate.

Theorem 2 (Statistical Error under Temporal Dependence). Suppose that Assumptions 1, 2, and 3(i)–(ii) hold. Write yt=f(Xt−1)+νt\mathbf{y}_t=f(X_{t-1})+\boldsymbol{\nu}_t, where f=gα+griskf=g_{\alpha}+g_{\mathrm{risk}} is the conditional mean return function. Let φ2\varphi^2 denote a deterministic statistical error term satisfying φ2=O~(NPdϵ2ϵ−d/β/T)\varphi^2=\widetilde{O}(NPd_{\epsilon}^{2} \epsilon^{-d/\beta}/T),

∥f^−f∥L2(Q‾T;RN)2+1T∑t=1T∥f^(Xt−1)−f(Xt−1)∥22=Op(ϵ2+φ2).\|\widehat{f}-f\|_{L^2(\overline{Q}_T;\mathbb{R}^N)}^2+\frac{1}{T}\sum_{t=1}^{T}\|\widehat{f}(X_{t-1})-f(X_{t-1})\|_2^2=O_p(\epsilon^2+\varphi^2).

In particular, choosing ϵ=T−β/(2β+d)\epsilon=T^{-\beta/(2\beta+d)} gives ∥f^−f∥L2(Q‾T;RN)2=O~p(NPd2T−2β/(2β+d))\|\widehat{f}-f\|_{L^2(\overline{Q}_T;\mathbb{R}^N)}^2=\widetilde{O}_p(NPd^2T^{-2\beta/(2\beta+d)}), where O~p\widetilde{O}_p suppresses logarithmic factors.

Theorem 2 builds on the approximation result in Theorem 1, with the required regularity of ff following from Assumption 2. It bounds the initial regression error under temporal dependence and time-varying state distributions. It controls both the L2(Q‾T;RN)L^2(\overline{Q}_T;\mathbb{R}^N) error4 and the average squared error over observed states in probability, providing the basis for subsequent factor and component estimation. For fixed dimensions and geometric quantities, the convergence exponent depends on the smoothness β\beta and intrinsic dimension dd, while the bound also reflects network complexity and the variability of return innovations.

For the in-sample analysis, we can extend the local PCA construction to all dates t=1,…,Tt=1,\ldots,T. For 0<h≤1/20<h\le1/2, Th∈NTh\in\mathbb{N}, and Th≥2Th\ge2, define

Wt:={{1,…,Th},1≤t≤Th,{t−Th,…,t−1},Th<t≤T,\mathcal{W}_t:=\begin{cases} \{1,\ldots,Th\}, & 1\le t\le Th,\\ \{t-Th,\ldots,t-1\}, & Th<t\le T, \end{cases}

and the initial windows use nearby observations within the training sample. All windows contain ThTh observations and satisfy ∣s−t∣≤Th|s-t|\le Th for s∈Wts\in\mathcal{W}_t. For this theoretical construction, the S3 targets are formed at all TT dates, and the S4 regressions use all these targets. Note that at each out-of-sample forecast origin, the entire construction uses only the available training sample.

Theorem 3 (Local Loading Recovery). *Suppose that the conditions of Theorem 2 and Assumptions 4–5 hold. Let 1≤K<N1\le K<N. Construct β^(Xt−1)\widehat{\beta}(X_{t-1}) by residual-based local PCA using the win-

dows defined above and the quartic kernel in (6). Write η2:=N/(Th)+Nδβ2+N−1\eta^2 := N/(Th)+N\delta_\beta^2+N^{-1}. There exist nonsingular matrices HtH_t satisfying N−1Ht⊤β(Xt−1)⊤β(Xt−1)Ht=IKN^{-1}H_t^\top\beta(X_{t-1})^\top\beta(X_{t-1})H_t=I_K and ∥Ht∥op+∥Ht−1∥op≤C\lVert H_t\rVert_{\mathrm{op}}+\lVert H_t^{-1}\rVert_{\mathrm{op}}\le C, such that

1T∑t=1T∥β^(Xt−1)−β(Xt−1)Ht∥F2=Op(ϵ2+φ2+η2).\frac{1}{T}\sum_{t=1}^{T}\left\lVert\widehat{\beta}(X_{t-1})-\beta(X_{t-1})H_t\right\rVert_F^2=O_p\left(\epsilon^2+\varphi^2+\eta^2\right).

Theorem 3 controls the average loading error in probability up to a change of factor coordinates. The bound accounts for first-stage estimation, local second-moment fluctuations, temporal variation of the loadings, and idiosyncratic covariance. The boundary windows satisfy the same error bound as the historical windows. Under Assumption 4, local variation contributes Nδβ2N\delta_\beta^2, which need not vanish when δβ\delta_\beta is fixed.

Then, the S4 regressions use the class F\mathcal{F} specified in Theorem 2. And by Theorem 1, this class contains separate approximations to gαg_\alpha and griskg_{\mathrm{risk}}, each with uniform error at most ϵ\epsilon.

Theorem 4 (Statistical Error Bounds for Conditional Pricing Components). Suppose that the conditions of Theorem 3 hold, then we have

1T∑t=1T∥g^α(Xt−1)−gα(Xt−1)∥22+∥g^α−gα∥L2(Q‾T;RN)2=Op(ϵ2+φ2+η2).\frac{1}{T}\sum_{t=1}^{T}\left\lVert\widehat{g}_\alpha(X_{t-1})-g_\alpha(X_{t-1})\right\rVert_2^2+\left\lVert\widehat{g}_\alpha-g_\alpha\right\rVert_{L^2(\overline{Q}_T;\mathbb{R}^N)}^2=O_p\left(\epsilon^2+\varphi^2+\eta^2\right).
1T∑t=1T∥g^risk(Xt−1)−grisk(Xt−1)∥22+∥g^risk−grisk∥L2(Q‾T;RN)2=Op(ϵ2+φ2+η2).\frac{1}{T}\sum_{t=1}^{T}\left\lVert\widehat{g}_{\mathrm{risk}}(X_{t-1})-g_{\mathrm{risk}}(X_{t-1})\right\rVert_2^2+\left\lVert\widehat{g}_{\mathrm{risk}}-g_{\mathrm{risk}}\right\rVert_{L^2(\overline{Q}_T;\mathbb{R}^N)}^2=O_p\left(\epsilon^2+\varphi^2+\eta^2\right).

Consequently, their sum satisfies

∥g^α+g^risk−f∥L2(Q‾T;RN)2=Op(ϵ2+φ2+η2).\left\lVert\widehat{g}_\alpha+\widehat{g}_{\mathrm{risk}}-f\right\rVert_{L^2(\overline{Q}_T;\mathbb{R}^N)}^2=O_p\left(\epsilon^2+\varphi^2+\eta^2\right).

Theorem 4 traces how the initial estimation and loading space recovery affect the estimated pricing components and their sum. The bounds account for approximation, statistical estimation, and local PCA errors through ϵ2\epsilon^2, φ2\varphi^2, and η2\eta^2, respectively, both over the observed states and under the average marginal distribution Q‾T\overline{Q}_T. The following corollary relates estimation of the combined pricing function to return prediction.

Corollary 5 (Mean Squared Forecast Error Decomposition). Under the conditions of Theorem 4, let g^α(t)\widehat{g}_\alpha^{(t)} and g^risk(t)\widehat{g}_{\mathrm{risk}}^{(t)} be fitted using information available at forecast origin tt, and define y^t+1∣t:=g^α(t)(Xt)+g^risk(t)(Xt)\widehat{y}_{t+1|t}:=\widehat{g}_\alpha^{(t)}(X_t)+\widehat{g}_{\mathrm{risk}}^{(t)}(X_t). Then we have

E∥yt+1−y^t+1∣t∥22=E∥νt+1∥22+E∥y^t+1∣t−f(Xt)∥22.\mathbb{E}\left\lVert y_{t+1}-\widehat{y}_{t+1|t}\right\rVert_2^2=\mathbb{E}\left\lVert\nu_{t+1}\right\rVert_2^2+\mathbb{E}\left\lVert\widehat{y}_{t+1|t}-f(X_t)\right\rVert_2^2.

The identity also holds after averaging over the evaluation dates.

The corollary separates forecast error into the conditional mean estimation error and irreducible innovation variance. The former arises from estimating the sum of the two pricing components, whose recovery is studied in Theorem 4; while the latter reflects factor innovations and idiosyncratic shocks with zero conditional mean given the information available at the forecast origin. This variation remains unpredictable under that information set, even when the pricing functions are known.

Simulation and Empirical Analysis

In this section, we assess CAP-T through simulation and application to U.S. equities. The simulation examines recovery of the pricing components and loading space when the underlying structure is known. The empirical analysis evaluates return prediction, the information captured by the estimated components, and their use in portfolio allocation.

In both analyses, we compare CAP-T with IPCA [25] and DNN-varying [13]. IPCA specifies factor loadings as linear functions of firm characteristics. DNN-varying combines neural networks estimated separately at each date with local PCA and subsequent component regressions. Its pricing functions use each firm’s own characteristics, whereas CAP-T incorporates information across firms. We consider two CAP-T variants with ReLU and Softmax attention, respectively. ReLU attention follows our theoretical specification, while Softmax provides a conventional alternative. Softmax assigns nonnegative weights that sum to one, whereas ReLU sets nonpositive scores to zero without normalizing the remaining weights. These coefficients aggregate predictive information, making normalization an architectural choice. We hold the remaining architectural and training settings fixed when comparing the two attention mechanisms. For each CAP-T variant, the networks for total expected returns, conditional mispricing, and risk compensation share an architectural template but are estimated separately. Both analyses consider one to three layers for CAP-T and DNN-varying.

Simulation

We examine the recovery of conditional mispricing, risk compensation, and the factor loading space under smoothly varying exposures. Each simulated panel contains N=120N = 120 firms observed over T=600T = 600 periods, with K=5K = 5 factors. The state matrix varies over time through a calendar index and includes 18 characteristics and an intercept, giving P=20P = 20:

(Xt−1)i1=1,(Xt−1)i,11=t/T,(X_{t-1})_{i1} = 1,\qquad(X_{t-1})_{i,11} = t/T,
(Xt−1)i,j+1+1{j≥10}=2cos⁡[πj(i−1/2)N],j=1,…,18.(X_{t-1})_{i,j+1+\mathbb{1}\{j\ge10\}} = \sqrt{2}\cos\left[\frac{\pi j(i-1/2)}{N}\right],\qquad j=1,\ldots,18.

The first loading is constant, [β(Xt−1)]i1=1[\beta(X_{t-1})]_{i1}=1, while the remaining loadings rotate smoothly:

[β(Xt−1)]ij=(Xt−1)ijcos⁡(1.6jt/T)+(Xt−1)i,j+4sin⁡(1.6jt/T),j=2,…,5.[\beta(X_{t-1})]_{ij}=(X_{t-1})_{ij}\cos(1.6jt/T)+(X_{t-1})_{i,j+4}\sin(1.6jt/T),\qquad j=2,\ldots,5.

gαg_{\alpha} and λ\lambda vary with the calendar index:

[gα(Xt−1)]i=0.005{1+0.1sin⁡(2πt/T)}(Xt−1)i,10,[g_{\alpha}(X_{t-1})]_i=0.005\{1+0.1\sin(2\pi t/T)\}(X_{t-1})_{i,10},
λ(Xt−1)={1+0.05sin⁡[π2(Xt−1)1,11]}(0.005, 0.003, −0.002, 0.002, 0.001)T.\lambda(X_{t-1})=\left\{1+0.05\sin\left[\frac{\pi}{2}(X_{t-1})_{1,11}\right]\right\}(0.005,\ 0.003,\ -0.002,\ 0.002,\ 0.001)^{\mathsf{T}}.

The construction satisfies β(Xt−1)Tβ(Xt−1)/N=I5\beta(X_{t-1})^{\mathsf{T}}\beta(X_{t-1})/N=I_5 and β(Xt−1)Tgα(Xt−1)=0\beta(X_{t-1})^{\mathsf{T}}g_{\alpha}(X_{t-1})=0. Factor innovations ζt\zeta_t are independent across dates and components, with zero-mean Gaussian distributions and standard deviations (0.020,0.018,0.016,0.014,0.012)(0.020,0.018,0.016,0.014,0.012). The errors uitu_{it} are iid N(0,0.0012)N(0,0.001^2) and independent of ζt\zeta_t. We generate ten panels with independent innovation paths and common pricing functions, estimating all models at K=5K=5.

The first 80% of each panel is reserved for initial estimation and the remaining 20% for out-of-sample evaluation. CAP-T and IPCA use 360-period rolling training windows, six-period parameter updates, and one-period-ahead forecasts. CAP-T constructs component targets from local PCA of residuals at the preceding 12 dates. A quartic kernel with bandwidth nine gives positive weight to the eight most recent residuals. The comparison includes Softmax and ReLU CAP-T and DNN-varying, each with one to three layers, and IPCA. For an evaluation period T\mathcal{T}, estimation accuracy is measured by RMSE(c^,c;T\widehat{c},c;\mathcal{T}) ={∑t∈T∥c^t−ct∥22/N∣T∣}1/2=\{\sum_{t\in\mathcal{T}}\|\widehat{c}_t-c_t\|_2^2/N|\mathcal{T}|\}^{1/2}. Predictions are evaluated against observed yy, while estimated components and decomposition targets are compared with their true values. We also report R2R^2 for recombined predictions, using fitted values in sample and forecasts out of sample. Following Yu et al. (2015), let θj,t=arccos⁡{σj(UtTU^t)}\theta_{j,t}=\arccos\{\sigma_j(U_t^{\mathsf{T}}\widehat{U}_t)\} be the principal angles between the true and estimated loading spaces, where UtU_t and U^t\widehat{U}_t are orthonormal bases and σj\sigma_j denotes the jjth largest singular value. We use a normalized version of the projection distance in Edelman et al. [12], averaged over the evaluation period:

d‾proj(T)=1∣T∣∑t∈T{1K∑j=1Ksin⁡2θj,t}1/2=1∣T∣∑t∈T∥UtUtT−U^tU^tT∥F2K.\overline{d}_{\mathrm{proj}}(\mathcal{T})=\frac{1}{|\mathcal{T}|}\sum_{t\in\mathcal{T}}\left\{\frac{1}{K}\sum_{j=1}^{K}\sin^2\theta_{j,t}\right\}^{1/2}=\frac{1}{|\mathcal{T}|}\sum_{t\in\mathcal{T}}\frac{\|U_tU_t^{\mathsf{T}}-\widehat{U}_t\widehat{U}_t^{\mathsf{T}}\|_F}{\sqrt{2K}}.

This basis-invariant distance ranges from zero for identical spaces to one for orthogonal spaces. All statistics are computed within each simulated panel and then averaged across ten panels. For DNN-varying, label and loading diagnostics use contemporaneous observations, whereas forecasts use previously estimated networks. For IPCA, label RMSE evaluates the mispricing estimate obtained by alternating least squares.

Table 1 shows that all six CAP-T specifications outperform DNN-varying and IPCA on the five error measures and Total R2R^2 in both samples. Outside the estimation sample, one-layer Softmax achieves the lowest return RMSE and the highest Total R2R^2, at 4.8601%, while two-layer ReLU gives the lowest mispricing RMSE. Three-layer Softmax gives the lowest loading-space error, 0.1169, while two-layer Softmax gives the lowest risk-compensation RMSE, 2.5911×10−32.5911 \times10^{-3}. Thus, additional depth does not uniformly improve performance. For every CAP-T specification, learning the mispricing function reduces RMSE relative to the constructed targets.

ModelLayersReturnsMispricingRiskLabelsdˉproj\bar{d}_{\mathrm{proj}}Total
In sample
CAP-T (ReLU)135.91261.00441.98661.68720.09735.8683
235.87350.93282.41231.77930.10196.0729
335.84930.89182.79171.77580.10146.1993
CAP-T (Softmax)135.92000.99951.91331.70120.09835.8291
235.89520.94992.08601.75110.10015.9593
335.87970.92982.31781.76880.10146.0400
DNN-varying136.57455.12104.60594.72420.76552.3648
236.55825.06524.59574.85930.77372.4516
336.54945.02164.59724.68310.77382.4994
IPCA—40.90918.398322.05228.39830.5290<0< 0
Out of sample
CAP-T (ReLU)136.53791.19022.73542.15510.12414.8527
236.54741.05512.73732.14230.12114.8050
336.58041.25573.15182.30120.12784.6300
CAP-T (Softmax)136.53641.12942.61212.30360.12934.8601
236.54601.29042.59112.13920.12284.8105
336.54971.24242.66562.02230.11694.7917
DNN-varying137.37895.40066.02115.54030.78690.4200
237.28425.11555.65465.67740.79990.9227
337.30295.08645.80335.29730.79330.8244
IPCA—37.60782.40399.60942.40390.6055<0< 0

Table 1. Recovery and prediction with five factors

Figure 1 shows that both CAP-T variants track the rotation of the loading space in sample and out of sample. In the left three columns, the estimated angles closely follow the true trajectories. In the right three columns, the first four ordered principal angles remain small for both variants, while the fifth is higher. Softmax has a lower average fifth angle than ReLU out of sample, indicating slightly better alignment with the true loading space. These patterns support accurate tracking and recovery beyond the initial estimation sample.

Figure 1. Tracking and recovery of the loading space. In the left three columns, lines show ϕj,t=(180/π)arccos⁡{σj(U∗⊤Ut)}\phi_{j,t} = (180/\pi)\arccos\{\sigma_j(U_*^\top U_t)\} and hollow diamonds show mean estimated counterparts, where U∗U_* is the true basis at the first date of each row. The right three columns show (180/π)θˉj,t(180/\pi)\bar{\theta}_{j,t}; smaller values indicate better recovery. The upper and lower rows show the estimation and out-of-sample periods.

Empirical Analysis

We apply the models introduced in Section 4.1 to a monthly panel spanning 1965-2020. Returns come from CRSP, and accounting information is from Compustat, following the data sources used by Fan et al. [13], Freyberger et al. [15], and Kelly et al. [25]. The inputs comprise 150 firm characteristics drawn primarily from Jensen et al. [23]. The underlying universe consists of U.S. common stocks listed on the NYSE, AMEX, and NASDAQ. To accommodate reporting lags, accounting characteristics for fiscal years ending in calendar year t−1t-1 are matched to returns from July of year tt through June of year t+1t+1.

After aligning firms across sources and applying the sample filters, we retain a balanced panel of 167 firms with 670 monthly observations and no missing returns. Characteristics observed from February 1965 through November 2020 are paired with excess returns in the following month, from March 1965 through December 2020. The first 536 observations form the initial estimation sample. The remaining 134 months, from November 2009 through December 2020, form the common evaluation period.

Using the same attention mechanisms and network depths as in the simulation, we assess return prediction, component prediction, and portfolio performance in the observed data. For a forecast Ct∣t−1C_{t|t-1}, we report the uncentered coefficient of determination

R2(C)=1−∑t∈TOOS∥yt−Ct∣t−1∥22∑t∈TOOS∥yt∥22,R^2(C) = 1 - \frac{\sum_{t\in\mathcal{T}_{\mathrm{OOS}}}\lVert y_t-C_{t|t-1}\rVert_2^2}{\sum_{t\in\mathcal{T}_{\mathrm{OOS}}}\lVert y_t\rVert_2^2},

where TOOS\mathcal{T}_{\mathrm{OOS}} contains the 134 evaluation months. We evaluate the Total forecast y^t∣t−1\hat{y}_{t|t-1} and its two components, g^α(Xt−1)\hat{g}_{\alpha}(X_{t-1}) and g^risk(Xt−1)\hat{g}_{\mathrm{risk}}(X_{t-1}). Risk compensation excludes realized factor innovations. Each component is scored against full realized excess returns, so the component scores are not additive. Scores are computed for each fitted member before averaging. CAP-T uses three independently fitted pipelines; DNN-varying uses three forecast members sharing the same decomposition targets; and IPCA provides a deterministic forecast. All R2R^2 values are reported in percent, with negative values displayed as < 0.

Table 2 examines monthly, quarterly, and semiannual parameter updating. The interval MM controls the re-estimation frequency, while the forecast horizon remains one month. Reported ranges cover all factor counts and network depths within each model family. CAP-T maintains positive scores for the Total forecast and both components throughout the grid. Its advantage over the benchmarks persists across updating schedules, factor counts, and network depths. Here we only report semiannual updating, i.e., M=6M = 6 to save space, while ?? reports the other schedules.

ModelLayersMMMinimum R2R^2 (%) — TotalMinimum R2R^2 (%) — RiskMinimum R2R^2 (%) — MispricingMaximum R2R^2 (%) — TotalMaximum R2R^2 (%) — RiskMaximum R2R^2 (%) — Mispricing
CAP-T (ReLU)1–311.52150.96220.77841.63091.28201.1289
31.54380.96510.80341.66111.28391.1618
61.56800.96890.81761.68151.29641.1866
CAP-T (Softmax)1–311.57560.96850.78991.67101.30171.1399
31.60011.00930.82561.69421.31231.1791
61.61401.01690.84351.71611.32011.2017
DNN-varying1–31< 00.0740< 0< 00.1944< 0
30.31060.4994< 00.48420.6278< 0
60.49840.6893< 00.62460.78810.0413
IPCA—1< 00.0294< 0< 00.1791< 0
3< 00.0448< 0< 00.1814< 0
6< 00.0543< 0< 00.1899< 0

Table 2. Predictive performance across updating schedules

Notes: MM is the parameter updating interval in months; all forecasts are for the following month. Ranges span K=1,…,7K = 1, \ldots, 7 and, for the neural networks, one to three layers, after averaging scores across fitted members. All models use the same 134 evaluation months. Risk and Mispricing are each evaluated against full realized excess returns. Bold identifies the highest endpoint in each numeric column.

Then, Table 3 separates the roles of attention, depth, and factor dimension. CAP-T retains its predictive advantage across configurations. Softmax generally provides the strongest Total forecasts, while ReLU remains effective throughout the grid. Factor dimension has a clearer effect on component prediction than on the Total forecast. Risk compensation scores rise with KK for every CAP-T specification, while mispricing scores decline but remain positive. This pattern is consistent with the changing definition of mispricing as the factor space expands. The Total forecast remains comparatively stable. For example, as KK increases from one to seven, the two layer Softmax model’s risk compensation R2R^2 rises from 1.0541% to 1.3201%, while its Total forecast R2R^2 changes only modestly, from 1.7161% to 1.6448%. The attention mechanisms exhibit complementary strengths: Softmax with two layers leads risk compensation prediction, while ReLU with three layers performs particularly well for mispricing at larger factor counts. At K=7K = 7, the three layer ReLU model achieves the highest mispricing R2R^2 of 0.8989%, while the two layer Softmax model leads risk compensation prediction at 1.3201%. These results motivate allowing the component networks to learn different relationships among firms.

Economic Performance Evaluation

We evaluate the investment value of CAP-T through portfolio allocation based on predicted total returns and stock selection based on conditional alpha. Both strategies are fully invested and long-only. The long-only restriction is motivated by its role in mitigating estimation risk and avoiding short-selling frictions [22, 10, 11]. Allocation rules are identical across models and factor counts within each strategy. Portfolios rebalance monthly, and performance is evaluated over the 134 months from November 2009 to December 2020. And reported Sharpe ratios are obtained by averaging the annualized ratios computed from net monthly excess returns.5

The first strategy uses predicted total returns as expected returns in the mean–variance framework of Markowitz [33], maximizing the forecast return-to-volatility ratio, with the portfolio fully invested and each asset weight no more than 12%. Performance is compared with equal-weighted (EW) and value-weighted (VW) portfolios of all 167 firms, with VW weights based on previous-month market capitalization. The same turnover-based transaction costs apply to all portfolios.

Table 4 shows that CAP-T achieves higher net Sharpe ratios than all DNN-varying and IPCA specifications at every factor count. Across architectures and factor dimensions, CAP-T’s net Sharpe ratios range from 1.0756 to 1.1402, compared with 0.8276 and 0.7341 for the full-universe EW and VW benchmarks. The highest value is attained with six factors. Performance varies modestly across factor counts within each architecture, indicating that CAP-T’s allocation advantage extends across the specifications considered.

ModelLayers1234567
CAP-T (ReLU)11.13821.12451.11011.10941.11221.10891.1091
CAP-T (ReLU)21.07561.08291.07591.08651.08171.08471.0807
CAP-T (ReLU)31.11171.12841.13221.13721.13641.14021.1361
CAP-T (Softmax)11.11461.12411.11281.11391.10941.10871.0978
CAP-T (Softmax)21.08611.10141.09811.10391.10701.10261.0930
CAP-T (Softmax)31.12271.13071.12061.11721.11471.10861.0963
DNN-varying11.03881.04191.03751.03891.04341.03161.0328
DNN-varying21.00371.00101.00521.01351.01930.99850.9973
DNN-varying31.06861.05771.04481.05991.06411.06441.0692
IPCA—0.79090.75130.73910.77040.80970.86350.8794

Table 4. Forecast-based mean–variance portfolios: net Sharpe ratios. Number of factors, KK.

The second strategy examines whether conditional alpha identifies attractive stocks within the long-only mandate. Adapting the forecast-based decile-sorting approach of Gu et al. (2020),

ModelLayersNumber of factors, KK
1234567
Panel A. Total return forecast
CAP-T (ReLU)11.68151.62401.61421.63161.62841.63341.6400
21.60341.59331.58211.56801.56931.57751.5916
31.64341.65761.64111.64081.62891.62351.6215
CAP-T (Softmax)11.67851.68651.64211.63451.62971.62021.6140
21.71611.69311.67021.67121.66521.65311.6448
31.69601.68601.67061.66621.66151.65321.6454
DNN-varying10.49840.55950.56500.56730.56760.57000.5751
20.52800.53490.53580.53860.54250.54690.5504
30.60960.61510.62460.61450.62380.62380.6218
IPCA—< 0< 0< 0< 0< 0< 0< 0
Panel B. Risk compensation
CAP-T (ReLU)11.02291.09371.13971.19551.22341.25971.2964
20.96891.09151.12751.15381.18591.22201.2668
30.99821.13921.17001.20491.22501.24581.2764
CAP-T (Softmax)11.01851.16561.18071.21421.24341.26581.2928
21.05411.17191.20991.25261.28051.29791.3201
31.01691.14241.18581.22371.25261.27311.2953
DNN-varying10.78810.78420.75480.75400.74870.75800.7538
20.75410.72280.69130.69060.68980.68930.6936
30.71120.74510.73680.71810.73660.72900.7208
IPCA—0.18990.11040.15320.11050.08880.05430.1163
Panel C. Conditional mispricing
CAP-T (ReLU)11.18011.07421.02340.98050.94380.90810.8696
21.12231.01360.96550.92580.89030.85680.8176
31.18661.08451.03881.00450.97140.93880.8989
CAP-T (Softmax)11.19421.08811.03730.99430.95570.91920.8772
21.20171.08751.03030.98390.94410.90710.8651
31.16551.05240.99840.95440.91660.88200.8435
DNN-varying1< 0< 0< 0< 0< 0< 0< 0
2< 0< 0< 0< 0< 0< 0< 0
30.0413< 0< 0< 0< 0< 0< 0
IPCA—< 0< 0< 0< 0< 0< 0< 0

Table 3. Table 3: Predictive accuracy by attention mechanism, depth, and factor dimension

Notes: Out-of-sample R2R^2 in percent under semiannual parameter updating (M=6M = 6). Scores for the neural network models are averaged across three random seeds; IPCA is deterministic. The total forecast is the sum of predicted mispricing and risk compensation. Corresponding results for M=1M = 1 and M=3M = 3 are reported in ??.

Notes: The full-universe EW and VW benchmarks have net Sharpe ratios of 0.8276 and 0.7341, respectively.

we rank firms by predicted conditional alpha and retain only the highest decile. This gives 17 of the 167 firms, with the decile size rounded up. Portfolio weights are proportional to predicted alpha within this group; all selected alpha forecasts are positive in the evaluated sample. The strategy therefore uses the ranking to select stocks and signal magnitudes to allocate capital, without a short leg.

Table 5 demonstrates the stock-selection value of CAP-T’s conditional alpha. For K=2,…,7K = 2, \ldots, 7, every CAP-T architecture delivers a higher net Sharpe ratio than all DNN-varying and IPCA specifications, while two architectures also lead at K=1K = 1. The strongest portfolio achieves a net Sharpe ratio of 0.8710 with seven factors. These results support the investment value of conditional alpha across factor dimensions and network architectures.

ModelLayers1234567
CAP-T (ReLU)10.74570.78900.83000.79510.85380.82680.8041
CAP-T (ReLU)20.71220.75280.82890.83160.83610.85340.8362
CAP-T (ReLU)30.79250.82110.80220.83390.84830.85760.8710
CAP-T (Softmax)10.77210.81690.82790.85510.85180.83750.8184
CAP-T (Softmax)20.75700.74690.74780.73810.72440.74250.7272
CAP-T (Softmax)30.76920.72660.71490.76450.76440.75550.7584
DNN-varying10.77030.72230.68980.69270.68690.66790.6576
DNN-varying20.52630.52170.56540.54530.55960.54110.5630
DNN-varying30.53010.56100.59020.59230.56810.59500.5529
IPCA—0.67800.70980.69350.71610.67000.64620.6395

Table 5. Top-decile conditional-alpha portfolios: net Sharpe ratios. Number of factors, KK.

Table 6 reports financial statistics for K=5K = 5. Panel A considers forecast-based allocation and Panel B conditional-alpha selection.

ModelLayersSRAR (%)Vol. (%)MDD (%)TOCost (%)CR (%)
CAP-T (ReLU)11.11212.34411.09918.3383.3140.663268.254
21.08212.18811.26418.8043.0360.607261.628
31.13612.39110.90417.1542.8120.562271.111
CAP-T (Softmax)11.10912.59811.35618.3774.2040.841277.672
21.10712.51711.30619.0613.6390.728274.740
31.11512.58811.29717.8443.4740.695277.697
DNN-varying11.04311.40710.93518.0545.0941.019232.641
21.01911.13710.92516.8474.8620.972224.269
31.06411.80911.09516.2624.2450.849247.534
IPCA—0.81012.97816.02729.1469.8081.962267.356
CAP-T (ReLU)10.85412.96115.07026.4469.4601.892283.232
20.83613.03615.75631.0359.4011.880272.606
30.84811.24913.33021.9839.6021.920218.971
CAP-T (Softmax)10.85214.04616.56930.3959.9861.997310.322
20.72412.07716.65033.9679.9341.987235.231
30.76412.56216.62731.7189.6331.927248.246
DNN-varying10.68712.10717.61929.2419.8771.975226.785
20.5609.07316.39531.1159.9121.982137.412
30.5689.77417.22834.9659.9021.980152.593
IPCA—0.67013.94820.81947.77810.1072.021269.103

Table 6. Financial performance with five factors. Panel A. Forecast-based mean–variance portfolios. Panel B. Top-decile conditional-alpha portfolios. Notes: AR denotes annualized mean net excess return; Vol., annualized volatility; MDD, maximum drawdown; TO, annualized one-way turnover; Cost, annualized trading cost; and CR, cumulative net excess return. CR and MDD are computed from an index that compounds net monthly excess returns, starting at one. TO is expressed in portfolio notional units; all other measures except SR are percentages. Bold denotes the highest SR, AR, and CR and the lowest Vol., MDD, TO, and Cost within each panel.

Forecast-based allocation delivers higher net Sharpe ratios for CAP-T across all specifications. Its portfolios combine annualized net excess returns of 12.188–12.598% with volatility of 10.904–11.356%, while turnover and trading costs remain below those of every competing model. These results highlight CAP-T’s ability to translate return forecasts into portfolios with favorable risk-adjusted performance and moderate trading requirements.

Conditional-alpha selection also delivers consistently higher net Sharpe ratios for CAP-T than for DNN-varying and IPCA, with annualized net excess returns reaching 14.046% and cumulative net excess returns reaching 310.322%. The two strategies serve complementary investment objectives: forecast-based allocation balances expected returns against their covariance structure, while alpha selection concentrates capital in firms with the strongest estimated mispricing. At five factors, forecast-based allocation achieves higher net Sharpe ratios and lower volatility, drawdown, and turnover for every CAP-T specification, illustrating the benefits of incorporating covariance information into portfolio construction.

Economic structure in cross-firm attention

Having evaluated CAP-T’s predictive and investment performance, we examine whether its attention representations capture economically meaningful relationships among firms. We select Softmax networks with strong task-specific out-of-sample R2R^2: three layers for first-stage prediction, two for risk compensation, and one for conditional alpha. We fix K=5K = 5, semiannual updating, covering 167 firms over all 134 out-of-sample months.

We apply spectral clustering to the firm representations produced by each network’s final combined multihead attention output, after projection and before residual and feedforward operations. Spectral clustering treats observations as nodes of a similarity graph and uses its eigenvectors to identify groups with strong internal connections relative to their connections with other groups [32]. This approach summarizes the geometry of the learned representations without imposing industry classifications. For each month, we construct Gaussian affinities from pairwise Euclidean distances, using the median positive distance as bandwidth, and normalize the affinity matrix before averaging it over time. Following the normalized spectral clustering procedure, we compute the eigenvectors associated with the smallest eigenvalues of the normalized graph Laplacian, normalize the rows of the resulting embedding, and apply kk-means. We select the number of groups from two to ten using the largest gap between consecutive eigenvalues, an eigengap heuristic described by Luxburg [32]. This criterion selects two groups for each of the three networks. We interpret these partitions through observed firm characteristics and assess selected pairwise associations against documentary evidence and characteristic similarity.

Figure 2 shows that the learned representations organize firms along economically interpretable dimensions. In the first-stage network, the groups contain 85 and 82 firms; the risk network produces a similar split of 88 and 79. Group 1 in both networks has greater short-term volatility, more extreme positive daily returns, and higher idiosyncratic volatility. The first stage partition also distinguishes the high-low spread measure. These patterns connect the information exchanged by the predictors to common risk and trading conditions. Alpha instead separates 95 and 72 firms. Its first group has higher gross and operating profitability and higher Altman ZZ-scores, whereas its second group has a higher Kaplan-Zingales index. The contrast gives the alpha representation a distinct interpretation in terms of operating strength and financing constraints.

Figure 2. Cross-firm attention networks and their economic profiles

Notes: Panels A–C display all 167 firms, colored by cluster. Edges connect pairs with co-clustering scores ≥0.30\ge0.30; scores are the smaller of the two half-sample averages across monthly partitions with 2–10 groups. Thicker lines indicate higher scores; darker lines connect different clusters. Each node represents a firm; the circular layout is used only for visualization. Panels D–F report the eight largest absolute group-mean differences in monthly cross-sectionally standardized characteristics averaged within groups over time.

We measure pairwise association through co-clustering frequencies, drawing on evidence accumulation and consensus clustering [14, 34]. For each pair, we average the same group indicators across monthly spectral partitions with two to ten groups, separately within each 67-month half, and take the smaller average to emphasize temporal persistence. For visualization, we retain scores of at least 0.30 among all 167 firms, using the same threshold within and between groups. This yields 554 within-group and 22 between-group links for the first stage, 823 and 23 for risk, and 272 and 12 for alpha. Cross group links capture recurring associations across months and group counts beyond the two-group partition of the time-averaged graph.

To assess the economic content of these associations, we retain each firm’s three highest scoring partners among the other 166 firms. Counting each unordered pair once yields 402, 409, and 373 distinct pairs for the first-stage, risk, and alpha networks, respectively. We examine annual reports, regulatory filings, official corporate releases, and court records for evidence of economic links, including named counterparties, shared customers, and common product or end markets. Pairs without documentary support are assessed for similarity in observed characteristics; those satisfying neither criterion are recorded as having insufficient evidence. We also assess similarity in firms’ 150 characteristics, requiring it to persist across both sample halves. Table 7 shows that a majority of selected pairs in each network are supported by documentary evidence or characteristic similarity.

First stageRiskAlpha
Documented links128124102
Characteristic similarity142157134
Unresolved132128137
Total pairs402409373
Share with supporting evidence (%)67.268.763.3

Table 7. Economic interpretation of firm associations. Notes: Documentary support takes precedence over characteristic similarity; the reported share combines these two categories. Unresolved pairs meet neither criterion in our ex post review.

Table 8 illustrates five reports supported pairs per network. The first stage recovers named customers, ventures, competitors, and financial service contracts. Meanwhile, risk adds engine production and gas-processing links. For alpha, the Abbott-Pfizer pair illustrates therapeutic overlap shared with the first stage selection, while Dover-SPX adds a distinct link through heating and cooling products. These cases give the component representations distinct economic interpretations and show how they recover complementary company relationships.

Firm pairDocumentary evidenceNetwork
Boeing / Curtiss-WrightBoeing is a principal customer of Curtiss-Wright’s commercial-aircraft business. [1]F
Boeing / Lockheed MartinPartners in the United Launch Alliance joint venture. [1]F
HP / XeroxXerox is a named competitor in HP’s printing business. [1]F
Harris / Northrop GrummanNorthrop Grumman is among Harris’s principal competitors. [1]F
Disney / Wells FargoWells Fargo Bank serves as an indenture trustee for Disney. [1]F
GE / Raytheon TechnologiesGE and Pratt & Whitney participate in an engine and aftermarket joint venture. [1]R
EQT / WilliamsWilliams Ohio Valley Midstream provides gas processing and NGL marketing under contract with EQT. [1]F, R
Hershey / AltriaMcLane is a major customer of both firms. [1], [2]F, R
ITT / Parker-HannifinOverlapping fluid-handling and motion-control products. [1], [2]R
MDU Resources / ONEOKRelated natural-gas pipeline and midstream operations. [1], [2]R
Abbott / PfizerOverlapping cardiovascular and metabolic pharmaceutical markets. [1], [2]F, A
Dover / SPXOverlapping heating, cooling and refrigeration equipment businesses. [1], [2]A
Baxter / HEICOBaxter’s medical devices and HEICO’s electronics serve medical applications. [1], [2]A
Spire / SempraSpire and Sempra’s SoCalGas operate regulated natural-gas distribution businesses. [1], [2]A
Honeywell / ITTBoth supply equipment and components for aircraft. [1], [2]A

Table 8. Notes: F, R, and A denote inclusion in the first-stage, risk, and alpha Top3 pair sets. Bracketed links point to supporting filings within each row.

Characteristic importance across pricing components

Having examined cross-firm relationships, we turn to the characteristics driving risk, alpha, and total return predictions, where total returns are predicted by summing risk and alpha. Importance is compared across ReLU and Softmax networks with one to three layers, using five factors, six-month updates, and 134 out-of-sample months for 167 firms.

Following the zero-out approach of Gu et al. (2020), we measure importance by the decrease in out-of-sample R2R^2 after setting a standardized characteristic to zero, holding the fitted parameters for each rolling forecast fixed. For total YY, we perturb both component inputs and sum the resulting predictions. All three predictors are evaluated against realized returns. Within each architecture and predictor, we average the signed decreases over three independently fitted members, truncate negative averages at zero, and normalize the resulting scores to sum to one across all 150 characteristics.

Figure 3 compares the characteristics most relevant to total return prediction and its risk and alpha components. For each prediction task, we select twelve characteristics, prioritizing those with consistently high importance across attention mechanisms and using average ranks to complete the selection. Displaying their importance separately across the six architectures reveals both the information shared by the predictors and the distinctive emphasis of each component.

Figure 3. Leading characteristics of total return prediction and its components. Notes: Cells report normalized importance (%). R1–R3 and S1–S3 denote ReLU and Softmax with one to three layers. Rows are ordered by average importance across architectures. All panels use a common color scale.

The risk component emphasizes market related characteristics, including proximity to the annual price high, the maximum-return-to-volatility measure, and recent returns. Meanwhile, the alpha component draws more heavily on dividend yield, leverage, cash-flow yield, and inventory growth, alongside residual momentum and seasonality. Total Y combines these sources, with leading characteristics spanning market behavior, valuation, and investment.

Heatmap of total Y characteristic importance

Figure 3a. Total Y.

Heatmap of risk compensation characteristic importance

Figure 3b. Risk compensation.

Heatmap of conditional alpha characteristic importance

Figure 3c. Conditional alpha.

Using the characteristic definitions documented by Jensen et al. [23], we organize selected characteristics into three broad economic groups: return dynamics and market risk (28 characteristics), valuation and payout yields (15), and investment and growth (22). Figure 4 reports equal-weight averages across the six CAP-T architectures. Group shares sum constituent importance scores, retaining normalization over all 150 characteristics.

Figure 4. Economic composition of characteristic importance.

Bar chart of return dynamics and market risk importance

Figure 4a. Return dynamics & market risk.

Bar chart of valuation and payout yields importance

Figure 4b. Valuation & payout yields.

Bar chart of investment and growth importance

Figure 4c. Investment & growth.

Market-related characteristics account for 38.72% of risk importance, compared with 30.12% for total Y and 16.98% for alpha. Valuation and payout yields instead receive their largest average share in alpha (22.17%), followed by total Y (11.91%) and risk (10.14%). Investment and growth show a similar pattern, accounting for 18.93% of alpha importance, 14.93% of total Y importance, and 9.47% of risk importance. Together, these patterns indicate that the estimated risk component places greater relative weight on market-related characteristics, whereas alpha emphasizes valuation and investment-related information. The latter pattern is consistent with evidence linking valuation and corporate investment to abnormal returns [29, 42]. Total return prediction draws on both sets of characteristics, complementing the broader evidence on the predictive importance of market-based signals in Gu et al. (2020).

Testing conditional alpha

The characteristic-importance results reveal distinct predictive roles for risk compensation and conditional alpha. We next examine whether the estimated alpha has a nonzero time average and whether its cross-sectional structure is preserved in out-of-sample forecasts. Using five factors and six-month updates, we study 167 firms from November 2009 to October 2020. Related to the pricing-error diagnostics of Gu et al. (2021), we focus on the time-average alpha.

Notes: Horizontal bars show the number of firms rejecting a zero time mean among 167 firms. Panel A tests the mean of the decomposition estimate g~α,i(Xt−1)\widetilde{g}_{\alpha,i}(X_{t-1}); Panel B tests the mean of the fitted residual ν^i,t\widehat{\nu}_{i,t} defined in Step S1. For CAP-T, the component identity in (9) gives the same residual after subtracting both decomposition components. Each model uses its own first-entry decomposition. Two-sided tests use Bartlett HAC standard errors with six lags [35] and 5% BH correction within each model and panel [3]. Tests condition on fitted estimates.

Figure 5 shows that the estimates identify a substantial average-return component outside the fitted risk-loading space. Across the six CAP-T architectures, 141–149 of the 167 firms exhibit nonzero average alpha after multiple-testing adjustment. This broad incidence persists across both attention mechanisms and all three network depths. The corresponding counts are 42–65 for DNN-varying and 59 for IPCA. Within the fitted CAP-T system, the alpha component therefore plays a substantive role in the return decomposition, and its presence is not confined to a particular architectural choice. The fitted residuals ν^i,t\hat{\nu}_{i,t} provide a complementary diagnostic. Across all six CAP-T architectures, no firm-level residual mean is significant after multiple-testing adjustment.

Figure 5. Tests of zero mean alpha and residuals

The preceding evidence identifies a persistent alpha component within the fitted return decomposition. Since portfolio formation uses predicted alpha to select stocks, we next examine whether out-of-sample alpha forecasts preserve its ordering across firms. For each architecture, we average fitted members within each month and calculate firm-level means over the same 132 months. Panels A and B of Figure 6 group the 167 firms into quintiles of mean decomposition alpha. Mean forecast alpha increases monotonically from Q1 to Q5 under both attention mechanisms and all three network depths. Panel C reports correlations between the two firm-level means, ranging from 0.69 to 0.79. These results indicate that the forecasts retain cross-sectional differences relevant to stock selection.

Figure 6. Cross-sectional alpha structure in out-of-sample forecasts

Conclusion

This paper develops CAP-T by combining cross-firm attention with an identified decomposition of conditional expected returns into alpha and risk compensation. The theoretical analysis establishes approximation and statistical error bounds under temporal dependence and traces error propagation through factor estimation and component learning. The empirical results document improved return prediction and useful portfolio signals after transaction costs. Component-level evidence reveals distinct patterns of characteristic importance and shows that out-of-sample alpha forecasts preserve the cross-sectional structure of decomposition alpha. Together, these findings demonstrate the value of combining flexible prediction with an economically interpretable return decomposition.

However, the current implementation relies on structured firm characteristics. A natural extension is to enrich this information set with financial text and images, motivated in part by econometric methods for measuring news sentiment [38]. Integrating these sources within cross-firm attention would allow us to examine whether information beyond conventional characteristics primarily refines estimated risk compensation or captures additional conditional alpha. Because this extension introduces estimated representations alongside the estimated factor space, it also calls for inference that accounts for uncertainty across learning stages under temporal dependence. Indeed, the literature on multistep estimation provides a methodological starting point for this task [7].

Declaration of generative AI

The authors used OpenAI’s ChatGPT and Codex to assist with language editing, code organization, and locating documentary evidence for the table of firm associations in the empirical analysis. The authors reviewed and verified the resulting text, code, and documentary evidence and take full responsibility for the final content.

References

References

  1. [1]Bai, J. and S. Ng (2002). “Determining the Number of Factors in Approximate Factor Models”. In: Econometrica 70.1, pp. 191–221.DOI
  2. [2]Barigozzi, M. et al. (2026). “Statistical inference for large-dimensional tensor factor model by iterative projections”. In: Journal of Multivariate Analysis 214, p. 105616. DOI: 10.1016/j.jmva.2026.105616.DOI
  3. [3]Benjamini, Y. and Y. Hochberg (1995). “Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing”. In: Journal of the Royal Statistical Society. Series B (Methodological) 57.1, pp. 289–300.DOI
  4. [4]Chang, J. et al. (2023). “Modelling matrix time series via a tensor CP-decomposition”. In: Journal of the Royal Statistical Society Series B: Statistical Methodology 85.1, pp. 127–148. DOI: 10.1093/jrsssb/qkac011.DOI
  5. [5]Chen, J. et al. (2025). “Estimating time-varying networks for high-dimensional time series”. In: Journal of Econometrics 249, p. 105941. DOI: 10.1016/j.jeconom.2024.105941.DOI
  6. [6]Chen, M. et al. (2022). “Nonparametric regression on low-dimensional manifolds using deep ReLU networks: Function approximation and statistical recovery”. In: Information and Inference: A Journal of the IMA 11.4, pp. 1203–1253.arxiv.org/abs/1908.01842
  7. [7]Chen, X. and Z. Liao (2015). “Sieve Semiparametric Two-Step GMM under Weak Dependence”. In: Journal of Econometrics 189.1, pp. 163–186.DOI
  8. [8]Chen, X. and X. Shen (1998). “Sieve Extremum Estimates for Weakly Dependent Data”. In: Econometrica 66.2, pp. 289–314.
  9. [9]Cohen, L. and A. Frazzini (2008). “Economic Links and Predictable Returns”. In: The Journal of Finance 63.4, pp. 1977–2011.DOI
  10. [10]D’Avolio, G. (2002). “The Market for Borrowing Stock”. In: Journal of Financial Economics 66.2-3, pp. 271–306. DOI: 10.1016/S0304-405X(02)00206–4.
  11. [11]Duffie, D., N. Gârleanu, and L. H. Pedersen (2002). “Securities Lending, Shorting, and Pricing”. In: Journal of Financial Economics 66.2-3, pp. 307–339. DOI: 10.1016/S0304-405X(02)00226–X.
  12. [12]Edelman, A., T. A. Arias, and S. T. Smith (1998). “The Geometry of Algorithms with Orthonality Constraints”. In: SIAM Journal on Matrix Analysis and Applications 20.2, pp. 303–353.arxiv.org/abs/physics/9806030
  13. [13]Fan, J. et al. (2026). “Structural Deep Learning in Conditional Asset Pricing”. Working paper, revised February 2026; first circulated in 2022. URL: https://ssrn.com/abstract=4117882.
  14. [14]Fred, A. L. N. and A. K. Jain (2005). “Combining Multiple Clusterings Using Evidence Accumulation”. In: IEEE Transactions on Pattern Analysis and Machine Intelligence 27.6, pp. 835–850.pubmed.ncbi.nlm.nih.gov/15943417
  15. [15]Freyberger, J., A. Neuhierl, and M. Weber (2020). “Dissecting Characteristics Nonparametrically”. In: The Review of Financial Studies 33.5, pp. 2326–2377.
  16. [18]Han, Y. et al. (2024). “Tensor factor model estimation by iterative projection”. In: The Annals of Statistics 52.6, pp. 2641–2667. DOI: 10.1214/24-AOS2412.DOI
  17. [19]Havrilla, A. and W. Liao (2024). “Understanding Scaling Laws with Statistical and Approximation Theory for Transformer Neural Networks on Intrinsically Low-dimensional Data”. In: Advances in Neural Information Processing Systems. Vol. 37. Curran Associates, Inc., pp. 42162–42210.arxiv.org/abs/2411.06646
  18. [22]Jagannathan, R. and T. Ma (2003). “Risk Reduction in Large Portfolios: Why Imposing the Wrong Constraints Helps”. In: The Journal of Finance 58.4, pp. 1651–1683. DOI: 10.1111/1540-6261.00580.DOI
  19. [23]Jensen, T. I., B. Kelly, and L. H. Pedersen (2023). “Is There a Replication Crisis in Finance?” In: The Journal of Finance 78.5, pp. 2465–2518.
  20. [24]Jiao, Y. et al. (2023). “Deep nonparametric regression on approximate manifolds: Nonasymptotic error bounds with polynomial prefactors”. In: The Annals of Statistics 51.2, pp. 691–716. DOI: 10.1214/23-AOS2266.DOI
  21. [25]Kelly, B. T., S. Pruitt, and Y. Su (2019). “Characteristics are covariances: A unified model of risk and return”. In: Journal of Financial Economics 134.3, pp. 501–524.
  22. [26]Kelly, B. T. et al. (2026). Artificial Intelligence Asset Pricing Models. Working Paper 33351. Revised June 2026; first issued January 2025. National Bureau of Economic Research. URL: https://www.nber.org/papers/w33351.
  23. [27]Kong, X.-B. (2017). “On the number of common factors with high-frequency data”. In: Biometrika 104.2, pp. 397–410.DOI
  24. [29]Lakonishok, J., A. Shleifer, and R. W. Vishny (1994). “Contrarian Investment, Extrapolation, and Risk”. In: The Journal of Finance 49.5, pp. 1541–1578.
  25. [30]Lee, J. et al. (2019). “Set Transformer: A Framework for Attention-Based Permutation-Invariant Neural Networks”. In: Proceedings of the 36th International Conference on Machine Learning. Vol. 97. Proceedings of Machine Learning Research. PMLR, pp. 3744–3753.arxiv.org/abs/1810.00825
  26. [31]Lettau, M. and M. Pelger (2020). “Estimating Latent Asset-Pricing Factors”. In: Journal of Econometrics 218.1, pp. 1–31.
  27. [32]Luxburg, U. von (2007). “A Tutorial on Spectral Clustering”. In: Statistics and Computing 17.4, pp. 395–416.arxiv.org/abs/0711.0189
  28. [33]Markowitz, H. (1952). “Portfolio Selection”. In: The Journal of Finance 7.1, pp. 77–91. DOI: 10.1111/j.1540-6261.1952.tb01525.x.DOI
  29. [34]Monti, S. et al. (2003). “Consensus Clustering: A Resampling-Based Method for Class Discovery and Visualization of Gene Expression Microarray Data”. In: Machine Learning 52.1–2, pp. 91–118.
  30. [35]Newey, W. K. and K. D. West (1987). “A Simple, Positive Semi-Definite, Heteroskedasticity and Autocorrelation Consistent Covariance Matrix”. In: Econometrica 55.3, pp. 703–708.DOI
  31. [36]Ross, S. A. (1976). “The Arbitrage Theory of Capital Asset Pricing”. In: Journal of Economic Theory 13.3, pp. 341–360.
  32. [37]Schmidt-Hieber, J. (2020). “Nonparametric Regression Using Deep Neural Networks with ReLU Activation Function”. In: The Annals of Statistics 48.4, pp. 1875–1897.
  33. [38]Shapiro, A. H., M. Sudhof, and D. J. Wilson (2022). “Measuring News Sentiment”. In: Journal of Econometrics 228.2, pp. 221–243.
  34. [39]Sharpe, W. F. (1964). “Capital Asset Prices: A Theory of Market Equilibrium under Conditions of Risk”. In: The Journal of Finance 19.3, pp. 425–442.DOI
  35. [42]Titman, S., K. C. J. Wei, and F. Xie (2004). “Capital Investments and Stock Returns”. In: Journal of Financial and Quantitative Analysis 39.4, pp. 677–700.
  36. [43]Vaswani, A. et al. (2017). “Attention is all you need”. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook, NY, USA: Curran Associates, Inc., pp. 6000–6010.arxiv.org/abs/1706.03762
  37. [44]Wang, D., X. Liu, and R. Chen (2019). “Factor Models for Matrix-Valued High-Dimensional Time Series”. In: Journal of Econometrics 208.1, pp. 231–248.arxiv.org/abs/1610.01889
  38. [45]Wang, F. (2022). “Maximum likelihood estimation and inference for high dimensional generalized factor models with application to factor-augmented regressions”. In: Journal of Econometrics 229.1, pp. 180–200. DOI: 10.1016/j.jeconom.2020.11.002.
  39. [48]Yun, C. et al. (2020). “Are Transformers Universal Approximators of Sequence-to-Sequence Functions?” In: International Conference on Learning Representations. URL: https://openreview.net/forum?id=ByxRM0Ntvr.arxiv.org/abs/1912.10077

Paper details

vlnc:W_2ksubpwm
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC0-1.0
Usage

views · 1

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

OpenAI’s ChatGPT, Codex

View statement
Looking for related papers…
Contents