betaFeedback

WorksComputer Science

SemiNovo: Learning Beyond Search-Identified Spectra for De Novo Peptide Sequencing

Abstract

Sensitive peptide identification is fundamental to protein characterization and novel protein discovery, for which de novo peptide sequencing (DNPS) from tandem mass spectra provides an important database-independent approach. Although deep learning has substantially advanced de novo sequencing, existing models are predominantly trained on high-confidence peptide-spectrum matches obtained through database search. This search-conditioned supervision excludes a large fraction of unidentified spectra, creating a distribution mismatch between the selectively labeled training data and the complete spectrum stream encountered at inference time, thereby limiting peptide identification performance. To address this challenge, we propose SemiNovo, a semi-supervised framework that learns jointly from identified and unlabeled spectra. SemiNovo first constructs DarkSpec, a large-scale quality-controlled corpus of unlabeled spectra collected from diverse public proteomics projects, thereby extending training beyond search-identified spectra and mitigating the resulting shift in DNPS supervision. It then combines stepwise mass-constrained autoregressive decoding with cumulative path confidence to derive reliable supervision from unlabeled spectra. Moreover, SemiNovo introduces head-wise chemical-evidence gating to improve spectral evidence aggregation under noisy and incomplete fragmentation. Experiments show that SemiNovo performs favorably against state-of-the-art methods on multiple benchmarks. Code and dataset are publicly available.

Introduction

Identifying proteins in complex biological samples is fundamental to understanding cellular processes, elucidating disease mechanisms, and discovering biomarkers and therapeutic targets [3, 13]. Tandem mass spectrometry (MS/MS) is the principal high-throughput technology for large-scale protein identification [2]. As illustrated in Figure 1(a), proteins are digested into peptides, precursor ions are isolated and fragmented, and the mass-to-charge ratios and intensities of the resulting fragment ions form an MS/MS spectrum. Because cleavage along a peptide backbone produces mass differences associated with amino-acid residues, the spectrum provides a chemical fingerprint from which the underlying sequence can be inferred [42]. Database search assigns this fingerprint to a peptide in a predefined candidate collection. By contrast, de novo peptide sequencing (DNPS) directly predicts the sequence from the spectrum and precursor information, without requiring the correct peptide to occur in a database. This property makes DNPS valuable for identifying sequence variants, unexpected post-translational modifications (PTMs), and peptides beyond reference proteomes [30, 37].

Figure 1. Database search induces biased supervision. (a) Tandem mass spectrometry fragments peptides into spectra that must be assigned amino-acid sequences. (b) Candidate-database search and false discovery rate control identify only part of the acquired spectra. (c) Retained peptide–spectrum matches occupy a selective subset of the complete spectral space. (d) Increasing identification stringency separates retained and excluded distributions, leaving informative dark spectra outside training.

Deep learning has substantially advanced DNPS, progressing from recurrent models over discretized spectra to point-based encoders, Transformers, retrieval-augmented generation, and diffusion models [7, 8, 10, 19, 24, 30, 35, 36, 37, 41]. Yet these architectures share a dependence on database-search supervision [37, 43]. Search engines score candidate peptide–spectrum matches (PSMs) [1, 9, 16, 17, 38], and only PSMs passing target–decoy false discovery rate (FDR) control are retained as labels [18]. Spectra remain unlabeled when the peptide is absent from the database, fragmentation evidence is incomplete, or the match score is insufficient. Many such spectra recur across experiments and collectively form a substantial, structured population [14]. We refer to them as dark spectra. Figure 1(b–Figure 1d) shows how candidate restriction and confidence filtering preserve only a selective region of the acquired spectral space. Consequently, a model that is database-independent at inference remains trained on the subset that database search can identify.

Learning from dark spectra is therefore important, but it presents three coupled challenges. First, identified PSMs favor database-covered peptides and interpretable fragmentation, whereas dark spectra exhibit different peak density, noise, and fragment completeness. This domain shift encourages models to rely on clean ion ladders and familiar peptide compositions that may not transfer to the full acquisition stream [14, 32]. Second, dark spectra provide no peptide annotations. Pseudo-labeling is possible, but autoregressive sequencing amplifies an early residue error through every later decoder state and prefix-mass decision; token-wise confidence alone cannot ensure that the resulting path remains reliable [20, 22, 27]. Third, dark spectra often contain weak, noisy, or incomplete fragmentation evidence [21]. Our analysis further finds that decoder attention can concentrate on structurally privileged prefix positions rather than residue context supported by diagnostic fragments. This misplaced concentration wastes representational capacity precisely when chemical informative evidence is scarce.

We propose SemiNovo, a unified framework comprising an acquisition-derived dataset, a chemical-evidence-gated model, and a chemically structured semi-supervised training paradigm. First, DarkSpec is refined from 44.9 million spectra collected across public projects. Quality filtering removes malformed and fragment-poor observations, while near-duplicate reduction prevents recurring acquisitions from dominating training; 4.5 million unlabeled spectra broaden coverage beyond search-identified PSMs. Second, SemiNovo generates pseudo-labels through stepwise mass-constrained autoregressive decoding. Cumulative path confidence retains the longest reliable prefix and terminates generation once the path becomes unreliable, preventing an early error from contaminating later supervision. Third, query-dependent chemical-evidence gates regulate how strongly attention output enters the residual stream. This mechanism suppresses updates dominated by structurally special but chemically weak positions and preserves sequence-informative fragment evidence under ambiguous fragmentation. Experiments show that SemiNovo performs favorably against state-of-the-art methods on multiple benchmarks. Our contributions are summarized as follows.

  • Dataset. We characterize the impact of search-conditioned distribution shift on DNPS and construct DarkSpec, a broad-source corpus of 4.5 million unlabeled spectra refined through quality filtering and near-duplicate clustering.

  • Semi-Supervised Learning. We develop a DNPS-specific strategy based on stepwise mass-constrained decoding, cumulative path confidence, reliable-prefix supervision, and monotone early termination.

  • Model. We identify systematic attention concentration on chemically weak prefix positions and introduce head-wise chemical-evidence gating to improve the use of sparse and incomplete fragment evidence.

  • Performance. SemiNovo achieves state-of-the-art peptide precision on multiple benchmarks and consistently outperforms its supervised counterpart across all three datasets.

Related Work

De Novo Peptide Sequencing. Classical de novo peptide sequencing searches for residue paths that explain observed fragment ions using spectrum graphs or probabilistic models [11]. Deep learning replaces hand-crafted scoring with learned spectrum–sequence mappings. DeepNovo combines convolutional spectrum features with recurrent decoding [30], PointNovo operates directly on unordered peaks [24], and Casanovo formulates sequencing as autoregressive Transformer generation [37]. Methods improve complementary aspects of this paradigm. AdaNovo addresses biases in PSMs [32], SearchNovo connects database search with de novo sequencing [33], ReNovo retrieves related peptides before generation [7], and DiffuNovo introduces diffusion-based generation with explicit precursor-mass guidance [8]. NovoBench standardizes datasets, modified-sequence vocabularies, and evaluation metrics for these models [43]. Nevertheless, both existing methods and benchmarks lean predominantly from confident database-search identifications. SemiNovo instead expands supervision to search-unidentified spectra while retaining the NovoBench protocol.

Semi-Supervised Learning. Modern semi-supervised learning (SSL) commonly combines teacher–student consistency with pseudo-label selection. Mean Teacher stabilizes targets through an exponential moving average of model weights [29], while Fix-Match transfers confident predictions from weakly to strongly augmented views [27]. FlexMatch adapts selection thresholds to class-dependent learning progress [40], and USB shows that SSL behavior varies substantially across tasks and domains [31]. Confidence is therefore not automatically a reliable selection criterion. CSL further separates correct and incorrect predictions using their confidence distributions [20]. These studies primarily consider independent classification outputs. Peptide pseudo-labels are autoregressive and chemically constrained, so an early residue error changes every subsequent prediction. SemiNovo consequently evaluates cumulative path confidence, retains a contiguous reliable prefix, and restricts generation to residues compatible with the remaining precursor mass.

Method

Preliminaries

Let x={(mi,li)}i=1Px = \{(m_i, l_i)\}_{i=1}^{P} denote a tandem mass spectrum with PP peaks, where mim_i and lil_i are the mass-to-charge ratio and intensity of peak ii. Its precursor is c=(mpre,z)c = (m_{\mathrm{pre}}, z), comprising the precursor m/zm/z and charge. Given monoisotopic proton mass mH+m_{\mathrm{H^+}} and water mass mH2Om_{\mathrm{H_2O}}, the precursor defines the target residue-sum mass

M(c)=z(mpre−mH+)−mH2O.(1)M(c) = z(m_{\mathrm{pre}} - m_{\mathrm{H^+}}) - m_{\mathrm{H_2O}}. \tag*{(1)}

A peptide is represented by y=(y1,…,yT)y = (y_1, \ldots, y_T), where yT=EOSy_T = \mathrm{EOS}. The vocabulary is Y=Vres∪{EOS}\mathcal{Y} = \mathcal{V}_{\mathrm{res}} \cup\{\mathrm{EOS}\}, where Vres\mathcal{V}_{\mathrm{res}} contains unmodified residues and each supported modified residue as an individual token. Let μ(a)\mu(a) be the residue mass of a∈Vresa \in\mathcal{V}_{\mathrm{res}}. The theoretical residue-sum mass is

m(y)=∑t:yt∈Vresμ(yt).(2)m(y) = \sum_{t:y_t \in\mathcal{V}_{\mathrm{res}}} \mu(y_t). \tag*{(2)}

A valid peptide should satisfy the precursor-mass constraint ∣m(y)−M(c)∣≤δ|m(y) - M(c)| \leq\delta, where δ\delta is the prescribed mass tolerance. Mass feasibility is evaluated at 50 ppm under isotope errors {0,1}\{0, 1\}, and EOS is permitted after at least six residues.

We consider a labeled set Dℓ\mathcal{D}_{\ell} of triples (x,c,y)(x, c, y) obtained from high-confidence PSMs and an unlabeled set Du\mathcal{D}_{u} of pairs (x,c)(x, c) sampled from acquisition-derived source pools independently of database-search peptide assignments. The conditional peptide likelihood is factorized autoregressively

pθ(y∣x,c)=∏t=1Tpθ(yt∣y<t,x,c).(3)p_{\theta}(y \mid x, c) = \prod_{t=1}^{T} p_{\theta}(y_t \mid y_{<t}, x, c). \tag*{(3)}

Here, TT includes the terminal EOS token. For a labeled minibatch Bℓ⊂Dℓ\mathcal{B}_{\ell} \subset\mathcal{D}_{\ell}, conventional supervised DNSP minimizes the teacher-forced token-average negative log-likelihood

Lsup=−∑(x,c,y)∈Bℓ∑t=1Tylog⁡pθ(yt∣y<t,x,c)∑(x,c,y)∈BℓTy.(4)\mathcal{L}_{\mathrm{sup}} = - \frac{ \sum_{(x,c,y) \in\mathcal{B}_{\ell}} \sum_{t=1}^{T_y} \log p_{\theta}(y_t \mid y_{<t}, x, c)}{ \sum_{(x,c,y) \in\mathcal{B}_{\ell}} T_y}. \tag*{(4)}

where TyT_y is the number of non-padding targets in yy. This objective learns only from search-identified spectra, leaving the much larger unlabeled spectrum population outside model optimization.

Figure 2 presents SemiNovo, which complements supervised DNPS with three components. Canonical spectrum processing places labeled and unlabeled observations in a shared input space. A non-causal spectrum encoder and a chemically gated causal peptide decoder map spectra to residue sequences. Finally, stepwise mass-constrained autoregressive decoding converts unlabeled spectra into reliable pseudo-prefix supervision.

Figure 2. Overview of SemiNovo. Labeled, weak, and strong spectrum views enter vertically aligned exponential moving average (EMA) teacher and student networks with head-wise chemical-evidence gating. Mass-constrained autoregressive decoding retains the longest prefix whose cumulative path confidence exceeds τ\tau. The student jointly optimizes identified peptides and reliable pseudo-prefixes.

DarkSpec Corpus

Supervised DNPS corpora retain spectra only when database search produces high-confidence PSMs. Their training distribution is consequently conditioned on database content and search sensitivity, while acquired spectra that remain unidentified receive no learning signal. To expose DNPS models to this omitted region, we construct DarkSpec from 653 acquisition runs across 43 public projects in the PRoteomics IDEntifications Database (PRIDE) [23]. Unlike a PSM corpus, inclusion depends on acquisition rather than peptide assignment. DarkSpec therefore mixes identifiable and unidentified spectra, approximating the acquisition-level population encountered at inference. Its source pool contains 44,963,379 MS/MS spectra from heterogeneous biological samples and experimental runs, covering spectra beyond conventional search-derived supervision. Appendix A lists the contributing projects and spectrum counts.

The acquisition-derived pool broadens coverage but contains spectra with malformed precursors or insufficient fragment evidence. We therefore filter spectra by physical input quality. Using the same NovoBench-compatible processing for labeled and unlabeled data [43], we remove out-of-range, precursor-adjacent, and low-intensity peaks, retain the 150 most intense peaks, and normalize intensities. We discard malformed spectra, invalid precursor charges, and spectra with fewer than 20 valid peaks, retaining over 41 million spectra. Shared processing prevents an artificial domain gap between labeled and unlabeled inputs, while quality-based selection preserves the acquisition-derived unlabeled distribution.

Pooling many projects introduces a second bias: repeated exports and overlapping acquisitions can make a small number of recurring spectral patterns dominate the unlabeled objective. We group near-duplicate spectra using a fixed SimHash signature [4, 6, 12] derived from fragment patterns, precursor mass, and charge, and retain one representative per group. A deterministic subset of 4,500,000 representatives forms the final training corpus, controlling redundancy and computational cost. Exact filtering and redundancy-control settings are provided in Appendix A.

Chemical-Evidence Gating

SemiNovo uses a Transformer encoder–decoder for spectrum-to-peptide generation. Each peak is represented by independent multi-scale Fourier encodings of its continuous m/zm/z and normalized intensity [8, 37].

ϕΩ(v)=[sin⁡(ωv),cos⁡(ωv)]ω∈Ω\phi_{\Omega}(v) = [\sin(\omega v),\cos(\omega v)]_{\omega\in\Omega}
ei=Fpeak([ϕΩm(mi),ϕΩI(Ii)])(5)e_i = F_{\mathrm{peak}}\left([\phi_{\Omega_m}(m_i),\phi_{\Omega_I}(I_i)]\right) \tag*{(5)}

where Ωm\Omega_m and ΩI\Omega_I are fixed frequency sets and FpeakF_{\mathrm{peak}} is a two-layer projection to width dd. A learned spectrum token is prepended, and a non-causal encoder models all peak representations bidirectionally because fragment peaks have no causal order. The decoder starts

from a precursor query formed by the Fourier encoding of neutral precursor mass and a learned charge embedding; the encoded spectrum then conditions residue generation through cross-attention.

DNPS requires the decoder to distinguish residue-localizing fragment evidence from global precursor context. However, Transformer attention can over-emphasize structurally privileged tokens [15, 28, 34, 39]. As shown in Figure 3, Casanovo allocates substantial attention to the precursor query while underweighting informative fragment evidence. This imbalance is especially harmful for dark spectra, whose informative ion series are often sparse or incomplete [42]. SemiNovo instead learns markedly smaller gates for the precursor query than for residue-prefix queries, attenuating chemically non-localizing updates before they enter the residual stream and preserving the decoder’s capacity to integrate diagnostic fragment ions through cross-attention.

Figure 3. Chemical-evidence gating suppresses non-localizing precursor-query updates. (a–b) Mean decoder self-attention in Casanovo and SemiNovo. (c) Layer-wise SemiNovo gate values for precursor and residue-prefix queries. Smaller precursor-query gates reduce the influence of global precursor context on residue prediction. Lines and shaded regions denote mean and standard deviation.

To regulate these prefix updates, SemiNovo applies chemical-evidence gating to causal self-attention in every peptide-decoder layer. For query states Hq∈RL×dH_q \in\mathbb{R}^{L\times d}, an input-dependent sigmoid gate is computed for all HH attention heads [5, 25, 26]:

G=σ(HqWg),G = \sigma(H_qW_g),
O~h=G:,h⊙Oh,\widetilde{O}_h = G_{:,h}\odot O_h,
Hq′=Hq+Wo[O~1;…;O~H].(6)H'_q = H_q + W_o[\widetilde{O}_1;\ldots;\widetilde{O}_H]. \tag*{(6)}

Here, LL is the number of decoder query positions, HH is the number of attention heads, σ\sigma is the sigmoid function, and ⊙\odot denotes element-wise multiplication with broadcasting over the head dimension. The learned matrix Wg∈Rd×HW_g \in\mathbb{R}^{d \times H} produces G∈(0,1)L×HG \in(0,1)^{L \times H}, which contains one gate per query and head; OhO_h is the standard scaled dot-product output of head hh, and WoW_o projects the concatenated gated outputs. The gate allows each decoder layer to attenuate unreliable prefix updates before they enter the residual stream, preserving capacity for residue context supported by the spectrum.

Autoregressive Semi-Supervision

Following common semi-supervised practice [27, 29], SemiNovo maintains an exponential moving average (EMA) teacher θˉ\bar{\theta} and enforces consistency between weakly and strongly augmented spectrum views. For an unlabeled example (xu,cu)∈Du(x_u,c_u) \in\mathcal{D}_u, the teacher generates pseudo-labels from a weak view xuwx_u^w that preserves canonical processing, while the student learns from a strong view xusx_u^s with mild peak removal and intensity perturbation. Peak m/zm/z and precursor information remain unchanged across views. Because peptide tokens form an autoregressive and mass-dependent path, SemiNovo retains only a contiguous pseudo-prefix that is both mass-admissible and statistically reliable.

Stepwise mass-constrained decoding. Token-wise pseudo-labeling ignores a defining property of DNPS: every generated residue consumes part of a fixed precursor mass. Consequently, a locally confident but mass-incompatible token can corrupt all subsequent pseudo-labels. SemiNovo therefore constrains each autoregressive decision by the remaining precursor mass. Let Mu=M(cu)M_u = M(c_u) and let mmin⁡m_{\min} be the lightest residue mass in VresV_{\mathrm{res}}. Appending candidate residue aa to prefix y^<ta\hat{y}_{<t}a leaves residual mass rt(a)=Mu−m(y^<ta)r_t(a) = M_u - m(\hat{y}_{<t}a). The admissible vocabulary is

Vt={a∈Vres:∣rt(a)∣≤δ∨rt(a)≥mmin⁡}∪{EOS:∣Mu−m(y^<t)∣≤δ}.(7)V_t = \{a \in V_{\mathrm{res}} : \lvert r_t(a) \rvert\le\delta\lor r_t(a) \ge m_{\min}\} \cup\{\mathrm{EOS} : \lvert M_u - m(\hat{y}_{<t}) \rvert\le\delta\}. \tag*{(7)}

Let pt(a)p_t(a) be the teacher’s full-vocabulary probability for token aa, conditioned on the current prefix, weak spectrum view, and precursor. We select the most probable mass-admissible token,

y^t=arg⁡max⁡a∈Vt pt(a).(8)\hat{y}_t = \underset{a \in V_t}{\arg\max}\ p_t(a). \tag*{(8)}

The selected token retains its original full-vocabulary probability, ensuring that chemical masking does not alter the confidence used for pseudo-label selection.

Path-consistent prefix selection. The reliability of an autoregressive residue depends on every preceding decision. SemiNovo therefore evaluates the joint confidence of each contiguous generation path. With qt=pt(y^t)q_t = p_t(\hat{y}_t) denoting the confidence of the selected residue conditioned on the preceding prefix and τ∈(0,1)\tau\in(0,1) the path-confidence threshold, we define

C0=1,Ct=Ct−1qt=∏i=1tqi,(9)C_0 = 1,\qquad C_t = C_{t-1}q_t = \prod_{i=1}^{t} q_i, \tag*{(9)}
T∗=max⁡({0}∪{t:Ct≥τ}).T^* = \max(\{0\} \cup\{t : C_t \ge\tau\}).

The accepted target is the longest reliable prefix y^1:T∗\hat{y}_{1:T^*}; if the first token fails, T∗=0T^* = 0 and the spectrum contributes no pseudo-label. Every retained residue is supported by its preceding path, while later suffix uncertainty does not remove reliable earlier supervision. Since 0≤qt≤10 \le q_t \le1, CtC_t is monotone non-increasing. Teacher decoding stops when Ct<τC_t < \tau, at mass-valid EOS, when the admissible set is empty, or at the maximum pseudo-sequence length Tmax⁡T_{\max}.

Algorithm 1 SemiNovo semi-supervised training
Require: batches Bℓ,BuB_\ell, B_u; student θ\theta; teacher θˉ\bar{\theta}; threshold τ\tau; weight λu\lambda_u; EMA decay β\beta
1: Construct weak and strong views (Xuw,Xus)(X_u^w, X_u^s) from BuB_u
2: Initialize accepted prefix set P←∅\mathcal{P} \leftarrow\varnothing
3: for u=1,…,∣Bu∣u = 1,\ldots,\lvert B_u\rvert do
4: y^u←∅\hat{y}_u \leftarrow\varnothing and ℓC,u←0\ell_{C,u} \leftarrow0
5: for t=1,…,Tmax⁡t = 1,\ldots,T_{\max} do
6: Construct Vu,t\mathcal{V}_{u,t} using (7)
7: if Vu,t=∅\mathcal{V}_{u,t} = \varnothing then break
8: end if
9: Select y^u,t←arg⁡max⁡a∈Vu,tpu,t(a)\hat{y}_{u,t} \leftarrow\arg\max_{a\in\mathcal{V}_{u,t}} p_{u,t}(a)
10: ℓC,u←ℓC,u+log⁡pu,t(y^u,t)\ell_{C,u} \leftarrow\ell_{C,u} + \log p_{u,t}(\hat{y}_{u,t})
11: if ℓC,u<log⁡τ\ell_{C,u} < \log\tau then break
12: end if
13: Append y^u,t\hat{y}_{u,t} to y^u\hat{y}_u
14: if y^u,t=EOS\hat{y}_{u,t} = \mathrm{EOS} then break
15: end if
16: end for
17: if ∣y^u∣>0\lvert\hat{y}_u\rvert> 0 then P←P∪{(Xus,cu,y^u)}\mathcal{P} \leftarrow\mathcal{P} \cup\{(X_u^s,c_u,\hat{y}_u)\}
18: end if
19: end for
20: Pack BℓB_\ell and P\mathcal{P} into one student forward pass
21: Update θ\theta using Lsup+λuLunsup\mathcal{L}_{\mathrm{sup}} + \lambda_u\mathcal{L}_{\mathrm{unsup}}
22: θˉ←βθˉ+(1−β)θ\bar{\theta} \leftarrow\beta\bar{\theta} + (1-\beta)\theta

Table 1.

Semi-supervised objective. Given minibatches BℓB_\ell and BuB_u, the student observes xusx_u^s and is teacher-forced on each accepted prefix. Let Tu∗T_u^* denote the accepted prefix length of example uu. The loss is normalized by the number of accepted tokens,

Lunsup=−∑u∈Bu∑t=1Tu∗log⁡pθ(y^u,t∣y^u,<t,xus,cu)∑u∈BuTu∗.(10)\mathcal{L}_{\mathrm{unsup}} = -\frac{\sum_{u\in B_u}\sum_{t=1}^{T_u^*}\log p_\theta(\hat{y}_{u,t}\mid\hat{y}_{u,<t},x_u^s,c_u)}{\sum_{u\in B_u}T_u^*}. \tag*{(10)}

The loss is set to zero when no token is accepted. The complete objective is L=Lsup+λuLunsup\mathcal{L} = \mathcal{L}_{\mathrm{sup}} + \lambda_u\mathcal{L}_{\mathrm{unsup}}, where λu\lambda_u controls the unlabeled term. Labeled spectra receive the same mild training perturbation as the strong unlabeled view, and labeled peptides and accepted pseudo-prefixes share one student forward pass. Algorithm 1 summarizes the training procedure.

Computational complexity. The encoder–decoder costs O(P2d+T2d+PTd+(P+T)d2)O(P^2d + T^2d + PTd + (P+T)d^2) per layer. Decoder gating adds only O(TdH)O(TdH), leaving the asymptotic inference complexity unchanged, while monotone early termination limits the training-only length of teacher decoding.

Experiments

Experimental Settings

Datasets. All experimental settings follow NovoBench [43]. We evaluate on three datasets spanning different resolutions and biological origins: Nine-species [30], HC-PT [10], and Seven-species [30]. Evaluation uses the official NovoBench implementation of peptide precision, precision–coverage AUC, amino-acid precision and recall, and PTM precision. DarkSPEC provides 4.5 million unlabeled spectra refined from 44,963,379 spectra across 43 public PRIDE projects. The same corpus and preprocessing are used for all targets without target-specific selection.

Method
Peptide levelAmino-acid level
Nine-speciesHC-PTSeven-speciesNine-speciesHC-PTSeven-species
Prec.AUCPrec.AUCPrec.AUCPrec.RecallPrec.RecallPrec.Recall
DeepNovo0.4280.3760.3130.2550.2040.1360.6960.6380.5310.5340.4920.433
PointNovo0.4800.4360.4190.3730.0220.0070.7400.6710.6230.6220.1960.169
Casanovo0.4810.4390.2110.1770.1190.0840.6970.6960.4420.4530.3220.327
AdaNovo0.5050.4690.2120.1780.1740.1350.6980.7090.4420.4510.3790.385
π\pi-HelixNovo0.5170.4530.3560.3180.2340.1730.7650.7580.5880.5820.4810.472
SearchNovo0.5500.4890.4470.4130.2590.1740.7480.7460.6520.6580.4890.488
ReNovo0.5680.5280.4670.4360.2780.2280.7700.7690.6510.6480.5120.514
DiffuNovo0.5720.4130.4850.3240.2330.1040.7850.7830.6480.6480.4300.428
SemiNovo (supervised)0.5820.5380.4900.4540.2550.1680.8000.7980.6650.6630.4250.424
SemiNovo0.6350.5930.5140.4750.2800.165<u>0.799</u>0.8000.6760.6750.4260.427

Table 1 (PDF p. 6). Table 1: De novo peptide sequencing performance under the NovoBench protocol. We report peptide precision and area under the precision–coverage curve (AUC) together with amino-acid precision and recall. Best and second-best results are shown in bold and underlined, respectively.

Implementation details. SemiNovo uses nine encoder and nine decoder blocks, hidden width 512, eight attention heads, and feed-forward width 1,024. We first train SemiNovo (supervised) from scratch for 30 epochs with batch size 32 and select the checkpoint with the highest validation sequence accuracy. Semi-supervised training starts from this checkpoint and jointly optimizes labeled NovoBench and unlabeled DarkSpec batches of 512 spectra for six unlabeled-data epochs, cycling the smaller labeled split. We use AdamW with peak learning rates of 5×10−45 \times10^{-4} and 10−310^{-3} for the supervised and semi-supervised stages, EMA decay 0.999, λu=1\lambda_u = 1, τ=0.9\tau= 0.9, and maximum pseudo-sequence length 32.

Main Results

Chemical-evidence gating yields a strong supervised model. Even without unlabeled spectra, SemiNovo (supervised) substantially improves over Casanovo on all three benchmarks. Peptide precision rises from 0.481 to 0.582 on Nine-species, from 0.211 to 0.490 on HC-PT, and from 0.119 to 0.255 on Seven-species. Curve AUC and amino-acid metrics improve consistently, including gains of more than twenty points in amino-acid precision and recall on HC-PT. Because this comparison uses only labeled NovoBench spectra, it establishes the gated spectrum-to-peptide model as a strong backbone before semi-supervised learning. The result is consistent with our motivation: suppressing chemically weak prefix updates allows diagnostic fragment evidence to contribute more effectively to sequence generation.

Unlabeled spectra primarily improve complete-peptide recovery. Adding DarkSpec raises peptide precision from 0.582 to 0.635 on Nine-species, from 0.490 to 0.514 on HC-PT, and from 0.255 to 0.280 on Seven-species. Nine-species and HC-PT achieve the strongest overall results, with AUC increasing to 0.593 and 0.475; Seven-species attains the highest peptide precision among the compared methods but not the best AUC or residue-level performance. Amino-acid metrics change only marginally on Nine-species and Seven-species and by about one point on HC-PT. Semi-supervision therefore does not simply make every residue easier to classify. Instead, exposure to unlabeled spectra reduces the small number of local errors that prevent an otherwise nearly correct sequence from becoming an exact peptide match. Since peptide precision requires every residue to be correct, modest residue-level changes can produce substantially larger sequence-level gains.

All three benchmarks exhibit domain-dependent transfer. DarkSpec is assembled from heterogeneous acquisitions and is not identically distributed to any NovoBench benchmark. This shift appears differently across the three targets. On Nine-species, semi-supervision produces large peptide-level gains but almost no amino-acid or PTM gain, indicating that compatible high-resolution contexts improve sequence consistency without substantially changing local residue discrimination. HC-PT shows smaller peptide-level gains, one-point amino-acid gains, and nearly unchanged PTM precision, consistent with a moderate mismatch in peptide composition and cleavage regime. Seven-species exhibits the strongest shift: peptide precision improves, but curve AUC decreases slightly from 0.168 to 0.165 and PTM precision falls from 0.495 to 0.382. Thus, useful sequence patterns transfer across all three domains, whereas confidence calibration and modification-specific evidence are more sensitive to acquisition mismatch.

Figure 4. Three central ablations on Nine-species. Grouped bars compare the complete model with removal of stepwise mass constraints, removal of chemical-evidence gating, and replacement of cumulative path confidence by mean token confidence.

PTM identification reveals the same division of labor. Table 2 shows that the supervised SemiNovo backbone raises PTM precision over Casanovo from 0.706 to 0.834 on Nine-species, from 0.501 to 0.731 on HC-PT, and from 0.360 to 0.495 on Seven-species. Chemical-evidence gating is therefore especially valuable for modified residues, whose localization depends on a small number of diagnostic fragment ions. Semi-supervision provides only modest additional gains on Nine-species and HC-PT, reaching 0.841 and 0.732, while Seven-species decreases to 0.382 despite stable amino-acid metrics. This disproportionate decline indicates confidence-calibration error under domain shift: confidence learned from DarkSpec is less aligned with modified-residue correctness in low-resolution Seven-species spectra, causing pseudo-label selection to provide less reliable PTM supervision even when ordinary residue accuracy remains stable.

MethodNine-species ↑\uparrowHC-PT ↑\uparrowSeven-species ↑\uparrow
DeepNovo0.5760.6260.391
PointNovo0.6290.6760.117
Casanovo0.7060.5010.360
AdaNovo0.6520.5520.448
π\pi-HelixNovo0.6800.5680.473
SearchNovo0.7640.7150.472
DiffuNovo0.8220.7050.515
SemiNovo (supervised)<u>0.834</u><u>0.731</u>0.495
SemiNovo0.8410.7320.382

Table 2. Table 2: PTM precision on the three NovoBench datasets.

Analysis and Ablations

We conduct all analyses on Nine-species because it combines multiple biological species, modified residues, and high-resolution fragment measurements. Unless stated otherwise, each variant changes only the named factor and retains the same architecture, data split, optimization schedule, and decoding configuration.

Component ablations. Figure 4 isolates three designs introduced by SemiNovo. Removing stepwise mass constraints only reduces peptide precision and curve AUC from 0.635/0.593 to 0.628/0.590. Its principal benefit is computational: pruning precursor-incompatible residues prevents decoding effort from being spent on paths that cannot produce a chemically feasible peptide. Removing chemical-evidence gating causes a larger decline to 0.593/0.571, supporting the attention analysis in Figure 3: regulating chemically weak updates is important for preserving diagnostic fragment evidence. Replacing cumulative path confidence with mean-token confidence, a matched generic SSL control, causes the largest degradation to 0.581/0.513 despite validation tuning over {0.85,0.90,0.95}\{0.85, 0.90, 0.95\}. A low-confidence erroneous residue can be diluted by confident predictions at other positions and enter the pseudo-label. Because subsequent residues condition on the erroneous prefix, this local mistake propagates through the autoregressive sequence and introduces substantially noisier supervision.

Pseudo-label reliability. We hide the annotations of all 28,572 Nine-species validation spectra during pseudo-label generation and reveal them only for evaluation. At matched coverage, cumulative path confidence yields the lowest exact-peptide error, with its largest advantage in the selective low-coverage region (Figure 5). This result confirms that sequence-level path confidence ranks reliable autoregressive pseudo-labels more effectively than aggregating independent token scores.

Exact pseudo-label error rate and error reduction as a function of accepted pseudo-label coverage

Figure 5. Exact pseudo-label error rate as a function of accepted coverage on the complete Nine-species validation split. The lower panel reports the error reduction of cumulative path confidence relative to the alternatives.

Hyperparameter sensitivity. Table 3 varies one factor at a time and reports validation performance. A path threshold of $\tau = 0.90 provides the strongest balance: lowering it to 0.85 introduces less reliable prefixes, whereas increasing it to 0.95 discards useful supervision. The unsupervised loss weight shows a similar interior optimum at λu=1.0\lambda_u = 1.0; weaker weighting underuses dark spectra, while stronger weighting allows pseudo-label noise to dominate the labeled objective. Limiting pseudo-sequences to 32 residues also performs best, avoiding both premature truncation and unnecessary decoding of long, increasingly uncertain paths.

Effect of unlabeled-data scale. The final block of Table 3 evaluates subsets of DarkSpec. Increasing the corpus from 0.5 million to 4.5 million spectra raises peptide precision from 0.580 to 0.631 and curve AUC from 0.523 to 0.595. Performance improves at each scale, establishing broad acquisition coverage as part of the data contribution. Quality filtering ensures valid model inputs, while near-duplicate control prevents recurring spectra from dominating this additional coverage.

FactorValuePep. prec.Curve AUC
Path threshold τ\tau0.850.6210.572
0.900.6310.595
0.950.6280.588
Loss weight λu\lambda_u0.50.6290.586
1.00.6310.595
1.50.6210.577
Maximum pseudo length240.6290.586
320.6310.595
400.6300.590
Unlabeled spectra0.5M0.5800.523
1.0M0.6030.544
2.0M0.6260.582
4.5M0.6310.595

Table 3. Hyperparameter and unlabeled-data sensitivity on the Nine-species validation split.

Qualitative analysis. Figure 6 presents three Nine-species spectra for which SemiNovo recovers the complete peptide while its supervised counterpart fails. The first prediction replaces the massequivalent N-terminal pair VA with GL; the second changes the local order TAE to TEA without altering peptide composition; and the third moves deamidation from N to the adjacent Q. These alternatives satisfy nearly identical precursor-mass constraints, but the highlighted b- and y-ion evidence supports the SemiNovo sequence. The cases show that unlabeled-spectrum learning improves local fragment interpretation where precursor feasibility alone cannot distinguish competing candidates.

Figure 6. Three qualitative Nine-species cases. Each row places a compact tandem mass spectrum beside its sequence-level predictions. Gray sticks denote unassigned peaks; blue and orange sticks denote peaks matching theoretical b- and y-ions within 0.02 Da. Blue and red highlights mark the corresponding correct and erroneous sequence regions, respectively; lowercase residues denote their modified forms.

Conclusion

We study how search-conditioned supervision bias affects database-free de novo inference. SemiNovo addresses this bias using DarkSpec, mass-constrained cumulative-path pseudo-labeling, and chemical-evidence gating. Together, these components extend supervision beyond identified spectra and improve the use of sparse fragment evidence. Unlabeled spectra improve peptide precision on all three benchmarks, while SemiNovo achieves state-of-the-art results on Nine-species and HC-PT. The remaining sensitivity

on Seven-species highlights domain alignment as an important direction for learning from heterogeneous public spectra.

Limitations and Ethical Considerations

SemiNovo formulates learning beyond search-identified spectra as a semi-supervised problem, but cross-dataset results also expose a domain-transfer challenge. DarkSpec is dominated by high-resolution acquisitions, and Seven-species reveals sensitivity to instrument resolution, fragmentation, and peptide distribution. Future work should combine semi-supervision with domain-aware corpus selection and acquisition conditioning.

The pseudo-label generator retains one mass-constrained path; preserving alternative feasible sequences may improve recall but requires efficient sequence-level calibration. Broader post-translational modification vocabularies also remain future work. All experiments use public PRIDE and NovoBench data and involve no new human-subject data collection.

Generative AI Usage

Generative AI tools assisted code development and manuscript drafting, translation, and editing. All AI-assisted material was reviewed by the authors, who retain responsibility for the research design, implementation, experiments, results, references, and final manuscript.

References

DarkSpec Corpus Details

Source Scope

DarkSpec is constructed from 653 acquisition runs in 43 public PRIDE projects. The source inventory contains 44,963,379 MS/MS spectra before model-compatible quality control. Table 4 reports the contribution of every project; these counts are measured before filtering and redundancy control.

ProjectSpectraProjectSpectra
PXD0557482,059,318PXD071883251,927
PXD060117666,617PXD072146933,697
PXD060201918,864PXD072226467,463
PXD060819591,603PXD072402217,986
PXD0614381,239,989PXD07257988,232
PXD0616443,001,044PXD072877983,211
PXD0617621,116,153PXD0730765,228,025
PXD0618081,010,229PXD073471427,952
PXD061929814,172PXD0737021,536,478
PXD062081125,936PXD073864177,875
PXD069979900,831PXD0739853,959,311
PXD070097689,375PXD0741793,934,079
PXD0701161,442,805PXD074303258,239
PXD07035587,691PXD074339226,303
PXD070597818,049PXD0743881,604,360
PXD0707761,437,776PXD074405647,384
PXD071007598,821PXD074406523,248
PXD071138900,601PXD074452378,687
PXD0711981,212,351PXD074558346,772
PXD07131918,312PXD074667431,124
PXD0714661,531,235PXD075330189,170
PXD071667970,084

Table 4. Public PRIDE projects in the DarkSpec source pool.

Canonical Processing

We apply the same peak representation to labeled NovoBench spectra and unlabeled PRIDE spectra. For each MS/MS spectrum, nonfinite peaks and peaks outside [50.52564895,2500][50.52564895, 2500] m/zm/z are removed, as are peaks within 2 Da of the precursor. Intensities are first normalized by the base peak; peaks below 1% relative intensity are discarded. We retain the 150 most intense remaining peaks, sort them by m/zm/z, apply square-root intensity scaling, and renormalize by the base peak. Spectra are excluded when the precursor charge is outside [1,10][1, 10], peak arrays are malformed, or fewer than 20 valid peaks remain. Table 5 summarizes this reduction.

StageSpectra
Raw source pool44,963,379
Rejected: fewer than 20 valid peaks3,700,995
Rejected: invalid precursor charge166,308
Rejected: malformed peak arrays7
Quality-controlled pool41,096,069
Fixed training corpus4,500,000

Table 5. Quality control and experimental corpus size.

Redundancy Control and Storage

Pooling public projects can over-represent repeated exports and closely related acquisitions. We therefore assign each processed spectrum a near-duplicate key. Fragment m/zm/z values are rounded to integer bins and converted to a 32-bit SimHash using intensity-weighted random projections with a fixed seed. The key combines this spectral signature with precursor charge and precursor MH+^{+} rounded to 0.5 Da. One representative is retained per key. To bound repeated experimental cost, a fixed 4.5-million-spectrum subset of representatives is used for the reported training runs. The final corpus size is therefore a computational budget rather than an additional search-derived quality criterion.

Each retained example stores a 150 ×\times 2 array of processed m/zm/z–intensity pairs together with precursor m/zm/z and charge. No peptide sequence, database-search score, or search-derived confidence enters the unlabeled training records. The arrays are memory-mapped during training and converted to the same precursor and peak tensors used by the labeled NovoBench loader.

Population Interpretation and Benchmark Overlap

DarkSpec is acquisition-derived rather than search-failure-derived. Because membership does not depend on peptide assignment, it contains both spectra that could be identified by a suitable search and spectra that remain unidentified. DarkSpec consequently samples the acquisition-level spectrum population presented to a database-free model at inference. Instrument, sample, and acquisition differences remain visible, as quantified in Appendix E.

We audit data-level overlap between all 4.5 million DarkSpec spectra and every train, validation, and test split of the three NovoBench datasets. An exact match requires equality of the canonical 150 ×\times 2 peak array, precursor m/zm/z, and charge. We additionally flag conservative near-duplicate candidates using the same 32-bit SimHash, charge, and 0.5 Da precursor-mass key used for DarkSpec redundancy control. Table 6 shows no exact match in any split. Near-key candidates comprise 0.023–0.041% of the test spectra, whereas semi-supervision improves peptide precision by 2.4–5.3 percentage points; their frequency is therefore insufficient to explain the measured gains.

DatasetSplitSpectraExactNear-key (%)
Nine-speciesTrain499,4020220 (0.044)
Nine-speciesValidation28,57207 (0.025)
Nine-speciesTest27,142010 (0.037)
HC-PTTrain213,284058 (0.027)
HC-PTValidation25,71807 (0.027)
HC-PTTest26,53606 (0.023)
Seven-speciesTrain317,009077 (0.024)
Seven-speciesValidation17,74002 (0.011)
Seven-speciesTest17,09407 (0.041)

Table 6. Data-level overlap between DarkSpec and NovoBench. Near-key entries report candidate count and percentage of the benchmark split.

Implementation Details

Architecture and Optimization

Table 7 gives the configuration used for the reported experiments. The value of nlayersn_\mathrm{layers} is applied independently to the spectrum encoder and peptide decoder. Head-wise chemical-evidence gates are used in decoder causal self-attention. Isoleucine is normalized to leucine in target sequences, while oxidized methionine and deamidated asparagine and glutamine remain distinct vocabulary entries.

ComponentSetting
Encoder / decoder blocks9 / 9
Hidden / feed-forward width512 / 1,024
Attention heads8
Trainable parameters48.2M
Dropout / weight decay0.1 / 0.05
Maximum peaks / sequence length150 / 100
Precursor tolerance / isotope errors50 ppm / {0,1}\{0, 1\}
Optimizer / scheduleAdamW / warm-up and cosine decay
Numerical precisionbfloat16 mixed
Gradient clipping1.0
Supervised epochs / batch / LR30 / 32 / 5×10−45 \times10^{-4}
Semi-supervised epochs / batch pair6 / 512 + 512
Semi-supervised LR / warm-up10−310^{-3} / 2,000 steps
Teacher EMA decay / λu\lambda_u0.999 / 1.0
Path threshold / pseudo length0.9 / 32
Peak dropout / intensity jitter0.05 / 0.02
Random seed454

Table 7. Model and optimization configuration.

The weak unlabeled view uses canonical processing without stochastic perturbation. The strong view independently drops each valid peak with probability 0.05 and multiplies retained intensities by factors sampled uniformly from [0.98, 1.02]; precursor values and peak m/zm/z positions are unchanged. Supervised targets and accepted pseudo-prefixes use hard cross-entropy without label smoothing.

Efficient Attention and Software Environment

All spectrum-encoder self-attention, peptide-decoder causal self-attention, and encoder–decoder cross-attention layers use the official FlashAttention 2 kernels. Variable-length peak and peptide sequences are unpadded before attention, and fused dropout–residual–normalization kernels are used within each Transformer block. The proposed head-wise gate is applied only to decoder causal self-attention and multiplies each head output immediately before the standard output projection; the remaining attention computation is unchanged.

The reported experiments use Python 3.11, PyTorch 2.6, Lightning 2.1.4, CUDA 12, and FlashAttention 2.7.4.post1 on an NVIDIA H200 GPU. Optimization uses AdamW with default momentum coefficients (0.9, 0.999) and ϵ=10−8\epsilon= 10^{-8}. The learning rate is updated after every optimization step using linear warm-up followed by cosine decay. Training uses bfloat16 mixed precision, gradient clipping at 1.0, and cuDNN benchmarking. Data loaders use 16 workers, pinned memory, persistent worker processes, and a prefetch factor of two.

Epoch and Checkpoint Semantics

Supervised training starts from random initialization and follows the NovoBench protocol for 30 epochs with batch size 32. The checkpoint with the highest validation sequence accuracy is reported as SemiNovo-Sup. and used to initialize semi-supervised training.

For the second stage, one epoch is defined by the larger of the labeled and unlabeled datasets. Let Nℓ=∣Dℓ∣N_\ell= |D_\ell| and Nu=∣Du∣N_u = |D_u|. A paired dataset has length max⁡(Nℓ,Nu)\max(N_\ell,N_u); examples from the smaller split are cycled while the paired indices are shuffled every epoch. Labeled and unlabeled minibatches each contain 512 spectra. With Nu=4,500,000N_u = 4{,}500{,}000, one epoch contains 8,789 optimization steps, and the reported six-epoch run contains 52,734 updates. This convention visits every unlabeled spectrum once per semi-supervised epoch while retaining labeled supervision at every update.

For confidence-criterion ablations, each criterion selects its best threshold from {0.85,0.90,0.95}\{0.85, 0.90, 0.95\} on the validation split. Validation is performed every 1,000 updates. We retain only the checkpoint with the highest validation sequence accuracy and do not save a separate last checkpoint. Semi-supervised training uses an EMA teacher with decay 0.999; the online student is optimized, while teacher parameters are updated after each optimizer step.

Open-source Resources

The SemiNovo repository at https://github.com/grandOrgan/SemiNovo provides the training and inference code, experiment configurations, DarkSpec construction pipeline, and NovoBench evaluation scripts. The versioned DarkSpec release is hosted on Hugging Face at https://huggingface.co/datasets/PanLiu/DarkSpec. Its Dataset Card documents the source PRIDE projects, data schema, quality-control procedure, provenance manifest, license, and loading interface.

Mass-Constrained Pseudo-Label Implementation

At each pseudo-decoding step, control tokens that cannot be emitted are first masked. Candidate residues are then screened against the precursor upper bound, and EOS is permitted only when the generated neutral peptide mass matches the observed precursor within 50 ppm under isotope errors 0 or 1 and the peptide contains at least six residues. The selected residue is the highest-probability member of this feasible set.

The probability used by cumulative path confidence is taken from the original softmax over the complete vocabulary rather than renormalized over the feasible set. Path confidence is accumulated in log space, and decoding terminates after confidence rejection, invalid mass progression, valid EOS, or 32 generated positions.

The student is teacher-forced only on the accepted contiguous prefix. Unsupervised cross-entropy is normalized by the total number of accepted pseudo-tokens in the minibatch rather than by the number of unlabeled spectra, so spectra with no reliable residue contribute zero loss. The supervised and accepted pseudo-labeled sequences are padded together and evaluated in one student forward pass.

Evaluation Details

All reported metrics are computed with the official NovoBench sequence parser and mass-aware matcher. Let NmatchaaN^{\mathrm{aa}}_{\mathrm{match}} be the number of matched residues, NpredaaN^{\mathrm{aa}}_{\mathrm{pred}} the number of predicted residues, and NtrueaaN^{\mathrm{aa}}_{\mathrm{true}} the number of ground-truth residues. A residue match requires an individual mass difference below 0.1 Da and a corresponding prefix or suffix cumulative-mass difference below 0.5 Da. We report

AA-Prec.=NmatchaaNpredaa,AA-Rec.=NmatchaaNtrueaa.(11)\mathrm{AA\text{-}Prec.} = \frac{N^{\mathrm{aa}}_{\mathrm{match}}}{N^{\mathrm{aa}}_{\mathrm{pred}}}, \qquad\mathrm{AA\text{-}Rec.} = \frac{N^{\mathrm{aa}}_{\mathrm{match}}}{N^{\mathrm{aa}}_{\mathrm{true}}}. \tag*{(11)}

A peptide is correct only when every residue is matched under this rule. Peptide precision is the number of completely matched peptides divided by the number of test spectra, including spectra for which no sequence is returned. PTM precision is computed analogously over predicted modified-residue tokens. Curve AUC is obtained by sorting predictions by their model confidence and integrating the resulting peptide precision–coverage curve. The same evaluator, residue masses, modification vocabulary, and confidence aggregation are used for every compared method.

The ablation and threshold analyses use Nine-species with seed 454. Each component ablation changes only the named factor and otherwise retains the complete model configuration. For threshold analysis, validation spectra are treated as unlabeled when measuring pseudo-prefix precision, accepted-token coverage, and mean accepted-prefix length; test labels are never used for threshold selection.

Supplementary Experiments

Statistical Stability

We repeat the complete Nine-species experiment with three random seeds while retaining the same data, optimization schedule, and model-selection criterion. Table 8 reports a mean peptide precision of 0.632 and a mean curve AUC of 0.589. Across runs, the standard deviations are 0.009 and 0.005, respectively, and every run remains above the supervised checkpoint.

Random seedPep. precisionCurve AUC
4540.6350.593
2,0260.6220.584
1,1450.6390.591
Mean ± std.0.632 ± 0.0090.589 ± 0.005

Table 8. Statistical stability on Nine-species across three random seeds.

Controlling for Additional Optimization

Semi-supervised training introduces additional optimization steps after the supervised checkpoint. We isolate the contribution of unlabeled spectra by continuing supervised training for the same number of updates, with the same labeled batches, learning-rate schedule, and model-selection protocol. As shown in Table 9, continued supervised optimization yields only a small improvement, whereas adding DarkSpec produces a substantially larger gain. This result attributes the improvement to unlabeled-spectrum learning rather than training duration alone.

Training settingPep. precisionCurve AUC
Supervised checkpoint0.5820.538
Supervised continuation0.5870.544
SemiNovo0.6350.593

Table 9. Matched-budget training control on Nine-species.

Pseudo-Label Reliability

The matched generic SSL control retains the same teacher–student views, EMA, unlabeled data, optimization budget, and model initialization as SemiNovo, changing only cumulative path selection to mean-token confidence. At full coverage, exact-peptide pseudo-label accuracy on the 28,572 Nine-species validation spectra is 56.78%. Table 10 complements the error–coverage curves in Figure 5 with accuracy at representative coverage levels.

Confidence criterionAUCAcc.@20%Acc.@40%Acc.@60%
Cumulative path0.853197.6592.9985.89
Mean token0.849395.5092.7585.80
Minimum token0.850497.3492.9385.45

Table 10. Pseudo-label ranking quality on the Nine-species validation split.

Semi-Supervised Training Dynamics

Figure 7 tracks a Nine-species semi-supervised run initialized from the best supervised checkpoint. As the teacher adapts to unlabeled spectra, reliable prefixes become longer, while complete-sequence acceptance rises and then fluctuates because longer paths are more likely to contain a position that violates the cumulative confidence threshold. Validation sequence accuracy improves during the initial expansion of reliable pseudo-supervision, but declines after the midpoint even as pseudo-label coverage continues to grow. This divergence exposes a quantity–quality trade-off: increasing teacher confidence admits more and longer sequences, yet confidence growth does not eliminate domain-dependent pseudo-label errors, which are progressively reinforced through autoregressive training.

Figure 7. Training dynamics for the Nine-species semi-supervised run initialized from the best supervised checkpoint. Reliable-prefix length, complete-sequence acceptance, and validation sequence accuracy follow distinct trajectories as the teacher adapts to 4.5 million unlabeled spectra.

Computational Efficiency

We benchmark GPU-resident computation on a single NVIDIA H200 with a batch size of 512 for both training and inference. Throughput is computed from the median stable step time, and peak memory denotes the maximum allocated GPU memory. For semi-supervised training, labeled throughput counts the 512 labeled spectra in each update, whereas total throughput additionally counts the 512 unlabeled spectra processed by the teacher and student.

MethodParamsLabeled/sTotal/sPeak memory
Casanovo47.35M3,513.93,513.914.60 GiB
SemiNovo (supervised)48.17M7,390.17,390.18.69 GiB
SemiNovo (semi-supervised)48.17M1,404.52,808.912.23 GiB

Table 11. Single-H200 training efficiency at batch size 512.

SemiNovo (supervised) reaches 2.10× the training throughput of Casanovo while using 40.5% less peak memory. Semi-supervised updates are slower because each step performs pseudo-label generation and optimization over both labeled and unlabeled spectra. The head-wise chemical-evidence gates introduce only 36,864 parameters, corresponding to 0.0765% of the complete model.

MethodSearchSpectra/sPeak memory
CasanovoOriginal6.794.81 GiB
SemiNovo (supervised)Casanovo-style93.153.04 GiB
SemiNovo (supervised)Mass-pruned611.893.04 GiB
SemiNovoCasanovo-style110.083.04 GiB
SemiNovoMass-pruned606.463.04 GiB

Table 12. Single-H200 inference efficiency at batch size 512.

Mass-feasibility pruning accelerates SemiNovo decoding by 6.57× for the supervised checkpoint and 5.51× for the complete model, without increasing peak memory. The optimized path processes approximately 89× more spectra per second than the original Casanovo implementation. The supervised and semi-supervised checkpoints share the same inference graph; pseudo-label generation introduces no deployment-time module.

Pseudo-label searchPseudo/sFull step/sMemory
Fixed-length greedy1,516.02,309.412.89 GiB
Confidence early stopping2,187.73,092.012.24 GiB
Early stopping + mass pruning2,160.13,046.212.24 GiB

Table 13. Efficiency of pseudo-label search at batch size 512.

Confidence-based early stopping provides the dominant pseudo-labeling speedup, improving search throughput by 44.3% and complete-step throughput by 33.9% over fixed-length greedy decoding. Adding mass-feasibility pruning retains nearly identical throughput, indicating that its primary role in pseudo-label generation is to prevent chemically infeasible continuations rather than to reduce the measured runtime.

Dataset Distribution Analysis

We compare DarkSpec with the labeled training splits of the three NovoBench datasets using 20,000 spectra sampled from each corpus. All statistics are computed after the canonical preprocessing in Appendix A. Spectral entropy is defined as −∑ipilog⁡pi-\sum_i p_i \log p_i, where pip_i is the normalized intensity of peak ii. Table 14 reveals substantial acquisition-level differences. DarkSpec contains heavier precursors and a markedly larger proportion of charge-three-or-higher ions than any benchmark. Its retained-peak distribution is denser than Nine-species and HC-PT, while Seven-species exhibits the highest spectral entropy. These differences confirm that unlabeled training exposes the model to spectral conditions that are not represented by a simple resampling of any individual NovoBench dataset.

DatasetPrecursor m/zm/zz≥3z \ge3 (%)PeaksEntropy
DarkSpec770.959.71504.55
Nine-species650.839.7894.32
HC-PT598.329.71094.55
Seven-species688.836.51504.88

Table 14. Distribution statistics after canonical preprocessing. Values are computed from 20,000 randomly sampled spectra per dataset.

References

  1. [1]Jun A, P. Liu, Yingying Sun, Jiaying Lin, Xiaofan Zhang, Zongxiang Nie, Jingnan Liu, Zhiguo Yu, Yuqi Zhang, Ziyuan Xing, Yi Chen, and Tiannan Guo. 2026. DDA-BERT: End-to-End Training for Data-Dependent Acquisition Mass Spectrometry-Based Proteomics. Nature Communications 17 (2026), 5752. doi:10.1038/s41467-026-72246-6
  2. [2]Ruedi Aebersold and Matthias Mann. 2003. Mass Spectrometry-Based Proteomics. Nature 422, 6928 (2003), 198–207.pubmed.ncbi.nlm.nih.gov/12634793
  3. [3]Ruedi Aebersold and Matthias Mann. 2016. Mass-spectrometric exploration of proteome structure and function. Nature 537, 7620 (2016), 347–355.DOI
  4. [4]Wout Bittemieux, Damon H. May, Jeffrey Bilmes, and William Stafford Noble. 2022. A Learned Embedding for Efficient Joint Analysis of Millions of Mass Spectra. Nature Methods 19, 6 (2022), 675–678. doi:10.1038/s41592-022-01496-1
  5. [5]Yelysei Bondarenko, Markus Nagel, and Tijmen Blankevoort. 2023. Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothing. In Advances in Neural Information Processing Systems, Vol. 36. 75067–75096.arxiv.org/abs/2306.12929
  6. [6]Moses S. Charikar. 2002. Similarity Estimation Techniques from Rounding Algorithms. In Proceedings of the 34th Annual ACM Symposium on Theory of Computing. 380–388. doi:10.1145/509907.509965DOI
  7. [7]Shaorong Chen, Jingbo Zhou, and Jia Zhou et al. 2025. ReNovo: Retrieval-Based De Novo Mass Spectrometry Peptide Sequencing. In International Conference on Learning Representations.
  8. [8]Shaorong Chen, Jingbo Zhou, and Jun Xia. 2026. Regressor-guided Diffusion Model for De Novo Peptide Sequencing with Explicit Mass Control. In Proceedings of the AAAI Conference on Artificial Intelligence.
  9. [9]Jürgen Cox and Matthias Mann. 2008. MaxQuant Enables High Peptide Identification Rates, Individualized p.b.-Range Mass Accuracies and Proteome-Wide Protein Quantification. Nature Biotechnology 26, 12 (2008), 1367–1372. doi:10.1038/nbt.1511
  10. [10]Kevin Eloff, Konstantinos Kalogeropoulos, Oliver Morell, et al. 2025. In-Stano Enables Diffusion-Powered De Novo Peptide Sequencing in Large-Scale Proteomics Experiments. Nature Machine Intelligence 7, 4 (2025), 565–579. doi:10.1038/s42256-025-01019-5
  11. [11]Ari Frank and Pavel Pevzner. 2005. PepNovo: De Novo Peptide Sequencing via Probabilistic Network Modeling. Analytical Chemistry 77, 4 (2005), 964–973.pubmed.ncbi.nlm.nih.gov/15858974
  12. [12]Ari M. Frank, Nuno Bandeira, Zhaoxin Shen, Stephen Tanner, Steven P. Briggs, Richard D. Smith, and Pavel A. Pevzner. 2008. Clustering Millions of Tandem Mass Spectra. Journal of Proteome Research 7, 1 (2008), 113–122. doi:10.1021/pr070361eDOI
  13. [13]Philipp E. Geyer, Lesca M. Holdt, Daniel Teupser, and Matthias Mann. 2017. Revisiting Biomarker Discovery by Plasma Proteomics. Molecular Systems Biology 13, 9 (2017), 942.DOI
  14. [14]Johannes Griss, Yasset Perez-Riverol, Steve Lewis, David L. Tabb, José A. Di-anes, Noemi Del-Toro, Marc Rurik, Mathias Walzer, Oliver Kolhbacher, Henning Hermjakob, Rui Wang, and Juan Antonio Vizcaíno. 2016. Recognizing Millions of Consistently Unidentified Spectra across Hundreds of Shotgun Proteomics Datasets. Nature Methods 13, 8 (2016), 651–656. doi:10.1038/nmeth.3902DOI
  15. [15]Xinxiang Gu, Tianyu Pang, Chao Du, et al. 2024. When Attention Sink Emerges in Language Models: An Empirical View. arXiv preprint arXiv:2410.10781 (2024).
  16. [16]Sangtae Kim and Pavel A. Pevzner. 2014. MS-GF+ Makes Progress towards a Universal Database Search Tool for Proteomics. Nature Communications 5 (2014), 5277. doi:10.1038/ncomms6277
  17. [17]Andrey T. Konein, Felipe V. Leprevost, Dmitry M. Avtonomov, Dattatreya Mel-lacheruvu, and Alexey I. Nesvizhskii. 2017. MSFragger: Ultrafast and Comprehensive Peptide Identification in Mass Spectrometry-Based Proteomics. Nature Methods 14, 5 (2017), 513–520. doi:10.1038/nmeth.4256
  18. [18]Levi I. Levitsky, Mark V. Ivanov, Anna A. Lobas, and Mikhail V. Gorshkov. 2017. Unbiased False Discovery Rate Estimation for Shotgun Proteomics Based on the Target–Decoy Approach. Journal of Proteome Research 16, 2 (2017), 393–397. doi:10.1021/acs.jproteome.6b00144
  19. [19]Kaiyuan Liu, Yuzhen Ye, Sujun Li, and Haixiu Tang. 2023. Accurate De Novo Peptide Sequencing using Fully Convolutional Neural Networks. Nature Communications 14 (2023), 7974. doi:10.1038/s41467-023-43010-x
  20. [20]Pan Liu and Jinshi Liu. 2025. When Confidence Fails: Revisiting Pseudo-Label Selection in Semi-supervised Semantic Segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 21874–21884.arxiv.org/abs/2509.16704
  21. [21]Zeping Mao, Ruixue Zhang, Lei Xin, and Ming Li. 2023. Mitigating the Missing-Fragmentation Problem in De Novo Peptide Sequencing with a Two-Stage Graph-Based Deep Learning Model. Nature Machine Intelligence 5 (2023), 1250–1260. doi:10.1038/s42256-023-00738-xDOI
  22. [22]Gaurav Patel, Jan Allebach, and Qiang Qiu. 2023. Seq-UPS: Sequential Uncertainty-Aware Pseudo-Label Selection for Semi-Supervised Text Recognition. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 6180–6190.arxiv.org/abs/2209.00641
  23. [23]Yasset Perez-Riverol et al. 2022. The PRIDE Database Resources in 2022: A Hub for Mass Spectrometry-Based Proteomics Evidences. Nucleic Acids Research 50, D1 (2022), D543–D552. doi:10.1093/nar/gkab1038
  24. [24]Rui Qiao, Ngoc Hieu Tran, Lei Xin, Xin Chen, Ming Li, Baozhen Shan, and Ali Ghodsi. 2021. Computationally instrument-resolution-independent de novo peptide sequencing. In International Conference on Learning Representations.DOI
  25. [25]Zihan Qiu, Zekun Wang, Bo Zheng, Zeyu Huang, Kaiyue Wen, Songlin Yang, Rui Men, Le Yu, Fei Huang, Suozhi Huang, Dayiheng Liu, Jingren Zhou, and Junyang Lin. 2025. Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free. In Advances in Neural Information Processing Systems, Vol. 38.arxiv.org/abs/2505.06708
  26. [26]Jason Ramapuram, Federico Danieli, Eshan Dhekhane, Floris Weers, Dan Bus-bridge, Pierre Ablin, Tatiana Likhomanenko, Jagrit Digani, Zijin Gu, Amitis Shidani, and Russ Webb. 2025. Theory, Analysis, and Best Practices for Sigmoid Self-Attention. In International Conference on Learning Representations.arxiv.org/abs/2409.04431
  27. [27]Kihyuk Sohn, David Berthelot, Nicholas Carlini, et al. 2020. FixMatch: Simplifying Semi-supervised learning with consistency and confidence. In Advances in Neural Information Processing Systems.
  28. [28]Mingjie Sun, Xinlei Chen, J. Zico Kolter, and Zhuang Liu. 2024. Massive Activations in Large Language Models. arXiv preprint arXiv:2402.17762 (2024).arxiv.org/abs/2402.17762
  29. [29]Antti Tarvainen and Harri Valpola. 2017. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. In Advances in Neural Information Processing Systems.arxiv.org/abs/1703.01780
  30. [30]Ngoc Hieu Tran, Xianglian Zhang, Lei Xin, Baozhen Shan, and Ming Li. 2017. De novo peptide sequencing by deep learning. Proceedings of the National Academy of Sciences 114, 31 (2017), 8247–8252.DOI
  31. [31]Yidong Wang, Hao Chen, Yue Fan, et al. 2022. USB: A Unified Semi-supervised Learning Benchmark for Classification. In Advances in Neural Information Processing Systems.arxiv.org/abs/2208.07204
  32. [32]Jun Xia, Shaorong Chen, Jingbo Zhou, Xiaojun Sun, Wei Du, Zhangyang Gao, Cheng Tan, Bozhen Hu, Jie Zheng, and Stan Z. Li. 2024. AdaNovo: Towards Robust De Novo Peptide Sequencing in Proteomics against Data Biases. In Advances in Neural Information Processing Systems, Vol. 37. 1811–1828.
  33. [33]Jun Xia, Sizhe Liu, Jingbo Zhou, Shaorong Chen, Huafeng Xiang, Ziqing Liu, Yizhou Liu, and Stan Z. Li. 2024. Bridging the Gap between Database Search and De Novo Peptide Sequencing with SearchNovo. bioRxiv (2024).
  34. [34]Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. 2024. Efficient Streaming Language Models with Attention Sinks. In International Conference on Learning Representations.arxiv.org/abs/2309.17453
  35. [35]Tingpeng Yang, Tianze Ling, Boyan Sun, et al. 2024. Introducing π-HelixNovo for practical large-scale de novo peptide sequencing. Briefings in Bioinformatics 25, 2 (2024), bbae021. doi:10.1093/bib/bbae021
  36. [36]Melih Yilmaz, William E. Fondrie, Wout Bittemieux, Carlo F. Melendez, Rowan Nelson, Varun Nanthan, Sewoong Oh, and William Stafford Noble. 2024. Sequence-to-Sequence Translation from Mass Spectra to Peptides with a Transformer Model. Nature Communications 15 (2024), 6427. doi:10.1038/s41467-024-49731-x
  37. [37]Melih Yilmaz, William E. Fondrie, Wout Bittemieux, Sewoong Oh, and William Stafford Noble. 2022. De novo mass spectrometry peptide sequencing with a transformer model. In Proceedings of the 39th International Conference on Machine Learning. 25514–25522.DOI
  38. [38]Yonghan Yang and Ming Li. 2025. Towards Highly Sensitive Deep Learning-Based End-to-End Database Search for Tandem Mass Spectrometry. Nature Machine Intelligence 7 (2025), 85–95. doi:10.1038/s42256-024-00960-1DOI
  39. [39]Zhongzhi Yu, Zheng Wang, Yonggan Fu, Huihong Shi, Khalid Shaikh, and Yingyan Chen Lin. 2024. Unveiling and Harnessing Hidden Attention Sinks: Enhancing Large Language Models without Training through Attention Calibration. In Proceedings of the 41st International Conference on Machine Learning, Vol. 235. 57659–57677.arxiv.org/abs/2406.15765
  40. [40]Bowen Zhang, Yidong Wang, Wenxin Hou, Hao Wu, Jindong Wang, Manabu Okumura, and Takahiro Shinozaki. 2021. FlexMatch: Bootstrapping Semi-Supervised Learning with Curriculum Pseudo Labeling. In Advances in Neural Information Processing Systems, Vol. 34.
  41. [41]Xiang Zhang, Tianze Ling, Zhi Jin, Sheng Xu, Zhiqiang Gao, Boyan Sun, Zijie Qiu, Jiaqi Wei, Nanqing Dong, Guangshuai Wang, et al. 2025. π-PrimeNovo: An Accurate and Efficient Non-Autoregressive Deep Learning Model for De Novo Peptide Sequencing. Nature Communications 16 (2025), 267. doi:10.1038/s41467-024-55021-3
  42. [42]Yaoyang Zhang, Bryan R. Fonslow, Baohai Shan, Min-Chul Baek, and John R. III Yates. 2013. Protein Analysis by Shotgun/Bottom-up Proteomics. Chemical Reviews 113, 4 (2013), 2343–2394.
  43. [43]Jingbo Zhou, Shaorong Chen, Jun Xia, et al. 2024. NovoBench: Benchmarking Deep Learning-based De Novo Peptide Sequencing Methods in Proteomics. In Advances in Neural Information Processing Systems, Vol. 37. doi:10.52202/070917-3327arxiv.org/abs/2406.11906

Paper details

vlnc:W_y6ca6v5s
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Generative AI tools

View statement
Looking for related papers…
Contents