WorksComputer Science
LBSNet: Lightweight Brain State Modeling via Learnable Spectral Prototypes for EEG-Based Fatigue Detection
Abstract
EEG-based fatigue detection requires modeling spectral information across bands and its temporal variation. Recent methods combine multi-band modeling with adaptive temporal processing, yet how spectral structure across bands should influence the contribution of each temporal interval remains insufficiently explored. We propose LBSNet, a lightweight network that uses prototype-based brain-state modeling to link local spectral structure with temporal aggregation. LBSNet extracts compact multi-band features through band-specific temporal convolution, shared spatial projections, and cross-band recalibration. Within each window, relative spectral activity is encoded as a normalized descriptor and softly matched to learnable spectral prototypes. Prototype assignments and learned prototype importance jointly determine temporal weights for power aggregation, allowing the spectral pattern of each window to guide its contribution to the prediction. Subject-independent experiments on SEED-VIG and SADT show that LBSNet outperforms seven baselines while maintaining low computational cost. Ablation and visualization analyses support prototype-guided temporal weighting, while the learned prototypes reveal a gradual redistribution of relative cross-band spectral states with increasing fatigue. The code is publicly available at https://github.com/zy0929/LBSNet.
Introduction
Driver fatigue impairs vigilance, sustained attention, and responsiveness, increasing the risk of driving errors and accidents [1, 2]. Electroencephalography (EEG) provides a noninvasive measure of brain activity and has been widely used for objective fatigue assessment [3, 4]. Prior studies have consistently linked fatigue and vigilance to changes in spectral activity across canonical EEG bands, and these spectral characteristics vary over the course of driving [5, 6]. EEG-based fatigue detection therefore requires modeling both fatigue-related spectral activity and its temporal variation.
Early EEG-based fatigue studies combined spectral features with temporal modeling, with Zheng and Lu [6] incorporating temporal dependencies into vigilance estimation using frequency-specific features. Subsequent convolutional approaches moved toward end-to-end learning. InterpretableCNN [7] learns fatigue-related features for cross-subject drowsiness recognition and reveals alpha spindles through interpretation analysis. Attention-based methods further strengthened spectral-temporal modeling. SACC-CapsNet [8] emphasizes temporal components and channels, while TFormer [9] integrates temporal and frequency-domain patterns through cross-attention. Recent models combine frequency-specific modeling with adaptive temporal processing. Multi-band modeling and temporal weighting capture different aspects of fatigue-related EEG, but existing designs do not explicitly link the local spectral structure of an interval to its temporal contribution [10, 11]. Overall, EEG-based fatigue detection has progressed toward representation learning and adaptive modeling across frequency and time.
Multi-band modeling and temporal weighting capture different aspects of fatigue-related EEG, but they do not define how the spectral structure of an interval should influence its temporal contribution [10, 11]. This issue is important because fatigue has been characterized not only by changes within individual frequency bands, but also by relative band power and relationships among bands [12]. Accordingly, fatigue-related information within a local interval may be reflected in joint patterns across frequency bands [13]. This raises the question of whether temporal aggregation can be guided by such local spectral patterns. Grouping similar local spectral patterns into shared states and learning their relevance to fatigue would provide a link between spectral structure and temporal weighting.
To this end, we propose LBSNet, a lightweight network for EEG-based fatigue detection. Its central idea is prototype-based brain-state modeling, which links local patterns of relative activity across frequency bands to temporal weighting. LBSNet first decomposes EEG into five canonical frequency bands, applies band-specific temporal convolution, and uses spatial projections shared across bands to obtain compact features. Cross-band recalibration then adjusts the relative contribution of the five bands based on global spectral activity. Within each temporal window, envelope-based band activity is transformed into a normalized relative spectral descriptor and softly matched to the learned prototypes. The resulting prototype assignments are combined with learned prototype importance to derive temporal weights, which are used to aggregate component-wise power into a compact representation for classification. The main contributions are summarized as follows:
We introduce prototype-based brain-state modeling that represents local relative spectral patterns with shared learnable prototypes and combines prototype assignments with learned importance to guide temporal aggregation.
We develop a lightweight architecture with shared spatial projections, low-dimensional state matching, and a compact classifier, achieving a favorable accuracy-efficiency trade-off with a small parameter budget and low computational cost.
We validate LBSNet on SEED-VIG and SADT and show that the learned spectral states exhibit a gradual redistribution with increasing fatigue, providing empirical support for prototype-guided temporal weighting.
Method
As illustrated in Figure 1, LBSNet comprises a spatiotemporal encoder, cross-band recalibration, and state-guided temporal aggregation. Starting from multichannel EEG signals, LBSNet extracts spatiotemporal features from five frequency bands, recalibrates their relative contributions, and uses learnable spectral prototypes to guide temporal aggregation, enabling lightweight modeling of latent brain states for EEG-based fatigue classification.
Figure 1. Overview of LBSNet, where learnable spectral prototypes represent local relative spectral patterns and guide temporal aggregation.
Spatiotemporal Encoder
The spatiotemporal encoder extracts five-band EEG features through temporal and shared spatial convolutions. For an input , with electrodes and samples, a fixed filter bank decomposes the signal into five frequency bands:
where applies Butterworth filtering independently to each electrode, yielding the band signal . The passbands are 1–4, 4–8, 8–14, 14–31, and 31–51 Hz.
Each band undergoes temporal convolution:
where indexes electrodes, the colon denotes the temporal sequence, and is a band-specific kernel shared across electrodes. These bias-free convolutions use kernel lengths (125, 63, 31, 15, 7) at 128 Hz and proportionally adjusted odd lengths at other sampling rates. Symmetric zero padding preserves the temporal length, yielding the temporally filtered signals .
Before spatial convolution, the signals are centered across electrodes and the spatial convolution kernels are normalized:
where contains the centered signals, indexes electrodes, and indexes samples. The vector is the th spatial convolution kernel, and is the number of output features per band. The operator removes the kernel mean and applies normalization.
Shared spatial convolution then produces
where is the coefficient for electrode in normalized kernel , and is its response in band at sample . The same kernels span all electrodes and are shared across bands. The output retains the band dimension and is passed to cross-band recalibration for joint band-weight estimation.
Cross-Band Recalibration
For the band-specific encoder output , cross-band recalibration first computes global log-power descriptors for the five frequency bands:
where is the global log-power of band , obtained by averaging power over spatial features and temporal samples and then taking the logarithm. The descriptors form .
Band weights are jointly estimated as
where is a two-layer MLP with an intermediate ReLU, and contains five positive weights summing to one. Each weight is conditioned on the descriptors of all five bands.
The encoder features are then recalibrated:
where is the weight for band , shared across its spatial features and temporal samples. Square-root scaling weights the corresponding power by . The recalibrated output is passed to state-guided temporal aggregation.
State-Guided Temporal Aggregation
Following cross-band recalibration, state-guided temporal aggregation weights temporal windows according to their latent spectral patterns. The recalibrated features are divided into non-overlapping windows of samples. Band activity within each window is computed from the analytic signal as
where contains the sample indices of window , denotes the Hilbert transform along the full temporal sequence, and measures the activity of band in that window.
The window activity vector is transformed into a normalized spectral descriptor that captures relative activity across bands:
where , the logarithm is applied elementwise, and performs mean removal and normalization across bands. This emphasizes relative spectral patterns and reduces sensitivity to variations in the overall magnitude of the EEG signal. Each window descriptor is matched against a set of learnable spectral prototypes to obtain soft state assignments:
where is prototype , is its normalized form, is the soft assignment of window to that prototype, and controls assignment sharpness. The prototypes represent latent spectral patterns learned jointly with the network.
The assignments are combined with learnable prototype importance to obtain temporal weights:
where denotes the importance of prototype , controls temporal weighting sharpness, and is the weight of window , with . Thus, each window contributes according to both its state assignments and the learned importance of those states.
The temporal weights are then used to aggregate squared recalibrated features into a weighted log-power representation:
where is the weighted log-power of spatial feature in band . The same temporal weights are shared across bands and spatial features. The matrix is flattened into a -dimensional representation and passed to a three-layer classifier with hidden dimensions 32 and 16. ReLU and dropout follow each hidden layer, and the final linear layer produces the class logits.
Experiments and Results
Datasets and Baselines
We evaluate our method on two EEG benchmarks for driver fatigue detection: SEED-VIG [6] and SADT [14]. SEED-VIG contains EEG recordings from 23 subjects during simulated driving. We retain 17 channels, downsample the signals from 1,000 to 200 Hz, and segment them into 20,355 non-overlapping 8-s samples. Vigilance is annotated using PERCLOS derived from eye-tracking recordings, with 0.35 as the threshold between alert and fatigued states. SADT is a balanced subset [14] of the sustained-attention driving dataset [15], comprising 2,022 3-s samples from 11 subjects at 128 Hz. We use all 30 EEG channels. The provided alert and drowsy labels were derived from local and global reaction-time criteria, considering both the current reaction time and recent response history.
For comparison, we select seven representative baselines. The convolutional baselines include EEGNet [16] and TCNet [17]. The attention-based baselines include Conformer [18], Deformer [19], BiConformer [20], DSAINet [21], and MB-STFormer [11].
Experimental Setup
Experiments are conducted on a server equipped with eight NVIDIA GeForce RTX 4090 GPUs. We use subject-wise five-fold cross-validation, where all samples from each subject are assigned to a single fold; three folds are used for training, one for validation, and one for testing. The model with the highest validation accuracy is selected for final evaluation. We set the temporal window length of LBSNet to 0.5 s for both datasets, with denoting the corresponding number of samples. The number of spectral prototypes is set to for SADT and for SEED-VIG, while the number of spatial features per band is set to and , respectively. All experiments are repeated with five random seeds, and the results are reported as mean standard deviation across seeds.
Performance Comparison
Table 1 compares the performance of LBSNet with seven baselines on SEED-VIG and SADT. LBSNet achieves the highest accuracy and F1-score on both datasets, with larger gains observed on SADT. Compared with the strongest baseline MB-STFormer, LBSNet improves accuracy/F1-score by 1.26/1.21 percentage points on SEED-VIG and 2.68/2.77 percentage points on SADT.
| Method | SEED-VIG | SADT | ||
| Accuracy | F1-score | Accuracy | F1-score | |
| EEGNet | ||||
| TCNet | ||||
| Conformer | ||||
| Deformer | ||||
| BiConformer | ||||
| DSAINet | ||||
| MB-STFormer | ||||
| LBSNet |
Table 1. Performance comparison (%) on SEED-VIG and SADT.
Ablation Analysis
To assess the components of LBSNet, we examine the role of state-guided temporal aggregation, whether prototype matching offers benefits beyond learnable temporal weighting alone, and the contribution of cross-band recalibration (CBR). As shown in Table 2, replacing the learned temporal weights with Uniform Pooling reduces accuracy/F1-score by 2.48/2.90 points on SEED-VIG and 1.78/0.81 points on SADT, showing the benefit of state-guided temporal aggregation. Replacing prototype matching with parameter-matched MLP Attention also reduces performance. Compared with this variant, LBSNet improves accuracy/F1-score by 2.10/3.50 points on SEED-VIG and 1.19/1.31 points on SADT. MLP Attention itself yields only small accuracy gains over Uniform Pooling and lower F1-scores on both datasets, indicating that learnable temporal weighting alone does not explain the improvement. Finally, removing CBR reduces both accuracy and F1-score on both datasets, showing that cross-band recalibration also contributes to the performance.
| Method | SEED-VIG | SADT | ||
| Accuracy | F1-score | Accuracy | F1-score | |
| Uniform Pooling | ||||
| MLP Attention | ||||
| w/o CBR | ||||
| LBSNet |
Table 2. Ablation results (%) on SEED-VIG and SADT.
Computational Efficiency Analysis
We compare models on SADT in terms of accuracy, multiply-accumulate operations (MACs), and trainable parameters. Figure 2 shows that LBSNet achieves the highest mean accuracy of 73.74% while requiring only 2.89 M MACs and 1.30 K trainable parameters, both the lowest among the compared models. Compared with EEGNet, the lightest baseline, LBSNet reduces MACs by 52.9% and the parameter count by 34.0%, while achieving higher accuracy. Compared with MB-STFormer, the strongest baseline, LBSNet improves accuracy by 2.68 percentage points while reducing MACs by 89.9% and the parameter count by 98.3%. The efficiency advantage is therefore reflected in both model size and computational cost. More importantly, the accuracy improvement is achieved without increasing model capacity or computation, showing that LBSNet does not rely on model scaling for better performance.
Figure 2. Comparison of model accuracy and computational efficiency on SADT. Each bubble corresponds to a model. The x-axis shows MACs (M) on a logarithmic scale, while the y-axis reports ACC (%). Bubble area reflects the logarithm of the number of trainable parameters, and the labels give the corresponding parameter counts.
Temporal Weighting Analysis
We examine how temporal weighting affects the aggregated features on SADT. In the shuffled setting, learned weights are randomly reassigned across temporal windows while their values remain unchanged. As shown in Figure 3, alert and fatigue samples overlap substantially under shuffled weights, whereas learned weights yield clearer class separation. This difference shows that temporal weight allocation affects the aggregated features, consistent with the ablation results showing better classification with learned temporal aggregation. Together, these results support temporal weighting in forming more discriminative representations.
Figure 3. t-SNE visualization of representations produced by temporal aggregation on SADT. (a) Shuffled weights. (b) Learned weights.
Spectral Prototype Analysis
We further analyze the learned spectral prototypes on SEED-VIG. As shown in Figure 4(a), P1 has positive components in the theta, alpha, and beta bands and negative components in the delta and gamma bands, whereas P2 exhibits an approximately opposite pattern. These differences indicate that the prototypes capture distinct spectral configurations across frequency bands. Figure 4(b) relates these configurations to vigilance state, with P1 accounting for 65.6% of the assignments in fatigue samples and P2 accounting for 71.6% in alert samples. This association is also reflected in the continuous PERCLOS analysis in Figure 4(c), where the proportion of P1 generally increases as PERCLOS rises, while that of P2 decreases. Taken together, these results suggest that increasing fatigue is accompanied by a gradual redistribution between relative cross-band spectral states. This finding supports the use of prototype-guided temporal aggregation for fatigue modeling.
Figure 4. Analysis of learned spectral prototypes on SEED-VIG. (a) Relative spectral patterns of the learned prototypes across the five frequency bands. (b) Prototype proportions for alert and fatigue samples. (c) Changes in prototype proportions across PERCLOS values. The dashed line marks the 0.35 threshold between alert and fatigue, and the histogram shows the sample distribution.
Conclusion
This work introduces LBSNet to connect local relative spectral patterns with temporal aggregation through learnable spectral prototypes, allowing spectral states to guide temporal weighting. Experiments on SEED-VIG and SADT show LBSNet outperforms all baselines while maintaining low computational cost. Learned prototypes reveal a gradual redistribution of relative cross-band spectral states with increasing fatigue, suggesting that joint spectral-state modeling offers a useful perspective for EEG-based fatigue assessment.
References
References
- [1]Daniel Mollicone, Kevin Kan, Chris Mott, Rachel Bartels, Steve Bruneau, Matthew van Wollen, Amy R Sparrow, and Hans PA Van Dongen, “Predicting performance and safety based on driver fatigue,” Accident Analysis & Prevention, vol. 126, pp. 142–145, 2019.DOI
- [2]A Gibbings, LB Ray, S Gagnon, CA Collin, R Robillard, and SM Fogel, “The EEG correlates and dangerous behavioral consequences of drowsy driving after a single night of mild sleep deprivation,” Physiology & Behavior, vol. 252, pp. 113822, 2022.DOI
- [3]Gianluca Borghini, Laura Astolfi, Giovanni Vecchiato, Donatella Mattia, and Fabio Babiloni, “Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness,” Neuroscience & Biobehavioral Reviews, vol. 44, pp. 58–75, 2014.
- [4]Raed Mohammed Hussein, Firas Sabar Miften, and Loay E George, “Driver drowsiness detection methods using EEG signals: a systematic review,” Computer methods in biomechanics and biomedical engineering, vol. 26, no. 11, pp. 1237–1249, 2023.DOI
- [5]Yvonne Tran, Ashley Craig, Rachel Craig, Rifai Chai, and Hung Nguyen, “The influence of mental fatigue on brain activity: Evidence from a systematic review with meta-analyses,” Psychophysiology, vol. 57, no. 5, pp. e13554, 2020.DOI
- [6]Wei-Long Zheng and Bao-Liang Lu, “A multimodal approach to estimating vigilance using EEG and forehead EOG,” Journal of neural engineering, vol. 14, no. 2, pp. 026017, 2017.DOI
- [7]Jian Cui, Zirui Lan, Olga Sourina, and Wolfgang Müller-Wittig, “EEG-based cross-subject driver drowsiness recognition with an interpretable convolutional neural network,” IEEE Transactions on Neural Networks and Learning Systems, vol. 34, no. 10, pp. 7921–7933, 2022.DOI
- [8]Chuangquan Chen, Zhouyu Ji, Yu Sun, Anastasios Bezerianos, Nitish Thakor, and Hongtao Wang, “Self-attentive channel-connectivity capsule network for EEG-based driving fatigue detection,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 31, pp. 3152–3162, 2023.DOI
- [9]Ruilin Li, Minghui Hu, Ruobin Gao, Lipo Wang, Ponnuthurai N Suganthan, and Olga Sourina, “TFormer: A time-frequency Transformer with batch normalization for driver fatigue recognition,” Advanced Engineering Informatics, vol. 62, pp. 102575, 2024.DOI
- [10]Isah Bello, Moeed Sehnan, Weidong Dang, Jamal F Banzi, Sha’awanatu Aminu, and Zhongke Gao, “A frequency-attentive spatio-temporal convolutional neural network with weighted feature fusion for EEG-based driver fatigue detection,” Biomedical Signal Processing and Control, vol. 113, pp. 108979, 2026.DOI
- [11]Ke Liu, Lilong Sun, Wenlong Wang, Zhenghui Gu, Zhuliang Yu, and Wei Wu, “MB-STFormer: A multi-band spectral-temporal transformer with efficient attention for enhanced EEG-based fatigue detection,” IEEE Journal of Biomedical and Health Informatics, 2026.DOI
- [12]Gang Li and Wan-Young Chung, “Electroencephalogram-based approaches for driver drowsiness detection and management: A review,” Sensors, vol. 22, no. 3, pp. 1100, 2022.DOI
- [13]Saad Arif, Saba Munawar, and Hashim Ali, “Driving drowsiness detection using spectral signatures of EEG-based neurophysiology,” Frontiers in physiology, vol. 14, pp. 1153268, 2023.DOI
- [14]Jian Cui, Zirui Lan, Yisi Liu, Ruilin Li, Fan Li, Olga Sourina, and Wolfgang Müller-Wittig, “A compact and interpretable convolutional neural network for cross-subject driver drowsiness detection from single-channel EEG,” Methods, vol. 202, pp. 173–184, 2022.DOI
- [15]Zehong Cao, Chun-Hsiang Chuang, Jung-Kai King, and Chin-Teng Lin, “Multi-channel EEG recordings during a sustained-attention driving task,” Scientific data, vol. 6, no. 1, pp. 19, 2019.
- [16]Vernon J Lawhern, Amelia J Solon, Nicholas R Waytowich, Stephen M Gordon, Chou P Hung, and Brent J Lance, “EEGNet: a compact convolutional neural network for EEG-based brain–computer interfaces,” Journal of neural engineering, vol. 15, no. 5, pp. 056013, 2018.DOI
- [17]Thórir Mar Ingolfsson, Michael Hersche, Xiaying Wang, Nobuaki Kobayashi, Lukas Cavillegi, and Luca Benini, “EEG-TCNet: An accurate temporal convolutional network for embedded motor-imagery brain–machine interfaces,” in 2020 IEEE international conference on systems, man, and cybernetics (SMC). IEEE, 2020, pp. 2958–2965.arxiv.org/abs/2006.00622
- [18]Yongchao Song, Qingqing Zheng, Bingchuan Liu, and Xiaorong Gao, “EEG Conformer: Convolutional Transformer for EEG Decoding and Visualization,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 31, pp. 710–719, 2023.DOI
- [19]Yi Ding, Yong Li, Hao Sun, Rui Liu, Chengxuan Tong, Chenyu Liu, Xinliang Zhou, and Cuntai Guan, “EEG-Deformer: A Dense Convolutional Transformer for Brain-Computer Interfaces,” IEEE Journal of Biomedical and Health Informatics, vol. 29, no. 3, pp. 1909–1918, 2025.arxiv.org/abs/2405.00719
- [20]Xu Yan, Yuhang Shi, Tingting Zhang, Xin Chen, and Yi Mao, “BiConformer: EEG-Based Cognitive Load Assessment Using a Bidirectional Convolutional Transformer Network,” IEEE Transactions on Instrumentation and Measurement, vol. 74, pp. 1–15, 2025.
- [21]Zhiyuan Ma, Zeyuan Li, Zihao Qiu, Jinhao Li, Lingqin Meng, Xinche Zhang, Yixuan Liu, Xinke Shen, and Sen Song, “Dsainet: An efficient dual-scale attentive interaction network for general eeg decoding,” arXiv preprint arXiv:2604.18095, 2026.arxiv.org/abs/2604.18095



