WorksComputer Science
Towards wearable-scale, transition-aware sleep staging for clinical populations
Abstract
Automatic sleep staging models are predominantly developed on healthy sleepers and generalize poorly to clinical populations with fragmented sleep. Across three cohorts (patients with stroke, isolated REM sleep behaviour disorder, and other sleep disorders), we observe 2-3x more awakenings per hour, nearly twice as much N1, and less REM sleep than in healthy sleepers. We propose a label-efficient teacher-student framework for wearable-scale sleep staging. SleepGPT (136M parameters), adapted using labels from 25% of training patients, supervises a causal CNN-GRU (31k parameters; 122 KB) that stages EEG, EOG, or both in real time. We introduce Gcv-TAD, a new black-box transition-aware distillation loss. It emphasizes uncommon stage transitions while discounting those that vary across patients, using a prior estimated from labelled patients. In patient-wise 10-fold cross-validation, Gcv-TAD improves per-patient macro F1 over supervised training on the same labels by +3.2 to +7.6 points on average. Gains are largest on N1, and performance approaches full-label training.
Introduction
Automatic sleep staging is essential for assessing sleep disorders, but manual polysomnography (PSG) scoring is labor-intensive and unsuitable for long-term monitoring. Wearable devices using one or two EEG/EOG channels could enable continuous sleep assessment outside the laboratory. However, most staging models are developed on healthy or community cohorts [1, 2] and often generalize poorly to clinical populations. Recent work demonstrated substantial performance degradation when a model trained on healthy sleepers was evaluated on ischemic stroke patients [3]. Clinical sleep is also more fragmented, with frequent awakenings, increased N1 sleep, and heterogeneous stage-transition patterns. These differences make staging more challenging, while the scarcity of expert-labelled recordings limits patient-specific model adaptation.
Recent sleep foundation models, such as SleepGPT [4], offer powerful representations but require substantial memory and bidirectional context, limiting their use in real-time wearable applications. Knowledge distillation (KD) [5] can transfer knowledge from large models to smaller students. However, conventional KD treats epochs independently and does not explicitly account for sleep-stage transitions. Existing sleep-specific approaches primarily transfer intermediate representations using teacher assistants [6, 7]. We instead propose a black-box distillation framework that incorporates the transition dynamics of the target clinical population while requiring labels from only a fraction of patients.
Our contributions are: (1) A novel transition-aware distillation loss, -TAD, that emphasizes uncommon stage transitions while accounting for between-patient variability. (2) A lightweight causal CNN-GRU (31k parameters; 122 KB) distilled from SleepGPT using only 25% labelled training patients, enabling real-time sleep staging from EEG and/or EOG. (3) A comprehensive evaluation across three clinical cohorts using patient-wise 10-fold cross-validation, demonstrating improved macro F1 and N1 staging over supervised training with the same labels and performance approaching full-label training.
Method
Teacher: We use SleepGPT [4], a 136M-parameter transformer pretrained on 86k h of PSG, followed by a 3-layer BiLSTM (100 hidden units). The model processes four PSG channels (C3, C4, EMG, EOG) using a 21-epoch context to classify five sleep stages. For each clinical cohort, we independently fine-tune the teacher using 25% and 100% of the labelled training patients. The fully labelled teacher serves as a reference, while the 25%-label teacher provides supervision for student distillation. Fine-tuning uses cross-entropy (CE) and AdamW ( learning rate, 0.75 layer decay, at most 25 epochs), with model selection based on inner-validation macro F1. The frozen 25%-label teacher generates soft predictions with and hard pseudo-labels for all training epochs.

Figure 1. Overview of the proposed -TAD framework. The teacher is adapted using 25% labelled patients and frozen during distillation. Its predictions guide a causal student through a transition-aware loss weighted by preceding-stage confidence and a cohort-specific prior. Only the student is intended for deployment.
Student: We design a lightweight causal CNN-GRU with three 1-D convolutional layers (16/32/32 filters; kernel sizes 50/10/5), followed by adaptive pooling to 32 steps, a 64-unit GRU, and a five-class classifier. The model contains 30,421 parameters for a single channel and 31,221 for EEG+EOG. By preserving the GRU hidden state across consecutive 30-s epochs, the student performs streaming sleep staging without future context. It is trained on 20-epoch sequences using Adam (learning rate , cosine schedule, maximum 40 epochs), with the best checkpoint selected by inner-validation macro F1.
Transition-aware distillation: Clinical sleep is characterized by frequent and heterogeneous stage transitions, making it important to account for transition dynamics during knowledge transfer. To address this, we propose -TAD, a black-box distillation loss that incorporates cohort-specific transition statistics and between-patient variability to guide student learning.
Let and denote the teacher and student probabilities at epoch , respectively, with temperature . The teacher’s preceding prediction defines and confidence . Our transition-aware loss is
where is a non-negative generalized KL term. The transition prior is defined as
where is the mean of the per-patient row-normalized transition matrices and measures their between-patient variability. Here, denotes the element-wise standard deviation.
The prior assigns greater weight to less probable next-stage transitions while discounting those that vary strongly between patients. It is estimated separately for each fold using only labelled training patients. The student minimizes , where CE uses scorer labels for labelled patients and teacher pseudo-labels for unlabelled patients.
Experiments
Datasets: We evaluate three clinical polysomnography (PSG) datasets. iSLEEPS [8] contains overnight PSG recordings from 100 ischemic stroke patients. NAPS [9, 10, 11] includes patients with isolated REM sleep behaviour disorder (iRBD), recorded across multiple clinical sites and centrally scored using AASM guidelines. We retain 133 of the 138 recordings that contain the required central EEG, chin EMG, and EOG channels. ISRUC-S1 [12] comprises patients with various sleep disorders. We use 99 recordings with scorer-1 annotations, excluding recording 40 due to its different montage.
Evaluation protocol: We use patient-wise 10-fold cross-validation with 80:10:10 training, validation, and test splits across all three cohorts. Teachers and students are independently trained with CE using 100% or 25% of training labels. The 100%-label models serve as supervised references. For distillation, the frozen 25%-label teacher supervises students using KD or -TAD, with the remaining training patients treated as unlabelled. All methods share identical splits, and checkpoints are selected using validation macro F1. Test patients remain held out throughout training and selection. We report accuracy, macro F1, Cohen’s , and per-stage F1.
Results and Discussion
Label efficiency (Table 1). Trained on all labelled patients, the 31k-parameter students stage patient sleep well, but their performance drops sharply across cohorts and input settings when only 25% of patients are labelled. The teacher is less affected by label scarcity, providing informative pseudo-labels for the unlabelled patients. -TAD emphasises less probable next-stage assignments while down-weighting transitions with high between-patient variability. This weighting is designed to help the student learn fragmented sleep patterns without relying excessively on inconsistent cohort-level transitions.
| Cohort | Model | 100% labels CE Acc | 100% labels CE MF1 | 100% labels CE | 25% labelled patients CE Acc | 25% labelled patients CE MF1 | 25% labelled patients CE | 25% labelled patients -TAD Acc | 25% labelled patients -TAD MF1 | 25% labelled patients -TAD | Per-class F1, 25% W | Per-class F1, 25% N1 | Per-class F1, 25% N2 | Per-class F1, 25% N3 | Per-class F1, 25% REM |
| iSLEEPS | Teacher | 0.77 | 0.71 | 0.68 | 0.75 | 0.68 | 0.65 | – | – | – | 0.81 | 0.36 | 0.80 | 0.70 | 0.74 |
| EOG (E1) | 0.72 | 0.64 | 0.60 | 0.66 | 0.57 | 0.51 | 0.71 | 0.63 | 0.58 | 0.76 | 0.28 | 0.77 | 0.68 | 0.66 | |
| EEG (C4) | 0.71 | 0.63 | 0.59 | 0.68 | 0.59 | 0.55 | 0.71 | 0.63 | 0.60 | 0.77 | 0.27 | 0.78 | 0.69 | 0.66 | |
| EEG+EOG | 0.74 | 0.67 | 0.64 | 0.71 | 0.62 | 0.58 | 0.75 | 0.68 | 0.64 | 0.80 | 0.33 | 0.81 | 0.71 | 0.73 | |
| NAPS | Teacher | 0.79 | 0.71 | 0.70 | 0.77 | 0.68 | 0.67 | – | – | – | 0.84 | 0.42 | 0.82 | 0.58 | 0.73 |
| EOG (E1) | 0.72 | 0.56 | 0.58 | 0.65 | 0.48 | 0.48 | 0.70 | 0.56 | 0.56 | 0.77 | 0.19 | 0.78 | 0.46 | 0.60 | |
| EEG (C4) | 0.74 | 0.59 | 0.61 | 0.68 | 0.52 | 0.54 | 0.73 | 0.60 | 0.61 | 0.82 | 0.29 | 0.81 | 0.45 | 0.63 | |
| EEG+EOG | 0.78 | 0.66 | 0.67 | 0.72 | 0.59 | 0.60 | 0.76 | 0.64 | 0.64 | 0.84 | 0.32 | 0.82 | 0.51 | 0.70 | |
| ISRUC-S1 | Teacher | 0.82 | 0.81 | 0.77 | 0.80 | 0.79 | 0.74 | – | – | – | 0.88 | 0.54 | 0.79 | 0.87 | 0.87 |
| EOG (E1) | 0.75 | 0.73 | 0.68 | 0.69 | 0.66 | 0.60 | 0.73 | 0.71 | 0.65 | 0.80 | 0.42 | 0.73 | 0.84 | 0.76 | |
| EEG (C4) | 0.77 | 0.75 | 0.71 | 0.73 | 0.70 | 0.66 | 0.76 | 0.73 | 0.69 | 0.83 | 0.46 | 0.77 | 0.85 | 0.76 | |
| EEG+EOG | 0.80 | 0.78 | 0.74 | 0.76 | 0.73 | 0.69 | 0.79 | 0.77 | 0.73 | 0.86 | 0.51 | 0.79 | 0.87 | 0.83 |
Table 1. Test accuracy (Acc), macro F1 (MF1) and Cohen’s (mean over 10 folds) of models trained on 100% of the labels (CE) and with 25% of the training patients labelled (CE or -TAD), and per-class F1 with 25% labels (teacher: CE; students: -TAD). Teacher: 4-channel SleepGPT-BiLSTM; students: 31k-parameter causal CNN-GRU. Best -TAD student of each cohort in bold.
With the same labels, -TAD improves macro F1 over CE by +3.2 to +7.6 points, leaving the students between −2.0 and +0.9 points from their full-label counterparts. All gains are means of paired per-patient macro-F1 differences, not differences of the fold means in Table 1. The largest gains occur in N1, a transitional stage that is more prevalent in fragmented clinical sleep and frequently missed by label-scarce CE students. With EEG+EOG, -TAD increases N1 F1 from 0.241 to 0.331, 0.210 to 0.324, and 0.430 to 0.514 (iSLEEPS, NAPS, ISRUC-S1), respectively. These results match or exceed those of the fully supervised CE students (0.305, 0.315, 0.511). Combining EEG and EOG provides the strongest student performance across the three cohorts.
Comparison with published work (Table 2): Our teacher has the highest macro F1 on both cohorts. With 25% of the labels and 31k parameters, the two-channel -TAD student matches the best published macro F1 on iSLEEPS and, on ISRUC-S1, comes within a few points of models with more channels [14, 15] while exceeding nine-channel SleepSMC [16].
| Method | Ch. | Params | Acc | MF1 | |
| iSLEEPS | |||||
| SE-ResNet-BiLSTM [3] | 1 | – | 0.75 | 0.68 | 0.64 |
| SleepStageNet [13] | 1 | 1.1M | 0.72 | 0.64 | 0.60 |
| SleepStageNet [13] | 5 | 1.1M | 0.74 | 0.67 | 0.63 |
| Ours: teacher | 4 | 136.2M | 0.77 | 0.71 | 0.68 |
| Ours: student, -TAD, 25% | 2 | 31k | 0.75 | 0.68 | 0.64 |
| ISRUC-S1 | |||||
| MSSC-BiMamba [14] | 10 | 470k | 0.83 | 0.80 | 0.77 |
| JK-STGCN [15] | – | – | 0.82 | 0.80 | 0.77 |
| SleepSMC [16] | 9 | – | 0.77 | 0.75 | 0.70 |
| SleepSMC, EEG [16] | 6 | – | 0.73 | 0.70 | 0.65 |
| Ours: teacher | 4 | 136.2M | 0.82 | 0.81 | 0.77 |
| Ours: student, -TAD, 25% | 2 | 31k | 0.79 | 0.77 | 0.73 |
Table 2. iSLEEPS and ISRUC-S1 against published results (100% labels unless marked 25%; Ch.: input channels). Ours: patient-wise 10-fold CV, fold means. Published protocols (folds, recordings) differ from ours. Best per cohort in bold.
Where the transition loss acts: Figure 2 illustrates how -TAD guides student learning on a fragmented iSLEEPS training night, containing 16.2 stage changes per hour compared with the cohort median of 9.6 among unlabelled patients. The scorer hypnogram (Figure 2a) shows frequent stage changes, while the teacher’s soft predictions (Figure 2b) remain confident during consolidated N2, N3 and REM sleep but become less certain around transitions, particularly N1. Rather than treating all epochs equally, uses the preceding-stage confidence and a cohort-specific transition prior to redistribute the distillation signal. The prior reduces emphasis on common self-transitions while discounting stage changes with high between-patient variability. Consequently, the loss places relatively greater weight near less frequent stage transitions (Figure 2c), where conventional epoch-wise supervision may provide limited guidance.

Figure 2. Illustration of on a highly fragmented, unlabelled iSLEEPS training night (fold 1; 16.2 stage changes per hour) (a) Scorer hypnogram, shown for reference only; red ticks: stage changes. (b) Teacher soft labels (), the distillation targets. (c) Weight of the transition loss per epoch, , relative to its night mean (dashed: equal weighting). (d) Epochs where the and CE students (25% labels) disagree: matches the scorer on 55, CE on 21, neither on 21.
This pattern extends across all three cohorts. In fold-1 training data, stage-change epochs account for 17, 21 and 29% of epochs but receive 20, 25 and 41% of the distillation weight in iSLEEPS, NAPS and ISRUC-S1, respectively. Similarly, N1 epochs represent 10, 10 and 13% of the data but receive 12, 13 and 21% of the weight. The stronger redistribution in ISRUC-S1 is consistent with its greater sleep fragmentation. This emphasis on less common transitions and N1 provides a plausible explanation for the improved N1 staging observed in the cross-validated results.
The student predictions offer further insight (Figure 2d). On the example night, and the CE student disagree on 97 epochs, of which correctly classifies 55, compared with 21 for CE. Across fold-1 training nights, is correct in 56, 48 and 60% of disagreements, versus 27, 29 and 29% for CE. However, improvements are not restricted to transitions; the example also shows gains during stable N3 sleep. Moreover, the teacher achieves only 64% accuracy on this night, illustrating that distillation remains sensitive to teacher reliability. These analyses use training recordings whose scorer labels were hidden during learning and examined retrospectively. These analyses illustrate how the proposed loss distributes its guidance, while generalization is evaluated on held-out patients in Table 1.
Wearable feasibility: The student has 30,421 parameters, 4,476× fewer than the teacher (136.2M), and needs 2.2M multiply–accumulates per 30-s epoch instead of 10.4G. Its weights take 121.7 KB in FP32 and 75 KB after dynamic INT8 quantization, within the SRAM of wearable microcontrollers, and its state between epochs is a single 64-dimensional vector. One epoch takes 1.16 ms on one thread of an Intel Xeon W2265 CPU (batch 1) and is scored as soon as it ends, whereas the teacher needs 271.90 ms per epoch even with cached embeddings and must wait 5 min for future context.
Conclusion
Long-term sleep monitoring in clinical populations is limited not only by wearable hardware but also by the scarcity of expert-labelled patient recordings. Our findings show that knowledge from a cohort-adapted foundation model can be transferred to a lightweight, causal student with substantially fewer labels. By accounting for differences in sleep transitions across patients, offers a promising approach to learning from heterogeneous clinical sleep. This could enable more accessible longitudinal monitoring of sleep disruption during recovery and disease progression, where repeated laboratory PSG is impractical. Future work will evaluate wearable electrode configurations, on-device performance, and generalization to additional clinical populations before real-world deployment.
Compliance with ethical standards
This retrospective study used de-identified PSG data from three existing clinical datasets: iSLEEPS, NAPS, and ISRUC-S1. The original iSLEEPS study received approval from the NIMHANS Institutional Ethics Committee. NAPS data were accessed through the NSRR under its data use agreement, while ISRUC-S1 is publicly available.
Acknowledgments
We acknowledge IHub-Data, IIIT Hyderabad (HI-002), for financial assistance.
Declaration of AI Use
ChatGPT (OpenAI) and Claude (Anthropic) were used solely to improve the quality and clarity of the writing and the aesthetic presentation of the model architecture image. All AIassisted outputs were reviewed by the authors, who take full responsibility for the content of the paper.
References
References
- [1]A. Supratak, H. Dong, C. Wu, and Y. Guo, “DeepSleep-Net: A model for automatic sleep stage scoring based on raw single-channel EEG,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 25, no. 11, pp. 1998–2008, 2017.
- [2]M. Perslev, S. Darkner, L. Kempfner, M. Nikolic, P. J. Jennum, and C. Igel, “U-Sleep: resilient high-frequency sleep staging,” npj Digital Medicine, vol. 4, pp. 72, 2021.DOI
- [3]Saswata Bose, Suvadeep Maiti, Shivam Kumar Sharma, Mythirayee S, Tapabrata Chakraborti, Srijitesh Rajendran, and Raju S. Bapi, “Ai generalisation gap in comorbid sleep disorder staging,” in 2026 IEEE 23rd International Symposium on Biomedical Imaging (ISBI), 2026, IEEE.DOI
- [4]W. Huang, Y. Wang, H. Cheng, W. Xu, T. Li, X. Wu, H. Xu, P. Liao, Z. Cui, Q. Zou, and J.-H. Gao, “A unified time-frequency foundation model for sleep decoding,” Nature Communications, vol. 17, pp. 1198, 2026.DOI
- [5]G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” arXiv preprint arXiv:1503.02531, 2015.arxiv.org/abs/1503.02531
- [6]H. Liang, Y. Liu, H. Wang, and Z. Jia, “Teacher assistant-based knowledge distillation extracting multi-level features on single channel sleep EEG,” in International Joint Conference on Artificial Intelligence (IJCAI), 2023, pp. 3948–3956.DOI
- [7]Z. Jia, H. Liang, Y. Liu, H. Wang, and T. Jiang, “DistillSleepNet: Heterogeneous multi-level knowledge distillation via teacher assistant for sleep staging,” IEEE Transactions on Big Data, 2024.DOI
- [8]S. Maiti, S. K. Sharma, S. Mythirayee, S. Rajendran, and R. S. Bapi, “Polysomnography dataset for sleep analysis in ischemic stroke patients,” Scientific Data, vol. 13, pp. 421, 2026.DOI
- [9]G.-Q. Zhang, L. Cui, R. Mueller, S. Tao, M. Kim, M. Rueschman, S. Mariani, D. Mobley, and S. Redline, “The National Sleep Research Resource: towards a sleep data commons,” Journal of the American Medical Informatics Association, vol. 25, no. 10, pp. 1351–1358, 2018.DOI
- [10]Jonathan E Elliott, Miranda M Lim, Allison T Keil, Ronald B Postuma, Amelie Pelletier, Jean-François Gagnon, Erik K St. Louis, Leah K Forsberg, Julie A Fields, Daniel E Huddleston, et al., “Baseline characteristics of the north american prodromal synucleinopathy cohort,” Annals of Clinical and Translational Neurology, vol. 10, no. 4, pp. 520–535, 2023.DOI
- [11]Miranda M Lim, Lee E Neilson, Jonathan E Elliott, Bianca Hoang-Dang, Jessica R Locke, Hannah Cunningham, Jessica Brewer, Laura Dennis, Laura Dovek, Amélie Pelletier, et al., “The north american prodromal synucleinopathy study: protocol for a multi-site, longitudinal, observational study of idiopathic/isolated rapid eye movement sleep behavior disorder,” Sleep advances, vol. 7, no. 1, pp. zpag007, 2026.DOI
- [12]S. Khalighi, T. Sousa, J. M. Santos, and U. Nunes, “ISRUC-Sleep: A comprehensive public dataset for sleep researchers,” Computer Methods and Programs in Biomedicine, vol. 124, pp. 180–192, 2016.DOI
- [13]A. Alhazmi, “SleepStageNet: A lightweight and explainable deep learning architecture for multi-channel sleep staging,” Brain Sciences, vol. 16, no. 8, pp. 860, 2026.DOI
- [14]C. Zhang, W. Cui, and J. Guo, “MSSC-BiMamba: Multimodal sleep stage classification and early diagnosis of sleep disorders with bidirectional Mamba,” arXiv preprint arXiv:2405.20142, 2024.arxiv.org/abs/2405.20142
- [15]X. Ji, Y. Li, and P. Wen, “Jumping knowledge based spatial-temporal graph convolutional networks for automatic sleep stage classification,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 30, pp. 1464–1472, 2022.DOI
- [16]S. Ma, Y. Zhang, Y. Chen, H. Wang, Y. Jin, W. Zhang, and Z. Jia, “SleepSMC: Ubiquitous sleep staging via supervised multimodal coordination,” in International Conference on Learning Representations (ICLR), 2025.