betaFeedback

WorksEngineering

Task-adaptive anonymisation enables privacy-preserving sharing of clinical speech data

Abstract

Clinical speech is increasingly used for disease screening and clinical assessment, but directly distributing speech recordings poses privacy risks because they contain both speaker-identifying voiceprints and sensitive spoken content. Existing speech anonymisation approaches are primarily developed for general speech applications or task-specific clinical speech and overlook that different clinical tasks rely on distinct speech characteristics, necessitating task-adaptive anonymisation strategies to protect patient privacy while preserving clinical utility. Here we introduce CASA, a clinically adaptive speech anonymisation agent that transforms anonymisation from a fixed processing procedure into a task-guided optimisation process, dynamically tailoring voiceprint and content anonymisation to task-specific privacy and clinical utility requirements. Through recursive self-improvement (RSI), CASA iteratively evaluates and refines anonymisation strategies against task-specific privacy and clinical utility requirements, while learning from these outcomes to guide more efficient planning in subsequent clinical tasks. CASA is validated across 14 clinical tasks encompassing five languages, diverse elicitation paradigms, and mental health, cognitive and voice disorders. CASA preserved disease-related classification performance while meeting speaker-anonymisation requirements across all clinical tasks, versus only 8/14 for the strongest fixed method. Assessments of resistance to speaker re-identification by external attackers and identification and redaction of sensitive personal information in clinical speech further demonstrated CASA's dual protection of speaker identity and sensitive spoken content. Clinical expert assessments further supported the preservation of clinically relevant information and the acceptability of anonymised speech for controlled research sharing.

Speech arises from the coordinated interaction of auditory perception, neural processing, motor planning and the respiratory, phonatory and articulatory systems, producing a complex behavioural signal that reflects both physiological and neurological function [1]. Because speech can be collected remotely and non-invasively, it has emerged as a promising digital biomarker for a broad range of clinical applications [2, 3, 4]. Previous studies have demonstrated the utility of speech-based analysis for mental health assessment [5, 6, 7, 8], cognitive impairment screening [9, 10, 11], and speech disorder evaluation [12, 13, 14]. However, speech simultaneously encodes highly identifiable information, including speaker identity and spoken linguistic content, creating substantial privacy risks for the collection, sharing and analysis of clinical speech data [3, 15, 16]. Effective anonymisation is therefore essential for enabling the safe sharing and analysis of clinical speech data while preserving information required for downstream clinical assessment.

A major challenge for clinical speech anonymisation is that different clinical tasks rely on distinct speech characteristics for assessment, including acoustic, phonetic, paralinguistic and linguistic information [15]. At the same time, different elicitation paradigms expose different forms of privacy-sensitive information. For example, reading speech may primarily require suppression of speaker-identifiable vocal characteristics [17], whereas spontaneous autobiographical speech may additionally contain sensitive semantic content [18]. Consequently, clinical speech anonymisation is unlikely to admit a one-size-fits-all solution. The goal is not simply to maximise anonymisation strength, but to achieve sufficient privacy protection while preserving the speech cues required for a given clinical task.

Practical clinical settings therefore require scalable speech anonymisation methods that can operate across diverse diseases and elicitation paradigms. Existing anonymisation approaches, however, remain largely fragmented between general-purpose speech anonymisation and disease-specific clinical solutions. Methods developed for general speech applications [19] primarily focus on suppressing speaker identity and often inadvertently degrade clinically relevant speech information, while disease-specific anonymisation approaches [20, 21, 22] are typically designed for individual disorders or narrowly defined tasks, limiting their applicability across heterogeneous clinical settings. To our knowledge, no existing framework adapts both voiceprint and spoken-content anonymisation to individual clinical tasks across diseases, languages and elicitation paradigms.

Here we present CASA, a clinically adaptive speech anonymisation agent that reformulates clinical speech anonymisation as a task-adaptive privacy–utility optimisation problem rather than a fixed preprocessing procedure. Given a natural-language description of clinical task properties and corresponding privacy requirements, CASA autonomously devises voiceprint and content anonymisation strategies to protect patient privacy while preserving disease-relevant speech information. Its recursive self-improvement mechanism iteratively evaluates and refines these strategies based on privacy–utility feedback for the current task, while distilling successful strategies and failure cases into memory to inform subsequent tasks. This creates a continuous learning cycle in which each task both draws on and contributes to CASA’s growing experience, progressively refining its strategy planning. In our evaluation, this experience-guided planning reduced the number of iterations required to meet the privacy–utility requirements by 31.3%. We validate CASA on 11 clinical speech datasets encompassing 14 clinical tasks, five languages and diverse mental health, cognitive and voice disorders. CASA satisfied the joint speaker-privacy and clinical-utility requirements across all 14 tasks, compared with at most eight for any individual fixed anonymisation method. The preservation of clinically relevant information and the acceptability of anonymised recordings for controlled research sharing are further supported by clinical expert evaluation. We envisage that this approach could facilitate broader clinical applications of speech-based assessment and privacy-preserving data sharing for multi-centre collaborative research.

Results

We conducted a series of evaluations to establish whether CASA can reconcile patient privacy with clinical utility across heterogeneous speech-based clinical tasks. We first compared CASA with other anonymisation methods to assess its ability to satisfy task-specific speaker-privacy and clinical-utility requirements across diverse tasks, and evaluated its protection against external speaker re-identification attackers. We then examined how different voiceprint anonymisation methods affect speaker privacy and the preservation of speech characteristics, investigating why task-specific strategy selection is necessary. We also evaluated CASA’s performance in identifying and protecting sensitive spoken content, as well as its ability to improve the efficiency of

task-adaptive anonymisation strategy planning through recursive self-improvement. Finally, clinical expert assessments examined whether anonymised recordings retained clinically relevant information and were considered acceptable for controlled research sharing.

CASA satisfies privacy–utility requirements across diverse clinical speech tasks

We first investigated whether CASA could satisfy joint speaker-privacy and clinical-utility requirements across heterogeneous clinical tasks where fixed anonymisation strategies fall short. Unlike fixed pipelines, CASA determines voiceprint and content anonymisation strategies according to task-specific requirements and iteratively adjusts them based on evaluation feedback (Figure 1a). It integrates five voiceprint anonymisation methods – pitch shifting (Pitch), linear predictive coding (LPC)-based formant transformation (FormantLPC_{\mathrm{LPC}}), spectral formant scaling (Formantspec_{\mathrm{spec}}), neural voice conversion (Neural VC) and transcription–resynthesis (Resyn)– alongside large language model (LLM)-based localisation and masking of sensitive spoken content. These components provide the available anonymisation operations, while the agent determines when and how to apply them for a given clinical task.

Figure 1. Overview of CASA and comparison with baselines. a, Architecture of the CASA agent. An LLM-based controller receives the clinical task, speech paradigm and privacy–utility requirements, plans an anonymisation strategy and invokes tools for voiceprint and content anonymisation. Evaluation feedback guides adjustment to methods and parameters. Cross-task memory retains coarse-grained summaries and fine-grained experimental records to guide subsequent planning. b, Number of datasets or task subsets meeting all applicable privacy and clinical-utility criteria, comparing five individual methods used as fixed strategies (grey), a SecureSpeech pipeline (orange) and CASA (blue). Success is defined as that the anonymised speech achieve an EER of at least 20%, sensitive-content localisation F1 of at least 0.75 when required, and a downstream BACC decrease of less than three percentage points relative to original speech. c, Delta BACC in percentage points after anonymisation with CASA and SecureSpeech. The dashed green circle marks the -3-percentage-point boundary, and the green shaded outer region indicates changes satisfying the utility requirement. The datasets are grouped into mental health, cognitive and voice disorders.

To assess the benefit of task-adaptive anonymisation strategy, we compared CASA with each of the five voiceprint anonymisation methods applied as a fixed strategy. We also included a multilingual implementation of SecureSpeech [28], which combines voiceprint and content anonymisation in a fixed pipeline, as a complete-system baseline. The evaluation covered 11 clinical speech datasets comprising 14 tasks or subsets. across five languages, spanning mental health, cognitive and voice disorders and including both single-speaker and multi-speaker recordings (Table 1). This heterogeneous setting allowed us to examine whether anonymisation strategies could satisfy the same privacy and utility requirements across different clinical contexts.

DatasetDiseaseLanguageN-SubjectsMulti-SpkSourceDownstream
MODMADepressionChinese52✗[23]Binary
Androids-CorpusDepressionItalian118✓[24]Binary
Suicide-YFSuicide riskChinese1223✗In-houseBinary
CrisisPsychological crisisChinese154✓In-house3-class
NCMMSC-ADAlzheimer’s DiseaseChinese175✗[25]3-class
ADReSSAlzheimer’s DiseaseEnglish156✗[9]Binary
IvanovaAlzheimer’s DiseaseSpanish357✗[10]3-class
NeuroVozParkinson’s DiseaseSpanish112✗[26]Binary
ENNILanguage impairmentEnglish352✓[27]Binary
SaarbrückenHyperfunktionelle DysphonieGerman424✗Binary
SaarbrückenLaryngitisGerman280✗[12]Binary
SaarbrückenRekurrenspareseGerman424✗Binary
PUMCHCleft-palateChinese244✓In-houseBinary

Table 1. Clinical speech datasets applied for validation, spanning mental health disorders, cognitive disorders, and voice disorders. Androids-Corpus was evaluated separately for interview and reading speech (Androids-I and Androids-R), and Saarbrücken was split into three disease-versus-control subsets (Saar-HD, Saar-L and Saar-R), resulting in 14 evaluation tasks/subsets. Disease type, language, number of subjects, whether recording is multi-speaker (Multi-Spk), data source, and downstream evaluation tasks, are listed. Downstream tasks include binary or multi-class classification.

We defined successful anonymisation using three simultaneous criteria: (i) a speaker-verification equal error rate (EER) of at least 20%, indicating that the specified speaker-privacy threshold was met; (ii) a F1 score at least 0.75 for localisation of sensitive spoken content when present and required by the task, indicating that the content privacy requirement was met; and (iii) a reduction in downstream balanced accuracy (BACC) of less than three percentage points relative to original speech, indicating preservation of disease-related classification performance.

As shown in Figure 1b, CASA succeeded on all 14 clinical tasks (100%) under these criteria, whereas the best-performing individual fixed method succeeded on only 8 tasks and SecureSpeech on 4 tasks. Notably, the Crisis and PUMCH datasets contain sensitive spoken information requiring both voiceprint and content anonymisation. The five individual voiceprint anonymisation methods could not fulfil this additional requirement, whereas CASA and SecureSpeech both incorporate content anonymisation, enabling a comparison of the two complete systems. As shown in Figure 1c, CASA achieved higher BACC than SecureSpeech on all tasks while maintaining the classification performance decline within the specified tolerance across all tasks.

We further assessed CASA’s voiceprint protection with two external attackers: (i) a semi-informed attacker trained on anonymised speech following the VoicePrivacy training recipe [29], which assesses whether an attacker informed prior knowledge of the speech transformation pipeline can successfully re-identify speakers, and (ii) a speech large language model (SpeechLLM) attacker based on SALMONN-2-8B[30] that directly judges whether two speech segments originate from the same speaker. The semi-informed attacker yielded a mean EER of 0.338 across the 14 clinical tasks, with all meeting the privacy threshold considered in VoicePrivacy 2024. The SpeechLLM attacker yielded a mean verification error of 0.441, calculated as one minus accuracy of same-speaker versus different-speaker classification. These results demonstrate CASA’s broad resistance to speaker re-identification across diverse clinical speech tasks.

Overall, task-adaptive strategy enables CASA to satisfy joint speaker-privacy and clinical-utility requirements across a wider range of clinical tasks. We next investigate how differences in disease-relevant speech characteristics and privacy-sensitive content across clinical tasks shape the requirements for voiceprint and content anonymisation, respectively.

Anonymisation of speaker voiceprint

To systematically understand the benefit of task-adaptive design, we examined how different voiceprint anonymisation methods affected different speech characteristics and downstream clinical performance. Beyond EER and BACC introduced above, our multi-dimensional evaluation further includes signal cleanliness (i.e., signal-to-noise ratio, SNR), perceptual quality (i.e., mean opinion score, MOS), preservation of paralinguistic information (i.e., Pearson correlation coefficient of fundamental frequency (F0), PCCF0_{\mathrm{F0}}, and emotion similarity, Emo), and preservation of linguistic information (i.e., character and word error rates, CER/WER). Full results are provided in Supplementary Table 1–Supplementary Table 3.

DatasetAnonymiseSNRMOSPCCF0_0CER/WEREmoEERBACCp-value
MODMAOrigin20.58±5.3620.58 \pm5.362.25±0.502.25 \pm0.50----0.782±0.0780.782 \pm0.078-
Pitch19.01±5.2819.01 \pm5.281.27±0.031.27 \pm0.030.903±0.1480.903 \pm0.1480.2040.934±0.0680.934 \pm0.0680.567 (0.529 - 0.619)0.787±0.1260.787 \pm0.1260.746
FormantLPC22.39±5.5122.39 \pm5.511.54±0.311.54 \pm0.310.851±0.3000.851 \pm0.3000.1170.944±0.0630.944 \pm0.0630.139 (0.125 - 0.153)0.807±0.0730.807 \pm0.0730.746
Formantspec19.00±5.1719.00 \pm5.171.31±0.071.31 \pm0.070.953±0.0970.953 \pm0.0970.0980.952±0.0560.952 \pm0.0560.163 (0.149 - 0.177)0.795±0.1460.795 \pm0.1460.604
Neural VC36.85±5.3636.85 \pm5.362.00±0.372.00 \pm0.370.279±0.4380.279 \pm0.4380.1700.936±0.0660.936 \pm0.0660.445 (0.426 - 0.465)0.773±0.0990.773 \pm0.0990.604
Resyn30.78±8.2930.78 \pm8.293.23±0.363.23 \pm0.360.070±0.2720.070 \pm0.2720.0400.928±0.0740.928 \pm0.0740.335 (0.318 - 0.351)0.842±0.1030.842 \pm0.1030.849
Androids-IOrigin22.72±4.5422.72 \pm4.541.52±0.191.52 \pm0.19----0.853±0.0900.853 \pm0.090-
Pitch22.04±4.3722.04 \pm4.371.25±0.011.25 \pm0.010.441±0.3550.441 \pm0.3550.1890.962±0.0510.962 \pm0.0510.319 (0.259 - 0.388)0.852±0.0920.852 \pm0.0920.696
FormantLPC21.16±4.5421.16 \pm4.541.26±0.041.26 \pm0.040.759±0.2330.759 \pm0.2330.1870.914±0.0740.914 \pm0.0740.362 (0.306 - 0.430)0.800±0.0930.800 \pm0.0930.090
Formantspec21.26±4.4821.26 \pm4.481.26±0.051.26 \pm0.050.412±0.3620.412 \pm0.3620.1250.950±0.0460.950 \pm0.0460.177 (0.134 - 0.238)0.825±0.0750.825 \pm0.0750.274
Neural VC40.73±1.8740.73 \pm1.871.72±0.111.72 \pm0.110.033±0.1490.033 \pm0.1490.2580.935±0.0590.935 \pm0.0590.483 (0.410 - 0.550)0.774±0.0880.774 \pm0.0880.047
Resyn39.02±4.3339.02 \pm4.332.44±0.462.44 \pm0.46−0.005±0.102-0.005 \pm0.1020.0210.957±0.0610.957 \pm0.0610.431 (0.367 - 0.500)0.896±0.0510.896 \pm0.0510.593
Androids-ROrigin22.89±5.0422.89 \pm5.041.71±0.291.71 \pm0.29----0.699±0.0880.699 \pm0.088-
Pitch21.69±4.8921.69 \pm4.891.25±0.021.25 \pm0.020.878±0.1830.878 \pm0.1830.0790.957±0.0580.957 \pm0.0580.330 (0.250 - 0.380)0.640±0.0840.640 \pm0.0840.229
FormantLPC21.39±4.8421.39 \pm4.841.29±0.051.29 \pm0.050.791±0.1880.791 \pm0.1880.0670.891±0.0660.891 \pm0.0660.353 (0.295 - 0.424)0.675±0.0820.675 \pm0.0820.500
Formantspec22.12±5.1222.12 \pm5.121.27±0.051.27 \pm0.050.808±0.2500.808 \pm0.2500.0360.962±0.0530.962 \pm0.0530.232 (0.183 - 0.295)0.745±0.1100.745 \pm0.1100.952
Neural VC39.82±2.5039.82 \pm2.501.81±0.141.81 \pm0.140.052±0.1420.052 \pm0.1420.0840.948±0.0660.948 \pm0.0660.509 (0.433 - 0.572)0.724±0.0610.724 \pm0.0610.837
Resyn35.50±4.4035.50 \pm4.402.24±0.392.24 \pm0.390.017±0.1140.017 \pm0.1140.0100.940±0.0690.940 \pm0.0690.460 (0.384 - 0.531)0.516±0.0630.516 \pm0.0630.008
Suicide-YFOrigin15.22±4.3015.22 \pm4.301.44±0.251.44 \pm0.25----0.668±0.0100.668 \pm0.010-
Pitch14.21±4.0414.21 \pm4.041.31±0.091.31 \pm0.090.702±0.2900.702 \pm0.2900.3650.922±0.0740.922 \pm0.0740.523 (0.502 - 0.551)0.604±0.0250.604 \pm0.0251.59×10−31.59 \times10^{-3}
FormantLPC14.83±4.6214.83 \pm4.621.31±0.081.31 \pm0.080.548±0.4980.548 \pm0.4980.2980.946±0.0600.946 \pm0.0600.180 (0.167 - 0.197)0.641±0.0430.641 \pm0.0430.069
Formantspec14.63±4.1314.63 \pm4.131.28±0.091.28 \pm0.090.702±0.3300.702 \pm0.3300.1490.939±0.0530.939 \pm0.0530.230 (0.212 - 0.249)0.614±0.0170.614 \pm0.0170.074
Neural VC33.52±6.0433.52 \pm6.041.82±0.301.82 \pm0.300.126±0.3300.126 \pm0.3300.3360.922±0.0730.922 \pm0.0730.467 (0.449 - 0.489)0.629±0.0230.629 \pm0.0230.014
Resyn35.52±4.2935.52 \pm4.291.82±0.401.82 \pm0.400.013±0.2180.013 \pm0.2180.0200.915±0.0760.915 \pm0.0760.412 (0.393 - 0.431)0.662±0.0180.662 \pm0.0180.376
CrisisOrigin13.47±4.3613.47 \pm4.361.33±0.051.33 \pm0.05----0.765±0.0700.765 \pm0.070-
Pitch10.73±3.9210.73 \pm3.921.23±0.011.23 \pm0.010.809±0.1670.809 \pm0.1670.3240.954±0.0540.954 \pm0.0540.586 (0.549 - 0.615)0.759±0.0670.759 \pm0.0670.721
FormantLPC13.69±4.3613.69 \pm4.361.22±0.011.22 \pm0.010.816±0.1920.816 \pm0.1920.1360.961±0.0490.961 \pm0.0490.228 (0.193 - 0.260)0.795±0.0660.795 \pm0.0660.639
Formantspec10.44±4.1110.44 \pm4.111.64±0.071.64 \pm0.070.694±0.2180.694 \pm0.2180.1160.965±0.0580.965 \pm0.0580.496 (0.460 - 0.534)0.676±0.0610.676 \pm0.0610.044
Neural VC40.44±1.6540.44 \pm1.651.64±0.071.64 \pm0.070.092±0.2020.092 \pm0.2020.1970.955±0.0360.955 \pm0.0360.515 (0.481 - 0.549)0.738±0.0350.738 \pm0.0350.229
Resyn52.74±6.2852.74 \pm6.282.22±0.082.22 \pm0.08−0.007±0.202-0.007 \pm0.2020.0310.852±0.0360.852 \pm0.0360.516 (0.481 - 0.549)0.683±0.1450.683 \pm0.1450.032

Supplementary Table 1. Supplementary Table 1 | Results on mental health disorder datasets. EER is reported with its 95% CI. Other metrics except CER/WER are reported as mean ±\pm standard deviation. For BACC, pp-values were computed using one-sided exact binomial tests on discordant paired-subject-level predictions, comparing the predictions from anonymised and original speech. No adjustment was made for multiple comparisons. Results with p<0.05p < 0.05 are shown in bold.

DatasetAnonymiseSNRMOSPCCF0_{F0}CER/WEREmoEERBACCp-value
Saarbrücken—HDOrigin25.10±5.9825.10 \pm5.983.29±0.453.29 \pm0.45----0.743±0.0130.743 \pm0.013-
Saarbrücken—HDPitch24.42±6.4024.42 \pm6.401.55±0.231.55 \pm0.230.966±0.1030.966 \pm0.1030.0050.0050.949±0.0550.949 \pm0.0550.422 (0.387−0.461)0.422\ (0.387 - 0.461)0.717±0.0170.717 \pm0.0170.1360.136
Saarbrücken—HDFormantLPC22.47±4.9422.47 \pm4.942.01±0.412.01 \pm0.410.780±0.4250.780 \pm0.4250.0290.0290.908±0.0640.908 \pm0.0640.258 (0.231−0.292)0.258\ (0.231 - 0.292)0.764±0.0110.764 \pm0.0110.8400.840
Saarbrücken—HDFormantspec22.98±5.7322.98 \pm5.731.92±0.471.92 \pm0.470.960±0.1340.960 \pm0.1340.0100.0100.973±0.0390.973 \pm0.0390.242 (0.213−0.272)0.242\ (0.213 - 0.272)0.736±0.0090.736 \pm0.0090.4170.417
Saarbrücken—HDNeural VC30.97±8.2530.97 \pm8.252.83±0.352.83 \pm0.350.443±0.4400.443 \pm0.4400.0220.0220.922±0.0720.922 \pm0.0720.510 (0.478−0.546)0.510\ (0.478 - 0.546)0.684±0.0210.684 \pm0.0210.0110.011
Saarbrücken—HDResyn48.41±7.9148.41 \pm7.913.20±0.243.20 \pm0.240.068±0.4570.068 \pm0.4570.0150.0150.913±0.0700.913 \pm0.0700.517 (0.478−0.550)0.517\ (0.478 - 0.550)0.486±0.0130.486 \pm0.0133.73×10−143.73 \times10^{-14}
Saarbrücken—LaryngitisOrigin24.93±6.1524.93 \pm6.153.19±0.503.19 \pm0.50----0.807±0.0310.807 \pm0.031-
Saarbrücken—LaryngitisPitch24.25±6.5024.25 \pm6.501.53±0.211.53 \pm0.210.966±0.1080.966 \pm0.1080.0270.0270.943±0.0570.943 \pm0.0570.438 (0.392−0.478)0.438\ (0.392 - 0.478)0.814±0.0420.814 \pm0.0420.6860.686
Saarbrücken—LaryngitisFormantLPC22.31±5.0422.31 \pm5.041.94±0.411.94 \pm0.410.796±0.4510.796 \pm0.4510.0290.0290.908±0.0640.908 \pm0.0640.224 (0.186−0.259)0.224\ (0.186 - 0.259)0.771±0.0570.771 \pm0.0570.0920.092
Saarbrücken—LaryngitisFormantspec22.85±5.8922.85 \pm5.891.94±0.501.94 \pm0.500.960±0.1310.960 \pm0.1310.0370.0370.969±0.0400.969 \pm0.0400.221 (0.183−0.250)0.221\ (0.183 - 0.250)0.789±0.0260.789 \pm0.0260.2710.271
Saarbrücken—LaryngitisNeural VC30.52±7.6530.52 \pm7.652.79±0.372.79 \pm0.370.463±0.4260.463 \pm0.4260.0430.0430.907±0.0690.907 \pm0.0690.460 (0.422−0.511)0.460\ (0.422 - 0.511)0.739±0.0680.739 \pm0.0680.0110.011
Saarbrücken—LaryngitisResyn48.17±7.2248.17 \pm7.223.24±0.263.24 \pm0.260.057±0.4690.057 \pm0.4690.0160.0160.901±0.0680.901 \pm0.0680.526 (0.480−0.566)0.526\ (0.480 - 0.566)0.489±0.0690.489 \pm0.0693.08×10−143.08 \times10^{-14}
Saarbrücken—RekurrenspareseOrigin24.74±5.7024.74 \pm5.703.04±0.663.04 \pm0.66----0.901±0.0120.901 \pm0.012-
Saarbrücken—RekurrensparesePitch24.18±6.1424.18 \pm6.141.52±0.211.52 \pm0.210.952±0.1570.952 \pm0.1570.0250.0250.951±0.0540.951 \pm0.0540.397 (0.358−0.427)0.397\ (0.358 - 0.427)0.870±0.0200.870 \pm0.0200.0200.020
Saarbrücken—RekurrenspareseFormantLPC22.29±4.8322.29 \pm4.831.88±0.411.88 \pm0.410.757±0.4400.757 \pm0.4400.0370.0370.911±0.0650.911 \pm0.0650.234 (0.207−0.263)0.234\ (0.207 - 0.263)0.894±0.0280.894 \pm0.0280.3800.380
Saarbrücken—RekurrenspareseFormantspec22.84±5.5122.84 \pm5.511.84±0.481.84 \pm0.480.955±0.1630.955 \pm0.1630.0180.0180.975±0.0380.975 \pm0.0380.200 (0.173−0.229)0.200\ (0.173 - 0.229)0.877±0.0210.877 \pm0.0210.0490.049
Saarbrücken—RekurrenspareseNeural VC30.94±7.7730.94 \pm7.772.74±0.362.74 \pm0.360.409±0.4220.409 \pm0.4220.0250.0250.916±0.0730.916 \pm0.0730.500 (0.465−0.535)0.500\ (0.465 - 0.535)0.830±0.0230.830 \pm0.0238.81×10−58.81 \times10^{-5}
Saarbrücken—RekurrenspareseResyn48.19±7.0948.19 \pm7.093.22±0.263.22 \pm0.260.017±0.4710.017 \pm0.4710.0050.0050.909±0.0690.909 \pm0.0690.493 (0.456−0.526)0.493\ (0.456 - 0.526)0.473±0.0250.473 \pm0.0257.10×10−147.10 \times10^{-14}
PUMCHOrigin23.17±7.8123.17 \pm7.811.32±0.071.32 \pm0.07----0.699±0.0900.699 \pm0.090-
PUMCHPitch22.62±7.2722.62 \pm7.271.37±0.121.37 \pm0.120.866±0.1580.866 \pm0.1580.1920.1920.951±0.0630.951 \pm0.0630.349 (0.324−0.379)0.349\ (0.324 - 0.379)0.644±0.0330.644 \pm0.0330.0580.058
PUMCHFormantLPC24.55±7.2324.55 \pm7.231.25±0.021.25 \pm0.020.693±0.3270.693 \pm0.3270.1540.1540.954±0.0560.954 \pm0.0560.214 (0.191−0.237)0.214\ (0.191 - 0.237)0.691±0.0320.691 \pm0.0320.4450.445
PUMCHFormantspec22.22±7.3622.22 \pm7.361.24±0.021.24 \pm0.020.934±0.0990.934 \pm0.0990.1150.1150.967±0.0480.967 \pm0.0480.119 (0.099−0.140)0.119\ (0.099 - 0.140)0.648±0.0660.648 \pm0.0660.0550.055
PUMCHNeural VC41.36±1.5341.36 \pm1.531.87±0.181.87 \pm0.180.214±0.2580.214 \pm0.2580.2410.2410.933±0.0730.933 \pm0.0730.488 (0.465−0.521)0.488\ (0.465 - 0.521)0.641±0.0630.641 \pm0.0630.0450.045
PUMCHResyn42.12±13.0042.12 \pm13.002.49±0.532.49 \pm0.530.004±0.1990.004 \pm0.1990.0450.0450.935±0.0680.935 \pm0.0680.440 (0.415−0.474)0.440\ (0.415 - 0.474)0.526±0.0700.526 \pm0.0701.35×10−141.35 \times10^{-14}

Supplementary Table 3. Results on voice disorder datasets.

Method-dependent differences in speaker traceability and speech information preservation. Results across datasets reveal consistent method-dependent patterns in speech quality (Figure 2a,Figure 2b), speech attributes preservation (Figure 2c-Figure 2e), and speaker anonymisation (Figure 2f). Signal-processing-based methods (Pitch, FormantspecFormant_{\mathrm{spec}}, FormantLPCFormant_{\mathrm{LPC}}) generally reduced perceptual quality while preserving pitch contours more effectively than neural voice conversion (Neural VC) and transcription–resynthesis (Resyn). Among these methods, FormantspecFormant_{\mathrm{spec}} showed the strongest overall preservation of linguistic and emotional information, but also yielded the lowest EER, indicating that preservation of speech characteristics came at the cost of weaker speaker anonymisation. Pitch provided stronger anonymisation, with EER values generally above 0.4, but introduced greater transcription errors. In contrast, Neural VC and Resyn exhibited a different trade-off profile. Neural VC consistently achieved near-random speaker-verification performance (EER 0.43–0.51), whereas Resyn showed greater variability across datasets (0.34–0.54). Both methods improved signal cleanliness (improved SNR), and Resyn further achieved the highest perceptual quality and lowest transcription error rates. However, these improvements were accompanied by substantial losses in pitch contour preservation (lower PCCF0_{\mathrm{F0}} values). Therefore, although anonymisation methods exhibited distinct behaviours, they consistently reflected the trade-off between preservation of speech information and effective speaker anonymisation.

Figure 2. Voiceprint anonymisation evaluation across methods and datasets. Boxplots show six evaluation metrics across datasets for each voiceprint anonymisation method: Δ\DeltaSNR (change in SNR, anonymised vs. original), Δ\DeltaMOS, PCCF0_{\mathrm{F0}}, CER/WER, Emo, and EER. Each dot represents a single subset/task. Box plots show the median (centre line), the first and third quartiles (box bounds), and whiskers extending to 1.5× the interquartile range. The dashed horizontal line in Δ\DeltaSNR and Δ\DeltaMOS at zero indicates no change relative to the original audio. The heatmap shows Δ\DeltaBACC. Negative values (blue) indicate performance degradation and positive values (red) indicate improvement. Statistical significance was assessed using one-sided exact binomial tests on discordant paired subject-level predictions. No adjustment was made for multiple comparisons. Exact PP values are provided in Supplementary Table 1–Supplementary Table 3. Asterisks denote significance levels: * P<0.05P < 0.05, ** P<0.01P < 0.01, *** P<0.001P < 0.001.

Task-dependent effects on downstream clinical classification performance. In contrast to the largely method-dependent patterns in speaker anonymisation and the preservation of general speech characteristics, changes in downstream clinical classification performance varied substantially across tasks (Figure 2g). Voice disorder tasks were particularly sensitive to anonymisation. Across the Saarbrücken subsets and the PUMCH dataset, Neural VC and Resyn caused significant performance degradation, with Resyn reducing classification accuracy to nearly random levels. FormantLPCFormant_{\mathrm{LPC}} and FormantspecFormant_{\mathrm{spec}} generally produced smaller declines, suggesting that local acoustic transformations are more likely to preserve fine-grained cues related to abnormal phonation and articulatory impairment. Mental health and cognitive impairment tasks generally showed greater tolerance to anonymisation. Their diagnostic cues may rely more on linguistic content, pause patterns, or broader prosodic characteristics than highly detailed acoustic abnormalities, allowing multiple anonymisation methods to preserve classification performance. The speech elicitation paradigms also affected the choice of anonymisation methods. In reading-speech datasets with fixed content, such as Androids-R and Ivanova, Resyn led to significant classification degradation. Resynthesising speech from transcriptions resets speaking rhythm, prosody, and abnormal voice characteristics, thereby removing informative cues which are relevant to disease recognition.

These findings highlight two considerations for voiceprint anonymisation in clinical speech. First, stronger speaker anonymisation does not necessarily lead to a better outcome, as methods that provide greater identity protection may also disrupt clinically relevant speech information, whereas methods that better preserve speech characteristics may offer weaker speaker protection. Second, the consequences of such information loss on clinical utility are task-dependent, as different disorders and elicitation paradigms rely on distinct speech characteristics. These findings motivate CASA’s task-adaptive optimisation of voiceprint anonymisation strategies to achieve sufficient speaker privacy while preserving the speech information required for each clinical task.

Anonymisation of sensitive spoken content

Voiceprint anonymisation alone does not address sensitive personal information explicitly disclosed through spoken content. CASA therefore performs content anonymisation by identifying privacy-sensitive information in speech transcripts and masking the corresponding spoken content. To avoid transmitting sensitive clinical information to external model services, CASA calls a locally deployed open-source LLM, Qwen2.5-32B-Instruct [31], for sensitive-content localisation. We analysed CASA’s content anonymisation on two in-house datasets containing sensitive spoken information: Crisis, comprising psychological hotline conversations that may disclose personal names, residential locations and telephone numbers, and PUMCH, in which a brief clinician–participant exchange involving names and ages precedes a speech repetition task.

We first examined CASA’s performance in sensitive-content localisation. We compared CASA’s LLM-based localisation with the named-entity recognition (NER)-based approach used in SecureSpeech, using manually annotated privacy-sensitive spans as ground truth. CASA outperformed SecureSpeech on both datasets (Figure 3a,Figure 3b), achieving F1 scores of 0.861 versus 0.570 on Crisis and 0.914 versus 0.343 on PUMCH.

Figure 3. Content anonymisation evaluation. a,b, Content anonymisation on two in-house datasets, comparing CASA’s locally deployed Qwen2.5-32B-Instruct model under default and task-adaptive prompts with the NER-based localisation used in SecureSpeech. c, Comparison of Qwen2.5-32B-Instruct and GPT-5-nano on precision, recall and F1 for sensitive-content localisation. Privacy-sensitive entities were replaced with synthetic names and locations for closed-source model evaluation. d,e, Downstream task performance with and without content anonymisation. Statistical significance was assessed using one-sided exact binomial tests on discordant paired subject-level predictions, and no statistically significant reductions were observed across all evaluations. No adjustment was made for multiple comparisons.

We next investigated the contribution of task-adaptive prompting to sensitive-content localisation. Using the same self-deployed LLM, we compared a default prompt with task-adaptive prompts refined by CASA according to dataset-specific contexts. The default prompt asked the model to identify common privacy-sensitive content, including names, numerical identifiers, and address information. The task-adaptive prompt was refined according to dataset-specific content, for example by excluding descriptions of psychological state from private identifiers in Crisis and by highlighting name-like nouns at the start of PUMCH recordings. As shown in Figure 3a,Figure 3b, task-adaptive prompting yields marked improvement on localisation performance compared with the default prompt on both datasets with F1 scores exceeding 0.85.

We then assessed whether CASA’s locally deployed open-source LLM could achieve competitive sensitive-content localisation performance compared with a closed-source model. We compared Qwen2.5-32B-Instruct with GPT-5-nano [32] using a synthetic-privacy version of Crisis (Figure 3c). To avoid exposing real private information to the closed-source model during this comparison, all privacy-sensitive spans were manually replaced with fictitious but realistic names, locations, and other identifiers, producing a synthetic-privacy version for evaluation. Results show that Qwen2.5 achieved comparable performance as GPT-5-nano under the same synthetic-privacy setting, suggesting that the open-source Qwen2.5 LLM, when paired with appropriate prompts, is sufficient for practical content anonymisation in clinical speech.

Finally, we assessed whether content anonymisation introduced additional losses in downstream clinical utility. We compared downstream BACC with and without content anonymisation across different voiceprint anonymisation methods on Crisis and PUMCH (Figure 3d,Figure 3e). No statistically significant reductions in classification performance were detected across the evaluated configurations, suggesting that privacy-sensitive spoken content can be removed without substantially affecting downstream clinical utility, enabling targeted content protection while preserving clinically relevant information.

Experience-guided self-improvement enables efficient task adaptation

The preceding analyses highlight the importance of adapting anonymisation strategies to the requirements of different clinical tasks. CASA achieves this through iterative evaluation and refinement of anonymisation strategies against task-specific privacy–utility requirements. To improve the efficiency of this iterative process, CASA incorporates a recursive self-improvement mechanism that leverages experience accumulation across tasks. Specifically, resulting experience is retained in two levels of memory for subsequent planning: fine-grained memory stores anonymisation configurations and evaluation results from previous runs, and coarse-grained memory summarises task-level lessons and recurring patterns. These memories guide subsequent strategy planning, reducing the need for repeated evaluations.

To quantify the contribution of accumulated experience, we compared CASA with and without memory under the same task-specific requirements. In the memory-enabled condition, CASA could retrieve experience from other tasks, whereas the memory-free condition had no access to previous cross-task experience. Both conditions retained within-task feedback-guided adaptation. Success required satisfying all privacy and utility thresholds specified for each run. As shown in Figure 4, with memory, the proportion of runs succeeding on the first evaluation increased from 46.4% to 62.9%, and the proportion succeeding within two evaluations increased from 57.1% to 81.4%. The mean number of strategy evaluations needed to reach a satisfactory configuration decreased by 31.3%, dropping from 2.27 to 1.56 steps. These results show that accumulated cross-task experience reduces the number of anonymisation and evaluation cycles needed for task adaptation, thus enabling CASA to identify satisfactory anonymisation strategies more efficiently.

Figure 4. Experience-guided self-improvement improves the efficiency of task adaptation. The upper right is a comparison of CASA without and with memory. Success@1 and Success@2 (left axis) denote the proportions of runs satisfying all specified privacy and utility requirements within one and two evaluation steps, respectively. Mean hit steps (right axis) denotes the average number of steps required to first reach a configuration satisfying these requirements. Higher success rates and fewer steps indicate more efficient adaptation. The other part shows a representative execution trace for Androids-R, showing how experiences from previous runs guide CASA in subsequent run.

A representative example from Androids-R illustrates how accumulated experience guides strategy planning (Figure 4). This involves an Italian reading task for depression detection, with user-defined requirements of PCCF0>0.70\mathrm{PCC}_{\mathrm{F0}} > 0.70 and EER>30%\mathrm{EER} > 30\%. CASA retrieved general lessons concerning the preservation of clinically relevant acoustic characteristics, together with detailed evaluation results from previous depression-related tasks. Guided by these experiences, CASA selected FormantLPC\mathrm{Formant}_{\mathrm{LPC}} as its first candidate. Evaluation on Androids-R yielded PCCF0=0.791\mathrm{PCC}_{\mathrm{F0}} = 0.791 and EER=35.3%\mathrm{EER} = 35.3\%, satisfying both requirements in a single trial. This example illustrates how CASA can leverage experience from related clinical tasks with different paradigms to prioritise promising anonymisation strategies, with suitability verified for the current task.

Clinical expert assessment of anonymised speech

We further involved human experts to assess the perceived clinical usefulness of anonymised speech and its acceptability for research sharing across three clinical tasks: psychological crisis assessment, suicide risk assessment, and cleft-palate speech assessment, corresponding to the Crisis, Suicide-YF, and PUMCH datasets, respectively. For psychological crisis and cleft-palate speech assessment, clinicians evaluated crisis level and nasality severity, respectively. Agreement was calculated between the corresponding labels assigned from original and anonymised speech. For suicide risk assessment, where reference labels were not derived from speech, clinicians rated the preservation of speech information relevant to suicide-risk prediction, including acoustic-feature-related and semantic-feature-related information. In addition, clinicians rated three aspects of anonymisation quality: (i) voiceprint anonymisation, which reflects whether the speaker could be tracked from voice characteristics; (ii) content anonymisation, which reflects whether sensitive spoken information was detected and masked; (iii) sharing acceptability, which reflects whether the anonymised recording was considered suitable for sharing with external researchers under a data-use agreement.

Figure 5a shows that clinical agreement remained high after anonymisation for both psychological crisis and cleft-palate speech assessment tasks. For suicide risk assessment, as shown in Figure 5b, semantic information was well preserved, with a score of 4.78, whereas acoustic information received a relatively lower score of 3.56. This is consistent with the voiceprint anonymisation results above, where Resyn preserved linguistic content but reduced preservation of original acoustic and prosodic cues. Clinicians also gave high scores for anonymisation quality and sharing acceptability, shown in Figure 5c. Voiceprint anonymisation was rated highly across all three datasets, including PUMCH, where clinicians considered FormantLPC\mathrm{Formant}_{\mathrm{LPC}} to provide acceptable voice protection despite its lower EER. Content anonymisation was rated highly on PUMCH, while the score was lower on Crisis. This may be because hotline conversations are long, open-ended and context-dependent, making complete localisation of sensitive spoken information more difficult. Importantly, all three datasets received sharing acceptability scores above 4, indicating that clinicians considered the anonymised recordings suitable for controlled external research sharing. These findings provide practical evidence that the proposed anonymisation agent can help improve clinical speech data sharing while preserving information needed for downstream clinical use.

Figure 5. Clinical expert evaluation of anonymised speech. a, Agreement between assessments of original and anonymised recordings, using crisis-level labels for Crisis and nasality-severity ratings for PUMCH. b, Preservation rating of acoustic and semantic information on the Suicide-YF dataset. c, Anonymisation quality and sharing acceptability rating. Clinicians rated three aspects: voiceprint anonymisation, content anonymisation, and sharing acceptability. All subjective ratings were collected on a scale of 1 to 5, with higher scores indicating better performance. Evaluators were blinded to the original reference labels.

Discussion

Clinical speech anonymisation requires balancing speaker identity protection with the preservation of information needed for clinical assessment. Voiceprint anonymisation methods that strongly suppress speaker identity may disrupt acoustic, prosodic or linguistic characteristics, whereas methods that better preserve these characteristics may offer weaker identity protection. The appropriate balance varies across clinical tasks because different disorders and speech elicitation paradigms rely on distinct speech characteristics for assessment. Our analyses showed that the same anonymisation method could preserve downstream classification performance in some tasks but substantially impair it in others. Beyond voiceprint anonymisation, clinical speech recordings may also disclose sensitive personal information through spoken content. Our content anonymisation experiments showed that identifying such information also benefits from task adaptation that accounts for differences in clinical context and speech elicitation paradigms. In sum, these findings suggest that clinical speech anonymisation needs to account for both the privacy–preservation balance and the specific requirements of different clinical tasks.

CASA addresses these challenges by reformulating clinical speech anonymisation as a task-adaptive optimisation problem, in which strategies are planned, executed and iteratively refined using multi-dimensional evaluation feedback. Incorporating measured outcomes as feedback is necessary since the clinical relevance of individual speech characteristics cannot always be determined a priori. Even similar clinical tasks may differ in data collection, participant populations and labelling criteria, affecting which speech characteristics are predictive of downstream outcomes. CASA uses downstream classification performance to directly quantify the utility of anonymised speech, reducing reliance on prior assumptions about clinically relevant speech characteristics. Furthermore, through recursive self-improvement, CASA accumulates successful strategies and lessons from previous evaluations to guide subsequent planning and reduce repeated exploration. This creates a continuous cycle of evaluation, experience accumulation and reuse, allowing CASA to adapt more efficiently to new clinical tasks as its experience grows.

CASA also supports human oversight of automated anonymisation. Intermediate outputs, including transcriptions, detected privacy-sensitive spans, timestamps and evaluation results, are maintained as transparent and editable representations, allowing clinical researchers and data managers to inspect, modify and approve anonymisation decisions when necessary. This human-in-the-loop design provides an additional layer of control when handling sensitive patient information. Clinical expert assessments further supported CASA’s practical value, showing high agreement between clinical judgements based on original and anonymised recordings, alongside favourable ratings for anonymisation quality and research-sharing acceptability. Following CASA’s anonymisation, a real-world cross-institutional clinical data sharing was conducted, where the Suicide-YF dataset was shared with Shanghai Artificial Intelligence Laboratory. By reducing speaker traceability and masking sensitive spoken content while preserving clinically relevant information, CASA could help overcome privacy-related barriers to cross-institutional clinical speech sharing. Such capabilities could facilitate collaborative research across hospitals and institutions, supporting the development of speech-based screening and assessment systems under appropriate data-use agreements.

Several limitations should be considered. First, sensitive-content localisation and voiceprint anonymisation methods such as transcription–resynthesis require automatic speech recognition (ASR) to generate speech transcripts. ASR performance may decline for low-resource languages, dialectal speech, child speech and severely pathological speech. Recognition errors can propagate to subsequent anonymisation procedures, potentially compromising privacy protection or the preservation of clinically relevant information. This limitation could be mitigated through domain-specific adaptation of ASR models for the target population or clinical setting. Second, the present study evaluated CASA using a limited set of anonymisation tools, comprising five voiceprint anonymisation methods and an LLM-based content anonymisation approach. Nevertheless, CASA’s modular design allows additional anonymisation and evaluation tools to be integrated, potentially broadening its applicability to a wider range of clinical tasks. Finally, human review may remain necessary for high-risk applications where residual privacy leakage is unacceptable. To accommodate this need, CASA’s human-in-the-loop design supports such oversight through transparent and editable intermediate outputs, allowing researchers and data managers to inspect, correct and approve anonymisation decisions when necessary.

Looking ahead, CASA could enable a positive feedback loop between privacy-preserving data sharing and experience-guided adaptation. By facilitating the controlled sharing of clinical speech, CASA could make more diverse data available for clinical model development, while generating anonymisation experience across a broader range of tasks. Through RSI, this accumulated experience could further improve strategy planning for new clinical tasks, potentially lowering barriers to data sharing. Future research should investigate how this reciprocal process develops across institutions and increasingly diverse clinical settings.

Methods

We developed and evaluated CASA through a modular processing pipeline comprising transcription, voiceprint anonymisation, content anonymisation, quantitative evaluation and clinical expert assessment.

CASA agent

CASA is implemented as an LLM-based agent that plans and executes clinical speech anonymisation according to task-specific requirements. The system takes original speech recordings as input and produces anonymised recordings through voiceprint modification and sensitive-content masking. The controller receives a natural-language description of the clinical objective, speech paradigm and user-defined privacy and utility requirements. These instructions guide planning, tool execution and subsequent adaptation.

Tools. Transcription, voiceprint anonymisation, content anonymisation and evaluation are implemented as independent tools with explicitly defined inputs, outputs and dependency constraints (see Supplementary Figure S1). Intermediate outputs are stored as structured JSON files, enabling both the agent and users to inspect and modify processing results before subsequent stages are executed.

Figure S1. Detailed workflow of CASA anonymisation. Original recordings are transcribed using an ASR module to obtain transcripts with speaker annotations and word-level timestamps. Available voiceprint anonymisation methods include pitch shifting (Pitch), spectral formant scaling (Formantspec_{\mathrm{spec}}), LPC-based formant transformation (FormantLPC_{\mathrm{LPC}}), neural voice conversion (Neural VC) and transcription–resynthesis (Resyn). For content anonymisation, a locally deployed LLM identifies sensitive spans in the transcripts and produces annotations to guide redaction. For Resyn, which doesn’t preserve temporal alignment, sensitive content is masked in the transcript before synthesis; while for other methods, the corresponding speech segments are replaced with noise. The resulting recordings are evaluated for speaker privacy (EER), downstream clinical utility (BACC), signal cleanliness (SNR), perceptual quality (MOS), pitch-contour preservation (PCCF0_0), linguistic preservation (CER/WER) and emotion preservation (Emo).

Memory. The agent maintains a two-level memory to support strategy planning and feedback-guided adaptation. A coarse-grained experience memory stores summaries of previous experiments, recurring processing patterns, and failure cases with lessons learned, providing reusable prior knowledge across tasks. A fine-grained experiment memory records run-specific details, including evaluation metrics, tool parameters, and intermediate JSON outputs and logs, enabling precise diagnosis and reproduction of each anonymisation run. In addition, the current-task history is maintained and updated throughout execution, linking the two memory levels to the ongoing planning and adaptation process. After each run, the agent stores the configuration and evaluation results in fine-grained memory and distils the outcome into reusable lessons for coarse-grained memory.

Planning and execution. Given a natural-language anonymisation request, the agent first constructs an anonymisation strategy by selecting processing modules and configuring their corresponding parameters. Strategy generation is performed using an LLM (i.e., Qwen3.8-27B[33]), which is provided with descriptions of available tools, parameter constraints, workflow dependencies, and relevant entries retrieved from memory. The model was run with a temperature of 0.01 and a repetition penalty of 1.05 to encourage deterministic strategy generation while reducing repetitive outputs. The resulting strategy is then executed to produce anonymised speech.

Evaluation and adaptation. Following execution, the evaluation module assesses the speaker privacy, downstream clinical utility and preservation of speech characteristics of the anonymised recordings, as detailed in Methods, “Evaluation methods”. The agent subsequently analyses the evaluation result to determine whether the anonymisation objectives have been satisfied [34]. If the current strategy fails to meet the specified requirements, the agent re-plans the anonymisation strategy according to evaluation feedback, the current-task history, and previously executed processing steps retrieved from memory, and iteratively explores alternative configurations until the objectives are satisfied or a maximum of 7 adaptation iterations is reached. The agent first explores alternative anonymisation methods at their default settings and adjusts method parameters when none satisfies the specified requirements. The agent also supports manually specified anonymisation workflows if needed.

To evaluate whether the memory module improves adaptation efficiency, we constructed 10 anonymisation requests for each of the 14 datasets or task subsets, yielding 140 requests in total. Each request identified the target dataset or task subset and specified two simultaneous requirements: an EER not below 20%, 30% or 40%, together with one constraint on downstream performance or speech preservation. The second constraint required a BACC decrease of less than three or six percentage points relative to original speech, a CER/WER below 15% or 20%, an emotion similarity above 0.93, or a pitch-contour correlation (PCCF0\mathrm{PCC}_{\mathrm{F0}}) above 0.70 or 0.80. We evaluated the same 140 requests with and without cross-task memory. In the memory-enabled condition, the agent could retrieve general lessons and detailed experimental records from the other 13 tasks, with experience from the task under evaluation excluded. In the memory-free condition, no cross-task experience was provided. Both conditions retained within-task planning and evaluation-guided adaptation. Success@1 and Success@2 as the proportions of requests reaching a satisfactory configuration within one and two evaluation steps, respectively, together with the mean number of steps required to reach the first satisfactory configuration.

Transcription methods

The transcription module converts raw speech recordings into structured intermediate representations containing sentence-level transcripts, word-level timestamps, sentence indices and speaker annotations. Speech segments associated with the same speaker are aggregated and processed using a pretrained age–gender recognition model [35, 36] to infer speakers’ age and gender attributes, which are subsequently stored as auxiliary metadata in the transcription files. Mandarin Chinese recordings are transcribed using Paraformer [37]. Recordings in other languages are transcribed using WhisperX [38].

Voiceprint anonymisation methods

The voiceprint anonymisation module supports three categories of anonymisation methods: signal-processing-based transformations, neural voice conversion, and transcription–resynthesis. Signal-processing-based methods include pitch shifting by semitone steps (Pitch), formant shifting through spectral scaling (Formantspec), and formant shifting through Linear Predictive Coding (LPC) pole transformation (FormantLPC) implemented by McAdams. Neural voice conversion is implemented using SeedVC, whereas transcription–resynthesis is implemented through an ASR–TTS (Text-to-Speech) cascade.

Pitch. Pitch-based anonymisation is performed using the librosa toolkit. According to speaker-level gender estimates obtained during transcription, a positive offset of 3 semitones is applied to male speakers and a negative offset of 3 semitones is applied to female speakers.

Formantspec_{\mathrm{spec}}. Formant-based anonymisation is implemented by applying short-time Fourier transform analysis followed by linear interpolation-based spectral scaling with a scaling factor of 1.1. This operation approximates modifying the effective vocal tract length of the speaker.

FormantLPC_{\mathrm{LPC}}. McAdams anonymisation [39] leverages Linear Predictive Coding (LPC) to alter vocal tract characteristics by shifting the formant locations. For each frame, the angles of the LPC poles are transformed by a predefined power coefficient, and the speech is then resynthesised from the original residual signal with the modified LPC filter, changing the distribution of formant frequencies. An LPC order of 20 and a power coefficient of 0.8 were used.

Neural VC. Neural voice conversion anonymises speech by modifying the original waveform or acoustic features. We perform neural VC using the SeedVC[40] framework. For multi-speaker recordings, speaker labels obtained during the transcription stage are used to assign different target speaker embeddings sampled from the AISHELL-1 [41] corpus to different speakers within the same recording. The converted speech preserves the original temporal alignment and therefore does not alter timestamp annotations.

Resyn. Transcription–resynthesis generates anonymised speech directly from transcribed text. We used FishSpeech [42] as the TTS backbone, with language-matched reference utterances from Mozilla Common Voice [43] and Multilingual LibriSpeech [44] used to specify target speaker identities.

Content anonymisation methods

The content anonymisation module involves two stages: privacy localisation and audio redaction. The privacy localisation stage identifies privacy-sensitive linguistic spans from transcription output and generates structured annotations containing detected text segments together with their corresponding timestamp ranges. Localisation is implemented using an LLM. By default, a locally deployed Qwen2.5-32B-Instruct model is used to avoid transmitting transcripts to external services and privacy leakage. The model is prompted to identify common forms of spoken private information, including names, numerical identifiers, and address information. To support long-form recordings, transcription outputs are segmented into groups of consecutive sentences before localisation.

The localisation module supports configurable prompts and model backends. Task-specific prompts can be provided to adapt privacy localisation to different recording protocols, clinical settings and languages. Alternative locally deployed LLMs can also be integrated to support different deployment environments.

Following privacy localisation, audio redaction is performed according to the anonymisation method. For methods that preserve timestamp alignment (Pitch, Formantspec, FormantLPC, and Neural VC), detected privacy-sensitive spans are redacted in the voiceprint-anonymised audio by replacing the corresponding speech segments with noise. For Resyn, privacy localisation and masking are first applied to the transcription, after which anonymised speech is generated from the modified text.

Evaluation methods

We build a multi-dimensional evaluation framework to comprehensively assess the effectiveness of anonymisation methods.

Speech quality. Speech quality is evaluated using signal-to-noise ratio (SNR) and mean opinion score (MOS). SNR measures cleanliness of the speech while MOS measures the overall perceptual quality of the speech. MOS is automatically estimated using UTMOS [45].

Contour preservation. Pitch preservation is evaluated through fundamental frequency (F0) contour analysis. F0 contours are extracted from both original and anonymised speech using the librosa toolkit. Preservation of pitch-contour shape is quantified using the Pearson correlation coefficient (PCC) between the two contours.

Linguistic preservation. Linguistic preservation is evaluated by comparing transcriptions generated from the original and anonymised speech. For Mandarin Chinese recordings, preservation is quantified using character error rate (CER). For other languages, word error rate (WER) is used.

Emotion preservation. Emotion preservation is evaluated using Emotion2Vec-Plus-Seed [46]. Emotion embeddings are extracted from both original and anonymised speech, and preservation is quantified using cosine similarity between the corresponding embeddings.

Speaker anonymisation. Speaker anonymisation is evaluated using equal error rate (EER), with higher EER indicating weaker speaker traceability. Speaker embeddings are extracted using the WavLM-Base-Plus-sv speaker verification model [47]. Positive verification pairs consist of original and anonymised recordings from the same subject, whereas negative pairs consist of original and anonymised recordings from different subjects. Verification scores are computed as cosine similarities between speaker embeddings. EER was estimated at the ROC operating point that minimised the absolute difference between the false acceptance and false rejection rates, and was calculated as the mean of these two rates at that point. The 95% confidence interval was estimated using a nonparametric percentile bootstrap with 1,000 resampling iterations, with EER recalculated for each resample, and the 2.5th and 97.5th percentiles of the valid bootstrap estimates defined the confidence interval. The framework supports user-provided subject mappings, allowing multiple recordings to be treated as originating from the same subject during verification. When no subject mapping is provided, each recording is treated as an independent subject.

Clinical utility. Downstream clinical utility is evaluated through disease-related classification. We use the speech LLM Qwen2.5-Omni-7B [48] as the backbone of an end-to-end classification model, following our previous work [22] (Supplementary Figure S2). The model consists of a speech encoder, an LLM, and a fully connected classification head that maps the final representation to disease-label predictions. Weight-decomposed low-rank adaptation (DoRA) [49] is used for parameter-efficient fine-tuning with a rank of 32 and an alpha of 64. The learning rate is set to 1×10−41 \times10^{-4}, and the number of training epochs is adjusted according to dataset size. To reduce the influence of class imbalance, minority classes in the training set are upsampled until all classes contain the same number of samples. Balanced accuracy (BACC) is used as the primary metric for downstream utility. Long recordings are divided into non-overlapping 4-minute segments, with each treated as a separate sample during training and inference. The final recording-level prediction is obtained by averaging segment-level class probabilities at test time. For each dataset, we trained a separate classifier on the original speech and on each anonymised version, using the corresponding speech version for both training and evaluation.

Figure S2. Downstream model structure.

For datasets with official test splits, evaluation is performed on the official test set using five independent runs with different random seeds. For datasets without official splits, five-fold cross-validation is used. In both settings, train, validation, and test sets are strictly subject-independent to avoid data leakage and to evaluate cross-subject generalisation. Mean BACC and standard deviation are reported across the five runs or folds. Statistical significance of utility degradation after anonymisation was assessed using one-sided exact binomial tests on discordant paired predictions, using scipy.stats.binomtest. No adjustment was made for multiple comparisons. A lower PP value indicates stronger evidence that anonymisation reduces downstream classification performance. Statistical significance was defined as P<0.05P < 0.05.

Datasets for validation

MODMA [23] is a Mandarin Chinese clinical speech dataset comprising 1,503 recordings from 52 participants aged 16–56 years, including outpatients diagnosed with depression (44.2%) and healthy controls (55.8%). Speech was collected through multiple elicitation paradigms, including interview, reading and picture-description tasks.

Androids-Corpus [24] comprises recordings from 118 native Italian speakers classified as healthy controls (HC) or patients (PT) with depression-related disorders. Reading recordings (54 HC, 58 PT) were collected using Aesop’s fable The North Wind and the Sun, whereas interview recordings (52 HC, 64 PT) were obtained through spontaneous responses to daily-life questions.

Suicide-YF consists of Mandarin Chinese speech recordings from 1,223 adolescents aged 10–18 years recruited in Guangdong, China. The participants were asked to complete an online survey, in which they were asked whether they had ever experienced suicidal thoughts or behaviours. Those who answered yes were invited to participate in the study. Additionally, a control group of children with no suicide risk was selected for comparison with the at-risk participants. All recordings are paired with suicide-risk assessments administered using the Mini International Neuropsychiatric Interview for Children and Adolescents (MINI-KID) suicidality module [50], with 53.4% of participants identified as at risk. Following previous studies [22, 51], only recordings from the self-description task were used. Informed consent was obtained from all participants and their guardians, and the study was approved by the Tsinghua University Institutional Review Board.

Crisis is an in-house Mandarin Chinese psychological-support hotline dataset comprising 154 recordings (approximately 100 hours in total; mean duration 39.49 minutes). The study was conducted at the Beijing Psychological Support Hotline which provides services to callers from around China, three-fourths of whom live outside of the Beijing Municipality. Hotline operators are full-time employees with a psychology or psychiatry background who receive at least 3 months full-time training in the provision of hotline services before independently taking calls. Recordings were independently annotated by two clinical experts into three crisis levels: no crisis (55 participants), low crisis (42), and medium-to-high crisis (57). Conversations cover a broad range of personal issues, including depression, anxiety, relationship stress and suicidal ideation, and may contain privacy-sensitive information such as names, locations and phone numbers. The study was approved by the Institutional Review Board of Beijing Huilongguan Hospital (2014-106, 2024-76). Before callers were connected to a hotline operator, they were informed by a voice message that all calls of the hotline would be tape-recorded and that data utilised for research purposes would be analysed anonymously.

NCMMSC-AD[25] is a Mandarin Chinese cognitive assessment dataset comprising 399 recordings from 175 participants labelled as healthy controls (HC), mild cognitive impairment (MCI), or Alzheimer’s disease (AD). Recordings include multiple cognitive tasks, including verbal fluency, lexical retrieval, picture description and self-introduction.

ADReSS[9] is an English Alzheimer’s disease dataset derived from the Pitt corpus, comprising recordings from 156 participants aged 50–79 years. The official split contains 54 HC and 54 AD participants for training, and 24 HC and 24 AD participants for testing. All recordings correspond to the Cookie Theft picture-description task.

Ivanova[10] is a Spanish cognitive assessment dataset comprising recordings from 357 participants aged 50–96 years. Participants were labelled as healthy controls (54.9%), mild cognitive impairment (24.4%), or Alzheimer’s disease (20.7%). All recordings correspond to a reading task in which participants read the first two sentences of Don Quixote by Cervantes.

NeuroVoz[26, 52] is a Spanish Parkinson’s disease dataset comprising recordings from 112 participants aged 31–88 years, including 58 healthy controls and 54 patients with Parkinson’s disease. The corpus includes multiple speech tasks, including sustained vowels, diadochokinetic tests, listen-and-repeat utterances, and spontaneous monologues.

ENNI[27] is an English narrative-speech dataset from the CHILDES Clinical English Edmonton Narrative Norms Instrument corpus, comprising recordings from Canadian children with typical development (286) or specific language impairment (66). The task is picture-elicited story generation, in which participants narrate a picture story to an examiner.

Saarbrücken[12] is a German pathological-voice dataset containing clinically validated voice-disorder labels. We selected three common voice-disorder categories from the database and constructed three disease-versus-control subsets from the read-sentence recordings: laryngitis (140 healthy, 140 pathological), Rekurrensparese (212 healthy, 212 pathological), and hyperfunctional dysphonia (212 healthy, 212 pathological). All recordings correspond to the sentence “Guten Morgen, wie geht es Ihnen?” (“Good morning, how are you?”)

PUMCH is an in-house Mandarin Chinese speech dataset for cleft-palate speech assessment, comprising recordings from 244 participants, including 128 patients with cleft palate and 116 healthy controls. Synchronized audio-video recordings were obtained primarily using an iPhone 16 in a quiet hospital room; additional home recordings were submitted using consumer devices under variable recording conditions. A researcher was present during hospital-based collection, and a parent or guardian could accompany minors when necessary; no researcher was present during home recording. Disease labels were annotated by experienced clinicians. Recordings consist of clinician–participant interactions beginning with identity-related questions (e.g., name and age), followed by a sentence-repetition task in which participants repeat utterances read by the clinician. This multicentre, prospective observational study was approved by the Ethics Committee of Peking Union Medical College Hospital, Chinese Academy of Medical Sciences, Beijing, China (approval number: I-25PJ0105). For adult participants, written informed consent was obtained from them directly; for minors, written informed consent was obtained from their parents or legal guardians, in accordance with the Declaration of Helsinki.

Fixed pipeline for baseline

To evaluate whether adaptive anonymisation provides advantages over conventional fixed strategies, we implemented SecureSpeech as a representative fixed anonymisation baseline. SecureSpeech adopts a sequential ASR–NER–LLM–TTS pipeline, where ASR is first applied to obtain transcripts, NER identifies privacy-sensitive entities, an LLM rewrites the detected spans, and a TTS model generates anonymised speech.

Since the original SecureSpeech implementation was developed for English speech only and its individual components do not directly support multilingual clinical speech evaluation, we adapted the pipeline for our experimental setting. Specifically, to enable privacy entity detection across multiple languages, the original English-only NER module was replaced with a RoBERTa-based NER model fine-tuned on the CLUENER2020 dataset [53] for Chinese speech processing, and a multilingual BERT-based NER model [54] trained on high-resource languages for multilingual scenarios. The original ParlerTTS-based synthesis module was replaced with FishSpeech to provide multilingual speech resynthesis capability. The LLM component for rewriting was replaced with locally deployed Qwen2.5-32B-Instruct. The ASR outputs were kept identical to those used in our framework to ensure a fair comparison.

External attacker evaluation

We evaluated CASA’s anonymised recordings using two external attackers: a semi-informed attacker trained on anonymised speech following the VoicePrivacy training protocol, and a SpeechLLM attacker using a pretrained speech–language model directly for speaker verification. Both attackers compared original and anonymised recordings to assess whether speaker identities remained linkable. Their outputs were used for external evaluation and were not provided to CASA during planning or adaptation.

Semi-informed attacker. We trained an ECAPA-TDNN [55] speaker-verification model on an anonymised version of LibriSpeech train-clean-360 [56]. Because the intended use of LibriSpeech is automatic speech recognition, CASA was instructed to preserve linguistic information with a CER/WER requirement below 10%. The anonymised training recordings were generated using CASA’s Resyn method. Model training followed the VoicePrivacy 2024 training recipe [29]. The resulting attacker was then applied to the anonymisation configurations produced by CASA for the clinical tasks. EER was calculated using the same pairing and scoring procedure described in “Evaluation methods”, replacing the agent’s speaker-embedding model with the ECAPA-TDNN attacker trained on CASA-anonymised LibriSpeech.

SpeechLLM attacker. We used SALMONN-2-8B [30] directly to classify pairs of recordings as originating from the same or different speakers. For each speaker shared between the original and anonymised recordings, we generated one same-speaker pair and one different-speaker pair, yielding 2N2N pairs for NN speakers. For every pair, the model received the first 15 seconds of the original recording and the last 15 seconds of the anonymised recording. When an individual recording was shorter than 30 seconds, its first or second half was used, respectively. A fixed prompt asked whether the main speakers in the two recordings were the same person and required exactly one response: “SAME” or “DIFFERENT”. Each pair was processed independently, with sampling disabled and a maximum output length of 256 tokens. The verification error, calculated as 1−accuracy1-\text{accuracy} was reported.

Clinical expert evaluation

We conducted clinical expert evaluations on three private clinical speech datasets to assess whether anonymised recordings preserved clinically relevant information and were acceptable for controlled research sharing. The expert panel comprised clinicians with relevant clinical experience and researchers extensively trained in the corresponding clinical domains. Evaluation protocols were tailored to the characteristics and assessment objectives of each clinical task. Throughout the clinical expert evaluations, evaluators were blinded to the original reference labels.

For the crisis assessment task, we randomly selected 60 recordings from the Crisis dataset for clinical relabelling, as the relatively long duration of the recordings made exhaustive expert evaluation impractical. Only the anonymised versions were presented to the experts, who independently assigned crisis labels following the same assessment procedure used to construct the original dataset. Labels derived from anonymised speech were compared with the original reference labels, and agreement was quantified as the proportion of matching labels. We additionally sampled a non-overlapping set of 30 recordings to assess anonymisation quality and acceptability for research sharing. For this evaluation, experts were provided with paired original and anonymised recordings. Two clinical experts jointly participated in the evaluation; each relabelled 30 anonymised recordings and assessed 15 original–anonymised pairs for anonymisation quality and sharing acceptability. The web interface is shown in Supplementary Figure 3 and Supplementary Figure 4.

Web interface for crisis relabelling

Supplementary Figure 3. Web interface for crisis relabelling.

Web interface for crisis anonymisation rating

Supplementary Figure 4. Web interface for crisis anonymisation rating.

For the suicide risk assessment task, 350 recordings were randomly selected from the Suicide-YF dataset and evaluated by one expert. Because the reference suicide-risk labels were derived from clinical assessments independent of the speech recordings, the expert was not asked to relabel the anonymised speech. Instead, paired original and anonymised recordings were presented to assess whether clinically relevant information for suicide-risk prediction was preserved after anonymisation. Preservation was evaluated along two dimensions: acoustic-feature-related information and semantic-feature-related information. As content anonymisation was not applied in this task, the anonymisation-quality assessment focused on voiceprint anonymisation and acceptability for research sharing. The web interface is shown in Supplementary Figure 5.

Web interface for Suicide-YF scoring

Supplementary Figure 5. Web interface for Suicide-YF scoring.

For the cleft-palate speech assessment task, all 244 recordings from the PUMCH dataset were evaluated by a speech-language pathologist. To simplify the evaluation procedure, rather than performing a comprehensive cleft-palate speech assessment covering multiple articulation and resonance errors, the evaluation focused on nasality severity as a clinically relevant perceptual feature, rated on a four-level ordinal scale: none, mild, moderate, and severe. Clinical information preservation was quantified by the exact agreement between ratings obtained from the original and anonymised recordings. The speech-language pathologist also assessed voiceprint anonymisation, content anonymisation, and acceptability for research sharing for each anonymised recording. The web interface is shown in Supplementary Figure 6.

Web interface for PUMCH evaluation

Supplementary Figure 6. Web interface for PUMCH evaluation.

Preservation, anonymisation-quality and sharing-acceptability scores were collected on a five-point scale, with higher scores indicating better preservation, stronger anonymisation or higher sharing acceptability. Mean scores were reported across evaluated recordings.

Reporting summary

Further information on research design is available in the Nature Research Reporting Summary linked to this article.

Data availability

All public datasets used in this study are available from their original sources, which are cited in the manuscript. The three in-house datasets, namely Suicide-YF, Crisis and PUMCH, are not publicly available because they contain sensitive clinical speech recordings. These datasets may be made available upon reasonable request to the corresponding authors, subject to the respective data-use agreements and ethical approval requirements.

Code availability

The code is publicly available at https://github.com/JaderOx/AnonymiseAgent under the Apache-2.0 license.

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Grant No. 62501336) and CAMS Innovation Fund for Medical Sciences (Grant No. 2023-I2M-C&T-B-050).

Author contributions

Z.C., W.W., and C.Z. designed the research; Z.C., H.N., H.S., and S.Z. performed the research; Y.L., R.C., L.Z., and Y.T. provided datasets; Z.C., H.N., W.W., and C.Z. analysed the data; J.W., L.C., C.L., Y.Z., C.S., and Y.T. contributed to human evaluation; and Z.C., H.N., W.W., and C.Z. wrote the paper. All authors read and approved the final manuscript.

Competing interests

The authors declare no competing interests.

Supplementary Information

DatasetAnonymiseSNRMOSPCCF0CER/WEREmoEERBACCp-value
NCMMSC-ADOrigin24.38±6.5424.38 \pm6.541.38±0.141.38 \pm0.14----0.765±0.0240.765 \pm0.024-
Pitch22.87±6.5722.87 \pm6.571.26±0.031.26 \pm0.030.783±0.1920.783 \pm0.1920.4040.4040.931±0.0730.931 \pm0.0730.5760.576 (0.5390.539 - 0.6150.615)0.688±0.0240.688 \pm0.0240.0160.016
FormantLPC26.45±6.5926.45 \pm6.591.25±0.031.25 \pm0.030.485±0.4760.485 \pm0.4760.2730.2730.941±0.0620.941 \pm0.0620.1470.147 (0.1240.124 - 0.1750.175)0.730±0.0290.730 \pm0.0290.0580.058
Formantspec23.17±6.4923.17 \pm6.491.24±0.021.24 \pm0.020.832±0.2150.832 \pm0.2150.2210.2210.960±0.0520.960 \pm0.0520.1900.190 (0.1650.165 - 0.2210.221)0.799±0.0140.799 \pm0.0140.6390.639
Neural VC39.19±2.7739.19 \pm2.771.73±0.171.73 \pm0.170.128±0.2680.128 \pm0.2680.3650.3650.942±0.0690.942 \pm0.0690.4670.467 (0.4320.432 - 0.5020.502)0.681±0.0070.681 \pm0.0070.0020.002
Resyn30.07±6.9030.07 \pm6.902.58±0.222.58 \pm0.220.012±0.1790.012 \pm0.1790.0480.0480.912±0.0760.912 \pm0.0760.4790.479 (0.4490.449 - 0.5140.514)0.754±0.0130.754 \pm0.0130.2430.243
ADReSSOrigin46.11±9.5246.11 \pm9.521.58±0.361.58 \pm0.36----0.757±0.0350.757 \pm0.035-
Pitch44.78±9.1744.78 \pm9.171.27±0.031.27 \pm0.030.846±0.1650.846 \pm0.1650.2350.2350.963±0.0470.963 \pm0.0470.3120.312 (0.2600.260 - 0.3680.368)0.815±0.0190.815 \pm0.0190.9900.990
FormantLPC43.48±8.9143.48 \pm8.911.26±0.041.26 \pm0.040.739±0.2940.739 \pm0.2940.3540.3540.938±0.0590.938 \pm0.0590.3610.361 (0.3040.304 - 0.4130.413)0.711±0.0510.711 \pm0.0510.1110.111
Formantspec43.97±9.1543.97 \pm9.151.26±0.041.26 \pm0.040.859±0.2100.859 \pm0.2100.1770.1770.974±0.0360.974 \pm0.0360.1480.148 (0.1070.107 - 0.1940.194)0.774±0.0180.774 \pm0.0180.7970.797
Neural VC41.00±4.3041.00 \pm4.301.88±0.371.88 \pm0.370.297±0.2690.297 \pm0.2690.3530.3530.913±0.0500.913 \pm0.0500.5130.513 (0.4450.445 - 0.5610.561)0.811±0.0200.811 \pm0.0200.9850.985
Resyn65.07±8.5565.07 \pm8.553.83±0.113.83 \pm0.110.045±0.2020.045 \pm0.2020.0250.0250.908±0.0670.908 \pm0.0670.5360.536 (0.4410.441 - 0.6130.613)0.754±0.0270.754 \pm0.0270.5610.561
IvanovaOrigin25.39±6.2025.39 \pm6.201.46±0.281.46 \pm0.28----0.448±0.0560.448 \pm0.056-
Pitch24.83±6.0324.83 \pm6.031.26±0.061.26 \pm0.060.834±0.1430.834 \pm0.1430.2650.2650.964±0.0420.964 \pm0.0420.3670.367 (0.2600.260 - 0.5340.534)0.396±0.0630.396 \pm0.0630.0150.015
FormantLPC22.94±5.7122.94 \pm5.711.25±0.051.25 \pm0.050.625±0.3520.625 \pm0.3520.3360.3360.910±0.0580.910 \pm0.0580.4000.400 (0.2670.267 - 0.5320.532)0.503±0.0410.503 \pm0.0410.6510.651
Formantspec23.71±5.9023.71 \pm5.901.26±0.071.26 \pm0.070.822±0.2410.822 \pm0.2410.1960.1960.968±0.0410.968 \pm0.0410.4170.417 (0.2670.267 - 0.5360.536)0.421±0.0230.421 \pm0.0230.1630.163
Neural VC40.57±2.8940.57 \pm2.891.72±0.191.72 \pm0.190.148±0.2210.148 \pm0.2210.3270.3270.915±0.0660.915 \pm0.0660.4330.433 (0.3180.318 - 0.5840.584)0.399±0.0420.399 \pm0.0420.1360.136
Resyn38.37±3.4838.37 \pm3.483.22±0.243.22 \pm0.240.036±0.1430.036 \pm0.1430.0990.0990.911±0.0520.911 \pm0.0520.4500.450 (0.2970.297 - 0.5530.553)0.396±0.0450.396 \pm0.0451.57×10−31.57 \times10^{-3}
NeuroVozOrigin18.44±11.7018.44 \pm11.702.85±0.672.85 \pm0.67----0.862±0.0770.862 \pm0.077-
Pitch18.30±11.4818.30 \pm11.481.38±0.141.38 \pm0.140.925±0.2190.925 \pm0.2190.2180.2180.950±0.0590.950 \pm0.0590.4350.435 (0.4200.420 - 0.4510.451)0.834±0.0530.834 \pm0.0530.2270.227
FormantLPC15.53±9.3515.53 \pm9.351.67±0.341.67 \pm0.340.832±0.3660.832 \pm0.3660.1840.1840.932±0.0670.932 \pm0.0670.2710.271 (0.2550.255 - 0.2840.284)0.850±0.0750.850 \pm0.0750.5000.500
Formantspec17.22±10.7617.22 \pm10.761.46±0.251.46 \pm0.250.919±0.2100.919 \pm0.2100.1280.1280.969±0.0500.969 \pm0.0500.2260.226 (0.2140.214 - 0.2420.242)0.841±0.0480.841 \pm0.0480.3770.377
Neural VC30.49±13.7830.49 \pm13.782.20±0.622.20 \pm0.620.315±0.5090.315 \pm0.5090.2300.2300.913±0.0700.913 \pm0.0700.4810.481 (0.4660.466 - 0.4970.497)0.843±0.0600.843 \pm0.0600.4120.412
Resyn38.22±13.6838.22 \pm13.683.58±0.593.58 \pm0.590.182±0.3980.182 \pm0.3980.1730.1730.908±0.0780.908 \pm0.0780.4860.486 (0.4700.470 - 0.5030.503)0.815±0.1120.815 \pm0.1121.18×10−71.18 \times10^{-7}
ENNIOrigin28.29±5.3628.29 \pm5.361.54±0.291.54 \pm0.29----0.849±0.0690.849 \pm0.0690.0900.090
Pitch26.92±5.8026.92 \pm5.801.25±0.031.25 \pm0.030.630±0.2310.630 \pm0.2310.2870.2870.939±0.0490.939 \pm0.0490.4670.467 (0.4290.429 - 0.5060.506)0.826±0.0900.826 \pm0.0900.8790.879
FormantLPC32.25±5.0032.25 \pm5.001.25±0.061.25 \pm0.060.485±0.3000.485 \pm0.3000.1720.1720.943±0.0580.943 \pm0.0580.2290.229 (0.1960.196 - 0.2600.260)0.842±0.0600.842 \pm0.0600.7900.790
Formantspec32.74±5.0732.74 \pm5.071.25±0.031.25 \pm0.030.686±0.1740.686 \pm0.1740.1620.1620.942±0.0560.942 \pm0.0560.2960.296 (0.2220.222 - 0.3390.339)0.824±0.0960.824 \pm0.0960.1140.114
Neural VC39.71±5.5139.71 \pm5.511.79±0.231.79 \pm0.230.134±0.2230.134 \pm0.2230.2390.2390.942±0.0650.942 \pm0.0650.4890.489 (0.4490.449 - 0.5230.523)0.810±0.0530.810 \pm0.0530.1140.114
Resyn38.00±5.4938.00 \pm5.494.28±0.214.28 \pm0.210.012±0.1280.012 \pm0.1280.1130.1130.927±0.0520.927 \pm0.0520.5000.500 (0.3030.303 - 0.8830.883)0.754±0.0650.754 \pm0.0650.1840.184

Supplementary Table 2. Results on cognitive disorder datasets.

References

  1. [1]Denes, P. B. & Pinson, E. N. The Speech Chain: The Physics and Biology of Spoken Language (Waveland Press, Long Grove, 2015).
  2. [2]Fagherazzi, G., Fischer, A., Ismael, M. & Despotovic, V. Voice for health: The use of vocal biomarkers from research to clinical practice. Digital Biomarkers 5, 78–88 (2021).DOI
  3. [3]Sara, J. D. S., Orbelo, D., Maor, E., Lerman, L. O. & Lerman, A. Guess what we can hear – novel voice biomarkers for the remote detection of disease. Mayo Clinic Proceedings 98, 1353–1375 (2023).DOI
  4. [4]De Silva, U. et al. Clinical decision support using speech signal analysis: Systematic scoping review of neurological disorders. Journal of Medical Internet Research 27, e63004 (2025).
  5. [5]Cummins, N. et al. A review of depression and suicide risk assessment using speech analysis. Speech Communication 71, 10–49 (2015).DOI
  6. [6]Ringeval, F. et al. AVEC 2019 workshop and challenge: State-of-mind, detecting depression with AI, and cross-cultural affect recognition. Proc. AVEC (Nice, France, 2019).arxiv.org/abs/1907.11510
  7. [7]Flanagan, O., Chan, A., Roop, P. & Sundram, F. Using acoustic speech patterns from smartphones to investigate mood disorders: Scoping review. JMIR mHealth and uHealth 9, e24352 (2021).DOI
  8. [8]Wu, W., Zhang, C. & Woodland, P. C. Self-supervised representations in speech-based depression detection. Proc. ICASSP (Rhodes, 2023).DOI
  9. [9]Luz, S., Haider, F., de la Fuente Garcia, S., Fromm, D. & MacWhinney, B. Alzheimer’s dementia recognition through spontaneous speech. Frontiers in Computer Science 3, 780169 (2021).arxiv.org/abs/2004.06833
  10. [10]Ivanova, O. et al. Discriminating speech traits of Alzheimer’s disease assessed through a corpus of reading task for Spanish language. Computer Speech & Language 73, 101341 (2022).DOI
  11. [11]Cui, Z., Wu, W., Zhang, W.-Q., Wu, J. & Zhang, C. Transferring speech-generic and depression-specific knowledge for alzheimer’s disease detection. Proc. ASRU (IEEE, Taipei, 2023).DOI
  12. [12]Pützer, M. & Barry, W. J. Saarbruecken voice database. https://doi.org/10.5281/zenodo.16874898 (2008).
  13. [13]Hasegawa-Johnson, M. et al. Community-supported shared infrastructure in support of speech accessibility. Journal of Speech, Language, and Hearing Research 67, 4162–4175 (2024).DOI
  14. [14]Jia, K., Tu, M., Jin, Z., Wang, S. & Zhang, C. Augmenting Dysarthric Speech Severity Assessment with MOS Supervision. Proc. Interspeech (Sydney, 2026).arxiv.org/abs/2606.18645
  15. [15]Diaz-Asper, C., Bongo, L. A. & Elvevåg, B. Navigating the tradeoff between personal privacy and data utility in speech anonymization for clinical research. npj Digital Medicine 8, 616 (2025).DOI
  16. [16]Malo, M.-F. et al. Using voice and speech data in healthcare: A scoping review of the ethical, legal and social implications. Frontiers in Digital Health 8, 1750111 (2026).DOI
  17. [17]Tayebi Arasteh, S. et al. The effect of speech pathology on automatic speaker verification: a large-scale study. Scientific Reports 13, 20476 (2023).DOI
  18. [18]Aggazzotti, C., Garg, A., Cai, Z. & Andrews, N. Content anonymization for privacy in long-form audio. Proc. ICASSP (Barcelona, 2026).DOI
  19. [19]Tomaschenko, N. et al. The VoicePrivacy 2020 Challenge: Results and findings. Computer Speech & Language 74, 101362 (2022).DOI
  20. [20]Ravuri, V., Gutierrez-Osuna, R. & Chaspari, T. Preserving mental health information in speech anonymization. Proc. ACIW (Nara, 2022).DOI
  21. [21]Tayebi Arasteh, S. et al. Addressing challenges in speaker anonymization to maintain utility while ensuring privacy of pathological speech. Communications Medicine 4, 182 (2024).DOI
  22. [22]Cui, Z. et al. Speaker anonymisation for speech-based suicide risk detection. Proc. ICASSP (Barcelona, 2026).DOI
  23. [23]Cai, H. et al. A multi-modal open dataset for mental-disorder analysis. Scientific data 9, 178 (2022).DOI
  24. [24]Tao, F., Esposito, A. & Vinciarelli, A. The Androids Corpus: A new publicly available benchmark for speech based depression detection. Proc. Interspeech (Dublin, 2023).DOI
  25. [25]Chen, X., Zhang, W.-Q. & Ma, Y. Raw waveform-based end-to-end Alzheimer’s disease detection method. Acta Electronica Sinica 51, 3582–3590 (2023).
  26. [26]Mendes-Laureano, J. et al. NeuroVoz: A Castillian Spanish corpus of Parkinsonian speech. Scientific Data 11, 1367 (2024).
  27. [27]Schneider, P., Hayward, D., Dubé, R. V. et al. Storytelling from pictures using the Edmonton narrative norms instrument. Journal of speech language pathology and audiology 30, 224 (2006).
  28. [28]Hui, B. S. H., Miao, X. & Wang, X. SecureSpeech: Prompt-based speaker and content protection. Proc. IJCB (Osaka, 2025).arxiv.org/abs/2507.07799
  29. [29]Tomaschenko, N. et al. The third VoicePrivacy Challenge: Preserving emotional expressiveness and linguistic content in voice anonymization. Computer Speech & Language 100, 101988 (2026).
  30. [30]Yang, X. et al. SALMONN-2: Advancing general-purpose hearing abilities with self-supervised representations. arXiv preprint arXiv:2607.17079 (2026).arxiv.org/abs/2607.17079
  31. [31]Yang, A. et al. Qwen2.5 technical report. arXiv preprint arXiv:2412.15115 (2024).arxiv.org/abs/2412.15115
  32. [32]OpenAI. GPT-5 nano model. https://developers.openai.com/api/docs/models/gpt-5-nano. Accessed: 2026-04.DOI
  33. [33]Qwen3.8-27B. Available at https://huggingface.co/Qwen/Qwen3.8-27B. Accessed: 2026-08.DOI
  34. [34]Shinn, N., Cassano, F., Gopinath, A., Narasimhan, K. & Yao, S. Reflection: Language agents with verbal reinforcement learning. Proc. NeurIPS (New Orleans, 2023).arxiv.org/abs/2303.11366
  35. [35]Burkhardt, F., Wagner, J., Wierstorf, H., Eyben, F. & Schuller, B. Speech-based age and gender prediction with Transformers. 15th ITG Conference (Aachen, 2023).arxiv.org/abs/2306.16962
  36. [36]Wagner, J., Wierstorf, H., Burkhardt, F., Eyben, F. & Schuller, B. Model for age and gender prediction based on wav2vec 2.0. https://doi.org/10.5281/zenodo.7761387 (2023).
  37. [37]Gao, Z., Zhang, S., McLoughlin, I. & Yan, Z. Paraformer: Fast and accurate parallel transformer for non-autoregressive end-to-end speech recognition. Proc. Interspeech (Incheon, 2022).arxiv.org/abs/2206.08317
  38. [38]Bain, M., Huh, J., Han, T. & Zisserman, A. Whisperx: Time-accurate speech transcription of long-form audio. Proc. Interspeech (Dublin, 2023).arxiv.org/abs/2303.00747
  39. [39]Patino, J., Tomashenko, N., Todisco, M., Nautsch, A. & Evans, N. Speaker anonymisation using the McAdams coefficient. Proc. Interspeech (Brno, 2021).arxiv.org/abs/2011.01130
  40. [40]Liu, S. Zero-shot voice conversion with diffusion Transformers. arXiv preprint arXiv:2411.09943 (2024).arxiv.org/abs/2411.09943
  41. [41]Bu, H., Du, J., Na, X., Wu, B. & Zheng, H. AISHELL-1: An open-source Mandarin speech corpus and a speech recognition baseline. Proc. O-COCOSDA (Seoul, 2017).arxiv.org/abs/1709.05522
  42. [42]Liao, S. et al. Fish-speech: Leveraging large language models for advanced multilingual text-to-speech synthesis. arXiv preprint arXiv:2411.01156 (2024).arxiv.org/abs/2411.01156
  43. [43]Ardila, R. et al. Common Voice: A massively-multilingual speech corpus. Proc. LREC (Marseille, 2020).arxiv.org/abs/1912.06670
  44. [44]Pratap, V., Xu, Q., Sriram, A., Synnaeve, G. & Collobert, R. MLS: A Large-Scale Multilingual Dataset for Speech Research. Proc. Interspeech (Conference held virtually, 2020).DOI
  45. [45]Saeki, T. et al. UTMOS: UTokyo-SaruLab system for VoiceMOS Challenge 2022. Proc. Interspeech (Incheon, 2022).arxiv.org/abs/2204.02152
  46. [46]Ma, Z. et al. Emotion2Vec: Self-supervised pre-training for speech emotion representation. Proc. ACL (Bangkok, 2024).arxiv.org/abs/2312.15185
  47. [47]WavLM-Base-Plus for speaker verification. Available at https://huggingface.co/microsoft/wavlm-base-plus-sv. Accessed: 2025-05.
  48. [48]Xu, J. et al. Qwen2.5-Omni technical report. arXiv preprint arXiv:2503.20215 (2025).arxiv.org/abs/2503.20215
  49. [49]Liu, S.-Y. et al. DoRA: Weight-decomposed low-rank adaptation. Proc. ICML (Vienna, 2024).arxiv.org/abs/2402.09353
  50. [50]Sheehan, D. V. et al. Reliability and validity of the mini international neuropsychiatric interview for children and adolescents (MINI-KID). The Journal of clinical psychiatry 71, 313–326 (2010).DOI
  51. [51]Cui, Z. et al. Spontaneous speech-based suicide risk detection using Whisper and large language models. Proc. Interspeech (Kos Island, 2024).DOI
  52. [52]Mendes-Laureano, J. et al. Neurovoz: a castillian spanish corpus of parkinsonian speech (1.0.0). https://doi.org/10.5281/zenodo.10777657 (2024).
  53. [53]Chinese RoBERTa-Base model for NER. Available at https://huggingface.co/uer/roberta-base-finetuned-cluener2020-chinese. Accessed: 2026-06.
  54. [54]bert-base-multilingual-cased-ner-hrl. Available at https://huggingface.co/Davlan/bert-base-multilingual-cased-ner-hrl. Accessed: 2026-06.
  55. [55]Desplanques, B., Thienpondt, J. & Demuynck, K. ECAPA-TDNN: Emphasized channel attention, propagation and aggregation in TDNN based speaker verification. Proc. Interspeech (Shanghai, 2020).DOI
  56. [56]Panayotov, V., Chen, G., Povey, D. & Khudanpur, S. Librispeech: An ASR corpus based on public domain audio books. Proc. ICASSP (South Brisbane, 2015).DOI

Paper details

vlnc:W_fvf5ppbq
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Engineering

Not recorded for this paper

View statement
Looking for related papers…
Contents