WorksComputer Science
Ruby-OPD: Enhancing Pronunciation Controllability and Accuracy for Flow-Matching Text-to-Speech via On-Policy Distillation
Abstract
Achieving both high pronunciation accuracy and fine-grained controllability remains a central challenge in Japanese text-to-speech (TTS), as Japanese is an unsegmented language mixing logographic Kanji and phonographic Kana with extensive polyphony. Traditional cascaded systems convert text into phoneme sequences via a front-end, ensuring pronunciation control but discarding lexical semantics and prosodic cues in surface orthography. Conversely, recent zero-shot TTS models directly consume raw orthographic text to improve naturalness, yet suffer from polyphone errors and lack explicit pronunciation control. To bridge this gap, we propose a two-stage framework for flow-matching TTS. First, we investigate Ruby text representations with orthography perturbation to jointly preserve surface forms and target pronunciations for reliable control. Second, we introduce On-Policy Distillation (OPD) on large-scale text-only corpora to enhance pronunciation accuracy on raw orthographic text without paired speech. On the Joyo-Kanji-Yomi benchmark, our model achieves 97.13% Accuracy, 2.02% Target PER, 0.70% Sentence PER, and 4.21% Sentence CER under raw text while retaining strong controllability.
Introduction
Traditional neural text-to-speech (TTS) systems [1, 2, 3] follow a cascaded pipeline comprising a text front-end, an acoustic model, and a vocoder, where the front-end extracts phonemes and prosodic features from raw text. Recently, scaling model capacity and data has driven the rise of zero-shot TTS across both autoregressive (AR) [4, 5, 6, 7, 8] and non-autoregressive (NAR) [9, 10, 11, 12] paradigms. These models bypass complex front-ends and directly consume raw orthographic text, leveraging surface lexical semantics to synthesize natural speech.
In Japanese TTS, however, directly consuming orthographic text poses severe challenges for pronunciation accuracy and controllability. As an unsegmented language, Japanese combines logograms (Kanji) and phonograms (Kana). While Kana maps almost deterministically to pronunciation, Kanji pronunciations vary drastically with morphology and context. For instance, 秋 (autumn) takes the native pronunciation (Kun yomi) aki standalone, the Sino-Japanese pronunciation (On yomi) shū in compounds such as 晩秋 (late autumn), and the metaphorical pronunciation (Gikun) toki (時, moment) in classical expressions such as 此れぞ誠に危急存亡の秋なり (this is truly a critical moment of survival).
Early Japanese TTS relied on morphological analyzers with dictionaries [13, 14, 15] and pretrained language models for polyphone disambiguation and accent prediction [16, 17, 18, 19]. Although phonemic inputs offer direct pronunciation control, stripping away Kanji orthography discards semantic and prosodic cues, limiting naturalness [20]. Conversely, recent end-to-end Japanese TTS models train directly on orthographic text. Irodori-TTS [21] models continuous acoustic latents [22] via conditional flow matching with a Diffusion Transformer (DiT) [23], while Sarashina2.2-TTS [24] uses an AR architecture and mitigates Kanji mispronunciations via local Kana substitution and offline speech synthesis over polyphones. Yet, this still sacrifices orthographic semantics and incurs costly off-policy synthesis.
In this paper, we propose a two-stage framework to enhance pronunciation controllability and accuracy in flow-matching Japanese TTS. First, we introduce Ruby text representations coupled with orthography perturbation to jointly preserve logographic surface forms and target pronunciations for precise pronunciation control. Second, we propose cross-condition On-Policy Distillation (OPD) on large-scale text-only corpora to improve raw-text pronunciation accuracy without paired speech. Experiments on the Joyo-Kanji-Yomi benchmark show that our method substantially improves both pronunciation controllability and accuracy, achieving 97.13% Accuracy, 2.02% Target PER, 0.70% Sentence PER, and 4.21% Sentence CER.
Method
Ruby text representation
Ruby is widely used in Japanese books, webpages, and subtitles to annotate the pronunciations of proper nouns, polyphones, rare words, or stylistically specialized terms. Visually, Ruby attaches small Kana characters above or beside target Kanji, simultaneously conveying both orthographic surface form and phonemic pronunciation within a single visual unit.
When linearizing two-dimensional Ruby typography into a sequential text input for a TTS text encoder, we explore two complementary representations:
Implicit Ruby: Target pronunciations are appended directly after annotated Kanji words using parentheses. This aligns naturally with human users’ everyday writing and interactive editing habits when manually annotating occasional rare words or proper nouns on the fly;
Explicit Ruby: One limitation of implicit Ruby is the lack of explicit left boundaries for the target span, which can sometimes result in ambiguous annotations in unsegmented text. Explicit Ruby, however, employs explicit boundary markers for both the target span and its pronunciation, making it well suited for unambiguous and precise pronunciation annotation.
An example is illustrated in Table 1. The compound word 超電磁砲 is canonically pronounced as ちょうでんじほう (chōdenjihō). However, in the title とある科学の超電磁砲 (A Certain Scientific Railgun), it takes the stylized pronunciation レールガン (rērugan). Under implicit Ruby, because this nonstandard pronunciation has no morphological correspondence to individual Kanji characters, the left boundary of the annotated span becomes ambiguous, where it could be interpreted as annotating either 電磁砲 or the entire 超電磁砲. In contrast, explicit Ruby eliminates such scope ambiguity by introducing explicit boundary markers.
| Orthographic text |
| とある科学の超電磁砲 |
| Implicit Ruby |
| とある科学(かがく)の超電磁砲(レールガン) |
| Explicit Ruby |
| とある|科学《かがく》の|超電磁砲《レールガン》 |
Table 1. An example of orthographic text, implicit Ruby, and explicit Ruby. |, 《, and 》 denote the start of the target word, and the start and end of the target pronunciation in explicit Ruby, respectively.
Training data construction
Constructing paired Ruby training data for supervised finetuning (SFT) solely via Grapheme-to-Phoneme (G2P) tools is prone to polyphone disambiguation errors that may mismatch the actual pronunciations uttered in the paired speech. To construct precisely aligned Ruby training data, we employ a crossmodal verification approach that leverages information from both text and audio modalities. Specifically, we first use a phoneme recognition model to transcribe the input speech into a pure Kana sequence, capturing the actual pronunciation of the utterance. Then, for the corresponding orthographic text, we use UniDic [14], a widely used morphological dictionary, to construct a word lattice encompassing all valid morphological segmentations and candidate pronunciations. Finally, we apply dynamic programming to retrieve the alignment path through the lattice that exactly matches the recognized Kana transcription, precisely pinpointing the character span and true contextual pronunciation of every Kanji word unit in the utterance.
In practical applications, users exhibit diverse annotation habits across different granularities, ranging from short single words to longer multi-word phrases, and may feed either completely unannotated orthographic text or partially annotated Ruby text where only selected words are annotated. To simulate these scenarios, rather than enumerating fixed full-annotation paths after the alignment step, we store the aligned Ruby units and adopt a dynamic multi-granularity sampling strategy during training. First, at the utterance level, we sample from a preset probability distribution to determine whether an utterance is formatted as unannotated, partially annotated, or fully annotated. Specifically, under the partially annotated mode, each Ruby unit is converted into Ruby format with a certain probability and otherwise reverts to raw orthographic text. Furthermore, adjacent Ruby units that are simultaneously selected for annotation are merged with a certain probability into a single contiguous Ruby unit. These data augmentation strategies enable the model to seamlessly handle unannotated raw text, sparse local annotations, and annotations of varying granularities.
Orthography perturbation
In our preliminary experiments, we found that directly training the model with Ruby text failed to make it properly interpret the Ruby format or follow the pronunciation annotations at inference time. Instead, the model often read aloud both the surface form and the annotated pronunciation within Ruby units. We conjecture that because the base model was pretrained on raw orthographic text, it can already infer canonical pronunciations for most Kanji solely from their surface glyphs, leaving little incentive to attend to the pronunciation cues provided in the Ruby annotations.
To address this issue, we keep the target pronunciation annotation intact while stochastically perturbing the surface form of each Ruby unit during training via three complementary operations:
Kanji masking: replaces partial or full Kanji spans in the surface form with special mask tokens. Partial masking encourages the model to align unmasked characters with the target pronunciation to infer span boundaries in implicit Ruby, whereas full masking forces the model to rely on the pronunciation annotation when surface glyph cues are absent;
Random substitution: replaces the surface form in explicit Ruby with random characters or unrelated words of arbitrary length, breaking character-to-mora correspondence and compelling the model to follow the annotated pronunciation regardless of surface glyphs;
Semantic substitution: replaces the surface form in explicit Ruby with semantically similar Kanji words that have distinctly different canonical pronunciations, retrieved via text embeddings over UniDic. This design specifically simulates the aforementioned Gikun phenomenon, where the surface form is semantically aligned with both the annotated pronunciation and surrounding context yet phonemically unrelated to its canonical pronunciation. For example, the Kana word ワールド (world), pronounced as wārudo, may be assigned a semantically similar yet orthographically and phonemically distinct surface form 世界, canonically pronounced as sekai. This strategy ensures that the model strictly adheres to the annotated pronunciation while leveraging surface orthography for semantic and prosodic cues.
On-policy distillation
After SFT with Ruby data, the model achieves high pronunciation accuracy when provided with full Ruby annotations. However, as noted earlier, real-world inputs are often unannotated orthographic text or only partially annotated, leaving unannotated spans vulnerable to polyphone disambiguation errors. Attaching an external G2P module at inference time or synthesizing paired text-speech data over polyphones in an off-policy manner [24] are viable workarounds, but they introduce extra pipeline modules or costly offline synthesis steps. Instead, we propose an On-Policy Distillation (OPD) approach for flow-matching TTS that relies solely on paired (orthographic text, Ruby text) data without requiring parallel audio, improving pronunciation accuracy when Ruby annotations are absent.
In conditional flow matching [25], a neural network learns a time-dependent velocity field that transports standard Gaussian noise at to clean acoustic latents at conditioned on text . With paired speech , noisy states are constructed via linear interpolation , and the model is trained by minimizing:
In our OPD framework, the SFT model serves as a frozen teacher conditioned on full Ruby text , while the student , initialized from , is conditioned on unannotated orthographic text (other conditioning is omitted for brevity). Because text-only samples lack ground-truth speech latents , we generate the intermediate state online via student rollout [26, 27]. Starting from pure Gaussian noise with target sequence length predicted by the teacher’s duration predictor, we integrate the student’s ordinary differential equation (ODE) trajectory up to a randomly sampled timestep without gradients to obtain the on-policy state . At , the student is trained to match the teacher’s velocity field:
where denotes the stop-gradient operator, alongside a Huber loss aligning the student’s duration predictor with the teacher’s. Since the rollout starts from pure noise, contains zero phonemic leakage from ground-truth audio, forcing the student to disambiguate polyphones from context to match the teacher’s trajectory. During training, we also interleave the paired speech data used in the SFT stage to anchor acoustic naturalness and retain Ruby controllability.
Experimental setup
Data
For the SFT stage, we use the Japanese subset of the open-source multilingual speech dataset Emilia [28]. To ensure strict consistency between pronunciation annotations and actual speech content, we transcribe all utterances using kana-whisper1 and filter for utterances whose Kana transcriptions exactly match the word lattice constructed via UniDic [14]. Combined with duration filtering (0.5 30 s), we obtain approximately 774 hours of high-quality aligned data.
For subsequent text-only OPD, we follow [19] by using entries from UniDic [14] as seed words and leveraging Gemini 3.1 Pro to synthesize text-pronunciation pairs. For each monophonic and polyphonic entry, we synthesize 5 and 10 contextual sentences, respectively. Specifically, rather than asking the LLM to directly output Ruby annotations, we prompt it to generate sentence-level Kana pronunciation sequences and apply the same lattice alignment method used for Emilia. This yields approximately 680k monophonic sentences and 100k polyphonic sentences.
Implementation details
We build upon the open-source Japanese flow-matching zero-shot TTS model Irodori-TTS-v4.1-Small [21]2 as our base model. It initializes its text encoder with a pretrained bidirectional masked language model [29] and predicts the conditional velocity field of 25 Hz, 32-dimensional continuous DACVAE latents [22] via a DiT backbone.
In the SFT stage, utterance-level annotation modes are sampled with probabilities of 50% full Ruby, 30% partial Ruby, in which each Ruby unit is annotated with a probability of 0.5, and 20% unannotated raw text. Each annotated Ruby unit is formatted as implicit or explicit Ruby with an equal probability of 0.5, and adjacent annotated Ruby units are merged with a probability of 0.5, up to 3 merges and 8 characters. For orthography perturbation, we use Qwen3-Embedding-0.6B3 to retrieve semantic substitution candidates over UniDic [14], and each annotated surface span undergoes full-span Kanji masking with a probability of 0.2, partial character-level masking with a probability of 0.25, semantic substitution with a probability of 0.2, and random substitution with a probability of 0.15. The model is trained for 50k steps with a global batch size of 256 using AdamW, with a learning rate of for the main model and for the pretrained text encoder.
In the OPD stage, we freeze the SFT model as the teacher, which receives 100% full Ruby text, and train the student for 50k steps with a global batch size of 256, a learning rate of , a text encoder learning rate of , and 8 rollout steps. Each training step samples text-only data with an 80% probability, equally weighted between monophonic and polyphonic subsets with 100% raw text input for the student, and paired speech data with a 20% probability, where student inputs are sampled using 80% raw text, 10% partial Ruby, and 10% full Ruby with the same orthography perturbation settings as in the SFT stage.
Evaluation metrics
To evaluate pronunciation accuracy and controllability, we conduct evaluations on a revised version of the Joyo-Kanji-Yomi benchmark (JKYB) [24]4. It comprises 13,536 sentences covering 2,136 Joyo Kanji (regular-use Kanji) and their 4,512 official pronunciations specified by the Agency for Cultural Affairs of Japan. JKYB provides the following metrics:
Accuracy: Percentage of evaluated Kanji pronunciations that exactly match the ground truth;
Target PER: Kana character error rate (Kana-CER) against the ground truth of the evaluated Kanji. Values are capped to 100%;
Sentence PER: Kana-CER for the full-sentence pronunciation. Values are capped to 100%;
Sentence CER: Standard CER between the ASR transcription of the utterance and the source text. Values are capped to 100%.
We also evaluate speech quality using UTMOSv2 [30] and DNSMOS [31], and voice cloning speaker similarity using CAM++ [32] and ERes2Net [33]. During inference, reference audio prompts are randomly sampled from the JVS corpus [34], which consists of 100 speakers with 100 normal-style utterances per speaker.
Experimental results
Pronunciation controllability
We first investigate the effectiveness of Ruby representations for pronunciation control. Using the reference text and ground-truth pronunciations from JKYB, we construct oracle Ruby inputs at both the target-word and full-sentence levels.
As shown in Table 3, for Irodori-Base, directly replacing the target Kanji word with Kana (Target Kana) effectively improves target pronunciation accuracy, whereas replacing the entire sentence with Kana (Full Kana) causes a notable performance drop at both the target-word and sentence levels, yielding an even higher Sentence PER than unannotated raw text. This confirms that stripping away Kanji orthography across an unsegmented sentence eliminates crucial lexical boundary and semantic cues. In contrast, by jointly preserving surface Kanji and phonemic annotations, both Irodori-Ruby-SFT and Irodori-Ruby-OPD avoid this degradation: expanding annotations from the target word (Target) to the full sentence (Full) maintains high target accuracy while further reducing Sentence PER. Moreover, explicit Ruby (Expl.) consistently outperforms implicit Ruby (Impl.) across both SFT and OPD models under both target-word and full-sentence settings. While implicit Ruby already provides strong controllability for convenient user interaction, the explicit boundary markers in explicit Ruby effectively eliminate left-boundary scope ambiguity, particularly for multi-character words with non-compositional pronunciations.
| Model | Input | Accuracy | Target PER | Sentence PER |
| Base | Raw text | 94.11% | 4.68% | 0.97% |
| Base | Target Kana | 97.89% | 1.37% | 0.56% |
| Base | Full Kana | 95.40% | 2.89% | 1.06% |
| SFT | Target Impl. | 97.56% | 1.71% | 0.75% |
| SFT | Target Expl. | 98.12% | 1.26% | 0.72% |
| SFT | Full Impl. | 98.05% | 1.33% | 0.53% |
| SFT | Full Expl. | 98.20% | 1.19% | 0.49% |
| OPD | Target Impl. | 98.09% | 1.30% | 0.57% |
| OPD | Target Expl. | 98.33% | 1.11% | 0.56% |
| OPD | Full Impl. | 98.14% | 1.23% | 0.45% |
| OPD | Full Expl. | 98.24% | 1.15% | 0.42% |
Table 3. Results of pronunciation controllability for Irodori-Base (Base), Irodori-Ruby-SFT (SFT), and Irodori-Ruby-OPD (OPD). Target and Full indicate target-word or full-sentence Kana replacement or Ruby annotation. Impl. and Expl. denote implicit and explicit Ruby.
Main results
Table 2 compares our models (Irodori-Ruby-SFT and Irodori-Ruby-OPD) against the base model (Irodori-Base) and public state-of-the-art (SOTA) baselines across pronunciation accuracy, speech quality, and voice cloning. At inference time, all models directly consume raw orthographic text without any Ruby annotations. First, while general multilingual TTS models, such as Qwen3-TTS, FireRedTTS3, OmniVoice, and FishAudio S2 Pro, achieve leading performance in speech quality and voice cloning, they still struggle with Japanese polyphonic Kanji pronunciation, exhibiting relatively low Accuracy (72.10%–89.90%) and high Target PER (8.27%–23.68%). In contrast, Japanese-specialized models such as Sarashina2.2-TTS and Irodori-Base benefit from large-scale Japanese pre-training and demonstrate a clear advantage in pronunciation accuracy, with Irodori-Base achieving 94.11% Accuracy and the lowest Sentence CER (4.04%).
| Model | Accuracy | Target PER | Sentence PER | Sentence CER | UTMOSv2 | DNSMOS | CAM++ | ERes2Net |
| Qwen3-TTS 1.7B Base [6] | 72.10% | 23.68% | 4.38% | 6.52% | 3.50 | 3.23 | 0.757 | 0.779 |
| FireRedTTS3 [8] | 73.29% | 22.35% | 4.04% | 6.12% | 3.36 | 3.26 | 0.833 | 0.832 |
| OmniVoice [12] | 83.66% | 13.24% | 2.91% | 5.18% | 3.19 | 3.21 | 0.797 | 0.786 |
| FishAudio S2 Pro [7] | 89.90% | 8.27% | 1.52% | 4.77% | 3.30 | 3.31 | 0.735 | 0.726 |
| Sarashina2.2-TTS [24] | 90.42% | 7.69% | 3.15% | 6.26% | 3.42 | 3.33 | 0.819 | 0.825 |
| Irodori-Base [21] | 94.11% | 4.68% | 0.97% | 4.04% | 3.21 | 3.23 | 0.728 | 0.697 |
| Irodori-Ruby-SFT | 92.32% | 6.05% | 1.32% | 4.40% | 3.18 | 3.22 | 0.703 | 0.663 |
| Irodori-Ruby-OPD | 97.13% | 2.02% | 0.70% | 4.21% | 3.19 | 3.23 | 0.703 | 0.663 |
Table 2. Comparison of pronunciation accuracy, speech quality, and voice cloning performance on the Joyo-Kanji-Yomi benchmark (JKYB) using JVS reference audio prompts. All models directly consume unannotated raw orthographic text at inference time.
Compared with Irodori-Base, Irodori-Ruby-SFT, fine-tuned solely on Emilia Ruby data, exhibits a slight drop on raw text (92.32% Accuracy and 6.05% Target PER) and minor shifts in acoustic and speaker metrics, reflecting the limited scale and modest recording quality of the SFT corpus as well as capacity allocated to learning Ruby conditioning. After large-scale text-only OPD, however, Irodori-Ruby-OPD achieves substantial gains on raw text without additional paired speech, boosting Accuracy to 97.13% (+4.81% over its teacher and +3.02% over Irodori-Base), cutting Target PER by more than half vs. Irodori-Base (4.68% → 2.02%), and achieving the lowest Sentence PER (0.70%). Furthermore, Irodori-Ruby-OPD consistently surpasses Irodori-Ruby-SFT across all four Ruby control modes (Table 3) while maintaining speech quality and voice cloning on par with its teacher. In particular, under target-only annotation (Target Impl. and Target Expl.), Irodori-Ruby-OPD notably reduces Sentence PER compared with Irodori-Ruby-SFT (0.56%–0.57% vs. 0.72%–0.75%), as the remaining unannotated Kanji words in the utterance simultaneously benefit from the enhanced raw-text pronunciation capability distilled during OPD.
Conclusion
In this paper, we proposed a two-stage framework for flow-matching Japanese TTS that first introduces implicit and explicit Ruby representations coupled with orthography perturbation during SFT to enable fine-grained pronunciation control while preserving surface Kanji semantics. Building upon the Ruby-capable SFT teacher, we further introduced cross-condition On-Policy Distillation (OPD) on large-scale LLM-synthesized text-pronunciation pairs, transferring context-dependent Kanji pronunciation knowledge into raw-text inference without requiring paired speech. Experimental results on the Joyo-Kanji-Yomi benchmark demonstrate that Irodori-Ruby-OPD achieves state-of-the-art pronunciation accuracy on unannotated orthographic text while providing flexible and reliable Ruby-based pronunciation control.
References
References
- [1]Y. Wang, R. Skerry-Ryan, D. Stanton, Y. Wu, R. J. Weiss, J. Jaitly, Z. Yang, Y. Xiao, Z. Chen, S. Bengio, Q. Le, Y. Agiomyrgiannakis, R. Clark, and R. A. Saurous, “Tacotron: Towards End-to-End Speech Synthesis,” in Proc. Interspeech, 2017, pp. 4006–4010.arxiv.org/abs/1703.10135
- [2]Y. Ren, C. Hu, X. Tan, T. Qin, S. Zhao, Z. Zhao, and T.-Y. Liu, “FastSpeech 2: Fast and high-quality end-to-end text to speech,” in International Conference on Learning Representations (ICLR), 2021.arxiv.org/abs/2006.04558
- [3]J. Kim, J. Kong, and J. Son, “Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech,” in Proceedings of the 38th International Conference on Machine Learning (ICML), 2021, pp. 5530–5540.arxiv.org/abs/2106.06103
- [4]Z. Du, Q. Chen, S. Zhang, K. Hu, H. Lu, Y. Yang, H. Hu, S. Zheng, Y. Gu, Z. Ma et al., “Cosyvoice: A scalable multilingual zero-shot text-to-speech synthesizer based on supervised semantic tokens,” arXiv preprint arXiv:2407.05407, 2024.arxiv.org/abs/2407.05407
- [5]Y. Zhou, G. Zeng, X. Liu, X. Li, R. Yu, Z. Wang, R. Ye, W. Sun, J. Gui, K. Li et al., “Voxcpml: Tokenizer-free tts for context-aware speech generation and true-to-life voice cloning,” arXiv preprint arXiv:2509.24650, 2025.arxiv.org/abs/2509.24650
- [6]H. Hu, X. Zhu, T. He, D. Guo, B. Zhang, X. Wang, Z. Guo, Z. Jiang, H. Hao, Z. Guo et al., “Qwen3-tts technical report,” arXiv preprint arXiv:2601.15621, 2026.arxiv.org/abs/2601.15621
- [7]S. Liao, Y. Wang, S. Liu, Y. Cheng, R. Zhang, T. Li, S. Li, Y. Zheng, X. Liu, Q. Wang et al., “Fish audio s2 technical report,” arXiv preprint arXiv:2603.08823, 2026.arxiv.org/abs/2603.08823
- [8]F. Shen, K. Xie, Y. Wu, Z. Dai, Y. Han, J. Li, X. Geng, F. Xie, L. Xie, X. Tang et al., “Firedtts3: Unified speech generation and editing with semantically enriched speech representations,” arXiv preprint arXiv:2608.17492, 2026.arxiv.org/abs/2608.17492
- [9]M. Le, A. Vyas, B. Shi, B. Karrer, L. Sari, R. Moritz, M. Williamson, V. Manohar, Y. Adi, J. Mahadeokar, and W.-N. Hsu, “Voicebox: Text-guided multilingual universal speech generation at scale,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 36, 2023.arxiv.org/abs/2306.15687
- [10]S. E. Eskimez, X. Wang, M. Thakker, C. Li, C.-H. Tsai, Z. Xiao, H. Yang, Z. Zhu, M. Tang, X. Tan et al., “Embarrassingly easy fully non-autoregressive zero-shot TTS,” in IEEE Spoken Language Technology Workshop (SLT), 2024, pp. 682–689.
- [11]Y. Chen, Z. Niu, Z. Ma, K. Deng, C. Wang, J. Zhao, K. Yu, and X. Chen, “F5-tts: A fairytaler that fakes fluent and faithful speech with flow matching,” in Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL), 2025, pp. 6255–6271.arxiv.org/abs/2410.06885
- [12]H. Zhu, L. Ye, W. Kang, Z. Yao, L. Guo, F. Kuang, Z. Han, W. Zhuang, L. Lin, and D. Povey, “Omnivoice: Towards omnilingual zero-shot text-to-speech with diffusion language models,” arXiv preprint arXiv:2604.00688, 2026.arxiv.org/abs/2604.00688
- [13]T. Kudo, “Mecab: Yet another part-of-speech and morphological analyzer,” http://mecab.sourceforge.net/, 2005.
- [14]Y. Den, T. Ogiso, H. Ogura, A. Yamada, N. Minematsu, K. Uchimoto, and H. Koiso, “The development of an electronic dictionary for morphological analysis and its application to japanese corpus linguistics,” in Japanese Linguistics, vol. 22, 2007, pp. 101–123.DOI
- [15]H. Morita, D. Kawahara, and S. Kurohashi, “Morphological analysis for unsegmented languages using recurrent neural network language model,” in Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, 2015, pp. 2292–2297.DOI
- [16]R. Hida, M. Hamada, C. Kamada, E. Tsunoo, T. Sekiya, and T. Kumakura, “Polyphone disambiguation and accent prediction using pre-trained language models in japanese TTS front-end,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2022, pp. 7132–7136.
- [17]B. Park, R. Yamamoto, and K. Tachibana, “A Unified Accent Estimation Method Based on Multi-Task Learning for Japanese Text-to-Speech,” in Proc. Interspeech, 2022, pp. 1931–1935.DOI
- [18]K. Kurihara and M. Sano, “Enhancing japanese text-to-speech accuracy with a novel combination transformer-BERT-based G2P: Integrating pronunciation dictionaries and accent sandhi,” in Proc. Interspeech, 2024.
- [19]R. Hu, Z. Zhan, and X. Lin, “Dictionary-constrained grapheme-to-phoneme for unsegmented languages from LLM-annotated data,” arXiv preprint arXiv:2609.19805, 2026.arxiv.org/abs/2609.19805
- [20]Y. Yasuda, X. Wang, S. Takaki, and J. Yamagishi, “Investigation of learning abilities on linguistic features in sequence-to-sequence text-to-speech synthesis,” Computer Speech & Language, vol. 58, p. 101008, 2019.arxiv.org/abs/2005.10390
- [21]C. Arata, “Irodori-TTS: A flow matching-based text-to-speech model with emoji-driven style control,” https://github.com/Aratako/Irodori-TTS, 2026.
- [22]R. Kumar, P. Seetharaman, A. Luebs, I. Kumar, and K. Kumar, “High-fidelity audio compression with improved RVQGAN,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 36, 2023.arxiv.org/abs/2306.06546
- [23]W. Peebles and S. Xie, “Scalable diffusion models with transformers,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023, pp. 4195–4205.
- [24]L. Liu, S. Zhu, K. Washizaki, R. Yoneyama, H. Jeon, M. Zhao, Y. Fujita, H. Shi, N. Yoshida, Y. Gao et al., “Sarashina2.2-TTS: Tackling kanji polyphony in japanese speech generation via data scaling and targeted data synthesis,” arXiv preprint arXiv:2606.25369, 2026.arxiv.org/abs/2606.25369
- [25]Y. Lipman, R. T. Q. Chen, H. Ben-Hamu, M. Nickel, and M. Le, “Flow matching for generative modeling,” in International Conference on Learning Representations (ICLR), 2023.arxiv.org/abs/2210.02747
- [26]R. Agarwal, N. Vieillard, Y. Zhou, P. Stanczyk, S. R. Garea, M. Geist, and O. Bachem, “On-policy distillation of language models: Learning from self-generated mistakes,” in International Conference on Learning Representations (ICLR), 2024.arxiv.org/abs/2306.13649
- [27]N. Shaul, C. Liu, A. Vahdat, and J. Berner, “Parallel decoding distillation for fast image and video generation,” arXiv preprint arXiv:2607.26004, 2026.arxiv.org/abs/2607.26004
- [28]H. He, Z. Shang, C. Wang, X. Li, Y. Gu, H. Hua, L. Liu, C. Yang, J. Li, P. Shi et al., “Emilia: An extensive, multilingual, and diverse speech dataset for large-scale speech generation,” in IEEE Spoken Language Technology Workshop (SLT), 2024.arxiv.org/abs/2407.05361
- [29]B. Warner, A. Chaffin, B. Clavié, O. Weller, O. Hallström, S. Taghadouini, A. Gallagher, R. Biswas, F. Ladhak, T. Aarsen et al., “Smarter, better, faster, longer: A modern bidirectional encoder for fast, memory efficient, and long context finetuning and inference,” arXiv preprint arXiv:2412.13663, 2024.arxiv.org/abs/2412.13663
- [30]K. Baba, W. Nakata, Y. Saito, and H. Saruwatari, “The T05 system for the VoiceMOS challenge 2024: Transfer learning from deep image classifier to naturalness MOS prediction of high-quality synthetic speech,” in IEEE Spoken Language Technology Workshop (SLT), 2024.arxiv.org/abs/2409.09305
- [31]C. K. A. Reddy, V. Gopal, and R. Cutler, “DNSMOS: A non-intrusive perceptual objective speech quality metric to evaluate noise suppressors,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2021, pp. 6493–6497.arxiv.org/abs/2010.15258
- [32]H. Wang, S. Zheng, Y. Chen, L. Cheng, and Q. Chen, “CAM++: A fast and efficient network for speaker verification using context-aware masking,” in Proc. Interspeech, 2023, pp. 5331–5335.arxiv.org/abs/2303.00332
- [33]Y. Chen, S. Zheng, H. Wang, L. Cheng, Q. Chen, and J. Qi, “An enhanced Res2Net with local and global feature fusion for speaker verification,” in Proc. Interspeech, 2023, pp. 2238–2242.arxiv.org/abs/2305.12838
- [34]S. Takamichi, K. Mitsui, Y. Saito, T. Koriyama, N. Tanji, and H. Saruwatari, “JVS corpus: free japanese multi-speaker voice corpus,” arXiv preprint arXiv:1908.06248, 2019.arxiv.org/abs/1908.06248