betaFeedback

WorksComputer Science

Spatial Audio Generation and Editing with Executable Plans and Self-Distilling Transfusion

Abstract

Spatial audio authoring coordinates source decisions with soundfield synthesis. For editing, plans specify what to change and retain, while reference audio supplies further acoustic detail. We present AMBIT, a unified framework for native First-Order Ambisonics (FOA) generation and editing with three connected contributions. First, a shared Transfusion backbone couples autoregressive planning with rectified-flow rendering through executable ScenePlans that specify source content, activity, and 3D trajectories. For editing, the plan describes the full target scene, including retained sources; contrastive features ground the requested change, while full reference latents supply residual acoustic detail for preservation. Second, an FOA-aware continuous VAE combines shared-decoder omnidirectional recovery, asymmetric grouped KL, and spatial covariance matching to preserve content and inter-channel structure. Third, joint Transfusion on-policy self-distillation (OPSD) refines planning on student-generated prefixes and rendering at sampling states reached under student plans, with target-conditioned teachers supervising both modes of the shared student. Evaluations across speech, music, and sound show improved generation content and spatial quality, acoustic fidelity during instruction-guided editing, and unedited-source preservation over the compared baselines. Project page: https://wjm-george.github.io/AMBIT/

References

  1. [1]Agarwal, R., Vieillard, N., Zhou, Y., Stanczyk, P., Ramos Garea, S., Geist, M., & Bachem, O. (2024). On-policy distillation of language models: Learning from self-generated mistakes. International Conference on Learning Representations, 2024, 21246–21263.
  2. [2]Agostinelli, A., Denk, T. I., Borsos, Z., Engel, J., Verzetti, M., Caillon, A., Huang, Q., Jansen, A., Roberts, A., Tagliasacchi, M., Sharifi, M., Zeghidour, N., & Frank, C. (2023). MusicLM: Generating Music From Text. https://arxiv.org/abs/2301.11325
  3. [3]Bakhturina, E., Lavrukhin, V., Ginsburg, B., & Zhang, Y. (2021). Hi-Fi multi-speaker English TTS dataset. https://arxiv.org/abs/2104.01497
  4. [4]Chen, H., Xie, W., Vedaldi, A., & Zisserman, A. (2020). VGGSound: A large-scale audio-visual dataset. ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 721–725.
  5. [5]Chen, Z., Fouhey, D. F., & Owens, A. (2022). Sound localization by self-supervised time delay estimation. European Conference on Computer Vision, 489–508.
  6. [6]Chen, W., Seetharaman, P., Kumar, R., Nieto, O., Watanabe, S., Salamon, J., & Jin, Z. (2026). AudioChat: Unified audio storytelling, editing, and understanding with Transfusion forcing. https://arxiv.org/abs/2602.17097
  7. [7]Défossez, A., Copet, J., Synnaeve, G., & Adi, Y. (2023). High Fidelity Neural Audio Compression. Transactions on Machine Learning Research.
  8. [8]Elizalde, B., Deshmukh, S., Al Ismail, M., & Wang, H. (2023). CLAP learning audio concepts from natural language supervision. ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 1–5.
  9. [9]Evans, Z., Parker, J. D., Carr, C., Zukowski, Z., Taylor, J., & Pons, J. (2025). Stable Audio Open. ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 1–5.
  10. [10]Fonseca, E., Favory, X., Pons, J., Font, F., & Serra, X. (2022). FSD50K: an open dataset of human-labeled sound events. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 30, 829–852.
  11. [11]Gandhi, S., Von Platen, P., & Rush, A. M. (2023). Distil-Whisper: Robust knowledge distillation via large-scale pseudo labelling. https://arxiv.org/abs/2311.00430
  12. [12]Gemmeke, J. F., Ellis, D. P., Freedman, D., Jansen, A., Lawrence, W., Moore, R. C., Plakal, M., & Ritter, M. (2017). Audio Set: An ontology and human-labeled dataset for audio events. 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 776–780.
  13. [13]Heydari, M., Souden, M., Conejo, B., & Atkins, J. (2025). ImmerseDiffusion: A generative spatial audio latent diffusion model. ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 1–5.
  14. [14]Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., & Chen, W. (2022). LoRA: Low-rank adaptation of large language models. International Conference on Learning Representations.
  15. [15]Hung, C.-Y., Majumder, N., Kong, Z., Mehrish, A., Zadeh, A., Li, C., Valle, R., Catanzaro, B., & Poria, S. (2026). TangoFlux: Super fast and faithful text to audio generation with flow matching and CLAP-ranked preference optimization. International Conference on Learning Representations, 2026, 150793–150816.
  16. [16]Ji, S., Jiang, Z., Wang, W., Chen, Y., Fang, M., Zuo, J., Yang, Q., Cheng, X., Wang, Z., Li, R., Zhang, Z., Yang, X., Huang, R., Jiang, Y., Chen, Q., Zheng, S., & Zhao, Z. (2025). WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modeling. International Conference on Learning Representations, 2025, 93809–93826. https://proceedings.iclr.cc/paper_files/paper/2025/file/ea1f5f0878d43ff4fb8bf64ef4a2326c-Paper-Conference.pdfproceedings.iclr.cc/paper_files/paper/2025/file/ea1f5f0878d43ff4fb8bf64ef4a2326c-Paper-Conference.pdf
  17. [17]Jia, Y., Chen, Y., Zhao, J., Zhao, S., Zeng, W., Chen, Y., & Qin, Y. (2025). AudioEditor: A training-free diffusion-based audio editing framework. ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 1–5.
  18. [18]Jiang, Y., Chen, Z., Ju, Z., Li, C., Dou, W., & Zhu, J. (2025). FreeAudio: Training-free timing planning for controllable long-form text-to-audio generation. Proceedings of the 33rd ACM International Conference on Multimedia, 9871–9880.
  19. [19]Jiang, Y., Chen, Z., Ju, Z., Dai, Y., Dou, W., & Zhu, J. (2026). ControlAudio: Tackling text-guided, timing-indicated and intelligible audio generation via progressive diffusion modeling. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 1394–1413.
  20. [20]Kim, C. D., Kim, B., Lee, H., & Kim, G. (2019). AudioCaps: Generating captions for audios in the wild. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 119–132.
  21. [21]Lan, Z., Hao, Y., & Zhao, M. (2025). Guiding audio editing with audio language model. https://arxiv.org/abs/2509.21625
  22. [22]Langman, R., Yang, X., Neekhara, P., Hussain, S., Casanova, E., Bakhturina, E., & Li, J. (2025). HiFiTTS-2: A large-scale high bandwidth speech dataset. https://arxiv.org/abs/2506.04152
  23. [23]Le Roux, J., Wisdom, S., Erdogan, H., & Hershey, J. R. (2019). SDR--half-baked or well done?. ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 626–630.
  24. [24]Lei, K., Zhang, Y., Pan, C., Pu, X., Guo, W., Li, R., & Zhao, Z. (2026). Towards Streaming Synchronized Spatial Audio Generation via Autoregressive Diffusion Transformer. https://arxiv.org/abs/2605.30940
  25. [25]Lei, K., Wen, C., Zhang, Y., Guo, W., Pan, C., Zhou, S., Li, Y., Li, R., Hu, R., Xu, H., et al. (2026). One-Stage Multi-Task Instruction-Guided 3D Spatial Audio Editing. https://arxiv.org/abs/2609.04975v2
  26. [26]Li, X., Liu, J., Chen, W., Zhu, H., Ma, Z., & Chen, X. (2026). Resonate: Reinforcing Text-to-Audio Generation via Online Feedback from Large Audio Language Models. https://arxiv.org/abs/2603.11661
  27. [27]Liu, X., Gong, C., & Liu, Q. (2023). Flow straight and fast: Learning to generate and transfer data with rectified flow. International Conference on Learning Representations.
  28. [28]Liu, X., Zhu, Z., Liu, H., Yuan, Y., Huang, Q., Cui, M., Liang, J., Cao, Y., Kong, Q., Plumbley, M. D., et al. (2025). WavJourney: Compositional audio creation with large language models. IEEE Transactions on Audio, Speech and Language Processing, 33, 2830–2844.
  29. [29]Liu, H., Yuan, Y., Liu, X., Mei, X., Kong, Q., Tian, Q., Wang, Y., Wang, W., Wang, Y., & Plumbley, M. D. (2024). AudioLDM 2: Learning holistic audio generation with self-supervised pretraining. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 32, 2871–2883.
  30. [30]Majumder, N., Hung, C.-Y., Ghosal, D., Hsu, W.-N., Mihalcea, R., & Poria, S. (2024). Tango 2: Aligning diffusion-based text-to-audio generations through direct preference optimization. Proceedings of the 32nd ACM International Conference on Multimedia, 564–572.
  31. [31]Manor, H., & Michaeli, T. (2024). Zero-shot unsupervised and text-based audio editing using DDPM inversion. Proceedings of the 41st International Conference on Machine Learning, 235, 34603–34629.
  32. [32]Meng, C., He, Y., Song, Y., Song, J., Wu, J., Zhu, J.-Y., & Ermon, S. (2022). SDEdit: Guided image synthesis and editing with stochastic differential equations. International Conference on Learning Representations.
  33. [33]Peebles, W., & Xie, S. (2023). Scalable diffusion models with transformers. 2023 IEEE/CVF International Conference on Computer Vision (ICCV), 4172–4182.
  34. [34]Qwen Team. (2026). Qwen3.5: Towards Native Multimodal Agents. https://qwen.ai/blog?id=qwen3.5qwen.ai/blog?id=qwen3.5
  35. [35]Saeki, T., Xin, D., Nakata, W., Koriyama, T., Takamichi, S., & Saruwatari, H. (2022). UTMOS: UTokyo-SaruLab system for VoiceMOS challenge 2022. https://arxiv.org/abs/2204.02152
  36. [36]Sarabia, M., Menyaylenko, E., Toso, A., Seto, S., Aldeneh, Z., Pirhosseinloo, S., Zappella, L., Theobald, B.-J., Apostoloff, N., & Sheaffer, J. (2023). Spatial LibriSpeech: An augmented dataset for spatial audio learning. https://arxiv.org/abs/2308.09514
  37. [37]Scheibler, R., Bezzam, E., & Dokmanić, I. (2018). Pyroomacoustics: A Python package for audio room simulation and array processing algorithms. 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 351–355.
  38. [38]Seki, K., Okamoto, Y., Yamaoka, K., Saito, Y., Takamichi, S., & Saruwatari, H. (2026). Spatial-CLAP: Learning spatially-aware audio--text embeddings for multi-source conditions. ICASSP 2026-2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 14742–14746.
  39. [39]Sudarsanam, P., Braun, S., & Gamper, H. (2025). FOA Tokenizer: Low-bitrate Neural Codec for First Order Ambisonics with Spatial Consistency Loss. https://arxiv.org/abs/2510.22241
  40. [40]Tao, Y., Wu, W., Zhang, C., Wu, M., Wang, S., & Xu, X. (2025). MMEdit: A unified framework for multi-type audio editing via audio language model. https://arxiv.org/abs/2512.20339
  41. [41]Templin, C., Zhu, Y., & Wang, H. (2026). Generating Moving 3D Soundscapes with Latent Diffusion Models. ICASSP 2026-2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2856–2860.
  42. [42]Tian, Z., Yang, B., Liu, Z., Zhang, J., Yuan, R., Yin, H., Chen, Q., Li, C., Lyu, J., Xue, W., et al. (2026). Audio-Omni: Extending Multi-modal Understanding to Versatile Audio Generation and Editing. https://arxiv.org/abs/2604.10708
  43. [43]Tian, Z., Ke, L., Liu, Z., Yuan, R., Xue, L., Yang, Y., Chen, W., Tan, X., Chen, Q., Xue, W., et al. (2026). AudioX-Turbo: A Unified Framework for Efficient Anything-to-Audio Generation. https://arxiv.org/abs/2606.12555
  44. [44]Tian, Z., Liu, Z., Jin, Y., Yuan, R., Xue, L., Tan, X., Chen, Q., Xue, W., & Guo, Y. (2026). AudioX: A Unified Framework for Anything-to-Audio Generation. International Conference on Learning Representations, 2026, 110625–110651. https://proceedings.iclr.cc/paper_files/paper/2026/file/b36dc39b319ba6ba2a0fd7601951efb4-Paper-Conference.pdfproceedings.iclr.cc/paper_files/paper/2026/file/b36dc39b319ba6ba2a0fd7601951efb4-Paper-Conference.pdf
  45. [45]Tian, Z., Liu, Z., Yuan, R., Pan, J., Liu, Q., Tan, X., Chen, Q., Xue, W., & Guo, Y. (2025). VidMuse: A simple video-to-music generation framework with long-short-term modeling. Proceedings of the Computer Vision and Pattern Recognition Conference, 18782–18793.
  46. [46]Wang, Y., Ju, Z., Tan, X., He, L., Wu, Z., Bian, J., et al. (2023). AUDIT: Audio editing by following instructions with latent diffusion models. Advances in Neural Information Processing Systems, 36, 71340–71357.
  47. [47]Wang, Y., Wang, X., Cheng, X., Chen, Y., & Song, R. (2026). Unified Synthesis of Compositional Speech and Sound from Free-Form Text Prompts. https://arxiv.org/abs/2605.28063
  48. [48]Wu, Y., Chen, K., Zhang, T., Hui, Y., Berg-Kirkpatrick, T., & Dubnov, S. (2023). Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation. ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 1–5.
  49. [49]Xie, Z., Xu, X., Wu, Z., & Wu, M. (2024). PicoAudio: Enabling precise timestamp and frequency controllability of audio events in text-to-audio generation. https://arxiv.org/abs/2407.02869
  50. [50]Xu, J., Liu, X., Wu, Y., Tong, Y., Li, Q., Ding, M., Tang, J., & Dong, Y. (2023). ImageReward: Learning and evaluating human preferences for text-to-image generation. Advances in Neural Information Processing Systems, 36, 15903–15935.
  51. [51]Xu, J., Guo, Z., Hu, H., Chu, Y., Wang, X., He, J., Wang, Y., Shi, X., He, T., Zhu, X., et al. (2025). Qwen3-Omni technical report. https://arxiv.org/abs/2509.17765
  52. [52]Yang, D., Wang, Y., Chong, D., Liu, S., Wu, X., & Meng, H. (2026). UniAudio 2.0: A unified audio language model with text-aligned factorized audio tokenization. https://arxiv.org/abs/2602.04683
  53. [53]Zen, H., Dang, V., Clark, R., Zhang, Y., Weiss, R. J., Jia, Y., Chen, Z., & Wu, Y. (2019). LibriTTS: A corpus derived from LibriSpeech for text-to-speech. https://arxiv.org/abs/1904.02882
  54. [54]Zhang, S., Liu, M., Tong, Y., He, W., Zou, S., Liu, J., Yu, Y., Song, J., Jiang, H., Huang, P., et al. (2026). Self-OPD: On-Policy Distillation for Flow Matching Models without Teacher. https://arxiv.org/abs/2608.26872
  55. [55]Zhou, C., Yu, L., Babu, A., Tirumala, K., Yasunaga, M., Shamis, L., Kahn, J., Ma, X., Zettlemoyer, L., & Levy, O. (2025). Transfusion: Predict the next token and diffuse images with one multi-modal model. International Conference on Learning Representations, 2025, 6446–6469.
  56. [56]Zhou, W., Zhu, X., Kong, L., Chen, B., Zhang, L., Liang, Y., Hou, X., Tian, Y., Sun, X., Wang, Y., et al. (2026). On-Policy Self-Distillation in Diffusion Models. https://arxiv.org/abs/2608.24646
  57. [57]Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Bi, X., Zhang, H., Zhang, M., Li, Y., Wu, Y., et al. (2024). DeepSeekMath: Pushing the limits of mathematical reasoning in open language models. https://arxiv.org/abs/2402.03300
  58. [58]Liu, J., Liu, G., Liang, J., Li, Y., Liu, J., Wang, X., Wan, P., Zhang, D., & Ouyang, W. (2025). Flow-GRPO: Training Flow Matching Models via Online RL. Advances in Neural Information Processing Systems, 38, Main Conference, 40783–40818. https://doi.org/10.52202/085713-1362
  59. [59]Xue, Z., Wu, J., Gao, Y., Kong, F., Zhu, L., Chen, M., Liu, Z., Liu, W., Guo, Q., Huang, W., et al. (2025). DanceGRPO: Unleashing GRPO on visual generation. https://arxiv.org/abs/2505.07818
  60. [60]Kumar, R., Seetharaman, P., Luebs, A., Kumar, I., & Kumar, K. (2023). High-fidelity audio compression with improved RVQGAN. Advances in Neural Information Processing Systems, 36, 27980–27993.
  61. [61]Zhao, S., Xie, Z., Liu, M., Huang, J., Pang, G., Chen, F., & Grover, A. (2026). Self-distilled reasoner: On-policy self-distillation for large language models. https://arxiv.org/abs/2601.18734
  62. [62]Jiang, D., Jin, X., Liu, D., Wang, Z., Zheng, M., Du, R., Yang, X., Wu, Q., Li, Z., Gao, P., et al. (2026). D-OPSD: On-Policy Self-Distillation for Continuously Tuning Step-Distilled Diffusion Models. https://arxiv.org/abs/2605.05204
  63. [63]Hu, H., Zhu, X., He, T., Guo, D., Zhang, B., Wang, X., Guo, Z., Jiang, Z., Hao, H., Guo, Z., et al. (2026). Qwen3-TTS technical report. https://arxiv.org/abs/2601.15621
  64. [64]Sun, P., Cheng, S., Li, X., Ye, Z., Liu, H., Zhang, H., Xue, W., & Guo, Y. (2025). Both ears wide open: Towards language-driven spatial audio generation. International Conference on Learning Representations, 2025, 42606–42649.
  65. [65]Hu, J., Cao, Y., Wu, M., Kang, F., Yang, F., Wang, W., Plumbley, M. D., & Yang, J. (2025). PSELDNets: Pre-trained neural networks on a large-scale synthetic dataset for sound event localization and detection. IEEE Transactions on Audio, Speech and Language Processing, 33, 2845–2860.
  66. [66]Kushwaha, S. S., Ma, J., Thomas, M. R., Tian, Y., & Bruni, A. (2025). Diff-SAGe: End-to-end spatial audio generation using diffusion models. ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 1–5.
  67. [67]Zhao, L., Chen, S., Feng, L., Zhang, J., Zhang, X.-L., Zhang, C., & Li, X. (2026). DualSpec: Text-to-spatial-audio generation via dual-spectrogram guided diffusion model. IEEE Transactions on Multimedia.
  68. [68]Yao, L., Huang, C., Yang, X., Geng, Z., Luo, C., Wang, H., Liu, J., & Pan, M. (2026). PhysWave: Physics-Guided Latent Diffusion Models for Controllable Spatial Audio Generation. https://arxiv.org/abs/2608.29549
  69. [69]Kim, J., Yun, H., & Kim, G. (2025). ViSAGe: Video-to-spatial audio generation. International Conference on Learning Representations, 2025, 46404–46424.
  70. [70]Kim, J., Yun, H., & Kim, G. (2026). Towards Scene-Aware Video-to-Spatial Audio Generation. International Journal of Computer Vision, 134(4), 184.
  71. [71]Feng, L., Zhao, L., Zhu, B., Zhang, X.-L., & Li, X. (2025). AudioSpa: Spatializing sound events with text. https://arxiv.org/abs/2502.11219
  72. [72]Pan, T., Liu, J., Huang, Z., Tang, J., & Wu, G. (2025). In-the-wild audio spatialization with flexible text-guided localization. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 1989–2001.
  73. [73]Lyth, D., & King, S. (2024). Natural language guidance of high-fidelity text-to-speech with synthetic annotations. https://arxiv.org/abs/2402.01912
  74. [74]Pulkki, V. (2006). Directional audio coding in spatial sound reproduction and stereo upmixing. Proc. Of the AES 28th Int. Conf, Pitea, Sweden.
  75. [75]Liu, H., Luo, T., Luo, K., Jiang, Q., Sun, P., Wang, J., Huang, R., Chen, Q., Wang, W., Li, X., Zhang, S., Yan, Z., Zhao, Z., & Xue, W. (2025). OmniAudio: Generating Spatial Audio from 360-Degree Video. Proceedings of the 42nd International Conference on Machine Learning, 267, 39060–39084. https://proceedings.mlr.press/v267/liu25as.htmlproceedings.mlr.press/v267/liu25as.html
  76. [76]Shimada, K., Politis, A., Sudarsanam, P., Krause, D. A., Uchida, K., Adavanne, S., Hakala, A., Koyama, Y., Takahashi, N., Takahashi, S., Virtanen, T., & Mitsufuji, Y. (2023). STARSS23: An Audio-Visual Dataset of Spatial Recordings of Real Scenes with Spatiotemporal Annotations of Sound Events. Advances in Neural Information Processing Systems, 36, 72931–72957. https://doi.org/10.52202/075280-3189
  77. [77]Liu, H., Wang, C., Gao, F., He, X., Ma, Y., Wan, Z., Zhang, Y., Wei, X., & Chen, Q. (2026). OPSD-V: On-Policy Self-Distillation for Post-Training Few-Step Autoregressive Video Generators. https://arxiv.org/abs/2607.08766
  78. [78]Ma, Y., Liu, X., Chen, X., Liu, W., Wu, C., Wu, Z., Pan, Z., Xie, Z., Zhang, H., Yu, X., et al. (2025). JanusFlow: Harmonizing autoregression and rectified flow for unified multimodal understanding and generation. 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 7739–7751.
  79. [79]Zhu, Z., Zhang, Y., Guo, W., Pan, C., & Zhao, Z. (2025). ASAudio: A Survey of Advanced Spatial Audio Research. https://arxiv.org/abs/2508.10924
  80. [80]Guo, W., Pan, C., Zhu, Z., Hu, X., Zhang, Y., Tang, L., Yang, R., Wang, H., Zhang, Z., Wang, Y., Chen, Y., Xu, H., Xu, K., Fan, P., Chen, Z., Yu, Y., Huang, Q., Wu, F., & Zhao, Z. (2025). MRSAudio: A Large-Scale Multimodal Recorded Spatial Audio Dataset with Refined Annotations. https://arxiv.org/abs/2510.10396
  81. [81]Zhang, Y., Guo, W., Pan, C., Zhu, Z., Jin, T., & Zhao, Z. (2025). ISDrama: Immersive Spatial Drama Generation through Multimodal Prompting. https://doi.org/10.1145/3746027.3755014
  82. [82]Zhang, Y., Li, R., Pan, C., Lei, K., Yin, X., & Yang, C. (2026). SwanTale: Unified Multi-Speaker Speech and Audio Generation for Instruct and Zero-Shot Tasks. https://arxiv.org/abs/2608.02023
  83. [83]Pan, C., Guo, W., Zhang, Y., Zhu, Z., Chen, Z., Wang, H., & Zhao, Z. (2025). A Multimodal Evaluation Framework for Spatial Audio Playback Systems: From Localization to Listener Preference. Proceedings of the 33rd ACM International Conference on Multimedia, 7006–7015. https://doi.org/10.1145/3746027.3755571
  84. [84]Li, R., Zhang, Y., Pan, C., Lei, K., Yin, X., & Yang, C. (2026). SwanVoice: Expressive Long-Form Zero-Shot Speech Synthesis for Both Monologue and Dialogue. https://arxiv.org/abs/2605.30993
  85. [85]Zhang, Y., Tian, B., & Duan, Z. (2025). Conan: A Chunkwise Online Network for Zero-Shot Adaptive Voice Conversion. https://arxiv.org/abs/2507.14534
  86. [86]Zhang, Y., Pan, C., Guo, W., Li, R., Zhu, Z., Wang, J., Xu, W., Lu, J., Hong, Z., Wang, C., Zhang, L., He, J., Jiang, Z., Chen, Y., Yang, C., Zhou, J., Cheng, X., & Zhao, Z. (2024). GTSinger: A Global Multi-Technique Singing Corpus with Realistic Music Scores for All Singing Tasks. Advances in Neural Information Processing Systems. https://doi.org/10.52202/079017-0034
  87. [87]Zhang, Y., Guo, W., Pan, C., Zhu, Z., Li, R., Lu, J., Huang, R., Zhang, R., Hong, Z., Jiang, Z., & Zhao, Z. (2025). Versatile Framework for Song Generation with Prompt-based Control. Findings of the Association for Computational Linguistics: EMNLP 2025. https://doi.org/10.18653/v1/2025.findings-emnlp.13
  88. [88]Zhang, Y., Guo, W., Pan, C., Yao, D., Zhu, Z., Jiang, Z., Wang, Y., Jin, T., & Zhao, Z. (2025). TCSinger 2: Customizable Multilingual Zero-shot Singing Voice Synthesis. Findings of the Association for Computational Linguistics: ACL 2025. https://doi.org/10.18653/v1/2025.findings-acl.687
  89. [89]Zhang, Y., Jiang, Z., Li, R., Pan, C., He, J., Huang, R., Wang, C., & Zhao, Z. (2024). TCSinger: Zero-Shot Singing Voice Synthesis with Style Transfer and Multi-Level Style Control. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 1960–1975. https://doi.org/10.18653/v1/2024.emnlp-main.117
  90. [90]Zhang, Y., Huang, R., Li, R., He, J., Xia, Y., Chen, F., Duan, X., Huai, B., & Zhao, Z. (2024). StyleSinger: Style Transfer for Out-of-Domain Singing Voice Synthesis. Proceedings of the AAAI Conference on Artificial Intelligence, 19597–19605. https://doi.org/10.1609/aaai.v38i17.29932
  91. [91]Guo, W., Zhang, Y., Pan, C., Zhu, Z., Li, R., Chen, Z., Xu, W., Wu, F., & Zhao, Z. (2025). STARS: A Unified Framework for Singing Transcription, Alignment, and Refined Style Annotation. https://arxiv.org/abs/2507.06670
  92. [92]Zhang, R., Wang, Q., Liu, J., Zhang, Y., Huo, Y., & Wu, C. (2025). Leveraging Pretrained Diffusion Models for Zero-Shot Part Assembly. https://arxiv.org/abs/2505.00426
  93. [93]Pan, C., Yao, D., Zhang, Y., Guo, W., Lu, J., Zhu, Z., & Zhao, Z. (2025). Synthetic Singers: A Review of Deep-Learning-based Singing Voice Synthesis Approaches. Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics, 396–416. https://doi.org/10.18653/v1/2025.ijcnlp-long.24
  94. [94]Guo, W., Zhang, Y., Pan, C., Huang, R., Tang, L., Li, R., Hong, Z., Wang, Y., & Zhao, Z. (2025). TechSinger: Technique Controllable Multilingual Singing Voice Synthesis via Flow Matching. https://arxiv.org/abs/2502.12572
  95. [95]Zhu, Z., Chen, Y., Shao, Y., Guo, W., Pan, C., Zhang, Y., Wang, Y., Liu, W., Zhang, H., Zeng, C., Cheng, W., Liu, Y., Yang, R., Yves, S., Bo, L., & Zhao, Z. (2026). Spatial-Omni: Spatial Audio Understanding Integration in Multimodal LLMs via FOA Encoding. https://arxiv.org/abs/2606.10738
  96. [96]Zhu, Z., Wang, H., Guo, W., Zhang, Y., Pan, C., Yang, R., & Zhao, Z. (2026). CSAVocoder: A Causal Spatial Audio Vocoder Towards Real-Time Spatial Audio Generation. https://arxiv.org/abs/2608.25404
  97. [97]Pan, C., Fan, Y., Zhuo, F., Chen, Y., Guo, W., Zhang, Y., Li, R., Zhu, Z., Yang, R., Ji, S., Wen, C., Xu, J., Lei, K., Yang, X., Lu, J., & Zhao, Z. (2026). Audio Editing in the Era of Foundation Models: A Survey. https://arxiv.org/abs/2606.23139
  98. [98]Feng, F., Fu, D., Xie, Z., Zhang, Y., Wu, Y., Zhao, Z., & Jin, T. (2026). Rectifying the Emotional Flow: Aligning Priors and Dynamic Guidance for High-Arousal Text-to-Speech. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 21874–21888. https://doi.org/10.18653/v1/2026.acl-long.998

Paper details

vlnc:W_yr9dcd2u
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents