WorksComputer Science
FoleyForge: Free-form, Controllable, Rectified-flow Editing of Sound Effects
Abstract
We present FoleyForge, a generative model for instruction-based editing of sound effects. Given an input clip and a free-form instruction describing the desired change, it applies the edit while preserving the input's identity. A single model handles semantic, stylistic, and DSP-like changes. We introduce a data pipeline that guides an audio language model to author both DSP-based acoustic edits and similarity-preserving semantic edits, thus addressing the lack of publicly available audio editing data. Such data is then used to fine-tune a rectified-flow audio generation model, which we extend to expose attribute controls for loudness change, edit strength, and an input-derived contour envelope, each tunable via classifier-free guidance, yielding a more expressive editing model. FoleyForge achieves state-of-the-art instruction adherence at competitive audio quality and source faithfulness.
abstract: We present FOLEYFORGE, a generative model for instruction-based editing of sound effects. Given an input clip and a free-form instruction describing the desired change, it applies the edit while preserving the input’s identity. A single model handles semantic, stylistic, and DSP-like changes. We introduce a data pipeline that guides an audio language model to author both DSP-based acoustic edits and similarity-preserving semantic edits, thus addressing the lack of publicly available audio editing data. Such data is then used to fine-tune a rectified-flow audio generation model, which we extend to expose attribute controls for loudness change, edit strength, and an input-derived contour envelope, each tunable via classifier-free guidance, yielding a more expressive editing model. FOLEYFORGE achieves state-of-the-art instruction adherence at competitive audio quality and source faithfulness.
Introduction
Text-to-audio generation now produces sound effects (SFX) of remarkable quality [1, 2, 3, 4], yet generation alone does not match how sound designers actually work. Sound design, especially for video content such as film, games, and short-form media, is fundamentally an iterative, per-source process: designers layer individual sources (e.g., footsteps, impacts, whoos, creaks) and repeatedly refine each one to lock it to the on-screen action and to creative direction: “make the punch heavier”, “muffle the bark”, or “brighten the sword”. With current generative models, the standard way to apply such a change is to rewrite the text prompt and resynthesize from scratch. This discards the very identity the designer has already dialed in (e.g., its timing, timbre, and/or texture) and, crucially, its synchronization to picture, producing unpredictable, uncontrolled clips. The image domain has already crossed this gap: instruction-based editors, whether open [5, 6] or proprietary [7] let a user recolor a shirt or swap a material while leaving the rest of the image untouched. Similar use cases would benefit audio, yet they remain largely unexplored.
Existing approaches fall into two families. The first edits a clip toward a full target description of the desired output [8, 9, 10]; the second, the strongest of which is SAO-Instruct [11], conditions directly on an edit instruction [12, 13, 14]. The instruction-based formulation is typically more natural for a designer, who would rather say what to change than fully re-describe the resulting sound, and it is the setting we adopt. Training-free methods forge fine-tuning: SDEdit [15] noises and re-denoises the input, regenerating a new audio, while EchoEdit+ [16] is inversion-free, editing via the difference of the rectified-flow velocities under the source and target prompts. Both lack explicit control over edit magnitude and tend to under-edit, preserving the input at the expense of actually applying the instruction, especially when the instruction is broad. Beyond method design, and for more semantic/creative edits, the core obstacle is often data: supervised approaches need curated edits, and to our knowledge no such SFX dataset is publicly available.
We introduce FOLEYFORGE, an instruction-driven, single-source SFX editor tailored to sound-design workflows (Figure 1). To overcome the data bottleneck, we build a synthetic data pipeline that combines complementary techniques to generate large-scale editing triplets, and use it to fine-tune a rectified-flow [17] version of Sketch2Sound [18]. The input clip is supplied through an alignment-free sequence-prepend conditioning that imposes no fixed duration or frame-level correspondence. The model is trained to preserve input identity across semantic, stylistic, and acoustic edits. FOLEYFORGE further exposes attribute controls (e.g., loudness, edit strength) that can be steered either as continuous scalars or as natural-language magnitude phrases (e.g., “a subtle” vs. “a drastic” edit), following SILA [19] and guided by classifier-free guidance [20]; this yields an explicit, controllable trade-off between edit magnitude and faithfulness to the reference. FOLEYFORGE achieves state-of-the-art instruction adherence and edit magnitude while maintaining competitive audio quality and source faithfulness, corroborated by standard automatic metrics, large audio language model judges, and a listening study. We strongly encourage readers to listen to samples at the demo URL shown at the top of Figure 1.
Figure 1. FOLEYFORGE overview. Left: data pipeline for DSP (top-left), generative (bottom), auxiliary (top-right) edits; input/output in red/blue, instructions in italic. Right: training and inference overview; dotted lines indicate train-only.
Method
Task formulation
Given an input sound effect and a free-form instruction naming the change to apply, the model produces an output that realizes the instructed change while preserving everything the instruction leaves untouched. The instruction describes the edit (“make it metallic”), not the target sound (“a metallic ball”), and a single model covers semantic edits, stylistic edits, and DSP-like edits (e.g., EQ, reverb, pitch, time-stretch). The user can also tune specific aspects of an edit (e.g., loudness) through fine-grained controls.
Data Pipeline
Our data pipeline (Figure 1 left) combines three types of edits: DSP for fine-grained edits; Generative for semantic, stylistic, and more creative ones; and Auxiliary for robustness. It yields triplets of the form {input audio, output audio, text instruction}.
DSP edits are achievable with signal processing alone. We select DSP operations that are common tools in sound design, such as reverb, low-pass filter, and pitch shift, and define a set of allowed parameters, similar to [11]. We then prompt Qwen3-Omni-30B-A3B-Captioner [21] (Qwen) to listen to the input audio and create an edit instruction achievable only with those operations, roughly following [22]. The output audio is rendered by applying the chain of operations with librosa [23] and torchaudio [24].
Generative edits cannot be achieved with DSP alone, e.g., turning a metal door sound into a wooden door sound. Given sound effect descriptions from annotated data, we prompt Qwen for an edit instruction and an output caption, restricting transformations so that input and output stay within domain: action changes within the same subject, small subject changes, location changes, and material changes are allowed; unrelated substitutions are not. Both audios are synthesized with Sketch2Sound, either by conditioning on the input energy curve or, when the edit requires a change in energy profile, by synthesizing both from the same noise latent, which yields closely related audio.
Auxiliary edits are added for robustness: compound edits combine DSP and generative operations, and therefore multiple goals in a single instruction; inverse edits, computed both A→B and B→A, to nudge the model toward cyclic edits (not applied in compound cases); and no-op edits that ask the model to reconstruct the input unchanged.
Model
FoleyForge’s backbone (Figure 1 right) is a rectified-flow version of Sketch2Sound, a text-to-audio model operating in the latent space of a neural audio codec, fine-tuned to allow control signals such as RMS envelope, and text-conditioned via cross-attention to FLAN-T5-XXL embeddings [25].
Source conditioning via sequence prepend. We condition the editor on the source clip by encoding it to clean latent tokens (at ) and prepending them as a separate, variable-length token block to the noisy output-latent sequence. A shared positional embedding is applied over the concatenated sequence, and the transformer’s self-attention lets the output tokens attend to the clean source tokens, and vice versa. The prepended block is sliced off before the loss, so the training objective remains output-only. Since the source enters as a variable-length prefix rather than a channel-concatenated, time-aligned condition, the output latent is decoupled from the source’s length and grid: it need not match the source’s duration or be re-sampled or padded onto a shared latent time axis (useful, e.g., when adding reverb). This differs from instruction-based editors such as SAO-Instruct, which concatenate the source latent to the model’s input channels and therefore register input and output on a common latent time grid, limiting the degree of the edit.
Instruction conditioning and guidance. The edit instruction is encoded by the text encoder with a fixed role-tag prefix (“Edit instruction: …”) and injected via cross-attention; a separate, optional output-caption slot describing the desired result enters the same way. We use two guidance configurations: (a) the output caption is the classifier-free-guidance (CFG) axis, with the instruction always on; and (b) the instruction is the guided axis, with the output caption retained only as a training aid. At inference the prepended source is held in the always-on set, i.e., present in every CFG partition and never zeroed. Thus CFG amplifies the guided text axis while the prepended source anchors input identity.
Editing inference. The output latent is initialized by SDEdit [15]: the encoded input is noised to an intermediate diffusion time and denoised from there, so the result stays anchored to the input’s structure while realizing the instructed change. A noise strength parameter sets how far from the input the initialization starts, so higher means more freedom to edit.
Fine-grained Controls
From each training pair we derive a set of relative controls that a user can also dial at inference. Unless noted, a control is a scalar or short vector that is projected and added to every latent frame (a per-frame additive term).
Loudness. The scalar , normalized; sets how loud the edit should be, independently of its content.
Edit strength. A multi-scale log-mel-spectrogram distance between input and output, computed on loudness-normalized audio so it is loudness-blind and orthogonal to the loudness control. It conditions the overall degree of change.
Energy envelope. The median-filtered RMS curve that reflects the output’s temporal amplitude contour (transients, onsets, rhythm). It is taken from the target during training and, since the target is unavailable at inference, from the input at test time, with a guidance strength that dials how strictly this contour is followed.
Natural-language magnitude (SILA). We apply a SILA-style strategy [19] to quantize the edit strength into phrases (“a subtle / moderate / pronounced / drastic edit”), appended to the instruction based on the same log-mel-spectrogram distance as in the edit strength control. This allows the amount of change to be also steerable in plain language.
Experiments
Objective Metrics
Our metrics fall into two groups: how well the edit was performed, and the overall audio quality of the result.
Editing strength. Three metrics are cosine similarities in the LAION-CLAP embedding space [26]. Alignment measures how well the output audio matches the target caption. Because it ignores the input, a system that simply generates a new clip matching the caption also scores well, so we read it alongside direction: the similarity between the audio- and text-embedding changes from input to output. Direction rewards moving the source toward the target, penalizing both no-op edits and off-target regenerations. Faithfulness is the input–output audio similarity, which is high when a system barely edits. As a purely acoustic counterpart to these CLAP metrics, edit magnitude (msMel, = larger edit) is the multi-scale log-mel-spectrogram distance between loudness-normalized input and output. Finally, adherence measures instruction following directly using large audio–language model (LALM) judges. Each judge sees the input clip, the edited clip, and the instruction, and rates from 1 to 5 how well the instruction was applied while the rest of the sound was left intact. We report the mean rating. Our primary judge is Qwen3-Omni-Instruct [21]. Because Qwen also wrote the edit instructions (§ 2.2), we additionally report Gemini 3.7 Flash [27], which plays no other role in our pipeline.
Audio quality. We report Fréchet Audio Distance (FAD, ) between the model outputs and a fixed, proprietary reference corpus of real sound effects. The same reference corpus is used for both test sets, so FAD is comparable across them. For high-level quality we report the Production Quality axis (PQ, ) of the Audiobox-Aesthetics predictor [28], a per-clip, reference-free aesthetic score averaged over the outputs.
Setup
Model. We fine-tune only the rectified-flow transformer, keeping the text and audio encoders frozen, for 80k steps on 8 GPUs with AdamW at a peak learning rate of (300-step linear warmup, a stable phase, then linear decay over the final of steps). The text guidance signals are trained with dropout so each learns a null direction: in configuration (a) the output caption is dropped 50% of the time with the instruction always on, and in configuration (b) the instruction is lightly dropped (10%) while the output caption is kept only as a high-dropout (50%) training aid. The prepended source is likewise lightly dropped (10%) during training but held always-on at inference. Each fine-grained control is trained with dropout as well (loudness and edit strength at 0.3, the energy envelope at 0.9) so it learns a per-control null direction; at inference, this lets any single control axis be amplified independently via CFG while the others are held at their trained strength.
Data. Our final training dataset consists of 464k triplets: 173k DSP (), 118k Generative (), and 173k Auxiliary (). The Auxiliary set contains three types of edits: compound edits (59k), inverse edits (86k; not applied to compound edits), and no-op edits (28k). The source data is an internally licensed dataset of high-quality sound-effect paired with human-annotated descriptions. We evaluate on two benchmarks, one of more subtle edits and one of more creative ones. Subtle Set is a 100-sample, fully human-reviewed set with ground truth, split evenly between DSP and generative edits. It is in-distribution for our models, as it is built with the same pipeline used to generate training data, but shares no overlap with the training or validation splits. Creative Set has no ground-truth: its 90 inputs are generated by a non-finetuned Sketch2Sound and paired with edit instructions only, since the edits are too broad to be achievable through traditional DSP operations. The instructions are out-of-distribution for all models, but the audio is in-distribution since it was generated by Sketch2Sound. All inputs and instructions were manually reviewed.
We compare FOLEYFORGE against three baselines. SAO-Instruct [11], the instruction-based state of the art, uses a different backbone and training data. We run its public checkpoint1 on our benchmarks, as its test set is unreleased. SDEdit [15], which noises the input and denoises it toward the instruction, and EchoEdit+ [16], an inversion-free editor that differences source- and target-conditioned velocities, are training-free methods we apply to the un-finetuned generator from which FOLEYFORGE is initialized. Because they share FOLEYFORGE’s architecture and differ only in how the edit is realized, they isolate the contribution of our fine-tuning, data, and conditioning. We also report SDEdit+Sketch, a training-free variant that adds the input’s energy envelope to SDEdit, as in Sketch2Sound. For FOLEYFORGE, we use SDEdit noise level (: clean, : pure noise) and envelope guidance , with other controls at their trained defaults (, energy envelope grounding on); edit strength and the loudness scalar are unused. FOLEYFORGE+ uses the same checkpoint as above but applies EchoEdit+’s training-free optimal-transport coupling, which noises the edited latent like the source so both velocity queries stay in-distribution, preventing drift along the trajectory.
Listening study. Twenty-five participants, screened for headphone use [29], rated 10 source clips under four anonymized, order-randomized conditions: SAO-Instruct, SDEdit, FOLEYFORGE, and a hidden anchor presenting the unedited source as an edit. Due to licensing issues, we randomly sample clips from the Creative Set inputs under 10 s (to keep the test short) and limited to three systems. Participants heard each source alongside its edit and rated audio quality and instruction adherence on a 1–5 scale.
Results and Discussion
Figure 2 plots adherence against faithfulness; an ideal editor lies toward the top right. The baselines each occupy a single corner: SDEdit (top left) adheres strongly, since it regenerates audio from the instruction, but nothing ties the output to the input, so faithfulness is low; EchoEdit+ (bottom right) is the most faithful but barely edits (direction ), often producing changes too subtle to perceive; and SAO-Instruct (middle right) is likewise faithful but subtle. FOLEYFORGE, by contrast, spans the frontier: it reaches the high-adherence end while staying more faithful than SDEdit, and sweeping the initialization noise traces the trade-off directly: higher raises adherence and lowers faithfulness. The same model also covers the opposite corner: FOLEYFORGE + moves into the faithful region near EchoEdit+, with the training-free SDEdit+Sketch in between at perceptible edit strengths.
Figure 2. Adherence vs. faithfulness, averaged over Subtle Set and Creative Set. An ideal editor lies toward the top right.
Quantitative results. Table 1 breaks this down per test set and metric group. FOLEYFORGE achieves the highest Qwen adherence on both sets and the highest alignment on Creative Set, while FOLEYFORGE + leads alignment on Subtle Set. Direction splits by set: FOLEYFORGE is best on Creative Set, while on Subtle Set SDEdit is nearest the reference, with SDEdit+Sketch and FOLEYFORGE just behind. The two variants bracket the faithfulness axis, FOLEYFORGE + matching the most faithful baseline, EchoEdit+. Audio quality is similar across systems, so we read it as a check that editing does not degrade the output rather than as a ranking: FOLEYFORGE + has the best PQ on both sets (often above the reference edits) while FOLEYFORGE sits at the low end, consistent with it moving furthest from the input.
| Model | Subtle Set (ground truth, ) | Creative Set (no ground-truth, ) | ||||||||||||
| Editing Strength | Audio Quality | Editing Strength | Audio Quality | |||||||||||
| alig | faith | dir | Qwen | Gemini | FAD | PQ | alig | faith | dir | Qwen | Gemini | FAD | PQ | |
| Ground truth | 0.388 | 0.747 | 0.133 | 3.28 | 4.72 | 0.340 | 6.77 | – | – | – | – | – | – | – |
| SAO-Instruct [11] | 0.361 | 0.720 | 0.066 | 2.94 | 4.58 | 0.382 | 6.71 | 0.389 | 0.831 | 0.093 | 2.69 | 3.65 | 0.406 | 6.76 |
| SDEdit [15] | 0.293 | 0.298 | 0.127 | 3.28 | 3.99 | 0.432 | 6.58 | 0.339 | 0.443 | 0.094 | 3.34 | 4.31 | 0.423 | 6.66 |
| EchoEdit+ [16] | 0.365 | 0.902 | 0.029 | 2.52 | 4.10 | 0.349 | 6.85 | 0.371 | 0.895 | 0.031 | 2.31 | 3.45 | 0.380 | 6.79 |
| SDEdit+Sketch (ours) | 0.339 | 0.545 | 0.113 | 3.06 | 4.55 | 0.387 | 6.58 | 0.393 | 0.666 | 0.092 | 3.27 | 4.27 | 0.409 | 6.67 |
| FoleyForge (ours) | 0.352 | 0.491 | 0.111 | 3.34 | 4.27 | 0.406 | 6.47 | 0.395 | 0.602 | 0.100 | 3.44 | 4.42 | 0.430 | 6.70 |
| FoleyForge+ (ours) | 0.370 | 0.856 | 0.050 | 2.36 | 4.16 | 0.351 | 6.96 | 0.373 | 0.889 | 0.032 | 2.47 | 3.55 | 0.381 | 6.94 |
Table 1. Objective results for our models and baselines on Subtle and Creative sets. Best results in bold, second bests underlined.
LALM judges. The signal-level metrics above measure how much the audio changed, not whether the instruction was followed, so we turn to the LALM judges for adherence. Creative Set makes the gap concrete: SAO-Instruct and FOLEYFORGE have nearly identical direction (0.093 vs. 0.100), yet both judges separate them by a wide margin favoring FOLEYFORGE. Neither judge is a ranking on its own (SDEdit ties the reference under Qwen on Subtle Set despite a faithfulness of only 0.298, and SAO-Instruct holds 0.831 faithfulness on Creative Set yet still scores well below FOLEYFORGE, producing edits too subtle to register) so we read the judges alongside alignment, direction, and faithfulness. On Subtle Set, FOLEYFORGE is statistically indistinguishable from the reference edits under Qwen (3.34 vs. 3.28; Wilcoxon signed-rank, ), while SAO-Instruct () and EchoEdit+ () fall significantly below it. The judges agree on Creative Set, where FOLEYFORGE leads both by a wide margin; they diverge only on Subtle Set, where Gemini places SAO-Instruct first (4.58 vs. 4.27).
Listening study. Our hidden anchor, as expected, obtains the highest audio quality (4.54 0.80) and the lowest adherence (1.76 1.24). Among the editing systems, FOLEYFORGE ranks first on adherence (3.22 1.29), ahead of both SDEdit (2.92 1.52, ) and SAO-Instruct (2.69 1.41, ); SDEdit and SAO-Instruct do not separate (). All tests are Wilcoxon signed-rank on per-rater, per-item pairs with Holm correction. On audio quality, FOLEYFORGE places second (3.43 1.19), below SAO-Instruct (3.73 1.16, ) and level with SDEdit (3.24 1.39). The study thus prices exactly the trade-off our design targets: FOLEYFORGE trades a small but reliable amount of audio quality against SAO-Instruct for a substantially larger gain in adherence. The human ranking agrees with Qwen’s on the Creating Set; licensing restrictions prevented a listening study on the Subtle Set, so the divergence there remains unresolved and is left to future work.
Controllability. We verify that each attribute control is calibrated and monotone by sweeping one control at a time, averaged over both test sets (Figure 3). We report two configurations: the main setting from the main results (energy envelope on) and an isolated setting that varies only the control of interest with the envelope off. Loudness moves delivered loudness monotonically toward the commanded target in both settings, with the isolated setting tracking closer to the ideal response. Edit strength and its SILA phrase counterpart both raise edit magnitude (msMel) monotonically, the SILA phrases stepping cleanly from none to drastic. The energy envelope grounds the output to the source: enabling it raises the input–output contour correlation (0.11 0.37). Across all four controls, the isolated setting spans a wider range than the main setting, i.e., the always-on envelope trades control authority for source grounding. FOLEYFORGE thus exposes a controllable family of operating points rather than a single behavior, making it flexible and easily tunable to user needs.
Figure 3. Control ablations. Each attribute control is monotone; the on-by-default energy envelope of the model with the main-results configuration trades control for faithfulness.
Conclusion
We presented FOLEYFORGE, an instruction-based editor for single-source sound effects that preserves input identity across semantic, stylistic, and DSP-like edits. It combines a synthetic data pipeline, in which a LALM authors DSP-verifiable and similarity-preserving semantic edits, with a length-flexible sequence–prepend condition that anchors the output to the source without tying it to the source’s time grid. FOLEYFORGE attains state-of-the-art instruction adherence at competitive audio quality and faithfulness. And where each baseline sits at a single fixed point on the faithfulness–adherence trade-off, FOLEYFORGE’s controls let the user move along it, with an inversion-free mode (FOLEYFORGE +) reaching the faithful end. Rather than one more point on the curve, FOLEYFORGE gives sound designers a highly flexible model tunable to their creative endeavors.
References
- [1]Z. Evans et al., “Stable Audio Open,” 2024, arXiv:2407.14358.arxiv.org/abs/2407.14358
- [2]F. Kreuk et al., “AudioGen: Textually Guided Audio Generation,” in International Conference on Learning Representations (ICLR), 2023.arxiv.org/abs/2209.15352
- [3]H. Liu et al., “AudioLDM: Text-to-Audio Generation with Latent Diffusion Models,” in International Conference on Machine Learning (ICML), 2023, vol. 202 of Proceedings of Machine Learning Research, pp. 21450–21474, PMLR.
- [4]H. Liu et al., “AudioLDM 2: Learning Holistic Audio Generation with Self-Supervised Pretraining,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 32, pp. 2871–2883, 2024.
- [5]T. Brooks et al., “InstructPix2Pix: Learning to Follow Image Editing Instructions,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023.arxiv.org/abs/2211.09800
- [6]A. Hertz et al., “Prompt-to-Prompt Image Editing with Cross-Attention Control,” in International Conference on Learning Representations (ICLR), 2023.arxiv.org/abs/2208.01626
- [7]Google, “Nano Banana (Version Pro),” Generative AI image model, 2026, Accessed: September 5, 2026.
- [8]Y. Wang et al., “AUDIT: Audio Editing by Following Instructions with Latent Diffusion Models,” in Advances in Neural Information Processing Systems (NeurIPS), 2023, vol. 36.arxiv.org/abs/2304.00830
- [9]H. Manor et al., “Zero-Shot Unsupervised and Text-Based Audio Editing Using DDPM Inversion,” in International Conference on Machine Learning (ICML), 2024, vol. 235 of Proceedings of Machine Learning Research, pp. 34603–34629, PMLR.arxiv.org/abs/2402.10009
- [10]M. Xu et al., “Prompt-Guided Precise Audio Editing with Diffusion Models,” in International Conference on Machine Learning (ICML), 2024, vol. 235 of Proceedings of Machine Learning Research, pp. 55126–55143, PMLR.arxiv.org/abs/2406.04350
- [11]M. Ungersböck et al., “SAO-Instruct: Free-Form Audio Editing Using Natural Language Instructions,” in Advances in Neural Information Processing Systems (NeurIPS), 2025, vol. 38.
- [12]Z. Lan et al., “Guiding Audio Editing with Audio Language Model,” in Workshop on Generative and Protective AI for Content Creation at the Conference on Neural Information Processing Systems (NeurIPS), 2025.arxiv.org/abs/2509.21625
- [13]L. Gao et al., “RFM-Editing: Rectified Flow Matching for Text-Guided Audio Editing,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2026.arxiv.org/abs/2509.14003
- [14]J. Liang et al., “WavCraft: Audio Editing and Generation with Large Language Models,” in Workshop on Large Language Model (LLM) Agents at the International Conference on Learning Representations (ICLR), 2024.arxiv.org/abs/2403.09527
- [15]C. Meng et al., “SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations,” in International Conference on Learning Representations (ICLR), 2022.arxiv.org/abs/2108.01073
- [16]Z. Fu et al., “EchoEdit: Stabilizing Inversion-Free Audio Editing via Optimal Transport Geometry,” 2026, arXiv:2606.15149.arxiv.org/abs/2606.15149
- [17]X. Liu et al., “Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow,” in International Conference on Learning Representations (ICLR), 2023.arxiv.org/abs/2209.03003
- [18]H. Flores García et al., “Sketch2Sound: Controllable Audio Generation via Time-Varying Signals and Sonic Imitations,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2025.arxiv.org/abs/2412.08550
- [19]S. Kumar et al., “SILA: Signal-to-Language Augmentation for Enhanced Control in Text-to-Audio Generation,” in IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), 2025.arxiv.org/abs/2412.09789
- [20]J. Ho et al., “Classifier-Free Diffusion Guidance,” in Workshop on Deep Generative Models and Downstream Applications at the Conference on Neural Information Processing Systems (NeurIPS), 2021.arxiv.org/abs/2207.12598
- [21]J. Xu et al., “Qwen3-Omni Technical Report,” 2025, arXiv:2509.17765.arxiv.org/abs/2509.17765
- [22]A. Chu et al., “Text2FX: Harnessing CLAP Embeddings for Text-Guided Audio Effects,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2025.arxiv.org/abs/2409.18847
- [23]B. McFee et al., “librosa: Audio and Music Signal Analysis in Python,” in Python in Science Conference (SciPy), 2015.DOI
- [24]J. Hwang et al., “TorchAudio 2.1: Advancing Speech Recognition, Self-Supervised Learning, and Audio Processing Components for PyTorch,” 2023, arXiv:2310.17864.arxiv.org/abs/2310.17864
- [25]H. W. Chung et al., “Scaling Instruction-Finetuned Language Models,” Journal of Machine Learning Research, vol. 25, no. 70, pp. 1–53, 2024.
- [26]Y. Wu et al., “Large-Scale Contrastive Language-Audio Pre-training with Feature Fusion and Keyword-to-Caption Augmentation,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2023.
- [27]Google, “Gemini 3.7 Flash,” Multimodal large language model, 2026, Accessed: September 10, 2026.
- [28]A. Tjandra et al., “Meta Audiobox Aesthetics: Unified Automatic Quality Assessment for Speech, Music, and Sound,” 2025, arXiv:2502.05139.arxiv.org/abs/2502.05139
- [29]K. J. P. Woods et al., “Headphone Screening to Facilitate Web-Based Auditory Experiments,” Attention, Perception, & Psychophysics, vol. 79, no. 7, pp. 2064–2072, 2017.


