WorksComputer Science
Think2Listen: Evidence-Centric Learning For Agentic Audio Understanding With LALMS
Abstract
Recent advances in large audio-language models (LALMs) have broadened audio understanding across speech, sound, and music, with tool-based agents further supporting these tasks by invoking predefined audio tools. However, a persistent gap remains between perceiving acoustic content and using it as evidence for reasoning: recognizing what is audible does not ensure that a model can identify which cues resolve an uncertain decision. To bridge this gap, we propose Think2Listen, an evidence-centric learning framework for agentic audio understanding. Guided by evidence before commitment, Think2Listen couples iterative reasoning with a targeted LALM interaction to actively seek acoustic cues that distinguish competing hypotheses and revise candidate judgments based on the observations returned. It learns what to ask, how to use evidence, and when to stop, using observation-induced changes in candidate-state to inform training credit. Extensive evaluations on two benchmarks spanning seven audio categories demonstrate that Think2Listen achieves (I) enhanced multi-step audio reasoning, with gains in overall accuracy of 5.08%–17.67% over competing audio agents; and (II) improved audio understanding and reasoning across speech, sound, and music, boosting overall accuracy by 2.29%–13.10% over the corresponding baselines. The source code is available at https://github.com/JinWang79/Think2Listen.
Introduction
Large Audio-Language Models (LALMs) have demonstrated increasingly broad capabilities across speech, sound, and music, enabling diverse auditory tasks through a shared language-based interface [53, 24]. In parallel, language-model agents have demonstrated how reasoning and environmental feedback can guide tool use and subsequent decisions [55, 36]. In the audio domain, agentic systems coordinate language models with specialized audio capabilities to support interactive task solving [16, 46]. Recent work has explored acquiring and refining task-relevant audio information during reasoning [50, 31, 59], as well as learning when and which audio tools to invoke [41, 2, 44].
Despite these advances, a persistent gap separates acoustic perception from the use of evidence in reasoning. Even when acoustic information is available, models may struggle to determine which cues are relevant to the question and how they support or rule out competing interpretations. Models can describe an audio scene yet fail to answer discriminative questions about it [19]. Even acoustic attributes recoverable from their encoders may remain poorly reflected in final predictions [37]. Together, these findings motivate treating acoustic cues as evidence for comparing candidate answers in agentic audio understanding.
Additional reasoning or interaction does not automatically bridge this gap. Reasoning may build on textual interpretations that are insufficiently grounded in acoustic properties [40], while intermediate stages of evidence collection can degrade performance relative to direct listening [43]. These limitations motivate interactions that seek acoustic cues relevant to unresolved distinctions between competing hypotheses. For example, Figure 1 presents an accent-identification case from MMSU [42]. Asking “Which accent?” requests another task-level prediction, whereas asking about /r/ realization or a word’s vowel quality seeks information about phonetic cues that can help distinguish candidate accents [14, 38]. This illustrates the principle of evidence before commitment: the agent should seek acoustic evidence that distinguishes competing hypotheses before committing to an answer.

Figure 1. An accent-identification interaction. An example showing the initial audio description, an end-to-end prediction, a label-level ASK and its observation, and a targeted ASK followed by its observation and candidate-set update.
However, a targeted query does not guarantee a justified revision of candidate judgments. Audio hallucination studies report confusion between semantic and acoustic attributes, as well as errors in event identification and temporal relations [5, 3]. Misleading observations may lead an agent to exclude the correct candidate, reducing apparent uncertainty on an unsupported premise. This poses a challenge for policy learning: final-answer correctness alone provides limited guidance on these revisions. An incorrect answer does not reveal whether the agent remained uncertain or excluded the correct candidate because of a misleading observation. Conversely, studies of mathematical reasoning show that correct answers can coexist with erroneous intermediate steps [62, 60]. This motivates complementing outcome feedback with information about intermediate progress [34] and, in our setting, how acquired observations change the agent’s candidate judgments.
Taken together, these limitations motivate learning evidence acquisition and candidate revision as a coupled process: unresolved distinctions between hypotheses should guide subsequent queries, and observation-induced changes in those hypotheses should inform training credit. Building on the interleaving of reasoning and action [55], we introduce Think2Listen, an evidence-centric learning framework for agentic audio understanding guided by evidence before commitment. Practically, its Adaptive Audio Evidence Loop allows an actor to choose between issuing a cue-specific query and submitting an answer. The LALM re-examines the recording for each query, while candidate revision and the final decision remain with the actor. To guide policy learning, we introduce Transition-as-Credit, inspired by feedback on intermediate progress [34]. It uses changes in candidate judgments to complement answer feedback, helping the agent learn from both its final answer and how it uses audio evidence. Our contributions can be summarized as follows:
Evidence-Seeking Interaction. We develop an Adaptive Audio Evidence Loop in which an actor formulates free-form queries to a LALM, relates the returned acoustic observations to competing hypotheses, and decides whether to seek further evidence or commit to an answer.
Transition-Aware Policy Learning. We introduce Transition-as-Credit, which reconstructs observation-induced candidate-state changes and uses them for group-adaptive calibration of trajectory rewards. This complements terminal correctness with information about intermediate candidate revisions, distinguishing unresolved ambiguity from misleading commitment. Together, these components enable the agent to learn what to ask, how to use evidence, and when to stop.
Empirical Validation. Extensive evaluations on two benchmarks spanning seven audio categories demonstrate that Think2Listen achieves (I) enhanced multi-step audio reasoning, with gains in overall accuracy of 5.08%–17.67% over competing audio agents; and (II) improved audio understanding and reasoning across speech, sound, and music, boosting overall accuracy by 2.29%–13.10% over the corresponding baselines.
Related Works
Large Audio-Language Models. Large audio-language models connect acoustic perception with language modeling, supporting audio understanding, conversation, and reasoning. Their development follows several overlapping directions: (1) audio understanding and conversation, which extends language interfaces to diverse auditory tasks, as in GAMA [10], Qwen2-Audio [6], DeSTA2.5-Audio [23], VITA-Audio [22], and SALMONN-2 [54]; (2) omni-modal audio understanding, which incorporates audio alongside text, images, and video, as in Mini-Omni2 [49], Gemini 2.0 Flash [13], Phi-4-Multimodal [1], AR-Omni [4], and Qwen3-Omni [52]; and (3) audio reasoning, which strengthens inference over acoustic content through reasoning-oriented data, prompting, or post-training, as explored in Audio-CoT [27], Audio-Reasoner [64], Step-Audio-R1 [40], Audio-Cogito [21], and Audio-DeepThinker [15].
Agentic Audio Understanding. Building on these audio capabilities, agentic systems use external tools and iterative interaction to acquire additional information during reasoning. Existing approaches differ in how they organize this interaction: (1) tool-based reasoning coordinates specialized audio capabilities, as in AudioGPT [16], AudioToolAgent [46], and Thinking-with-Sound [50]; (2) multi-agent evidence refinement progressively enriches audio-derived textual evidence through collaboration, as in AudioGenie-Reasoner [31] and AsymAudio [32]; and (3) learned tool selection and routing optimizes when and how external tools are invoked, as in AuTAgent [41] and AudioRouter [2]. In the related task of audio captioning, ACE-Cap [25] learns active evidence acquisition for caption generation and evaluates caption utility on downstream audio QA.
Background & Motivation
We contrast end-to-end prediction with interactive evidence acquisition and examine how untrained agents use this additional audio access.
Prediction paradigms (End-to-end and interactive inference). In the end-to-end setting, a LALM predicts an answer from the recording, task instruction, and answer candidates [51]. This protocol accommodates intermediate reasoning [12] while excluding explicit queries for additional acoustic evidence. Interactive inference extends the protocol through query-conditioned evidence acquisition prior to the final decision.
Empirical diagnostics (Interaction before policy training). We evaluate untrained interaction along two dimensions: performance relative to direct prediction and the outcomes of continued querying. Figure 2 summarizes the resulting patterns.
Figure 2. Interaction before policy training. (a–b) Pass@5 of the end-to-end LALM and untrained interactive agents. Filled ♣/♢/♡/♠ denote synthetic speech, non-verbal sound, duration, and dialogue turns; outlined suits denote stress, pitch, vocal range, and speaker counting in the same order. (c–d) Outcomes and candidate transitions by ASK depth and stage.
Task-dependent interaction gains. Figure 2(a–Figure 2b) compares the end-to-end LALM with untrained interactive agents on selected MMSU tasks. Their relative strengths vary, motivating policies that learn when to query and which acoustic distinctions to investigate.
Continued querying and unresolved decisions. Figure 2(c–Figure 2d) shows that continued querying can coexist with redundant requests and unsuccessful completion. This does not establish that extra queries cause errors, but highlights the need to learn how to use observations and when to stop.
Design motivation. Reliable audio interaction requires learning what to ask, how to revise answers, and when to stop. Training should use changes in candidate judgments to distinguish useful evidence seeking from redundant or misleading interaction.
Learning Evidence-Centric Audio Interaction
Section 3 shows that interaction can expose acoustic evidence but remains unreliable without policy training. We therefore introduce an evidence-centric framework with two components: the Adaptive Audio Evidence Loop for inference and Transition-as-Credit for training. Figure 3 summarizes the framework; the following sections describe the inference-time evidence loop, candidate-state reconstruction, group-adaptive credit assignment, and policy update.

Figure 3. Evidence-centric audio interaction and policy learning. Top: the actor requests audio evidence or submits an answer. Bottom: transition extraction and gold-aware reward calibration guide policy updates. Dashed paths indicate difficulty-aware sampling and teacher supervision.
Adaptive Audio Evidence Loop
The loop implements evidence before commitment through interaction between an actor and a LALM. The actor uses textual evidence to decide what to ask and when to answer, while the LALM accesses the recording to provide query-specific observations.
Given an instance , where is the recording, is the task instruction, and is the set of answer candidates, the actor first receives an initial audio description from the LALM. This description supplies the actor’s initial acoustic evidence. At turn , its state is
where contains the preceding query–observation pairs and is the remaining query budget. Initially, and , with denoting the maximum number of additional audio queries.
The actor policy samples an action:
Here is a free-form audio query and is the submitted answer. The query content itself is therefore a policy action, intended to resolve an acoustic distinction between candidates, such as a word’s pronunciation or the order of two events.
For , the audio model processes the recording conditioned on the query and returns a textual observation . Different queries can reveal different aspects of the same recording, allowing the actor to seek information missing from its current context.
After each query, the history and budget are updated:
where appends the query–observation pair. Using the updated context, the actor reassesses which candidates remain plausible and whether further evidence is needed, adapts its next query, and makes a stopping decision accordingly. An action submits the answer and ends the interaction. The next subsection describes how candidate revisions are reconstructed for training.
Candidate-State Transition Evaluation
Terminal correctness alone does not establish whether an individual interaction contributed useful evidence to the reasoning process. Two trajectories may arrive at the same final answer, even though one eliminates a plausible distractor while the other merely reiterates information already available. We therefore assess the evidential contribution of interactions in terms of the resulting changes in candidate state, rather than the surface form of their queries or observations.
For each ASK turn selected for evaluation, a semantic extractor , parameterized by , reconstructs how the returned observation changes the actor’s candidate state:
Here and are the task instruction and answer candidates, is the initial audio description, contains the preceding query–observation history, and is the actor reasoning associated with the current query . The output marks whether the query is specific and decision-relevant; and are the plausible candidate sets before and after the observation; and is the set of eliminated candidates based on the returned observation.
The structured output is accepted only when its labels belong to and satisfy , , and ; otherwise the extraction is recorded as unresolved. Importantly, receives only information available along the trajectory and never observes the gold answer . It therefore performs candidate-state reconstruction rather than reward assignment. § 4.3 describes how these structured transitions are subsequently converted into group-relative training credit. This reconstruction assesses the effect of textual feedback on candidate judgments; it does not verify the feedback against the waveform. Answers are used afterward to flag erroneous candidate elimination.
Group-Adaptive Process Credit Calibration
We deterministically flag a trajectory as misleading if its final answer is incorrect and at least one evaluated query with yields an observation that contracts a two-candidate set containing the gold answer to an incorrect singleton. A useful query can receive inaccurate feedback, so assigning the full failure penalty to the trajectory may suppress useful evidence seeking. Penalty attenuation accounts for this uncertainty in attributing failure. The actor may also misuse feedback, or reconstruction may be inaccurate; we therefore retain a negative calibrated reward rather than treating a flagged transition as evidence of a successful interaction.
For each instance , we sample G on-policy trajectories and denote the misleading subset by . Calibration is activated only when the group contains both correct and incorrect trajectories and . Otherwise, terminal rewards remain unchanged.
Within an activated group, let be the fraction of incorrect trajectories flagged as misleading. This fraction determines the weight assigned to the incorrect-answer reward of :
Here is the logistic sigmoid, is the minimum weight, and and control the midpoint and smoothness of its dependence on .
We define a group reference as the mean terminal reward of the active, non-misleading trajectories, capped at zero. The calibrated reward for trajectory is
Here is the original terminal reward. Misleading trajectories receive a reward interpolated between and ; as misleading errors become more prevalent, increases and the calibrated reward moves closer to . Other trajectories retain their terminal rewards. Finally, we standardize the calibrated rewards to obtain advantages for policy learning.
Learning the Interaction Policy
Advantage-weighted policy update. The calibrated advantage from § 4.3 weights all actor-generated response tokens in trajectory . For token with full prefix , the current policy and rollout policy define the probability ratio . The clipped token loss is
Here is the clipping radius.
Verified teacher supervision. We supplement all-incorrect on-policy groups with available answer-verified teacher demonstrations, following ReLIFT (Ma et al., 2026b). These demonstrations provide additional reasoning examples for questions that remain unsolved in the current rollouts. We expand demonstration coverage using targeted audio queries and their observations as hints; generation and supervision details are provided in Appendix § 11.§ 1.
Joint training objective. We combine the policy loss, reference-policy KL regularization, and teacher supervision:
Here collects the token losses, aggregates over non-teacher on-policy trajectories, is the reference-policy KL term, and is the masked teacher cross-entropy. The coefficients and weight the KL and teacher losses, respectively. Separately, difficulty-aware sampling controls which instances are revisited without changing rewards or this objective, as detailed in Appendix § 11.§ 2.
Experiment
We organize our experimental evaluation around four research questions: RQ1: How does Think2Listen perform against end-to-end LALMs and existing audio agents? RQ2: How does policy training shape when and how agents seek additional audio evidence? RQ3: Does learning what to ask help agents acquire useful evidence for better audio reasoning? RQ4: Does transition-level credit assignment improve reliability in agentic audio reasoning?
Experimental Setup
Datasets and Benchmarks. We evaluate Think2Listen on audio understanding and reasoning across three domains: sound, music, and speech. For evaluation, we use two complementary benchmarks: (1) MMAU-test-mini, the test-mini subset of MMAU [33], to assess understanding and reasoning within each audio domain; and (2) MMAR [28], to evaluate multi-step reasoning over sound, music, speech, and their mixtures. For policy training, we construct a dataset of 1,770 examples drawn from MMAU-Pro [20], which covers reasoning across audio domains; MuChoMusic [45], which focuses on music understanding; and MMSU [42], which targets spoken language understanding and reasoning. We select examples to provide balanced coverage of sound, music, and speech.
Method Framework. As shown in Figure 3, Think2Listen assigns roles to three models. The actor policy (), initialized with Qwen3.5-9B [30], uses the task, textual interaction history, and remaining budget to emit ASK() or ANSWER(). The audio model (), Qwen3-Omni-30B-A3B-Instruct [52], re-reads the original recording for each query and returns a textual observation . For transition-as-Credit, the semantic extractor (), Qwen3-6?B [29], reconstructs candidate-state transitions for deterministic reward calibration.
Overall Performance Comparison (RQ1)
Table 1 compares Think2Listen with end-to-end LALMs and audio agents on MMAR and MMAU test-mini. We highlight two observations.
Table 1: Main results on audio understanding and reasoning benchmarks. For each subcategory, bold and underlined scores indicate the best and second-best results, respectively. In each Overall column, circled superscripts ①–⑤ indicate the five highest scores in descending order.
| Method | MMAR | MMAU test-mini | |||||||||||
| S | M | Sp. | S–M | S–Sp. | M–Sp. | S–M–Sp. | Overall | Sound | Music | Speech | Overall | ||
| End-to-End LALMs | |||||||||||||
| Gemini 2.5 Flash | 57.42 | 48.04 | 79.23 | 45.45 | 75.37 | 74.67 | 54.55 | 67.07 ② | 73.27 | 65.57 | 76.58 | 71.80 | |
| Qwen3-Omni | 56.16 | 52.87 | 73.65 | 72.72 | 71.72 | 69.86 | 68.18 | 66.07 ③ | 78.72 | 71.00 | 74.10 | 74.18 ⑤ | |
| GPT-4o | 53.94 | 50.97 | 70.41 | 63.64 | 72.48 | 62.20 | 75.00 | 63.50 | 64.56 | 56.29 | 66.67 | 62.50 | |
| MiMo-Audio | – | – | – | – | – | – | – | 61.70 | 81.68 | 74.25 | 68.17 | 74.70 ② | |
| Gemini 2.0 Flash-Lite | 52.26 | 45.25 | 66.92 | 72.73 | 66.01 | 66.67 | 54.55 | 59.67 | 69.97 | 65.27 | 74.47 | 69.90 | |
| EvA | – | – | – | – | – | – | – | 59.30 | 80.78 | 74.65 | 68.47 | 74.63 ③ | |
| Step-Audio-2-mini | – | – | – | – | – | – | – | 56.80 | 79.30 | 68.44 | 68.16 | 72.73 | |
| Qwen2.5-Omni | 58.79 | 40.78 | 59.86 | 54.55 | 61.93 | 67.07 | 58.33 | 56.70 | 78.10 | 65.90 | 70.60 | 71.50 | |
| Kimi-Audio | 50.32 | 37.99 | 56.15 | 36.36 | 61.58 | 60.00 | 45.45 | 52.60 | 74.17 | 58.38 | 66.07 | 66.20 | |
| Audio Flamingo 3 | 52.26 | 32.40 | 49.62 | 27.27 | 54.19 | 52.00 | 31.82 | 47.18 | 74.47 | 67.37 | 61.26 | 67.70 | |
| MiDashengLM | 43.87 | 40.22 | 51.15 | 18.18 | 45.81 | 57.33 | 36.36 | 46.30 | 69.67 | 58.98 | 62.16 | 63.60 | |
| Audio Reasoner | 43.64 | 33.50 | 32.99 | 45.45 | 42.66 | 31.71 | 25.00 | 36.80 | 67.87 | 69.16 | 66.07 | 67.70 | |
| SALMONN | 30.30 | 31.07 | 34.69 | 9.09 | 34.86 | 35.37 | 41.67 | 33.20 | 41.14 | 37.13 | 26.43 | 34.90 | |
| Audio Flamingo 2 | 24.85 | 17.48 | 20.75 | 18.18 | 26.61 | 23.17 | 8.33 | 21.90 | 71.47 | 70.96 | 44.74 | 62.40 | |
| Audio Agents | |||||||||||||
| AudioToolAgent | 61.21 | 57.28 | 69.05 | 54.55 | 69.27 | 78.05 | 66.67 | 65.90 ④ | 78.38 | 69.46 | 75.08 | 74.30 ④ | |
| AudioRouter | 61.40 | 66.10 | 77.20 | 57.20 | 65.10 | 68.30 | 55.50 | 64.40 ⑤ | – | – | – | – | |
| ACE-Cap | – | – | – | – | – | – | – | 64.10 | – | – | – | 73.00 | |
| AuTAgent | 51.52 | 45.15 | 73.81 | – | – | – | – | 63.00 | 66.97 | 61.80 | 72.97 | 67.20 | |
| AudioGenie-Reasoner | 49.68 | 43.26 | 69.23 | 45.45 | 64.53 | 65.33 | 59.09 | 58.85 | 75.08 | 66.17 | 76.58 | 72.60 | |
| Think2Listen | 63.90 | 55.62 | 77.70 | 60.78 | 74.66 | 69.87 | 69.57 | 69.25 ① | 74.63 | 70.97 | 82.22 | 76.00 ① |
Table 1.
Obs. 1. Think2Listen achieves the best overall performance among the baselines. On MMAR, Think2Listen achieves 69.25 ± 0.28%, surpassing the strongest end-to-end baseline, Gemini 2.5 Flash, at 67.07% by 2.18 pp. On MMAU test-mini, it reaches 76.00 ± 0.18% (mean ± standard deviation across five evaluation rounds), exceeding the corresponding baseline, MiMo-Audio, at 74.70% by 1.30 pp. It also outperforms the strongest audio agent, AudioToolAgent, by 3.35 pp and 1.70 pp, respectively, with larger overall gains on MMAR. These gains extend across multi-step reasoning and broader audio understanding. Appendix § 15 reports query controls, stronger text-actor comparisons, and recent audio-model results.
Obs. 2. Category-level strengths vary across benchmarks. On MMAR, Think2Listen leads in sound, surpassing AudioRouter by 2.50 pp: 61.40% → 63.90%. It also ranks second in speech, sound–speech, and sound–music–speech, scoring 77.70%, 74.66%, and 69.57%, respectively. On MMAU test-mini, its strongest advantage is in speech, where it exceeds the best competing score shared by Gemini 2.5 Flash and AudioGenie-Reasoner by 5.64 pp: 76.58% → 82.22%.
Audio-Seeking Behavior and Feedback Value (RQ2)
We compare natural rollouts before and after training on MMSU using identical initial audio descriptions. A shared-prefix diagnostic further examines whether the acquired feedback helps the policy revise its answer and make better decisions based on the additional acoustic evidence.
Obs. 3. Training broadens audio seeking primarily through single-query interactions. The trained policy more often seeks additional audio evidence before answering. As shown in Figure 4(a), single-query interactions become the dominant pattern, increasing from 31.84% → 53.47%, while no-query interactions decline from 64.49% to 41.22%. This broader use of audio evidence is accompanied by higher inclusive accuracy, 64.08% → 68.98%, with mean additional ASK calls remaining below one. The budget permits four additional ASK calls; the separate trajectory-length penalty is described in Appendix § 12.
Figure 4. Natural audio seeking before and after training. (a) Additional ASK distributions; (b) Outcomes for questions initially answered incorrectly without querying.
Obs. 4. Audio feedback improves decision quality at newly queried states. The shift toward audio seeking also reaches questions that were initially answered incorrectly without any querying. After training, 72.86% of the questions in this cohort receive an ASK, and 38.57% of the full cohort both query and answer correctly, as shown in Figure 4(b). To assess the contribution of the acquired audio feedback, the shared-prefix diagnostic holds the answering actor fixed on newly queried states. Providing the actor with the first query–observation pair raises accuracy from 21.57% → 47.06%, supporting the value of the acquired acoustic evidence for subsequent answer revision. Additional cohort and paired-outcome diagnostics appear in Appendix § 15.§ 3.
Learning What to Ask for Audio Reasoning (RQ3)
We compare No ASK, Generic query, Untrained query, and Learned query on a fixed dataset to assess whether learning what to ask provides benefits beyond generic descriptions and untrained querying. Detailed configurations are provided in the appendix D. These comparisons evaluate complete interaction policies, including query generation and answer reasoning, rather than isolating query learning under a fixed answering actor. The shared-prefix diagnostic in § 5.3 separately holds the answering actor fixed to assess the utility of acquired feedback.
Obs. 5. Generic descriptions benefit reasoning more than perception. Additional descriptions provide useful support for reasoning: Generic query raises Pass@1 from 42.35% to 50.78%. Perception, however, remains nearly unchanged, 35.43% → 35.69%. The same contrast holds for Pass@5 in Figure 5(a), suggesting that additional audio access through generic descriptions offers uneven benefits across task groups.
Figure 5. Performance of audio-query strategies on MMSU. (a) Pass@1 and Pass@5 by task group. (b) Mean Pass@1 on eight tasks, with 1–8 questions per task. SYL: syllables; SPK: speaker counting; CSQ: code-switching QA; DLG: dialogue reasoning; PAU: pauses; SBR: stress-based reasoning; PRO: prolonged sounds; BSR: background scenes.
Obs. 6. The learned interaction policy benefits both reasoning and perception. Learned query outperforms both generic and untrained querying, supporting the value of jointly learning evidence acquisition and answer revision. Compared with Untrained query, Pass@1 improves 51.17% → 53.33% for reasoning and 37.93% → 41.29% for perception. Relative to Generic query, the corresponding Pass@1 gains are 2.55 pp and 5.60 pp, respectively, with the larger gain in perception highlighting the benefit of learned queries where generic descriptions provide limited improvement. The advantage persists under repeated sampling, with Pass@5 reaching 78.79% and 70.20%, respectively. Figure 5(b) illustrates where these improvements are most pronounced within the evaluated subset.
Training Diagnostics of Transition-as-Credit (RQ4)
We assess candidate-state reconstruction through answer agreement and human audit, then compare Transition-as-Credit (TaC) with a historical run without TaC on training accuracy, misleading transitions, and query usage. Detailed experimental settings are provided in the appendix E.
| Metric | TaC | w/o TaC |
| Parsed ASK | 20,095/20,341 | 22,247/22,352 |
| Parse coverage | 98.79% | 99.53% |
| Singleton turns | 9,273 | 9,985 |
| Agrees with answer | 8,034 | 8,706 |
| Agreement | 86.64% | 87.19% |
| Human audit | 75% | 77% |
caption: Validation of candidate-state reconstruction. Final-answer agreement and human assessment of reconstructed transitions.
:::
Obs. 7. Reconstructed candidate states are broadly consistent with actor choices and human judgments. Table 2 shows near-complete parse coverage and comparable answer agreement across runs: 86.64% for TaC and 87.19% for w/o TaC. Human reviewers accept both candidate sets in 75% and 77% of samples, respectively. These results indicate that reconstructed candidate sets are generally consistent with the actor’s choices and the available interaction evidence.
Obs. 8. Higher training accuracy accompanies a lower prevalence of misleading transitions. A misleading trajectory contains an ASK that contracts a two-candidate state containing the gold answer into an incorrect singleton and ultimately ends with an incorrect answer. Figure 6 shows that TaC generally achieves higher final-answer accuracy and lower misleading rates than w/o TaC. From rollouts 0–34 to 35–69, TaC’s misleading rate among fully terminal errors declines 16.85% → 13.47%, while overall trajectory accuracy increases 48.52% → 50.99%. The accuracy advantage also holds over the full run, with 50.81% for TaC versus 50.04% for w/o TaC, an absolute improvement of 0.77 pp, alongside fewer ASK calls on average.
Figure 6. Training accuracy and misleading candidate transitions. (a) Accuracy over all trajectories. (b) Misleading trajectories as a percentage of fully terminal errors.
Conclusion
In this paper, We presented Think2Listen, an evidence-centric learning framework for agentic audio understanding guided by evidence before commitment. It enables an interactive reasoning paradigm in which an agent actively seeks acoustic evidence through targeted LALM queries, revises competing hypotheses as new evidence emerges, and decides when to commit to an answer supported by acoustic evidence. To learn this interaction, we utilize observation-induced candidate-state changes to calibrate trajectory rewards, complementing terminal correctness with feedback on intermediate evidence use. Evaluations on two benchmarks spanning seven audio categories show gains in multi-step audio reasoning and audio understanding across speech, sound, and music. These results support jointly learning evidence acquisition and candidate revision. Learning what to ask, how to use evidence, and when to stop helps connect acoustic perception with evidence-grounded decisions.
AI Use Statement
We used AI tools to assist with writing and to polish the language of this paper. The authors remain fully responsible for the content, experimental results, analyses, and claims presented in this paper.
Ethics Statement
Think2Listen uses existing audio datasets and pretrained models to study audio understanding and reasoning. Model-generated observations and candidate-state reconstructions may contain errors and reflect biases in the underlying data and models. Applications involving personal audio should respect consent and privacy and include safeguards against unsupported inferences about individuals.
Reproducibility Statement
Data sources and model roles are described in § 5.1. Sources of baseline results, teacher supervision and difficulty-aware sampling, training and decoding settings, audio-query configurations, reconstruction-validation protocols and metric definitions, prompts, and qualitative interaction traces are documented in Appendices § 10–§ 17.
Baseline Models and Sources of Reported Results
We select a diverse set of representative baselines for Table 1, spanning end-to-end LALMs and audio agents. The LALMs range from proprietary multimodal systems, including Gemini 2.5 Flash [7], GPT-4o [17], and Gemini 2.0 Flash-Lite [13], to open models for audio understanding and reasoning, including Qwen3-Omni [52], MiMo-Audio [58], EvA [48], Step-Audio-2-mini [47], Qwen2.5-Omni [51], Kimi-Audio [8], Audio Flamingo 3 [12], MiDashengLM [9], Audio Reasoner [64], SALMONN [39], and Audio Flamingo 2 [11]. We further include audio agents that span tool-based interaction, multi-agent evidence refinement, and learned tool use: AudioToolAgent Wijngaard et al. [46], AudioRouter Chen et al. [2], ACE-Cap Ma et al. [25], AuTAgent Tong et al. [41], and AudioGenie-Reasoner Rong et al. [31].
For MMAU, we report results on the test-mini split.
Additional Methodological Details
Verified Teacher Supervision
Following ReLIFT Ma et al. (2026b), we supplement unsuccessful policy groups with verified demonstrations. We expand teacher coverage through targeted audio queries, drawing on privileged conditioning in OPSD Zhao et al. [61] and audio-based supervision in AudioRubrics Yu et al. [56].
We first collect answer-verified teacher trajectories, retaining one per covered instance. For a scheduled instance absent from this bank, a planner generates a hint query and obtains its audio observation:
Here P is the planner and summarizes failed attempts. Queries requesting answer labels or reproducing candidate text are rejected. A fresh actor continues from the original run augmented with this query–observation pair, without gold, planner reasoning, or failed rollout history. We retain one answer-correct continuation per newly covered instance.
During training, teacher supervision is activated by
Here G is the rollout group size, is the binary correctness reward, and is an indicator. The indicators and require no removed trajectories and a usable bank entry, respectively. An activated group replaces its longest trajectory with the demonstration, preserving the original rewards and advantages. Teacher rows contribute the auxiliary loss
Here is the total batch size, indexes teacher rows, and and denote a target token and its full prefix. The binary mask selects supervised tokens. For rescued trajectories, only actor-generated continuation actions are supervised; hints, observations, and prompts remain context without gradients. Policy-gradient and reference-KL losses apply only to on-policy rows in Eq. (8).
Difficulty-Aware Sampling
Under binary rewards, uniformly correct or incorrect groups have zero reward variance and provide no relative policy-gradient signal in GRPO Shao et al. [35]. We therefore use fixed stratified sampling to prioritize potentially informative questions while preserving exposure to harder and unobserved instances.
Historical performance informs difficulty-aware selection Kong et al. [18]; ); Zheng et al. [63]. We use each observed instance’s empirical success rate:
Here counts all attempts for instance within a fixed historical window, including invalid attempts, and counts those with positive base rewards. We aggregate repeated occurrences and define two preferential sampling pools:
Here contains hard instances with nonzero success, and contains medium-difficulty instances, with thresholds and . We combine these groups with uniform sampling throughout the training set :
Here is an indicator, denotes set size, and are source weights summing to one. The uniform source retains access to easy, unseen, and historically unsolved instances; source weights thus differ from realized difficulty proportions.
We realize the mixture through fixed offline quotas, batch-level instance uniqueness, and a checked repetition cap within the saved sequence. Unlike DAPO’s online filtering [57] and CDAS’s adaptive difficulty updates [18], our schedule is constructed before audio rollouts and shared unchanged across configurations. This avoids additional generation solely for batch refilling and holds instance selection and ordering fixed, supporting controlled comparisons of method variants and hyperparameter settings.
| Configuration | Acc. (%) |
| Baseline GRPO | 66.0 |
| Difficulty-aware sampling | 66.9 |
| Verified teacher supervision | 68.3 |
| Transition-as-Credit | 68.7 |
| Think2Listen | 71.2 |
Table 7. (a) MMSU test accuracy.
Figure 7b. (b) Teacher-trajectory coverage.
Ablation Results and Teacher-Trajectory Coverage
Figure 7 reports ablation results on the MMSU test set. Transition-as-Credit reaches 68.7% accuracy, compared with 66.0% for baseline GRPO, while the full Think2Listen system achieves an overall accuracy of 71.2%. To examine verified teacher supervision (§ 4.4), we compare the initial set of answer-verified teacher trajectories with the expanded set obtained through targeted audio queries. The expanded set maintains higher smoothed coverage of all-negative rollout groups.

Figure 7. Ablation results and teacher-trajectory coverage. (a) MMSU test accuracy. The last row reports the full system’s overall accuracy. (b) Availability of verified teacher trajectories for rollout groups whose trajectories all receive negative terminal rewards.
Training and Evaluation Configurations
We initialize the actor and the reference policy from Qwen3.5-9B and train on two NVIDIA B200 GPUs. Table 3 summarizes the optimization, interaction, and decoding parameters.
| Parameter | Value |
| Training epochs | 5 |
| Prompts per rollout batch | 32 |
| Trajectories per prompt, | 8 |
| Global batch size | 256 |
| Advantage estimator | GRPO |
| Optimizer | Adam |
| Learning rate / schedule | / constant |
| Adam moments / weight decay | (0.9, 0.98) / 0 |
| Clipping radius, | 0.2 |
| KL coefficient, | 0.001 |
| KL estimator | low_var_kl |
| Entropy coefficient | 0 |
| Dynamic batching budget per GPU | 8,192 tokens |
| Maximum additional ASK calls, | 4 |
| Checkpoint and evaluation interval | 20 rollout batches |
| Actor sampling temperature during training | 1.0 |
| Actor sampling temperature / top- at evaluation | 0.7 / 0.95 |
Table 3. Training and evaluation parameters.
Evaluation metric and repeated sampling. Our evaluations for Table 1 estimate single-response accuracy with the unbiased Pass@1 estimator. For question , we sample five responses and count correct answers, giving over questions. Repeated sampling estimates the performance of one sampled response per question, and thus does not select the best answer or apply majority voting. Think2Listen achieves on MMAR and on MMAU test-mini. These values are the mean and standard deviation of the overall accuracies across five evaluation rounds, each using one sampled response per question. The mean is equivalent to the Pass@1 estimator above. Externally sourced baselines retain the reporting protocols of their sources (Appendix § 10); supplementary controls with one sample per question are explicitly identified in Appendix § 15.§ 1.
Note on MMSU data separation. We divide MMSU into disjoint training, validation, and test partitions before constructing the mixed training set. Its 755 MMSU examples are selected exclusively from the training partition. The analyses in § 5.3 and § 5.4 and Appendix § 11.§ 3 draw on the same held-out test partition of 490 questions; RQ3 uses selected questions from this partition. The stored training and test sample lists share neither sample IDs nor audio filenames. Validation data, rather than test results, determine the checkpoint choices in Figure 7.
Inspired by the overlong reward shaping in DAPO ([57]), we use total trajectory length to determine a length penalty: within the 4,096–8,192-token interval, each additional 1,024 tokens beyond 4,096 contributes a further to the reward. Trajectories exceeding 8,192 tokens are excluded from reward computation and policy updates. At evaluation time, temperature 0.7 controls sampling sharpness, and top- 0.95 restricts sampling to the smallest set of most probable tokens covering at least 95% of the probability mass.
For Transition-as-Credit, sets the minimum weight on the incorrect-answer reward of in Eq. (5); places the sigmoid midpoint where half of the group’s incorrect trajectories are misleading, and controls the smoothness of this transition. The semantic extractor with temperature 0 and top- 0.95 for greedy decoding, with an output limit of 8,192 tokens. For verified teacher supervision, scales the masked teacher cross-entropy in Eq. (8).
Audio-Query Configurations for RQ3
Evaluation protocol. We evaluate four configurations on selected MMSU questions, with ten runs per question per configuration, yielding 6,680 evaluation runs in total.
Query configurations. No ASK uses the base actor to answer from the initial audio description without additional audio calls. Generic query selects a predefined generic query template according to the audio type associated with each question, as shown in Table 4. Untrained query allows the base actor to formulate queries from the task and interaction history, with up to four additional audio calls. Learned query uses the trained actor for both query generation and final answering, conditioned on the evolving interaction history.
| Modality | Prompt |
| Speech | Describe the audible speech content, speaker count, and relevant background scene. State only facts that can be heard. |
| Music | Describe the audible musical sources and texture: instruments, voices, timbre, accompaniment, and production effects. |
| Sound | Identify the audible sound sources, events, and background ambience. Distinguish foreground from background when possible. |
| Mixed | Inventory the audible speech, music, and non-speech sound sources, separating foreground and background. |
Table 4. Modality-specific prompts for generic audio queries.
Reconstruction Validation for RQ4
Evaluation protocol. We compare Transition-as-Credit (TaC) with a historical run without TaC (w/o TaC), using the same prompt schedule and 25,600 trajectories per run over 100 training rollouts. Both runs use the same setup and training settings, with TaC disabled in the historical comparison run. We assess candidate-state reconstruction through answer agreement and human audit, and examine training accuracy, misleading trajectories, and query usage.
Observed calibration frequency. In the available training audit, 17.47% of eight-trajectory prompt groups contain at least one rescaled trajectory, and 5.26% of trajectory records are rescaled. A record is counted as rescaled when its logged calibration scale differs from one. These are empirical audit frequencies; the activation rule remains that a group contains both correct and incorrect trajectories and at least one trajectory flagged as misleading (§ 4.3).
Candidate-state reconstruction. We use the semantic extractor defined in § 4.2 and Eq. (4). For each evaluated ASK turn, it returns from the task, answer candidates, initial audio description, preceding interaction history, actor reasoning, query, and observation. Here indicates whether the query is specific and decision-relevant; and are the plausible candidate sets before and after the observation; and contains the eliminated candidates. The extractor does not observe the gold answer . Following the same acceptance rule, all output candidate labels must belong to , with , , and . Outputs that cannot be parsed or fail these checks are recorded as unresolved.
Answer agreement. Automatic reconstruction statistics cover ASK turns from trajectories with valid actions but incorrect final answers. Parse coverage is the fraction of these turns with an accepted extraction. Among accepted extractions with , we compare the sole remaining candidate with the actor’s recorded final answer. Agreement is the fraction of these singleton turns that match that answer, with each eligible turn counted separately. The reference is the actor’s final choice, rather than , so this check measures consistency with the actor’s decision rather than task correctness.
Human audit. We manually assess 100 sampled transitions per method. For each sample, reviewers judge whether and are reasonable given the interaction context before and after the observation, respectively. A sample passes only when both candidate sets are judged reasonable; the reported audit rate is the fraction of the 100 samples that pass. This assessment complements answer agreement by checking the plausibility of both reconstructed states, rather than only agreement with a final choice.
Additional Evaluation and Control Experiments
Query Controls on the Main Benchmarks
We add query controls on MMAR and MMAU test-mini questions using the Qwen3.5-9B actor and the same cached initial descriptions. Query-enabled settings use Qwen3-Omni-30B-A3B-Instruct with up to four additional ASK calls. The two trained-actor settings use the same checkpoint. The untrained-query and trained-description-only controls use one sample per question; learned-query results use the five-sample Pass@1 estimator from Table 1.
| Configuration | MMAR | MMAU test-mini |
| Qwen3-Omni (direct answering) | 66.07 | 74.18 |
| Untrained query | 60.00 | 68.82 |
| Initial description only (trained actor) | 62.83 | 73.03 |
| Learned query (trained actor) | 69.25 | 76.00 |
Table 5. Query controls on MMAR and MMAU test-mini. Accuracy (%); query-policy settings share cached initial descriptions. Sampling counts differ as specified in the evaluation protocol.
Learned query exceeds untrained query by 9.25 and 7.18 percentage points on MMAR and MMAU test-mini, respectively. With the trained checkpoint fixed, allowing additional queries improves the reported accuracy by 6.42 and 2.97 points over answering from the initial description alone. These comparisons support the value of learning the interaction policy and acquiring additional evidence, while their sampling counts differ. They address querying behavior on MMAR/MMAU; the TaC-specific ablation remains the MMSU comparison in Appendix § 11.§ 3.
Repeated Answering and Matched Audio-Call Controls
We also test majority voting over Qwen3-Omni’s direct answers. Relative to direct answering, voting yields two fewer correct answers on MMAR and three fewer on MMAU test-mini, offering no accuracy gain in this experiment.
To compare targeted querying with repeated generic descriptions, we evaluate independent audio descriptions followed by untrained-actor answer voting on fixed subsets of 200 questions per dataset. For each question, this control uses the same number of audio calls as its untrained-query trajectory, including the initial description. Table 6 shows that untrained querying exceeds description-based voting by 2.00 points on MMAR and 6.00 on MMAU test-mini under the matched audio-call budget. This matching concerns audio calls, not total inference compute.
| Configuration | MMAR | MMAU test-mini |
| Untrained query | 58.00 | 73.50 |
| Independent descriptions + untrained-actor voting | 56.00 | 67.50 |
Table 6. Matched audio-call comparison. Accuracy (%) on the same fixed 200-question subset of each dataset.
Additional Cohort and Paired-Outcome Diagnostics for RQ2
The cohort in Figure 4(b) comprises questions initially answered incorrectly without querying. On this cohort, the existing untrained-actor rerun achieves 15.71% accuracy, compared with 44.29% for the trained natural policy. The rerun allows at most two repeated label-level queries, whereas the natural trajectories allow up to four ASK calls. This is therefore a descriptive comparison, rather than a control with identical query budgets.
Across the full paired evaluation set, 5.51% of questions change from correct to incorrect after training, while 10.41% change from incorrect or invalid to correct. The net gain is 4.90 percentage points, consistent with the overall accuracy increase from 64.08% to 68.98% in Obs. 3. These paired changes complement the selected-cohort analysis. The shared-prefix diagnostic in Obs. 4 separately fixes the answering actor and measures the contribution of the first query–observation pair.
Failure Recovery with Acoustically Verified Feedback
Protocol. We re-evaluate originally incorrect answers on MMAR and MMAU test-mini to examine the contribution of audio feedback. We regenerate both the initial descriptions and query observations with Gemini 2.5 Pro; human listening confirms that the replacement observations match the audio. We keep the actor, recorded query sequence, and final-answer template fixed, and allow no additional queries. As a control for recovery from re-answering alone, we also ask the same actor to answer once more using the original feedback.
Results and metric definitions. All rates in Table 7 use the corresponding original failure set as the denominator. Replacing feedback recovers 38.71% of MMAR failures and 37.50% of MMAU test-mini failures, whereas re-answering with the original feedback recovers 0.00% and 4.17%, respectively. We report the difference between these recovery rates as the net recovery gain; thus, the MMAU test-mini gain is 33.33 percentage points, distinct from its 37.50% total verified-feedback recovery rate.
| Failure set | Original | Verified | Net gain | Residual |
| MMAR | 0.00 | 38.71 | 38.71 | 61.29 |
| MMAU test-mini | 4.17 | 37.50 | 33.33 | 62.50 |
| Combined | 1.82 | 38.18 | 36.36 | 61.82 |
Table 7 (PDF p. 20). Recovery on the original failure sets. Original and verified columns give recovery rates (%); net gain is their difference in percentage points. Residual is the percentage still incorrect with verified feedback. Combined pools the failure sets.
The pooled net recovery gain is 36.36 percentage points of the original failures without changing the actor or query sequence. The remaining 61.82% persist with acoustically verified feedback, suggesting limitations in reasoning, evidence use, or query coverage. This is a controlled feedback-replacement diagnostic: the net gain is a difference between recovery rates, not a per-example causal assignment of each failure to perception or reasoning. In particular, fixing the query sequence leaves missing evidence outside its coverage unaddressed.
Comparison with Stronger Pretrained Text Actors
Protocol. We evaluate GPT-6.1 Terra and GPT-6.1 Sol at medium, high, and xhigh reasoning effort without task-specific training. All actors use the same Qwen3-Omni-30B-A3B-Instruct audio model, tool interface, prompts, cached initial descriptions, and budget of four additional audio queries. Every result in Table 8, including Think2Listen, uses the same fixed subset of 100 questions from MMAR and 100 questions from MMAU test-mini. These subset results are reported separately from the main benchmark comparison in Table 1.
| Text actor | Reasoning effort | MMAR | MMAU test-mini |
| Think2Listen (trained 9B) | – | 69 | 76 |
| GPT-6.1 Terra | medium | 69 | 76 |
| GPT-6.1 Terra | high | 70 | 77 |
| GPT-6.1 Terra | xhigh | 73 | 76 |
| GPT-6.1 Sol | medium | 73 | 77 |
| GPT-6.1 Sol | high | 75 | 74 |
| GPT-6.1 Sol | xhigh | 76 | 79 |
Table 8. Text-actor comparison with a shared audio interface. Accuracy (%) on fixed 100-question subsets of each benchmark.
Table 8: Text-actor comparison with a shared audio interface. Accuracy (%) on fixed 100-question subsets of each benchmark.
The trained 9B actor matches Terra at medium effort on both subsets and is within one percentage point of Terra at high effort. Sol at xhigh effort exceeds Think2Listen by 7 percentage points on MMAR and 3 on MMAU test-mini. These results place the trained actor’s performance in context under a shared audio interface; the comparison does not equate the actors’ inference compute or establish statistical equivalence.
Sensitivity to a Smaller Semantic Extractor
Replay protocol. We replace the semantic extractor with Qwen3-8B and replay the recorded interactions to assess changes in parse coverage and calibrated advantages. Parse coverage is compared on the same-budget subset; advantage statistics use the full replay against the recovered 27B reference. These are different evaluation scopes and are reported separately in Table 9. This replay evaluates the credit signal without retraining the actor with the smaller extractor.
| Metric | Result |
| Parse coverage on same-budget subset: original → 8B | 98.93% → 87.23% |
| Mean absolute advantage difference: overall | 0.0079 |
| Mean absolute advantage difference: groups with reward differences | 0.0544 |
| Advantage-sign changes | 0 |
Table 9. Smaller-extractor replay. Advantage differences compare the 8B replay with the recovered 27B reference; the restricted mean uses groups with reward differences.
Table 9: Smaller-extractor replay. Advantage differences compare the 8B replay with the recovered 27B reference; the restricted mean uses groups with reward differences.
Although the smaller extractor lowers parse coverage, the evaluated advantage directions remain unchanged. This supports stability of the credit signal in this replay, rather than identical reconstruction or unchanged downstream performance after retraining. Extracted structures remain subject to the validity checks in § 4.2, and eligible calibrated rewards remain negative. The downstream TaC ablation in Appendix § 11.§ 3 separately reports 68.7% MSU test accuracy versus 66.0% for baseline GRPO, evaluated at checkpoints selected using validation performance before training step 160. Together with the 75% human acceptance rate in Table 2, these results document benefits at an operating point with imperfect reconstruction.
Reported Results for Recent Audio Systems
Table 10 supplements the main comparison with recent audio systems. The external scores retain the protocols of their source papers, and each dataset’s source is identified separately. In particular, the MMAU test-mini scores for Step-Audio-R1 and AF-Next-Think come from independent evaluations in SKAD and Audio-R1, respectively.
| Model | MMAR | MMAU mini | Source |
| Think2Listen | 69.25 | 76.00 | Table 1 |
| Step-Audio-R1 | 69.15 | 71.10 | MMAR: MOSS-Audio, Table 1; MMAU: SKAD, tbl:2 |
| SALMONN-2 9B | 64.50 | – | SALMONN-2, tbl:VI |
| SALMONN-2 30B-A3B | 67.60 | – | SALMONN-2, tbl:VI |
| AF-Next Instruct | 59.70 | – | Audio Flamingo Next, Table 1 |
| AF-Next Think | 61.00 | 74.50 | MMAR: Audio Flamingo Next, Table 1; MMAU: Audio-R1, Table 1 |
| AF-Next Captioner | 63.00 | – | Audio Flamingo Next, Table 1 |
Table 10. Published results for recent audio systems. Accuracy (%); source links identify the paper and table for each score. AF-Next denotes Audio Flamingo Next.
Think2Listen’s reported MMAR accuracy is numerically close to Step-Audio-R1’s 69.15%, and its MMAU test-mini score exceeds the two external test-mini results listed here. Differences in evaluation protocols preclude interpreting these margins as controlled comparisons or evidence of statistical superiority.
Prompts
We provide the prompts for actor action selection, initial audio description, targeted audio observation, and candidate-state reconstruction. Each box separates the system message from the user message or template. Braced placeholders are populated at runtime; the original audio accompanies the two audio-model prompts. The candidate-state extractor receives only actor-visible information and has no access to the gold answer.
Actor action selection
SYSTEM
You are an active audio-perception agent.
You cannot hear the audio. You receive a task question, its candidate answers, an
initial audio description, and any prior ASK plus Omni observation pairs.
On every turn, output exactly one action and nothing else:
<ask>one targeted question about audible evidence</ask>
or
<answer>X</answer>
For <answer>, replace X with exactly one listed answer label.
For <ask>, request factual evidence that can be heard in the original audio. Do not
ask Omni to choose an answer, name an answer label, enumerate candidate answers, or
explain which candidate is correct. Do not output reasoning, Markdown, or text
outside the action tag.
Ask only when another audible fact would materially improve the decision. When the
available evidence is sufficient, output <answer> instead of another ASK.
USER TEMPLATE
[Task Question]
{question}
[Candidate Answers]
{choices}
[Initial Omni Description]
{initial_description}
[Prior Qwen ASK and Omni Observation Pairs]
{interaction_history}
Choose the next action now.
Initial audio description
SYSTEM
You are an audio analysis assistant.
Your task is to answer user questions strictly based on the factual content of the
provided audio clips. Carefully listen to and analyze the audio before answering.
Audio reference notation:
- A bare reference like <audio> refers to the full duration of that audio clip.
- A reference with [start:end] denotes a time slice of that clip (e.g.
<audio[0.0s:3s]> means the segment from 0.0s to 3.0s).
- An omitted start means the beginning of the clip (e.g. <audio[:2.0s]> means from
the beginning up to 2.0s).
- An omitted end means the end of the clip (e.g. <audio[1.5s:]> means from
the end).
- A negative value counts backward from the end (e.g. <audio[-2.5s:]> means the
last 2.5 seconds of that clip).
- The slicing semantics follow Python-style [start:end] conventions.
Guidelines:
- Base your answers only on information that is clearly present or can be directly
inferred from the audio.
Initial audio description CONTINUED
Do NOT make up details that are not supported by the audio. Do not hallucinate sounds or speech that you did not hear.
When describing audio content, always use the <audio[start:end]> notation to indicate when each element occurs in the audio.
Be specific and factual about: speech content, speaker's emotional tone, speaking pace, voice characteristics, background sounds, acoustic environment.
If the audio is noisy, ambiguous, or unclear, acknowledge the uncertainty rather than guessing.
Your goal is to provide reliable, evidence-based answers grounded in the audio content only.
USER PROMPT
Listen to the provided audio <audio> carefully and create a chronological timeline of what you hear.
MANDATORY SPEECH LABELING:
EVERY speech segment MUST be labeled with <speech N> (N = 0, 1, 2, ... in chronological order)
EVERY speaker MUST be labeled with <speaker N> (use the same N for the same speaker throughout)
You MUST transcribe the spoken words as accurately as possible inside each speech segment
If the audio contains ANY human voice, you MUST use <speech> type and provide transcription
Do NOT describe speech as [sound_effect], [noise], or any other type. Speech = [speech] with <speech N> <speaker N> tags.
Even brief utterances (hi, oh, hmm) must be tagged as <speech N>
For each distinct audio element you detect, provide:
Time range using <audio[start:end]> notation
Type: [speech, music, sound_effect, silence, noise, other]
For speech: <speech N> <speaker N> followed by the exact transcription in quotes For non-speech: brief factual description
Output format: <audio[X.Xs:Y.Ys]> | [speech] | <speech 0> <speaker 0> "transcribed words here" <audio[Y.Ys:Z.Zs]> | [music] | description of music <audio[Z.Zs:W.Ws]> | [speech] | <speech 1> <speaker 1> "transcribed words here" <audio[W.Ws:V.Vs]> | [sound_effect] | description of sound ...
After the timeline, provide a scene summary:
[Scene Summary]
<Summarize the overall scene: what is happening, where, who is involved, and the general atmosphere.>
List each <speaker N> and describe their voice characteristics and, if identifiable, their likely role (e.g., "<speaker 0>: female, concerned tone, appears to be the mother; <speaker 1>: male, casual tone, appears to be a friend").
Base this only on what you actually heard.
Rules:
Only describe sounds, speech, or music you are CERTAIN you heard
Do NOT guess or invent any sounds you did not clearly hear
If the audio is unclear or noisy, say so
Use precise timestamps based on what you actually hear
If uncertain about a boundary, use ~ (e.g., <audio[~5.0s:~8.0s]>)
Targeted audio observation
SYSTEM
You are an audio question answering model. Use the audio clip to answer the question.
USER CONTENT
audio_url: file://{audio_path}
text: {actor_ask_query}
Candidate-state extractor
SYSTEM
You extract actor-visible candidate-state transitions. Return JSON only.
USER TEMPLATE
Reconstruct the candidate-answer state transition caused by one ASK in an audio reasoning trajectory. Use only the actor-visible question, options, audio description, reasoning, ASK, and Omni observation. Do not infer a hidden correct answer.
[Task Question]
{question}
[Candidate Answers]
{choices}
[Initial Audio Description]
{initial_description}
[Prior ASK and Actor-Visible Observation History]
{prior_history}
[Actor Reasoning Before Current ASK]
{actor_reasoning}
[Current ASK]
{current_ask}
[Actual Actor-Visible Omni Observation]
{current_observation}
Identify the options that were genuinely plausible before this ASK and those still plausible after the observation. An ASK is specific only when it targets a concrete, decision-relevant uncertainty that the observation could resolve. List only options whose status is supported by the visible trajectory. Eliminated distractors must be plausible before the ASK and no longer plausible after it.
Return JSON only:
{
"ask_is_specific": "<yes|no>",
"plausible_before": ["<answer labels>"],
"plausible_after": ["<answer labels>"],
"eliminated_distractors": ["<answer labels>"],
"reason": "<one or two sentence evidence-based explanation>"
}
Qualitative Examples of Targeted Audio Interaction
These recorded trajectories illustrate how targeted questions inform final answers. Observations are model reports; [...] marks omitted source text. Case titles are editorial.
Case 1. Rhythm and timbre refine genre identification — 1 ASK
TASK QUESTION
Which musical genre would this bass piece fit into?
CANDIDATE ANSWERS
A. Blues B. Chamber music C. Funk D. Solo performance on bass
INITIAL AUDIO DESCRIPTION
<audio[0.0s:10.0s]> | [music] | Acoustic guitar playing a rhythmic, percussive melody.
[Scene Summary]
The audio contains only instrumental music. There is no speech, sound effects, or background noise. The scene is a solo acoustic guitar performance.
ACTOR / REASONING BEFORE ASK
The initial description mentions acoustic guitar with rhythmic, percussive melody, but doesn’t describe bass elements specifically. I need to ask about what I can hear regarding bass characteristics like rhythm, texture, instrumentation, or musical style cues.
ACTOR / ASK 1
<ask>What specific rhythmic patterns, bass line characteristics, or instrumental textures are audible in the piece?</ask>
AUDIO MODEL / OBSERVATION 1
The piece features a prominent, groovy, and syncopated bass line played on an electric bass guitar. The rhythm is built on a classic "slap" technique, where the player uses the thumb to strike the lower strings for a percussive "thump" and uses the fingers to pull the higher strings for a sharp "pop." This creates a distinct, funky, and driving rhythmic texture that is both melodic and percussive.
ACTOR / CONCLUSION
The Omni Observation clearly describes a prominent electric bass line with syncopated, "slap" technique creating a funky, driving rhythmic texture. This aligns strongly with Funk music characteristics.
FINAL ACTION GROUND-TRUTH ANSWER
<answer>C</answer> C. Funk
Case 2. Targeted listening reveals a background sound 1 ASK
TASK QUESTION
What non-musical sound can be heard in the recording?
CANDIDATE ANSWERS
A. Clicking sounds
B. String scrapes
C. Footsteps in the background
D. Amateur recording quality
INITIAL AUDIO DESCRIPTION (EXCERPT)
<audio[0.0s:10.0s]> | [music] | Soft, ambient music with a gentle, ethereal quality, featuring a female vocalist singing in a high, melodic tone.
[...]`
There are no other discernible sounds or speakers.
ACTOR / REASONING BEFORE ASK
The initial description mentions soft ambient music with a female vocalist and a speech fragment. I need to ask about specific non-musical sounds to distinguish between the options provided.
ACTOR / ASK 1
<ask>Ask about specific non-musical sounds or ambient noises that occur throughout the recording to identify what is other than the music and vocal performance.</ask>
AUDIO MODEL / OBSERVATION 1
The most prominent non-musical sound is a rhythmic, mechanical clicking or tapping noise that occurs consistently throughout the entire recording, underneath the music and vocals.
ACTOR / CONCLUSION
The Omni Observation clearly states there is a rhythmic, mechanical clicking or tapping noise throughout the recording, which corresponds to candidate A.
FINAL ACTION <answer>A</answer>
GROUND-TRUTH ANSWER A. Clicking sounds
Case 3. Deriving a disjunctive conclusion
2 ASKs
TASK QUESTION
What would be the most appropriate conclusion based on the given context?
CANDIDATE ANSWERS
A. I don’t save money on gas or I take a taxi B. I save money on gas or I don’t take a taxi C. I choose to take public transport and end up paying more for the ride D. I take a taxi and save money on gas
INITIAL AUDIO DESCRIPTION (EXCERPT)
<audio[0.0s:18.8s]> | [speech] | <speech 0> <speaker 0> "If I choose to take a public transport, I would be able to save money on gas. On the other hand, if I decide to take a taxi, I will end up paying more for the ride. It is known that at least one of the following statements is true. Either I opt for a public transport or I won’t have to pay more for the ride. However, it is uncertain which one is true. It could be the case that only the first statement is true, or only the second statement is true, or perhaps both statements are true."
ACTOR / REASONING BEFORE ASK 1
The speaker presents a logical puzzle about transportation choices. Let me analyze what is stated:
If public transport save money on gas
If taxi pay more for ride
At least one of these is true: (a) take public transport OR (b) won’t pay more
The scenario seems to be about logical consistency around choices and costs.
ACTOR / ASK 1
<ask>What specific logical constraints does the speaker state must hold true?</ask>
AUDIO MODEL / OBSERVATION 1
Based on the audio, the speaker states the following specific logical constraints:
At least one statement is true: "It is known that at least one of the following statements is true."
The statements are:
Statement 1: "Either I opt for a public transport or I won’t have to pay more for the ride."
Statement 2: "I will end up paying more for the ride."
The truth of the statements is uncertain: "However, it is uncertain which one is true. It could be the case that only the first statement is true, or only the second statement is true, or perhaps both statements are true."
In summary, the speaker states that the disjunction (logical OR) of the two statements must be true, but they do not specify which one(s) are true. Case 3. Deriving a disjunctive conclusion 2 ASKS / CONTINUED
ACTOR / REASONING BEFORE ASK 2
Let me formalize this: Let P = take public transport Let S = save money on gas Let T = take taxi Let M = pay more for ride From the audio:
If P then S (public transport saves gas)
If T then M (taxi means pay more)
At least one true: (P OR not M)
Uncertain which statement is true
The question asks what conclusion follows. Let me think about what logically follows from these premises.
If we know (P OR not M) is true, and we need to find what must follow...
Looking at the answer choices, they’re about combinations of choices and costs.
ACTOR / ASK 2
<ask>Does the audio indicate what statement(s) are actually true?</ask>
AUDIO MODEL / OBSERVATION 2
No, the audio does not indicate which statement(s) are actually true. It explicitly states that it is uncertain which one is true and that it could be the case that only the first statement is true, only the second statement is true, or both statements are true.
ACTOR / CONCLUSION
So the constraint is: (P OR not M) is TRUE.
From the conditional statements:
P → S (public transport → save gas)
T → M (taxi → pay more)
Given (P OR not M), what follows?
If P is true, then S is true (save money on gas). If not M is true, then either P or not T (since T → M, contrapositive gives not M → not T).
Looking at answer B: "I save money on gas OR I don’t take a taxi"
This would be S OR not T.
If we have (P OR not M):
Case 1: P is true → S is true → (S OR not T) is true
Case 2: not M is true → From T → M, if not M then not T → (S OR not T) is true
Either way, (S OR not T) must be true.
So answer B follows logically.
FINAL ACTION GROUND-TRUTH ANSWER
<answer>B</answer> B. I save money on gas or I don’t take a taxi
Case 4. Prosodic evidence supports sarcasm detection 1 ASK
TASK QUESTION
Does the speaker express sarcasm or irony in the audio?
CANDIDATE ANSWERS
A. true D. false
INITIAL AUDIO DESCRIPTION (EXCERPT)
<audio[5.8s:8.3s]> | [speech] | <speech 2> <speaker 2> "Are we going to the drive-thru at Jack in the Box?" <audio[8.3s:9.5s]> | [noise] | audience laughter
[...]
<speaker 2>: male, sarcastic and humorous tone, likely a character in a skit.
ACTOR / REASONING BEFORE ASK
The scene summary notes that speaker 2 is described as having a "sarcastic and humorous tone." I should ask to confirm what specific evidence of tone or delivery supports this characterization.
ACTOR / ASK 1
<ask>What specific auditory evidence supports the characterization of speaker 2 as having a sarcastic tone?</ask>
AUDIO MODEL / OBSERVATION 1
The specific auditory evidence supporting the characterization of speaker 2 as having a sarcastic tone is his exaggerated, mocking tone of voice.
Instead of a neutral or genuinely inquisitive tone, he delivers the line, "Are you going to the drive-through at Jack in the Box?" with a high-pitched, incredulous inflection. This delivery, combined with the immediate audience laughter that follows, indicates he is using the question rhetorically to express his disbelief and mock the idea that someone would spend an entire dinner discussing their research.
ACTOR / CONCLUSION
The Omni Observation confirms that speaker 2 uses an exaggerated, mocking tone with incredulous inflection, suggesting irreverence or mockery. The audience laughter reinforces the humorous intent. This points toward sarcasm/irony in the delivery.
FINAL ACTION GROUND-TRUTH ANSWER
<answer>A</answer> A. true
Case 5. Vocal artifacts inform speech authenticity 1 ASK
TASK QUESTION
Is the audio spoken by a real person or synthesized (fake)?
CANDIDATE ANSWERS
A. fake B. real
INITIAL AUDIO DESCRIPTION
<audio[0.0s:2.1s]> | [speech] | <speech 0> <speaker 0> "Is there something in particular that you want to eat?"
[Scene Summary] The audio contains a single speaker asking a question in a calm, conversational tone. The environment is quiet, with no background music or noise.
<speaker 0>: male, calm tone, appears to be asking a question in a casual setting.
ACTOR / REASONING BEFORE ASK
I should ask for specific evidence about vocal characteristics that would indicate whether the speech is real or synthesized.
ACTOR / ASK 1
<ask>What specific vocal qualities, artifacts, or patterns appear in the audio that distinguish between natural human speech and synthesis?</ask>
AUDIO MODEL / OBSERVATION 1 (EXCERPTS)
Monotonous Pitch and Intonation: The pitch of the voice remains extremely flat and unchanging throughout the sentence.
[...]
Lack of Natural Rhythm and Prosody: The rhythm is perfectly regular and robotic.
[...]
Precise, Artificial Articulation: The pronunciation is unnaturally clear and consistent.
[...]
Absence of Breath Sounds and Micro-Phonations: Natural speech includes subtle breaths, lip smacks, and other physiological sounds. This voice is completely devoid of such background
The source observation ends mid-sentence here.
ACTOR / CONCLUSION
The Omni Observation provides clear evidence of artificial characteristics: monotonous pitch, lack of natural rhythm, precise artificial articulation, and absence of breath sounds. These are telling signs of synthesized speech rather than real human speech.
| FINAL ACTION | GROUND-TRUTH ANSWER |
<answer>A</answer> | A. fake |
References
- [1]Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J Hewett, Mojan Javaheripi, Piero Kauffmann, et al. Phi-4 technical report. arXiv preprint arXiv:2412.08905, 2024.arxiv.org/abs/2412.08905
- [2]Liyang Chen, Hongkai Chen, Yujun Cai, Sifan Li, Qingwen Ye, and Yiwei Wang. Audiourouter: Data efficient audio understanding via rl based dual reasoning. arXiv preprint arXiv:2602.10439, 2026a.arxiv.org/abs/2602.10439
- [3]Yanxi Chen, Wenhui Zhu, Xiwen Chen, Zhipeng Wang, Xin Li, Peijie Qiu, Hao Wang, Xuanzhao Dong, Yujian Xiong, Anderson Schneider, et al. Aha: Aligning large audio-language models for reasoning hallucinations via counterfactual hard negatives. In Findings of the Association for Computational Linguistics: ACL 2026, pp. 29294–29306, 2026b.
- [4]Dongjie Cheng, Ruifeng Yuan, Yongqi Li, Runyang You, Wenjie Wang, Liqiang Nie, Lei Zhang, and Wenjie Li. Ar-omni: A unified autoregressive model for any-to-any generation. arXiv preprint arXiv:2601.17761, 2026a.arxiv.org/abs/2601.17761
- [5]Xize Cheng, Dongjie Fu, Chenyuhao Wen, Shannon Yu, Zehan Wang, Shengpeng Ji, Siddhant Arora, Tao Jin, Shinjii Watanabe, and Zhou Zhao. Aha-bench: Benchmarking audio hallucinations in large audio-language models. Advances in Neural Information Processing Systems, 38, 2026b.DOI
- [6]Yunfei Chu, Jin Xu, Qian Yang, Haojie Wei, Xipin Wei, Zhifang Guo, Yichong Leng, Yuanjun Lv, Jinzheng He, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen2-audio technical report, 2024. URL https://arxiv.org/abs/2407.10759.
- [7]Gheorghe Comanici, Eric Bieber, Mike Schaeckermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261, 2025.arxiv.org/abs/2507.06261
- [8]Ding Ding, Zecian Ju, Yichong Leng, Songxiang Liu, Tong Liu, Zeyu Shang, Kai Shen, Wei Song, Xu Tan, Heyi Tang, et al. Kimi-audio technical report. arXiv preprint arXiv:2504.18425, 2025.arxiv.org/abs/2504.18425
- [9]Heinrich Dinkel, Gang Li, Jizhong Liu, Jian Luan, Yadong Niu, Xingwei Sun, Tianzi Wang, Qiyang Xiao, Junbo Zhang, and Jiahao Zhou. Midashe: Efficient audio understanding with general audio captions. arXiv preprint arXiv:2508.03983, 2025.arxiv.org/abs/2508.03983
- [10]Sreyan Ghosh, Sonal Kumar, Ashish Seth, Chandra Kiran Reddy Evuru, Utkarsh Tyagi, S Sakshi, Oriol Nieto, Ramani Duraiswami, and Dinesh Manocha. Gama: A large audio-language model with advanced audio understanding and complex reasoning abilities. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 6288–6313, 2024.arxiv.org/abs/2406.11768
- [11]Sreyan Ghosh, Zhifeng Kong, Sonal Kumar, S Sakshi, Jaehyeon Kim, Wei Ping, Rafael Valle, Dinesh Manocha, and Bryan Catanzaro. Audio flamingo 2: An audio-language model with long-audio understanding and expert reasoning abilities. arXiv preprint arXiv:2503.03983, 2025.arxiv.org/abs/2503.03983
- [12]Sreyan Ghosh, Arushi Goel, Kaousheik Jayakumar, Lasha Koroshinadze, Nishit Anand, Zhifeng Kong, Siddharth Gururani, Sang-gil Lee, Jaehyeon Kim, Aya Aljafari, et al. Audio flamingo next: Next-generation open audio-language models for speech, sound, and music. arXiv preprint arXiv:2604.10905, 2026.arxiv.org/abs/2604.10905
- [13]Google. Gemini 2.0 flash model card, 2025. URL https://storage.googleapis.com/model-cards/documents/gemini-2-flash.pdf.
- [14]Dieter Hartmann and Sabine Zerbian. Rhoticity in black south african english–a sociolinguistic study. Southern African linguistics and applied language studies, 27(2):135–148, 2010.DOI
- [15]Xiang He, Chenxing Li, Jinting Wang, Yan Rong, Tianxin Xie, Wenfu Wang, Li Liu, and Dong Yu. Audio-deepthinker: Progressive reasoning-aware reinforcement learning for high-quality chain-of-thought emergence in audio language models. arXiv preprint arXiv:2604.18187, 2026.arxiv.org/abs/2604.18187
- [16]Rongjie Huang, Mingze Li, Dongchao Yang, Jiatong Shi, Xuankai Chang, Zhenhui Ye, Yuning Wu, Zhiqiang Hong, Jiawei Huang, Jinglin Liu, et al. Audiogpt: Understanding and generating speech, music, sound, and talking head. In Proceedings of the AAAI conference on artificial intelligence, volume 38, pp. 23802–23804, 2024.arxiv.org/abs/2304.12995
- [17]Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024.arxiv.org/abs/2410.21276
- [18]Deyang Kong, Qi Guo, Xiangyu Xi, Wei Wang, Jingang Wang, Xunliang Cai, Shikun Zhang, and Wei Ye. Rethinking the sampling criteria in reinforcement learning for llm reasoning: A competence-difficulty alignment perspective. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pp. 31438–31446, 2026.
- [19]Chun-Yi Kuan, Wei-Ping Huang, and Hung-yi Lee. Understanding sounds, missing the questions: The challenge of object hallucination in large audio-language models. arXiv preprint arXiv:2406.08402, 2024.arxiv.org/abs/2406.08402
- [20]Sonal Kumar, Šimon Sedláček, Vaibhavi Lokegaonkar, Fernando López, Wenyí Yu, Nishit Anand, Hyeonggon Ryu, Lichang Chen, Maxim Plička, Miroslav Hlaváček, et al. MMAU-Pro: A challenging and comprehensive benchmark for holistic evaluation of audio general intelligence. arXiv preprint arXiv:2508.13992, 2025. URL https://arxiv.org/abs/2508.13992.
- [21]Longhao Li, Hongjie Chen, Zehan Li, Qihan Hu, Jian Kang, Jie Li, Lei Xie, and Yongxiang Li. Audio-cogito: Towards deep audio reasoning in large audio language models. arXiv preprint arXiv:2604.12527, 2026.arxiv.org/abs/2604.12527
- [22]Zuwei Long, Yunhang Shen, Chaoyou Fu, Heting Gao, Lijiang Li, Peixian Chen, Mengdan Zhang, Hang Shao, Jian Li, Jinlong Peng, et al. Vita-audio: Fast interleaved cross-modal token generation for efficient large speech-language model. arXiv preprint arXiv:2505.03739, 2025.arxiv.org/abs/2505.03739
- [23]Ke-Han Lu, Zhehuai Chen, Szu-Wei Fu, Chao-Han Huck Yang, Sung-Feng Huang, Chih-Kai Yang, Chee-En Yu, Chun-Wei Chen, Wei-Chih Chen, Chien-yu Huang, et al. Desta2. 5-audio: Toward general-purpose large audio language model with self-generated cross-modal alignment. IEEE Transactions on Audio, Speech and Language Processing, 2026.DOI
- [24]Kaiwen Luo, Zhenhong Zhou, Leyan Wang, Liang Lin, Tianyu Shao, Yuanhe Zhang, Yang Xiao, Yuxuan Li, Miao Yu, Kailin Lyu, et al. A survey of large audio language models: Generalization, trustworthiness, and outlook. arXiv preprint arXiv:2605.20266, 2026.arxiv.org/abs/2605.20266
- [25]Fengji Ma, Yan Rong, Xu Li, Xuenan Xu, Chen Zhang, and Li Liu. Ace-cap: Active evidence acquisition via agentic co-evolution for long-paragraph fine-grained audio captioning. arXiv preprint arXiv:2608.16162, 2026a.arxiv.org/abs/2608.16162
- [27]Ziyang Ma, Zhuo Chen, Yuping Wang, Eng-Siong Chng, and Xie Chen. Audio-cot: Exploring chain-of-thought reasoning in large audio language model. In 2025 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pp. 1–6. IEEE, 2025.arxiv.org/abs/2501.07246
- [28]Ziyang Ma, Yinghao Ma, Yangqiao Zhu, Chen Yang, Yi-Wen Chao, Ruiyang Xu, Wenxi Chen, Yuanzhe Chen, Zhuo Chen, Jian Cong, et al. Mmar: A challenging benchmark for deep reasoning in speech, audio, music, and their mix. Advances in Neural Information Processing Systems, 38, 2026c.arxiv.org/abs/2505.13032
- [29]Qwen Team. Qwen3.6-27B: Flagship-level coding in a 27B dense model, April 2026a. URL https://qwen.ai/blog?id=qwen3.6-27b.
- [30]Qwen Team. Qwen3.5: Towards native multimodal agents, February 2026b. URL https://qwen.ai/blog?id=qwen3.5.
- [31]Yan Rong, Chenxing Li, Dong Yu, and Li Liu. Audiogenie-reasoner: A training-free multi-agent framework for coarse-to-fine audio deep reasoning. In ICASSP 2026-2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 14497–14501. IEEE, 2026a.arxiv.org/abs/2509.16971
- [32]Yan Rong, Jinting Wang, Tianxin Xie, Xiang He, Chenxing Li, Dong Yu, and Li Liu. Beyond Symmetric Interaction: Capability-Aware Asymmetric Multi-Agent Collaboration for Audio Deep Reasoning. In Interspeech 2026, pp. 211–216, 2026b. doi: 10.21437/Interspeech.2026-2273.DOI
- [33]S Sakshi, Utkarsh Tyagi, Sonal Kumar, Ashish Seth, Ramaneswaran Selvakumar, Oriol Nieto, Ramani Duraiswami, Sreyan Ghosh, and Dinesh Manocha. Mmau: A massive multi-task audio understanding and reasoning benchmark, 2024. URL https://arxiv.org/abs/2410.19168.
- [34]Amrith Setlur, Chirag Nagpal, Adam Fisch, Xinyang Geng, Jacob Eisenstein, Rishabh Agarwal, Alekh Agarwal, Jonathan Berant, and Aviral Kumar. Rewarding progress: Scaling automated process verifiers for llm reasoning. In International Conference on Learning Representations, volume 2025, pp. 60808–60838, 2025.arxiv.org/abs/2410.08146
- [35]Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024. URL https://arxiv.org/abs/2402.03300.
- [36]Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflection: Language agents with verbal reinforcement learning. Advances in neural information processing systems, 36:8634–8652, 2023.arxiv.org/abs/2303.11366
- [37]Yirong Sun, Yanjun Chen, Xin Qiu, Gang Zhang, Hongyu Chen, Daokuan Wu, Chengming Li, Min Yang, Dawei Zhu, Wei Zhang, et al. Sonicbench: Dissecting the physical perception bottleneck in large audio language models. arXiv preprint arXiv:2601.11039, 2026.arxiv.org/abs/2601.11039
- [38]Tuende Szalay, Mostafa Shahin, Beena Ahmed, and Kirrie Ballard. Knowledge of accent differences can be used to predict speech recognition. In Proc. Interspeech 2022, pp. 1372–1376, 2022.DOI
- [39]Changli Tang, WenyI Yu, Guangzhi Sun, Xianzhao Chen, Tian Tan, Wei Li, Lu Lu, Zejun Ma, and Chao Zhang. Salmonn: Towards generic hearing abilities for large language models. In International Conference on Learning Representations, volume 2024, pp. 16607–16629, 2024.arxiv.org/abs/2310.13289
- [40]Fei Tian, Xiangyu Tony Zhang, Yuxin Zhang, Haoyang Zhang, Yuxin Li, Dajiao Liu, Yayue Deng, Donghang Wu, Jun Chen, Liang Zhao, et al. Step-audio-r1 technical report. arXiv preprint arXiv:2511.15848, 2025.arxiv.org/abs/2511.15848
- [41]Siqian Tong, Xuan Li, Yiwei Wang, Baolong Bi, Yujun Cai, Shenghua Liu, Yuchen He, and Chengpeng Hao. Autagent: A reinforcement learning framework for tool-augmented audio reasoning. arXiv preprint arXiv:2602.13685, 2026.arxiv.org/abs/2602.13685
- [42]Dingdong Wang, Junnan Li, Jincenzi Wu, Dongchao Yang, Xueyuan Chen, Tianhua Zhang, and Helen Meng. Mmsu: A massive multi-task spoken language understanding and reasoning benchmark. In International Conference on Learning Representations, volume 2026, pp. 31374–31410, 2026a.arxiv.org/abs/2506.04779
- [43]Yucheng Wang, Jing Peng, Hanqi Li, Chenghao Wang, Wenming Tu, Yu Xi, Zhaokai Sun, Kai Yu, and Shuai Wang. Audio-mind: An auditable agentic framework for audio understanding. arXiv preprint arXiv:2605.28480, 2026b.arxiv.org/abs/2605.28480
- [44]Yuwen Wang, Tian-Hao Zhang, Minghao Cai, Yilin Ren, Ziyang Jiang, Xin Wang, Zhichao Wang, Pan Zhou, Kun Zhan, and Xinyuan Qian. Hear, invoke, and understand: A skill-calling multimodal agent for large audio language models. arXiv preprint arXiv:2608.01881, 2026c.arxiv.org/abs/2608.01881
- [45]Benno Weck, Ilaria Manco, Emmanouil Benetos, Elio Quinton, György Fazekas, and Dmitry Bogdanov. MuChoMusic: Evaluating music understanding in multimodal audio-language models. In Proceedings of the 25th International Society for Music Information Retrieval Conference (ISMIR), 2024. URL https://arxiv.org/abs/2408.01337.
- [46]Gijs Wijngaard, Elia Formisano, Michel Dumontier, and Jenia Jitsev. Audiotoolagent: An agentic framework for audio-language models. arXiv preprint arXiv:2510.02995, 2025.arxiv.org/abs/2510.02995
- [47]Boyong Wu, Chao Yan, Chen Hu, Cheng Yi, Chengli Feng, Fei Tian, Feiyu Shen, Gang Yu, Haoyang Zhang, Jingbei Li, et al. Step-audio 2 technical report. arXiv preprint arXiv:2507.16632, 2025.arxiv.org/abs/2507.16632
- [48]Xinyuan Xie, Shunian Chen, Zhiheng Liu, Yuhao Zhang, Zhiqiang Lv, Liyin Liang, and Benyou Wang. Eva: An evidence-first audio understanding paradigm for llams. arXiv preprint arXiv:2603.27667, 2026.arxiv.org/abs/2603.27667
- [49]Zhifei Xie and Changqiao Wu. Mini-omni2: Towards open-source gpt-4o with vision, speech and duplex capabilities. arXiv preprint arXiv:2410.11190, 2024.arxiv.org/abs/2410.11190
- [50]Zhen Xiong, Yujun Cai, Zhecheng Li, Junsong Yuan, and Yiwei Wang. Thinking with sound: Audio chain-of-thought enables multimodal reasoning in large audio-language models. arXiv preprint arXiv:2509.21749, 2025.arxiv.org/abs/2509.21749
- [51]Jin Xu, Zhifang Guo, Jinzheng He, Hangrui Hu, Ting He, Shuai Bai, Keqin Chen, Jialin Wang, Yang Fan, Kai Dang, Bin Zhang, Xiong Wang, Yunfei Chu, and Junyang Lin. Qwen2.5-omni technical report, 2025a. URL https://arxiv.org/abs/2503.20215.
- [52]Jin Xu, Zhifang Guo, Hangrui Hu, Yunfei Chu, Xiong Wang, Jinzheng He, Yuxuan Wang, Xian Shi, Ting He, Xinfa Zhu, et al. Qwen3-omni technical report. arXiv preprint arXiv:2509.17765, 2025b.arxiv.org/abs/2509.17765
- [53]Chih-Kai Yang, Neo S Ho, and Hung-yi Lee. Towards holistic evaluation of large audio-language models: A comprehensive survey. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 10155–10181, 2025.arxiv.org/abs/2505.15957
- [54]Xiaoyu Yang, Xuenan Xu, Wenyi Yu, Siyin Wang, Changli Tang, Terumi Chiba, Siyuan Hou, Ziyang Zhang, Wen Wu, Baoxiang Li, et al. Salmonn-2: Advancing general-purpose hearing abilities with self-supervised representations. arXiv preprint arXiv:2607.17079, 2026.arxiv.org/abs/2607.17079
- [55]Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629, 2022.arxiv.org/abs/2210.03629
- [56]Fangxu Yu, Tao Feng, Dehai Min, Zinan Lin, Weijia Xu, Michael Xu, Philip S. Yu, Ge Liu, and Tianyi Zhou. Reinforcement learning with evolving rubrics as rewards for audio reasoning, 2026a. URL https://arxiv.org/abs/2608.02831.
- [57]Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, et al. Dapo: An open-source llm reinforcement learning system at scale. Advances in Neural Information Processing Systems, 38:113222–113244, 2026b.arxiv.org/abs/2503.14476
- [58]Dong Zhang, Gang Wang, Jinlong Xue, Kai Fang, Liang Zhao, Rui Ma, Shuhui Ren, Shuo Liu, Tao Guo, Weiji Zhuang, et al. Mimo-audio: Audio language models are few-shot learners. arXiv preprint arXiv:2512.23808, 2025a.arxiv.org/abs/2512.23808
- [59]Siyuan Zhang, Jian Zong, Junyu Wang, Peiyuan Jiang, Jiahao Yan, Jingyu Zhang, Tianrui Wang, Xiaobao Wang, Longbiao Wang, and Jianwu Dang. Echo-agent: Evidence chain orchestration agent for audio reasoning. arXiv preprint arXiv:2606.15141, 2026.arxiv.org/abs/2606.15141
- [60]Zhenru Zhang, Chujie Zheng, Yangzhen Wu, Beichen Zhang, Runji Lin, Bowen Yu, Dayiheng Liu, Jingren Zhou, and Junyang Lin. The lessons of developing process reward models in mathematical reasoning. In Findings of the Association for Computational Linguistics: ACL 2025, pp. 10495–10516, 2025b.
- [61]Siyan Zhao, Zhihui Xie, Mengchen Liu, Jing Huang, Guan Pang, Feiyu Chen, and Aditya Grover. Self-distilled reasoner: On-policy self-distillation for large language models, 2026. URL https://arxiv.org/abs/2601.18734.
- [62]Chujie Zheng, Zhenru Zhang, Beichen Zhang, Runji Lin, Keming Lu, Bowen Yu, Dayiheng Liu, Jingren Zhou, and Junyang Lin. Processbench: Identifying process errors in mathematical reasoning. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1009–1024, 2025.arxiv.org/abs/2412.06559
- [63]Haizhong Zheng, Yang Zhou, Brian Bartoldson, Bhavya Kailkhura, Fan Lai, Jiawei Zhao, and Beidi Chen. Act only when it pays: Efficient reinforcement learning for llm reasoning via selective rollouts. Advances in Neural Information Processing Systems, 38:124321–124346, 2026.arxiv.org/abs/2506.02177
- [64]Xie Zhifei, Mingbao Lin, Zihang Liu, Pengcheng Wu, Shuicheng Yan, and Chunyan Miao. Audio-reasoner: Improving reasoning capability in large audio language models. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 23840–23862, 2025.arxiv.org/abs/2503.02318




