WorksComputer Science
One Waveform, Two Stories: Benchmarking Speech-Acoustic Conflict in Audio-Language Models
Abstract
Audio-language models (ALMs) are evaluated on their understanding of both speech and environmental sounds. What remains underexplored is how they respond when spoken claims contradict acoustic evidence within the same recording. We introduce SAC-Bench, a benchmark of intra-audio speech–acoustic conflict comprising 616,031 unique clips from 188 sound-event classes across seven semantic, temporal, and spatial tasks. It pairs environmental recordings with compatible or contradictory synthesized spoken claims and includes audio-only controls where applicable. Across five open-weight ALMs evaluated on semantic and temporal tasks, high audio-only accuracy often coexists with large losses under conflicting claims; Event Identity accuracy is 24.8–46.5 percentage points lower in conflict than compatible conditions. Temporal Order failures persist even when speech follows the complete event sequence, so simultaneous speech–event masking cannot fully explain them. A spatial case study with Spatial-Omni finds lower motion accuracy associated with both spoken claims and the speech source's physical trajectory. Prompt interventions sometimes restore acoustic accuracy, but can also reduce claim-aligned answers without improving grounding or worsen performance. SAC-Bench thus separates acoustic competence, conflict robustness, and intervention effects.
Introduction
Audio-language models (ALMs) can process spoken language alongside non-speech acoustic information, including environmental sounds [24, 30, 10]. Unlike cascaded dialogue systems, whose downstream components largely rely on ASR transcripts [20], ALMs can in principle use acoustic evidence that spoken words alone do not convey. However, making this evidence available to a model does not establish whether it actually contributes to the model’s predictions.
This distinction is difficult to assess when spoken claims agree with the environmental acoustic evidence. If a speaker says that a dog is barking and a bark is audible, a model can answer correctly by following the spoken claim without recognizing the bark itself. The same ambiguity arises when speech accurately describes an event’s timing or a sound source’s location. Consequently, success under aligned conditions cannot distinguish faithful acoustic grounding from reliance on linguistic shortcuts [29]. Figure 1 illustrates how conflicting spoken claims and acoustic evidence are instantiated across the four task groups.

Figure 1. Overview of SAC-Bench. A single audio waveform contains spoken claims and environmental acoustic evidence that may imply conflicting interpretations. SAC-Bench evaluates this conflict across four task groups: event presence, event identity, temporal structure, and spatial configuration.
Conflict-based evaluation provides a direct way to expose such shortcut behavior. Prior work has studied contradictions between visual and textual evidence, audio and video, and audio and externally supplied text, revealing systematic preferences among competing information sources [33, 23, 36, 7, 3]. These settings primarily concern inter-modal conflict, where contradictory evidence is provided through separately represented modalities. Speech-centered audio creates a distinct setting: the same audio input may contain foreground linguistic claims together with non-linguistic evidence about the surrounding events, their temporal structure, and their spatial configuration. We refer to this setting as intra-audio speech–acoustic conflict, where spoken claims and acoustic evidence within the same audio input imply incompatible interpretations of what occurred, when it occurred, or where and how a sound source is situated or moves.
Recent benchmarks have separately investigated several components of this problem. Intra-audio conflict benchmarks primarily study lexical–prosodic inconsistencies, speaker attributes, or semantic background events [6, 11, 29]. Temporal audio benchmarks evaluate temporal ordering, duration, or timestamp grounding [9, 2, 21], while spatial-audio benchmarks test source localization, distance, spatial relations, and motion trajectories [35, 13, 37, 22]. Yet existing intra-audio conflict evaluations primarily focus on semantic or paralinguistic inconsistencies, whereas temporal and spatial benchmarks generally assess acoustic capabilities without contradictory spoken claims. These settings leave underexplored how spoken claims influence grounding across event semantics, temporal structure, and spatial configuration. Spatial grounding additionally requires distinguishing the environmental target’s location or trajectory from both the spoken claim and the speech source’s own spatial cues.
We introduce the Speech–Acoustic Conflict Benchmark (SAC-Bench), with 616,031 unique audio clips across seven tasks spanning semantic, temporal, and spatial grounding. It combines compatible and conflicting spoken claims with single-modality controls where applicable, while matched spatial conditions vary claim content and speech-source geometry. Across five ALMs on semantic and temporal tasks, strong audio-only recognition coexists with substantial speech-aligned errors. Temporal Order failures persist even when speech follows the complete event sequence. A spatial case study further reveals sensitivity to both semantic and physical speech cues. Prompt interventions show that reducing speech following does not necessarily improve acoustic accuracy.
Related Work
Multimodal and Intra-Audio Conflict
Conflicting vision–text and audio–visual inputs reveal systematic preferences among competing evidence sources [33, 23, 5]. In audio–text settings, MCR-BENCH and ALME document reliance on externally supplied text despite contradictory audio [26, 3]. Mechanistic analysis further shows that textual pathways can suppress intact audio representations during late-layer integration [7]. These findings motivate distinguishing available acoustic information from its use in prediction, although mechanisms identified under inter-modal conflict need not transfer directly to intra-audio settings.
Within a single waveform, DEAF evaluates conflicts involving emotional prosody, background scenes, and speaker attributes, separately varying spoken-content conflict and misleading textual prompts [29]. Its background-sound task asks which environment the sounds suggest, providing a direct precedent for environmental speech–acoustic conflict at the scene level. VoxParadox instead targets paralinguistic attributes through contradictions between transcript claims and vocal delivery [16]. Audio Hallucination Attacks introduces AHA-Eval, combining misleading queries with injected speech describing absent events [19]. These studies establish overlapping forms of linguistic interference and motivate examining its variation across more structured environmental judgments.
Audio Grounding Benchmarks
General suites such as AIR-Bench, AudioBench, MMAU, and MMAR assess broad audio understanding and reasoning [32, 25, 18, 14]. Reliability-oriented evaluations examine unsupported responses: AHa-Bench covers semantic, acoustic, and semantic–acoustic confusion, including speaker-attribute contradictions and speech-induced inference of absent sounds [6], while HalluAudio evaluates hallucinations across speech, sounds, and music [34]. Such coverage complements conflict-specific evaluation; task accuracy and hallucination rates alone do not characterize robustness to systematically varied spoken claims.
Temporal benchmarks probe event structure: CompA-order tests occurrence and ordering, TREA evaluates ordering, duration, and counting, and SpotSound-Bench targets fine-grained event localization [9, 2, 21]. SpatialSoundQA/BAT evaluates binaural spatial perception, Sci-Phi describes spatial acoustic scenes, and SO-Bench and AudioMotionBench assess spatial relations or source motion [35, 13, 37, 22]. These tasks characterize temporal and spatial capabilities without systematically crossing them with contradictory spoken descriptions.
SAC-Bench builds on these conflict and capability evaluations through seven environmental-event tasks. Available audio-only controls measure performance without speech; non-overlapping Temporal Order sequences test conflict beyond simultaneous masking. Spatial conditions separately specify the environmental target, spoken claim, and physical speech source, enabling matched comparisons of semantic and physical interference. Table 1 distinguishes event-level conflict coverage from scene-level and paralinguistic conflict; its coverage markers describe evaluation designs, not model performance.
| Benchmark | Grounding tasks | P | I | St | Or | Lc | Sm | Rm | Other conf. | SF ctrl. | 3-src GT | Audio Total | Audio Eval | QA pairs Total | QA pairs Eval |
| CompA (Ghosh et al., [9]) | Order, attributes (600 inst.) | – | – | – | – | – | – | – | ◦ | – | – | 1,300 | 1,300 | n/a | n/a |
| AHa-Bench† (Cheng et al., [6]) | Presence, duration, order, distance | – | – | – | – | – | – | – | ◦ | – | – | 396 | 396 | 906 | 906 |
| HalluAudio (Zhao et al., [34]) | Presence, overlap, order | – | – | – | – | – | – | – | – | – | – | n/r | n/r | 5,720 | 5,720 |
| SO-Bench (Zhu et al., [37]) | Onset, location, distance, motion | – | – | – | – | – | – | – | – | – | – | 7,877 | 7,877 | 7,877 | 7,877 |
| AHA-Eval (Seth et al., [19]) | Absent-event queries | • | – | – | – | – | – | – | – | ◦ | – | n/r | n/r | ∼6.5K | ∼6.5K |
| DEAF† (Xiong et al., [29]) | Background, prosody, speaker | – | – | – | – | – | – | – | • | – | – | 2,756 | 2,756 | n/r | n/r |
| VoxParadox (Pang et al., [16]) | Paralinguistics, speaker | – | – | – | – | – | – | – | • | – | – | 2,000 | 2,000 | 2,000 | 2,000 |
| SAC-Bench (Ours) | Seven event tasks | • | • | • | • | • | • | • | – | • | • | 616,031 | 223,021 | 2,043,836 | 744,086 |
Table 1. Comparison of audio-language benchmarks in speech–acoustic conflict coverage and dataset scale.
Dataset Construction Method
Figure 2 summarizes our data construction pipeline. Each task pairs environmental acoustic evidence () with a spoken claim () that may agree or conflict with it. The evaluated answer concerns the environmental event. For example, a dog-barking recording paired with a spoken claim about barking forms a compatible condition, whereas the same recording paired with a claim about a siren forms a conflicting condition. Temporal State also uses the speech segment as a temporal reference; spatial tasks additionally distinguish the claim from the speech source’s physical geometry (). In figures and relation labels, and denote Legend: • = evaluated; ◦ = partial; – = absent; n/r = not reported; n/a = not applicable. Conflict dimensions: P = presence; I = identity; St = state; Or = order; Lc = location; Sm = spherical motion; Rm = radial motion. Other = non-event conflict; SF = speech-free control; 3-src = independent acoustic, claim, and speaker geometry. Audio counts denote clips or stimuli; SAC-Bench counts each distinct final audio path once across task references; QA counts include diagnostics but exclude prompt variants. CompA reports matching instances; DEAF reports stimuli.

Figure 2. SAC-Bench construction pipeline, from environmental audio selection and spoken claim synthesis to conflict generation, audio rendering, and evaluation.
†DEAF’s BSC task asks which environment the background sounds suggest. Its scene-level speech–acoustic conflict is marked separately from event-label Presence and Identity. AHa-Bench has partial Presence coverage through speech-induced inference of absent sounds.
matches or mismatches in task-relevant attributes, not identity among audio signals, spoken claims, and speech sources.
We curate FSD50K recordings (Fonseca et al., [8]) by removing redundant ancestor labels and retaining recordings with one remaining event label and unanimously positive available PP/PNP ratings. We preserve the original splits and evaluate models only on the exhaustively annotated evaluation split, treating unannotated in-vocabulary events as absent under its annotation protocol. Recordings span six duration bins from 2 to 31 seconds, with at least five clips per class within each split and bin. Claims are generated with GPT-4o or task-specific templates and synthesized with F5-TTS (Hurst et al., [12]; Chen et al., [4]). Synthesizing speech lets us pair the same environmental recording with different claims while controlling speaker identity, timing, level, and spatial geometry across matched conditions. Recording natural speech for all such combinations would be harder to scale and standardize. Selection and synthesis details appear in Appendix § 8.
Semantic Grounding: Event Presence and Identity
Presence asks whether a target event is audible; Identity asks which environmental event is audible. Claims vary in polarity and, where available, explicitness. Explicit claims name the target, implicit claims avoid its label and screened synonyms, and negative Identity claims deny the target without implying silence. Each claim is paired with the target event, no environmental event, or a near/far alternative. CLAP text embeddings [28] define a dissimilar candidate pool, within which near and far alternatives are selected using Lin similarity on the AudioSet ontology graph. Positive claims agree only with the target event; negative claims conflict only when it is present. Speech is center-aligned with the event at speech-to-event RMS ratios of dB. No-event conditions contain speech only; audio-only controls preserve the environmental recording. See Appendices § 8.§ 2 and § 9.§ 1.
Temporal Grounding: Event Timing and Order
Temporal State places an event before or after a spoken timing claim. Speech provides the temporal reference, so this task has no audio-only control. Temporal Order presents two source recordings in their claimed or reversed order, with audio-only controls for both sequences. Speech follows the complete event sequence, avoiding simultaneous speech–event masking; inter-event and speech–event gaps are 0.2 seconds. See Appendices § 8.§ 2 and § 9.§ 2.
Spatial Grounding: Source Location and Motion
Static Location, Spherical Motion, and Radial Motion use 38 spherical Voronoi zones for direction [17]. Five distinct recordings per eligible class and split are reused across matched conditions. The speech-source geometry is undefined for non-spatial speech.
Static Location samples claims from five angular-distance strata relative to : Same, Near, Medium, Far, and Antipodal. For this task, is the acoustic ground-truth zone, is the zone specified by the spoken claim, and is the zone assigned to the physical speech source. Spatialized speech uses five relations among these zones: L1 (all three match), L2 (only and match), L3 (only and match), L4 (only and match), and L5 (all three differ). One L1, one L2, four of each L3–L5, and five non-spatial conditions yield 19 conditions per source.
Spherical Motion follows a constant-radius great-circle arc. Claims describe its actual, reversed, or midpoint-orthogonal trajectory; speech is non-spatial or follows one of these three trajectories, yielding conditions. Radial Motion crosses two acoustic directions (approaching or receding), two claim directions, and three speech geometries (non-spatial, same direction, or opposite direction), yielding conditions. Both motion tasks require recordings of at least 4 seconds.
Environmental events and spatialized speech are rendered in binaural and FOA formats with anechoic propagation, no added synthetic noise, and 0-dB active-RMS SNR. Audio-only controls preserve the rendered environmental event. See Appendices § 8.§ 4 and § 9.§ 3 for sampling and rendering details.
Dataset Statistics
SAC-Bench contains 616,031 unique audio clips (2,069.9 hours), derived from 16,340 environmental recordings spanning 188 sound-event classes (94% of FSD50K). The development and evaluation splits contain 393,010 and 223,021 clips, respectively. Table 2 reports task-level counts and their global union; Figure 3 characterizes event-vocabulary coverage. Each final audio path is counted once. Source recordings, synthesized speech identifiers, and final audio paths are disjoint across splits. Modality breakdowns, source durations, text overlap, and counting details appear in Appendix § 10.
Figure 3. Taxonomic and semantic coverage of sound-event labels in AudioSet, FSD50K, and SAC-Bench. (a) Label distribution across ten ontology categories with fractional assignments. (b) UMAP projection of label-name embeddings.
| Task | Dev | Eval | Total |
| Presence | 160,663 | 81,892 | 242,555 |
| Identity | 160,658 | 81,892 | 242,550 |
| Temporal State | 2,884 | 1,520 | 4,404 |
| Temporal Order | 4,304 | 2,258 | 6,562 |
| Static Location | 36,346 | 28,200 | 64,546 |
| Spherical Motion | 17,620 | 14,638 | 32,258 |
| Radial Motion | 18,930 | 15,764 | 34,694 |
| All tasks (union) | 393,010 | 223,021 | 616,031 |
Table 2. Unique audio clips by task and FSD50K split. The final row reports the deduplicated total across all tasks.
Experimental Setup
Our evaluation tests whether ALMs use environmental evidence when spoken claims conflict with it. Where available, audio-only controls assess acoustic competence; compatible–conflict gaps describe performance across conditions; and speech-following rate (SFR) identifies claim-aligned responses. High audio-only accuracy with low conflict accuracy and high SFR indicates competence in isolation but frequent claim-aligned errors in mixtures. Prompt interventions are judged by both accuracy and SFR, because reduced speech following alone need not improve grounding. Matched spatial conditions distinguish claim content from speech-source geometry.
Evaluated Models and Input Representations
We evaluate five open-weight audio-language models: Qwen3-Omni-30B-A3B-Instruct, Step-Audio-2-mini, Audio Flamingo Next, MOSS-Audio-8B-Instruct, and Spatial-Omni SO30B [30, 27, 10, 31, 37]. All five are evaluated on presence, identity and temporal tasks; Spatial-Omni uses its mono pathway for these tasks. Spatial tasks are evaluated only with Spatial-Omni’s four-channel first-order Ambisonics (FOA) encoder. FOA inputs longer than 20 seconds and binaural inputs are skipped. Checkpoints, preprocessing, and inference settings appear in Appendix § 11.§ 4.
Evaluation Questions and Answer Choices
Presence poses Yes/No questions about the acoustic event and four alternative labels, adding the spoken target when needed and merging duplicates. Identity uses five choices containing the acoustic answer and, for positive conflicts, the speech-asserted identity; absent-event recordings include “No environmental sound is present” as the correct choice. Temporal tasks use binary choices. Spatial primary questions use 38 zones for Static Location, four directed trajectories for Spherical Motion, and two directions for Radial Motion.
Choices and their ordering are fixed within each matched source family across speech conditions, representations, and prompt policies. Separate spatial diagnostics assess the spoken claim, speaker geometry, their relation to the environmental source, and motion endpoints; these diagnostics are excluded from primary acoustic accuracy. Semantic analyses additionally stratify by claim formulation, polarity, and duration. Question and distractor construction appear in Appendix § 11.§ 1.
Prompt Policies and Inference
We compare neutral prompting (no additional instruction), acoustic-grounded prompting (answer from environmental sounds), and conflict-aware prompting (also verify potentially false spoken claims). Temporal State instructions retain the speech segment as the temporal reference without treating its timing claim as evidence. Audio-only controls and diagnostic questions use neutral prompting.
Each zero-shot request contains the recording and a textual question, without demonstrations, transcripts, or reference answers. We use model-native input formats, disable sampling, and generate one response per question without retries or answer selection. Full instructions and inference settings appear in the appendix.
Evaluation Metrics
Completed responses form the scoring population, where unparseable outputs count as incorrect. Inference failures (e.g., skips, missing predictions) are excluded from the denominator.
Non-spatial performance. Identity and temporal tasks use question-level acoustic accuracy. Presence uses recording-level balanced accuracy: Yes and No recalls receive equal weight within each recording, and recordings receive equal weight. Conditions with only one reference class use recording-averaged recall for that class. Appendix § 11.§ 6 gives the formal definitions.
Speech-following rate (SFR). For each conflicting speech–audio question, let be the answer option or options implied by the spoken claim, derived directly from the claim and question. We include a completed question in only when this speech-implied reference is defined and disjoint from the acoustically correct answer set. The speech-implied reference is derived from the claim annotation, not from model predictions on speech-only inputs. For example, negative Identity claims without a unique implied answer are excluded. SFR is the proportion of eligible mixed-input responses that select a speech-implied option:
A higher SFR indicates more claim-aligned responses under conflict, rather than merely more errors. A lower SFR does not by itself imply better acoustic grounding: the model may select another incorrect option or produce an invalid response. We therefore interpret SFR together with acoustic accuracy.
Spatial performance and diagnostics. Spatial primary accuracy averages completed questions within each source recording, then recordings within each event class, and finally classes, preventing sources with more rendered variants from dominating. Random chance baselines are 1/38, 1/4, and 1/2 for Location, Spherical Motion, and Radial Motion. Diagnostic questions targeting spoken claims or speaker geometry are scored separately. Conflict-target attraction measures the change in selecting a conflicting claim or speaker state relative to a matched audio-only control. Formal definitions appear in Appendix § 11.7.
Comparisons and uncertainty. Paired comparisons use completed matched observations with identical answer choices and ordering. We report 95% confidence intervals using source-cluster bootstrap for nonspatial tasks (connected components of shared sources for Temporal Order) and event-class bootstrap for spatial tasks, retaining recordings and their rendered variants within each resampled class.
Results and Analysis
Acoustic Competence and Speech-Aligned Failures
Acoustic competence without speech. Table 3 compares three representative grounding tasks under neutral prompting. Conflict accuracy covers all scored conflict questions, whereas SFR requires a defined speech-implied answer. The columns therefore have different denominators and need not sum to 100%, particularly for Identity. Without spoken claims, Qwen3-Omni achieves 84.6% accuracy on Event Identity and 98.2% on Temporal Order, while Step-Audio-2-mini reaches 88.0% on Identity. These audio-only results establish substantial acoustic competence in several models. Compatible-condition performance alone cannot establish this competence, since the correct spoken claim may already provide the answer.
| Model | Audio-only | Compatible | Conflict | Gap | SFR |
| Event Presence (BAcc) | |||||
| MOSS-Audio-8B | 81.2 | 78.5 | 65.7 | 12.9 | 67.5 |
| Audio Flamingo Next | 84.0 | 85.2 | 71.1 | 14.1 | 67.8 |
| Qwen3-Omni | 91.3 | 89.0 | 79.3 | 9.7 | 64.2 |
| Step-Audio-2-mini | 85.8 | 79.9 | 67.8 | 12.0 | 74.7 |
| Spatial-Omni (SO-30B) | 73.1 | 72.8 | 69.3 | 3.5 | 64.6 |
| Event Identity (accuracy) | |||||
| MOSS-Audio-8B | 46.7 | 57.4 | 29.1 | 28.4 | 60.9 |
| Audio Flamingo Next | 81.4 | 76.2 | 35.6 | 40.6 | 71.5 |
| Qwen3-Omni | 84.6 | 83.0 | 39.8 | 43.2 | 70.5 |
| Step-Audio-2-mini | 88.0 | 79.4 | 32.9 | 46.5 | 80.8 |
| Spatial-Omni (SO-30B) | 78.2 | 76.1 | 51.3 | 24.8 | 47.0 |
| Temporal Order (accuracy) | |||||
| MOSS-Audio-8B | 94.0 | 94.2 | 43.6 | 50.7 | 56.4 |
| Audio Flamingo Next | 87.4 | 89.6 | 27.6 | 62.0 | 71.8 |
| Qwen3-Omni | 98.2 | 99.7 | 58.4 | 41.3 | 41.6 |
| Step-Audio-2-mini | 72.6 | 96.8 | 7.5 | 89.3 | 92.4 |
| Spatial-Omni (SO-30B) | 92.7 | 86.2 | 70.5 | 15.7 | 29.5 |
Table 3. Neutral-prompt acoustic performance and SFR (%). Presence uses recording-level BAcc; other tasks use accuracy. Gap = compatible minus conflict (pp). SFR uses only eligible conflict questions.
Performance under conflicting evidence. All five models score lower under conflicting than compatible speech, with gaps of 3.5–14.1 percentage points (pp) for Presence, 24.8–46.5 pp for Identity, and 15.7–89.3 pp for Temporal Order. These are descriptive condition-level gaps, not paired effects holding the environmental waveform fixed. Step-Audio-2-mini illustrates the pattern: its Temporal Order accuracy is 72.6% with audio only and 96.8% with compatible speech, but 7.5% under conflict. Strong audio-only recognition thus does not guarantee high accuracy when speech contradicts the acoustic evidence.
Claim-aligned responses under conflict. Among conflicting five-choice Identity questions with a defined speech-implied answer, SFR ranges from 47.0% to 80.8%, above the 20% uniform-choice reference. It reaches 80.8% for Step-Audio-2-mini, 71.5% for Audio Flamingo Next, and 70.5% for Qwen3-Omni. Eligible conflict responses therefore frequently select the misleading spoken option. SFR describes this directional response pattern; it does not measure the fraction of all conflict errors caused by speech.
Variation in Conflict Sensitivity
Conflict sensitivity varies across speech level, claim formulation, and temporal task structure. We examine these factors before separately analyzing semantic and physical spatial cues.
Relative acoustic prominence. Increasing the speech-to-event ratio (SER) from to dB increases SFR by 11.9–25.0 pp for Identity and 6.3–17.7 pp for Presence across the five models (Figure 4). Even at dB, Identity SFR remains 39.2–65.6%, above the 20% uniform-choice reference. Speech-aligned responses therefore persist when speech is quieter than the environmental event. However, SER changes both speech intelligibility and acoustic masking, preventing attribution of this trend solely to linguistic preference.
Figure 4. SFR versus speech-to-event ratio (SER) under neutral prompting. Shading indicates source-cluster bootstrap 95% confidence intervals; dashed lines indicate uniform-choice references.
Claim formulation and direction. Among positive conflicting Identity claims, explicit statements yield SFRs 13.7–29.1 pp higher than implicit statements over shared duration bins, including 79.8% versus 50.7% for Audio Flamingo Next. Presence also exhibits strong polarity asymmetries: positive versus negative claim SFR is 95.0% versus 22.9% for Qwen3-Omni and 95.4% versus 1.2% for Spatial-Omni. Because Presence claim polarity is coupled with target-event presence, these differences characterize condition-dependent behavior rather than an isolated wording effect.
Temporal task structure. Temporal Order exhibits substantial conflict-associated accuracy decreases across all five models, whereas Temporal State shows no comparable pattern (Figure 5). In fact, compatible-to-conflict accuracy rises from 73.9% to 84.1% for Audio Flamingo Next and from 70.4% to 78.9% for StepAudio-2-mini on Temporal State. Importantly, Temporal Order places the misleading speech after the entire environmental event sequence. Its failures therefore cannot be explained solely by simultaneous speech–event masking. Conflict sensitivity differs considerably even between tasks involving temporal acoustic information.

Figure 5. Acoustic accuracy under compatible and conflicting claims for (a) Temporal Order and (b) Temporal State. Error bars show 95% source-cluster bootstrap confidence intervals ( per model–condition cell).
Semantic and Physical Cues in Spatial Grounding
Task-dependent acoustic competence. Table 4 reports raw MID-macro accuracy for Spatial-Omni on eligible FOA inputs. Audio-only accuracy is 31.3% for Location and 50.5% for Motion, above their respective 2.6% and 25% uniform-choice references. Radial accuracy is 50.3%, near its 50% reference, limiting competence-based interpretations of its conflict diagnostics.
| Task | Chance | Audio-only | Speech (NS) | Speech (S) |
| Location | 2.6 | 31.3 | 21.0 | 21.5 |
| Motion | 25.0 | 50.5 | 35.0 | 33.2 |
| Radial | 50.0 | 50.3 | 49.2 | 49.7 |
Table 4. Spatial-Omni acoustic accuracy (%) on eligible FOA inputs under neutral prompting. NS/S denote non-spatial/spatialized speech; both columns pool compatible and conflicting conditions.
Semantic and physical interference. We analyze complete matched four-cell families with fixed environmental audio, crossing aligned or alternative semantic claims () and physical speech trajectories (). Relative to , Motion accuracy is 9.9 pp lower (95% CI: [6.0, 13.7]) when only the semantic claim is reversed, and 20.1 pp lower ([15.2, 25.2]) when only the physical speech trajectory is reversed (Figure 6a). For orthogonal trajectories, the corresponding differences are 15.8 pp ([11.7, 20.2]) and 13.0 pp ([8.6, 17.5]; Figure 6b). Both cues therefore interfere with environmental trajectory judgments. Their relative point estimates vary across contrasts; these comparisons do not establish internal causal mechanisms.
Figure 6. Claim and speaker cues in Spatial-Omni’s FOA motion task. (a,b) Matched acoustic accuracy when claim and speech trajectory match ground truth or an alternative . (c) Attraction to conflicting and relative to audio-only controls. Error bars show 95% class-bootstrap confidence intervals; dashed lines mark 25% chance in (a,b) and zero attraction in (c).
Target attraction and joint conflict. With spatialized speech, Motion CTA-C and CTA-S are 17.1 and 19.3 pp, with both 95% intervals above zero (Figure 6c). Predictions thus shift toward both the spoken claim and the speech-source trajectory relative to matched audio-only controls. Radial CTA-S is only 0.6 pp (95% CI: [-0.9, 2.1]), illustrating task-dependent attribution. Across all seven available factorial contrasts, interaction intervals include zero; the data do not establish nonadditive interference.
Prompt Intervention and Conflict Robustness
We compare neutral, acoustic-grounded (AG), and conflict-aware (CA) prompting with unchanged task questions and answer options (Table 5; Figure 7). We assess both acoustic correctness and speech-following behavior.
Figure 7. Paired changes in acoustic performance and SFR under prompt interventions relative to neutral prompting. Rightward indicates reduced SFR; upward indicates improved acoustic performance. Presence uses recording-level BAcc. Model–task comparisons without eligible matched pairs are omitted.
| Model | Task | Acoustic score (%) Neu | Acoustic score (%) AG | Acoustic score (%) CA | SFR (%) Neu | SFR (%) AG | SFR (%) CA |
| MOSS-Audio-8B | Presence | 65.7 | 68.6 | 64.9 | 67.5 | 51.8 | 24.1 |
| MOSS-Audio-8B | Identity | 29.1 | 32.9 | 39.2 | 60.9 | 51.8 | 38.1 |
| Audio Flamingo Next | Identity | 35.6 | 38.5 | 37.5 | 71.5 | 59.1 | 64.1 |
| Qwen3-Omni | Identity | 39.8 | 49.6 | 59.3 | 70.5 | 54.7 | 40.9 |
| Step-Audio-2-mini | Identity | 32.9 | 33.5 | 33.0 | 80.8 | 78.7 | 79.4 |
| Spatial-Omni (SO-30B) | Identity | 51.3 | 56.5 | 60.6 | 47.0 | 34.4 | 37.7 |
| Qwen3-Omni | Temporal Order | 58.4 | 71.3 | 85.1 | 41.6 | 28.7 | 14.9 |
| Step-Audio-2-mini | Temporal State | 78.9 | 55.5 | 56.3 | 21.1 | 44.5 | 43.7 |
Table 5. Prompt-policy performance on representative conflict tasks (%). Neu: neutral; AG: acoustic-grounded; CA: conflict-aware. Presence uses recording-level BAcc; other tasks use accuracy.
Selective recovery of acoustic grounding. CA prompting improves Qwen3-Omni’s Identity accuracy from 39.8% to 59.3%, while reducing SFR from 70.5% to 49.9%. On Temporal Order, accuracy rises from 58.4% to 85.1%, with 204 incorrect-to-correct repairs against only one correct-to-incorrect change in the matched comparison. MOSS-Audio and Spatial-Omni also gain 10.1 and 9.3 pp on Identity, respectively.
However, Step-Audio-2-mini remains near 33% Identity accuracy under all three policies. Prompting can therefore restore acoustically correct responses, but its effectiveness varies substantially across models.
Reduced speech following without improved grounding. Under CA prompting, MOSS-Audio’s Event Presence SFR falls from 67.5% to 24.1%, yet recording-level BAcc remains nearly unchanged (65.7% to 64.9%), with a paired 95% confidence interval that includes zero. Class-specific recalls reveal the underlying response shift: Yes recall decreases from 69.9% to 35.4%, whereas No recall increases from 60.8% to 94.5%. Thus, the substantial reduction in speech following coincides with a pronounced preference for negative responses rather than improved balanced recognition of environmental events. This example shows that SFR suppression can arise from response redistribution without a corresponding gain in acoustic grounding.
Negative transfer under intervention. Prompt interventions can also substantially worsen acoustic grounding. For Step-Audio-2-mini on Temporal State, accuracy decreases from 78.9% under neutral prompting to 55.5% with AG and 56.3% with CA, while SFR increases from 21.1% to 44.5% and 43.7%, respectively. Under AG prompting, 216 initially correct responses become incorrect, compared with only 38 repairs among 760 paired questions. This yields a -23.4 pp accuracy change, with a source-cluster bootstrap 95% confidence interval of [-27.2, -19.5] pp. Notably, the task-specific instructions explicitly preserve speech as a temporal reference, yet performance still deteriorates. The results therefore reveal an intervention-induced failure pattern, although they do not establish its underlying mechanism.
Limited changes in spatial grounding. For Motion with spatialized speech, neutral, AG, and CA prompting yield acoustic accuracies of 33.2%, 32.9%, and 33.6%, respectively. CTA-C is 17.1, 13.9, and 15.0 pp, whereas CTA-S remains 19.3, 19.8, and 20.4 pp. The lower semantic-attraction point estimates therefore do not coincide with higher acoustic accuracy. These are descriptive policy-specific comparisons: the current spatial report does not provide paired cross-policy confidence intervals.
Scope of comparisons. Compatible–conflict gaps are condition-level aggregates, not paired instance effects. Spatial interference contrasts use matched families with fixed environmental audio, but speech waveforms vary across cells. Spatial results are limited to eligible Spatial-Omni FOA inputs; cross-policy differences remain descriptive because paired cross-policy confidence intervals are not reported. These analyses describe observable responses, not internal mechanisms. Synthesized speech and rendered spatial scenes may limit generalization to spontaneous speech and real-world environments.
Conclusion
We introduced SAC-Bench, a controlled benchmark for evaluating speech–acoustic conflict across semantic, temporal, and spatial grounding. Strong audio-only recognition does not guarantee robustness to conflicting speech, and Temporal Order failures occur even when speech follows the complete event sequence. Within the evaluated spatial setting, matched motion contrasts reveal accuracy losses associated with both spoken claims and physical speech trajectories.
Prompt interventions yield heterogeneous outcomes: they can restore correct judgments, reduce speech following without improving accuracy, or degrade performance. These findings distinguish acoustic competence, susceptibility to competing speech cues, and intervention effectiveness as separate evaluation properties. SAC-Bench provides controlled conflicts and matched comparisons to assess these properties through both acoustic correctness and cue-aligned responses.
Acknowledgments
We thank Jingze Zhang, Xiaoya Zhou and Yingpin Fang (Beijing Foreign Studies University), and Lee Sean Joe (National University of Singapore) for manually reviewing and refining ChatGPT-generated synonyms, including removing inappropriate candidates and adding missing alternatives.
Dataset Construction Details
Source recordings are curated independently within the original FSD50K development and evaluation splits. We first remove redundant ancestor labels using the AudioSet ontology, then discard annotations whose available PP/PNP ratings are not uniformly positive. Annotations without available ratings are retained. A recording is eligible only if exactly one event label remains after these filtering steps. All reported model evaluations use recordings originating from the exhaustively annotated FSD50K evaluation split.
Duration Bins
Environmental recordings are grouped into , , , , , and seconds. Class eligibility and contrastive-label selection are computed independently within each FSD50K source split and duration bin.
Contrastive Event Selection
Let denote the CLAP text embedding of event label . Semantic distance is
For each target, contains the most distant eligible labels.
For AudioSet ontology node , Seco intrinsic information content is
where is the ontology and contains its descendants. Lin similarity is
where is the common ancestor with the highest information content.
The far candidate is
Let contain candidates having an ancestor–descendant relation with . The near candidate is
If is empty, the maximization uses . Temporal-order construction instead samples one label uniformly from .
For semantic question construction, four additional distractor labels are selected independently of the near and far acoustic alternatives. We exclude the target label, the previously selected near and far labels, their duplicate display names, and candidates having an ancestor–descendant relationship with the target. The remaining eligible labels are sorted by ascending CLAP distance, with ties resolved by AudioSet class ID. Four distinct alternatives are then selected at the 12.5th, 37.5th, 62.5th, and 87.5th percentiles of this ordered candidate list, using deterministic nearest-position selection. These alternatives are used to construct question options rather than additional acoustic conflict conditions.
Speech Synthesis
Speech is synthesized using F5-TTS with two speaker identities, A and B, defined by separate reference waveforms and reference transcripts. Speaker-specific speed factors of 1.0 and 0.81672 are applied during synthesis. Each generated speaker segment is normalized to a target RMS of 0.055, with peak attenuation applied when necessary to prevent clipping. Multi-turn dialogue segments are concatenated in their original order and saved as 16-bit PCM audio at 44.1 kHz.
Table 6 records configuration values needed to reproduce curation and speech synthesis.
| Configuration | Value |
| CLAP checkpoint | 630k-audioset-best.pt |
| CLAP label-text template | Corrected human-readable label |
| Temporal-order sampling seed | 42 |
| GPT-4o model version | gpt-4o-2024-11-20 |
| F5-TTS checkpoint | F5TTS_Base |
| M-AILABS reference recording | elliot_miller |
| LJSpeech reference recording | Linda Johnson |
| Speaker-specific speed factors | Speaker A: 1.0; Speaker B: 0.81672 |
| Segment RMS target | 0.055 |
| TTS inference seed | 4242 |
| TTS NFE steps | 32 |
| TTS CFG strength | 2.5 |
| TTS deterministic mode | enabled |
| TTS output sample rate | 44.1 kHz |
Table 6. Dataset-generation configuration.
Spatial Coordinates and Conflict Conditions
Spherical discretization. We use a listener-centered coordinate system with pointing forward, leftward, and upward. Azimuth is measured from toward , and elevation is measured from the horizontal plane.
We construct 38 spherical Voronoi zones with the topology of the octahedral Goldberg polyhedron [17]. The generating sites are normalized permutations and sign variations of , , and , yielding 6, 24, and 8 sites, respectively. Their spherical Voronoi dual contains six quadrilateral and 32 hexagonal regions.
For a unit direction , zone membership is determined by
where denotes the unit vector of site . Source directions are sampled uniformly in solid angle within their assigned Voronoi cells using rejection sampling. The regions are not assumed to have equal area.
Static location. Let , , and denote the acoustic event direction, spoken claim direction, and physical speaker direction, respectively. For each acoustic anchor , we construct five semantic conditions according to their angular separation:
These comprise Same (), Near (), Medium (), Far (), and Antipodal (). The four non-matching claims occupy distinct zones.
For spatialized speech, five three-source relations are defined:
The construction contains one L1, one L2, and four instances each of L3–L5. Together with five non-spatial speech conditions ( undefined), this produces 19 conditions per source recording.
For L2, the speaker direction is selected from the four conflicting semantic directions, permitting an L2–L4 comparison at fixed and . L3 and L4 share and while varying . L5 selects from a separate set of Near, Medium, Far, and Antipodal candidates, excluding the zones occupied by and . Candidate selection balances angular-distance strata across source families.
Spherical motion. We select two non-neighboring zones whose sampled directions are separated by 60°–120°. The acoustic source follows the shorter great-circle arc between its endpoints. We construct three trajectory variants: the acoustic ground truth, its reversal, and a midpoint-orthogonal trajectory. The latter is obtained by rotating the entire arc by ±90° about the radial axis through its midpoint. The trajectories have orthogonal tangents at the shared midpoint, rather than a constant 90° angular separation.
The three variants define the possible semantic claims . Physical speech motion independently follows one of the same three trajectories or remains non-spatial, producing conditions.
Radial motion. For each fixed direction, the acoustic source either approaches the listener from 2.0 to 0.5 m or recedes from 0.5 to 2.0 m. Each acoustic direction is paired with both approaching and receding semantic claims. Speech is either non-spatial, moving in the same radial direction as the acoustic event, or moving in the opposite direction. This yields conditions.
Source sampling and matching. Within each FSD50K split and event class, we select five distinct source recordings across eligible duration bins using deterministic sampling. Selected WAVs must decode completely, contain finite non-silent audio, and match their declared duration bins. Recordings are not duplicated to fill a class quota.
Each selected recording is reused across its eligible matched spatial conditions. Clips shorter than 4 seconds contribute only to Static Location; longer clips additionally contribute to Spherical and Radial Motion. This preserves the acoustic material across controlled variations in and .
Table 7 summarizes the compatible and conflicting conditions.
| Task | Compatible | Conflict |
| Presence (+) | target | none / near / far |
| Presence (−) | none / near / far | target |
| Identity (+) | target | none / near / far |
| Identity (−) | none / near / far | target |
| Temporal State | matched | reversed |
| Temporal order | matched | reversed |
| Location | ||
| Spherical motion | ||
| Radial motion |
Table 7. Semantic compatibility conditions. In spatial tasks, semantic compatibility requires the spoken claim to match the acoustic ground truth , independent of speaker geometry .
Audio Rendering Details
Semantic Rendering
Speech waveform and environmental waveform are temporally center-aligned. Their relative level is
Speech level is fixed and event gain is chosen to attain the requested . If peak protection is required, common attenuation is applied to the complete mixture. Speech-only and audio-only semantic controls reference the corresponding source waveforms without additional mixture rendering.
Temporal Rendering
For Temporal State, the environmental event is placed before or after the complete speech waveform with gap seconds. For Temporal Order, two environmental waveforms are concatenated with an inter-event gap seconds, and speech follows the complete sequence after an additional gap seconds. Source recordings are assigned using a matched policy: for each event-class pair within the same source split and duration bin, the same recording for each event class is reused across compatible, conflicting, and audio-only conditions. Compatible and conflicting conditions use the same spoken claim but reverse the acoustic event order. Audio-only controls preserve the corresponding event sequence and inter-event gap while removing speech.
Level calibration uses pooled environmental RMS,
where is the length of event . Inserted silence is excluded. Speech gain is selected relative to , and clipping protection attenuates speech and events jointly.
Spatial Rendering
Source trajectories. Environmental events and spatialized speech are rendered using independently specified geometries. Static sources maintain a fixed direction at 2.0 m. Spherical motion follows a shortest great-circle arc at constant radius, while radial motion maintains a fixed direction and varies distance linearly between 0.5 and 2.0 m.
Moving sources remain stationary for 0.3 s at each endpoint and move at constant angular or radial velocity during the intervening interval. Trajectories are sampled at 10-ms intervals.
Binaural rendering. We use SADIE II D2 KEMAR HRIRs in SOFA format [1]. Static sources undergo direction-specific convolution. Moving sources use blockwise time-varying convolution, with inverse-distance-weighted HRIR interpolation (, exponent 2) and a 256-sample block size. SOFA delay information is preserved to maintain interaural timing cues.
FOA encoding. We use first-order Ambisonics with AmbiX ACN/SN3D conventions [15]. For a mono source at unit direction , the encoded signal is
where is the distance attenuation. Both environmental events and spatialized speech follow this encoding.
Distance and Doppler. Free-field distance attenuation is modeled as
with m. Radial motion additionally uses retarded-time resampling:
where m/s is the speed of sound. This produces Doppler shifts consistent with the time-varying radial distance. Constant-radius spherical motion has no radial Doppler shift.
| Parameter | Value |
| Speech–event gap | 0.2 s |
| Inter-event gap | 0.2 s |
| Speech position for Temporal order | after sequence |
| Semantic SER settings | , , 0, 5, 10 dB |
| Acoustic environment | free-field |
| Spatial formats | binaural, FOA |
| Outer/static radius | 2.0 m |
| Near radius | 0.5m |
| Trajectory sampling interval | 10ms |
| Endpoint hold | 0.3 s |
| Reference distance | 1.0 m |
| Speed of sound | 343 m/s |
| Doppler | enabled |
| HRTF set | SADIE II D2 KEMAR |
| Moving-source interpolation | IDW, |
| IDW exponent | 2 |
| Convolution block size | 256 samples |
| Active-RMS frame | 50 ms |
| Activity threshold | dB |
| Spatial speech modes | non-spatial, spatialized |
| Spatial additive noise | none |
| Spatial SER levels | 0 dB only |
Table 8. Audio rendering parameters used in dataset generation.
Speech spatialization. We distinguish two speech geometry modes. In the non-spatial mode, speech is added equally to both binaural channels or to the FOA channel only. No physical speaker direction is assigned.
In the spatialized mode, speech is rendered as an independent acoustic source with physical geometry , using the same binaural or FOA rendering pipeline as the environmental event. The speaker’s physical geometry is specified independently of the spoken semantic claim . Both speech and event waveforms are temporally center-aligned before mixing.
Level calibration and controls. Speech level is calibrated after spatial rendering to a 0-dB speech-to-event ratio (SER). Active RMS is measured over non-overlapping 50-ms frames within the speech–event temporal overlap in which both sources exceed their respective activity thresholds of dB relative to peak frame RMS. A single gain is applied to the speech component:
Binaural calibration uses joint-ear energy, whereas FOA calibration uses the component.
Both formats use anechoic rendering without added synthetic noise. The original recording background, if present, is retained. For each distinct spatially rendered environmental event, an audio-only control is retained and reused across matching speech conditions.
Rendering Parameters
Table 8 summarizes the rendering configuration.
Extended Dataset Statistics
The benchmark draws on 16,340 environmental recordings (45.9 hours) and 42,379 synthesized speech waveforms (93.7 hours), corresponding to 40,536 distinct speech texts. Its 616,031 unique final audio clips total
| Task | Unique clips | Hours |
| Presence | 242,555 | 781.8 |
| Identity | 242,550 | 786.6 |
| Temporal State | 4,404 | 24.2 |
| Temporal Order | 6,562 | 54.9 |
| Static Location | 64,546 | 194.6 |
| Spherical Motion | 32,258 | 142.1 |
| Radial Motion | 34,694 | 121.7 |
| All tasks (union) | 616,031 | 2,069.9 |
Table 9. Unique audio clips and path-deduplicated duration by task. Task rows overlap; the total is their union, not their sum.
2,069.9 hours. Clips are deduplicated by normalized final audio path, not waveform-content hashing. Each path and its duration are counted once in the global total; all referenced final files passed the existence check.
Of the unique clips, 558,333 contain speech and environmental audio, 28,580 are speech-only, and 29,118 are audio-only. Development contains 393,010 clips (1,326.1 hours), and evaluation contains 223,021 clips (743.8 hours). Environmental source paths, pair identifiers, synthesized speech identifiers, and final audio paths are disjoint across splits; 1,415 exact speech texts occur in both. Audio-clip counts are distinct from question and inference-request counts.
Evaluation Implementation Details
Question Construction
Presence For recordings with environmental audio, queries include the acoustic event label and its registered alternatives. For recordings without an environmental event, the speech/target label anchors the alternative registry. Conflicting conditions additionally include the spoken event label when available. Repeated query labels are merged. A query receives acoustic answer Yes only when it identifies the actual event; all queries receive No in no-event recordings.
Speech-implied Yes/No references are assigned only to queries about the spoken target label when its claim defines the corresponding polarity. Queries about unrelated alternatives do not automatically receive a speech reference.
Identity Each question contains five distinct choices, with distractors drawn from the registered contrastive alternatives described in Appendix § 8.§ 2. The acoustic event, or “No environmental sound is present” for no-event audio, is always included. Positive conflicting claims additionally require the speech-asserted event. Remaining slots are filled from the registered alternatives, excluding duplicates of required choices. Alternative selection and final ordering are deterministic. Negative identity claims do not generally specify a unique speech-implied identity answer.
Temporal and radial motion Temporal State chooses between event-before-speech and event-after-speech. Temporal order chooses between the two possible orders of the specified events. Radial motion chooses approaching or receding.
Spatial questions. Static Location uses all 38 registered zones as distinct choices, with option keys A–Z and AA–AL. Spherical Motion asks for the complete directed trajectory using four distinct alternatives: the actual trajectory, its reversal, and its two midpoint-orthogonal rotations by and . The bank includes both rotations even though each source family renders only one as a semantic alternative. Radial Motion uses approaching and receding. Each primary question has one correct key, with uniform-choice references of , , and , respectively.
Spatial diagnostics and ordering. Neutral-prompt diagnostics query the spoken claim , physical speech state when defined, and their relation to the environmental target . Claim and physical-state questions use the corresponding primary bank. Relation questions distinguish L1–L5 for Location and Motion, or R1–R4 for Radial. Motion endpoints are separate 38-way diagnostics. All diagnostics are excluded from primary acoustic scores. The spatial build has no validated speech-only controls; mixed-input diagnostics do not measure independent single-source competence.
Spatial banks and permutations are fixed within each source family, independently of , , representation, and prompt policy. Mixed primary questions use all three policies; diagnostics and audio-only controls use neutral prompting. Nonspatial query and choice ordering follows the original deterministic seed-42 procedure.
Question Templates
The neutral policy adds no intervention instruction. The model-facing questions are:
Presence: “Is the sound of {query label} present in the recording?”
Identity: “Which of the following best identifies the environmental sound in the recording?”
Temporal State: “When does the environmental sound occur relative to the spoken segment?”
Temporal order: “In what order do the two environmental sounds occur?”
Location: “Which direction does the non-speech environmental sound come from, relative to the listener?”
Spherical Motion: “Which directed trajectory best describes the movement of the non-speech environmental sound?”
Radial Motion: “Is the non-speech environmental sound approaching or receding from the listener?”
Presence appends “Answer only with Yes or No.” Multiple-choice questions display the ordered options and append “Answer only with the corresponding letter.”
Spatial multiple-choice questions display their ordered choices and append “Answer only with the corresponding option key.” Motion start and end questions are retained only as diagnostics.
Intervention Instructions
We report neutral prompting as the baseline and evaluate acoustic-grounded and conflict-aware prompting as two intervention conditions.
Presence and Identity acoustic_grounded: “Base your answer on the actual environmental sounds in the recording. Do not use statements made by the speakers as evidence about whether an environmental sound is present or what that sound is.”
conflict_aware: “The spoken content may contain incorrect claims about the environmental sounds in the recording. Treat such claims as untrusted and independently determine the answer from the actual environmental sounds.”
Temporal State acoustic_grounded: “Determine the answer from the actual timing in the recording. Use the spoken segment only as a temporal reference, not as evidence about when the environmental sound occurs.”
conflict_aware: “The spoken content may incorrectly describe the timing of the environmental sound. Use the spoken segment only as a temporal reference and independently determine the event timing from the recording.”
Temporal order acoustic_grounded: “Determine the order from the environmental sounds you actually hear. Do not use statements made by the speakers as evidence about the event order.”
conflict_aware: “The spoken content may incorrectly describe the order of the environmental sounds. Treat such claims as untrusted and independently determine the actual order from the recording.”
Spatial localization acoustic_grounded: “Determine the source location from the spatial acoustic cues in the recording. Do not use statements made by the speakers as evidence about the source location.”
conflict_aware: “The spoken content may incorrectly describe the location of the environmental sound source. Treat such claims as untrusted and independently determine the source location from the spatial acoustic cues.”
Spherical motion acoustic_grounded: “Determine the source trajectory from the spatial acoustic cues in the recording. Do not use statements made by the speakers as evidence about the source trajectory.”
conflict_aware: “The spoken content may incorrectly describe the trajectory of the environmental sound source. Treat such claims as untrusted and independently determine the source trajectory from the spatial acoustic cues.”
Radial motion acoustic_grounded: “Determine whether the sound source is approaching or receding from the spatial acoustic cues in the recording. Do not use statements made by the speakers as evidence about the source motion.”
conflict_aware: “The spoken content may incorrectly describe whether the sound source is approaching or receding. Treat such claims as untrusted and independently determine the motion from the spatial acoustic cues.”
Inference Configuration
MOSS-Audio and Qwen3-Omni use vLLM with their model-specific audio interfaces and chat templates. Step-Audio-2-mini uses a vLLM-hosted chat-completion endpoint. Audio Flamingo Next uses Transformers with MusicFlamingoForConditionalGeneration in bfloat16. Spatial-Omni uses its official PyTorch implementation, Qwen3-Omni base model, adaptation checkpoint, and spatial audio encoder.
MOSS-Audio averages input channels before mono inference. Audio Flamingo Next reads float32 audio, averages channels, and uses polyphase resampling to 16 kHz. Step-Audio converts input to 16-kHz mono, 16-bit PCM WAV. Qwen3-Omni loads audio through qwen_omni_utils.process_mm_info; its processor behavior is tied to the recorded package version.
Spatial-Omni’s FOA path requires four channels, preserves their order during polyphase resampling to 16 kHz, and uses a maximum 20-second input window. Binaural requests and overlength recordings are explicitly skipped rather than truncated or substituted with mono audio.
Table 10 records the effective configuration of the reported runs.
| Checkpoint | Backend version | Precision | Max. tokens | Revision |
| MOSS-Audio-8B-Instruct | vLLM: 0.31.0 | BF16 | 256 | 66326e6 |
| Spatial-Omni-SO30B | PyTorch: 2.9.0 | BF16 | 256 | ba33a4c |
| audio-flamingo-next-hf | Transformers: 5.19.0 | BF16 | 256 | 5634886 |
| Qwen3-Omni-30B-A3B-Instruct | vLLM: 0.9.3 | BF16 | 256 | 26291f7 |
| Step-Audio-2-mini | vLLM: 0.1.dev1+g9ecabf22f | BF16 | 256 | e36fdd5 |
Table 10. Model checkpoints and inference configurations for the reported experiments.
Requests are assigned to deterministic logical shards by resolved audio path, keeping questions and policies for the same recording together. Stored outputs retain request identifiers, task metadata, reference sets, raw responses, and execution signatures.
Answer Parsing and Outcome Accounting
Nonspatial results use the relaxed parser. Spatial results use a separate final-line choice parser supporting both one- and two-letter option keys. Closed reasoning blocks and special-token markup are removed; an unclosed reasoning block is unparseable. Accepted spatial forms include a standalone displayed key, supported answer wrappers, and a key accompanied by its matching displayed option text. Text-only, fuzzy, or synonym-based spatial answer matching is not used.
Completed invalid or unparseable responses count as incorrect. Inference errors, missing predictions, and explicit skips are excluded from primary accuracy and reported separately. The reported Spatial-Omni run contains 327,084 requests across primary questions, diagnostics, and prompt policies: 132,585 completed responses (132,420 valid and 165 invalid), 163,542 unsupported-format skips, and 30,957 overlength skips, with no missing predictions or inference errors. These are request counts, not unique-audio or independent-source counts.
Presence Metrics
For recording , let contain completed queries with acoustic reference . Define
The primary score is
where contains recordings with completed queries of both reference classes. All queries are used, without negative subsampling; recordings receive equal weight.
Invalid completed responses remain incorrect. Recordings losing a completed reference class through inference failures are excluded from BAcc and counted separately. For analysis conditions with only one acoustic reference class by construction, the primary score is recording-averaged recall for that class, not BAcc. Question-pooled recalls and BAcc are supplementary diagnostics.
Spatial Scoring and Matched Diagnostics
Primary aggregation and uncertainty. Let denote correctness on a completed environmental-target question. The primary score is
where contains contributing event-class MIDs, their original environmental recordings, and the completed primary questions for a recording. Primary scores are raw accuracy. The supplementary adjustment is , with fixed , , or for Location, Motion, or Radial.
Spatial 95% intervals resample MIDs with replacement while retaining their within-class recording groups. They are conditional on the selected recordings. Nonspatial intervals instead use environmental-source clusters; Temporal Order uses connected components of shared sources. Both procedures use 1,000 bootstrap replicates with base seed 42. Paired contrast intervals are computed from paired differences.
Matched factorial contrasts. Hold environmental state fixed within a family and select an alternative . Let denote accuracy with for and for , and for and for . The aligned-other-cue interference contrasts and interaction are
Only families with four completed responses, the same source, and identical choices contribute. Differences are formed within families before source/MID aggregation. Location contributes only distance contrasts with a complete four-cell structure; not every source supplies all distance contrasts. These contrasts are descriptive because speech waveforms can differ. An interaction interval containing zero does not establish equivalence.
Conflict-target attraction. For target role , let denote its spatial value for question , restricting eligibility to . Let and denote the spatial values selected on a mixture and its unique matched neutral audio-only control. Both responses must be completed and share environmental source, spatial state, representation, and ordered choices. Define
CTA-T is the source/MID macro-average of . The control need not be correct. Invalid completed responses contribute zero to target indicators. CTA-C and CTA-S can use different eligible sets and share a target when ; CTA alone does not separate the two mechanisms.
Conditional acoustic retention. Retention is mixed-input primary accuracy restricted to matched audio-only controls answered correctly, with the same source/MID aggregation. Correct-control selection does not establish competence when baseline performance is near chance. Spatial prompt-policy scores remain descriptive because the current report supplies no paired cross-policy intervals.
References
- [1]Cal Armstrong, Lewis Thresh, Damian Murphy, and Gavin Kearney. A perceptual evaluation of individual and non-individual hrtfs: A case study of the sadie ii database. Applied Sciences, 8(11):2029, 2018.DOI
- [2]Debarpan Bhattacharya, Apoorva Kulkarni, and Sriram Ganapathy. Benchmarking and confidence evaluation of lalms for temporal reasoning. arXiv preprint arXiv:2505.13115, 2025.arxiv.org/abs/2505.13115
- [3]Jayadev Billa. When audio-llms don’t listen: A cross-linguistic study of modality arbitration. arXiv preprint arXiv:2602.11488, 2026.arxiv.org/abs/2602.11488
- [4]Yushen Chen, Zhikang Niu, Ziyang Ma, Keqi Deng, Chunhui Wang, JianZhao JianZhao, Kai Yu, and Xie Chen. F5-tts: A fairytaler that fakes fluent and faithful speech with flow matching. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 6255–6271, 2025.
- [5]Zixuan Chen, Depeng Wang, Hao Lin, Li Luo, Ke Xu, Ya Guo, Huijia Zhu, Tanfeng Sun, and Xinghao Jiang. Avid: A benchmark for omni-modal audio-visual inconsistency understanding via agent-driven construction. arXiv preprint arXiv:2604.13593, 2026.arxiv.org/abs/2604.13593
- [6]Xize Cheng, Dongjie Fu, Chenyuhao Wen, Shannon Yu, Zehan Wang, Shengpeng Ji, Siddhant Arora, Tao Jin, Shinji Watanabe, and Zhou Zhao. Aha-bench: Benchmarking audio hallucinations in large audio-language models. Advances in Neural Information Processing Systems, 38, 2026.DOI
- [7]Hyebin Cho, Suho Yoo, Jaehyuk Jang, Changick Kim, and Joon Son Chung. Who wins the conflict? mechanistic interpretability of text bias in audio llms. arXiv preprint arXiv:2606.18924, 2026.arxiv.org/abs/2606.18924
- [8]Eduardo Fonseca, Xavier Favory, Jordi Pons, Frederic Font, and Xavier Serra. Fsd50k: an open dataset of human-labeled sound events. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 30: 829–852, 2021.arxiv.org/abs/2010.00475
- [9]Sreyan Ghosh, Ashish Seth, Sonal Kumar, Utkarsh Tyagi, Chandra Kiran Evuru, S Sakshi, Oriol Nieto, Ramani Duraiswami, Dinesh Manocha, et al. Compa: Addressing the gap in compositional reasoning in audio-language models. In International Conference on Learning Representations, volume 2024, pp. 15486–15511, 2024.arxiv.org/abs/2310.08753
- [10]Sreyan Ghosh, Arushi Goel, Kaousheik Jayakumar, Lasha Koroshinadze, Nishit Anand, Zhifeng Kong, Siddharth Gururani, Sang-gil Lee, Jaehyeon Kim, Aya Aljafari, et al. Audio flamingo next: Next-generation open audio-language models for speech, sound, and music. arXiv preprint arXiv:2604.10905, 2026.arxiv.org/abs/2604.10905
- [11]Dawei Huang, Yongjie Lv, Ruijie Xiong, Chunxiang Jin, and Xiaojiang Peng. When tone and words disagree: Towards robust speech emotion recognition under acoustic-semantic conflict. arXiv preprint arXiv:2601.04564, 2026.arxiv.org/abs/2601.04564
- [12]Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perlman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024.arxiv.org/abs/2410.21276
- [13]Xilin Jiang, Hannes Gamper, and Sebastian Braun. Sci-phi: A large language model spatial audio descriptor. IEEE Open Journal of Signal Processing, 2026.arxiv.org/abs/2510.05542
- [14]Ziyang Ma, Yinghao Ma, Yanqiao Zhu, Chen Yang, Yi-Wen Chao, Ruiyang Xu, Wenxi Chen, Yuanzhe Chen, Zhuo Chen, Jian Cong, et al. Ammar: A challenging benchmark for deep reasoning in speech, audio, music, and their mix. Advances in Neural Information Processing Systems, 38, 2026.arxiv.org/abs/2505.13032
- [15]Christian Nachbar, Franz Zotter, Etienne Deleflie, and Alois Sontacchi. Ambix-a suggested ambisonics format. In Ambisonics Symposium, volume 2011, 2011.
- [16]Jiacheng Pang, Ashutosh Chaubey, and Mohammad Soleymani. Do audio llms listen or read? analyzing and mitigating paralinguistic failures with voxparadox. arXiv preprint arXiv:2605.27772, 2026.arxiv.org/abs/2605.27772
- [17]Robert J. Renka. Algorithm 772: Stripack: Delaunay triangulation and voronoi diagram on the surface of a sphere. ACM Transactions on Mathematical Software (TOMS), 23(3):416–434, 1997.DOI
- [18]Sakshi Sakshi, Utkarsh Tyagi, Sonal Kumar, Ashish Seth, Rameswaran Selvakumar, Oriol Nieto, Ramani Duraiswami, Sreyan Ghosh, and Dinesh Manocha. Mmau: A massive multi-task audio understanding and reasoning benchmark. In International Conference on Learning Representations, volume 2025, pp. 84929–84964, 2025.arxiv.org/abs/2410.19168
- [19]Ashish Seth, Sonal Kumar, Rameswaran Selvakumar, Nishit Anand, Utkarsh Tyagi, Prem Seetharaman, Ramani Duraiswami, and Dinesh Manocha. Audio hallucination attacks: Probing the reliability of large audio language models. arXiv preprint arXiv:2603.29263, 2026.arxiv.org/abs/2603.29263
- [20]Siri-Team. Hey siri: An on-device dnn-powered voice trigger for apple’s personal assistant. Apple Machine Learning Journal, October 1 2017. URL https://machinelearning.apple.com/research/hey-siri.
- [21]Luoyi Sun, Xiao Zhou, Zექian Li, Ya Zhang, Yanfeng Wang, and Weidi Xie. Spotsound: Enhancing large audio-language models with fine-grained temporal grounding. arXiv preprint arXiv:2604.13023, 2026.arxiv.org/abs/2604.13023
- [22]Zhe Sun, Yujun Cai, Jiayu Yao, and Yiwei Wang. Audiomotionbench: Evaluating auditory motion perception in audio llms. arXiv preprint arXiv:2511.13273, 2025.arxiv.org/abs/2511.13273
- [23]Kim Sung-Bin, Oh Hyun-Bin, JungMok Lee, Arda Senocak, Joon Son Chung, and Tae-Hyun Oh. Avh-bench: A cross-modal hallucination benchmark for audio-visual large language models. arXiv preprint arXiv:2410.18325, 2024.arxiv.org/abs/2410.18325
- [24]Changli Tang, Wenyi Yu, Guangzhi Sun, Xianzhao Chen, Tian Tan, Wei Li, Lu Lu, Zejun Ma, and Chao Zhang. Salmonn: Towards generic hearing abilities for large language models. In International Conference on Learning Representations, volume 2024, pp. 16607–16629, 2024.arxiv.org/abs/2310.13289
- [25]Bin Wang, Xunlong Zou, Geyu Lin, Shuo Sun, Zhuohan Liu, Wenyu Zhang, Zhengyuan Liu, AiTi Aw, and Nancy Chen. Audiobench: A universal benchmark for audio large language models. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pp. 4297–4316, 2025a.arxiv.org/abs/2406.16020
- [26]Cheng Wang, Gelei Deng, Xianglin Yang, Han Qiu, and Tianwei Zhang. When audio and text disagree: Revealing text bias in large audio-language models. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 4878–4888, 2025b.arxiv.org/abs/2508.15407
- [27]Boyong Wu, Chao Yan, Chen Hu, Cheng Yi, Chengli Feng, Fei Tian, Feiyu Shen, Gang Yu, Haoyang Zhang, Jingbei Li, et al. Step-audio 2 technical report. arXiv preprint arXiv:2507.16632, 2025.arxiv.org/abs/2507.16632
- [28]Yusong Wu, Ke Chen, Tianyu Zhang, Yuchen Hui, Taylor Berg-Kirkpatrick, and Shlomo Dubnov. Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1–5. IEEE, 2023.
- [29]Jiaqi Xiong, Yunjia Qi, Qi Cao, Yu Zheng, Yutong Zhang, Ziteng Wang, Ruofan Liao, Weisheng Xu, and Sichen Liu. Deaf: A benchmark for diagnostic evaluation of acoustic faithfulness in audio language models. arXiv preprint arXiv:2603.18048, 2026.arxiv.org/abs/2603.18048
- [30]Jin Xu, Zhifang Guo, Hangrui Hu, Yunfei Chu, Xiong Wang, Jinzheng He, Yuxuan Wang, Xian Shi, Ting He, Xinfa Zhu, et al. Qwen3-omni technical report. arXiv preprint arXiv:2509.17765, 2025.arxiv.org/abs/2509.17765
- [31]Chen Yang, Chufan Yu, Hanfu Chen, Jie Zhu, Jingqi Chen, Ke Chen, Wenxuan Wang, Yang Wang, Yaozhou Jiang, Yi Jiang, et al. Moss-audio technical report. arXiv preprint arXiv:2606.01802, 2026.arxiv.org/abs/2606.01802
- [32]Qian Yang, Jin Xu, Wenrui Liu, Yunfei Chu, Ziyue Jiang, Xiaohuan Zhou, Yichong Leng, Yuanjun Lv, Zhou Zhao, Chang Zhou, et al. Air-bench: Benchmarking large audio-language models via generative comprehension. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1979–1998, 2024.arxiv.org/abs/2402.07729
- [33]Zongmeng Zhang, Wengang Zhou, Jie Zhao, and Houqiang Li. Robust multimodal large language models against modality conflict. arXiv preprint arXiv:2507.07151, 2025.arxiv.org/abs/2507.07151
- [34]Feiyu Zhao, Yiming Chen, Wenhuan Lu, Daipeng Zhang, Xiangyu Yue, and Jianguo Wei. HalluAudio: A comprehensive benchmark for hallucination detection in large audio-language models. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 38797–38816, 2026.
- [35]Zhisheng Zheng, Puyuan Peng, Ziyang Ma, Xie Chen, Eunsol Choi, and David Harwath. Bat: Learning to reason about spatial sounds with large language models. arXiv preprint arXiv:2402.01591, 2024.arxiv.org/abs/2402.01591
- [36]Muyuan Zhong, Erfei Cui, Sen Xing, Weiyun Wang, Wen Wu, Yuchen Hu, Yanting Zhang, Xiaowei Hu, Wenhai Wang, Chao Zhang, et al. Have-bench: Hierarchical audio-visual evaluation from perception to interaction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8801–8812, 2026.
- [37]Zhiyuan Zhu, Yixuan Chen, Yiwen Shao, Wenxiang Guo, Changhao Pan, Yu Zhang, Yuxiang Wang, Wei Liu, Houhua Zhang, Chengkuan Zeng, et al. Spatial-omni: Spatial audio understanding integration in multimodal llms via foa encoding. arXiv preprint arXiv:2606.10738, 2026.arxiv.org/abs/2606.10738



