Overview and Meta-Analysis of DCASE 2026 Challenge Task 6: Audio Moment Retrieval from Long Audio

Introduction

Audio Moment Retrieval (AMR) refers to the task of retrieving specific temporal moments within a long audio recording that align with a given complex free-form text query [1]. For example, given a several-minute-long audio recording and a query such as “Spectators are cheering and shouting in a sports game”, a system is expected to output the start and end timestamps of the moments that match the query, as shown in Figure 1. Here, a moment is a contiguous interval defined by its start and end timestamps. A single recording may contain one or several moments that match the query.

Vector (PDF)

Long, untrimmed recordings such as meeting archives, lifelogs, broadcast media, podcasts, and acoustic monitoring are increasingly common, and manually searching for the portion of interest by listening is time-consuming and does not scale. AMR addresses this by allowing users to describe the target content in free-form text and returning the corresponding moments directly, which helps in indexing and navigating large audio archives.

AMR is positioned among several neighboring tasks. Clip-level language-based audio retrieval ranks short, pre-segmented clips by their relevance to a query [2, 3, 4]. This task assumes the audio is already segmented and does not detect when the queried content occurs, whereas AMR handles long and untrimmed recordings, and performs temporal detection. Sound event detection estimates temporal activities of a predefined, closed set of event classes [5], whereas AMR assumes an open-vocabulary setting specified by a given free-form query, which may describe several overlapping events, or a broader acoustic scene. The closest equivalent to AMR is video moment retrieval, where Detection Transformer (DETR) [6] models such as Moment-DETR [7], QD-DETR [8], UVCOM [9], and CG-DETR [10] regress the start and end timestamps directly. AMR transfers this paradigm to the acoustic domain, where temporal cues must be inferred from sound alone.

The initial study proposed the Audio Moment DETR (AM-DETR) baseline together with the large-scale synthetic Clotho-Moment dataset [1], later complemented by the manually annotated real-recording dataset CASTELLA [11]. The central challenge of AMR is to capture long-range temporal context while aligning it with the query using strong audio-text representations, despite the scarcity of annotated long audio data. Previous work [11] has shown that pre-training on synthetic data and fine-tuning on annotated data substantially improve accuracy.

DCASE 2026 Challenge Task 6 is the first DCASE task to address AMR, extending clip-level language-based audio retrieval to temporal detection. The organizers provide the datasets, a baseline system, and an evaluation protocol, so that participants can focus on modeling. Participants were encouraged to develop advanced audio-text models, to design networks that effectively capture temporal structure, and to generate or augment synthetic data to improve training. In total, 21 teams submitted 59 systems. The three best-performing systems achieved [email protected] of 48.59%. While simply replacing the audio-text feature extractor and moment-detection network with more powerful versions led to score improvements, the top-performing teams additionally implemented post-processing techniques such as confidence score calibration and ensembling.

Task Definition

This section presents the formal definition, notation, and evaluation metrics of DCASE 2026 Challenge Task 6.

Problem formulation and notation

The input to an AMR system is a long audio recording $x \in\mathbb{R}^{L}$ of duration $L$ samples and $T$ seconds, and a free-form text query denoted by $q$. The system outputs a set of $N'$ predicted moments,

where $0 < \hat{t}^{s}_{n} \le\hat{t}^{e}_{n} \le T$ denote the start and end timestamps (in seconds) of the $n$-th predicted moment and $\hat{c}_{n} \in\mathbb{R}$ is its confidence score produced by the system. The predicted moments are ranked by their confidence scores.

The goal of this challenge is to accurately predict the timestamps for moments that match the given query and to assign them high confidence scores. The query $q$ used in this challenge is relevant to a set of $N$ moments $\mathcal{M} = \{(t_n^s,t_n^e)\}_{n=1}^{N}$, where $N \ge1$, and a single recording may also be associated with multiple queries. Participants are required to submit at least one moment per query and may submit multiple moments in descending order of confidence.

Evaluation metrics

The correctness of a retrieved moment is determined by its temporal intersection over union (IoU) with the reference moments. For a predicted moment $\hat{m} = (\hat{t}^s,\hat{t}^e)$ and a reference moment $m = (t^s,t^e)$,

where $|\cdot|$ denotes the temporal length of the (intersected or unioned) interval. A predicted moment is regarded as correct when its IoU with a reference moment is at least a threshold $\theta$.

Systems are evaluated with two ranking metrics, Recall1@$\theta$ and mean average precision (mAP@$\theta$). Let $Q$ denote the set of evaluation queries. For a query $q \in Q$, let $\hat{m}^{(1)}_q$ be its highest-confidence predicted moment and $M_q$ be the set of reference moments. Recall1@$\theta$ considers only $\hat{m}^{(1)}_q$ and measures the fraction of queries whose highest-confidence moment attains IoU $\ge\theta$ with a reference moment:

where $\mathbf{1}[\cdot]$ is the indicator function. mAP@$\theta$ instead accounts for all predicted moments. For each query, the predictions are ranked by confidence and matched one-to-one to the reference moments at threshold $\theta$, yielding an average precision $\operatorname{AP}_q(\theta)$. mAP@$\theta$ is its mean over queries,

Following [1], the averaged mAP further averages mAP@$\theta$ over $\theta\in\{0.50, 0.55, \ldots, 0.95\}$ in steps of 0.05, and Recall1 is reported at $\theta= 0.5$ and $\theta= 0.7$. The exact matching and interpolation rules of $\operatorname{AP}_q(\theta)$ follow the official evaluation script released with the baseline code.

Since this challenge focuses on how accurately the most confident retrieved moment corresponds to a reference moment, the primary metric used for ranking is [email protected], i.e., the top-ranked moment must attain IoU $\ge0.7$. Note that participants are only required to predict a single moment, even when multiple reference moments exist for a query, and submitting additional moments only affects mAP and does not change the primary ranking.

Datasets

For this task, we use two development datasets, Clotho-Moment [1] and CASTELLA [11], and a separately collected evaluation dataset. Each dataset consists of long audio recordings, free-form English captions, and the timestamps corresponding to each description.

To avoid issues related to downloading audio, the organizers also distribute pre-extracted MS-CLAP audio-text features of all three datasets, so that participants can train and evaluate systems without downloading the raw audio. Table 1 summarizes the datasets.

Dataset	Annotation	#Rec.	#Captions	Length
Clotho-Moment	synthetic	51,240	44,261	1 min
CASTELLA	manual	1,862	3,881	1–5 min
Evaluation	manual	100	177	1–5 min

Development dataset

Clotho-Moment is a large-scale synthetic dataset designed to boost the training of AMR models [1]. It is constructed by overlaying foreground audio clips from Clotho [12], an audio-captioning dataset, onto long background recordings from the Walking Tours videos [13] at random intervals drawn from an exponential distribution with a mean of 30 seconds. Silence at the onset and offset of each Clotho clip is removed before overlaying to clarify the start and end of the moments. The foreground and background are mixed at randomly sampled relative levels. Because the timestamps are obtained automatically from the overlay procedure, no manual annotation is required, and the dataset is large, containing 51,240 one-minute recordings with 44,261 captions.

CASTELLA is a manually annotated dataset for training and evaluation [11]. It contains 1,862 recordings over 120 hours in total. This dataset is split into training, validation, and test splits, containing 2,182, 352, and 1,347 annotated captions, respectively. Recordings range from one to five minutes and is annotated with up to five moments. The recordings were collected from the original long-form YouTube videos corresponding to a subset of AudioCaps. The collected recordings shorter than one minute were excluded and longer ones were trimmed to five minutes. The captions and one-second-resolution timestamps were collected via crowdsourcing. Annotators selected the salient moments of each recording and wrote their captions, and the annotations were further reviewed by listening to the audio only, so as to remove errors caused by visual bias. On average, each recording contains 2.1 moments, and each caption is 7.8 words long.

CASTELLA and Clotho-Moment are originally divided into training, validation, and test splits. To ensure consistency, the split names are redefined using DCASE terminology: development-training comprises the training splits of both CASTELLA and Clotho-Moment; development-validation is the CASTELLA validation split; and development-testing is the CASTELLA test split.

Evaluation dataset

The evaluation dataset consists of 100 real audio recordings collected from YouTube, ranging from one to five minutes, with 177 text queries in total. The audio captions and timestamps were collected in the same manner as CASTELLA. Each query may be associated with one or more ground-truth timestamp pairs, which were withheld during the challenge.

BASELINE SYSTEM

The organizers provide a baseline system, AM-DETR [1], that is simple but effective for this challenge. It consists of two modules: an audio-text feature extractor and a moment-detection network that adapts the video-moment retrieval model QD-DETR [8] to the acoustic domain. QD-DETR builds on the Detection Transformer (DETR) [6] and Moment-DETR [7].

Feature extraction. The feature extractor is based on the pre-trained MS-CLAP 2023 model [4], which maps audio and text into a shared cross-modal embedding space. The text query is encoded by the text encoder into a sequence of 768-dimensional text embeddings. The audio recording is split using a sliding window with a 1-second window and a 1-second hop, and each frame is encoded by the audio encoder into a 768-dimensional audio embedding [1]. Because both modalities are embedded in the same space, the resulting embeddings already capture the alignment between acoustic content and language.

Operating on these pre-computed embeddings rather than on raw audio keeps the detection network lightweight.

Moment-detection network. Following QD-DETR [8], a transformer encoder contextualizes the sequence of audio embeddings conditioned on the text-query embedding, and a transformer decoder with $K$ learnable moment queries predicts a set of $K$ candidate moments parameterized by a center and width with a confidence score. The default value of $K$ is 10 in the baseline setup. At inference time, the candidate moments are ranked by their confidence scores, and the most confident one is used for the primary [email protected] metric.

Training and reproduction. The network is trained with bipartite matching between predicted and reference moments. The training objective is a moment loss that combines an $L_1$ loss on the predicted center and width with a generalized IoU (gIoU) loss [14], and a cross-entropy score loss on the confidence, weighted by $\lambda_{L1}$, $\lambda_{\mathrm{gIoU}}$, and $\lambda_{\mathrm{score}}$, respectively. In addition, the baseline uses loss functions from QD-DETR [8], such as the highlight detection loss and the negative pair loss, enhancing the retrieval performance. To make the setup reproducible, we state the protocol explicitly: models are trained on the development-training split, selected on development-validation, and evaluated on development-testing (§ 3.1), and two configurations are provided: one trained on CASTELLA only and one trained on CASTELLA together with Clotho-Moment. The baseline uses a two-layer transformer encoder and decoder with a hidden size of 256, eight attention heads, and a feed-forward size of 1,024, and is optimized with AdamW (learning rate $10^{-4}$, a batch size of 32, for 200 epochs). The baseline is implemented based on the Lighthouse library [15], and the full training and evaluation recipe has been released so that the reported numbers can be reproduced¹.

Results and Discussion

This section summarizes the challenge outcome and discusses the trends across the submitted systems.

Results

The challenge attracted 21 teams, which submitted 59 systems in total. Each team was allowed up to four submissions, and the submissions were evaluated on the evaluation dataset of 100 real recordings and 177 queries (§ 3.2). Table 2 shows the performance and ranking of the best system for each team. The top three systems tied on the primary metric, each achieving [email protected] of 48.59% [16, 17, 18], with the fourth-ranked system achieving 46.89% [19]. These scores are roughly 3.5 times the baseline score. The three tied systems are distinguished by the other metrics. While Kibata (YCU) and Kim (CAU) also achieved strong results in [email protected] and average mAP, Sugawara (YCU) achieved a lower [email protected] score. How these differences arise from the system designs is discussed in the following subsection.

#	Team (institution)	Feature extractor	Detection network	[email protected]	[email protected]	mAP
1	Kibata (YCU) [16]	M2D-CLAP	CG-DETR*	48.59	69.49	40.67
2	Kim (CAU) [17]	Multiple	Original	48.59	63.84	41.39
1	Sugawara (YCU) [18]	Multiple	UVCOM	48.59	59.89	22.69
4	Ogawa (YCU) [19]	M2D-CLAP	CG-DETR*	46.89	60.45	34.59
5	Calvet (AUDIAS) [20]	Multiple	UVCOM*	43.50	64.41	33.15
6	Choi (KAIST) [21]	M2D-CLAP*	QD-DETR*	41.24	55.93	33.11
6	Usui (YCU) [22]	M2D-CLAP	UVCOM	41.24	58.19	34.01
8	Nakazawa (AM) [23]	Multiple	QD-DETR	36.16	49.72	17.84
9	Kang (ISCT) [24]	FLAM	QD-DETR	35.59	50.28	26.84
10	Xiao (HEU) [25]	MS-CLAP	UVCOM*	33.90	48.02	27.18
11	Khan (WPI) [26]	Multiple	UVCOM*	32.20	48.02	20.49
12	Chunarak (NUTH) [27]	Multiple	QD-DETR*	29.38	52.54	22.73
13	Huang (WHU) [28]	MS-CLAP	UVCOM*	27.68	48.02	23.58
14	Nishijima (UTokyo) [29]	Qwen2-Audio*	Qwen2-Audio	22.60	32.77	15.48
15	Chen (CHET 30)	MS-CLAP	UVCOM	22.03	37.29	12.06
16	Lu (YZU) [31]	MS-CLAP	QD-DETR	21.47	31.64	17.35
17	Xu (GZHU) [32]	MS-CLAP	QD-DETR*	15.25	31.07	13.42
18	Huck (NV) [33]	MS-CLAP*	-	13.56	22.03	9.40
17	Zhang (XJTLU) [34]	MS-CLAP	Original	13.56	22.03	14.16
20	Kret (Cooper Union) [35]	MS-CLAP	-	11.30	19.21	6.57
21	Minh (GWU) [36]	MS-CLAP	Original	5.65	12.99	6.89
–	Baseline	MS-CLAP	QD-DETR	13.56	28.25	12.11

Discussion

Top-ranked systems. All three top-ranked teams [16, 17, 18] adopted approaches to refine the baseline. While all three teams comprehensively improved the feature extractor, moment-detection network, loss functions, and post-processing techniques such as confidence score refinement, they focused on different components. Kim (CAU) and Kibata (YCU) focused on making various architectural improvements to enhance overall retrieval performance. Sugawara (YCU) focused on accurately estimating moment widths rather than moment

centers through model ensembling. This difference is reflected in the gap between [email protected] and [email protected], suggesting that Sugawara (YCU) predicted more precise timestamps for easier-to-detect moments, whereas Kim (CAU) and Kibata (YCU) were better at roughly detecting more difficult moments.

Feature extraction. Table 2 shows that the teams that replaced the feature extractor dominated the leaderboard, and their median [email protected] was about 41%. These teams adopted M2D-CLAP [37], LAION-CLAP [38], FLAM [39], or self-supervised models like EAT and BEATs [40, 41]. Kibata (YCU) reported that simply replacing MS-CLAP features with M2D-CLAP features while keeping the architecture unchanged raised [email protected] from 26.80% to 40.70% on the development-testing dataset. In contrast, systems that kept the provided MS-CLAP features achieved lower team-best scores, with a median [email protected] of 18%. However, a substantial gap remained between the top-ranked teams and teams such as Nakazawa (AM) that focused on exploring feature extractors [23], indicating that there is potential for further gains beyond just the feature extractor.

Moment-detection network. Most systems used DETR-based moment-detection networks, often replacing QD-DETR with stronger variants. Top-tier teams like [16], [18, 19, 20], [22] utilized existing powerful networks such as UVCOM [9], CG-DETR [10], and BAM-DETR [42]. Kim (CAU) and Choi (KAIST) independently replaced the transformer self-attention with a Mamba [43] temporal encoder and reported this replacement as one of their largest single-component gains [17, 21]. In contrast, most systems based on a training-free approach did not outperform the baseline system, suggesting that a detection network trained on real data is essential for improving retrieval performance.

Model parameters. Figure 2 shows the relationship between the retrieval performance and the number of parameters. The performance and the total number of parameters are only weakly correlated ($r = 0.22$). Note that the systems with more than $10^4$ M parameters use audio LLMs. The performance and the number of trainable parameters are more strongly correlated ($r = 0.57$), whereas we can observe a system that achieves high performance without using more than ten times as many parameters as the baseline. These results imply that performance gains depend not only on scaling model parameters, but also on adding appropriate loss functions and incorporating postprocessing methods such as confidence-score refinement and model ensembling.

Confidence score refinement. Several teams addressed the misalignment between confidence scores and localization quality, where accurately localized moments could receive lower confidence scores than inaccurate ones. Approaches to this problem can be broadly divided into two categories: adding a loss function and introducing a score calibrator. For the loss function, IoU-aware approaches like varifocal loss [44] were widely introduced. Usui (YCU) reported that simply adding varifocal loss significantly improved [email protected] from 20.30% to 27.62%. Score calibrators were implemented in various forms, such as adding a lightweight branch, constructing unique dedicated models, and leveraging Audio LLMs. Two teams, including Kim (CAU), added a branch for the score calibration to the network to directly predict IoU values during training, and the predicted IoU values were used for the calibration at inference time [17, 28]. Two other teams employed their own score calibrators [20, 34]. Notably, Calvet (AUDIAS) reported that an MLP-based calibrator improved [email protected] from 34.45% to 40.01%. Khan (WPI) used an external model trained to be specialized in calibration [26], while others directly used models without any additional training [17].

Timestamp refinement. There were also approaches that focused on refining timestamps rather than confidence scores. Sugawara (YCU) reported that while detection networks can correctly predict the center of moments, they have a tendency to misestimate their widths [18]. To achieve accurate timestamp prediction, this team employed a model ensemble based on weighted box fusion [45], specifically incorporating models with varied temporal resolutions of acoustic features. This team tied for the highest [email protected] but had a lower [email protected] than the other two top teams, leaving a smaller gap between the two metrics. This is likely because their approach improved the ability to estimate moment widths, but did not significantly improve the ability to predict moment centers. Although Kang (ISCT) reported that a large-scale ensemble using models with the same architecture led to significant performance improvements [24], the gap between their [email protected] and [email protected] scores does not show a clear trend compared to teams with similar rankings. This result suggests that rather than relying on a simple ensemble, considering temporal resolution is crucial for improving the accuracy of moment-width estimation.

Audio LLMs. While some teams exploited audio LLMs for the score refinement and data augmentation, Nishijima (Utokyo) leveraged Qwen2-Audio [46] to directly predict the moments [29]. Since Qwen2-Audio cannot directly handle the long audio in AMR, this system compressed features along the sequence dimension and fine-tuned the model for AMR. Although this system substantially outperformed the baseline in terms of [email protected], the gap with the top system remains significant.

Synthetic-to-real gap. Several teams observed that systems tuned only on the synthetic development data often failed to transfer because of a mismatch between synthetic and real data. Indeed, when validation was performed on synthetic data, improvements in the validation score occasionally corresponded to worse performance on real data, so validating on the real development-testing split proved essential. To better exploit synthetic data, Ogawa (YCU) and Kang (ISCT) adopted data augmentation during pre-training. These teams narrowed the gap with concatenation-based synthesis and mixing augmentations such as MomentMix [47], using external corpora [48].

Development-testing-to-evaluation gap. The left panel of Figure 3 demonstrates the correlation between [email protected] on the development-testing dataset and that on the evaluation dataset. While both datasets were collected from YouTube, the development-testing dataset was obtained from a subset of AudioCaps, and the distribution of moments reflects the bias caused by AudioCaps’ filtering process. However, the very strong correlation between these scores suggests that the domain mismatch between the development-testing and evaluation datasets is limited.

Comparison between metrics. The center and right graphs of Figure 3 show relationships between metrics on the evaluation dataset. Comparing [email protected] and [email protected], we can find that the gap between these metrics widens as the score increases, moving further away from the $y=x$ line. In light of the discussion on timestamp refinement, systems with a large gap between the two metrics are likely to achieve further gains in [email protected] if they improve their performance in predicting moment widths. Comparing [email protected] and average mAP, the gap between these metrics also widens as the scores increase. This indicates that predicting multiple intervals remains challenging.

Conclusion

This paper describes DCASE 2026 Challenge Task 6, Audio Moment Retrieval from Long Audio, the first DCASE task to address the retrieval of text-queried temporal moments from minutes-long recordings. We provide the datasets, the baseline model and the evaluation protocol. For the evaluation, we constructed a manually annotated evaluation dataset dedicated to this challenge. The baseline system combines an MS-CLAP feature extractor with a DETR-based moment-detection network. The systems are evaluated using the IoU-based ranking metrics with [email protected] as the primary metric. The challenge drew 21 teams and 59 systems, and the three best-performing systems achieved a [email protected] of 48.59%, roughly 3.5 times the baseline score. All the top-performing systems enhanced their feature extractors and moment-detection networks. In particular, the three top-ranked systems incorporated further ideas, such as introducing a score calibrator and ensembling models with varying temporal resolutions of features. The gap between [email protected] and [email protected] suggests that there is still room for further performance improvement. Future work will focus on constructing a simple yet higher-performing model by integrating and refining these approaches.

Acknowledgments

We thank Paul Primus from Johannes Kepler University Linz for his invaluable support in preparing the challenge.