betaFeedback

WorksEngineering

BAmbiDrop: Array-Agnostic Speech Enhancement with Zero-Shot binaural Reproduction using Multi-center Ambisonics Encoding

Abstract

Array-agnostic enhancement networks generalize across microphone geometries but produce a single channel, while binaural enhancement methods preserve interaural cues only for one fixed device geometry. We propose BAmbiDrop, a framework that extends array-agnostic enhancement to binaural reproduction without retraining. Exploiting the fact that the Ambisonics origin is an arbitrary point in space, a frozen single-channel AmbiDrop network estimates a complex mask at multiple centers across the array; these masks are fused by a complex mean into a single shared mask, applied to the microphone signals and rendered by fixed BSM filters. Across 21 free-field and rigid-sphere arrays, the fused mask improves enhancement and interaural cue reproduction simultaneously, whereas masking each microphone independently degrades the latter without a consistent gain in the former.

Introduction

Capturing an acoustic scene with a microphone array and reproducing it for a listener is a central problem in spatial audio, underlying immersive telepresence, augmented reality, hearing assistance and teleconferencing [1]. In binaural reproduction, the recording is rendered to the two ears of a listener, such that the interaural cues supporting localization and externalization are preserved [1]. Real scenes may contain competing talkers, noise and reverberation, so the recording must also be enhanced for the target talker to be intelligible [2]. Achieving high performance on both operations may therefore impose a challenge [1].

Systems that address both objectives differ mainly in the domain in which the noise reduction is applied. One line operates in the Ambisonics domain, where the enhanced signal remains a sound-field representation that can still be rendered binaurally, by masking designed for binaural reproduction [3], direction- and reverberation-preserving noise reduction [4], or deep Ambisonics-to-Ambisonics networks [5, 6]. A second line operates at the ears, where the trade-off between noise reduction and cue preservation is well documented: beamformers for binaural hearing devices constrain the interaural cues of the residual signal explicitly [7, 8], and learning-based systems produce a two-channel output under cue-preservation loss terms [9]. A third line enhances the microphone signals themselves, estimating one output per microphone so that a renderer may follow [10, 11], an architecture recently motivated by binaural reproduction [12]. Although these methods were found useful, the learning-based ones are typically preferable over classical approaches due to their improved speech enhancement [13]. However, learning-based neural networks are usually optimized for one microphone array layout and generalize poorly to others [14], and although they can be retrained per device, this requires substantial data which may not always be available. This shortcoming has motivated array-agnostic processing approaches.

Array-agnostic enhancement has been approached by architectures that remain invariant to the number and the placement of the microphones, employing transform-average-concatenate layers [15] or attention across channels [16], at the cost of large multi-geometry training sets. Recently, we proposed AmbiDrop [14, 17], which instead trains on ideal Ambisonics with channel-wise dropout and encodes an arbitrary array by Ambisonics Signal Matching (ASM) [18] at inference, so that no array geometry is observed during training. These methods, however, produce a single channel, and the spatial attributes of their output are not preserved. The closest exception maps array signals to a binaural output through a network trained with a geometry-invariant coherence feature [19], but its renderer is itself learned from data of one array family, and its invariance is demonstrated on a small set of compact configurations. In summary, the development and study of methods for array-agnostic speech enhancement that also deliver binaural audio remains a gap in current literature.

We propose BAmbiDrop to bridge this gap. The proposed method applies the mask of a trained AmbiDrop network [14] by re-positioning the Ambisonics coordinate-system origin to align with each of the array microphones, then applying Binaural Signal Matching (BSM) filters [20] to the enhanced microphone signals to produce binaural signals. The method exploits the generalization of the spherical harmonics representation: relocation can be applied to any microphone position using the same trained mask without loss of performance. We further show that a common mask shared between microphones provides a more accurate binaural signal as it reduces inter-channel variability compared to microphone-dependent masks. The contributions of this work are as follows: (i) the development of an array-agnostic speech enhancement framework with an accurate and efficient projection to binaural signals, (ii) once trained, the network can be applied to any array without re-training, (iii) the proposed method has been evaluated for both speech enhancement and binaural reproduction, showing improved performance compared to baselines.

Background

This section presents the array signal model, Ambisonics encoding by ASM, binaural reproduction by BSM, and the AmbiDrop enhancement model. The spherical coordinate system (r,θ,ϕ)(r, \theta, \phi) denotes radius, elevation and azimuth, and the wavenumber is k=2πcfk = \frac{2\pi}{c} f, with cc the speed of sound and ff the frequency.

Array Signal Model

Consider an arbitrary array of DD omnidirectional microphones positioned at (rd,θd,ϕd)(r_d,\theta_d,\phi_d), ∀ 1≤d≤D\forall\ 1 \le d \le D, in a sound field composed of QQ plane waves which represent the direct sound from far-field sources together with their reflections from the enclosure boundaries. The plane-wave arrival directions are denoted by uq=(θq,ϕq)\mathbf{u}_q=(\theta_q,\phi_q), ∀ 1≤q≤Q\forall\ 1 \le q \le Q. The array steering matrix V(k)\mathbf{V}(k) of dimensions D×QD \times Q, with element [V(k)]d,q[\mathbf{V}(k)]_{d,q} represents the transfer function between the qq-th plane wave and the dd-th microphone. With this model, the microphone signals can be formulated as:

x(k)=V(k)s(k)+n(k),(1)\mathbf{x}(k)=\mathbf{V}(k)\mathbf{s}(k)+\mathbf{n}(k), \tag*{(1)}

where s(k)\mathbf{s}(k) holds the plane-wave amplitudes at the origin and n(k)\mathbf{n}(k) is additive sensor noise, assumed spatially white and uncorrelated with the source signals.

Ambisonics Encoding and ASM

The Ambisonics signals up to order NaN_a are obtained from the plane-wave amplitudes as a(k)=YΩQHs(k)\mathbf{a}(k)=\mathbf{Y}_{\Omega Q}^{H}\mathbf{s}(k) [21], where a(k)\mathbf{a}(k) stacks the Ambisonics signals anm(k)a_{nm}(k) and YΩQ\mathbf{Y}_{\Omega Q} is the SH matrix of dimensions Q×(Na+1)2Q \times(N_a+1)^2 whose columns ynm\mathbf{y}_{nm} contain the SH basis functions of order 0≤n≤Na0 \le n \le N_a and degree −n≤m≤n-n \le m \le n evaluated at the QQ plane-wave directions. This representation depends on the sound field alone and not on the array that observes it.

ASM [18] estimates these signals from an arbitrary array by a linear mapping of the microphone signals,

a^nm(k)=[cnmASM(k)]Hx(k),(2)\hat{a}_{nm}(k)=\left[\mathbf{c}_{nm}^{ASM}(k)\right]^H\mathbf{x}(k), \tag*{(2)}

with the coefficients c(k)\mathbf{c}(k) designed to minimize the NMSE between the estimated and the true Ambisonics signals. Under a diffuse field with source covariance σs2I\sigma_s^2\mathbf{I} and spatially white noise of power σn2\sigma_n^2, the minimizer is:

cnmASM(k)=(V(k)VH(k)+σn2σs2I)−1V(k)ynm,(3)\mathbf{c}_{nm}^{ASM}(k)=\left(\mathbf{V}(k)\mathbf{V}^{H}(k)+\frac{\sigma_n^2}{\sigma_s^2}\mathbf{I}\right)^{-1}\mathbf{V}(k)\mathbf{y}_{nm}, \tag*{(3)}

which requires (Na+1)2≤D(N_a+1)^2 \le D for all channels to be encoded [21]. The accuracy of individual channels nonetheless depends on the array geometry, and so some channels may be poorly estimated [18].

Binaural Reproduction With BSM

BSM [20] reconstructs the binaural signal at a listener’s ears from signals recorded by an arbitrary microphone array, sharing the same linear structure as ASM (Eq. (2)). The binaural signals p^l,r(k)=[cl,rBSM(k)]Hx(k)\hat{p}_{l,r}(k)=[\mathbf{c}_{l,r}^{BSM}(k)]^H\mathbf{x}(k), combined into vector p(k)=[pl(k),pr(k)]T\mathbf{p}(k)=[p_l(k),p_r(k)]^T, are estimated by the filter cl,rBSM(k)\mathbf{c}_{l,r}^{BSM}(k). This filter minimizes the NMSE to the true binaural signals, and has a form similar to Eq. (3). Above a crossover frequency the least-squares fit is replaced by a magnitude least-squares minimization [20], which relaxes the phase constraint where interaural phase is perceptually less important.

Speech Enhancement With AmbiDrop

AmbiDrop [17] performs the task of speech enhancement in the Ambisonics domain, using an FT-JNF network [22] trained exclusively on ideal Ambisonics signals of order Na=2N_a=2, with channel-wise dropout applied during training. Since these signals depend on the sound field alone, as noted in Sec. 2.2, the network training is independent of any microphone array, while the dropout improves robustness to the channel-dependent encoding errors that ASM introduces at inference.

At inference, the microphone signals of an arbitrary array are encoded by ASM, transformed to the STFT domain, and processed by the network, which estimates a complex mask M(t,f)M(t,f) at each time-frequency bin (t,f)(t,f). The mask is applied to the omnidirectional channel,

a~00(t,f)=M(t,f)a^00(t,f),(4)\tilde{a}_{00}(t,f)=M(t,f)\hat{a}_{00}(t,f), \tag*{(4)}

yielding a single-channel enhanced signal. The model used here differs from the published one in the training reference alone, which is the reverberant target speaker rather than its direct-path component: reverberation carries the externalization, perceived distance and envelopment cues a binaural renderer must reproduce [23, 24, 25], a well-documented consideration in spatial audio reproduction.

Proposed Method

We propose BAmbiDrop, a binaural extension of the single-channel-output AmbiDrop framework, in which speech enhancement is applied separately to each microphone by repositioning the center of the Ambisonics coordinate system for each microphone across the array, and fusing the microphone-dependent masks into a single shared mask prior to binaural rendering. The proposed method is outlined in Figure 1, and detailed below.

Figure 1. The BAmbiDrop pipeline (Sec. 3). The microphone array feeds DD microphone-aligned ASM encoders, one per microphone position rdr_d (Sec. 3.1); the same frozen AmbiDrop network (Sec. 3.1) is evaluated once per microphone, giving one mask Md(t,f)M_d(t, f) each, and Fusion (Sec. 3.1) averages them into a single shared mask Mˉ(t,f)\bar{M}(t, f). Masking (Sec. 3.1) applies that mask to the unprocessed microphone signals, and BSM (Sec. 3.1) renders the result to the binaural signal. Dashed outlines mark repeated instances of the same block, one per microphone.

BAmbiDrop

Microphone-Aligned ASM: The Ambisonics channel for which AmbiDrop applies masking as in Eq. (4) is a00a_{00} which represent the sound pressure at the origin of the Ambisonics coordinate system [21]. Because the approach of the proposed method is based on applying the mask computed for channel a00a_{00} to an array microphone, it is hypothesized that this approach will work better when the microphone is aligned with the origin of the Ambisonics coordinate system. For this reason, the Microphone-Aligned ASM block aligns the origin of the coordinate system with one array microphone before applying ASM and the AmbiDrop mask to that microphone. In the following, this alignment process is presented in detail.

The main effect of this origin alignment is on the steering matrix V(k)\mathbf{V}(k). Because it represents the transfer function from a plane-wave of an amplitude defined at the origin to the output of the array microphones, aligning the origin of the coordinate system simply leads to a shift in the plane-wave amplitude, as also exploited in [26] for HRTF alignment. The elements of the aligned steering matrix can therefore be formulated as:

[Vδ(k)]d,q=[V(k)]d,qexp⁡(jkuq⋅δ).(5)[\mathbf{V}^{\delta}(k)]_{d,q}=[\mathbf{V}(k)]_{d,q}\exp(jk\mathbf{u}_q\cdot\boldsymbol{\delta}). \tag*{(5)}

where uq\mathbf{u}_q denotes the direction of the qq-th plane-wave and δ\boldsymbol{\delta} denotes the displacement of the origin relative to the origin defined by V(k)\mathbf{V}(k). Substituting Eq. (5) in Eq. (3) leads to

cnmδ(k)=(V(k)VH(k)+σn2σs2I)−1Vδ(k)ynm,(6)\mathbf{c}_{nm}^{\delta}(k)=\left(\mathbf{V}(k)\mathbf{V}^{H}(k)+\frac{\sigma_n^2}{\sigma_s^2}\mathbf{I}\right)^{-1}\mathbf{V}^{\delta}(k)\mathbf{y}_{nm}, \tag*{(6)}

where it is noted that Vδ(k)[Vδ(k)]H=V(k)[V(k)]H\mathbf{V}^{\delta}(k)[\mathbf{V}^{\delta}(k)]^{H}=\mathbf{V}(k)[\mathbf{V}(k)]^{H} due to the cancellation of the exponential function.

BAmbiDrop sets δ=rd\boldsymbol{\delta}=\mathbf{r}_d, the position of the dd-th microphone of Sec. 2.1, for every 1≤d≤D1 \le d \le D: the microphone signals are re-encoded using ASM DD times, once per microphone position, giving DD sets of microphone-aligned Ambisonics channels, transformed to the STFT domain for further processing.

AmbiDrop: The network of Sec. 2.4, trained once on ideal Ambisonics, independent of any array, is frozen (no further training or adaptation), and evaluated DD times - once on the microphone-aligned channels of each microphone dd. Each evaluation returns one complex mask, Md(t,f)M_d(t, f), the instance of (4) M(t,f)M(t, f) at that microphone’s origin.

Fusion: The AmbiDrop stage as above estimated a total of DD masks, one for each microphone. In this fusion stage, these masks are averaged to form a single mask:

Mˉ(t,f)=1D∑d=1DMd(t,f).(7)\bar{M}(t, f) = \frac{1}{D}\sum_{d=1}^{D} M_d(t, f). \tag*{(7)}

It is expected that such a mask common to all microphones will reduce inter-channel distortion and improve the binaural reproduction quality that follows. Nevertheless, this may come at the expense of speech enhancement performance. The variations described in Sec. 3.2 are designed to study this tradeoff.

Masking: In this stage, the common mask is applied directly to each of the unprocessed microphone signals,

X~d(t,f)=Mˉ(t,f)Xd(t,f),1≤d≤D,(8)\widetilde{X}_d(t, f) = \bar{M}(t, f)X_d(t, f), \qquad1 \le d \le D, \tag*{(8)}

leading to speech-enhanced microphone signals ready for binaural reproduction.

BSM: The masked signals X~d(t,f)\widetilde{X}_d(t, f) are decoded to binaural signals by the fixed BSM filters, detailed in Sec. 2.3. Note that this BSM filter is common to each variation detailed next in Sec. 3.2.

Variations of the Proposed Method

In this paper the proposed method is studied alongside several variations as detailed next.

Fusion: This is the main method, as detailed in Sec. 3.1.

Center: sets δ=0\delta= 0 in (5), as used originally in [14] and detailed in Sec. 2.2, and applies this single mask to each microphone via (8).

Per-Mic: this variation skips the Fusion stage, such that the DD masks Md(t,f)M_d(t, f) are applied individually to each microphone, X~d(t,f)=Md(t,f)Xd(t,f)\widetilde{X}_d(t, f) = M_d(t, f)X_d(t, f), therefore not sharing a common mask. This configuration may not preserve the inter-channel relations, but may improve speech enhancement due to the microphone-specific masks.

Baselines

Two methods are used as baselines for comparing to the proposed method and its variants. Both use the same FT-JNF architecture as the proposed method, but trained directly on the DD microphone signals, extended to a DD-mask MIMO output, one mask per microphone, in two different variations, as detailed below. Both baselines are trained on a training set of various array geometries.

MIMO FT-JNF applies a separate mask to each channel, the baseline analogue of Per-Mic.

MIMO FT-JNF (Fusion) averages the DD masks by a complex mean before applying them, which is the baseline analogue of Fusion.

Experiment

This experiment evaluates the performance of the proposed method and its variations against two baselines and across various array configurations and acoustic conditions.

Setup

Simulated acoustics scenes are generated with a target speaker and five interfering speakers recorded by an array in a reverberant room, simulated using the image method, with varying acoustic conditions as detailed in [17]. Clean speech signals are drawn from the WSJ0 corpus [27]. Twenty simulated arrays of [17], together with a simulated model of the Aria glasses array, include linear, planar and volumetric arrays of various configurations, confined to a sphere of radius 0.1 m, each with D=7D = 7 microphones. These are simulated using an Ambisonics representation of the sound field of order N=20N = 20 and include both free-field and rigid-sphere steering functions. The inference dataset comprises 100 examples per array, and is disjoint from the training and validation sets.

Methodology and Metrics

In the realization of the proposed method and its variants, the network of Sec. 2.4 is used unmodified, composed of a two-layer 64-unit BLSTM with a complex-mask output head, 142,594 parameters and channel dropout of up to three channels at probability 0.4, with all parameters identical to [17]. For these methods, microphone signals are preprocessed with an STFT using a 32 ms Hamming window and 50% overlap, and encoded by microphone-aligned ASM (Sec. 3.1) as a Tikhonov-regularized least-squares solution ((3)), with a regularization per frequency as in [20]. Fusion is evaluated over the DD microphone positions, (7). The BSM filters (Sec. 3.1) follow the same regularized solution ((3), with the magnitude least-squares crossover frequency set as in [20]). These filters are designed once per array and reused unchanged for each of the methods.

The two baselines of Sec. 3.3 share the network hyperparameters above and the same negative SI-SDR objective as in [17], with one scaling shared across all DD microphones. They are trained with the 10 training arrays of [17] and evaluated on those as well as on the 10 test arrays together with the simulated Aria array (§ 4.3). Both collapse well below Fusion on every column, several going negative on SI-SDR and SI-SDR-Dir: fitting a mask directly to the microphone domain does not generalize the way the array-agnostic Ambisonics-domain design does, as was already shown in [14].

The reference binaural signals are rendered from the full N=20N = 20 order Ambisonics representation of the scene using the Neumann KU 100 HRTF [28]. Two such references are used: the reference binaural signal, rendered from the full reverberant scene, and the direct binaural signal, rendered from the direct-path component alone. Performance is reported by the following metrics. SI-SDR is measured by the SI-SDR [29] of the binaural signals, with respect to the reference binaural signal, averaged over both ears, as are PESQ [30] and STOI [31]. SI-SDR-Dir measures the distortion caused by the mask to the direct sound from the sources, and is the SI-SDR of the masked direct-path signal, with respect to the direct binaural signal, averaged over both ears. ΔC50\Delta C50 is measured by the absolute error of the clarity index C50C_{50} [32] of the masked reverberant target signal with respect to that of the reference binaural signal. Interaural level difference (ILD) [33] and interaural coherence (IC) [25] are measured from the binaural covariances of the enhanced binaural signal, and evaluated against the same measures computed from the direct binaural signal and the reference binaural signal, respectively, leading to Δ\DeltaILD and Δ\DeltaIC. They are computed as the RMSE over frequency after time alignment.

Results

Table 1 reports the mean over the 100 examples of each array. Fusion attains the highest SI-SDR, PESQ, STOI and SI-SDR-Dir, and the lowest Δ\DeltaIC. This shows that Fusion, by averaging the Per-Mic mask estimates and using a shared mask, reduces binaural errors, as it leaves the inter-channel relations that BSM exploits in its design intact, so both enhancement and interaural cues improve. Center with its single mask falls only shortly behind, showing that averaging the Per-Mic masks is slightly better than using one central mask. An additional study not presented in the table investigated the addition of three more masks to the averaging of Fusion, which are centered at the two ears and the center of the head, but this improved the SI-SDR by at most 0.1 dB when averaged over all arrays, and so these positions were eventually excluded from Fusion.

MethodSI-SDR dB↑PESQ ↑STOI ↑ΔIC ↓ΔC50 dB↓ΔILD dB↓SI-SDR-Dir dB↑
BSM Noisy-8.801.230.5250.250.005.07-11.98
BSM Clean2.253.690.9450.140.221.441.34
Center-0.871.540.7560.261.544.770.13
Per-Mic-2.511.570.7530.261.035.07-3.23
Fusion-0.301.620.7850.231.544.710.69
MIMO FT-JNF-6.131.460.6500.291.124.80-4.01
MIMO FT-JNF (Fusion)-7.161.450.6340.281.184.72-4.68

Table 1. Mean performance of each method over all 21 arrays, with metrics defined in § 4.2. The BSM Noisy row is the noisy binaural signal, unprocessed, as computed using BSM; the BSM Clean row is the same but using the clean target signal only; MIMO FT-JNF and MIMO FT-JNF (Fusion) are the baselines of § 3.3, evaluated on the same full dataset as the proposed method’s variants. The best value per column, among the ranked methods, is in bold.

Per-Mic behaves as anticipated in § 3.2, with its Δ\DeltaILD, SI-SDR-Dir, and Δ\DeltaIC degrade compared to both Fusion and Center. In particular, SI-SDR-Dir degrades to −3.23-3.23 dB, probably because the independent per-microphone masks modify the channels by different complex gains and break the relations BSM assumed in its design. It nonetheless attains the lowest ΔC50\Delta C_{50}, although the differences to the other methods are not large (about 0.5 dB).

Table 1 also includes the two baselines of § 3.3, trained on only 10 of the 21 arrays (§ 4.2) but evaluated here, like the proposed method’s variants, on the full 21-array set. Both collapse well below Fusion on every column, several going negative on SI-SDR and SI-SDR-Dir: fitting a mask directly to the microphone domain does not generalize the way the array-agnostic Ambisonics-domain design does, as was already shown in [14].

Note that although the values of SI-SDR are not very high for all methods, they are close to that of BSM Clean, which incorporates both residual noise and distortion of the binaural signal, and indicate a high standing reference point. In conclusion, the proposed Fusion showed the feasibility of extending the previously proposed AmbiDrop method for array-agnostic speech enhancement to binaural reproduction without any additional training – simply by applying the trained network to ASM computed around each microphone position.

Conclusions

BambiDrop extends array-agnostic single-channel speech enhancement to binaural reproduction. Relocating the Ambisonics origin allows a frozen AmbiDrop network, trained only on ideal Ambisonics, to provide a mask at any point in space; fusing the masks estimated across the array into one shared complex mask and rendering the masked microphone signals with fixed BSM filters then yields a binaural output with no component retrained. This configuration improved both enhancement and interaural cue reproduction, whereas masking each microphone independently degraded the binaural signals. Future work is proposed to address on- and dynamic target speaker, improving the practical aspects of the method.

References

References

  1. [1]B. Rafaely, V. Tourbabin, E. Habets, Z. Ben-Hur, H. Lee, H. Gamper, L. Arbel, L. Birnie, T. Abhayapala, and P. Samarasinghe, “Spatial audio signal processing for binaural reproduction of recorded acoustic scenes—review and challenges,” Acta Acust., vol. 6, pp. 47, 2022.DOI
  2. [2]D. O’Shaughnessy, “Speech enhancement—a review of modern methods,” IEEE Trans. Human-Machine Syst., vol. 54, no. 1, pp. 110–120, 2024.DOI
  3. [3]M. Lugasi and B. Rafaely, “Speech enhancement using masking for binaural reproduction of Ambisonics signals,” IEEE/ACM Trans. Audio, Speech, Language Process., vol. 28, pp. 1767–1777, 2020.
  4. [4]A. Herzog and E. A. P. Habets, “Direction and reverberation preserving noise reduction of Ambisonics signals,” IEEE/ACM Trans. Audio, Speech, Language Process., vol. 28, pp. 2461–2475, 2020.
  5. [5]A. Herzog, S. R. Chetupalli, and E. A. P. Habets, “AmbiSep: Joint ambisonic-to-ambisonic speech separation and noise reduction,” IEEE/ACM Trans. Audio, Speech, Language Process., vol. 31, pp. 3081–3094, 2023.DOI
  6. [6]M. Lugasi, J. Donley, A. Menon, V. Tourbabin, and B. Rafaely, “Multi-channel to multi-channel noise reduction and reverberant speech preservation in time-varying acoustic scenes for binaural reproduction,” IEEE/ACM Trans. Audio, Speech, Language Process., vol. 32, pp. 3283–3295, 2024.DOI
  7. [7]B. Cornelis, S. Doclo, T. Van den Bogaert, M. Moonen, and J. Wouters, “Theoretical analysis of binaural multichannel noise reduction techniques,” IEEE Trans. Audio, Speech, Language Process., vol. 18, no. 2, pp. 342–355, 2010.DOI
  8. [8]D. Marquardt, V. Hohmann, and S. Doclo, “Interaural coherence preservation in multi-channel Wiener filtering-based noise reduction for binaural hearing aids,” IEEE/ACM Trans. Audio, Speech, Language Process., vol. 23, no. 12, pp. 2162–2176, 2015.
  9. [9]V. Tokala, E. Grinstein, M. Brookes, S. Doclo, J. Jensen, and P. A. Naylor, “Binaural speech enhancement using deep complex convolutional transformer networks,” in Proc. IEEE Int. Conf. Acoust., Speech Signal Process. (ICASSP), 2024, pp. 681–685.arxiv.org/abs/2403.05393
  10. [10]A. Pandey, B. Xu, A. Kumar, J. Donley, P. Calamia, and D. Wang, “TPARN: Triple-path attentive recurrent network for time-domain multichannel speech enhancement,” in Proc. IEEE Int. Conf. Acoust., Speech Signal Process. (ICASSP), 2022, pp. 6497–6501.arxiv.org/abs/2110.10757
  11. [11]M. Lugasi, A. Menon, V. Tourbabin, and B. Rafaely, “Spatial audio signal enhancement by a two-stage source-system estimation with frequency smoothing for improved perception,” IEEE/ACM Trans. Audio, Speech, Language Process., vol. 30, pp. 2584–2596, 2022.DOI
  12. [12]T. Deppisch, “Direction-preserving MIMO speech enhancement using a neural covariance estimator,” arXiv preprint arXiv:2604.11179, 2026.arxiv.org/abs/2604.11179
  13. [13]G. Huang, J. R. Jensen, J. Chen, J. Benesty, M. G. Christensen, A. Sugiyama, G. Elko, and T. Gaensler, “Advances in microphone array processing and multichannel speech enhancement,” in Proc. IEEE Int. Conf. Acoust., Speech Signal Process. (ICASSP), 2025, pp. 1–5.arxiv.org/abs/2502.09037
  14. [14]M. Tatarjitzky and B. Rafaely, “AmbiDrop: Array-agnostic speech enhancement using ambisonics encoding and dropout-based learning,” in Proc. IEEE Int. Conf. Acoust., Speech Signal Process. (ICASSP), 2026, pp. 14732–14736.DOI
  15. [15]Y. Luo, Z. Chen, N. Mesgarani, and T. Yoshioka, “End-to-end microphone permutation and number invariant multi-channel speech separation,” in Proc. IEEE Int. Conf. Acoust., Speech Signal Process. (ICASSP), 2020, pp. 6394–6398.arxiv.org/abs/1910.14104
  16. [16]T. Yoshioka, X. Wang, D. Wang, M. Tang, Z. Zhu, Z. Chen, and N. Kanda, “VarArray: Array-geometry-agnostic continuous speech separation,” in Proc. IEEE Int. Conf. Acoust., Speech Signal Process. (ICASSP), 2022, pp. 6027–6031.
  17. [17]M. Tatarjitzky, V. Tourbabin, and B. Rafaely, “AmbiDrop: Ambisonics-based array-agnostic neural speech enhancement,” arXiv preprint arXiv:2607.00548, 2026.arxiv.org/abs/2607.00548
  18. [18]Y. Gayer, V. Tourbabin, Z. Ben-Hur, J. Donley, and B. Rafaely, “Ambisonics encoding for arbitrary microphone arrays incorporating residual channels for binaural reproduction,” in Proc. IEEE Int. Conf. Acoust., Speech Signal Process. Workshops (ICASSP Workshops), 2024, pp. 244–248.DOI
  19. [19]Y. Hsu and M. R. Bai, “Learning-based array configuration-independent binaural audio telepresence with scalable signal enhancement and ambience preservation,” arXiv preprint arXiv:2311.12706, 2023.arxiv.org/abs/2311.12706
  20. [20]L. Madmoni, J. Donley, V. Tourbabin, and B. Rafaely, “Beamforming-based binaural reproduction by matching of binaural signals,” in Proc. AES Int. Conf. Audio for Virtual and Augmented Reality, 2020.
  21. [21]B. Rafaely, Fundamentals of Spherical Array Processing, Springer Topics in Signal Processing. Springer, Berlin, Germany, 1st edition, 2015.
  22. [22]K. Tesch and T. Gerkmann, “Insights into deep nonlinear filters for improved multi-channel speech enhancement,” IEEE/ACM Trans. Audio, Speech, Language Process., vol. 31, pp. 563–575, 2023.DOI
  23. [23]J. Catic, S. Santurette, and T. Dau, “The role of reverberation-related binaural cues in the externalization of speech,” J. Acoust. Soc. Amer., vol. 138, no. 2, pp. 1154–1167, 2015.DOI
  24. [24]A. W. Bronkhorst and T. Houtgast, “Auditory distance perception in rooms,” Nature, vol. 397, no. 6719, pp. 517–520, 1999.
  25. [25]J. S. Bradley and G. A. Soulodre, “Objective measures of listener envelopment,” J. Acoust. Soc. Amer., vol. 98, no. 5, pp. 2590–2597, 1995.
  26. [27]J. S. Garofolo, D. Graff, D. Paul, and D. Pallet, “CSR-I (WSJ0) Complete,” Linguistic Data Consortium, Philadelphia, PA, USA, 2007, LDC Catalog No. LDC93S6A.DOI
  27. [28]B. Bernschütz, “A spherical far field HRIR/HRTF compilation of the Neumann KU 100,” in Proc. AIA-DAGA Conf. Acoust., 2013, pp. 592–595.
  28. [29]J. Le Roux, S. Wisdom, H. Erdogan, and J. R. Hershey, “SDR—half-baked or well done?,” in Proc. IEEE Int. Conf. Acoust., Speech Signal Process. (ICASSP), Brighton, U.K., 2019, pp. 626–630.arxiv.org/abs/1811.02508
  29. [30]A. W. Rix, J. G. Beerends, M. P. Hollier, and A. P. Hekstra, “Perceptual evaluation of speech quality (PESQ)—a new method for speech quality assessment of telephone networks and codecs,” in Proc. IEEE Int. Conf. Acoust., Speech Signal Process. (ICASSP), 2001, vol. 2, pp. 749–752.
  30. [31]C. H. Taal, R. C. Hendriks, R. Heusdens, and J. Jensen, “An algorithm for intelligibility prediction of time–frequency weighted noisy speech,” IEEE Trans. Audio, Speech, Language Process., vol. 19, no. 7, pp. 2125–2136, 2011.DOI
  31. [32]International Organization for Standardization, “ISO 3382-1:2009 acoustics — measurement of room acoustic parameters — part 1: Performance spaces,” 2009.
  32. [33]J. Vilkamo and T. Bäckström, “Time-frequency processing: Methods and tools,” in Parametric Time-Frequency Domain Spatial Audio, pp. 1–24. Wiley, 2017.DOI

Paper details

vlnc:W_4mwtdyjz
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 11

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Engineering

Not recorded for this paper

View statement
Looking for related papers…
Contents