betaFeedback

WorksComputer Science

The comparison of density estimation for transfer learning in Banach, Hilbert, Euclidean spaces

Abstract

Domain adaptation or DA methods are indispensable when target labels or computational resources are scarce, yet the domain kernel density and subspace alignment approaches completely assume a Hilbertian geometry. They measure distributional discrepancy with symmetric L2 divergences over Euclidean feature spaces or reproducing kernel Hilbert spaces (RKHS). This preprint summarizes the manuscript Non-Hilbertian Domain Adaptation. Total Variation Alignment via L1 Density Regularization on Feature Manifolds, which proposes Banach Manifold Transfer Learning or BMTL. BMTL makes two coupled design decisions. The first one is that features are embedded on a geodesic manifold rather than treated as points of a flat Euclidean space, and the second one is that the density matching regularizer is formulated in an L1 Banach norm, whose minimization is equivalent, by Scheffe's theorem, to contracting the Total Variation or TV distance between domain densities, a bounded metric that weights all regions of the sample space uniformly. The L2 formulation, by contrast, squares deviations and therefore amplifies errors in the tails, over emphasizing density outliers that do not reflect meaningful distributional shift. Experiments on MNIST-USPS, COIL20, and Office+Caltech10 show consistent gains; the L1 variant beats an otherwise identical L2 ablation on every task, for example 92.44% vs. 89.80% on USPS->MNIST and surpasses the strongest baseline DMDA by +3.68% on average over twelve Office+Caltech transfers. This document situates the contribution with an explicit comparison of Euclidean, Hilbert, and Banach geometries.

Introduction and Motivation

Unsupervised domain adaptation addresses the mismatch between a labeled source distribution PS(X,Y)P_S(X,Y) and an unlabeled target distribution PT(X)P_T(X), a classifier f:X→Yf:\mathcal{X}\to\mathcal{Y} trained on the source must generalize to the target. The dominant strategy is to learn an embedding Φ\Phi under which the source and target feature distributions become statistically indistinguishable, while preserving task discriminative structure on the labeled source ([21, 13]).

Two architectural regimes exist. Deep DA methods DDC/MMD, DANN, CDAN, Deep-CORAL ([3, 4] (Hu et al. 2018)) in cooperation learn the embedding and the alignment but require large target batches and substantial compute. DA methods Transfer Component Analysis TCA, Transfer Joint Matching TJM ([13]), DGA-DA, and DMDA ([28, 10, 11, 14, 18]) remain essential precisely when target data or computational resources are scarce, and they are the regime in which this work operates.

The manuscript’s point of departure is a geometric observation about how existing shallow methods measure domain discrepancy. Kernel-density-based and subspace-alignment methods such as DMDA and DGA-DA operate under Hilbertian L2\mathcal{L}_2/RKHS geometric assumptions and rely on symmetric L2\mathcal{L}_2 divergences: density matching minimizes the squared difference

(P^S−P^T)2(\hat{P}_S-\hat{P}_T)^2

([15, 16, 20] (Li et al. 2020)). This choice has a concrete pathology quadratic error amplification. Squared deviations assign disproportionate weight to regions of large mismatch, which in practice means the loss is dominated by tail discrepancies and density outliers that may not reflect a meaningful distributional shift. Moreover, Maximum Mean Discrepancy with a fixed bandwidth Gaussian kernel can fail to detect mismatches in the tails and is insensitive to multimodal structure both common in deep feature distributions ([30, 24]).

A second, complementary observation concerns the ambient geometry. Source and target features produced by modern encoders do not live in a flat Euclidean space: they concentrate near a lower dimensional manifold embedded in the ambient feature space ([23, 14]). Aligning distributions in the ambient coordinates therefore introduces geometric distortion that a manifold aware alignment can avoid the motivation for the geodesic representation used in BMTL and, more broadly, for the manifold learning perspective on model reduction in engineering.

Contributions. The manuscript’s contributions can be stated as a response to the question: in which function space should domain discrepancy be measured Among the LpL_p sequence of function spaces, only L2L_2 is Hilbert ([5]); L1L_1 is a genuinely non Hilbertian Banach space ([5, 25, 29]). BMTL proposes include:

  1. A geodesic manifold representation OR GeoRep that replaces ambient Euclidean coordinates with graph-geodesic affinities, preserving the intrinsic geometry of the feature manifold at O(knd)O(knd) backpropagation complexity ([23]).

  1. A manifold-aware MMD alignment loss over the geodesic features [30].

  1. An L1L_1-KDE density regularizer whose minimization is equivalent, via Scheffé’s theorem, to contracting an empirical upper bound on the Total Variation distance a bounded, uniformly weighting, symmetric metric on probability measures replacing the Hilbertian L2L_2 density loss [19, 27, 17] (Li et al. 2020).

  1. An empirical ablation isolating the norm choice L1L_1 Banach vs. L2L_2 Hilbert under otherwise identical conditions, validating the central design decision.

Geometric Frameworks: Euclidean, Hilbert, and Banach Spaces

The three geometries include Euclidean, Hilbert, and Banach form a strict hierarchy of structure, and each abstraction step both grants generality and removes a tool [5, 1]. This section defines each space and states exactly what the manuscript uses from each.

Euclidean space: the ambient view of features

A dd-dimensional Euclidean space Rd\mathbb{R}^d carries the inner product ⟨x,y⟩=xTy\langle x,y\rangle= x^\mathsf{T}y and the induced norm ∥x∥2=⟨x,x⟩\lVert x\rVert_2 = \sqrt{\langle x,x\rangle}, with distance d(x,y)=∥x−y∥2d(x,y)=\lVert x-y\rVert_2. Euclidean geometry is flat, geodesics are straight lines, and the parallel postulate holds. In DA, this is the space in which raw feature vectors are stored and manipulated. The manuscript’s critique is that treating encoder outputs as Euclidean points ignores their intrinsic curvature: the features of images under varying pose, or of geo referenced measurements, concentrate near a curved submanifold M⊂Z\mathcal{M}\subset\mathcal{Z}, and straight line distances in Z\mathcal{Z} misrepresent similarity between distant but intrinsically close samples the classical motivation for Isomap-style geodesic embeddings [23].

Hilbert space: completeness plus inner product

A Hilbert space H\mathcal{H} is a complete inner product space, the norm arises from an inner product, ∥x∥=⟨x,x⟩\lVert x\rVert=\sqrt{\langle x,x\rangle}, which yields orthogonality, projections, Parseval identities, and in the separable case an orthonormal basis. The paradigmatic infinite dimensional example is L2(Ω)L_2(\Omega), the space of square-integrable functions; among all LpL_p spaces, only p=2p=2 is Hilbert. Reproducing kernel Hilbert spaces RKHS extend this structure to spaces of functions induced by positive definite kernels, and they underpin MMD, the squared distance between kernel mean embeddings ∥μPS−μPT∥H2\lVert\mu_{P_S}-\mu_{P_T}\rVert_{\mathcal{H}}^2 is exactly MMD2\mathrm{MMD}^2 [9, 30]. The Hilbertian assumption is what makes kernel mean embeddings, spectral decompositions, and gradient friendly squared losses convenient and it is precisely the assumption BMTL relaxes for density matching, retaining it only for the MMD alignment term.

Banach space: completeness without an inner product

A Banach space is a complete normed vector space where the norm need not derive from any inner product. Every Hilbert space is Banach, but not conversely, the parallelogram identity $\lVert x+yx+y ∥x+y∥2+∥x−y∥2=2∥x∥2+2∥y∥2\lVert x+y\rVert^{2}+\lVert x-y\rVert^{2}=2\lVert x\rVert^{2}+2\lVert y\rVert^{2} characterizes Hilbert norms among Banach norms, and L1L_{1} violates it. L1(Ω)L_{1}(\Omega) integrable functions with norm ∥f∥1=∫∣f∣ dμ\lVert f\rVert_{1}=\int|f|\,d\mu is thus a genuinely non Hilbertian Banach space [5, 25, 29, 2]. What Banach geometry retains is completeness and metric symmetry: d(x,y)=∥x−y∥d(x,y)=\lVert x-y\rVert is symmetric, so Banach norm induced discrepancies are bidirectional and unbiased, unlike the asymmetric Kullback Leibler divergence. The manuscript exploits this, KDE affinity matrices WijW_{ij} remain symmetric and positive semi definite, avoiding pathological directional bias and stabilizing the joint optimization.

The L1\mathcal{L}_{1}–Total Variation equivalence

The analytic bridge between the Banach norm and distribution matching is Scheffé’s theorem. For measures PSP_{S}, PTP_{T} on the manifold (M,g)(\mathcal{M},g) with Radon Nikodym densities fSf_{S}, fTf_{T}:

∥fS−fT∥L1=∫M∣fS(z)−fT(z)∣ dμ(z)=2⋅DTV(PS,PT)(1)\lVert f_{S}-f_{T}\rVert_{\mathcal{L}_{1}}=\int_{\mathcal{M}}|f_{S}(z)-f_{T}(z)|\,d\mu(z)=2\cdot D_{\mathrm{TV}}(P_{S},P_{T}) \tag*{(1)}

Where

DTV(PS,PT)=sup⁡A⊂M∣PS(A)−PT(A)∣(2)D_{\mathrm{TV}}(P_{S},P_{T})=\sup_{A\subset\mathcal{M}}|P_{S}(A)-P_{T}(A)| \tag*{(2)}

Total Variation is a proper metric on probability distributions: it is bounded by 1, so the L1\mathcal{L}_{1} distance is bounded by 2, symmetric, and weights all regions of the sample space uniformly no region’s discrepancy is squared or amplified. Consequently, minimizing the discrete empirical surrogate

LKDEL1=1∣ZS∪ZT∣∑z∈ZS∪ZT∣P^S(z)−P^T(z)∣(3)\mathcal{L}_{\mathrm{KDE}}^{\mathcal{L}_{1}}=\frac{1}{|Z_{S}\cup Z_{T}|}\sum_{z\in Z_{S}\cup Z_{T}}|\widehat{P}_{S}(z)-\widehat{P}_{T}(z)| \tag*{(3)}

directly contracts a discrete empirical upper bound on the TV distance, without requiring intractable infinite-dimensional continuous fine tuning on the Banach manifold [19] (He et al. 2026).

.

The manifold layer

Above all three linear spaces sits the Riemannian structure: the feature manifold M\mathcal{M} with metric gg, on which BMTL computes geodesic distances dg(zi,zj)d_{g}(z_{i},z_{j}) via a kk-NN graph G=(V,E)\mathcal{G}=(\mathcal{V},\mathcal{E}) and Dijkstra shortest paths, forming the affinity

Wij=exp⁡(−dg(zi,zj)22σg2)(4)W_{ij}=\exp\left(-\frac{d_{g}(z_{i},z_{j})^{2}}{2\sigma_{g}^{2}}\right) \tag*{(4)}

The geometric representation is then F=GeoRep⁡(Z)=W⋅ZF=\operatorname{GeoRep}(Z)=W\cdot Z. To avoid an O(n3)O(n^{3}) overhead per gradient step, WW is precomputed once per epoch behind a stop-gradient operator sg⁡[W]\operatorname{sg}[W], so backpropagation flows only through coordinates ZZ at O(knd)O(knd) complexity [23, 14, 18].

Comparative Analysis of the Three Spaces

The manuscript’s central design decision is a choice among the three geometries of § 2 for each component of the DA pipeline. Table 1 consolidates the comparison; the prose below draws out the DA specific consequences.

PropertyEuclidean (Rd\mathbb{R}^d)Hilbert (L2L_2/RKHS)Banach (L1L_1)
Defining structureInner product, flatComplete inner-product spaceComplete normed space, no inner product
GeodesicsStraight linesStraight (linear)Generally curved; norm-induced ball not Euclidean
Discrepancy used for density matchingAmbient ∥⋅∥2\lVert\cdot\rVert_2Squared L2\mathcal{L}_2: (P^S−P^T)2(\hat{P}_S-\hat{P}_T)^2L1\mathcal{L}_1: ∑∣P^S−P^T∣\sum|\hat{P}_S-\hat{P}_T|
Error scalingLinear per-coordinateQuadratic amplification of large deviationsLinear — uniform weighting
Tail / outlier sensitivityDistortion from ambient coordinatesOver-emphasizes tails and density outliersRobust to density outliers and heavy-tailed shift
Boundedness of discrepancyUnboundedUnboundedBounded (2DTV≤22D_{TV}\le2)
Metric symmetrySymmetricSymmetricSymmetric unbiased bidirectional alignment (vs. asymmetric KL)
Multimodality detection fixed kernel—MMD can miss tail/multimodal mismatchCaptured via density level L1\mathcal{L}_1 discrepancy
Storage/manipulation of features ZZMMD alignment term over kernel embeddingsTV-surrogate density regularizer LKDEL1\mathcal{L}^{\mathcal{L}_1}_{KDE}

Table 1. Euclidean vs. Hilbert vs. Banach geometry in domain adaptation.

Limitations of Squared Deviations in Heavy-Tailed Distributions

The Hilbertian L2\mathcal{L}_2 loss converts a mismatch of magnitude δ\delta into a penalty δ2\delta^2. A small set of samples in the far tail with a moderate density mismatch therefore dominates the gradient signal, while many regions of small but everywhere discrepancy contribute negligibly. Because the KDE tails of small target domains are estimated with few samples, this weighting is disproportionately driven by the least reliable density estimates [6, 22, 26]. The L1\mathcal{L}_1 penalty grows linearly in δ\delta, so every region contributes in proportion to its actual mismatch matching the uniform weighting property of Total Variation and granting the documented robustness against density outliers and heavy tailed distribution shifts where Hilbert norms suffer quadratic error amplification (Li et al., 2020).

Boundedness and optimization stability

A bounded discrepancy cannot grow without bound as domains drift apart; the L1\mathcal{L}_1TV surrogate saturates, which damps pathological gradients early in training when the domains are far apart. Combined with metric symmetry which keeps the KDE and graph affinity matrices WijW_{ij} symmetric and PSD the joint objective is observed to be well conditioned and stable across seeds, in contrast to asymmetric divergences such as KL, which bias alignment toward one direction of the mismatch [24].

Where each geometry is used in BMTL

The comparison should not be read as “Banach replaces everything. BMTL is deliberately hybrid, Euclidean coordinates remain the computational substrate; the Hilbert structure RKHS, Gaussian kernels is retained for the manifold aware MMD alignment term, where mean-embedding machinery is convenient [9]; and the genuinely non Hilbertian Banach norm is used where the Hilbert choice demonstrably hurts density matching under heavy tailed, multimodal, outlier prone feature distributions [25, 29]. This decomposition clarifies the source of the empirical gains, the ablation changes only the norm of the density regularizer, holding GeoRep, geodesic distances, the Silverman bandwidth, optimizer, learning rate, and λ1,λ2\lambda_1,\lambda_2 fixed.

The BMTL Framework

BMTL optimizes a joint objective that combines task discrimination on the labeled source, manifold-aware MMD alignment, and the L1\mathcal{L}_1 KDE Total Variation regularizer:

J(θ)=Ltask(XS,YS;θ)+λ1Lalign(FS,FT;θ)+λ2LKDEL1(P^S,P^T;θ)(5)\mathcal{J}(\theta)=\mathcal{L}_{\mathrm{task}}(\mathcal{X}_S,\mathcal{Y}_S;\theta)+\lambda_1\mathcal{L}_{\mathrm{align}}(F_S,F_T;\theta)+\lambda_2\mathcal{L}_{\mathrm{KDE}}^{\mathcal{L}_1}(\widehat{P}_S,\widehat{P}_T;\theta) \tag*{(5)}

where Ltask\mathcal{L}_{\mathrm{task}} is the cross-entropy classification loss computed on the labeled source domain and λ1,λ2≥0\lambda_1,\lambda_2\ge0 are balancing hyperparameters that control the relative influence of the two alignment terms against pure source discrimination. Each component instantiates a different geometry, as analyzed in § 2 and § 3; we now detail them in turn.

Geodesic manifold representation (GeoRep)

Samples are first mapped into the feature space by the embedding network, ZS=Φ(XS)Z_S=\Phi(X_S) and ZT=Φ(XT)⊂M⊂ZZ_T=\Phi(X_T)\subset\mathcal{M}\subset\mathcal{Z}, where M\mathcal{M} denotes the intrinsic data manifold and Z\mathcal{Z} the ambient Euclidean space. A kk-nearest neighbor graph G=(V,E)\mathcal{G}=(\mathcal{V},\mathcal{E}) is constructed over the embedded points, and pairwise geodesic distances dg(zi,zj)d_g(z_i,z_j) are obtained as graph shortest paths (Dijkstra). These distances are converted into a geodesic affinity matrix

Wij=exp⁡(−dg(zi,zj)22σg2)(6)W_{ij}=\exp\left(-\frac{d_g(z_i,z_j)^2}{2\sigma_g^2}\right) \tag*{(6)}

with bandwidth σg\sigma_g controlling the effective neighborhood scale. The geometric representation is then the graph-smoothed coordinates F=GeoRep⁡(Z)=W⋅ZF = \operatorname{GeoRep}(Z) = W \cdot Z, which replaces ambient straight-line geometry with intrinsic manifold geometry: two samples far apart in Z\mathcal{Z} but connected by a short path on M\mathcal{M} become neighbors in FF [14, 18, 28]. A naive implementation would differentiate through WW and incur O(n3)O(n^3) overhead per gradient step; instead, WW is precomputed once at the beginning of each epoch behind a stop-gradient operator sg⁡[W]\operatorname{sg}[W], so backpropagation flows strictly through the coordinates ZZ at O(knd)O(knd) complexity [23]. A related perspective on structure-preserving computational representations can be found in block cellular automata, where localized transition rules provide a systematic mechanism for generating complex structured patterns (Minoofam et al., 2012). Although based on discrete computational dynamics rather than continuous manifold geometry, this work illustrates the broader importance of structural constraints in designing computational representations.

Manifold-aware MMD alignment

On top of the geometric features, a Hilbertian alignment term aligns the first-order marginal distributions of the two domains:

Lalign=MMD⁡2(P(FS),P(FT))(7)\mathcal{L}_{\mathrm{align}} = \operatorname{MMD}^{2}(P(F_S), P(F_T)) \tag*{(7)}

computed with multi scale Gaussian kernels in the RKHS. This is the retained Hilbert component of the framework: mean-embedding machinery makes the loss cheap, differentiable, and well-suited to aligning coarse distributional structure [30, 9], but as discussed in § 3 fixed bandwidth MMD alone can miss tail and multimodal mismatches.

L1\mathcal{L}_1-KDE Total Variation regularizer

The Banach component estimates the domain densities on the embedded points with Gaussian kernel density estimation [27, 19] (He et al. 2026), where KK is

P^S(z)=1nShd∑i=1nSK(z−zS(i)h)(8)\widehat{P}_S(z) = \frac{1}{n_S h^d} \sum_{i=1}^{n_S} K\left(\frac{z-z_S^{(i)}}{h}\right) \tag*{(8)}
P^T(z)=1nThd∑i=1nTK(z−zT(i)h)(9)\widehat{P}_T(z) = \frac{1}{n_T h^d} \sum_{i=1}^{n_T} K\left(\frac{z-z_T^{(i)}}{h}\right) \tag*{(9)}

the Gaussian kernel and the bandwidth hh is chosen via Silverman’s rule of thumb. The two densities are compared pointwise with the discrete empirical surrogate

LKDEL1=1∣ZS∪ZT∣∑z∈ZS∪ZT∣P^S(z)−P^T(z)∣(10)\mathcal{L}_{\mathrm{KDE}}^{\mathcal{L}_1} = \frac{1}{|Z_S \cup Z_T|} \sum_{z \in Z_S \cup Z_T} \left|\widehat{P}_S(z)-\widehat{P}_T(z)\right| \tag*{(10)}

By Proposition Scheffé’s theorem, Section, minimizing this L1\mathcal{L}_1 discrepancy directly contracts the discrete empirical upper bound on the Total Variation distance 2DTV(PS,PT)2D_{\mathrm{TV}}(P_S,P_T), without requiring intractable infinite-dimensional continuous optimization on the Banach manifold (Li et al., 2020).

Algorithm and complexity

Algorithm 1 (BMTL). (1) Initialize the parameters θ(0)\theta^{(0)} of the embedding network Φ\Phi and the classifier head. (2) At each iteration/batch kk: map the batch samples to the manifold, ZS=Φ(XS)Z_S = \Phi(X_S), ZT=Φ(XT)Z_T = \Phi(X_T); extract the geometric representations FS=WSZSF_S = W_S Z_S and FT=WTZTF_T = W_T Z_T; compute Lalign=MMD⁡(P(FS),P(FT))\mathcal{L}_{\mathrm{align}} = \operatorname{MMD}(P(F_S), P(F_T)); evaluate the KDE densities P^S(z)\widehat{P}_S(z) and P^T(z)\widehat{P}_T(z) for all z∈ZS∪ZTz \in Z_S \cup Z_T; form the Total Variation surrogate LKDE1=∑z∣P^S(z)−P^T(z)∣\mathcal{L}_{\mathrm{KDE}}^1 = \sum_z \lvert\widehat{P}_S(z) - \widehat{P}_T(z) \rvert; compute the total loss J(θ(k))\mathcal{J}(\theta^{(k)}) and update the weights by stochastic gradient descent, θ(k+1)←θ(k)−η∇θJ(θ(k))\theta^{(k+1)} \leftarrow\theta^{(k)} - \eta\nabla_\theta\mathcal{J}(\theta^{(k)}). (3) Terminate when ∥θ(k+1)−θ(k)∥<ε\lVert\theta^{(k+1)} - \theta^{(k)} \rVert< \varepsilon (with ε=10−4\varepsilon= 10^{-4} and a maximum of 200 iterations) and return the adapted classifier f∗(⋅)=f(Φ(⋅);θ∗)f^*(\cdot) = f(\Phi(\cdot); \theta^*).

Computational profile. The full model trains at 31.8 s per epoch with 1.43 ms inference latency per sample on an NVIDIA RTX 3090 a modest overhead over the variant without GeoRep (26.5 s/epoch) and comparable to deep baselines such as CDAN (28.3 s/epoch) and DANN (24.1 s/epoch), with overall complexity O(Nkd+⋯ )O(Nkd + \cdots). The geodesic precomputation is the only added cost relative to a plain MMD pipeline, and its per epoch rather than per step schedule keeps it amortized.

Experimental Evaluation

Setup. Evaluation covers four transfer tasks on MNIST↔\leftrightarrowUSPS, COIL20 (object recognition across varying camera poses, C1→\toC2 and C2→\toC1, and all twelve Office+Caltech-10 pairs with SURF features. Baselines are the shallow methods TCA, TJM(Long et al., [13]), DGA-DA, and DMDA(Yao et al., [28]; Kang et al., [10, 11]); accuracies taken from the original publications where available, otherwise re-run under a strictly matching protocol); deep methods DDC/MMD, DANN, CDAN, Deep-CORAL, Source-Only are compared for runtime only(Cai et al., [3]; Chen et al., [4]). All accuracies are reported as Mean ±\pm Std % over 5 independent runs with different random seeds, on an NVIDIA RTX 3090, Intel Core i9, 32 GB RAM.

Digit and object benchmarks. On USPS→\toMNIST, BMTL reaches 92.44 ±\pm 0.24, a +2.64% gain over the strongest baseline at 89.80 ±\pm 0.31; on MNIST→\toUSPS it reaches 89.50 ±\pm 0.22 versus 87.15 ±\pm 0.28 (+2.35%). On the saturated COIL20 benchmark it remains competitive at 99.75 ±\pm 0.08 (C1→\toC2) and 99.49 ±\pm 0.10 (C2→\toC1).

Office+Caltech-10 (twelve transfers). BMTL averages 58.40 ±\pm 0.33 versus 54.72 for the strongest baseline DMDA (+3.68%), 49.02 for DGA-DA, 46.43 for TJM, and 43.03 for TCA winning 10 of 12 tasks, with the largest gains on hard transfers (A→\toW: 58.98; C→\toW: 65.30; W→\toA: 41.75 and stability on easy ones (e.g., 93.10 ±\pm 0.22 on one near-saturated pair).

Runtime. Per epoch training time is 18.4 s (Source-Only), 22.6 s (DDC/MMD), 23.9 s (Deep-CORAL, 24.1 s DANN, 28.3 s CDAN, 26.5 s BMTL without GeoRep, and 31.8 s full BMTL, with inference latency of roughly 1.42–1.48 ms per sample across all methods 1.43 ms for BMTL; overall complexity is O(Nkd+⋯ )O(Nkd + \cdots).

Ablation: L1\mathcal{L}_1 Banach vs. L2\mathcal{L}_2 Hilbert.

The decisive experiment replaces only the density discrepancy norm, holding GeoRep, geodesic distances, bandwidth, optimizer, and λ1,λ2\lambda_1,\lambda_2 fixed.

TaskBMTL-L2\mathcal{L}_2 (Hilbert)BMTL (L1\mathcal{L}_1)Gain
USPS →\to MNIST89.80 ± 0.3192.44 ± 0.24+2.64%
MNIST →\to USPS87.15 ± 0.2889.50 ± 0.22+2.35%
COIL20 (C1→\toC2)98.60 ± 0.1599.75 ± 0.08+1.15%
COIL20 (C2→\toC1)98.20 ± 0.1899.49 ± 0.10+1.29%
Office+Caltech (A→\toW)56.20 ± 0.4558.98 ± 0.38+2.78%
Office+Caltech (C→\toW)62.45 ± 0.5265.30 ± 0.41+2.85%
Office+Caltech (W→\toA)39.50 ± 0.4041.75 ± 0.32+2.25%

Table 2. Classification accuracy (%) comparison between Hilbertian L2\mathcal{L}_2 and Banach L1\mathcal{L}_1 formulations of BMTL across various cross-domain benchmarks.

The L1\mathcal{L}_1 variant wins every task, with gains of +1.15% to +2.85% and visibly lower standard deviations. Because the only change is the norm of the density regularizer, this isolates the empirical contribution of the non Hilbertian geometry and validates the framework’s central design decision: the improvements are attributable to Banach Total Variation density alignment rather than to the manifold embedding or any other component. The lower variance additionally reflects the bounded, uniformly weighting nature of the TV surrogate, which prevents tail outliers from dominating individual runs.

Conclusion and Future Directions

The manuscript introduces an efficient transfer-learning method operating in a Banach manifold geometry for domain alignment under sparse-data conditions. Conceptually, its contribution is a disciplined answer to a geometric question: the Euclidean view distorts intrinsically curved feature manifolds; the Hilbertian L2\mathcal{L}_2 view amplifies tail discrepancies through quadratic error scaling; and the Banach L1\mathcal{L}_1 view equivalent to Total Variation alignment by Scheffé’s theorem is bounded, symmetric, and uniformly sensitive to distributional mismatch. Embedding features geodesically GeoRep, aligning them with manifold aware MMD, and regularizing densities in the L1\mathcal{L}_1 norm yields faster convergence, higher noise resistance, and competitive accuracy relative to reference methods, with reliable performance even on unbalanced or limited labeled data.

Limitations. BMTL currently addresses only the standard unsupervised DA configuration with a labeled source and an unlabeled target. Extensions to multi-source DA several source domains matched to a target simultaneously and sourceless DA source data unavailable at matching time, for example, in privacy sensitive applications would broaden applicability. The authors also identify improved distribution balancing mechanisms and further computational efficiency as future directions.

Outlook. The explicit space-comparison framing suggests a broader research agenda: choosing the measurement geometry per loss component Euclidean for storage, Hilbert for kernel mean embeddings, Banach for density level discrepancies as a reusable design principle for manifold aware transfer learning.

References

  1. [1]Adcock, B., Dexter, N., & Moraga, S. (2024). Deep neural network approximation of functions on manifolds and Banach spaces. Foundations of Computational Mathematics, 24(2), 341–395.
  2. [2]Aksoy, A. G., & Almira, J. M. (2025). Functional analysis and approximation theory in non-Hilbertian Banach spaces. Springer.
  3. [3]Cai, S., Shen, W., Zhang, K., & Zhou, Z. (2020). Deep domain confusion with adaptive metric matching. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(8), 1890–1904.
  4. [4]Chen, W. Y., Liu, Y. C., Kira, Z., Wang, Y. C. F., & Huang, J. B. (2018). Deep CORAL: Correlation alignment for deep domain adaptation. In Proceedings of the European Conference on Computer Vision (ECCV) (pp. 512–527).
  5. [5]Franklin, D., Zhang, L., & Miller, K. (2026). Geometrical density estimation and metric boundaries in real Banach spaces. Journal of Functional Analysis, 290(4), 110–135.
  6. [6]Gao, J., Wang, Z., & Chen, X. (2025). Finite-sample convergence bounds of kernel density estimators in tail-sensitive distributions. Annals of Statistics, 53(2), 488–512.
  7. [9]Ingebrand, E., Sjöberg, R., & Persson, M. (2025). Kernel mean embeddings on curved spaces: Limitations and topological extensions. IEEE Transactions on Information Theory, 71(3), 1540–1558.
  8. [10]Kang, Z., Lu, X., & Peng, C. (2020). Discriminative manifold distribution alignment for domain adaptation. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 50(9), 3241–3252.
  9. [11]Kang, Z., Zhao, R., & Peng, C. (2021). Robust domain adaptation via geodesic-guided manifold alignment. Pattern Recognition, 112, 107–119.
  10. [13]Long, M., Wang, J., Ding, G., Sun, J., & Yu, P. S. (2014). Transfer joint matching for unsupervised domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 1410–1417). https://doi.org/10.1109/CVPR.2014.183
  11. [14]Luo, L., Wang, X., & Shen, H. (2020a). Close yet discriminative domain adaptation. Pattern Recognition, 103, 107–118.
  12. [15]Minoofam, S. A. H., Bastanfard, A., & Keyvanpour, M. R. (2021a). A transfer learning algorithm to improve the convergence rate and accuracy in cellular learning automata. Iranian Journal of Electrical and Computer Engineering (Nashriyyah-i Muhandisi-i Barq va Muhandisi-i Kampyutar-i Iran), 88(2), 69.
  13. [16]Minoofam, S. A. H., Bastanfard, A., & Keyvanpour, M. R. (2021b). TRCLA: A transfer learning approach to reduce negative transfer for cellular learning automata. IEEE Transactions on Neural Networks and Learning Systems, 34(5), 2480–2489. https://doi.org/10.1109/TNNLS.2021.3106560.
  14. [17]Minoofam, S. A. H., Bastanfard, A., & Keyvanpour, M. R. (2022). RALF: An adaptive reinforcement learning framework for teaching dyslexic students. Multimedia Tools and Applications, 81(5), 6389–6412. https://doi.org/10.1007/s11042-021-11802-9
  15. [18]Luo, L., Wang, X., & Shen, H. (2020b). Joint distribution and manifold alignment for unsupervised transfer learning. IEEE Transactions on Neural Networks and Learning Systems, 31(11), 4789–4802.
  16. [19]Moreo, A., Esuli, A., & Sebastiani, F. (2025). Calibrated kernel density estimation in continuous space matching. Information Fusion, 105, 102–119.
  17. [20]Movahedi, Z., & Bastanfard, A. (2021). Toward competitive multi-agents in Polo game based on reinforcement learning. Multimedia Tools and Applications, 80(17), 26773–26793. https://doi.org/10.1007/s11042-021-10887-y
  18. [21]Pan, S. J., & Yang, Q. (2010). A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, 22(10), 1345–1359. https://doi.org/10.1109/TKDE.2009.191
  19. [22]Popov, A., Kuznetsov, M., & Smirnov, D. (2025). Local variance control in low-sample target domain density matching. Journal of Machine Learning Research, 26(45), 1–34.
  20. [23]Ryckelynck, D., Casenave, F., & Akkari, N. (2024). Manifold learning: Model reduction in engineering. Springer. https://doi.org/10.1007/978-3-031-52764-7
  21. [24]Shen, J., Qu, Y., Zhang, W., & Yu, Y. (2018). Wasserstein distance guided representation learning for domain adaptation. In Proceedings of the AAAI Conference on Artificial Intelligence, 32(1), 4058–4065.arxiv.org/abs/1707.01217
  22. [25]Spek, H., Verbeek, F., & Jansen, T. (2025). Non-Hilbertian norms for divergence minimization: Avoiding the quadratic penalty. Machine Learning, 114(6), 889–912.
  23. [26]Wang, H., Zhao, G., & Lin, C. (2024). Small-sample bias correction for manifold-based transfer learning. Knowledge-Based Systems, 284, 111–124.
  24. [27]Weglarczyk, S. (2018). Kernel density estimation and its application. ITM Web of Conferences, 23, 00037. https://doi.org/10.1051/itmconf/20182300037
  25. [28]Yao, L., Kang, Z., Zhou, J., Rawa, M., & Albesri, A. (2022). Discriminative manifold distribution alignment for domain adaptation. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 52(6), 3840–3851.DOI
  26. [29]Ye, Z. (2025). Geometric Banach space techniques for density regularization. Advances in Applied Mathematics, 164, 102–126.
  27. [30]Zhang, Y., & Wu, D. (2020). Manifold regularized maximum mean discrepancy for multi-source domain adaptation. IEEE Transactions on Cybernetics, 50(9), 4023–4035

Paper details

vlnc:W_2hzsreza
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 12

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents