betaFeedback

WorksComputer Science

Structured Channel Network Pruning with Bi-Sparse Matrix Factorization

Abstract

Structured channel pruning for convolutional neural networks (CNNs) plays an important role in pattern recognition and deep learning. Nevertheless, existing methods usually calculate channel importance scores based on the magnitudes of individual filters, making it difficult to distinguish shared redundancy from individual channel contributions. To tackle this limitation, we propose a novel structured channel pruning method termed bi-sparse matrix factorization (BSMF). Specifically, BSMF characterizes the convolutional weight matrix as output-channel coefficients over a shared basis with row-wise ℓ2,p\ell_{2,p}-norm and column-wise ℓ2,q\ell_{2,q}-norm regularization terms, respectively. Note that p,q∈[0,1)p,q\in[0,1), which, to our best knowledge, has not been explored in the literature. To solve the resulting non-convex optimization problem, we develop an efficient update scheme built upon the alternating direction method of multipliers (ADMM), in which all subproblems admit closed-form solutions. After that, a compact CNN is obtained by reconstructing the convolutional filters from the learned factors and discarding the redundant channels. Extensive experiments on the CIFAR-10 and ImageNet-1K datasets demonstrate that our proposed BSMF achieves significant network compression while maintaining high reconstruction accuracy. The source code is publicly available at https://github.com/xianchaoxiu/BSMF

Email addresses: [email protected] (Jingjing Liu), [email protected] (Ziyang Wang), [email protected] (Xianchao Xiu), [email protected] (Wanquan Liu)

Keywords: Network compression, structured pruning, convolutional neural networks, matrix factorization, sparse optimization

Introduction

Convolutional neural networks (CNNs) have achieved considerable success in computer vision [1]. However, when deployed on mobile devices, deeper and wider networks must operate within limited memory and computation resources, which limits their practical implementation [2]. In this regard, model compression techniques have been developed to reduce network complexity while preserving predictive accuracy. Interested readers are referred to recent surveys [3, 4].

Generally speaking, existing model compression methods are grouped into four categories: pruning, quantization, knowledge distillation, and low-rank decomposition. Among them, pruning eliminates redundant weights or structured units [5], quantization reduces the precision of weights or activations [6], knowledge distillation trains a compact student model under the supervision of a teacher model [7], and low-rank decomposition factorizes dense weight matrices or tensors into smaller components [8].

Compared with the other three categories, pruning possesses the advantage of achieving flexible trade-offs between model performance and computational complexity. As discussed in [9], network pruning methods fall into unstructured and structured pruning. Technically, unstructured pruning produces irregular sparsity and usually demands dedicated software or hardware support for speedup. By contrast, structured pruning discards complete convolutional filters, yielding a compact network compatible with standard dense convolution implementations [10]. Consequently, structured pruning has garnered significant attention from both academia and industry [11].

Within structured pruning, filters are commonly selected using cross-layer importance evaluation [12], global channel attention [13], or gradient information [14]. Other paradigms include channel masks [15] and dynamic strategies realized by input-dependent gates [16]. Nevertheless, scores based on individual filter responses may fail to characterize shared components and channel-wise contributions. Recent efforts have sought to capture interdependencies among filters using independence criteria or tensor decomposition techniques [17, 18]. For example, efficient tensor decomposition-based filter pruning represented individual filters using higher-order singular value decomposition and identified similar filters from distances between their factors [18]. To exploit filter similarity during optimization, co-synthesizing multiple similar filters (CSMF) grouped similar filters and synchronized their updates using fused gradients and individual weight increments, consolidating their information into representative filters before removal [10]. More recently, dynamic filter pruning via unified importance and redundancy combined filter magnitudes with inter-filter correlations to consider filter importance and redundancy [19]. Although these methods alleviate inter-filter redundancy, they do not learn output-channel coefficients over a shared basis. Therefore, by aligning the coefficients in the decomposed representation with the output channels, one can build an explicit connection between shared structural redundancies and channel selection.

A complementary strategy is to use sparse regularization terms to induce removable channel structures. Specifically, network slimming using ℓp\ell_p-norm (0<p<10 < p < 1) regularization term approximated ℓ0\ell_0-norm sparsity [5], while channel pruning for convolutional neural networks using ℓ0\ell_0-norm constraints directly constrained nonzero channel indicators [20]. Although these objectives facilitate channel removal, they do not explicitly model the shared structure among filters. This motivates a combined treatment of shared redundancy and channel sparsity. In practice, channel selection must satisfy various constraints, such as layer-width requirements, residual connections, and global resource budgets [21, 22]. At the deployment stage, some methods retain the decomposed factors as separate operators during inference, which can introduce additional computational overhead [23]. Therefore, a practical channel pruning framework should connect shared redundancy modeling with channel selection while preserving standard dense convolutions for deployment, without introducing additional factorized operators during inference.

Motivated by the above observations, we propose bi-sparse matrix factorization (BSMF) for structured channel pruning. This method decomposes each convolutional weight matrix into channel coefficients and a shared basis, where each output channel corresponds to a specific coefficient vector. In addition, we impose structured sparsity on the rows of the coefficient factor and the columns of the basis factor. Here, the row-wise norms of the coefficient are used as a scoring criterion to select channels, while adhering to constraints on layer width, connectivity, and computational resources. Before removing the channels, the factors are recombined to reconstruct the convolutional filters, ultimately yielding a compact network composed of standard dense convolutions. Figure 1 illustrates the differences between our proposed BSMF and existing CSMF [10]. It can be seen that CSMF coordinates the update process of filters by combining a common fusion gradient with individual weight increments, while our BSMF constructs channel scores by utilizing shared convolutional filter information, and each output filter is represented by a row of coefficients over the shared basis. Furthermore, the norm of this row determines the priority for retaining the corresponding filter, which provides greater flexibility.

Comparison between BSMF and CSMF, showing channel scoring and pruning

Figure 1. Comparison between our BSMF and CSMF [10], where BSMF scores channels using the row-wise norms of ZAZ_A, while CSMF co-synthesizes filters using fused gradients and individual weight increments.

The main contributions of this paper can be summarized as follows.

  • We propose a novel matrix factorization model by imposing non-convex structured regularization on both the output-channel coefficients and the shared basis.

By leveraging the row-wise norms of the learned coefficients, we derive channel importance scores that account for filter relationships, thereby providing guidance for structured channel pruning.

  • We develop an efficient optimization algorithm based on the alternating minimization framework, where both variables can be solved using ADMM. Although establishing strict theoretical convergence is challenging, we empirically verify the convergence numerically.

  • We demonstrate the superiority of the proposed method through sufficient experiments. Moreover, we visualize the learned importance scores using channel-selection heatmaps and quantify the selection differences by comparing the sets of retained channels via the Jaccard coefficient.

The remainder of this paper is organized as follows. Section 2 reviews the related work. Section 3 presents our mathematical formulation, optimization algorithm, and pruning procedure. Section 4 reports the experimental setup, numerical results, ablation studies, and further discussions. Section 5 concludes the paper.

Related Work

This section reviews recent work on channel pruning, structured pruning, and sparse regularization.

Channel Pruning

Channel scoring must distinguish information content from inter-channel redundancy. In previous studies, filter pruning using high-rank feature map ranks feature responses but may overlook cross-channel duplication [24]. Later, channel independence-based pruning [25] and automatic network pruning via Hilbert-Schmidt independence criterion lasso under information bottleneck principle [26] capture channel dependence or information preservation. However, these methods rely on activation statistics derived from sampling when assigning scores. Although the above methods reduce redundancy, they do not learn sparse channel coefficients within a shared-basis framework. Beyond similarity-based pruning, more recent methods including singular value-driven automated filter pruning [27], adding before pruning (ABP) [28], and iterative pruning with prompt knowledge distillation (IPPKD) [29] preserve information through spectral structure, attention-guided fusion, or distillation, respectively. By contrast, our BSMF uses the row-wise norms of the shared-basis coefficients to score channels, aligning scoring with sparse optimization yet without activation statistics.

Structured Pruning

Structured pruning removes complete filters, and existing methods mainly rely on masks, filter similarity, or factorized representations. Although generative adversarial learning (GAL) [30] and leanflex grouped filter pruning (LeanFlex-GKP) [31] learn structural masks or group configurations, they do not explicitly represent shared filter components. Besides, tensor decomposition-based filter pruning [18] and CSMF [10] characterize inter-filter similarity through decomposition-factor distances and coordinated fusion, respectively. However, these methods do not learn channel-aligned sparse coefficients over a shared basis [23]. Our BSMF addresses these limitations by learning a shared representation for channel selection and recombining its factors before removal, thereby preserving standard dense convolutions at deployment.

Sparse Regularization

Sparse regularization methods also differ in the granularity of the structures they sparsify. Beyond explicit channel regularizations, feature flow regularization promotes structured sparsity through feature propagation [32]. More recently, double sparse factorization learns two element-wise sparse factor matrices [33]. Nevertheless, feature-level or element-wise sparsity does not necessarily eliminate an entire output-channel group. In contrast, our BSMF couples weight reconstruction with ℓ2,p\ell_{2,p}-norm regularization on coefficient rows and ℓ2,q\ell_{2,q}-norm regularization on basis columns. The row regularization suppresses complete output-channel groups, whereas the basis-column regularization encourages a compact shared representation.

Methodology

This section first introduces the proposed formulation and optimization algorithm, followed by the pruning strategy.

Our Formulation

For a convolutional layer, unfold its weights into W∈Rm×nW \in\mathbb{R}^{m \times n} with m=Coutm = C_{\mathrm{out}} and n=Cinh2n = C_{\mathrm{in}}h^2, where CoutC_{\mathrm{out}} and CinC_{\mathrm{in}} denote the number of output and input channels, respectively, and hh represents the width of the convolution filter. For the sake of convenience, we omit the layer index and approximate the weight matrix as

W≈AB,(1)W \approx AB, \tag*{(1)}

where A∈Rm×rA \in\mathbb{R}^{m \times r} and B∈Rr×nB \in\mathbb{R}^{r \times n}, with integer rank 1≤r≤min⁡(m,n)1 \leq r \leq\min(m,n). In fact, each row of AA corresponds to an output channel, while the rows of BB form a shared basis and its columns correspond to the input-filter coordinates.

To enhance pruning capability, we consider the following model

min⁡A,B 12∥W−AB∥F2+λA∥A∥2,pp+λB∥B∥2,qq,(2)\min_{A,B}\ \frac{1}{2}\lVert W-AB\rVert_F^2+\lambda_A\lVert A\rVert_{2,p}^p+\lambda_B\lVert B\rVert_{2,q}^q, \tag*{(2)}

where ∥⋅∥F\lVert\cdot\rVert_F is the Frobenius norm, λA,λB≥0\lambda_A,\lambda_B \geq0 are parameters to weight the two sparsity-inducing regularization terms. We would like to point out that

∥A∥2,pp=∑i=1m∥Ai,:∥2p,∥B∥2,qq=∑j=1n∥B:,j∥2q(3)\lVert A\rVert_{2,p}^p=\sum_{i=1}^{m}\lVert A_{i,:}\rVert_2^p,\qquad\lVert B\rVert_{2,q}^q=\sum_{j=1}^{n}\lVert B_{:,j}\rVert_2^q \tag*{(3)}

where Ai,:A_{i,:} denotes the ii-row of AA and B:,jB_{:,j} denotes the jj-th column of BB. This means that ∥A∥2,pp,∥B∥2,qq\lVert A\rVert_{2,p}^p,\lVert B\rVert_{2,q}^q are row-wise and column-wise structured sparsity, respectively. In addition, p,q∈[0,1)p,q \in[0,1), which includes p,q∈(0,1)p,q \in(0,1) in [34] and the special case p,q=0p,q=0 in [35]. Specifically, when p,q=0p,q=0, they are defined as the counts of nonzero rows and nonzero columns, respectively.

Optimization Algorithm

Problem (3) involves two variables, making direct optimization quite difficult. Inspired by [33], we adopt an alternating minimization scheme by computing

Ak+1=arg⁡min⁡A12∥W−ABk∥F2+λA∥A∥2,pp,Bk+1=arg⁡min⁡B12∥W−Ak+1B∥F2+λB∥B∥2,qq.(4)\begin{aligned} A^{k+1} &= \arg\min_A \frac{1}{2}\lVert W-AB^k\rVert_F^2+\lambda_A\lVert A\rVert_{2,p}^p,\\ B^{k+1} &= \arg\min_B \frac{1}{2}\lVert W-A^{k+1}B\rVert_F^2+\lambda_B\lVert B\rVert_{2,q}^q. \tag*{(4)} \end{aligned}

In what follows, we describe how to solve the subproblems using the alternating direction method of multipliers (ADMM).

A-subproblem

With BkB^k fixed, A-subproblem can be reformulated as

min⁡A,ZA12∥W−ABk∥F2+λA∥ZA∥2,pps.t.A=ZA.(5)\begin{aligned} \min_{A,Z_A}\quad&\frac{1}{2}\lVert W-AB^k\rVert_F^2+\lambda_A\lVert Z_A\rVert_{2,p}^p\\ \text{s.t.}\quad&A=Z_A. \tag*{(5)} \end{aligned}

The corresponding augmented Lagrangian is

LA(A,ZA;UA)=12∥W−ABk∥F2+λA∥ZA∥2,pp+⟨UA,A−ZA⟩+ρ2∥A−ZA∥F2,(6)\mathcal{L}_A(A,Z_A;U_A)=\frac{1}{2}\lVert W-AB^k\rVert_F^2+\lambda_A\lVert Z_A\rVert_{2,p}^p+\langle U_A,A-Z_A\rangle+\frac{\rho}{2}\lVert A-Z_A\rVert_F^2, \tag*{(6)}

where ⟨⋅,⋅⟩\langle\cdot,\cdot\rangle denotes the inner product and ρ>0\rho>0 is the tuning parameter.

  • Update AA according to

Ak+1=arg⁡min⁡A12∥W−ABk∥F2+ρ2∥A−ZAk+UAk/ρ∥F2.(7)A^{k+1}=\arg\min_A\frac{1}{2}\lVert W-AB^k\rVert_F^2+\frac{\rho}{2}\lVert A-Z_A^k+U_A^k/\rho\rVert_F^2. \tag*{(7)}

Setting the gradient with respect to AA to zero yields

(Ak+1Bk−W)Bk,⊤+ρ(Ak+1−ZAk+UAk/ρ)=0,(8)(A^{k+1}B^k-W)B^{k,\top}+\rho(A^{k+1}-Z_A^k+U_A^k/\rho)=0, \tag*{(8)}

where Bk,⊤B^{k,\top} denotes the transpose of BkB^k. Therefore, the closed-form solution of problem (7) is

Ak+1=(WBk,T+ρZAk−UAk)(BkBk,T+ρI)−1,(9)A^{k+1} = \left(WB^{k,\mathsf{T}} + \rho Z_A^k - U_A^k\right)\left(B^kB^{k,\mathsf{T}}+\rho I\right)^{-1}, \tag*{(9)}

where I∈Rr×rI \in\mathbb{R}^{r \times r} is the identity matrix.

  • Update ZAZ_A by

ZAk+1=arg⁡min⁡ZA λA∥ZA∥2,pp+ρ2∥Ak+1−ZA+UAk/ρ∥F2=arg⁡min⁡ZA λA∥ZA∥2,pp+ρ2∥ZA−VAk+1∥F2,(10)\begin{aligned} Z_A^{k+1} &= \underset{Z_A}{\arg\min}\ \lambda_A\lVert Z_A\rVert_{2,p}^p+\frac{\rho}{2}\lVert A^{k+1}-Z_A+U_A^k/\rho\rVert_F^2 \\ &= \underset{Z_A}{\arg\min}\ \lambda_A\lVert Z_A\rVert_{2,p}^p+\frac{\rho}{2}\lVert Z_A-V_A^{k+1}\rVert_F^2, \tag*{(10)} \end{aligned}

where VAk+1=Ak+1+UAk/ρV_A^{k+1}=A^{k+1}+U_A^k/\rho. Thus, the closed-form is

ZAk+1∈prox⁡λAρ∥⋅∥2,pp(VAk+1).(11)Z_A^{k+1}\in\operatorname{prox}_{\frac{\lambda_A}{\rho}\lVert\cdot\rVert_{2,p}^p}\left(V_A^{k+1}\right). \tag*{(11)}

Specifically, for the ii-th row, it becomes

(ZAk+1)i,:∈{prox⁡λAρ∥⋅∥pp((VAk+1)i,:)≐(VAk+1)i,:∥(VAk+1)i,:∥2ωλAρ(∥(VAk+1)i,:∥2),∥(VAk+1)i,:∥2>0,{0},∥(VAk+1)i,:∥2=0,(12)\left(Z_A^{k+1}\right)_{i,:}\in \begin{cases} \operatorname{prox}_{\frac{\lambda_A}{\rho}\lVert\cdot\rVert_p^p}\left(\left(V_A^{k+1}\right)_{i,:}\right) \doteq\frac{\left(V_A^{k+1}\right)_{i,:}}{\left\lVert\left(V_A^{k+1}\right)_{i,:}\right\rVert_2} \omega_{\frac{\lambda_A}{\rho}}\left(\left\lVert\left(V_A^{k+1}\right)_{i,:}\right\rVert_2\right), & \left\lVert\left(V_A^{k+1}\right)_{i,:}\right\rVert_2>0,\\ \{0\}, & \left\lVert\left(V_A^{k+1}\right)_{i,:}\right\rVert_2=0, \end{cases} \tag*{(12)}

and prox is defined in the following lemma.

Lemma 3.1. For x∈Rx\in\mathbb{R} and p∈[0,1)p\in[0,1), consider the proximal operator

prox⁡λ∣⋅∣p(t)=arg⁡min⁡x λ∣x∣p+12(x−t)2\operatorname{prox}_{\lambda\lvert\cdot\rvert^p}(t)=\arg\min_x\ \lambda\lvert x\rvert^p+\frac{1}{2}(x-t)^2
={{0},∣t∣<κ(λ,p),{0,sgn⁡(t)c(λ,p)},∣t∣=κ(λ,p),{sgn⁡(t)ϖp(∣t∣)},∣t∣>κ(λ,p),(13)=\begin{cases} \{0\}, & \lvert t\rvert<\kappa(\lambda,p),\\ \{0,\operatorname{sgn}(t)c(\lambda,p)\}, & \lvert t\rvert=\kappa(\lambda,p),\\ \{\operatorname{sgn}(t)\varpi_p(\lvert t\rvert)\}, & \lvert t\rvert>\kappa(\lambda,p), \end{cases} \tag*{(13)}

where

c(λ,p)=(2λ(1−p))12−p>0,κ(λ,p)=(2−p)λ12−p(2(1−p))p+1p−2,ϖp(t)∈{x∣x−t+λp sgn⁡(x)xp−1=0, x>0}.(14)\begin{aligned} c(\lambda,p) &= (2\lambda(1-p))^{\frac{1}{2-p}} > 0,\\ \kappa(\lambda,p) &= (2-p)\lambda^{\frac{1}{2-p}}(2(1-p))^{\frac{p+1}{p-2}},\\ \varpi_p(t) &\in\{x \mid x-t+\lambda p\,\operatorname{sgn}(x)x^{p-1}=0,\ x>0\}. \tag*{(14)} \end{aligned}

More details can be found in [36].

  • Update UAU_A via

UAk+1=UAk+ρ(Ak+1−ZAk+1).(15)U_A^{k+1}=U_A^k+\rho(A^{k+1}-Z_A^{k+1}). \tag*{(15)}

B-subproblem

With the updated ZAk+1Z_A^{k+1} fixed, BB-subproblem can be written as

min⁡B,ZB12∥W−ZAk+1B∥F2+λB∥ZB∥2,qqs.t.B=ZB,(16)\begin{aligned} \min_{B,Z_B}\quad&\frac{1}{2}\lVert W-Z_A^{k+1}B\rVert_F^2+\lambda_B\lVert Z_B\rVert_{2,q}^q\\ \text{s.t.}\quad&B=Z_B, \tag*{(16)} \end{aligned}

and the augmented Lagrangian is

LB(B,ZB;UB)=12∥W−ZAk+1B∥F2+λB∥ZB∥2,qq+⟨UB,B−ZB⟩+ρ2∥B−ZB∥F2.(17)\mathcal{L}_B(B,Z_B;U_B)=\frac{1}{2}\lVert W-Z_A^{k+1}B\rVert_F^2+\lambda_B\lVert Z_B\rVert_{2,q}^q+\langle U_B,B-Z_B\rangle+\frac{\rho}{2}\lVert B-Z_B\rVert_F^2. \tag*{(17)}

Similar to the above updates, the following iterative forms can be obtained.

  • Update BB by

Bk+1=(ZAk+1,⊤ZAk+1+ρI)−1(ZAk+1,⊤W+ρZBk−UBk),(18)B^{k+1}=(Z_A^{k+1,\top}Z_A^{k+1}+\rho I)^{-1}(Z_A^{k+1,\top}W+\rho Z_B^k-U_B^k), \tag*{(18)}

where where ZAk+1,⊤Z_A^{k+1,\top} denotes the transpose of ZAk+1Z_A^{k+1}.

  • Update ZBZ_B by

ZBk+1∈prox⁡λBρ∥⋅∥2,qq(VBk+1).(19)Z_B^{k+1}\in\operatorname{prox}_{\frac{\lambda_B}{\rho}\lVert\cdot\rVert_{2,q}^q}(V_B^{k+1}). \tag*{(19)}

where VBk+1=Bk+1+UBk/ρV_B^{k+1}=B^{k+1}+U_B^k/\rho and its column is defined as

(ZBk+1):,j∈{prox⁡λBρ∣⋅∣q(∥(VBk+1):,j∥2)⋅(VBk+1):,j∥(VBk+1):,j∥2,∥(VBk+1):,j∥2>0,{0},∥(VBk+1):,j∥2=0.(20)(Z_B^{k+1})_{:,j}\in \begin{cases} \operatorname{prox}_{\frac{\lambda_B}{\rho}\lvert\cdot\rvert^q}\left(\lVert(V_B^{k+1})_{:,j}\rVert_2\right)\cdot \frac{(V_B^{k+1})_{:,j}}{\lVert(V_B^{k+1})_{:,j}\rVert_2}, & \lVert(V_B^{k+1})_{:,j}\rVert_2>0,\\ \{0\}, & \lVert(V_B^{k+1})_{:,j}\rVert_2=0. \end{cases} \tag*{(20)}

Algorithm 1 Alternating Minimization for Solving Problem (3)

Input: Weight W∈Rm×nW \in\mathbb{R}^{m \times n}, rank rr, iteration KK, parameters λA\lambda_A, λB\lambda_B, pp, qq, ρ\rho.

Initialize: Compute Wr=PrΣrQrTW_r = P_r \Sigma_r Q_r^{\mathsf{T}}, let ZA0=PrΣr1/2Z_A^0 = P_r \Sigma_r^{1/2}, ZB0=Σr1/2QrTZ_B^0 = \Sigma_r^{1/2} Q_r^{\mathsf{T}}, UA0U_A^0, UB0=0U_B^0 = 0.

while k<Kk < K do

&nbsp;&nbsp;&nbsp;&nbsp;A-subproblem

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Update Ak+1A^{k+1} using Eq. (9).

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Update ZAk+1Z_A^{k+1} using Eq. (11).

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Update UAk+1U_A^{k+1} using Eq. (15).

&nbsp;&nbsp;&nbsp;&nbsp;end

&nbsp;&nbsp;&nbsp;&nbsp;B-subproblem

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Update Bk+1B^{k+1} using Eq. (18).

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Update ZBk+1Z_B^{k+1} using Eq. (19).

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Update UBk+1U_B^{k+1} using Eq. (21).

&nbsp;&nbsp;&nbsp;&nbsp;end

&nbsp;&nbsp;&nbsp;&nbsp;k←k+1k \leftarrow k + 1.

end

Output: ZAKZ_A^K, ZBKZ_B^K and compute W^\widehat{W} using Eq. (22).

  • Update UBU_B by

UBk+1=UBk+ρ(Bk+1−ZBk+1).(21)U_B^{k+1} = U_B^k + \rho(B^{k+1} - Z_B^{k+1}). \tag*{(21)}

Overall Scheme

After KK outer iterations, the weight matrix is reconstructed as

W^=ZAKZBK.(22)\widehat{W} = Z_A^K Z_B^K. \tag*{(22)}

Finally, Algorithm 1 summarizes the initialization and alternating updates.

Pruning Strategy

This part describes how to use the factor matrix ZAKZ_A^K to rank output channels, construct masks, and then convert the masked model into a compact dense network.

Channel Importance

Let Cl=mlC_l = m_l denote the number of output channels in the ll-th layer. Its importance score is defined as

sl,i=∥ZA,lK(i,:)∥2.(23)s_{l,i} = \lVert Z_{A,l}^K(i,:) \rVert_2. \tag*{(23)}

A smaller score indicates a weaker coefficient group and a lower retention priority. Since the raw score scales may vary across different layers, the scores are normalized independently for each layer. Let ala_l and blb_l be the 5th and 95th percentiles of the scores in the ll-th layer, respectively. When bl−al≥εb_l-a_l \ge\varepsilon, define

s~l,i=clip⁡[0,1](sl,i−albl−al+ε)(24)\widetilde{s}_{l,i}=\operatorname{clip}_{[0,1]}\left(\frac{s_{l,i}-a_l}{b_l-a_l+\varepsilon}\right) \tag*{(24)}

where ε=10−12\varepsilon= 10^{-12} and clip⁡[0,1](x)=min⁡{1,max⁡{0,x}}\operatorname{clip}_{[0,1]}(x)=\min\{1,\max\{0,x\}\}. This robust scaling limits the effect of extreme scores while placing layers on a common range. If bl−al<εb_l-a_l < \varepsilon, all normalized scores for that layer are set to one, thereby avoiding division by an extremely small value.

Mask Selection

Let Lprune⊆LL_{\mathrm{prune}} \subseteq L be the set of layers eligible for output channel pruning, and let N=∑l∈LpruneClN=\sum_{l\in L_{\mathrm{prune}}} C_l be the total number of channels across these layers. For a given nominal pruning ratio π∈[0,1)\pi\in[0,1), the normalized scores of all eligible channels are sorted in ascending order as t(1)≤⋯≤t(N)t_{(1)} \le\cdots\le t_{(N)}. In addition, the initial threshold is defined as the following empirical lower-tail order statistic

jπ=min⁡{⌊πN⌋+1,N},τch=t(jπ).(25)j_\pi=\min\{\lfloor\pi N\rfloor+1,N\},\qquad\tau_{\mathrm{ch}}=t_{(j_\pi)}. \tag*{(25)}

The initial mask is defined as

ml,i(0)={1,s~l,i>τch,0,s~l,i≤τch,(26)m_{l,i}^{(0)}= \begin{cases} 1, & \widetilde{s}_{l,i}>\tau_{\mathrm{ch}},\\ 0, & \widetilde{s}_{l,i}\le\tau_{\mathrm{ch}}, \end{cases} \tag*{(26)}

where 1 indicates that the channel is retained, and 0 indicates that it is marked for removal. When all channel scores are distinct, this threshold results in the removal of the jπj_\pi channels with the lowest scores, and if any channels have scores equal to the threshold, they are also removed. Thus, π\pi determines the nominal lower-tail proportion. Subsequently, the threshold mask must be refined to satisfy constraints regarding layer width and connectivity. Let cl−c_l^{-} and cl+c_l^{+} be the allowed minimum and maximum retained widths, where 1≤cl−≤cl+≤Cl1 \le c_l^{-} \le c_l^{+} \le C_l. If a layer retains fewer than cl−c_l^{-} channels, its highest-scoring discarded channels are restored until the lower bound is met. If it retains more than cl+c_l^{+}, its lowest-scoring retained channels are removed until the upper bound is met. Let ml,im_{l,i} denote the resulting mask and Il={i:ml,i=1}\mathcal{I}_l = \{i : m_{l,i} = 1\} be the corresponding retained index set. At the adjustment boundaries, if multiple indices share the same score, it is possible to derive several valid masks. Furthermore, if removing certain channels would violate the width constraints imposed by residual addition operations, those channels are protected and retained.

Network Construction

For each layer, the dense convolutional filter is reconstructed using Eq. (22), and the output filters indexed by Il\mathcal{I}_l are retained. Simultaneously, the associated normalization parameters and input-channel slices are removed from the subsequent connected layers. After that, the batch-normalization statistics are recalibrated, and the compact model is fine-tuned to recover its accuracy.

Experiments

To validate the superiority of our proposed BSMF, some state-of-the-art pruning methods are selected, including HRank2, CHIP3, APIB4, CORING5, NORTON6, CSMF7, DFPV8, GAL9, LeanFlex-GKP10, ℓ0\ell_011 ℓp\ell_p12, ABP, and IPPKD13.

Subsection 4.1 describes the datasets, models, and training settings. Subsection 4.2 evaluates the proposed BSMF on the CIFAR-10 and ImageNet-1K datasets in terms of accuracy, floating-point operations (FLOPs), and the number of parameters. Subsection 4.3 examines the roles of individual components, including matrix factorization, bi-sparsity optimization, and channel scoring. Subsection 4.4 provides further analysis of the convergence, resulting channel selections, sensitivity to the regularization order, and distribution of effective weights.

Experimental Setup

Datasets and Models

Experiments are conducted on the CIFAR-1014 dataset using ResNet-56, ResNet-110, and VGG-16 models, while the ResNet-50 model is used for the ImageNet-1K15 dataset. Here, the CIFAR-10 dataset consists of 50,000 training images and 10,000 testing images across 10 classes, with an image size of 32×3232 \times32 pixels. The ImageNet-1K dataset contains approximately 1.28 million training images and 50,000 validation images across 1,000 classes.

Training Settings

During the fine-tuning stage, stochastic gradient descent (SGD) is employed. For the CIFAR-10 dataset, the batch size is set to 128 for the ResNet-56, ResNet-110, and VGG-16 models. The learning rates for ResNet-56 at 40% FLOPs reduction, ResNet-110 at approximately 50% FLOPs reduction, and VGG-16 are 0.009, 0.01, and 0.015, respectively. The corresponding momentum values are 0.95, 0.95, and 0.97, with weight decay 5×10−45 \times10^{-4} for all three. For the ImageNet-1K dataset, ResNet-50 is fine-tuned using Nesterov SGD with a batch size of 256, a learning rate of 0.025, a momentum of 0.9, and a weight decay coefficient of 5×10−55 \times10^{-5}. Furthermore, normalization parameters and bias terms are excluded from weight decay. For ResNet-56 and ResNet-110, all comparison methods are reimplemented with their recommended hyperparameters, using the same pretrained checkpoint, pruning targets, and fine-tuning protocol within each architecture. For VGG-16 and ResNet-50, the comparison results are taken directly from the corresponding papers, together with their original training recipes and dense baselines.

Evaluation Metrics

To evaluate the performance of the comparative methods, Top-1 and Top-5 accuracy are used, representing the proportion of cases where the ground-truth label appears within the top 1 and top 5 categories, respectively, based on the predicted scores [37]. A higher value indicates better performance. Additionally, FLOPs and the number of parameters (Params) [24] are considered, where lower values are preferred when accuracy is comparable. For any of the aforementioned resource metrics, the pruning ratio (PR) is calculated as

PR=(1−CprunedCdense)×100%.(27)\mathrm{PR} = \left(1 - \frac{C_{\mathrm{pruned}}}{C_{\mathrm{dense}}}\right) \times100\%. \tag*{(27)}

where CdenseC_{\mathrm{dense}} and CprunedC_{\mathrm{pruned}} are the corresponding counts for the dense baseline and pruned model. A higher PR value indicates a greater relative reduction.

Comparison Results

Table 1 shows that our BSMF achieves the highest accuracy across the evaluated pruning settings. Its gain over the strongest competing method increases from 0.26% and 0.01% under mild pruning to 0.63% and 0.47% under tight pruning on ResNet-56 and ResNet-110, respectively. Its shared-representation-based channel scoring captures inter-filter dependencies, helping BSMF retain informative and complementary channels under tight pruning budgets, thereby retaining a more informative channel subset under the same pruning budget. Under mild pruning, ranking differences have limited impact because many redundant channels remain. Under tight pruning, however, an incorrect ranking causes a larger loss of feature coverage, making channel-selection quality decisive. The ablation results in Subsection 4.3 support this interpretation. Overall, our BSMF achieves a more favorable accuracy-efficiency trade-off under comparable FLOPs and parameter constraints.

ModelMethodTop-1 AccuracyFLOPs (PR)Params (PR)
ResNet-56Baseline93.26%127.09M (0.0%)0.85M (0.0%)
ℓp\ell_p [5]93.87%75.62M (40%)0.56M (34%)
CORING [18]93.79%76.12M (40%)0.47M (45%)
APIB [26]93.92%76.35M (40%)0.53M (37%)
BSMF (Ours)94.18%75.92M (40%)0.44M (48%)
HRank [24]92.49%62.72M (50%)0.49M (42%)
NORTON [23]92.41%63.33M (50%)0.38M (55%)
ℓ0\ell_0 Pruner [20]93.03%64.04M (50%)0.36M (58%)
BSMF (Ours)93.84%62.23M (50%)0.34M (60%)
ResNet-110Baseline93.50%257.09M (0.0%)1.73M (0.0%)
ℓp\ell_p [5]94.41%126.70M (50%)1.11M (36%)
APIB [26]94.45%127.29M (50%)0.88M (49%)
BSMF (Ours)94.46%125.00M (51%)0.89M (48%)
ℓ0\ell_0 Pruner [20]93.26%114.92M (55%)0.69M (60%)
HRank [24]92.96%105.63M (58%)0.72M (60%)
NORTON [23]93.58%102.84M (60%)0.65M (62%)
BSMF (Ours)94.05%102.04M (60%)0.69M (60%)

Table 1. Pruning results for ResNet-56 and ResNet-110 on the CIFAR-10 dataset, where the best and second-best pruned values are shown in bold and underlined, respectively.

Table 2 reports the pruning results of VGG-16 on the CIFAR-10 dataset. Under the high-compression setting, our BSMF achieves an accuracy of 94.48%, the highest among the compared methods, while substantially reducing both computation and model size. This result shows that BSMF preserves classification performance even when pruning is aggressive. Together with the results on ResNet-56 and ResNet-110, it demonstrates that our BSMF is effective on the CIFAR-10 dataset across both sequential architectures such as VGG-16 and residual architectures, rather than being limited to a particular network design.

ModelMethodTop-1 AccuracyFLOPs (PR)Params (PR)
VGG-16Baseline93.96%313.73M (0%)14.98M (0%)
LeanFlex-GKP [31]94.15%178.17M (43%)8.29M (44%)
GAL-0.1 [30]93.42%171.89M (45%)2.67M (82%)
HRank [24]93.43%145.61M (54%)2.51M (83%)
DFPV [19]93.76%139.30M (56%)3.25M (78%)
IPPKD [29]94.24%139.36M (56%)6.89M (54%)
CHIP [25]93.86%131.17M (58%)2.76M (82%)
CORING [18]94.42%131.44M (58%)2.76M (82%)
APIB [26]94.08%125.48M (60%)3.60M (76%)
ℓp\ell_p [5]94.36%123.60M (61%)2.67M (82%)
BSMF (Ours)94.48%122.98M (61%)2.66M (82%)

Table 2. Pruning results for VGG-16 on the CIFAR-10 dataset, where the best and second-best pruned values are shown in bold and underlined, respectively.

Furthermore, Table 3 presents the pruning results for ResNet-50 on the ImageNet-1K dataset. Our BSMF achieves the highest reported Top-1 accuracy of 76.27%, exceeding CHIP by 0.12% while requiring fewer FLOPs (2.07B versus 2.11B) and parameters (13.03M versus 14.23M). This performance is consistent with the coupling of reconstruction and sparse channel selection in BSMF. The reconstruction term penalizes deviations from the original weights, while row-wise regularization suppresses weak channel coefficients in the shared basis. Ranking channels by these regularized coefficient norms carries the learned suppression directly into pruning decisions, aligning channel selection with sparse optimization. Together with the CIFAR-10 results, this finding supports the effectiveness of BSMF across the evaluated architectures and its applicability to large-scale image classification.

Table 3: Pruning results for ResNet-50 on the ImageNet-1K dataset, where the best and second-best pruned values are shown in bold and underlined, respectively.

ModelMethodTop-1 AccuracyTop-5 AccuracyFLOPs (PR)Params (PR)
ResNet-50Baseline [38]76.15%92.87%4.12B (0.0%)25.50M (0.0%)
LeanFlex-GKP [31]75.62%N/A2.76B (33%)17.80M (30%)
GAL-0.5 [30]71.95%90.94%2.33B (43%)21.20M (17%)
ABP [28]74.80%92.36%2.36B (43%)13.01M (49%)
HRank-1 [24]74.98%92.33%2.30B (44%)16.15M (37%)
ℓ0\ell_0 Pruner [20]75.18%92.39%2.27B (45%)13.39M (48%)
ℓp\ell_p [5]76.06%_92.91%_2.22B (46%)17.14M (33%)
CSMF [10]75.81%N/A2.20B (47%)14.92M (42%)
CHIP [25]_76.15%__92.91%_2.11B (49%)14.23M (44%)
DFPV [19]75.89%93.04%1.99B (52%)14.05M (45%)
BSMF (Ours)76.27%92.89%2.07B (50%)_13.03M (49%)_

Table 3. Pruning results for ResNet-50 on the ImageNet-1K dataset, where the best and second-best pruned values are shown in bold and underlined, respectively.

Overall, the results on the CIFAR-10 and ImageNet-1K datasets demonstrate that our BSMF achieves a favorable accuracy-efficiency trade-off across both sequential and residual architectures. The larger accuracy gains under more aggressive pruning budgets on ResNet-56 and ResNet-110 further show its effectiveness when the retained model capacity is limited.

Ablation Study

The ablation study compares factorization, bi-sparse optimization, and channel-scoring variants on the CIFAR-10 dataset, then examines their scores and retained-channel sets to explain the resulting selections.

Component Contributions

Table 4 reports the ResNet-56 ablation results on the CIFAR-10 dataset, where accuracy is reported as the mean and standard deviation over three random seeds. It can be concluded that

Table 4: Component ablation results for ResNet-56 on the CIFAR-10 dataset, where the best and second-best pruned values are shown in bold and underlined, respectively.

MethodFactor.Sparse opt.Channel scoreTop-1 Accuracy (%)FLOPs Red.Params
BaselineNoNoNone93.260%0.85M
BSMF without pruningYesYesNone93.09 ± 0.020%0.85M
ℓ2\ell_2 pruningNoNofilter ℓ2\ell_2-norm93.65 ± 0.2340%0.55M
BSMF + filter ℓ2\ell_2YesYesfilter ℓ2\ell_2-norm93.74 ± 0.1040%0.52M
BSMF + filter ℓ1\ell_1YesYesfilter ℓ1\ell_1-norm_93.91 ± 0.09_40%_0.51M_
Factorization-only pruningYesNoA-row norm93.83 ± 0.1840%0.44M
BSMF (Ours)YesYesA-row norm93.99 ± 0.0740%0.44M

Table 4. Component ablation results for ResNet-56 on the CIFAR-10 dataset, where the best and second-best pruned values are shown in bold and underlined, respectively.

  • Without pruning, BSMF achieves 93.09% accuracy versus 93.26% for the dense baseline, with unchanged FLOPs and 0.85M parameters. The 0.17-percentage-point decrease shows that factorization, bi-sparse optimization, and filter recovery preserve near-baseline performance before channel removal.

  • With identical AA-row scoring, 40% FLOPs reduction, and 0.44M parameters, adding bi-sparse optimization to factorization-only pruning raises mean accuracy from 93.83% to 93.99% and reduces the observed standard deviation from 0.18 to 0.07, supporting its effectiveness beyond factorization alone.

  • At 40% FLOPs reduction, adding factorization and bi-sparse optimization to direct ℓ2\ell_2 pruning raises mean accuracy from 93.65% to 93.74% and reduces parameters from 0.55M to 0.52M. This modest benefit with recovered-filter ℓ2\ell_2 scoring supports the pipeline’s effectiveness and provides a reference for evaluating AA-row scoring.

  • Among the three BSMF scoring variants, filter-ℓ1\ell_1 improves accuracy over filter-ℓ2\ell_2 from 93.74% to 93.91%, while AA-row scoring achieves the highest accuracy (93.99%) with the fewest parameters (0.44M). By scoring the regularized coefficient groups rather than recovered filters, AA-row scoring directly connects sparse optimization to channel selection, yielding the best pruning performance among the evaluated configurations.

Overall, BSMF maintains near-baseline accuracy before pruning, benefits from bi-sparse optimization, and improves conventional norm-based pruning. Notably, with AA-row scoring, it achieves the highest accuracy with the fewest parameters among the evaluated configurations.

Effect of Factorized Channel Scoring

Figure 2 compares baseline filter ℓ2\ell_2-norms (a), optimized AA-row scores (b), and the final retention mask (c) for ResNet-56. The scores in (b), defined in Eq. (23), measure each channel’s coefficient strength in a shared basis. Compared with (a), they show clearer contrast among channels, particularly in the deeper stages, consistent with row-wise regularization suppressing weak coefficient groups. The mask in (c) is constructed from these scores under the network’s structural constraints, linking the regularized factorized representation to channel selection.

Figure 2. Channel-selection heatmaps for ResNet-56, where (a)-(c) show baseline filter ℓ2\ell_2-norms, BSMF A-row scores, and retention masks, respectively.

Comparison of Retained-Channel Sets

To examine how its channel selections differ from those of existing criteria, we compare the retained-channel sets at the same layer using the Jaccard similarity. For layer ii, it is defined as

Ji(A,B)=∣Si(A)∩Si(B)∣∣Si(A)∪Si(B)∣(28)J_i(A,B)=\frac{\left|S_i^{(A)}\cap S_i^{(B)}\right|}{\left|S_i^{(A)}\cup S_i^{(B)}\right|} \tag*{(28)}

where Si(A)S_i^{(A)} and Si(B)S_i^{(B)} are the channel sets retained by methods AA and BB, respectively. Figure 3 shows pairwise similarities for representative layers of ResNet-56 and ResNet-19

Figure 3. Pairwise Jaccard similarity of retained-channel sets on the CIFAR-10 dataset, where the top and bottom rows show ResNet-56 and ResNet-110, respectively.

  1. The mean-similarity plots indicate that BSMF’s selections are generally less similar to those of the other criteria in the middle and deeper layers, while the trend in shallow layers varies between the two networks. This suggests that BSMF can retain channel sets that differ from those selected by conventional criteria, consistent with its factorized channel-scoring mechanism. However, the reasons for the layer-dependent differences in similarity remain unclear and warrant further investigation.

The ablations show higher mean accuracy with bi-sparse optimization than with factorization alone at the tested budget, while the retained-set comparisons reveal selection differences from the evaluated criteria. BSMF’s key feature is the use of regularized, channel-aligned coefficient groups in a learned shared basis for importance scoring, directly linking sparse optimization to channel selection.

Discussion

This section provides a convergence analysis, examines how regularization orders and architectural constraints affect pruning, and uses effective weight distributions to relate factor sparsity to channel selection.

Convergence Analysis

Figure 4(a)-(Figure 4d) shows the objective values of problem (3) during Algorithm 1 for VGG-16, ResNet-56, and ResNet-110 on the CIFAR-10 dataset, and ResNet-50 on the ImageNet-1K dataset, respectively. Across all four settings, the objective decreases rapidly in the first few shrink iterations and then approaches a stable level, with only small subsequent changes. BSMF couples the row- and column-group penalties in Eq. (3) to regularize channel coefficients and the shared basis, respectively. The observed stabilization provides numerical support for optimizing this coupled nonconvex formulation through alternating ADMM.

Convergence curves of BSMF on different models and datasets

Figure 4. Convergence curves of BSMF on different models and datasets, where (a)-(d) show the objective function values.

Sensitivity to Regularization Orders

We evaluate nine pairs (p,q)∈{0,1/2,2/3}2(p, q) \in\{0, 1/2, 2/3\}^{2} for the regularization orders in Eq. (3) on ResNet-56 and ResNet-110 using the CIFAR-10 dataset. Figure 5 reports the mean Top-1 accuracy over three random seeds. The results suggest selecting pp and qq jointly. For ResNet-110, q=1/2q = 1/2 generally performs well, suggesting half-order regularization on the shared-basis side while considering several candidate orders on the coefficient side. Here, pp and qq govern row sparsity in channel coefficients and column sparsity in the shared basis, respectively. Overall, the mean accuracy ranges span 0.28 and 0.24 percentage points for ResNet-56 and ResNet-110, respectively, suggesting relative insensitivity to these orders within the tested range. However, the limited grid and small differences between configurations require further evaluation with more seeds and combined consideration of regularization weights, factorization rank, and fine-tuning settings to determine preferred choices. Their applicability to other models also requires validation.

Figure 5. Sensitivity to pp and qq on the CIFAR-10 dataset, where (a,b) show mean best-checkpoint Top-1 accuracy over three seeds for ResNet-56 and ResNet-110, respectively.

Channel Allocation and Deployment

Figure 6 illustrates channel protection by layer function and pruning sensitivity [22]. VGG-16 (a) compresses earlier layers while preserving its final output width for the classifier, consistent with ThiNet-Tiny’s milder final-layer pruning [39]. ResNet-56 and 110 (b,c) retain block output widths for shortcut compatibility and protect the stem and Stage 1 against early information loss, with stronger protection at the entrances of Stages 2 and 3 for cross-scale transfer and at the end of Stage 3 for classifier inputs. ResNet-50 (d) prunes bottleneck interiors while retaining block outputs and projection shortcuts, as in ThiNet [39]. These constraints preserve feature transfer, residual addition, and classification while concentrating compression on internal computation. Within these constraints, BSMF selects channels using coefficient-row scores learned through bi-sparse shared filter modeling, then recombines the factors and physically removes channels to obtain a compact network with standard dense convolutions.

Figure 6. Channel counts before and after pruning, where counts are shown per VGG-16 layer and summed over the main-branch convolutions of each ResNet block.

Effective Weight Distributions

Figure 7 compares original weights (a)-(d) with effective weights (e)-(h), defined for visualization as Weff,l=diag⁡(gl)Wrec,lW_{\mathrm{eff},l} = \operatorname{diag}(g_l)W_{\mathrm{rec},l}, where Wrec,lW_{\mathrm{rec},l} contains dense filters after factor reconstruction and recovery, and gl,i=0.05+0.95sˉl,ig_{l,i} = 0.05 + 0.95\bar{s}_{l,i} uses the normalized AA-row score in Eq. (24). Across all four architectures, effective weights concentrate more strongly near zero, as illustrated in (a,e). Since each AA row represents an output channel, score-based attenuation visualizes the learned coefficient-row sparsity. Under structural constraints, BSMF converts these scores into pruning masks, translating the sparse structure learned through bi-sparse shared filter modeling into a channel-pruning strategy.

Weight distributions for ResNet-56, ResNet-110, ResNet-50, and VGG-16, showing original and effective weights

Figure 7. Weight distributions, where (a)-(d) show original weights and (e)-(h) show effective weights constructed using normalized A-row scores.

Conclusion

This paper proposes a novel structured channel pruning method called BSMF. By expressing convolutional filters as channel-aligned coefficients over a shared basis and applying structured sparsity regularization terms to both factors, our BSMF assesses channel importance within a representation that captures relationships among filters. In algorithms, an efficient alternating minimization strategy is designed based on the ADMM framework. Experiments on the CIFAR-10 and ImageNet-1K datasets demonstrate competitive classification accuracy with reduced FLOPs and parameter counts. These results confirm the significant potential of our proposed pruning model.

Of course, there remain some limitations. On one hand, the compression stage requires a significant number of iterative computations, thus a faster and more efficient algorithm should be developed. On the other hand, this work focuses exclusively on CNN-based image models and extending it to large language models would also be a meaningful direction.

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China under Grants 62204044 and 12371306, the State Key Laboratory of Integrated Chips and Systems under Grant SKLICS-K202302, and the Shenzhen Nature Science Research Foundation Key Project under Grant ZDYJ20251211121303005.

References

  1. [1]X. Zhao, L. Wang, Y. Zhang, X. Han, M. Deveci, M. Parmar, A review of convolutional neural networks in computer vision, Artificial Intelligence Review 57 (2024) 99.DOI
  2. [2]Z. Li, F. Liu, W. Yang, S. Peng, J. Zhou, A survey of convolutional neural networks: Analysis, applications, and prospects, IEEE Transactions on Neural Networks and Learning Systems 33 (2022) 6999–7019.DOI
  3. [3]D. Liu, Y. Zhu, Z. Liu, Y. Liu, C. Han, J. Tian, R. Li, W. Yi, A survey of model compression techniques: Past, present, and future, Frontiers in Robotics and AI 12 (2025) 1518965.DOI
  4. [4]F. M. A. Khan, M. Hallaq, H. Abou-Zeid, O. Erak, O. Waqar, S. A. Hassan, O. Alhussein, E. Hossain, Model compression for sustainable AI in xG wireless networks: Recent advances, challenges, and future directions, IEEE Communications Surveys & Tutorials (2026).
  5. [5]F. Ji, X. Chen, R. Chu, B. Liu, Network slimming using Lₚ (p < 1) regularization, Pattern Recognition 167 (2025) 111711.DOI
  6. [6]L. Wang, C. Zeng, M. Zhang, J. Wu, L. Nie, Domain aware post training quantization for vision transformers in deployment, Pattern Recognition 171 (2026) 112182.DOI
  7. [7]T. Huang, S. You, F. Wang, C. Qian, C. Xu, DIST+: Knowledge distillation from a stronger adaptive teacher, IEEE Transactions on Pattern Analysis and Machine Intelligence 47 (2025) 5571–5585.DOI
  8. [8]M. Gabor, R. Zdunek, Reduced storage direct tensor ring decomposition for convolutional neural networks compression, Neural Networks 193 (2026) 107994.DOI
  9. [9]H. Cheng, M. Zhang, J. Q. Shi, A survey on deep neural network pruning: Taxonomy, comparison, analysis, and recommendations, IEEE Transactions on Pattern Analysis and Machine Intelligence 46 (2024) 10558–10578.DOI
  10. [10]J. Yan, W. Huang, L. Yin, W. Xie, Y. Zhao, Co-synthesizing multiple similar filters to enhance the feature diversity of pruned network, Pattern Recognition 179 (2026) 113948.DOI
  11. [11]Y. He, L. Xiao, Structured pruning for deep convolutional neural networks: A survey, IEEE Transactions on Pattern Analysis and Machine Intelligence 46 (2024) 2900–2919.DOI
  12. [12]Y. Lian, P. Peng, K. Jiang, W. Xu, Cross-layer importance evaluation for neural network pruning, Neural Networks 179 (2024) 106496.DOI
  13. [13]Y. Wang, S. Guo, J. Guo, J. Zhang, W. Zhang, C. Yan, Y. Zhang, Towards performance-maximizing neural network pruning via global channel attention, Neural Networks 171 (2024) 104–113.DOI
  14. [14]Z. Zhang, C. Cao, F. Zheng, T. Sun, L. Zhao, GCPNet: Gradient-aware channel pruning network with bilateral coupled sampling strategy, Expert Systems with Applications 266 (2025) 126104.DOI
  15. [15]M. Ma, T. Zhang, Z. Wang, Y. Wang, T. Du, W. Li, Global channel pruning with self-supervised mask learning, IEEE Transactions on Circuits and Systems for Video Technology 35 (2025) 2013–2025.DOI
  16. [16]M. Shi, C. Liu, J. Jiao, Q. Ye, Self-supervised feature-gate coupling for dynamic network pruning, Pattern Recognition 154 (2024) 110594.DOI
  17. [17]L. Yang, S. Gu, C. Shen, X. Zhao, Q. Hu, Soft independence guided filter pruning, Pattern Recognition 153 (2024) 110488.
  18. [18]V. T. Pham, Y. Zniyed, T. P. Nguyen, Efficient tensor decomposition-based filter pruning, Neural Networks 178 (2024) 106393.DOI
  19. [19]A. M. Shaikh, Y. Kang, A. Kumar, Y.-b. Zhao, Dynamic filter pruning via unified importance and redundancy, Information Sciences 729 (2026) 122845.DOI
  20. [20]E. Chen, H. Wang, Z. Shi, W. Zhang, Channel pruning for convolutional neural networks using l₀-norm constraints, Neurocomputing 636 (2025) 129925.DOI
  21. [21]L. Gonzalez-Carabarin, I. A. M. Huijben, B. Veeling, A. Schmid, R. J. G. van Sloun, Dynamic probabilistic pruning: A general framework for hardware-constrained pruning at different granularities, IEEE Transactions on Neural Networks and Learning Systems 35 (2024) 733–744.
  22. [22]H. Ye, B. Zhang, T. Chen, J. Fan, B. Wang, Performance-aware approximation of global channel pruning for multitask CNNs, IEEE Transactions on Pattern Analysis and Machine Intelligence 45 (2023) 10267–10284.arxiv.org/abs/2303.11923
  23. [23]V. T. Pham, Y. Zniyed, T. P. Nguyen, Enhanced network compression through tensor decompositions and pruning, IEEE Transactions on Neural Networks and Learning Systems 36 (2025) 4358–4370.DOI
  24. [24]M. Lin, R. Ji, Y. Wang, Y. Zhang, B. Zhang, Y. Tian, L. Shao, HRank: Filter pruning using high-rank feature map, in: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, 2020, pp. 1526–1535.arxiv.org/abs/2002.10179
  25. [25]Y. Sui, M. Yin, Y. Xie, H. Phan, S. Aliari Zonouz, B. Yuan, CHIP: CHAnnel independence-based pruning for compact neural networks, in: M. Ranzato, A. Beygelzimer, Y. Dauphin, P. Liang, J. W. Vaughan (Eds.), Advances in Neural Information Processing Systems, volume 34, Curran Associates, Inc., 2021, pp. 24604–24616.arxiv.org/abs/2110.13981
  26. [26]S. Guo, L. Zhang, X. Zheng, Y. Wang, Y. Li, F. Chao, C. Wu, S. Zhang, R. Ji, Automatic network pruning via Hilbert-Schmidt independence criterion lasso under information bottleneck principle, in: 2023 IEEE/CVF International Conference on Computer Vision, IEEE, 2023, pp. 17412–17423.DOI
  27. [27]V. T. Pham, Y. Zniyed, T. P. Nguyen, Singular values-driven automated filter pruning, Neural Networks 192 (2025) 107857.DOI
  28. [28]G. Tian, Y. Sun, Y. Liu, X. Zeng, M. Wang, Y. Liu, J. Zhang, J. Chen, Adding before pruning: Sparse filter fusion for deep convolutional neural networks via auxiliary attention, IEEE Transactions on Neural Networks and Learning Systems 36 (2025) 3930–3942.
  29. [29]Y. Li, L. Wang, X. Tao, IPPKD: An iterative pruning method with prompt knowledge distillation, Pattern Recognition 180 (2026) 114640.DOI
  30. [30]S. Lin, R. Ji, C. Yan, B. Zhang, L. Cao, Q. Ye, F. Huang, D. Doermann, Towards optimal structured CNN pruning via generative adversarial learning, in: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, 2019, pp. 2785–2794.arxiv.org/abs/1903.09291
  31. [31]J. Zhang, S. Zhong, A. Ye, Z. Liu, S. Zhao, K. Zhou, L. Li, S.-H. Choi, R. Chen, X. Hu, S. Xu, V. Chaudhary, Flexible group count enables hassle-free structured pruning, in: 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, 2025, pp. 4807–4818.
  32. [32]Y. Wu, Y. Lan, L. Zhang, Y. Xiang, Feature flow regularization: Improving structured sparsity in deep neural networks, Neural Networks 161 (2023) 598–613.DOI
  33. [33]V. Boža, V. Macko, Two sparse matrices are better than one: Sparsifying neural networks with double sparse factorization, in: International Conference on Learning Representations, 2025.arxiv.org/abs/2409.18850
  34. [34]Z. Li, F. Nie, J. Bian, D. Wu, X. Li, Sparse PCA via ℓ₂,ₚ-norm regularization for unsupervised feature selection, IEEE Transactions on Pattern Analysis and Machine Intelligence 45 (2023) 5322–5328.
  35. [35]J. Liu, X. Ju, X. Xiu, W. Liu, Bi-level unsupervised feature selection with ℓ₂,₀-norm, Pattern Recognition 181 (2027) 114687.DOI
  36. [36]X. Xiu, C. Huang, P. Shang, W. Liu, Bi-sparse unsupervised feature selection, IEEE Transactions on Image Processing 34 (2025) 7407–7421.DOI
  37. [37]O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, L. Fei-Fei, ImageNet large scale visual recognition challenge, International Journal of Computer Vision 115 (2015) 211–252.
  38. [38]K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: 2016 IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2016, pp. 770–778.arxiv.org/abs/1512.03385
  39. [39]J.-H. Luo, H. Zhang, H.-Y. Zhou, C.-W. Xie, J. Wu, W. Lin, ThiNet: Pruning CNN filters for a thinner net, IEEE Transactions on Pattern Analysis and Machine Intelligence 41 (2019) 2525–2538.DOI

Paper details

vlnc:W_qaxnt6b4
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 10

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents