WorksComputer Science
MATA: Motion-aligned Temporal Adapter
Abstract
Few-shot fine-grained video action recognition requires recognizing novel action classes from only a few labeled examples. The difficulty is that visually similar actions often share actors, scenes, poses, and objects, differing only in the order, timing, and short duration of motion changes. We study this problem under three practical constraints: few tunable parameters, low compute cost, and use of the video modality alone. We propose Motion-aligned Temporal Adapter (MATA), a parameter-efficient adapter on top of a frozen CLIP backbone. MATA specializes CLIP features along the temporal dimension using two components: temporal phase alignment through learnable temporal positional embeddings, and a local motion transition operator through depth-wise convolution. With only ~3.6M tunable parameters, MATA achieves state-of-the-art performance across multiple fine-grained benchmarks while remaining robust to both visual corruptions and temporal disruptions induced by frame dropping. Code is available at https://github.com/prakashchhipa/MATA-Motion-aligned-Temporal-Adapter
Introduction
Few-shot fine-grained action recognition requires separating novel actions given only a few labeled examples. The challenge is that visually similar actions often share actors, scenes, poses, and objects [20, 34], while differing in brief motion changes, timing, and action phases (e.g., preparation, execution, and completion stages). Thus, recognition is both low-data and temporally sensitive: support-query matching must align action phases and specialize temporal features to capture the motion cues that separate fine-grained classes.
Existing few-shot action recognition methods face two practical limitations. Metric- and alignment-based methods such as OTAM [3], TRX [29], STRM [37], and HyRSM [41] improve support-query matching, but lack the transferable priors of foundation models. CLIP-based methods address this limitation, yet often rely on full backbone fine-tuning, additional modalities as in CLIP-FSAR [43], MA-FSAR [45], or heavier meta-learning designs [43, 45]. On the other hand, parameter-efficient video adapters on CLIP reduce tunable parameters, but typically combine spatial and temporal adaptation through multi-branch designs (AIM [48], attention-based modules [2]–[28]), or 3D convolutions (ST-Adapter [26]). This can add computation and can dilute the motion-specific cues that are most critical in few-shot fine-grained action recognition. A third line of work improves temporal reasoning through explicitly extracted motion structure: Trokens [16] builds trajectory tokens in a separate offline stage before training, which adds preprocessing time and storage, and makes recognition sensitive to the quality of the extracted tokens.
This leads to the research question: Can CLIP be adapted for few-shot fine-grained video action recognition using a minimal, compute-efficient mechanism that focuses on temporal dynamics while using only the video modality? Therefore, we study few-shot fine-grained video action recognition under the accuracy–compute efficiency–unimodality objective: i) improving recognition performance with few tunable parameters, ii) being compute efficient, and iii) using only the readily available video modality. This objective follows directly from the challenges above. The model must preserve CLIP’s transferable spatial and semantic priors, recover discriminative temporal evidence from support and query videos, and avoid the cost of full backbone tuning, heavy spatio-temporal adaptation, or additional modality pipelines.
We propose Motion-aligned Temporal Adapter (MATA in Figure 1), a parameter-efficient adapter for a frozen CLIP visual backbone. MATA is motivated by two needs in few-shot fine-grained recognition: support-query action phases must be aligned, and brief motion transitions must be modeled explicitly. MATA addresses the first need through temporal phase alignment (i.e., representing each frame with its relative position in the action sequence), achieved by adding learnable temporal positional embeddings to patch-level bottleneck features so that each visual state is tied to its relative action phase. It addresses the second need through local motion transition modeling, achieved by depth-wise one-dimensional temporal convolution along the frame dimension to capture fine-grained details. The resulting temporal response is projected back to the CLIP token dimension through identity-preserving residual reconstruction, keeping adaptation compact while preserving CLIP’s pretrained priors.

Figure 1. Overview of the MATA few-shot inference pipeline. Support and query videos are encoded by a shared frozen CLIP backbone, and query labels are predicted via prototype matching. Compared to full fine-tuning [43], MATA updates only a small set of parameters, improving accuracy while reducing latency.
With only ∼3.6M tunable parameters, MATA sets a new state of the art on fine-grained few-shot benchmarks under a CLIP backbone, reaching 80.12% and 81.28% 1-shot accuracy on Diving48 and FineGym288 and improving over D2ST [28] by 6.83 and +10.95 points while reducing per-episode latency by . MATA also remains the most accurate method under all five test-time corruptions applied to query clips, showing that focused temporal adaptation can be accurate, compact, fast, and robust from the video modality alone. Our contributions are:
We show that temporal-only adaptation is sufficient for few-shot fine-grained action recognition, effectively capturing short-lived and rapid motion differences between visually similar actions without adapting spatial features.
We introduce MATA, a parameter-efficient temporal adapter that adapts a frozen CLIP visual backbone for few-shot recognition from short video clips, requiring neither additional modalities nor auxiliary cues such as pose or offline preprocessing, while substantially reducing tunable parameters and computational cost.
We establish MATA’s effectiveness and robustness through extensive evaluation across fine-grained action benchmarks, achieving state-of-the-art few-shot recognition while remaining robust to both visual corruptions and temporal disruptions induced by frame dropping.
Related Work
Vision–language pretraining for video actions. CLIP [31] provides open-vocabulary visual representations that have been extended to video by ActionCLIP [46], X-CLIP [25], ViFi-CLIP [32], and CAST [17] through video–text alignment, cross-frame attention, temporal alignment, and spatial–temporal cross-attention. These rely on full backbone fine-tuning or heavy video-specific modules, which is inefficient for fine-grained actions with rapid, subtle motion.
Parameter-efficient video adaptation. Following lightweight adapters in NLP [13], ST-Adapter [26], DUALPATH [27], D²ST [28], and AIM [48] adapt ViT backbones through spatial–temporal modules built from multi-branch or attention-based designs, while Task-Adapter [1] and Task-Adapter++ [2] add task-specific and order-aware alignment. All jointly model spatial appearance and temporal structure. We instead target fine-grained recognition, where short-duration motion is the main discriminative signal, through focused temporal adaptation without additional spatial modeling.
Few-shot action recognition. Classical FSAR methods such as ProtoNet [35], OTAM [3], TRX [29], PAL [55], STRM [37], and HyRSM [41] learn global similarity metrics or temporal alignments, while VPD [12] distills pose supervision into RGB features. Such cues are less suited to actions separated by brief motion evolution rather than coarse appearance. CLIP-based methods exploit stronger semantic priors: CLIP-FSAR [43] fine-tunes the full backbone with temporal transformers, and MA-FSAR [45] freezes it but adds multi-modal and meta-learning components. Others refine matching or temporal structure: TEAM [18] removes explicit temporal alignment, MPL [47] learns match-guided prototypes, TSAM [19] models frame order as a sequence, HMP-Net [5] builds hierarchical meta-prototypes, and EMP-Net [44] reduces cost via multi-level post reasoning. TATs [15] and Trokens [16] instead represent motion as trajectory tokens, which is effective on fine-grained data but requires an offline extraction stage and depends on tracker quality.
MATA retains a frozen CLIP visual backbone and introduces a minimal temporal-only adapter, avoiding attention-heavy or multi-branch architectures, auxiliary modalities, and offline preprocessing, while achieving strong performance on FineGym99, FineGym288, and Diving48.
Method
Problem setting
We consider -way -shot fine-grained action recognition under the episodic setting. Each episode contains a support set
and a query set
The support set provides labeled videos for each of the action classes. For each query video , the recognition objective is to predict
using only the labeled support videos from the same episode. Videos are encoded by a shared frozen CLIP visual backbone with segment-sampled frames, so index denotes a normalized position within the action. A video is represented by the mean of its frame-level class tokens, , which is -normalized. Each class prototype is the mean of its support representations, and a query is assigned to the prototype of highest cosine similarity. Training minimizes the cross-entropy of query predictions over episodes; the CLIP text encoder is not used.
Motion-aligned temporal adapter (MATA)
MATA formulates temporal adaptation (in Figure 1) in a low-dimensional bottleneck space. The key components of MATA are temporal phase alignment, a local motion transition operator, and residual reconstruction. Temporal phase alignment adds frame-order information, the local motion transition operator models short temporal changes, and residual reconstruction maps the temporal response back to the original CLIP token dimension.
Input tokens. Let be the batch size, the number of frames, the spatial patch grid, and the token dimension. The token sequence entering a transformer block is
where the first token is the class token and the remaining tokens are spatial patch tokens:
so only is processed in the bottleneck space and remains unchanged. Since the class token attends to patch tokens in the following block, the update applied to reaches , and hence , through the remaining blocks.
Three properties of the few-shot fine-grained setting motivate this design. First, CLIP provides strong spatial and semantic priors, and adaptation should refine them toward temporal discrimination rather than overwrite them, so MATA updates patch tokens through a residual correction
Second, support-query comparison is reliable only when temporally corresponding action phases are represented consistently, so MATA assigns each frame a learnable phase coordinate before any temporal modeling. Third, fine-grained actions are distinguished by short temporal changes (eg., preparation, onset, release, reversal, delay, or completion), so MATA models transitions among neighboring latent frame states rather than relying on frame-wise appearance alone.
MATA Formulation
Bottleneck temporal projection. Patch tokens are first projected into a compact adapter space:
where is the bottleneck ratio. The tensor is then rearranged so that each spatial location forms an independent temporal sequence:
This operation isolates temporal evolution while keeping spatial locations structurally separated.
Temporal phase alignment. Let denote the bottleneck patch sequence, where each index corresponds to the same spatial patch location observed across frames. MATA utilizes a learnable temporal positional embedding
The phase-aligned representation is
This phase conditioning is achieved through the learnable temporal positional embedding , which injects relative frame-order information directly into each patch-level bottleneck feature. Because is indexed by frame position, it is defined for a fixed and re-learned when the number of sampled frames changes. The novel design choice is that temporal phase is introduced before local transition modeling. Thus, MATA does not treat a patch feature only as a visual state; it represents it as a visual state tied to its relative action phase.
Local motion transition operator. After temporal phase alignment, MATA applies temporal specialization along the frame dimension. For fine-grained action recognition, this is the relevant adaptation axis: visually similar classes are often separated by short temporal changes rather than by global appearance differences. For each spatial location , channel , and time index , the temporal response is
where is the temporal kernel size, is the dilation rate, is the channel-specific temporal weight, and is the padding offset.
MATA formulates temporal specialization as depth-wise one-dimensional filtering in the bottleneck space, yielding only temporal parameters. Per block MATA adds parameters; with , , , over 12 blocks this gives the 3.6M reported in Table 1. The operation is also compute-efficient because it acts locally along frames at fixed spatial locations, avoiding dense temporal attention over all video tokens.
Table 1. Comparison with representative parameter-efficient video adapters. MATA performs temporal-only adaptation while maintaining linear temporal complexity. Cost is per patch for the temporal operator alone, at the width at which each operator acts. : bottleneck width; : kernel size; S: spatial; T: temporal.
Residual reconstruction. The temporal response is mapped back to the original token dimension and added to the input patch tokens:
The up-projection is zero-initialized, so the adapter starts as an identity mapping and gradually learns motion-specific corrections.
Compact form. The complete transformation can be summarized as
Thus, MATA consists of three coupled operations: temporal phase alignment, local motion transition operator, and residual reconstruction.
Table 1 compares MATA among representative parameter-efficient video adapters. Where prior adapters jointly model spatial appearance and temporal structure through multi-branch, 3D-convolutional, or attention-based modules, MATA adapts only the temporal axis, and assigns each frame a learnable temporal phase before modeling local transitions, an ordering that ties each patch feature to its relative action phase and that we isolate in Table 8. Acting in the bottleneck space, its temporal cost is linear rather than quadratic in . MATA thus trades generality for focus, which we show is advantageous when the discriminative signal is short-lived motion rather than appearance.
| Component | Diving48 | FineGym99 | ||
| 1-shot | 5-shot | 1-shot | 5-shot | |
| Local motion transition | 78.69 | 88.76 | 80.02 | 89.26 |
| + Temporal phase alignment | 80.12 | 90.00 | 82.21 | 91.29 |
Table 8. Component analysis of MATA on Diving48 and FineGym99. Each row cumulatively adds the indicated component.
Results and discussion
Implementation details
We evaluate MATA on six challenging few-shot action benchmarks: Diving48 [20], FineGym99, FineGym288 [34], Toyota SmartHome [7], HAA500 [6] and MPII Cooking2 [33]. These datasets cover fine-grained sports, daily activities, human-object interactions, and temporally sensitive action classes. We use the same class splits as Net [36] where applicable for fair comparison.
We follow a 5-way episodic meta-learning protocol with 60,000 training episodes. Only the MATA adapter is optimized with query loss; the CLIP backbone remains frozen. Unless otherwise stated, each video is represented by 8 segment-sampled frames, and training uses batch size of 8, standard video augmentations, and NVIDIA H100 GPUs. At test time, we evaluate 10,000 episodes under 1-shot, 3-shot, and 5-shot settings and report Top-1 accuracy with single-view inference. All experiments use a frozen CLIP ViT-B/16 backbone and 8 input frames by default. MATA uses temporal kernel size , dilation , and bottleneck ratio 0.25.
Establishing state-of-the-art performance across benchmarks
MATA is the most accurate and the most parameter-efficient method in Table 2 in nearly all settings. It improves over the strongest non-foundation baseline by up to +12.62 points at 1-shot with 90% fewer parameters than Net, and over ST, the strongest prior adapter, by up to +10.95 points with half the tunable parameters. Gains are largest in the low-shot regime, where support-query temporal alignment matters most. Trokens is the closest competitor, but on unequal terms: it uses a stronger DINOv2 backbone and trajectory tokens built in an offline stage before training, paying in preprocessing cost, storage, and sensitivity to tracker quality. It leads on FineGym99 at 1-shot; MATA leads at 3- and 5-shot and Diving48 by +12.92 points, with 79% fewer parameters and a single pass over raw frames. Gains extend beyond sports motion. Table 3 shows consistent improvements on Toyota SmartHome, HAA500, and MPII Cooking2, where the cues are object manipulation and interaction order rather than body dynamics. Coarse-grained comparisons are in Sec. D of the supplementary.
| Method | Venue | Tunable Params (M) | Diving48 | FineGym99 | FineGym288 | ||||||
| 1-shot | 3-shot | 5-shot | 1-shot | 3-shot | 5-shot | 1-shot | 3-shot | 5-shot | |||
| TSN+Cosine [38] | ECCV’16 | 25M | 29.72 | 35.69 | 38.81 | 36.63 | 41.35 | 42.76 | 36.68 | 42.87 | 44.82 |
| ProtoNet [35] | NeurIPS’17 | 25M | 59.80 | 74.74 | 77.69 | 62.99 | 74.25 | 77.99 | 58.98 | 70.56 | 74.87 |
| OTAM [3] | CVPR’20 | 23.5M | 41.64 | 47.96 | 49.78 | 48.93 | 52.88 | 55.61 | 47.58 | 53.67 | 55.62 |
| PAL [55] | BMVC’21 | 25M | 50.40 | 57.57 | 61.17 | 58.85 | 65.13 | 68.64 | 58.32 | 63.62 | 65.77 |
| TRX [29] | CVPR’21 | 47.1M | 62.53 | 77.81 | 82.01 | 66.55 | 80.15 | 83.86 | 64.03 | 76.73 | 81.05 |
| STRM [37] | CVPR’22 | 30M | 62.29 | 77.79 | 81.03 | 68.09 | 80.93 | 84.33 | 64.81 | 76.98 | 80.64 |
| HyRSM [41] | CVPR’22 | 65.6M | 61.04 | 76.04 | 81.74 | 67.09 | 78.15 | 82.92 | 62.86 | 73.12 | 78.81 |
| MoLo [42] | CVPR’23 | 89.6M | - | - | - | 73.3 | 80.2 | 84.8 | - | - | - |
| TATs [15] | ECCV’24 | 10.8M | - | - | - | 81.8 | 86.0 | 87.9 | - | - | - |
| M3Net [36] | MM’23 | 30M | 72.35 | 82.50 | 85.44 | 72.70 | 83.03 | 86.71 | 68.66 | 79.73 | 83.05 |
| DINOv2 Backbone | |||||||||||
| Trokens [16]ϕ | ICCV’25 | 17.4M | 67.20 † | 77.14 † | 81.30 † | 84.40 | 88.30 | 89.90 | - | - | - |
| CLIP Backbone | |||||||||||
| D2ST [28]† | ICCV’25 | 7.3M | 73.29 | 81.64 | 83.85 | 76.81 | 85.52 | 87.55 | 70.33 | 76.26 | 82.08 |
| EMP-Net [44]†*` | ECCV’24 | 7.5M | 71.00 | - | 69.20 | - | - | - | - | - | - |
| Task-Adapter [1]* | MM’24 | 7.2M | 68.04 | 72.47 | 75.09 | 64.48 | 67.12 | 70.54 | - | - | - |
| Task-Adapter++ [2]* | PR’26 | 8.0M | 46.45 | 60.65 | 64.17 | 60.82 | 65.43 | 68.12 | 63.07 | - | 72.76 |
| TEAM [18]† | CVPR’25 | 87.0M | 38.50 | 50.40 | 54.90 | - | - | - | - | - | - |
| MPL [47]† | CVPR’26 | 14.0M | 39.29 | 44.61 | 48.7 | - | - | - | - | - | - |
| MATA | - | 3.6M | 80.12 ± 0.15 | 88.31 ± 0.14 | 90.00 ± 0.12 | 82.21 ± 0.18 | 88.97 ± 0.16 | 91.29 ± 0.13 | 81.28 ± 0.17 | 89.39 ± 0.15 | 90.13 ± 0.19 |
Table 2. Few-shot fine-grained action recognition on Diving48, FineGym99, and FineGym288 (%) under 5-way 1/3/5-shot settings. This table compares prior few-shot methods that do not use foundation models with MATA. Best results are in bold.† means results reproduced by us. “*” denotes multi-modal methods incorporating additional text input and a text encoder. ϕ denotes additional preprocessing.
Table 3. Few-shot fine-grained action recognition on additional datasets (%) under 5-way 1/3/5-shot settings.
Achieving the accuracy–compute efficiency–unimodality objective
The accuracy–compute efficiency–unimodality objective asks whether a few-shot video model can improve recognition while remaining compact, fast, and single-stream. Therefore, we compare episodic inference efficiency between MATA and ST under an identical few-shot protocol on a single NVIDIA H100 80GB HBM3 GPU using FP16. FLOPs are measured with fvcore, and latency is measured with CUDA events over timed blocks of consecutive meta-forwards. Each episode uses a 5-way 1-shot setting with one query per class and frames, corresponding to 40 support and 40 query frame-rows per episode.
Table 4 shows that MATA improves both accuracy and runtime efficiency under the same episodic protocol. At comparable nominal FLOPs, MATA is more accurate and cuts per-episode latency from 111.26 to 27.16 ms, a speedup that raises throughput from 9.07 to 36.81 episodes/s and realized compute from 26.92 to 107.53 TFLOP/s.
| Method | 1-shot Acc. (%) ↑ | GFLOPs Episode ↓ | Latency Episode (ms) ↓ | Throughput (episodes/s) ↑ | Effective Compute (TFLOP/s) ↑ |
| ST † [28] | 73.29 | 2990.0 | 111.26 | 9.07 | 26.92 |
| Task-Adapter † [2] | 46.45 | 3335.95 | 68.54 | 14.60 | 48.67 |
| EMP-Net † [44] | 71.0 | 2964.24 | 128.16 | 7.80 | 23.12 |
| MATA | 80.12 | 2921.3 | 27.16 | 36.81 | 107.53 |
| Gain (MATA vs. ST) | ↑ +6.83 | ↓ 2.3% | ↓ 4.10× | ↑ 4.06× | ↑ 3.99× |
Table 4. MATA delivers both higher accuracy and substantially faster few-shot adaptation. The results reveal a clear advantage in both recognition performance and practical inference efficiency, rather than a trade-off between the two. † denotes results reproduced by us; ↓ lower is better and ↑ higher is better.
FLOPs therefore do not predict few-shot inference cost. ST’s deformable attention issues many small, irregular memory accesses, keeping GPU utilization low regardless of arithmetic volume (26.92 TFLOP/s realized). MATA’s temporal operator is a depth-wise convolution over a contiguous frame axis, which maps to dense kernels and leaves the frozen CLIP pathway untouched (107.53 TFLOP/s). The gain comes from what MATA does not add to that pathway, not from doing less arithmetic.
MATA is scalable and applicable to a fully-supervised setting.
MATA is not restricted to the few-shot regime. Under the standard fully-supervised protocol, it reaches 89.7% on Diving48 (Table 5) and 95.1% / 90.0% on FineGym99 and FineGym288 (Table 6), matching or exceeding AIM and DualPath with roughly a third of their tunable parameters and outperforming backbones an order of magnitude larger. MATA also scales with backbone capacity: Table 7 shows consistent gains from ViT-B/16 to ViT-L/14 across all three fine-grained benchmarks, indicating that temporal-only adaptation continues to benefit from stronger spatial priors rather than saturating.
| Method | Venue | Tunable Params (M) | Frames | Top-1 |
| Non-CLIP-based methods | ||||
| ORViT TimeSformer [11] | CVPR’22 | 160.0 | 32 | 88.0 |
| BEVT [39] | CVPR’22 | 88.1 | 16 | 86.7 |
| VideoSwin-B [23] | CVPR’22 | 88.1 | 32 | 81.9 |
| SlowFast R101 [8] | ICCV’19 | 62.8 | 64+16 | 77.6 |
| TQN [50] | CVPR’21 | – | all | 81.8 |
| CLIP-based methods ViT-B/16 | ||||
| BDC-CLIP † [24] | ICML’25 | 125.4 | 16 | 76.6 |
| PeVL [51] | CVPR’24 | 28.0 | 16 | 84.5 |
| DiST † [30] | ICCV’23 | 19.0 | 16 | 76.85 |
| AIM † [48] | ICLR’23 | 11.0 | 32 | 88.9 |
| DualPath † [27] | CVPR’23 | 10.0 | 32 | 88.7 |
| ST Adapter † [26] | NeurIPS’22 | 7.2 | 16 | 84.2 |
| MATA (ours) | – | 3.6 | 16 | 85.8 ± 0.13 |
| MATA (ours) | – | 3.6 | 32 | 89.7 ± 0.15 |
Table 5. Fully-supervised results on Diving48. Top-1 accuracy under the standard fully-supervised protocol, included to show MATA also applies beyond the few-shot setting.
| Method | Venue | Frames | Tunable Params (M) | FineGym99 | FineGym288 |
| Non-CLIP-based methods | |||||
| TQN [50] | CVPR’21 | all | 78.0 | 93.8 | 89.6 |
| TSM [21] | ICCV’19 | 16+16 | 48.6 | 88.4 | 83.1 |
| TRNms [53] | ECCV’18 | 8+8 | 36.6 | 87.8 | 82.0 |
| SlowOnly [8] | ICCV’19 | 32 | 32.4 | 93.46 | 86.8 |
| I3D [4] | CVPR’17 | 16 | 28.0 | 75.6 | 77.3 |
| CLIP-based methods ViT-B/16 | |||||
| PeVL† [51] | CVPR’24 | 16 | 28.0 | 91.2 | 85.4 |
| AIM † [48] | ICLR’23 | 32 | 11.0 | 94.9 | 89.1 |
| MATA (ours) | – | 32 | 3.7 | 95.1 ± 0.13 | 90.0 ± 0.10 |
Table 6. Fully-supervised results on FineGym99 and FineGym288. PeVL [51] results are reported using video modality.
| MATA | Params | Diving48 | FineGym99 | FineGym288 | |||
| 1-shot | 5-shot | 1-shot | 5-shot | 1-shot | 5-shot | ||
| ViT-B/16 | 3.6 | 80.12 | 90.00 | 82.21 | 91.29 | 81.28 | 90.13 |
| ViT-L/14 | 12.5 | 82.65 | 90.00 | 83.07 | 92.53 | 83.00 | 91.64 |
Table 7. Backbone scalability of MATA. Top-1 accuracy (%) under 1-shot and 5-shot settings with frozen CLIP backbones. Gains carry over to the larger backbone across all three benchmarks.
Ablation study
We ablate the core components of MATA to isolate the role of temporal phase alignment. Table 8 shows that adding temporal phase alignment to the local motion transition operator consistently improves both 1-shot and 5-shot accuracy on Diving48 and FineGym99. This indicates that local motion transition modeling alone is not sufficient: fine-grained support-query matching also benefits from knowing where a motion state occurs within the action sequence. Additional ablations on adapter depth, input frames, and supervision are provided in Sec. A in the supplementary. temporal stress test. Corruptions are applied in pixel space before CLIP normalization, and each method is evaluated on 2,000 episodes.
MATA remains robust against distribution shifts.
We evaluate 5-way 1-shot recognition under five test-time corruptions at severity levels . Support videos remain clean and only query clips are corrupted, isolating robustness to degraded test inputs. We consider Gaussian noise, Gaussian blur, JPEG compression, brightness, and frame drop; the latter removes 12.5%/37.5%/62.5% of the 8 frames and fills missing positions by hold-last, providing a
We report mean Corruption Accuracy (mCA) and mean Corruption Error (mCE). Following the mCE metric introduced with ImageNet-C [10], we adapt mCE to our video robustness setting. Let denote the Top-1 accuracy of method on corruption , averaged over severities . We define
where Task-Adapter++ is the reference method. Thus, mCA measures absolute corrupted accuracy, while mCE measures corruption error relative to a common reference, with lower mCE indicating greater robustness.
Table Table 9 shows that MATA achieves the highest accuracy under every corruption, improving mCA over ST by 8.0 points (72.8 vs. 64.8). MATA also obtains the lowest mCE, reducing normalized corruption error from 57.5 for ST to 44.3, a 13.2-point improvement. These results show that MATA’s gains are not limited to clean inputs: its temporal specialization remains effective under both spatial corruptions and temporal degradations.
| Method | Clean Top-1 ↑ | Gaussian noise | Gaussian blur | Frame drop | JPEG compr. | Brightness | mCA (%) ↑ | mCE ↓ |
| Task-Adapter++ | 45.8 | 32.1 | 36.6 | 44.1 | 40.8 | 41.0 | 38.9 | 100.0 |
| D²ST | 73.8 | 55.4 | 62.9 | 66.9 | 68.5 | 70.1 | 64.8 | 57.5 |
| MATA | 80.1 | 63.7 | 70.7 | 74.2 | 77.4 | 77.9 | 72.8 | 44.3 |
Table 9. Test-time robustness on Diving48 (5-way 1-shot); only queries are corrupted. Mean Top-1 (%) over severities ; mCE is normalized by Task-Adapter++ (=100, lower is better).
Why MATA’s temporal modeling captures fine-grained details
We provide qualitative and statistical analyses to explain why MATA consistently outperforms ST. We further include a single-frame blackout sensitivity analysis in Sec. C to measure temporal selectivity under targeted frame removal and nearest-neighbor retrieval across confusing classes in Sec. B in the supplementary.
Qualitative diagnostic on temporal saliency
We analyze temporal saliency to explain why MATA improves fine-grained discrimination through temporally localized adaptation. Fine-grained actions often differ by brief stage transitions and motion ordering, so effective adaptation should be temporally selective rather than uniformly modifying representations across the clip. MATA explicitly biases adaptation toward local temporal cues via a depth-wise 1D temporal convolution and learned temporal positions, aiming to spend limited capacity on the most discriminative frames.
We measure temporal saliency using the per-frame magnitude of the adapter-induced residual update. Let denote the adapter residual update in a transformer block (§ 3). Reshaping to isolate time, we define
where indexes spatial tokens and indexes frames. Intuitively, quantifies how strongly the adapter modifies the representation at time . We report the mean curve with variability (shaded region) across multiple episodes on randomly sampled FineGym classes, aggregating across examples to reflect a generalized trend.
Selective peaks vs. uniform updates. Figure 2 shows that MATA yields a lower baseline update magnitude with sharp temporal peaks concentrated around discriminative stages, consistent with fine-grained cues being short-lived. In contrast, D²ST [28] exhibits a higher and flatter response, suggesting more uniform perturbations across frames. These saliency patterns support the intended mechanism of MATA: it applies small, targeted modifications and activates strongly only near key transition moments, reducing unnecessary feature drift and thereby aiding few-shot generalization.

Figure 2. Temporal saliency via adapter update magnitude: Per-frame update magnitude (Eq. (15)) for MATA and D²ST on randomly sampled FineGym clips (mean ± variability). MATA shows sharper temporal peaks, indicating more selective activation on discriminative motion stages. More comparison examples in Sec. E in the supplementary.
Conclusion and limitations
We presented MATA, a motion-aligned temporal adapter for few-shot fine-grained action recognition with a frozen CLIP backbone. By combining temporal phase alignment with local modeling of short motion transitions, MATA achieves state-of-the-art results on fine-grained benchmarks with only ∼3.6M tunable parameters, lower latency, and the best accuracy under every test-time corruption, without full backbone tuning or auxiliary modalities. MATA is specialized by design: it assumes a fixed frame count and does not target long-range or appearance-driven recognition, leaving long-duration benchmarks and streaming recognition for future work.
Acknowledgment: This work has emanated from research supported by the Kempe Foundations (Kempestiftelserna), JCSMK23-0109, Sweden.
Scaling behavior of MATA across capacity, context, and supervision
We analyze how MATA scales with adaptation capacity, temporal context, and supervision, using as the strongest adapter-based baseline. Across all three axes, MATA consistently exhibits more favorable scaling behavior, indicating that its temporal adaptation mechanism makes more effective use of available resources.
Adaptation depth. Table 10a shows that MATA outperforms even when adapters are placed only in the last few transformer blocks, and this margin persists as more blocks are adapted, indicating that MATA does not require deep or widespread architectural modification to model fine-grained temporal structure.
Temporal context. From Table 10b, both methods improve with more input frames, but MATA consistently gains more as clip length grows, suggesting it leverages extended temporal context more effectively to discriminate fine-grained actions.
| Method | Blocks | Params | Diving48 1-shot | Diving48 5-shot | FineGym99 1-shot | FineGym99 5-shot |
| 6 | 3.6 | 70.01 | 79.84 | 73.84 | 84.80 | |
| MATA | 6 | 1.8 | 78.43 | 87.95 | 79.80 | 89.60 |
| 8 | 4.8 | 71.02 | 81.43 | 75.12 | 86.10 | |
| MATA | 8 | 2.4 | 79.80 | 88.02 | 80.74 | 90.05 |
| 10 | 6.1 | 72.70 | 82.64 | 76.32 | 87.13 | |
| MATA | 10 | 3.0 | 80.05 | 88.83 | 81.66 | 90.97 |
Table 10a. Comparison of MATA and when inserting adapters into the last 6, 8, and 10 CLIP blocks. We report 1-shot and 5-shot accuracy on Diving48 and FineGym99.
| Method | Frames | Diving48 1-shot | Diving48 5-shot | FineGym99 1-shot | FineGym99 5-shot |
| 8 | 73.29 | 83.85 | 76.81 | 87.55 | |
| MATA | 8 | 80.12 | 90.00 | 81.37 | 90.44 |
| 12 | 79.88 | 87.84 | 80.00 | 89.98 | |
| MATA | 12 | 84.64 | 90.68 | 84.14 | 92.33 |
| 16 | 82.75 | 89.82 | 80.41 | 89.38 | |
| MATA | 16 | 85.90 | 92.03 | 88.55 | 93.76 |
Table 10b. Effect of input frame count on MATA and . We report 1-shot and 5-shot accuracy using 8, 12, and 16 frames on Diving48 and FineGym99.
Supervision. Table 11 shows that MATA maintains a stable advantage over from 1-shot to 10-shot settings on both datasets, indicating smooth scaling with increased supervision without signs of overfitting.
| Method | Diving48 | FineGym99 | ||||||||||
| 1-shot | 3-shot | 5-shot | 7-shot | 9-shot | 10-shot | 1-shot | 3-shot | 5-shot | 7-shot | 9-shot | 10-shot | |
| DST | 73.29 | 80.64 | 83.85 | 84.59 | 84.54 | 84.96 | 76.81 | 85.52 | 87.55 | 89.30 | 90.03 | 90.70 |
| MATA | 79.37 | 86.46 | 88.48 | 88.76 | 89.31 | 89.50 | 81.37 | 88.12 | 90.44 | 91.05 | 92.35 | 92.40 |
Table 11. Evaluation of MATA and DST across different shot settings. MATA consistently outperforms DST in all shot regimes.
Nearest-neighbor retrieval across confusing fine-grained classes
We probe how MATA reshapes representation structure using nearest-neighbor (NN) retrieval on confusable FineGym99 classes. We construct a small set of 80 videos from five confusable class pairs and, for each query, retrieve the top- nearest gallery videos under cosine similarity using three embeddings: (i) frozen CLIP (no adapter), (ii) , and (iii) MATA. Each video is represented by a single clip embedding computed with the same 8-frame sampling and preprocessing as episodic evaluation; we -normalize so cosine similarity reduces to a dot product. Retrieval results are visualized as grids (Query | NN-1..NN-) and summarized in Figure 3.

Figure 3. Nearest-neighbor retrieval on confusable FineGym99 classes. For the same query and gallery, MATA yields cleaner top- neighborhoods than frozen CLIP and DST, consistent with improved fine-grained discrimination.
To quantify neighborhood quality, we report Top-5 label purity (Purity@5), defined as the number of same-label neighbors among the top-5 retrieved videos. Table 12 shows that MATA produces substantially cleaner neighborhoods than frozen CLIP and : averaged over 20 queries (5 pairs 4 queries per pair), Purity@5 increases from 0.85/5 (frozen CLIP) to 4.00/5 for MATA, with reaching 2.80/5, confirming MATA’s ability to discriminate among fine-grained confusing classes. This qualitative and statistical analysis supports our claim that MATA reshapes the local neighborhood structure to better align with discriminative action dynamics rather than appearance alone.
| Method | Purity@5 ↑ | Purity rate ↑ | Purity@5 vs. CLIP ↑ |
| Frozen CLIP | 0.85 | 0.17 (0.08–0.27) | 0.00 |
| DST | 2.80 | 0.56 (0.39–0.73) | +1.95 (1.20–2.70) |
| MATA | 4.00 | 0.80 (0.64–0.93) | +3.15 (2.40–3.90) |
Table 12. Nearest-neighbor retrieval quality on FineGym99. Purity@5 counts how many of the top-5 retrieved neighbors share the query label. Parentheses denote 95% bootstrap confidence intervals computed over the same 20 queries.
Single-Frame Blackout Sensitivity Analysis
To understand temporal selectivity, we perform single-frame blackout sensitivity analysis on 8-frame clips from FineGym99. For each frame , we compute the margin drop , where is the decision margin for the original clip , is the margin when frame is replaced with a normalized black frame, is the cosine similarity score for the ground-truth class , and is the score for class . A large magnitude indicates strong temporal impact, while the sign indicates whether frame is helpful (positive ) or harmful (negative ) to the prediction. We evaluate both MATA and on 20 randomly sampled 5-way 1-shot episodes from the test split, computing for each frame and aggregating results across episodes.
Across 20 episodes, MATA demonstrates higher peak sensitivity (mean ) and higher temporal selectivity (standard deviation of : ) compared to ( and , respectively). The aggregate analysis (Figure 4) shows MATA consistently identifies key frames (frames 3–4) with sharp peaks, while exhibits a flatter temporal response across all frames. A representative example (Figure 4, Episode 1) demonstrates MATA’s sharp peak at frame 4 (), indicating strong temporal focus, while DST shows a flatter response with lower peak sensitivity (). These results demonstrate that MATA’s depthwise temporal convolution enables improved temporal selectivity by (1) identifying and focusing on discriminative frames with sharp sensitivity peaks, (2) efficiently distinguishing between critical and non-critical temporal phases, and (3) enabling cheap identification of key frames through margin drop analysis, thereby supporting more efficient temporal modeling compared to attention-based methods that process all frames uniformly.

Figure 4. Single-frame blackout sensitivity analysis. (a) Example episode showing MATA’s sharp peak at frame 4 () vs D²ST’s flatter response (peak ). (b) Aggregate plot (mean over 20 episodes). MATA shows higher peak sensitivity (0.091 vs 0.064) and higher temporal selectivity (0.036 vs 0.025), demonstrating MATA’s focused temporal modeling.
Comparison with other methods on Coarse-grained Datasets
Table 13 reports results on coarse-grained benchmarks, where MATA is competitive but does not lead. This is consistent with its design rather than a shortcoming of it. On HMDB51, UCF101, and Kinetics, classes are largely separable from appearance and scene context, and a single well-chosen frame is often sufficient; the discriminative signal these datasets reward is precisely the spatial and semantic content that MATA leaves untouched, since it adapts only the temporal axis. Methods that adapt spatially, or that recruit the CLIP text encoder as an additional supervisory signal such as CLIP-FSAR [43], MA-FSAR [45], and Task-Adapter++ [2], can exploit that signal directly and hold an advantage here. MATA nevertheless matches or exceeds several of these methods while operating from the video modality alone, without text prompts, pose, depth, or offline preprocessing, and with a fraction of their tunable parameters. Its advantage appears where the discriminative cue is short-lived motion: on the fine-grained benchmarks of Table 2 and Table 3, where classes share actors, scenes, and objects and differ only in the timing and ordering of motion, temporal-only adaptation is the more effective use of a limited parameter budget. We therefore position MATA as a fine-grained specialist rather than a general-purpose few-shot recognizer.
| Method | Reference | Pretrain | HMDB51 1-shot | HMDB51 5-shot | UCF101 1-shot | UCF101 5-shot | Kinetics 1-shot | Kinetics 5-shot | SSv2-Small 1-shot | SSv2-Small 5-shot | SSv2-Full 1-shot | SSv2-Full 5-shot |
| CMN [54] | ECCV’18 | ResNet50 | – | – | – | – | 60.5 | 78.9 | 34.4 | 43.8 | 36.2 | 48.9 |
| OTAM [3] | CVPR’20 | ResNet50 | – | – | – | – | 73.0 | 85.8 | – | – | 42.8 | 52.3 |
| AmeFuNet [9] | MM’20 | ResNet50 | 60.2 | 75.5 | 85.1 | 95.5 | 74.1 | 86.8 | – | – | – | – |
| TRX [29] | CVPR’21 | ResNet50 | – | 75.6 | – | 96.1 | 63.6 | 85.9 | – | 59.1 | – | 64.6 |
| SPRN [40] | MM’21 | ResNet50 | 61.6 | 76.2 | 86.5 | 95.8 | 75.2 | 87.1 | – | – | – | – |
| HyRSM [41] | CVPR’22 | ResNet50 | 60.3 | 76.0 | 83.9 | 94.7 | 73.7 | 86.1 | 40.6 | 56.1 | 54.3 | 69.0 |
| TA2N+Sampler [22] | MM’22 | ResNet50 | 59.9 | 73.5 | 83.5 | 96.0 | 73.6 | 86.2 | – | – | 47.1 | 61.6 |
| MoLo [42] | CVPR’23 | ResNet50 | 60.8 | 77.4 | 86.0 | 95.5 | 74.0 | 85.6 | 41.9 | 56.2 | 55.0 | 69.6 |
| MGCSM [49] | MM’23 | ResNet50 | 61.3 | 79.3 | 86.5 | 97.1 | 74.2 | 88.2 | – | – | – | – |
| SA-CT [52] | MM’23 | ResNet50 | 60.4 | 78.3 | 85.4 | 96.4 | 71.9 | 87.1 | – | – | 48.9 | 69.1 |
| CLIP-FSAR [43] | IJCV’24 | ResNet50 | 69.4 | 80.7 | 92.4 | 97.0 | 90.1 | 92.0 | 52.1 | 55.8 | 58.7 | 62.8 |
| SOAP [14] | MM’24 | ResNet50 | 76.5 | 88.4 | 94.1 | 99.3 | 81.1 | 93.8 | – | – | 61.9 | 79.8 |
| HMP-Net [5] | MM’25 | ResNet50 | 77.6 | 88.7 | 94.9 | 99.1 | 92.0 | 94.1 | 54.6 | 60.9 | 61.2 | 76.5 |
| MPL [47] | CVPR’26 | ResNet50 | 63.3 | 78.1 | 90.1 | 97.5 | 77.4 | 89.5 | - | - | 43.5 | 56.6 |
| Trokens [16] | ICCV’25 | DinoV2 | 69.8 | 82.3 | 94.0 | 97.9 | 82.9 | 91.2 | 53.4 | 68.9 | 61.5 | 76.7 |
| TEAM [18] | CVPR’25 | ResNet50 | 62.8 | 78.4 | 87.2 | 96.2 | 75.1 | 88.2 | - | - | - | - |
| CLIP-Freeze [31] | ICML’21 | ViT-B | 51.4 | 71.0 | 84.6 | 94.5 | 68.2 | 85.3 | 29.5 | 42.5 | 26.8 | 36.3 |
| CLIP-FSAR [43]* | IJCV’24 | ViT-B | 77.1 | 87.7 | 97.0 | 99.1 | 94.8 | 95.4 | 54.6 | - | - | - |
| MA-FSAR [45]* | PR’26 | ViT-B | 83.4 | 87.9 | 97.2 | 99.2 | 95.7 | 96.0 | 59.1 | 64.5 | 63.3 | 72.3 |
| EMP-Net [44]* | ECCV’24 | ViT-B | 76.8 | 85.8 | 94.3 | 98.2 | 89.1 | 93.5 | 57.1 | 65.7 | 63.1 | 73.0 |
| Task-Adapter [1]* | MM’24 | ViT-B | 83.6 | 88.8 | 98.0 | 99.4 | 95.0 | 96.8 | 60.2 | 70.2 | 71.3 | 74.2 |
| Task-Adapter++ [2]* | PR’26 | ViT-B | 86.5 | 92.0 | 98.9 | 99.7 | 96.2 | 97.5 | 63.6 | 73.2 | 72.8 | 75.5 |
| SOAP [14] | MM’24 | ViT-B | 79.3 | 89.8 | 96.8 | 99.5 | 89.9 | 95.5 | – | – | 66.7 | 81.2 |
| HMP-Net [5] | MM’25 | ViT-B | 83.8 | 89.9 | 98.7 | 99.5 | 95.7 | 97.5 | 60.5 | 70.4 | 68.4 | 79.5 |
| AIM [48] | ICLR’23 | ViT-B | 74.2 | 86.9 | 95.4 | 98.5 | 88.4 | 95.3 | 52.8 | 67.5 | 63.7 | 79.2 |
| DUALPATH [27] | CVPR’23 | ViT-B | 74.9 | 87.5 | 95.7 | 98.7 | 88.8 | 95.4 | 53.5 | 68.1 | 64.5 | 79.8 |
| ST-Adapter [26] | NeurIPS’22 | ViT-B | 74.1 | 87.3 | 95.9 | 98.9 | 88.5 | 95.1 | 53.1 | 68.0 | 64.2 | 79.5 |
| D2ST [28] | ICCV’25 | ViT-B | 77.1 | 88.2 | 96.4 | 99.1 | 89.3 | 95.5 | 55.0 | 69.3 | 66.7 | 81.9 |
| TSAM [19] | AAAI’25 | ViT-B | 84.5 | 88.9 | 98.3 | 99.3 | 96.2 | 97.1 | 60.5 | 66.7 | 65.8 | 74.6 |
| MPL [47] | CVPR’26 | ViT-B | 68.0 | 84.9 | 93.0 | 98.5 | 83.7 | 93.8 | - | - | 40.1 | 55.0 |
| TEAM [18] | CVPR’25 | ViT-B | 70.9 | 85.5 | 94.5 | 98.8 | 83.3 | 92.9 | 47.2 | 63.1 | - | - |
| MATA | – | ViT-B | 74.10 | 86.54 | 94.82 | 98.50 | - | - | 55.2 | 73.2 | 58.7 | 74.8 |
Table 13. Comparison with current SOTA few-shot action recognition methods on HMDB51, UCF101, Kinetics, SSv2-Small, and SSv2-Full datasets. “*” denotes multi-modal methods incorporating additional text input and a text encoder.
More examples on Temporal Saliency of Adapter Updates
This section shows more examples for temporal saliency to further strengthen MATA’s temporal capacity.

Figure 5. Temporal saliency examples.
References
- [1]Congqi Cao, Yueran Zhang, Yating Yu, Qinyi Lv, Lingtong Min, and Yanning Zhang. Task-adapter: task-specific adaptation of image models for few-shot action recognition. In Proceedings of the ACM International Conference on Multimedia (ACM MM), pages 9038–9047, 2024. 3, 6, 15arxiv.org/abs/2408.00249
- [2]Congqi Cao, Peiheng Han, Yueran Zhang, Yating Yu, Qinyi Lv, Lingtong Min, and Yanning Zhang. Task-adapter++: Task-specific adaptation with order-aware alignment for few-shot action recognition. Pattern Recognition, page 113237, 2026. 3, 6, 7, 14, 15
- [3]Kaidi Cao, Jingwei Ji, Zhangjie Cao, Chien-Yi Chang, and Juan Carlos Niebles. Few-shot video classification via temporal alignment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10618–10627, 2020. 1, 3, 6, 15arxiv.org/abs/1906.11415
- [4]Joao Carreira and Andrew Zisserman. Quo vadis, action recognition? a new model and the kinetics dataset. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 21–26, 2017. 7arxiv.org/abs/1705.07750
- [5]Xiaoyu Chen, Yigang Cen, Wanru Xu, Yue Zhang, Yi Jin, Yidong Li, and Linna Zhang. Hierarchical meta-prototypes network for few-shot action recognition. In Proceedings of the 33rd ACM International Conference on Multimedia, page 2643–2652, New York, NY, USA, 2025. Association for Computing Machinery. 3, 15DOI
- [6]Jihoon Chung, Cheng-hsin Wu, Hsuan-ru Yang, Yu-Wing Tai, and Chi-Keung Tang. HAA500: human-centric atomic action dataset with curated videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 13465–13474, 2021. 5arxiv.org/abs/2009.05224
- [7]Srijan Das, Rui Dai, Michal Koperski, Luca Minciullo, Lorenzo Garattoni, Francois Bremond, and Giampiero Francesca. Toyota smarthome: Real-world activities of daily living. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 833–842, 2019. 5DOI
- [8]Christoph Feichtenhofer, Haoqi Fan, Jitendra Malik, and Kaiming He. Slowfast networks for video recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 6202–6211, 2019. 7arxiv.org/abs/1812.03982
- [9]Yuqian Fu, Li Zhang, Junke Wang, Yanwei Fu, and Yu-Gang Jiang. Depth guided adaptive meta-fusion network for few-shot video recognition. In Proceedings of the 28th ACM international conference on multimedia, pages 1142–1151, 2020. 15arxiv.org/abs/2010.09982
- [10]Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. arXiv preprint arXiv:1903.12261, 2019. 6
- [11]Roei Herzig, Elad Ben-Avraham, Karttikeya Mangalam, Amir Bar, Gal Chechik, Anna Rohrbach, Trevor Darrell, and Amir Globerson. Object-region video transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3148–3159, 2022. 7arxiv.org/abs/2110.06915
- [12]James Hong, Matthew Fisher, Michaël Gharbi, and Kayvon Fatahalian. Video pose distillation for few-shot, fine-grained sports action recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 9254–9263, 2021. 3arxiv.org/abs/2109.01305
- [13]Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attarian, and Sylvain Gelly. Parameter-efficient transfer learning for nlp. In Proceedings of the International Conference on Machine Learning (ICML), pages 2790–2799, 2019. 3arxiv.org/abs/1902.00751
- [14]Wenbo Huang, Jinghui Zhang, Xuwei Qian, Zhen Wu, Meng Wang, and Lei Zhang. Soap: Enhancing spatio-temporal relation and motion information capturing for few-shot action recognition. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 4572–4580, 2024. 15DOI
- [15]Pulkit Kumar, Namitha Padmanabhan, Luke Luo, Sai Saketh Rambhatla, and Abhinav Shrivastava. Trajectory-aligned space-time tokens for few-shot action recognition. In European Conference on Computer Vision (ECCV), pages 474–493. Springer, 2024. 3, 6arxiv.org/abs/2407.18249
- [16]Pulkit Kumar, Shuaiyi Huang, Matthew Walmer, Sai Saketh Rambhatla, and Abhinav Shrivastava. Trokens: Semantic-aware relational trajectory tokens for few-shot action recognition. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV), pages 13544–13556. IEEE, 2025. 1, 3, 6, 15arxiv.org/abs/2508.03695
- [17]Dongho Lee, Jongseo Lee, and Jinwoo Choi. Cast: cross-attention in space and time for video action recognition. Advances in Neural Information Processing Systems, 36: 79399–79425, 2023. 3arxiv.org/abs/2311.18825
- [18]SuBeen Lee, WonJun Moon, Hyun Seok Seong, and Jae-Pil Heo. Temporal alignment-free video matching for few-shot action recognition. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5412–5421. IEEE, 2025. 3, 6, 15arxiv.org/abs/2504.05956
- [19]Bozheng Li, Mushui Liu, Gaoang Wang, and Yunlong Yu. Frame order matters: A temporal sequence-aware model for few-shot action recognition. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 18218–18226, 2025. 3, 15arxiv.org/abs/2408.12475
- [20]Yingwei Li, Yi Li, and Nuno Vasconcelos. RESOUND: towards action recognition without representation bias. In Proceedings of the European Conference on Computer Vision (ECCV), pages 513–528, 2018. 1, 5DOI
- [21]Ji Lin, Chuang Gan, and Song Han. Tsm: Temporal shift module for efficient video understanding. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 7083–7093, 2019. 7arxiv.org/abs/1811.08383
- [22]Huabin Liu, Weixian Lv, John See, and Weiyao Lin. Task-adaptive spatial-temporal video sampler for few-shot action recognition. In Proceedings of the 30th ACM International Conference on Multimedia, pages 6230–6240, 2022. 15arxiv.org/abs/2207.09759
- [23]Ze Liu, Jia Ning, Yue Cao, Yixuan Wei, Zheng Zhang, Stephen Lin, and Han Hu. Video swin transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3202–3211, 2022. 7arxiv.org/abs/2106.13230
- [24]Fei Long, Xiaoou Li, Jiaming Lv, Haoyuan Yang, Xianjun Cheng, and Peihua Li. BDC-CLIP: Brownian distance covariance for adapting CLIP to action recognition. In Proceedings of the International Conference on Machine Learning (ICML), 2025. 7
- [25]Bolin Ni, Hou wen Peng, Minghao Chen, Songyang Zhang, Gao feng Meng, Jianlong Fu, Shiming Xiang, and Haibin Ling. Expanding language-image pretrained models for general video recognition. In Proceedings of the European Conference on Computer Vision (ECCV), pages 1–18, 2022. 3arxiv.org/abs/2208.02816
- [26]Junting Pan, Ziyi Lin, Xiatian Zhu, Jing Shao, and Hong-sheng Li. ST-adapter: parameter-efficient image-to-video transfer learning. Advances in Neural Information Processing Systems, 35:26462–26477, 2022. 1, 3, 4, 7, 15
- [27]Jungin Park, Jiyoung Lee, and Kwanghoon Sohn. Dual-path adaptation from image to video transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2203–2213, 2023. 3, 4, 7, 15arxiv.org/abs/2303.09857
- [28]Wenjie Pei, Qizhong Tan, Guangming Lu, Jiandong Tian, and Jun Yu. D2ST-adapter: disentangled and deformable spatio-temporal adapter for few-shot action recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 11317–11326, 2025. 1, 2, 3, 4, 6, 7, 8, 15arxiv.org/abs/2312.01431
- [29]Toby Perrett, Alessandro Masullo, Tilo Burghardt, Majid Mirmethi, and Dima Damen. Temporal-relational crosstransformers for few-shot action recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 475–484, 2021. 1, 3, 6, 15arxiv.org/abs/2101.06184
- [30]Zhiwu Qing, Shiwei Zhang, Ziyuan Huang, Yingya Zhang, Changxin Gao, Deli Zhao, and Nong Sang. Disentangling spatial and temporal learning for efficient image-to-video transfer learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 13934–13944, 2023. 1, 2arxiv.org/abs/2309.07911
- [31]Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In Proceedings of the International Conference on Machine Learning (ICML), pages 8748–8763, 2021. 2, 15arxiv.org/abs/2103.00020
- [32]Hanoona Rasheed, Muhammad Uzair Khattak, Muhammad Maaz, Salman Khan, and Fahad Shahbaz Khan. Fine-tuned CLIP models are efficient video learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6545–6554, 2023. 3arxiv.org/abs/2212.03640
- [33]Marcus Rohrbach, Anna Rohrbach, Michaela Regneri, Sikandar Amin, Mykhaylo Andriluka, Manfred Pinkal, and Bernt Schiele. Recognizing fine-grained and composite activities using hand-centric features and script data. International Journal of Computer Vision, pages 1–28, 2015. 5DOI
- [34]Dian Shao, Yue Zhao, Bo Dai, and Dahua Lin. FineGym: a hierarchical video dataset for fine-grained action understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2616–2625, 2020. 1, 5arxiv.org/abs/2004.06704
- [35]Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. Advances in neural information processing systems, 30, 2017. 3, 6arxiv.org/abs/1703.05175
- [36]Hao Tang, Jun Liu, Shuanglin Yan, Rui Yan, Zechao Li, and Jinhui Tang. M3Net: Multi-view encoding, matching, and fusion for few-shot fine-grained action recognition. In Proceedings of the ACM International Conference on Multimedia (ACM MM), pages 1719–1728, 2023. 5, 6arxiv.org/abs/2308.03063
- [37]Anirudh Thatipelli, Sanath Narayan, Salman Khan, Rao Muhammad Anwer, Fahad Shahbaz Khan, and Bernard Ghanem. Spatio-temporal relation modeling for few-shot action recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 19958–19967, 2022. 1, 3, 6arxiv.org/abs/2112.05132
- [38]Limin Wang, Yuanjun Xiong, Zhe Wang, Yu Qiao, Dahua Lin, Xiaoou Tang, and Luc Van Gool. Temporal segment networks: Towards good practices for deep action recognition. In Proceedings of the European Conference on Computer Vision (ECCV), pages 20–36, 2016. 6arxiv.org/abs/1608.00859
- [39]Rui Wang, Dongdong Chen, Zuxuan Wu, Yinpeng Chen, Xiang Dai, Mengchen Liu, Yu-Gang Jiang, Luowei Zhou, and Lu Yuan. BEVT: Bert pretraining of video transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 14733–14743, 2022. 7arxiv.org/abs/2112.01529
- [40]Xiao Wang, Weirong Ye, Zhongang Qi, Xun Zhao, Guangge Wang, Ying Shan, and Hanzi Wang. Semantic-guided relation propagation network for few-shot action recognition. In Proceedings of the 29th ACM international conference on multimedia, pages 816–825, 2021. 15DOI
- [41]Xiang Wang, Shiwei Zhang, Zhiwu Qing, Mingqian Tang, Zhengrong Zuo, Changxin Gao, Rong Jin, and Nong Sang. Hybrid relation guided set matching for few-shot action recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 19948–19957, 2022. 1, 3, 6, 15arxiv.org/abs/2204.13423
- [42]Xiang Wang, Shiwei Zhang, Zhiwu Qing, Changxin Gao, Yingya Zhang, Deli Zhao, and Nong Sang. Molo: Motion-augmented long-short contrastive learning for few-shot action recognition. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 18011–18021. IEEE, 2023. 6, 15arxiv.org/abs/2304.00946
- [43]Xiang Wang, Shiwei Zhang, Jun Cen, Changxin Gao, Yingya Zhang, Deli Zhao, and Nong Sang. CLIP-guided prototype modulating for few-shot action recognition. International Journal of Computer Vision, 132(6):1899–1912, 2024. 1, 2, 3, 14, 15arxiv.org/abs/2303.02982
- [44]Cong Wu, Xiao-Jun Wu, Linze Li, Tianyang Xu, Zhenhua Feng, and Josef Kittler. Efficient few-shot action recognition via multi-level post-reasoning. In Proceedings of the European Conference on Computer Vision (ECCV), pages 38–56, 2024. 3, 6, 7, 15DOI
- [45]Jiazheng Xing, Jian Zhao, Chao Xu, Mengmeng Wang, Guang Dai, Yong Liu, Jingdong Wang, and Xuelong Li. Ma-fsar: Multimodal adaptation of CLIP for few-shot action recognition. Pattern Recognition, 169:111902, 2026. 1, 3, 14, 15
- [46]Mengmeng Xu, Han Zhang, Jian Xu, et al. ActionCLIP: a new paradigm for video action recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021. 2
- [47]Feng Yang, Jie Zhao, Fulin Luo, Anyong Qin, Tiecheng Song, Yue Zhao, Chenqiang Gao, and Junwei Han. Mpl: Match-guided prototype learning for few-shot action recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 34439–34448, 2026. 3, 6, 15
- [48]Taojian Yang, Yi Zhu, Yusheng Xie, Aston Zhang, Chen Chen, and Mu Li. AIM: adapting image models for effi- cient video action recognition. In International Conference on Learning Representations (ICLR), 2023. 1, 3, 4, 7, 15
- [49]Tianwei Yu, Peng Chen, Yuanjie Dang, Ruohong Huan, and Ronghua Liang. Multi-speed global contextual subspace matching for few-shot action recognition. In Proceedings of the 31st ACM International Conference on Multimedia, pages 2344–2352, 2023. 15
- [50]Chuhan Zhang, Ankush Gupta, and Andrew Zisserman. Temporal query networks for fine-grained video understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4486–4496, 2021. 7arxiv.org/abs/2104.09496
- [51]Haosong Zhang, Mei Chee Leong, Liyuan Li, and Weisi Lin. PEVL: pose-enhanced vision-language model for fine-grained human action recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 18857–18867, 2024. 7DOI
- [52]Yilun Zhang, Yuqian Fu, Xingjun Ma, Lizhe Qi, Jingjing Chen, Zuxuan Wu, and Yu-Gang Jiang. On the importance of spatial relations for few-shot action recognition. In Proceedings of the 31st ACM international conference on multimedia, pages 2243–2251, 2023. 15arxiv.org/abs/2308.07119
- [53]Bolei Zhou, Alex Andonian, Aude Oliva, and Antonio Torralba. Temporal relational reasoning in videos. In Proceedings of the European Conference on Computer Vision (ECCV), pages 803–818, 2018. 7arxiv.org/abs/1711.08496
- [54]Linchao Zhu and Yi Yang. Compound memory networks for few-shot video classification. In European Conference on Computer Vision (ECCV), pages 782–797. Springer, 2018. 15DOI
- [55]Xiatian Zhu, Antoine Toisoul, Juan-Manuel Perez-Rua, Li Zhang, Brais Martinez, and Tao Xiang. Few-shot action recognition with prototype-centered attentive learning. arXiv preprint arXiv:2101.08085, 2021. 3, 6arxiv.org/abs/2101.08085