GradCO: Behavior-Aware Gradient-Conflict Offloading for Text-Based Person Anomaly Search

Introduction

Text-to-Image Person Retrieval (TIPR) is a key technology in the field of multimodel learning ([1, 2, 3, 4, 5, 6, 7, 8, 9]). Recent multimodal large language models further address data selection and modality balance ([10, 11, 12, 13]). As illustrated on the left side of Figure 1 (a), this technique allows users to locate target pedestrians using natural language descriptions, offering greater flexibility and intuitiveness compared to traditional image-based ReID. However, existing person retrieval research has largely concentrated on matching the static appearance of individuals, such as “a man wearing a red T-shirt,” while often overlooking the comprehension of dynamic behaviors, particularly abnormal behaviors that are crucial in safety-critical scenarios ([14, 15, 17, 19, 20, 21] (Bialas 2010)).

To bridge the behavioral blind spot in ReID and the identity neglect in traditional visual anomaly detection [22, 23, 24, 25] (Flaborea et al. 2023), Yang et al. (Yang et al., 2025) introduced the Text-based Person Anomaly Search (TPAS) task (Yang et al., 2025; Zhang et al., 2026; Xie et al., 2026). As shown in right side of Figure 1 (a), TPAS aims to retrieve a specific pedestrian from a large-scale image database using a natural language query describing both appearance and behavior (normal or abnormal), e.g., “the man in the blue jacket who is falling down” (Yang et al., 2026).

TPAS requires both fine-grained appearance discrimination from person re-identification (ReID) and subtle behavior perception from anomaly detection, posing a fundamental invariance–sensitivity conflict. Specifically, the model must learn appearance features invariant to behavioral changes for identity recognition, while remaining sensitive to postural variations for behavior recognition. A natural solution is to disentangle the feature learning of “invariant” appearance and “sensitive” behavior, for example, by using independent identity and behavior expert streams. However, this seemingly reasonable disentanglement scheme, under the unique data distribution of the TPAS task, inevitably induces the following two deeper technical challenges.

C1: Gradient Conflict in Granular Alignment. We observe that hard negative samples in the TPAS task (e.g., “the same man in a blue jacket who is standing”) are designed to share identity with the positive sample (e.g., “the man in the blue jacket who is falling down”). A conventional contrastive objective therefore receives competing signals in an appearance-sensitive stream: identity similarity favors preserving the pair’s proximity, whereas behavior mismatch favors separating it. This conflict is induced by the joint retrieval objective and does not imply that the corresponding expert stream is assigned an explicit semantic label. As illustrated in Figure 1(b), the COL-enhanced objective exhibits smaller loss fluctuations than standard InfoNCE on PAB, indicating improved optimization stability rather than directly proving semantic disentanglement.

Related Work

C2: Aggregation Distortion of the Alignment Paradigm. Furthermore, another traditional “Aggregate-then-Align” paradigm is also incompatible with the disentanglement scheme. As illustrated in Figure 1(c), if the identity and behavior features are forcibly aggregated (e.g., via concatenation or summation) into a single global vector, the feature strength of the appearance signal can be significantly greater than that of the behavior signal. This aggregation can cause the strong identity (appearance) signal to overwhelm the weak yet critical behavior signal. Moreover, this context-agnostic global feature cannot respond to the dynamic intent of the text (for instance, when querying “falling down,” the focus should be primarily on behavior features), leading to an “intent mismatch.”

To address these challenges, we propose GradCO (Behavior-Aware Gradient-Conflict Offloading), a TPAS framework that combines three complementary designs. (a) Behavior-Aware Mixture-of-Experts (B-MoE) forms appearance, normal-behavior, and abnormal-behavior streams, whose relative roles emerge from retrieval training without expert-specific semantic labels. (b) Conflict Offload Learning (COL) introduces pair-conditioned offload anchors on the text side to redirect conflict-bearing gradients from hard negatives away from the main text representations. (c) Hierarchical Intent Alignment (HIA) uses text-conditioned weights to aggregate expert similarities and features, mitigating signal domination and intent mismatch.

The main contributions are summarized as:

• We identify and analyze two TPAS-specific challenges: gradient conflict from same-identity, different-behavior hard negatives and query-agnostic aggregation distortion.

• We propose GradCO, which integrates B-MoE, COL, and HIA for structured expert representation, gradient offloading, and intent-aware aggregation.

• Experiments on five TPAS/TIPR benchmarks demonstrate improved retrieval, corruption robustness, and out-of-distribution generalization, supported by ablation and visualization results.

Related Work

Text-based Person Anomaly Search

Text-to-Image Person Retrieval (TIPR) retrieves pedestrians from natural-language descriptions [1, 2, 8] (Chen et al. 2026; Gao et al. 2021; Song et al. 2022; Li et al. 2022; Song et al. 2023; Niu et al. 2020; Qian et al. 2017; Wang et al. 2020; Li et al. 2025; Song et al. 2026). Existing work uses cross-modal attention [3, 4, 5] (Shao et al. 2022; Shu et al. 2022; Wang et al. 2022; Li et al. 2026a), auxiliary cues [6] (Jing et al. 2020; Li et al. 2026b), task-specific objectives (Sarafianos et al., 2019; Liu et al., 2019; Zheng et al., 2020), and vision-language pretraining or large language models [9, 13] (Shu et al. 2022; Bi et al. 2026a; Bai et al. 2023; Song et al. 2024; Han et al. 2021; Hu et al. 2025; Bi et al. 2026b; Yan et al. 2023; Bi et al. 2026c; Li et al. 2023a) to align images and text [20] (Li et al. 2026c; Chen et al. 2025; Li et al. 2026d). In parallel, Video Anomaly Detection (VAD) identifies deviations from normal patterns through latent-, skeleton-, and reconstruction-based paradigms [22, 23, 24] (Yuan and Kitani 2020; Nguyen et al. 2019; Wu et al. 2025; Tang et al. 2025; Gong et al. 2019; Chong and Tay 2017). TPAS unifies these settings by requiring retrieval conditioned on both pedestrian appearance and normal or anomalous behavior (Yang et al., 2025). Yet TIPR and VAD methods generally optimize identity matching and behavior recognition separately, leaving their joint alignment unresolved. GradCO addresses this gap by learning a query-responsive joint representation that preserves identity-consistent appearance cues while retaining behavior-sensitive evidence for TPAS retrieval.

Mixtures of Experts

Mixture-of-Experts (MoE) architectures route inputs to multiple experts and have been applied to large-model optimization, multitask learning, transfer learning, and ReID (Jacobs et al., 1991; Fedus et al., 2022; Chen et al., 2023; Zoph et al., 2022; Jaiswal et al., 2025; Li et al., 2023b, 2026e; Xu et al., 2022; Liu et al., 2024). ReID MoE methods mainly model modality variation, biometric-factor fusion, or domain generalization (Liu et al., 2024; Feng et al.; Su et al., 2025). GradCO extends MoE to this setting by redirecting conflicting optimization signals during granular alignment and making expert aggregation responsive to text intent, so identity consistency and behavior sensitivity can coexist under TPAS hard negatives.

Methodology

GradCO organizes coupled appearance, normal-behavior, and anomalous-behavior semantics into parallel expert streams, while offload anchors mitigate alignment-related gradient conflicts. As shown in Figure 2, GradCO consists of three jointly trained components: (a) Behavior-Aware Mixture-of-Experts (B-MoE, detailed in § 3.2); (b) Conflict Offload Learning (COL, detailed in § 3.3); and (c) Hierarchical Intent Alignment (HIA, detailed in § 3.4*). We first define the TPAS task and then describe each component.

Problem Formulation

The TPAS task is defined as follows: given a textual query describing the appearance of a pedestrian and a specific normal or abnormal behavior, the objective is to retrieve images from a large scale database where the pedestrian matches both the identity and the described behavior. Formally, given a set of image text pairs $\mathcal{T} = \{(x_{\mathrm{T}}, x_{\mathrm{I}})_n\}_{n=1}^{N}$ (where $x_{\mathrm{T}}$ and $x_{\mathrm{I}}$ denote the text and image, respectively, and $N$ is the total number of pairs), the objective is to learn a text encoder $\Phi_{\mathrm{T}}$ and an image encoder $\Phi_{\mathrm{I}}$ that map the corresponding text and image into a unified embedding space, such that $\Phi_{\mathrm{T}}(x_{\mathrm{T}}) \rightarrow\Phi_{\mathrm{I}}(x_{\mathrm{I}})$.

Behavior-Aware Mixture-of-Experts (B-MoE)

To model appearance and behavior cues with different invariance and sensitivity requirements, we design the Behavior-Aware Mixture-of-Experts (B-MoE) module. As shown in Figure 2(a), B-MoE constructs three parallel streams for appearance, normal behavior, and abnormal behavior, which provide the representations for subsequent cross-modal alignment.

abnormal-behavior stream, the top-level gate $\mathcal{G}^{\mathrm{T}}_{\mathrm{top}}$ produces a scalar weight $\mathbf{w}^{\mathrm{T}}_{\mathrm{top,anom}} \in\mathbb{R}^{1}$, yielding:

Expert Construction. For each modality, every stream contains $K$ sub-experts and a stream-specific gating network. We use the text-side abnormal-behavior stream as an example. Given the text encoder output, the sub-experts process the input from different perspectives and are dynamically combined by $\mathcal{G}_{\mathrm{ext}}$:

where $\mathbf{H}^{\mathrm{T}}_{\mathrm{anom}} \in\mathbb{R}^{S \times D}$, $S$ is the text sequence length, $D$ is the embedding dimension, and $\mathbf{w}^{\mathrm{T},k}_{\mathrm{ext,anom}}$ is the scalar weight of the $k$-th sub-expert. The appearance and normal-behavior streams use the same construction to obtain $\mathbf{H}^{\mathrm{T}}_{\mathrm{app}}$ and $\mathbf{H}^{\mathrm{T}}_{\mathrm{norm}}$. Although all streams share the encoder output, their independent experts and gates produce different input-dependent representations.

Implicit Specialization. Since three initial representations share the same encoder output, we reweight each according to input context. For the

The appearance and normal-behavior streams use their corresponding top-level gates to obtain $\mathbf{U}^{\mathrm{T}}_{\mathrm{app}}$ and $\mathbf{U}^{\mathrm{T}}_{\mathrm{norm}}$. No stream-specific semantic labels are used; all sub-experts and gates are jointly optimized by the image-text alignment and matching objectives.

Finally, text projection produces the global text expert features $\mathbf{f}^{\mathrm{T}}_{\mathrm{app}}, \mathbf{f}^{\mathrm{T}}_{\mathrm{norm}}, \mathbf{f}^{\mathrm{T}}_{\mathrm{anom}} \in\mathbb{R}^{D}$. The image branch follows the same procedure with its own modality-specific experts and gates, producing $\mathbf{U}^{\mathrm{I}}_{\mathrm{app}}, \mathbf{U}^{\mathrm{I}}_{\mathrm{norm}}, \mathbf{U}^{\mathrm{I}}_{\mathrm{anom}} \in\mathbb{R}^{C \times D}$, where $C$ is the number of image patches. After image projection, we obtain the global image expert features $\mathbf{f}^{\mathrm{I}}_{\mathrm{app}}, \mathbf{f}^{\mathrm{I}}_{\mathrm{norm}}, \mathbf{f}^{\mathrm{I}}_{\mathrm{anom}} \in\mathbb{R}^{D}$.

Conflict Offload Learning (COL)

To address the gradient conflict problem, we design the Conflict Offload Learning (COL) module based on the expert features from B-MoE. As shown in Figure 2(b), it introduces pairconditioned offload anchors for each expert and modifies the gradient path of contrastive learning, thereby providing an offload path for the conflicts caused by “same-identity, different-behavior” hard negatives.

Offload-anchor formulation. Before describing the implementation of COL, we first formulate the effect of an offload anchor on gradient propagation. For a batch of $B$ paired samples, let $\mathbf{S}$ denote image-text similarity matrix, where $s_{ij}=\cos(\mathbf{f}^{\mathrm{I}}_i,\mathbf{f}^{\mathrm{T}}_j)$. The InfoNCE loss is defined as follows,

where $\tau$ is the temperature coefficient.

For each sample pair, we introduce an offload anchor $\Delta_{ij}$ and redefine the similarity as follows,

Let $\hat{p}_{ij}=\exp(\hat{s}_{ij}/\tau)/\sum_{\ell=1}^{B}\exp(\hat{s}_{i\ell}/\tau)$, $y_{ij}=\mathbf{1}_{[j=i]}$, $\mathbf{g}_{ij}=\nabla_{\mathbf{z}_{ij}}\hat{s}_{ij}$ with $\mathbf{z}_{ij}=\mathbf{f}^{\mathrm{I}}_i+\Delta_{ij}$, and $\mathbf{J}_{ij}=\partial\Delta_{ij}/\partial\mathbf{f}^{\mathrm{T}}_i$. The corresponding gradients are

where $\mathbf{I}$ is the identity matrix. The first line gives the gradient through the offload path, while the second accounts for both the direct path through $\mathbf{f}^{\mathrm{T}}_i+\Delta_{ij}$ and the dependence of the predicted anchor on the text feature through $\mathbf{J}_{ij}$.

We focus on text-side gradients because text acts as the dynamic query while image gallery remains fixed at retrieval time. The pair-conditioned anchor absorbs part of conflicting alignment pressure while preserving text feature’s semantic content.

Conflict offloading. For each expert image-text pair $(\mathbf{f}^{\mathrm{T}}_i,\mathbf{f}^{\mathrm{I}}_j)$, COL computes an offload anchor from their feature difference and the image expert tokens. We illustrate the process using the abnormal-behavior expert:

where $\mathrm{TF}_{\mathrm{anom}}$ denotes the Transformer used by the abnormal-behavior expert. The corresponding offload-based similarity is as follows,

The appearance and normal-behavior experts follow the same procedure, yielding $\mathbf{S}_{\mathrm{app}}$ and $\mathbf{S}_{\mathrm{norm}}$.

Direction diversity regularization. To prevent the offload anchors from collapsing into similar directions, we introduce:

where $\mathcal{B}_{\mathbb{T}}$ denotes the text samples in the batch, $\sigma$ is a scaling factor, and $\mathbf{d}_{ij}$ and $\mathbf{d}_{ik}$ are the normalized forms of $\Delta^{ij}_{\mathrm{anom}}$ and $\Delta^{ik}_{\mathrm{anom}}$, respectively. $\langle\cdot,\cdot\rangle$ denotes cosine similarity. This regularizer encourages different pair-conditioned anchors associated with the same text query to maintain distinct directions.

Hierarchical Intent Alignment (HIA)

To address aggregation distortion, we design Hierarchical Intent Alignment (HIA). As shown in Figure 2(c), HIA predicts query-conditioned weights from text expert features and uses them to aggregate expert similarities and features.

Query-conditioned similarity aggregation. Taking the text side as an example, we feed the three global expert features from B-MoE into the text intent gating network $\mathbf{G}^{\mathrm{T}}_{\mathrm{H}}$. For the $i$-th text query, it produces $\mathbf{W}^{\mathrm{T}}_{\mathrm{gate},i}\in\mathbb{R}^{3}$.

where $\mathbf{W}^{\mathbb{T}}_{\mathrm{gate},i} = [w^{\mathbb{T}}_{\mathrm{app},i}, w^{\mathbb{T}}_{\mathrm{norm},i}, w^{\mathbb{T}}_{\mathrm{anom},i}]$. For a text-image pair $(i,j)$, these weights aggregate the expert similarities as $S^{ij}_{\mathrm{final}} = \sum_{r \in\mathcal{K}} w^{\mathbb{T}}_{r,i} S^{ij}_{r}$, where $\mathcal{K} = \{app, norm, anom\}$. We then optimize the bidirectional image-text contrastive loss:

where $\mathcal{L}_{\mathrm{NCE}}(\cdot)$ is defined in Eq. (3). This loss jointly trains expert streams and HIA gate, allowing their relative contributions to adapt to each query.

Intent-aware feature aggregation and matching. Following Yang et al. (2025), we further introduce an image-text matching loss to model semantic combinations. We reuse the weights in Eq. (9) to aggregate the text expert features as $\mathbf{f}^{\mathbb{T}}_{\mathrm{final},i} = \sum_{r \in\mathcal{K}} w^{\mathbb{T}}_{r,i}\mathbf{f}^{\mathbb{T}}_{r,i}$. For each image-text pair, the same weights are applied to the corresponding image expert features. The resulting representations are fused by last six layers of BERT, which act as a cross encoder and produce the matching logit $s_{itm}$. We optimize the binary cross-entropy loss $\mathcal{L}_{itm}$ using the label $y_{itm} \in\{0,1\}$.

Finally, the overall objective is as follows,

where $\gamma$ weights the direction-diversity regularizer. Following previous works (Yang et al., 2025; Cao et al., 2025), the coefficients of $\mathcal{L}_{itc}$ and $\mathcal{L}_{itm}$ are set to 1.

Experiments

In this section, we evaluate GradCO on the target TPAS task and conventional TIPR benchmarks, followed by ablation, sensitivity, and qualitative analyses.

Experimental Settings

Datasets.

For TPAS, we use PAB (Yang et al. (2025)) as the in-distribution benchmark, Multi-Weather to evaluate robustness to weather corruptions, and UCC

as the out-of-distribution (OOD) test set. For TIPR, we evaluate on CUHK-PEDES (Li et al. (2017)), ICFG-PEDES (Ding et al. (2021)), RSTPReID (Zhu et al. (2021)), and UFineBench (Zuo et al. (2024)), which includes the UFine6926 and UFine3C settings.

PAB. Pedestrian Anomaly Behavior (PAB) Yang et al. (2025) is a benchmark for text-based person anomaly search, which retrieves pedestrians engaged in normal or anomalous activities using natural-language descriptions. It contains 1,015,583 image-text pairs, including 1,013,605 synthetic training pairs generated with diffusion models using descriptions from OOPS! as prompts, and 1,978 real-world test pairs from OOPS!. PAB covers 1,600 anomalous action types (e.g., falling and being hit) and 1,000 normal action types (e.g., running and playing soccer). Its frame-level descriptions cover appearance, actions, and scenes, with an average length of 50.3 words; the training set uses an anomalous-to-normal ratio of 3:2. To construct the test set, OpenPose and ResNet-50 are used for filtering, followed by professional manual verification. Qwen2-VL generates the test descriptions, which are then checked through human quality control.

Multi-Weather Setting of PAB. Multi-Weather is an evaluation protocol applied to the PAB test set to measure retrieval robustness under weather and lighting corruptions. It includes ten conditions: Normal, Wind, Rain, Snow, Rain+Snow, Dark, Dark+Wind, Dark+Rain, Dark+Snow, and Over-exposure. We report results for each condition and their arithmetic mean.

UCC. UCC is a real-world test set for evaluating out-of-distribution (OOD) generalization from PAB to an unseen data source. It is derived from UCF-Crime and contains 5,320 image-text pairs. The images are selected keyframes from 13 normal and anomalous video categories, and Qwen2-VL generates the corresponding textual descriptions.

CUHK-PEDES. CUHK-PEDES Li et al. (2017) is a text-to-image person retrieval benchmark containing 40,206 images and 80,440 descriptions from 13,003 identities. Its training set contains 34,054 images, 68,126 descriptions, and 11,003 identities; the test set contains 3,074 images, 6,156 descriptions, and 1,000 identities.

ICFG-PEDES. ICFG-PEDES Ding et al. (2021) contains 54,522 images from 4,102 identities, with a text description paired with each image. Its training set contains 34,674 images from 3,102 identities, and its test set contains 19,848 images

from the remaining 1,000 identities.

RSTPReID. RSTPReID Zhu et al. (2021) contains 20,505 images of 4,101 identities collected from 15 cameras. Each identity has five images captured by different cameras, and each image is annotated with two text descriptions. The dataset is split into 3,701 training, 200 validation, and 200 test identities.

UFineBench. UFineBench Zuo et al. (2024) contains UFine6926 and the UFine3C evaluation set. UFine6926 has 26,206 images and 52,412 descriptions from 6,926 identities. Its descriptions are ultra-fine-grained, with an average length of 80.8 words. The training set contains 18,577 images and 37,154 descriptions from 4,926 identities, while the test set contains 7,629 images and 15,258 descriptions from 2,000 identities. UFine3C evaluates retrieval across domains, text granularities, and writing styles. It is formed by sampling 750 identities from the test sets of each of CUHK-PEDES, ICFG-PEDES, and UFine6926, representing coarse-, medium-, and fine-grained descriptions, respectively. Qwen-14B and Llama2-70B are used to restructure the original descriptions, and the generated text is manually revised. UFine3C contains 2,250 identities, 7,446 images, and 37,939 text queries.

Implementation details.

Following previous works (Cao et al., 2025), GradCO is initialized with the encoder weights of X2VLM (Zeng et al., 2023). We use the AdamW optimizer with a batch size of 22 and a weight decay of 0.01. B-MoE contains $K = 4$ sub-experts, and direction-diversity regularization weight is set to $\gamma= 0.5$. All experiments are conducted on a NVIDIA V100 GPU with 32GB memory and trained for 10 epochs.

Evaluation. For TPAS, we report Recall@K (R@K, $K \in\{1, 5, 10\}$) and mean Average Precision (mAP). For TIPR, R-Avg is the arithmetic mean of R@1, R@5, and R@10. On Multi-Weather, we report R@1 and mAP for each of the ten weather conditions and their arithmetic means. PAB-Avg and UCC(OOD)-Avg follow the corresponding protocols. Higher scores indicate better retrieval performance.

Performance Comparison

We compare GradCO with existing methods on the TPAS and TIPR tasks. For TPAS, we evaluate in-distribution retrieval on PAB, robustness under the Multi-Weather protocol, and transfer to the UCC out-of-distribution (OOD) test set. We then evaluate whether the same model remains effective on conventional TIPR benchmarks.

TPAS task

PAB evaluation. Table 1 reports the results on PAB under the 0.1M and 1M gallery settings. GradCO achieves the highest R@1, R@5, and mAP in both settings, and ties CMP at 1M R@10. Under the 0.1M setting, GradCO improves R@1 from 83.06 to 85.39 and mAP from 90.41 to 92.13 over CMP. Under the 1M setting, the corresponding gains are 1.67 and 1.22 percentage points. These results show that GradCO improves text-to-image retrieval when the query combines identity-related appearance with normal or anomalous behavior.

Method	#Data	R@1	R@5	R@10	mAP
CLIP (ICML’21)	-	47.57	81.55	89.03	62.73
X-VLM (ICML’22)	-	71.94	97.78	98.99	83.96
RaSa (IJCAI’23)	-	21.74	27.30	27.96	24.35
APTM (MM’23)	-	22.90	45.80	52.38	33.56
IRRA (CVPR’23)	-	30.59	59.61	68.91	44.41
MRA (arXiv’25)	-	9.91	23.66	31.45	17.15
WoRA (WWW’25)	-	22.25	45.91	53.54	33.39
CAMeL (TIFS’25)	-	24.47	50.00	58.75	36.75
CLIP (ICML’21)	0.1M	77.60	98.84	99.75	87.35
X-VLM (ICML’22)	0.1M	81.95	98.84	99.19	89.86
RaSa (IJCAI’23)	0.1M	80.79	98.89	99.65	89.20
APTM (MM’23)	0.1M	72.14	95.30	97.17	82.78
IRRA (CVPR’23)	0.1M	76.39	97.62	99.14	86.33
MRA (arXiv’25)	0.1M	70.53	94.69	97.47	81.59
WoRA (WWW’25)	0.1M	74.47	96.82	98.48	84.60
CAMeL (TIFS’25)	0.1M	74.30	96.79	98.84	84.20
CMP (ICCV’25)	0.1M	83.06	98.89	99.49	90.41
GradCO (Ours)	0.1M	85.39	99.34	99.60	92.13
CMP (ICCV’25)	1M	84.93	99.09	99.75	91.66
GradCO (Ours)	1M	86.60	99.60	99.75	92.88

Multi-Weather evaluation. Table 2 reports R@1 and mAP under ten weather conditions. GradCO achieves the highest value for both metrics in every condition, including the challenging “Dark+Rain” setting. Its mean R@1 and mAP are 72.2 and 80.67, compared with 67.12 and 77.44 for CMP. The consistent gains across conditions indicate that the multistream representation and query-conditioned aggregation remain effective when the visual evidence is degraded by weather changes.

Method	Normal R@1	Normal mAP	Wind R@1	Wind mAP	Rain R@1	Rain mAP	Snow R@1	Snow mAP	Rain+Snow R@1	Rain+Snow mAP	Dark R@1	Dark mAP	Dark+Wind R@1	Dark+Wind mAP	Dark+Rain R@1	Dark+Rain mAP	Dark+Snow R@1	Dark+Snow mAP	Over-exposure R@1	Over-exposure mAP	Mean R@1	Mean mAP
X-VLM (ICML’22)	83.47	90.94	79.02	88.10	54.40	67.40	59.10	72.34	49.95	62.09	79.58	88.24	75.53	85.47	34.93	45.98	50.25	63.32	74.87	84.82	64.11	74.87
CMP (ICCV’25)	84.93	91.66	81.24	89.34	60.06	72.53	63.40	75.74	54.85	67.31	80.89	89.00	77.20	86.55	39.03	50.58	53.49	66.12	76.09	85.56	67.12	77.44
GradCO (Ours)	86.60	92.88	83.69	91.25	68.00	78.85	69.55	80.68	61.77	69.20	83.24	90.08	82.08	89.33	47.38	55.22	60.31	70.21	79.32	88.95	72.20	80.67

OOD evaluation. Table 3 evaluates transfer to UCC. With the 1M gallery, GradCO improves over CMP from 55.23 to 62.53 in R@1 and from 44.35 to 52.01 in mAP, corresponding to gains of 7.30 and 7.66 percentage points. The improvement is also observed in the 1M gallery setting, showing that GradCO transfers more effectively to the tested OOD domain.

Method	R@1	R@5	R@10	mAP
CLIP (ICML’21)	51.60	68.31	76.43	43.05
X-VL (ICML’22)	52.33	66.73	72.54	40.87
APTM (MM’23)	27.86	40.41	46.77	22.61
IRRA (CVPR’23)	40.28	57.24	65.98	33.53
RaSa (IJCAI’23)	54.12	70.32	75.96	39.71
CMP (ICCV’25)	54.12	71.07	77.90	43.13
CMP(1M) (ICCV’25)	55.23	71.67	77.99	44.35
GradCO	61.85	78.31	86.09	51.78
GradCO (1M)	62.53	78.81	86.89	52.01

TIPR task

TIPR evaluation. Table 4 summarizes the average R@K on five TIPR evaluation sets. GradCO obtains the highest score on CUHK-PEDES, ICFG-PEDES, RSTPReID, and UFine3C. On UFine6926, Bi-IRRA reaches 95.18, while GradCO obtains 95.13. The largest improvement appears on UFine3C, where GradCO reaches 82.01 compared with 74.63 for CFAM. These results show that the appearance–behavior modeling and intent-conditioned matching developed for TPAS also improve retrieval under conventional and cross-domain TIPR settings.

Method	CUHK Avg	ICFG Avg	RSTP Avg	UFineBench 6926-Avg.	UFineBench 3C-Avg.
NAFS (ECCV’18)	-	-	-	76.49	58.25
SSAN (arXiv’21)	-	-	-	85.52	67.32
CMKA (TIP’21)	70.07	-	-	-	-
LapsCore (ICCV’21)	75.60	-	-	-	-
LGUR (MM’22)	-	-	-	81.72	65.42
SAF (ICASSP’22)	78.38	-	-	-	-
TextPed (NeurIPS’22)	78.85	-	-	-	-
AXM-Net (MM’22)	77.24	-	-	-	-
Magnet (TNNLS’23)	79.14	73.00	-	-	-
CFine (TIP’23)	82.22	73.27	68.22	-	-
IRRA (CVPR’23)	85.67	76.51	76.57	90.81	68.99
BiLMa (ICCV’23)	85.75	76.57	77.17	-	-
RaSa (IJCAI’23)	87.02	76.93	81.58	-	-
TBPS-CLIP (AAAI’24)	84.69	75.95	78.08	-	-
CADA-G (TMM’24)	85.72	75.71	77.75	-	-
UMSA (AAAI’24)	85.89	77.33	79.00	-	-
FSRL (ICMR’24)	86.32	77.28	77.77	-	-
Propot (MM’24)	86.32	77.89	78.40	-	-
CFAM (CVPR’24)	86.83	77.63	79.03	93.86	74.63
RDE (CVPR’24)	86.73	79.17	79.73	-	-
Bi-IRRA (TPAMI’25)	88.77	79.79	84.17	95.18	-
GradCO (Ours)	88.87	79.85	84.73	95.13	82.01

Efficiency Evaluation Analysis

To assess efficiency and resource consumption, we compare CMP and GradCO under the same hardware conditions and batch size of 22. Results are reported in Table 5.

Method	Parameters(M)	GPU Memory(MiB)	Test time(s/sample)	Train Time(s/iteration)	PAB-Avg	MultiWeather-Avg	UCC(OOD)-Avg
CMP	226.13	14575 (bs=22)	0.0015	0.5326 (bs=22)	94.59	67.12	68.30
GradCO (Ours)	247.36	18413 (bs=22)	0.0023	0.7213 (bs=22)	95.32	72.19	76.08

GradCO has 247.36M parameters, compared with 226.13M for CMP. This increase mainly comes from B-MoE, which processes appearance, normal behavior, and anomalous behavior through separate expert streams and their sub-experts. This design is associated with retrieval scores of 95.32% on PAB and 76.08% on UCC (OOD).

GradCO uses 18,413 MiB of GPU memory, compared with 14,575 MiB for CMP. This higher training memory consumption is partly due to COL, which computes pair-conditioned offload anchors for expert streams during training to mitigate gradient conflicts caused by “same identity, different behavior” hard negatives.

At inference, GradCO takes 2.3 ms per sample, compared with 1.5 ms for CMP, an absolute increase of 0.8 ms (53.3%). This overhead mainly comes from HIA, which uses text-conditioned gating to aggregate expert similarities. Despite the

increase, the 2.3 ms per-sample retrieval latency indicates that GradCO can support real-time retrieval in text-based person anomaly search. Meanwhile, Multi-Weather-Avg improves from 67.12% to 72.19%, and UCC (OOD)-Avg from 68.30% to 76.08%. These results show that GradCO improves retrieval performance under weather corruptions and distribution shifts while maintaining low-latency inference, making it suitable for integration into real-time anomaly-search systems.

Ablation Studies

We evaluate the contributions of B-MoE, COL, and HIA under the common training and evaluation settings described in § 4.1. All values are compared with the complete GradCO model.

B-MoE. As shown in Figure 3, removing the appearance (D#1), normal-behavior (D#2), and anomaly-behavior (D#3) streams reduces PAB-Avg to 94.71, 94.64, and 93.41, respectively, and Multi-Weather-Avg to 67.25, 65.90, and 70.16. Removing the image-side experts (D#4) or text-side experts (D#5) lowers UCC-Avg to 62.95 and 63.13; removing B-MoE entirely (D#6) further reduces it to 62.60. These complementary gains indicate that appearance, normal-behavior, and anomaly-behavior information jointly improve retrieval performance.

COL. As shown in Figure 4(a), removing the directional diversity regularizer (D#7) lowers UCC-Avg to 70.58, while removing the offload-anchor path (D#8) further reduces it to 55.43. The larger performance drop indicates that the pair-conditioned anchor is the primary mechanism for mitigating gradient conflicts between samples with the same identity but different behaviors. The directional diversity regularizer helps prevent anchor collapse and maintain effective gradient offloading.

HIA. As shown in Figure 4(b), removing similarity aggregation (D#9, w/o HIA_agg_sims) reduces PAB-Avg by 4.69 points. Removing intent weighting (D#10, w/o HIA_intent) and feature aggregation (D#11, w/o HIA_agg_feat) primarily affects PAB and Multi-Weather, indicating that expert information should be dynamically aggregated according to text intent. Removing the image-text contrastive objective (D#12, w/o HIA_align) causes performance to nearly collapse. Removing the matching objective (D#13, w/o matching) reduces PAB-Avg, Multi-Weather-Avg, and UCC-Avg to 83.70, 47.66, and 56.86. These results show that intent aggregation and both training objectives jointly support retrieval performance.

Sensitivity Analysis

To analyze the sensitivity of GradCO to the number of B-MoE sub-experts and the direction-diversity weight $\gamma$, we vary both hyperparameters and report the results in Figure 5. For both parameters, the performance first increases and then decreases. Setting $K = 4$ provides sufficient perspectives for separating appearance and behavior cues, whereas a larger value (e.g., $K \ge5$) may over-partition features and make experts focus on local details. The best performance is obtained at $\gamma= 0.5$, which helps prevent anchor collapse. A larger $\gamma$ over-constrains anchor directions, while a smaller value provides insufficient regularization, leading to less reliable offloading and stronger residual conflicts. We use $K = 4$ and $\gamma= 0.5$ in all experiments.

Mechanism Analysis

Performance Analysis on PAB Test Subsets (Normal vs. Anomaly)

To validate the effectiveness of GradCO in addressing the “invariance” versus “sensitivity” conflict inherent in feature learning, we partitioned the PAB test set into two subsets: PAB-Normal (regular behaviors) and PAB-Anomaly (anomalous behaviors). This division aims to separately evaluate the decoupling capability of the model in terms of identity preservation and fine-grained action capture. As shown in Table 6, compared to the baseline model CMP, GradCO achieved performance improvements across all metrics on both subsets, with the specific analysis as follows:

Method	PAB-Normal		PAB-Anomaly
	R@1	mAP	R@1	mAP
CMP	85.04	92.04	83.22	90.24
GradCO (Ours)	86.55	92.82	86.65	92.94

1. On the PAB-Normal subset, GradCO elevates the R@1 metric from the baseline of 85.04% to 86.55%. This demonstrates that after the Behavior-Aware Mixture-of-Experts (B-MoE) disentangles highly coupled semantics into independent expert streams, it does not compromise the capability of the model to extract appearance features. Coupled with Conflict Offload Learning (COL) for the mitigation of gradient conflict among experts, the model maintains robust identity recognition performance when processing regular behaviors.

2. On the PAB-Anomaly subset, the performance of CMP exhibits a decline, with its R@1 being only 83.22% (lower than its performance on PAB-Normal). This is attributed to its traditional “Aggregate-then-Align” paradigm causing “Aggregation Distortion”, which renders subtle anomaly behavior signals susceptible to being overwhelmed by strong appearance features. In contrast, GradCO achieves an R@1 of 86.65% and an mAP of 92.94% on this subset. Benefiting from the intent-guided dynamic aggregation of Hierarchical Intent Alignment (HIA) and the offload anchors mechanism of COL, the model successfully preserves and precisely aligns crucial evidence of anomaly behavior without the need for additional supervision signals.

The Evolution Analysis of Offload Anchors

To characterize the evolution of COL’s offload anchors during training, we track their norms and their cosine similarity with the text-image semantic difference across the appearance, normal-behavior, and anomalous-behavior streams. These metrics describe the anchors’ magnitude and direction relative to the main alignment signal.

Figure 6 shows that anchor norms first decrease, then rise, and finally stabilize. During the first 500 steps, the model builds a coarse global feature space, and the anchors remain small. From steps 500 to 2,000, as training focuses more on hard negatives with the same identity but different behaviors, anchor norms increase, consistent with COL adapting its correction strength to fine-grained alignment conflicts. The anomaly stream maintains the largest norm (approximately 9.0), followed by the appearance stream (approximately 7.5) and the normal-behavior stream (approximately 7.0). This pattern is consistent with the greater difficulty of preserving subtle anomaly cues against strong appearance signals.

Figure 7 shows that the cosine similarity between the anchors and the text-image semantic difference remains near zero throughout training, indicating that the anchors tend to introduce corrections in directions distinct from the main alignment signal. The anomaly stream has a slight negative offset (approximately $-0.1$), suggesting a weak opposing component for this stream. Together, these trends show that COL learns stable, stream-dependent corrections while preserving the main semantic alignment direction. The stable late-stage curves are also consistent with $L_{\mathrm{dir}}$ helping prevent anchor-direction collapse.

Table 7. Optimization diagnostics of COL variants. Gradient conflict rate and loss volatility measure optimization interference, while retrieval margin and anchor directional similarity characterize retrieval separation and offload geometry, respectively.

How COL Resolves Identity–Behavior Conflicts

This section analyzes the optimization mechanism of COL from three perspectives: gradient interaction, optimization stability, and anchor geometry. In TPAS, hard negatives often share similar identity-related appearance information while exhibiting different behaviors, creating competing optimization signals between identity preser vation and behavior discrimination.

COL introduces sample-specific offload anchors to provide an additional optimization pathway for conflict-sensitive information. For expert branch $k$, the anchor is generated as

and the similarity is reformulated as

where $\Delta^{k}_{ij}$ provides a sample-specific correction direction while preserving the original feature pathway for identity-related representation.

Gradient conflict reduction. We quantify the interaction between positive attraction and hard-negative repulsion by measuring their gradient cosine similarity. Let $g_i^{+}$ and $g_i^{-}$ denote the corresponding gradients propagated to the same text-side representation:

A negative cosine value indicates conflicting optimization directions. The conflict rate is computed as

The hardest negative sample is selected from nonmatching candidates within the same mini-batch according to the highest similarity score.

Optimization stability and retrieval separation. To evaluate optimization stability, we measure normalized loss volatility:

Meanwhile, retrieval separation is quantified by the gallery-level margin:

Anchor diversity preservation. To investigate the effect of directional regularization, the normalized anchor direction is defined as

The average directional similarity is calculated as

Lower $\mu_{\Delta}$ indicates more diverse anchor directions.

As shown in Figure 8 and Table 7, COL reduces gradient conflicts, stabilizes optimization, and improves retrieval separation. Removing anchors results in stronger optimization interference, while removing $\mathcal{L}_{dir}$ leads to less diverse anchor directions. These results demonstrate that COL resolves identity–behavior conflicts by introducing adaptive sample-specific offloading pathways.

Method	$r_{\mathrm{conf}} \downarrow$	$V_L \downarrow$	$m \uparrow$	$\mu_{\Delta} \downarrow$
w/o anchors	87.0%	0.072	0.18	–
w/o $L_{\mathrm{dir}}$	45.4%	0.043	0.46	0.681
Full COL	12.2%	0.019	0.78	0.129

Instance-level correction of hard negatives. Beyond optimization-level analysis, we further examine the instance-level effect of COL on confusing hard negative samples. Figure 9 compares the appearance-expert similarity scores of the same hard negative pairs before and after introducing COL. Most samples are distributed below the diagonal line $y = x$, indicating that COL consistently reduces the similarity assigned to hard negatives. This sample-level correction reduces the tendency of behavior-conflicting samples to compete with true matches in the retrieval space, providing additional evidence that the offload anchors translate optimization benefits into improved discrimination of difficult cases.

How COL Improves Feature Alignment and Embedding Geometry

Beyond optimization dynamics, we further investigate how COL affects the geometry of the learned embedding space. Specifically, we analyze three aspects: cross-modal positive-pair alignment, global feature distribution, and the distribution structure of different semantic groups. Figure 10 compares the embedding geometry under three settings: w/o COL, COL w/o $\mathcal{L}_{dir}$, and Full COL. All variants use identical feature extraction, projection, and density estimation procedures.

For each ground-truth text-image pair, let $\mathbf{z}_i^T$ and $\mathbf{z}_i^{I+}$ denote the $\ell_2$-normalized text and image embeddings, respectively. The positive-pair alignment distance is defined as

A smaller $\bar{d}_+$ indicates better cross-modal alignment between matched text and image representations.

To characterize the global embedding geometry, we further measure feature uniformity in the original normalized feature space:

Following previous analysis, a lower value indicates a more uniformly distributed embedding space with reduced local concentration.

For visualization, a shared projection matrix $P$ is applied to obtain two-dimensional representations:

where the angular distribution is estimated using kernel density estimation. The same projection and KDE configurations are applied to all variants for a fair comparison.

As shown in Figure 10, Full COL achieves the smallest positive-pair distance, with the mean distance reduced to 1.078 compared with 1.206 for w/o COL and 1.238 for COL w/o $\mathcal{L}_{dir}$. This indicates that COL improves the alignment between matched text-image representations.

Meanwhile, removing $\mathcal{L}_{dir}$ leads to a more concentrated embedding distribution, suggesting that unconstrained correction directions may reduce the diversity of learned offsets and distort the feature geometry. In contrast, Full COL maintains a more balanced angular coverage while preserving the separation among different semantic groups, including appearance, normal behavior, abnormal behavior, and hard negatives.

How Does HIA Improve Query-level Retrieval Consistency?

After analyzing the optimization mechanism and feature geometry of COL, we further investigate how HIA improves query-level retrieval consistency through adaptive expert aggregation. All comparisons use the same official query set, complete gallery, ground-truth annotations, and evaluation protocol.

To understand the role of expert aggregation, we first analyze the retrieval behavior of different semantic branches. The three expert branches capture complementary information, including appearance identity, normal behavior patterns, and abnormal behavior cues. Since different queries may rely on different semantic factors, fixed expert weighting may not provide optimal retrieval performance.

For query $q_i$, let $S_{ij}^{app}$, $S_{ij}^{norm}$, and $S_{ij}^{anom}$ denote the similarity scores produced by the three expert branches. Uniform aggregation and HIA aggregation are defined as

where $k\in\{\mathrm{app},\mathrm{norm},\mathrm{anom}\}$, $w_{ik}$ denotes the query-conditioned HIA weight.

For each query, we calculate the average precision (AP) under individual experts, uniform aggregation, and HIA aggregation. Figure 11(c) presents the corresponding query-level AP distributions. Individual experts show different strengths across queries, while uniform aggregation provides a fixed combination of semantic cues. HIA further improves retrieval consistency by dynamically adjusting expert contributions according to query characteristics.

We further analyze the resulting ranking behavior through positive-negative score separation. For query $q_i$, let $\mathcal{P}_i$ denote the set of ground-truth positive samples. The highest positive and non-positive scores are defined as

The retrieval margin is calculated as

A larger margin indicates a clearer ranking boundary between correct matches and confusing negatives. Figure 11(a) compares the margin distribution under different COL settings, while Figure 11(b) visualizes the joint distribution of $(s_i^-, s_i^+)$. These results characterize how the learned representations translate into final retrieval decisions.

Overall, the analysis demonstrates that HIA improves query-level retrieval consistency by adaptively combining complementary expert representations. Instead of relying on a fixed expert contribution, HIA dynamically adjusts semantic importance according to each query, resulting in more reliable retrieval behavior.

How Does HIA Adapt Expert Weights to Query Semantics?

This section analyzes whether HIA performs query-dependent expert adaptation by examining gate distributions and controlled text interventions. For each test query, we extract the three expert weights $\mathbf{w}_i = (w_{i,\mathrm{app}}, w_{i,\mathrm{norm}}, w_{i,\mathrm{anom}})$ and analyze their responses under different semantic conditions.

The three weights satisfy

Figure 12(a–Figure 12c) shows the KDE distributions of the three expert weights for official Normal and Anomaly queries. The distributions reveal different routing preferences between the two query groups: Normal queries assign relatively larger weights to normal-behavior experts, while Anomaly queries increase the contribution of abnormal-behavior experts. Meanwhile, the appearance expert maintains a distinct weighting pattern across the two groups, indicating that HIA considers both identity and behavior-related semantics during aggregation.

To further examine whether the gate responds to explicit semantic changes, we perform controlled text interventions. For each query with deterministic semantic phrases, we construct modified descriptions by masking appearance phrases, masking behavior phrases, or replacing behavior descriptions. All interventions preserve the same gallery and evaluation protocol.

For intervention condition $c$, the paired gate variation is defined as

Figure 12(d) presents the joint density of $\Delta w_{i,\mathrm{norm}}$ and $\Delta w_{i,\mathrm{anom}}$. The intervention results show that changing behavior-related descriptions induces coordinated shifts between normal- and abnormal-behavior expert weights, whereas appearance masking mainly affects the corresponding routing response with limited behavior-gate variation. These observations indicate that HIA dynamically adjusts expert importance according to query semantics rather than relying on fixed expert aggregation.

Together, these analyses demonstrate that HIA provides adaptive expert routing by responding to query semantics. The learned gate weights reflect the relative importance of appearance and behavior cues, enabling flexible aggregation of complementary expert representations.

Qualitative Results

Visualization of B-MoE Expert Gating Weights

To analyze whether B-MoE learns input-dependent routing patterns, we visualize the top-level gating weights $w^{\mathbb{I}}_{top}$ generated by $\mathcal{G}^{\mathbb{I}}_{top}$ in Figure 13 and Figure 14. The weights correspond to the relative preferences assigned to the appearance ($U^{\mathbb{I}}_{app}$), normal-behavior ($U^{\mathbb{I}}_{norm}$), and anomaly-behavior ($U^{\mathbb{I}}_{anom}$) streams. Since all experts receive the same encoder input and no expert-specific labels are provided during training, these visualizations should be interpreted as learned routing preferences rather than explicit semantic categorization.

In the selected anomaly examples, the anomaly-behavior stream receives relatively larger weights compared with the other streams. For example, Case 341 assigns a weight of 0.594 to the anomaly stream, indicating that the gate increases its preference for this stream under the shown abnormal visual context. Meanwhile, the appearance stream remains nonzero, suggesting that the routing process preserves contributions from multiple expert streams rather than selecting a single exclusive pathway.

For the selected normal examples, the normal-behavior stream obtains higher relative weights in scenarios involving controlled or skillful activities. For instance, in the Rubik’s cube example, the normal-behavior stream receives a weight of 0.529, while the anomaly-behavior stream receives 0.137. This indicates that B-MoE assigns different routing preferences according to the observed visual content, allowing different expert streams to contribute differently across samples.

The appearance stream remains active in the displayed examples, including a weight of 0.393 in Case 85, while behavior-related streams become more prominent when stronger action cues are present. This observation suggests that the learned routing mechanism maintains a balance among different expert streams instead of relying exclusively on appearance or behavior information. Overall, these visualizations provide qualitative evidence that the B-MoE gating network learns sample-dependent expert preferences. Together with the quantitative ablation results, they support the motivation of decomposing TPAS representations into multiple specialized streams, illustrating relative expert preferences rather than explicit semantic labels assigned to each expert.

Visualization of Hard-negative Feature Separation

To qualitatively analyze the representation structure under challenging identity–behavior conflicts, we visualize the image-to-image cosine similarity matrices of ten selected hard-negative pairs from the PAB test set Yang et al. (2025). The comparison between the X2VLM baseline and GradCO is shown in Figure 15. Both models are evaluated on the same selected samples, and the visualization provides an intuitive comparison of their learned feature organization. Note that this qualitative analysis does not isolate the individual effects of B-MoE and COL, but illustrates the overall representation changes introduced by the proposed framework.

As shown in Figure 15a, the X2VLM baseline exhibits a relatively diffuse similarity pattern. Several off-diagonal entries between different identities remain highly similar, with values mainly distributed in the range of 0.70–0.80. This indicates that hard-negative samples with similar appearance or behavior-related patterns remain difficult to separate in the learned feature space, leading to ambiguous identity-level representation.

In contrast, Figure 15b shows that GradCO produces a clearer block-like similarity structure. Samples belonging to the same identity maintain higher similarity, while samples from different identities show reduced similarity. For example, the similarity between $ID1\_ActA$ and $ID1\_ActB$ remains high (0.86), whereas some cross-identity pairs, such as $ID1$ and $ID9$, are reduced to 0.44. These observations suggest that GradCO learns a more discriminative organization of identity-related and behavior-related information, improving the separation of confusing hard-negative samples.

Multi-dimensional Performance Comparison

As shown in Figure 16, the radar chart summarizes the reported TPAS and TIPR benchmark results to provide an overall view of GradCO’s performance across different evaluation settings. Unlike individual metric comparisons, this visualization highlights the relative performance trends among different methods, while the detailed numerical comparisons are provided in the main tables.

For TPAS benchmarks, GradCO achieves the strongest performance on the displayed PAB, Multi-Weather, and OOD metrics. In particular, the results on Multi-Weather and UCC indicate improved performance under the tested environmental variations and OOD conditions. These observations demonstrate the effectiveness of GradCO on the evaluated TPAS scenarios, while they do not imply generalization to all possible domain shifts.

For the UCC OOD evaluation, GradCO improves over CMP (1M) by 7.30 and 7.66 percentage points in R@1 and mAP, respectively. These improvements provide empirical evidence of stronger retrieval performance on the tested OOD benchmark. The contribution of individual components is further analyzed through the corresponding ablation studies.

For TIPR benchmarks, GradCO remains competitive across the reported datasets. As summarized in the radar chart and quantitative tables, GradCO achieves the best displayed averages on CUHK-PEDES, ICFG-PEDES, and RSTPReID, leads UFine3C, while Bi-IRRA remains higher on UFine6926 R-Avg. These results indicate that the proposed framework maintains strong cross-modal retrieval capability beyond the TPAS setting.

Case Study

We compare GradCO with the representative baseline CMP on representative queries from the PAB dataset. Figure 17 presents three queries, followed by twelve additional cases in Figure 18. These examples illustrate retrieval under identity–behavior ambiguity, where visually similar candidates may exhibit different behaviors.

Results in Figure 17. From left to right, the three queries describe a woman and a man falling from a swing, runners participating in a race, and a football game. GradCO ranks the target first in the first two cases. In the swing example, the query combines appearance cues, such as “woman sitting on a man” and “swing”, with the behavior cue “falling down”. CMP ranks a visually similar image with mismatched behavior first, whereas GradCO retrieves the target at Rank-1. In the third case, GradCO ranks a highly similar hard negative first and the target second, while CMP ranks the target fifth.

Additional successful cases in Figure 18. Reading Figure 18 from left to right and from top to bottom, the first three rows show nine further cases in which GradCO ranks the target at Rank-1. The first row contains the imbalanced-cycling, tilting-gate, and dance-floor cases. In the tilting-gate example, CMP retrieves the same person standing upright in front of the gate, whereas GradCO retrieves the target leaning against it.

The second row contains the handstand-at-the-pier, kneeling-with-a-bottle, and bending-over-on-grass cases. In the handstand example, CMP retrieves the correct scene and person but mismatches the behavior, while GradCO ranks the handstand target first. The third row contains the lamppost-collision, skateboard-fall, and indoor-bending cases. In the lamppost example, GradCO distinguishes the collision moment from a visually similar walking case.

Failure cases. The final row of Figure 18 shows three further queries, including the gymnasium example. In that example, GradCO ranks a same-identity but behavior-mismatched sample first and the target second, while CMP ranks the target fifth or lower. Thus, in both the football case in Figure 17 and the gymnasium case in Figure 18, GradCO misses Rank-1 but ranks the target above CMP. These cases show that GradCO can provide a more discriminative ordering among difficult candidates, while also indicating that behavior retrieval remains challenging in the presence of highly similar candidates, visual clutter, or complex multi-person interactions.

Conclusion

In this paper, we investigate the TPAS task and reveal two key challenges of existing decoupling paradigms: gradient conflict and aggregation distortion. To address these challenges, we propose GradCO, a hierarchical Mixture-of-Experts framework that decouples appearance and behavior, alleviates hard-negative-induced optimization conflicts through Conflict Offload Learning, and enables intent-aware feature aggregation with Hierarchical Intent Alignment. Extensive experiments on five TPAS/TIPR benchmark datasets validate the effectiveness and robustness of our approach.