betaFeedback

WorksComputer Science

IIT Patna at "ICPR 2026 TVRID Competition on Privacy-Preserving Person Re-Identification from Top-View RGB-Depth Camera"

Abstract

Privacy-aware person re-identification (Re-ID) from top-view cameras replaces frontal appearance with anthropometric shape, while depth sensing further reduces color-based identity cues. The ICPR 2026 TVRID competition evaluates this setting using 86 identities captured by four synchronized Intel RealSense D455 cameras with RGB and Depth streams across flat, ascent, descent, and oblique viewpoints. Performance is measured using mAP and CMC@1 across within-camera in/out, ascent-descent, and flat-oblique scenarios. The competition had 147 registered participants and 33 final submissions. Three teams from IIT Patna participated: 'abhinavkumar03' and 'sjkdgfhl_' ranked 5th and 7th among 31 ranked participants, and 'vivek katuri' reached 77.67% CMC@1 / 85.20% mAP on the RGB track. This paper presents the methods of the two top-10 teams. Methodology 1 (abhinavkumar03) achieves 93.51% mAP and 90.42% CMC@1 on Depth, ranking 4th of 28, while Methodology 2 (sjkdgfhl_) reaches 99.63%/99.35% on RGB and 86.26%/80.34% on Depth. Both substantially outperform the 57.19% track-average Depth mAP, with Methodology 1 yielding a 7.3-point gain over Methodology 2. These results demonstrate the importance of depth representation and supervision for privacy-preserving Re-ID.

Introduction

Person re-identification (Re-ID) determines whether observations from different cameras belong to the same person and is widely used in surveillance, retail analytics, and crowd analysis [1]. While most Re-ID methods rely on side-view RGB images and appearance cues such as clothing color and texture, overhead cameras provide a more privacy-aware alternative by reducing facial visibility, while depth sensors further reduce appearance information by capturing 3D body geometry without color [2]. However, limited silhouettes, variable scale, short 2-3 second observations, and fewer color and texture cues make re-identification challenging. The ICPR 2026 TVRID competition [3] evaluates this setting using synchronized RGB and depth data from four overhead cameras with different viewpoints. Three teams from IIT Patna participated, with two reaching the top 10. Methodology 1 (M1: abhinavkumar03) converts depth maps into JET colorized images and uses OSNet [4], while Methodology 2 (M2: sjkdgfhl_) uses separate CNNs backbone for RGB and depth and mean-pools frame-level features to form a tracklet representation. Both methods use only the official TVRID training data and achieve strong performance on the competition benchmarks.

Task formulation. For each modality m∈{rgb,depth}m \in\{\mathrm{rgb},\mathrm{depth}\}, a tracklet Tm={xtm}t=1T\mathcal{T}^{m}=\{x_{t}^{m}\}_{t=1}^{T} is the set of TT person crops from one camera pass, where xtmx_{t}^{m} is the crop at frame tt. Using the N=62N=62 labeled training identities, each method learns an embedding vm=Ψm(Tm)∈Rdv^{m}=\Psi^{m}(\mathcal{T}^{m}) \in\mathbb{R}^{d} with ∥vm∥2=1\lVert v^{m}\rVert_{2}=1 (d=512d=512 for M1, d=2048d=2048 for M2). At test time, the identities are unseen. Each query tracklet qq is compared with every gallery tracklet gg using Dm(q,g)=∥vqm−vgm∥2D^{m}(q,g)=\lVert v_{q}^{m}-v_{g}^{m}\rVert_{2} and gallery tracklets are ranked by increasing distance.

Methodology 1

Depth colorization: Since raw depth images are single-channel and lack the color and texture cues used by ImageNet-pretrained CNNs, M1 converts each depth frame into a three-channel JET pseudo-color image using a fixed colormap C\mathcal{C} [6]:

x~t=C(xtdepth)∈RH×W×3.(1)\tilde{x}_{t}=\mathcal{C}(x_{t}^{\mathrm{depth}})\in\mathbb{R}^{H\times W\times3}. \tag*{(1)}

This converts depth variations into color gradients and highlights structures such as the head and shoulders without adding learnable parameters.

OSNet backbone: The colorized image is resized to 256×\times128 and processed using OSNet x1_0 [4]. Its omni-scale blocks capture features at different spatial scales, helping handle changes in viewpoint, elevation, and body scale in top-view images. Global average pooling and a fully connected layer produce a 512-dimensional frame feature ftf_t. The backbone has only 2.2M parameters.

Training and Inference

The network is trained to classify each frame into one of the N=62N=62 training identities. For a mini-batch of BB frames, let fi∈R512f_i\in\mathbb{R}^{512} be the embedding of frame ii, yiy_i its identity label, and W∈RN×512W\in\mathbb{R}^{N\times512} the weights of a linear classifier. The loss is label-smoothed cross-entropy:

LLS=−1B∑i=1B∑k=1Ny~i,klog⁡[softmax⁡(Wfi)k],(2)\mathcal{L}_{\mathrm{LS}}=-\frac{1}{B}\sum_{i=1}^{B}\sum_{k=1}^{N}\tilde{y}_{i,k}\log\left[\operatorname{softmax}(Wf_i)_k\right], \tag*{(2)}

where the smoothed target is y~i,k=(1−ε)δk,yi+ε/N\tilde{y}_{i,k}=(1-\varepsilon)\delta_{k,y_i}+\varepsilon/N, δk,yi\delta_{k,y_i} equals 1 when k=yik=y_i and 0 otherwise, and ε=0.1\varepsilon=0.1. Training uses only the official TVRID training data, starting from ImageNet weights. We use Adam for 60 epochs with batch size 32, an initial learning rate of 3×10−43\times10^{-4} and weight decay 5×10−45\times10^{-4}, and reduce the learning rate by 10×10\times at epochs 30 and 50.

Depth-frame OSNet methodology diagram

Figure 1. Methodology 1 (abhinavkumar03): Depth frames are JET-colorized and encoded by OSNet [4]

Modality-specific CNN backbone methodology diagram

Figure 2. Methodology 2 (sjkdgfhl_): Modality-specific CNN backbone encodes frames, followed by mean pooling for tracklet retrieval [5]

For augmentation, each image is flipped horizontally with probability 0.5. With probability 0.5 it is also enlarged to 288×144 and randomly cropped back to 256×128; otherwise it is resized directly to 256×128.

During inference, the classifier is removed and each frame feature is ℓ2\ell_{2}-normalized: vtdepth=ft/∥ft∥2v_{t}^{\mathrm{depth}}=f_{t}/\lVert f_{t}\rVert_{2}. The resulting frame embeddings are used to represent the tracklets, and query-gallery pairs are ranked using Dm(q,g)D^{m}(q,g).

Methodology 2

Modality-specific backbones: For each modality m∈{rgb,depth}m \in\{\mathrm{rgb},\mathrm{depth}\}, every frame xtmx_{t}^{m} of a tracklet is resized to 256×128 and encoded independently by its own backbone ϕm\phi^{m}, a convolutional network with its ImageNet classification head removed. We use ResNeSt-50d [7] for RGB (ϕrgb\phi^{\mathrm{rgb}}) and ResNet-50 [5] for depth (ϕdepth\phi^{\mathrm{depth}}). Both start from ImageNet weights, and depth frames are read as grayscale images replicated to three channels so that they match the RGB input format. The backbone outputs a feature map ϕm(xtm)∈R2048×h×w\phi^{m}(x_{t}^{m}) \in\mathbb{R}^{2048 \times h \times w}, where h×wh \times w is its spatial size. Global average pooling (GAP), which averages this map over all h×wh \times w positions, gives a 2048-dimensional frame feature:

ft=GAP⁡(ϕm(xtm))∈R2048(3)f_{t}=\operatorname{GAP}\left(\phi^{m}\left(x_{t}^{m}\right)\right)\in\mathbb{R}^{2048} \tag*{(3)}

Tracklet aggregation: At test time the classifier WW is removed. A tracklet Tm\mathcal{T}^{m} contains TT frames, which are short and unordered, so instead of modeling time we average the TT frame features ftf_{t} from Eq. (3) and scale the result to unit length:

fˉ=1T∑t=1Tft,vm=fˉ∥fˉ∥2,(4)\bar{f}=\frac{1}{T}\sum_{t=1}^{T}f_{t},\qquad v^{m}=\frac{\bar{f}}{\lVert\bar{f}\rVert_{2}}, \tag*{(4)}

where ∥⋅∥2\lVert\cdot\rVert_{2} is the Euclidean (ℓ2\ell_{2}) norm. For depth, the BN layer of the training head is kept, so each ftf_{t} is batch-normalized before averaging. Averaging adds no parameters, does not depend on frame order or on TT, and smooths out frame-level noise such as motion blur, partial views and detector errors. The tracklet embedding vmv^{m} is then compared across query and gallery tracklets with Dm(q,g)D^{m}(q,g).

Training Strategy

Score-level fusion: When both RGB and depth are available, their query-gallery distances are averaged:

Dfused=12(Drgb+Ddepth)(5)D^{\mathrm{fused}}=\frac{1}{2}\left(D^{\mathrm{rgb}}+D^{\mathrm{depth}}\right) \tag*{(5)}

This provides RGB+Depth matching using both modalities for the query and gallery.

Training Strategy

Each modality branch is trained independently to classify frames into the N=62N=62 training identities. For a mini-batch B\mathcal{B} of frame–label pairs (xtm,y)(x_{t}^{m},y), the frame feature ftf_{t} (Eq. (3)) passes through a BN layer and a linear classifier W∈RN×2048W\in\mathbb{R}^{N\times2048}, trained with cross-entropy:

LID=−1∣B∣∑(xtm,y)∈Blog⁡[softmax⁡(WBN⁡(ft))y](6)\mathcal{L}_{\mathrm{ID}}=-\frac{1}{|\mathcal{B}|}\sum_{(x_{t}^{m},y)\in\mathcal{B}}\log\left[\operatorname{softmax}\left(W\operatorname{BN}(f_{t})\right)_{y}\right] \tag*{(6)}
TrackMetricAverageMedianM1M2
RGBmAP89.6693.3599.8899.63
CMC@185.7391.9299.8499.35
DepthmAP57.1949.8293.5186.26
CMC@147.1030.5590.4280.34

Table II. Our entries against the official aggregate statistics [3]

Official Depth-track mAP for 28 entries

Figure 3. Official Depth-track mAP for 28 entries [3], sorted in decreasing order. Our entries are in the top tier, above the track average

We use only TVRID training data with ImageNet initialization and train with Adam (constant learning rate 3.5×10−43.5 \times10^{-4}, batch size 64). Augmentation includes horizontal flipping, rotation (±10∘\pm10^\circ), color jitter and random erasing [8]. During inference, the BatchNorm layer and classifier are removed. Frame features are mean-pooled using Eq. (4) to obtain the ℓ2\ell_2-normalized tracklet embedding. When both RGB and depth are available, their distances are fused using Eq. (5).

Experimental Setup

Common protocol: Both methods use ImageNet initialization and train only on the 62 official TVRID training identities. We evaluate final rankings on the blind Codabench server using CMC@1 and mAP [3].

Implementation: Table I summarizes the two configurations. M1 & M2 are implemented and trained on the Aryabhatta Paramrudra Server.

M1 (abhinavkumar03)M2 (sjkdgfhl_)
InputDepth → JET pseudo-RGBRGB / raw depth
BackboneOSNet x1_0 (2.2M)ResNeSt-50d (27M) / ResNet-50 (26M)
Embedding512-d, ℓ2\ell_22048-d, mean-pooled, ℓ2\ell_2
LossLLS\mathcal{L}_{\mathrm{LS}}LID\mathcal{L}_{\mathrm{ID}}
OptimizerAdam, lr 3×10−33 \times10^{-3}Adam, lr 3.5×10−43.5 \times10^{-4}
Schedule60 ep., batch 32; ×0.1\times0.1 at 30, 50—
AugmentationFlip; crop from 288×144288 \times144Flip, rotation ±10∘\pm10^\circ, jitter, erasing

Table I. Implementation details of the two methods.

Dataset Analysis:

TVRID [3] contains 86 identities recorded by four synchronized overhead Intel RealSense D455 cameras at 15 FPS (640×480640 \times480), covering flat, ascent, descent, and oblique views. Each passage is observed twice (IN/OUT), giving eight observation sets, with aligned RGB and depth data and depth-based person crops. The dataset uses 62 identities for training, with disjoint test identities. The main challenges are short 2-3 sec tracklets, viewpoint and scale changes, and limited color and texture in depth images. Evaluation considers within-camera IN↔OUT, ascent↔descent, and flat↔oblique scenarios using CMC@1 and mAP. The competition had 147 registered participants, 732 development submissions, and 33 final submissions, with average RGB mAP/CMC@1 of 89.66%/85.73% compared with 57.19%/47.10% for depth,

89.66%/85.73% compared with 57.19%/47.10% for depth, highlighting the greater difficulty of depth-based Re-ID.

Result Analysis:

M1 and M2 achieve strong performance across both tracks, obtaining 99.88% and 99.63% RGB mAP with 99.84% and 99.35% CMC@1, respectively, and 93.51% and 86.26% Depth mAP with 90.42% and 80.34% CMC@1. Both exceed the official Depth-track averages of 57.19% mAP and 47.10% CMC@1 [3]. On Depth, M1 outperforms M2 by 7.25 mAP and 10.08 CMC@1 points despite using a much smaller backbone (2.2M vs. 26M parameters), securing 4th rank among 28 submissions. This highlights the effectiveness of JET-based depth colorization with lightweight multi-scale OSNet. Both methods use only official TVRID training data and standard CNN-based training; while M1 provides a lightweight depth solution, M2 employs modality-specific backbones and tracklet-level feature aggregation. Both process RGB and Depth independently without requiring cross-modal alignment during training. However, the lack of controlled ablations and differences in training settings limit the attribution of performance gains to individual components.

Conclusion

We presented two IIT Patna entries in the ICPR 2026 TVRID competition using only the official training data. M1 combines JET depth colorization with a lightweight 2.2M-parameter OSNet, while M2 uses modality-specific backbones (ResNeSt-50d & ResNet-50) with tracklet aggregation. They achieve up to 99.88% mAP on RGB and 93.51% on Depth, with M1 ranking 5th overall and 4th on the Depth track, while M2 ranks 7th overall. The results show that effective depth representation and simple feature aggregation can provide strong Re-ID performance without large models or external data.

Acknowledgment: The organizers of the ICPR 2026 TVRID Competition and the Aryabhatta Supercomputing Centre (ASC), IIT Patna, established under NSM, Govt. of India, for providing computational resources, are gratefully acknowledged.

References

  1. [1]M. Ye et al., “Deep Learning for Person Re-Identification: A Survey and Outlook,” IEEE TPAMI, vol. 44, no. 6, pp. 2872–2893, 2022.DOI
  2. [2]R. Delécluse et al., “Privacy-Preserving Person Re-Identification from Temporal Sequences with Transformer and Hungarian Optimization,” in IEEE FG, 2025, pp. 1–10.DOI
  3. [3]R. Delécluse et al., “ICPR 2026 competition on privacy-preserving person re-identification from top-view rgb-depth camera (tvrid),” in Pattern Recognition, 2027, pp. 229–242.
  4. [4]K. Zhou, Y. Yang, A. Cavallaro, and T. Xiang, “Omni-scale feature learning for person re-identification,” in ICCV, 2019.arxiv.org/abs/1905.00953
  5. [5]K. He et al., “Deep Residual Learning for Image Recognition,” in CVPR, 2016.arxiv.org/abs/1512.03385
  6. [6]A. Eitel et al., “Multimodal Deep Learning for Robust RGB-D Object Recognition,” in IEEE IROS, 2015, pp. 681–687.arxiv.org/abs/1507.06821
  7. [7]H. Zhang et al., “ResNeSt: Split-Attention Networks,” in IEEE CVPRW, 2022, pp. 2735–2745.arxiv.org/abs/2004.08955
  8. [8]Z. Zhong, L. Zheng, G. Kang, S. Li, and Y. Yang, “Random erasing data augmentation,” in AAAI, 2020.arxiv.org/abs/1708.04896

Paper details

vlnc:W_a6mmvhwk
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 7

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents