betaFeedback

WorksComputer Science

Consensus Capture in Reference-Based UAV Video Stabilization: A Benchmark on Dynamic Urban Scenes

Abstract

Reference-frame stabilization registers every frame of a hovering-drone video onto one reference frame, keeping object positions comparable. The registration pipelines it relies on are selected on feature-matching benchmarks in which only the camera moves. Over a busy intersection, vehicles moving together supply mutually consistent correspondences that can draw the robust homography estimate toward their motion: consensus capture, which inlier statistics cannot flag. A two-part benchmark measures it and the object masking meant to prevent it. A static protocol scores ten pipelines under exactly known homographies on 4K aerial frames, tuned on a site-disjoint split at a matched resolution and feature budget. A dynamic protocol scores them at an offline and a real-time budget on hovers selected from the video alone to span the amount and coherence of moving foreground, plus a stationary control. It measures background alignment, object position error and the share of a moving group's displacement an estimate follows, against a pseudo-ground-truth reference audited with manually annotated ground control points, which also score every candidate. On static scenes, tuned AKAZE, RootSIFT, SIFT and BRISK lead, retuning the matcher thresholds raises the mean average accuracy (mAA) by at most 0.0062, and masking costs less than 0.005. Unmasked ORB is captured on 28 of 304 real-time dynamic pairs and on 6 offline; its real-time captured estimates have a median corner error of 18.8 px yet typical inlier counts and ratios. Masking removes capture for every pipeline and leaves the leading pipelines unchanged. Code, data and results are released.

Introduction

Reference-frame stabilization is the registration model used when object positions must stay consistent over a long hover, during which wind keeps shifting and rotating the view: every frame is aligned directly onto one fixed reference frame, so that error does not accumulate and a position measured in frame kk is comparable with one measured minutes earlier. Trajectory extraction from high-altitude unmanned aerial vehicle (UAV) video is the application studied here, and traffic monitoring supplies the footage, but the requirement is shared by any analysis that compares moving objects across time in image or map coordinates, such as river surface velocimetry ([63]). The estimate is a homography, and the pipeline that produces it (feature detector, descriptor, matcher, robust estimator, and, where objects are annotated, a masking strategy that withholds them from feature detection) is assembled from components each of which has been benchmarked extensively ([27, 50, 9]).

Those benchmarks evaluate correspondence accuracy on scenes without independently moving content: planar surfaces, landmark façades, building walls. The only source of disagreement between their two views is the camera. A hovering drone over a busy intersection presents the opposite case. The camera barely moves, and dozens of objects in the frame move without it, many of them together: a platoon released by a green light translates as a unit, and every vehicle in it supplies correspondences that agree with one another under a single transform, the one describing their motion. If those correspondences win the consensus, the robust estimator does not fail. It solves the problem it was posed, returning a homography that is internally consistent and well supported but describes the vehicles rather than the road (Figure 1).

Consensus capture on real footage

Figure 1. Consensus capture on real footage. Unmasked ORB at the real-time budget (left) registers a frame ten frames (0.33 s) after the reference frame of case S2 (Table 5). Both panels show the same zoomed-in crop of the 3840 × 2160 frame, centered on the largest moving group; orange boxes are the vehicles that moved together between the two frames, with their motion. The estimate registers the group back onto where it stood (capture fraction 0.96), so its error, drawn as black arrows on a grid and magnified by the factor in the legend, carries the platoon’s motion across the whole crop (10.0 px corner error against the reference estimator), while 23 % of its 466 inliers lie on the moving vehicles (orange dots). The same pipeline under oriented-box masks (right) keeps 783 inliers, none on those vehicles, at a corner error of 0.9 px. Both are re-registered with the stored parameters and seed of the real-time campaign (§ 7.3).

We call this failure consensus capture, and it has a consequence for evaluation before it has one for method design. Inlier count, inlier ratio, and reprojection error measure the self-consistency of a fit, and a coherently moving foreground is self-consistent by construction. Where it wins, these quantities are no longer merely weak proxies for accuracy, as they are already known to be on static scenes [50, 9]; they become anti-informative, because the moving-object consensus can be the cleaner of the two available fits. An evaluation that reads them cannot detect capture. What is needed scores the alignment of the static world directly, makes a moving-object consensus observable, and relates it to how much of the frame moves and how coherently. To our knowledge no matching or registration benchmark provides this: none combines independently moving content with reference geometry (§ 2), so a risk flagged for airborne video [55] has gone unmeasured.

Capture matters most where trajectories are used one at a time. Macroscopic quantities such as flow and density average a transient offset away, but microscopic analyses read individual trajectories and their derivatives: car following, lane changes [54], per-vehicle noise emission [28], and surrogate safety measures such as time-to-collision, post-encroachment time and the deceleration rate to avoid a crash [1, 29], which drone trajectory datasets now provide or are built to support [98, 38]. Under reference-frame registration each frame is registered on its own, so nothing ties a captured estimate to those of neighboring frames: its error follows the group’s motion and is present only while the group holds the consensus. Speeds and accelerations are differences of positions over fractions of a second, so an error that varies from frame to frame in this way is amplified in them [72]. The accuracy these datasets aim for is correspondingly tight: the inD dataset requires a positioning error below 0.1 m [14], under two pixels at the ground sample distance (GSD) of the footage studied here (§ 3), whereas the captured estimates of § 7.3 have a median corner error of 18.8 px, about a meter on the ground, against the reference estimator. Since the estimates pulled toward the group keep typical inlier statistics, a pipeline captured on a few frames passes their error, unflagged, to every trajectory in those frames. Several of these datasets are also recorded lower than the 140 m to 150 m studied here: inD from up to 100 m at about 4 cm per pixel [14] and the intersections of CitySim from 120 m [98]; a view that covers less ground can give a vehicle group of the same size a larger share of the frame and of its features.

We provide such an evaluation in two parts. Part I is a static protocol under exactly known homographies and photometric degradation, on 4K frames from a multi-drone experiment over Songdo, South Korea: one from each of the 29 viewpoints from which its ten drones recorded 20 intersections. It establishes what each pipeline can do when nothing moves and which of its settings matter, extending the campaign of Fonod et al. (2025a) with paired trials, per-pipeline tuning on a site-disjoint split, and a matched resolution and feature budget. Part II is a dynamic protocol on nine hovers over busy intersections, each registered onto a reference frame selected from the video alone to span the amount and coherence of moving foreground, plus a stationary control that separates the removal of texture. It runs every pipeline at an offline and a real-time point of Part I’s budget grid. Real footage has no true homography, so Part II measures background alignment outside the annotated objects, the position error a trajectory inherits, the direction of the error relative to the dominant motion, and the share of a moving group’s displacement an estimate follows, which measures capture where it happens. Its pseudo-ground-truth reference estimator is audited against manually annotated ground control points (GCPs) (image positions of static ground points, not surveyed coordinates), which also score every candidate directly, so that the masking comparison does not depend on the reference estimator’s own masking. Masking with horizontal bounding boxes (HBBs) and oriented bounding boxes (OBBs) is a controlled axis throughout, since the shape of a mask could matter in either direction (§ 5).

On hovers over Songdo intersections, the measurement finds capture. At the real-time budget, unmasked ORB is captured in all five cases that present a coherently moving group, the four with much moving foreground and the stationary control, on 28 of the 304 pairs that carry one, though 22 of them in a single case. The 17 within the first second are pulls toward the group; 10 of the 11 beyond it are breakdowns of the estimate, which unmasked ORB also shows with no group in view. At the offline budget capture is rarer (unmasked ORB, 6 pairs in two cases). Estimates pulled toward a group cannot be told apart by their inlier statistics: their inlier counts and ratios match those of the same pipeline at the nearest offsets of the same case, at both budgets. Masking removes capture for every pipeline at both budgets while leaving the leading pipelines, SIFT, RootSIFT and AKAZE, where they were; for real-time ORB, whose feature budget it also returns to the ground, the captured pairs can account for at most a quarter of the gain. HBBs and OBBs do not differ at these densities and orientations, where the two geometries withhold nearly the same pixels. Averaged over all nine cases, four of which never present a moving group, the masking benefit is either not significant or smaller than the repeatability of the GCPs (§ 7.5). The conclusions are bounded by the moving share of these cases; the protocol is not, and it applies unchanged to footage from lower altitude or over denser traffic, where that share is larger.

Contributions

  1. A benchmark for the complete registration pipeline of reference-based UAV stabilization in dynamic urban scenes: a static protocol under exactly known homographies (§ 6) and a dynamic protocol on real hovering-drone footage at an offline and a real-time compute budget (§ 7). It covers the pipelines a deployed stabilization library exposes, as a measurement of that system rather than a leaderboard of state-of-the-art matchers (§ 3.2).

  1. An evaluation methodology for dynamic scenes, where self-consistency statistics are unreliable: background-alignment measures that need no ground-truth homography, object-level errors that bound the position error a trajectory inherits, two measures that make consensus capture observable (the direction of the error and the share of a moving group’s displacement an estimate follows), and a GCP audit that measures error of the pseudo-ground-truth and supplies an independent comparison (§ 4), under a pre-specified analysis plan (§ 4.9).

  1. A case-selection procedure that measures from the video alone how much of the foreground moves, how coherently, and when, and chooses the cases and their reference frames to span these conditions, under which capture can arise; its measurements are validated against published vehicle trajectories (§ 7.1).

  1. A measurement of consensus capture and of object-aware masking on real footage: the real-time budget unmasked ORB is captured in every case that presents a coherent group, its pull toward the group is invisible to inlier statistics, and capture disappears under masking at detection (§ 7.3 and § 7.5).

  1. The public release of the benchmark implementation (Fonod, 2026c) and of its data (Fonod and Geroliminis, 2026a) (annotated frames, cases, GCPs and every campaign’s results), from which every benchmark number is regenerated by a command. Also released are hbb2obb (Fonod, 2026b), the SAM-assisted conversion of HBBs to OBBs that makes OBB masks available at benchmark scale, and Songdo Vision OBB (Fonod and Geroliminis, 2026b), the OBB version of Songdo Vision (Fonod et al., 2026), together with the human-verified oriented reference against which the conversion is validated (§ 5.1 and Appendix § 14).

Related work

Local features and matching. Correspondence estimation has been surveyed repeatedly as it moved from hand-crafted operators to learned ones (Ma et al., 2021). The pipelines benchmarked here span three generations of that work. Hand-crafted detectors and descriptors, from SIFT (Lowe, 2004) and its RootSIFT normalization (Arandjelović and Zisserman, 2012) to the binary ORB (Rublee et al., 2011) and BRISK (Leutenegger et al., 2011) and the nonlinear scale spaces of KAZE (Alcantarilla et al., 2012) and AKAZE (Alcantarilla et al., 2013), remain the default in deployed stabilization pipelines, because they need no training data, no accelerator, and no assumption about the imagery. Learned detectors and descriptors replace one or both stages with a network trained for repeatability and matchability, among them SuperPoint (DeTone et al., 2018), DISK (Tyszkiewicz et al., 2020), DeDoDe (Edstedt et al., 2024a), Key.Net (Barroso-Laguna et al., 2019) and XFeat (Potje et al., 2024); learned matchers such as LightGlue (Lindenberger et al., 2023) replace the nearest-neighbor search and its ratio test with attention over the two keypoint sets; and detector-free methods such as LoFTR (J. Sun et al., 2021) drop keypoints and match dense features directly. Dense matchers such as RoMa (Edstedt et al., 2024b) go further and estimate a warp and its certainty for every pixel, from which correspondences are sampled. A stabilization pipeline is chosen as a whole and its components interact (a matcher accepts only certain descriptors, a detector-free method has no keypoint budget), so the comparison of § 6.2 spans all three. Two properties of this literature shape the protocol. Orientation estimation is not universal, so several learned models are effectively upright, which is why rotation is swept, as Tareen and Saleem (2018) sweep it for the classical detectors, which they find unequally invariant to it; and the input resolution of a learned model is a memory bound rather than a free choice, which is why the compute budget is an axis of the benchmark.

Direct homography regression. A parallel line of work regresses the transform from an image pair (DeTone et al., 2016). Zhang et al. (2020) observe that a loss computed uniformly over the image cannot keep moving or non-planar objects from contributing to it, and learn a content mask that rejects those regions: the problem measured here, approached by learning what to ignore instead of by annotating it. These methods lie outside the feature-based family deployed in the trajectory extraction systems of § 2.

Robust estimation and dominant structure. RANSAC (Fischler and Bolles, 1981) and its modern variants (Barath and Matas, 2018; Barath et al., 2020) make a homography recoverable from contaminated correspondences. That a coherent structure of outliers can capture the consensus is known (Stewart, 1997), and Chum et al. (2005) detect a dominant plane inside the estimator; § 4.1 explains why that defense does not transfer to a coherently moving group. Filters that prune matches before the estimator rest on a similar vote: S. Jiang and W. Jiang (2018) keep matches whose motion agrees in direction and length, Zhou et al. (2024) build local motion consensus into a learned matcher, and the motions of a coherently moving group agree as well.

Scenes with more than one structure. Multi-model fitting accepts that a scene may contain several coherent structures and recovers them jointly, by clustering correspondences (Toldo and Fusiello, 2008; Wong et al., 2011) or by fitting instances progressively inside the robust estimator (Barath and Matas, 2019). Vehicles crossing an intersection are such a second structure, and AdelaideRMF (Wong et al., 2011) does contain independently moving content. These methods are evaluated by how well correspondences are partitioned among structures, not by the accuracy of one recovered transform, and AdelaideRMF supplies no reference homography for the background. Recovering several homographies also leaves open which one describes the ground, which correspondence counts cannot decide when the moving structure is the larger. Multi-model fitting is a remedy that changes the estimator; what is missing first is a measurement of the failure under the estimators actually deployed.

Benchmarks for matching and homography estimation. Comparative studies of classical detectors and descriptors (Tareen and Saleem, 2018) and HPatches (Balntas et al., 2017) established the vocabulary this field still uses, and Schonberger et al. (2017) set the pattern of judging hand-crafted and learned features on a common downstream task. Three benchmarks set the methodological standard we adopt. Jin et al. (2021) argue that local features must be judged by a downstream task rather than by intermediate proxies, and show that carefully tuned classical pipelines remain competitive with learned ones, a conclusion Efe et al. (2021) reach on HPatches by optimizing a single parameter; Barath et al. (2023) contribute a large-scale homography benchmark with an explicit tuning-and-test protocol. Their shared lesson is that a comparison is only as meaningful as the care taken to place every method on equal footing (§ 4.7). None of them measures registration accuracy on a scene whose content moves independently of the camera (Table 1), because their reference geometry, structure from motion (SfM) or manually labeled correspondences on static structure, must exclude independently moving objects to work at all. Registration benchmarks that do model change place it between acquisitions (T. Sun et al., 2025). Pseudo-ground truth has a further cost: Brachmann et al. (2021) show that the algorithm generating it can reorder the methods evaluated against it, which is why the reference estimator of Part II is audited against GCPs (§ 4.8).

BenchmarkContentReference geometryIndependent motion
HPatches ((Balntas et al., 2017))planar surfacesmanual homography✗
Efe et al. ((2021))planar surfacesmanual homography✗
IMC-PT (Jin et al., 2021)landmark photosSfM poses✗
HEB (Barath et al., 2023)landmark photosSfM planes✗
AdelaideRMF (Wong et al., 2011)multi-structure pairsnone (memberships)✓
Ljubicić et al. (2021)river videosnone (static features)✓
This work, Part IUAV images, intersectionsknown by construction✗
This work, Part IIUAV videos, intersectionsaudited pseudo-GT✓

Table 1. Benchmarks for feature matching, homography estimation and stabilization, and the property this work adds. “Independent motion” asks whether any part of the scene moves between the two views independently of the camera.

Suppressing moving objects before geometry is estimated. Excluding independently moving content before a geometric model is fitted is established practice in dynamic simultaneous localization and mapping (SLAM). DynaSLAM (Bescos et al., 2018) and DS-SLAM (C. Yu et al., 2018) segment likely-moving classes and withhold those regions from tracking and mapping. Both report that suppression keeps the estimate usable in populated scenes; what is missing is a measurement. Those systems are evaluated end to end, by trajectory error on indoor sequences with a handful of walking people or on driving sequences, which does not say how much of the frame the objects may cover before the consensus shifts to them, which box geometry is worth its cost, or what the failure looks like without suppression. A hovering drone over an intersection is the opposite regime: a nearly static camera and dozens of vehicles moving in coherent streams. Isolating the effect needs the masking strategy varied as a controlled axis while everything else is held fixed (§ 7.5), and a metric that does not read the objects themselves (§ 4.3).

Video stabilization as a different objective. Stabilization also names work whose goal is a pleasant camera path rather than a fixed frame of reference: smoothing the path under geometric constraints ([44]), over a mesh of local homographies ([62]), or by learning the warp ((J. Yu and Ramamoorthi, 2020)). Its measures (cropping ratio, distortion, a frequency-domain stability score) follow from a perceptual objective, and none asks where a pixel actually is. A smoothed path may drift from any particular frame, which is inadmissible when object positions are compared across a recording, and a mesh of local warps can absorb independent object motion instead of rejecting it. Moving objects enter this literature as a threat to the global motion estimate, met by adaptive RANSAC ([16]), clustered motion vectors ((R. Wu et al., 2021)) or foreground trajectories ([97]), and scored by the similarity of the output frames, not against reference geometry.

Registration and stabilization of aerial video. Registering airborne video to a fixed reference is long established in photogrammetry ([57]). The remote sensing literature on image registration concentrates on matching across modalities, sensors and acquisition times ([42]), with structural or phase-congruency descriptors built to be insensitive to radiometric change ([60] (Ye et al. 2017)) and networks trained on large multimodal datasets ((P. Wu et al., 2026)), so transient content is a nuisance to be removed rather than a property to be measured.

Moving traffic in airborne registration. Vehicle measurement from the air rests on this registration ([52, 75]), and moving traffic is a recognized hazard to it. Kirchhof and Stilla (Kirchhof and Stilla [55]) warn that many objects moving in the same direction, as on dense roads, can survive RANSAC and bias the homography, and exclude points at detected motion. Karimi Nejadasl and Lindenbergh (Karimi Nejadasl and Lindenbergh [53]) rule out direct SIFT with RANSAC against the first image of a helicopter sequence, in part because of the moving vehicles, and chain consecutive homographies instead. Ljubicić et al. (Ljubicić et al. [63]) compare seven stabilization tools on UAV river footage, where the water moves and the banks do not, by residual motion at static features. None of these measures capture or what it costs in position.

Aerial detection benchmarks and trajectory datasets. Aerial object detection has well-established benchmarks ([24, 100]), DOTA made OBBs standard for aerial objects, and UAVid ([65]) separates moving from static cars in 4K UAV video for semantic segmentation. They evaluate recognition, not the registration that must precede any use of object positions when the camera moves. Drone-based trajectory datasets show the dependency most clearly: pNEUMA ([10]), inD ([14]), CitySim ([98]) and Songdo Traffic ([38]) all extract positions from hovering footage after a registration step, whose error georeferenced positions ([69]) inherit directly, and the intersections they are recorded over are where a moving-object consensus is most likely.

UAV stabilization for traffic monitoring. The work closest to this paper is Fonod et al. (Fonod et al. [38]), which extracts georeferenced vehicle trajectories from hovering-drone recordings of urban intersections and stabilizes each frame onto a reference frame with an object-masked homography, releasing that stage as stabilo ([35]) and its parameter study as stabilo-optimize. The vehicles are masked to keep correspondences on moving objects out of the fit, the failure this paper names consensus capture, but the pipeline was selected on static scenes under synthetic warps, so neither that failure nor what masking does about it was measured on the recordings. That is the gap addressed here: the static campaign is extended (§ 6) and paired with a dynamic one on the footage the pipeline is deployed on (§ 7), where the masking axis can be measured rather than assumed.

Problem setting

Scene and platform assumptions. Recordings come from hovering drones at approximately constant altitude (140 m to 150 m) in bird’s-eye view (BEV) at 4K resolution (3840 × 2160), so inter-frame motion is small and wind-induced. The ground sample distance (GSD) is about 5.5 cm per pixel: the scale of the homographies that register hovers of this experiment onto its georeferenced orthophoto [38] ranges from 0.0548 to 0.0564 m per pixel over eleven hovers. A pixel of registration error is thus about 5.5 cm on the ground, and the 5 px upper limit of the accuracy interval below is about 28 cm. Every pixel quantity in this paper converts at that rate. The stabilization model is a reference-frame homography, which holds the static scene in place over a hover exactly only where that scene is planar. Above-ground structures (buildings, poles, vegetation) depart from the plane in proportion to how far the platform translates; a hovering platform translates little but does drift, so the evaluation excludes buildings and large tree canopy, the largest such structures, by polygons drawn on each reference frame (§ 4.3) and scores the rest of the static scene. The deployed regime is a hovering platform, so the geometric perturbation a pipeline has to recover in practice is a few pixels of drift and a fraction of a degree of yaw per frame relative to the reference, growing slowly over a recording. Part I samples a wider regime than that, so that a pipeline’s operating limits are measured, and § 6 states where the two regimes differ.

Images, transforms, and the registration model

Let Ω={0,…,W−1}×{0,…,H−1}\Omega= \{0,\ldots,W-1\} \times\{0,\ldots,H-1\} be the pixel domain of a recording and let Fk:Ω→RF^k : \Omega\to\mathbb{R} denote its kk-th frame, with FrefF^{\mathrm{ref}} a designated reference frame (one fixed frame per case, chosen by the case selection of § 7.1). Write π(x1,x2,x3)=(x1/x3,x2/x3)\pi(x_1,x_2,x_3) = (x_1/x_3, x_2/x_3) for the projection from homogeneous to Euclidean coordinates, and let H\mathbb{H} be the group of planar homographies, that is, the real projective linear group PGL⁡(3,R)\operatorname{PGL}(3,\mathbb{R}) of non-singular 3×33 \times3 matrices identified up to a non-zero scalar [46]. Table 2 collects the notation. A homography H∈H\mathbf{H} \in\mathbb{H} acts on a point x∈R2\mathbf{x} \in\mathbb{R}^2 by

Table 2: Notation used throughout. Transform superscripts give the direction of the mapping, so Href→k=(Hk→ref)−1\mathbf{H}^{\mathrm{ref}\to k} = \left(\mathbf{H}^{k\to\mathrm{ref}}\right)^{-1}. Matrices and vectors are bold and scalars italic, so the homography H\mathbf{H} is distinct from the frame height HH; frame dimensions are H×WH \times W and object box extents w×hw \times h, following the usual image and detection conventions.

SymbolMeaning
Ω\Omega, H×WH \times Wpixel domain of a recording, and its size
FkF^k, FrefF^{\mathrm{ref}}frame kk; the reference frame
H\mathbb{H}group of planar homographies, PGL⁡(3,R)\operatorname{PGL}(3,\mathbb{R})
H⋅x\mathbf{H} \cdot\mathbf{x}, π\boldsymbol{\pi}action of H\mathbf{H} on a Euclidean point; homogeneous projection
Hk→ref\mathbf{H}^{k\to\mathrm{ref}}, Href→k\mathbf{H}^{\mathrm{ref}\to k}true transform current →\to reference, and its inverse
H^k→ref\widehat{\mathbf{H}}^{k\to\mathrm{ref}}a candidate pipeline’s estimate of Hk→ref\mathbf{H}^{k\to\mathrm{ref}}
H~k→ref\widetilde{\mathbf{H}}^{k\to\mathrm{ref}}the reference estimator (pseudo-ground truth)
HGCPk→ref\mathbf{H}^{k\to\mathrm{ref}}_{\mathrm{GCP}}transform fitted to the GCPs of an audited frame
⊥\perpfailure to produce an estimate
P\mathcal{P}, μ\mua pipeline and its masking strategy
Bk\mathcal{B}^k, qj\mathbf{q}_j, q‾j\overline{\mathbf{q}}_jobject annotations of frame kk; corners and center of box jj
C\mathcal{C}, ci\mathbf{c}_i, di\mathbf{d}_iframe corners and their transfer displacements
R\mathcal{R}, V\mathcal{V}, M\mathcal{M}, G\mathcal{G}evaluation region; its validity, object, and static-structure terms
τ\tau, ρ\rhoobject dilation and border erosion, in pixels
u\mathbf{u}, θˉ\bar{\theta}, κ\kappadominant motion axis, its angle, and its concentration
e∥e_{\parallel}, e⊥e_{\perp}, AAerror along and across that axis; their alignment ratio
ε\varepsilon, εfloor\varepsilon_{\mathrm{floor}}HEA threshold; GCP-audited equivalence margin
δ\deltathe equivalence margin of a comparison (§ 4.9)

Table 2.

H⋅x=π(H[x1]),(1)\mathbf{H} \cdot\mathbf{x} = \pi\left(\mathbf{H}\begin{bmatrix}\mathbf{x}\\1\end{bmatrix}\right), \tag*{(1)}

which is defined whenever the third homogeneous coordinate is non-zero. A degenerate or non-finite estimate drives that coordinate to zero, and § 4.6 explains why the resulting non-finite coordinates are propagated rather than suppressed.

Direct-to-reference registration. Each frame FkF^k is aligned to FrefF^{\mathrm{ref}} by a homography Hk→ref∈H\mathbf{H}^{k\to\mathrm{ref}} \in\mathbb{H}, estimated directly for every kk; the registered frame samples FkF^k through its inverse:

Fref(x)≈Fk(Href→k⋅x),x∈Ω.(2)F^{\mathrm{ref}}(\mathbf{x}) \approx F^k\left(\mathbf{H}^{\mathrm{ref}\to k}\cdot\mathbf{x}\right), \qquad\mathbf{x}\in\Omega. \tag*{(2)}

Sequential accumulation, Hk→ref=∏j=1kHj→j−1\mathbf{H}^{k\to\mathrm{ref}} = \prod_{j=1}^{k}\mathbf{H}^{j\to j-1}, is not used. Its error grows with kk by construction, and it is not the algorithm deployed in reference-frame stabilization [38]. Eq. (2) rests on the ground-plane assumption, which above-ground structure violates.

Pipelines under study

A pipeline is the complete chain that turns an image pair into a transform. We write it as a tuple

P=(det, desc, match, rob, μ),(3)\mathcal{P} = (\mathrm{det},\ \mathrm{desc},\ \mathrm{match},\ \mathrm{rob},\ \mu), \tag*{(3)}

whose components are, in order, the feature detector, the descriptor, the matcher, the robust estimator, and the masking strategy μ∈{none,hbb,obb}\mu\in\{\mathrm{none},\mathrm{hbb},\mathrm{obb}\} of § 5. Applied to a pair it yields

P(Fref, Fk, Bref, Bk)∈H∪{⊥},(4)\mathcal{P}(F^{\mathrm{ref}},\ F^k,\ \mathcal{B}^{\mathrm{ref}},\ \mathcal{B}^k) \in\mathbb{H}\cup\{\perp\}, \tag*{(4)}

where Bref\mathcal{B}^{\mathrm{ref}} and Bk\mathcal{B}^k are the object annotations of the reference frame and of frame $k ({xref}Section 5) and $\perp$ denotes a failure to produce an estimate at all. Admitting $\perp$ explicitly, rather than substituting the identity, is what makes the failure rate a reported quantity instead of a silent bias ({xref}Section 4.6`). We write H^k→ref\widehat{\mathbf{H}}^{k\to\mathrm{ref}} for a successful estimate.

The five axes of Eq. (3) are evaluated jointly. Component-wise evaluation is the norm in the matching literature, but the quantity a stabilization system delivers is the transform, and the interactions between the axes, particularly between μ\mu and rob, are what this benchmark is designed to expose.

The components come from two libraries: the classical detectors and descriptors and every robust estimator are OpenCV’s, and the learned detectors, descriptors and matchers are kornia’s [76]. Both are driven through stabilo [35], the stabilization library of Fonod et al. (2025a), which this benchmark evaluates and does not modify. Which implementation supplies a component is part of what is compared, for the reason given below.

Selection of pipelines. The detector axis holds ten pipelines. Five are classical, matched by brute force with a ratio test: ORB [78], SIFT [64], RootSIFT [4], BRISK [59] and AKAZE [3]. Four are learned and sparse: DISK [87], DeDoDe [25] and Key.Net [11] (KeyNet in the tables), each matched by LightGlue [61], and XFeat [71], matched by brute force because the library’s LightGlue has no weights for it. One is detector-free, LoFTR [83], which has no keypoint budget to set. The deployed library exposes one more detector, KAZE [2], which is left out: AKAZE is its accelerated variant, and it is the slowest classical detector by a wide margin. Otherwise, the set is everything the library exposes, which is the inclusion criterion: the object of study is the system a trajectory-extraction pipeline actually runs and the alternatives a practitioner can select between by configuration alone, not the state of the art in matching at large. Methods outside that system, such as SuperPoint [23] and the dense matchers of § 2, fall outside this scope; nothing in the protocol depends on the method, and it applies to them unchanged.

Robust estimators are identified by implementation, not by name. The rob axis sweeps nine OpenCV method codes spanning seven algorithms: LMEDS [77], RANSAC [31], RHO [12], GC-RANSAC [6], PROSAC [17], MAGSAC++ [8], and LO-RANSAC [18] in three configurations that differ in local-optimization iterations and in whether they run in parallel, the last four within OpenCV’s implementation of the USAC framework [73]. They are reported under OpenCV’s flag names with the algorithm in parentheses, because the three LO-RANSAC configurations are separate entries a practitioner selects from, and because two of the flags are widely described as DEGENSAC, including in the documentation of the library evaluated here, although neither carries the dominant-plane check of Chum et al. (2005) into a homography estimate (§ 4.1). Following Barath et al. (2023), a method is fixed by its implementation and version rather than by its published name (Appendix § 10).

Masking acts on feature detection. The order is: detection or segmentation →\rightarrow mask construction →\rightarrow feature detection →\rightarrow description →\rightarrow matching →\rightarrow robust estimation. No keypoint inside a mask reaches matching, so metrics defined on “removed features” or on a “dynamic inlier ratio” are not applicable here, and are not used. Where the mask enters the detector differs between implementations, and it decides what masking can do under a feature budget; every detector is used with the masking its implementation provides, unmodified. ORB, BRISK and AKAZE detect outside the mask and then keep their strongest responses up to the budget, so the keypoints a mask removes are replaced by others. SIFT, as OpenCV implements it, keeps its strongest responses over the whole frame and then drops those inside the mask; the learned detectors accept no mask, so the library drops their selected keypoints that fall inside it. For both, masking can only remove keypoints.

Evaluation methodology

This section shows why the internal statistics of a robust fit cannot rank pipelines on a dynamic scene and defines what replaces them: the metrics, the treatment of failures, comparability, the audit of the reference estimator, and the analysis plan.

Why internal-consistency metrics are insufficient

Inlier count, inlier ratio, and reprojection error measure the internal consistency of a fit: how well the correspondences the estimator retained agree with the model fitted to them. On static scenes they are already known to be weak proxies for accuracy. Barath et al. (2023) report that the inlier count depends on image resolution, on the inlier threshold and on the estimator, so that MAGSAC returns far more inliers than its competitors at comparable pose accuracy, and in the tabulation of Jin et al. (2021, Table 13) the method with the highest inlier ratio, R2D2 at 52.9%, is outperformed downstream by RootSIFT at 39.6%.

The claim here is stronger, and it follows from the scene rather than from the estimator. A homography needs four correspondences; a group of objects moving together, such as a stream of vehicles crossing an intersection, supplies hundreds that are mutually consistent under a single transform, the one describing their motion. Their coverage ranges from a small fraction of a high-altitude view to a large share of a low-altitude view over congested lanes, but the estimator weighs their share of the correspondences, not of the area, so even a small coverage can cast many votes. Outliers that form a coherent structure of their own bias a robust estimate toward a compromise between the structures or capture it outright [82], and a busy intersection produces that situation persistently. Wherever the moving group wins, every internal-consistency check is satisfied by the wrong answer: these statistics are not merely uninformative but anti-informative. They cannot tell whether it has won; § 7.3 measures that for the cases studied here.

The estimator cannot detect this failure on its own. Chum et al. [19] detect a scene dominated by a single plane with a test inside the estimator, because that case is a degeneracy: the data underconstrain the fundamental matrix, and the deficiency is visible in the sample. A homography fitted to coherently moving objects is fully determined and over-supported; it differs from the intended solution only in which part of the scene it describes, which no individual correspondence encodes. None of the estimators compared here contains a check that could apply; OpenCV runs DEGENSAC only for the fundamental matrix, and the only tests reaching a homography are conditions on the minimal sample (no three collinear points, unchanged orientation), which a stream of vehicles over several lanes satisfies. The information needed, which image content is static, is external to the estimation problem, so the failure has to be handled by the protocol.

These quantities are therefore recorded as diagnostics, to be explained rather than optimized, and never rank a pipeline. What replaces them is the rest of this section: agreement with an external geometry where one exists (§ 4.2), direct measurement of background alignment where one does not (§ 4.3), and two measures that make a moving-object consensus observable, the direction of the error (§ 4.4) and the share of a moving group’s displacement an estimate follows (§ 4.5).

Geometric metrics

Corner transfer error. Let C={c1,…,c4}\mathcal{C} = \{\mathbf{c}_1,\ldots,\mathbf{c}_4\} be the frame corners, c1=(0,0)\mathbf{c}_1 = (0,0), c2=(W,0)\mathbf{c}_2 = (W,0), c3=(W,H)\mathbf{c}_3 = (W,H), c4=(0,H)\mathbf{c}_4 = (0,H). Given a reference transform H\mathbf{H} and an estimate H^\widehat{\mathbf{H}} of its inverse direction, a corner sent forward through H\mathbf{H} and back through H^\widehat{\mathbf{H}} should return to where it started; the residual is the error. Define the per-corner displacement

di=H^⋅(H⋅ci)−ci,i=1,…,4,(5)\mathbf{d}_i = \widehat{\mathbf{H}} \cdot\left(\mathbf{H} \cdot\mathbf{c}_i\right) - \mathbf{c}_i,\qquad i = 1,\ldots,4, \tag*{(5)}

and summarize it by

CTEmean=14∑i∥di∥,CTEmax=max⁡i∥di∥,CTErms=(14∑i∥di∥2)1/2.(6)\mathrm{CTE}_{\mathrm{mean}}=\frac{1}{4}\sum_i\lVert\mathbf{d}_i\rVert,\qquad\mathrm{CTE}_{\mathrm{max}}=\max_i\lVert\mathbf{d}_i\rVert,\qquad\mathrm{CTE}_{\mathrm{rms}}=\left(\frac{1}{4}\sum_i\lVert\mathbf{d}_i\rVert^2\right)^{1/2}. \tag*{(6)}

The corner transfer error (CTE) is measured in pixels, is invariant to the parameterization of the homography, and weights the frame periphery, where a small angular error produces the largest displacement. We keep the displacement vectors of Eq. (5) and not only their norms, because their directions carry the evidence used in § 4.4.

Two directions must be distinguished. In Part I the perturbation H\mathbf{H} maps reference →\to warped and the pipeline estimates the inverse, so Eq. (5) applies directly. In Part II the reference estimator H^k→ref\widehat{\mathbf{H}}^{k\to\mathrm{ref}} and the candidate H^k→ref\widehat{\mathbf{H}}^{k\to\mathrm{ref}} both map current →\to reference, so one is inverted first and the disagreement between two same-direction transforms is

di=H^k→ref⋅(H^ref→k⋅ci)−ci.(7)\mathbf{d}_i=\widehat{\mathbf{H}}^{k\to\mathrm{ref}}\cdot\left(\widehat{\mathbf{H}}^{\mathrm{ref}\to k}\cdot\mathbf{c}_i\right)-\mathbf{c}_i. \tag*{(7)}

Homography estimation accuracy. Following DeTone et al. [23] and Efe et al. [27], we aggregate over a set of NN trials by the homography estimation accuracy (HEA): the fraction of trials that succeed within a pixel budget ε\varepsilon,

HEAε=1N∑n=1N[Pn≠⊥ ∧ CTEmean(n)≤ε].(8)\mathrm{HEA}_{\varepsilon}=\frac{1}{N}\sum_{n=1}^{N}\left[\mathbb{P}_n\ne\perp\ \wedge\ \mathrm{CTE}_{\mathrm{mean}}^{(n)}\le\varepsilon\right]. \tag*{(8)}

A failed trial counts in the denominator but never in the numerator, so declining to answer cannot raise accuracy; unlike a mean CTE, the rate is not dominated by one catastrophic trial.

Integrating the accuracy curve. Any single ε\varepsilon is arbitrary: tuning can make a method win at one threshold, yielding several simultaneous states of the art (Efe et al. [27]). Following Efe et al. [27], Jin et al. [50] and Barath et al. [9], we summarize HEAε\mathrm{HEA}_{\varepsilon} by the area under its curve, the mean average accuracy (mAA) over a threshold interval,

mAA[εlo,εhi]=1εhi−εlo∫εloεhiHEAε dε=1N∑n=1Nεhi−clip⁡(CTEmean(n),εlo,εhi)εhi−εlo,(9)\mathrm{mAA}_{[\varepsilon_{\mathrm{lo}},\varepsilon_{\mathrm{hi}}]}=\frac{1}{\varepsilon_{\mathrm{hi}}-\varepsilon_{\mathrm{lo}}}\int_{\varepsilon_{\mathrm{lo}}}^{\varepsilon_{\mathrm{hi}}}\mathrm{HEA}_{\varepsilon}\,\mathrm{d}\varepsilon=\frac{1}{N}\sum_{n=1}^{N}\frac{\varepsilon_{\mathrm{hi}}-\operatorname{clip}\left(\mathrm{CTE}_{\mathrm{mean}}^{(n)},\varepsilon_{\mathrm{lo}},\varepsilon_{\mathrm{hi}}\right)}{\varepsilon_{\mathrm{hi}}-\varepsilon_{\mathrm{lo}}}, \tag*{(9)}

where the right-hand form, which is what we evaluate, is exact because each trial enters HEAε\mathrm{HEA}_{\varepsilon} as a step in ε\varepsilon; a failed trial’s error is taken as infinite. Each trial contributes linearly in its error inside the interval and nothing beyond it: past a few pixels an estimate is wrong, and how wrong carries little information.

Both parts report mAA[0,5][0,5] px as the headline quantity; the cited benchmarks integrate over other ranges and units, so their values are not comparable with these. In Part II the integral is evaluated per case and averaged with equal weight per case, the sampling unit. The reference estimator’s audited residual (§ 4.8) does not raise the lower limit, since integrating from it would still order two candidates a tenth of a pixel apart above it. It is the equivalence margin of the reference-relative pixel metrics (§ 4.9), mAA keeps its margin of 0.01, and mAA[εfloor,5][\varepsilon_{\mathrm{floor}},5] is reported beside the headline as a sensitivity check.

Object-level agreement. Frame corners describe the transform globally; the annotated objects describe where the content of interest actually lands. For each annotated object jj let qj∈R4×2\mathbf{q}_j\in\mathbb{R}^{4\times2} hold the corners of its box as the reference geometry places them in the reference frame, and q^j\hat{\mathbf{q}}_j the corners of the same box as the candidate places them, each corner transformed on its own. In Part I these are a box of Bref\mathcal{B}^{\mathrm{ref}} and its round trip through Eq. (5); in Part II they are a box of Bk\mathcal{B}^{k} carried into the reference frame by H^k→ref\widehat{\mathbf{H}}^{k\to\mathrm{ref}} and by H^k→ref\widehat{\mathbf{H}}^{k\to\mathrm{ref}}. We define two per-object errors,

IoUj=IoU⁡(qj,q^j),OCEj=∥qˉj−q^ˉj∥,(10)\mathrm{IoU}_j=\operatorname{IoU}(\mathbf{q}_j,\hat{\mathbf{q}}_j),\qquad\mathrm{OCE}_j=\lVert\bar{\mathbf{q}}_j-\bar{\hat{\mathbf{q}}}_j\rVert, \tag*{(10)}

where IoU⁡(⋅,⋅)\operatorname{IoU}(\cdot,\cdot) is the ratio of intersection to union area of two convex quadrilaterals and qˉj=14∑iqj,i\bar{\mathbf{q}}_j=\frac{1}{4}\sum_i\mathbf{q}_{j,i} is the center of box jj. IoUj_j is threshold-free and bounded in [0,1][0,1]; the object center error (OCE) is measured in pixels and converts to meters through the GSD of § 3. Over the objects of one frame, each is summarized by its mean (the headline value), standard deviation, quartiles and extremes; the minimum of IoUj_j and the maximum of OCEj_j expose a registration that is accurate on average but fails at one edge of the frame, which a mean hides.

The two errors answer different questions. IoUj_j depends on object size as well as on registration: a displacement that barely changes the overlap of a bus removes much of a motorcycle’s overlap. OCEj_j does not, and it is the quantity a trajectory-extraction pipeline inherits, since such a pipeline carries each detected box into the reference frame corner by corner and reads the object’s position off the carried quadrilateral. Because qj\mathbf{q}_j and q^j\hat{\mathbf{q}}_j carry the same box, the detector’s localization error cancels, and OCE isolates what registration adds to a trajectory.

Both errors use the OBBoxes for every pipeline, whatever its masking strategy μ\mu, so no candidate is scored on a geometry its own strategy chose; each q^j\hat{\mathbf{q}}_j is compared as the quadrilateral it becomes, with no rectangle refitted and no axis-aligned hull, which would conflate registration with orientation.

Reference-free background-alignment metrics

On real footage no true homography exists, so alignment is measured directly: FrefF^{\mathrm{ref}} is compared with the warped current frame Fk∘H^ref→kF^{k}\circ\widehat{H}^{\mathrm{ref}\to k} over an evaluation region that excludes the annotated objects and raised structures.

The evaluation region. Write S⊕DrS\oplus D_r and S⊖DrS\ominus D_r for the morphological dilation and erosion of a pixel set SS by a disc DrD_r of radius rr [81]. Let MrefM^{\mathrm{ref}} be the pixels covered by the reference frame’s annotated objects and MkM^k those of the current frame’s, both rasterized in reference coordinates. The region is

R=(V∩Mref⊕Dτ‾∩Mk⊕Dτ‾∩G)⊖Dρ,(11)\mathcal{R}=\left(\mathcal{V}\cap\overline{M^{\mathrm{ref}}\oplus D_\tau}\cap\overline{M^k\oplus D_\tau}\cap\mathcal{G}\right)\ominus D_\rho, \tag*{(11)}

where V={x∈Ω:H^ref→k⋅x∈Ω}\mathcal{V}=\{\mathbf{x}\in\Omega:\widehat{H}^{\mathrm{ref}\to k}\cdot\mathbf{x}\in\Omega\} is the set of pixels receiving real data under the warp, less a two-pixel fringe of interpolated values, G\mathcal{G} is the frame less polygons drawn by hand over the buildings, other raised structures and large contiguous tree canopy of the reference frame, τ\tau is a constant dilation in pixels, and the erosion by DρD_\rho keeps the assembled region a margin away from every one of its boundaries: the frame edge, the edge of the warped data, and the dilated objects. Each term is necessary: without V\mathcal{V} a homography is rewarded for pushing empty area into frame; without the object terms a busier scene scores worse for every pipeline; without G\mathcal{G} the parallax of roofs and canopy under the platform’s drift is charged to the pipeline; without the erosion, pixels at those boundaries, where the warp interpolates and the masks meet the background, enter the metric. The polygons are drawn once per case, before any of its results are read, and depend on no pipeline.

Two properties keep Eq. (11) from being tunable by the pipeline under test. The region is common to all candidates: MkM^k is placed into reference coordinates by the reference estimator’s H^k→ref\widehat{H}^{k\to\mathrm{ref}}, never by the candidate’s own H^k→ref\widehat{H}^{k\to\mathrm{ref}}, and only V\mathcal{V} is candidate-specific, because it must describe the image that candidate produced. And the dilation τ\tau belongs to the protocol: it is a fixed pixel count, independent of the margin by which a pipeline grows its feature-suppression masks (§ 5). ∣R∣/∣Ω∣|\mathcal{R}|/|\Omega| is recorded with every measurement, and a measurement whose region falls below a minimum area fraction is scored as the identity warp (§ 4.6). Part I computes no reference-free measure.

Four complementary measures. Over R\mathcal{R} we compute structural similarity (SSIM) [89], zero-mean normalized cross-correlation (ZNCC), mutual information (MI) [67, 88], and a magnitude-weighted agreement of gradient orientation (GOS), a normalized form of the gradient terms used in multimodal registration [70, 45]. Writing A=FrefA=F^{\mathrm{ref}} and B=Fk∘H^ref→kB=F^k\circ\widehat{H}^{\mathrm{ref}\to k} restricted to R\mathcal{R}, with means Aˉ,Bˉ\bar{A},\bar{B} and standard deviations σA,σB\sigma_A,\sigma_B,

ZNCC⁡(A,B)=1∣R∣σAσB∑x∈R(A(x)−Aˉ)(B(x)−Bˉ)∈[−1,1],(12)\operatorname{ZNCC}(A,B)=\frac{1}{|\mathcal{R}|\sigma_A\sigma_B}\sum_{\mathbf{x}\in\mathcal{R}}(A(\mathbf{x})-\bar{A})(B(\mathbf{x})-\bar{B})\in[-1,1], \tag*{(12)}
MI⁡(A,B)=∑a,bpAB(a,b)log⁡2pAB(a,b)pA(a) pB(b),(13)\operatorname{MI}(A,B)=\sum_{a,b}p_{AB}(a,b)\log_2\frac{p_{AB}(a,b)}{p_A(a)\,p_B(b)}, \tag*{(13)}

in bits, with pABp_{AB} the joint intensity histogram over R\mathcal{R} (64 × 64 bins) and pA,pBp_A,p_B its marginals, and

GOS⁡(A,B)=∑x∈Rw(x)cos⁡(∠∇A(x)−∠∇B(x))∑x∈Rw(x)∈[−1,1],w(x)=(∥∇A(x)∥∥∇B(x)∥)1/2,(14)\operatorname{GOS}(A,B)=\frac{\sum_{\mathbf{x}\in\mathcal{R}}w(\mathbf{x})\cos\left(\angle\nabla A(\mathbf{x})-\angle\nabla B(\mathbf{x})\right)}{\sum_{\mathbf{x}\in\mathcal{R}}w(\mathbf{x})}\in[-1,1],\qquad w(\mathbf{x})=\left(\lVert\nabla A(\mathbf{x})\rVert\lVert\nabla B(\mathbf{x})\rVert\right)^{1/2}, \tag*{(14)}

with Sobel gradients. The weighting makes Eq. (14) usable on aerial asphalt: flat regions, whose gradient direction is arbitrary, contribute nothing, and road markings and curbs dominate; the geometric mean of the two magnitudes keeps a few high-contrast edges from outweighing the rest quadratically.

The four differ in what they tolerate: ZNCC an affine change of intensity, MI any one-to-one remapping, Eq. (14) any monotonically increasing change, and SSIM, which compares local luminance, contrast and structure separately, the least photometric change of the four. Agreement among them is evidence that a measured alignment is not an artifact of one measure’s blind spot.

Decomposing error along the dominant motion axis

The mechanism studied here leaves a geometric signature: a homography dragged by a moving-object consensus has its error along the direction of motion, whereas ordinary estimation noise has no preferred direction. Measuring that signature makes capture testable.

Recovering the motion axis. Elongated objects such as vehicles travel along their own long axis, so the orientation distribution of the OBBs recovers the direction of flow without any correspondence between frames, which matters because the pairs here reach thousands of frames apart and no tracker links their objects. Let θj\theta_j be the orientation of the long axis of box jj of the reference frame and aja_j its area. Orientations are axial data: a box rotated by π\pi is the same box, so the mean is taken over doubled angles and halved back ([68]). With the weighted resultant

R=∑jaje2iθj∑jaj,θˉ=12arg⁡R,κ=∣R∣∈[0,1],(15)R = \frac{\sum_j a_j e^{2i\theta_j}}{\sum_j a_j}, \qquad\bar{\theta} = \frac{1}{2}\arg R, \qquad\kappa= |R| \in[0,1], \tag*{(15)}

the dominant axis is the unit vector u=(cos⁡θˉ,sin⁡θˉ)\mathbf{u} = (\cos\bar{\theta},\sin\bar{\theta}) and κ\kappa is its concentration.

Decomposing the error. With u⊥=(−sin⁡θˉ,cos⁡θˉ)\mathbf{u}^{\perp} = (-\sin\bar{\theta},\cos\bar{\theta}) and the corner displacements di\mathbf{d}_i of Eq. (7), define

e∥=14∑i∣di⋅u∣,e⊥=14∑i∣di⋅u⊥∣,A=e∥e∥+e⊥∈[0,1].(16)e_{\parallel} = \frac{1}{4}\sum_i|\mathbf{d}_i\cdot\mathbf{u}|,\qquad e_{\perp} = \frac{1}{4}\sum_i|\mathbf{d}_i\cdot\mathbf{u}^{\perp}|,\qquad A = \frac{e_{\parallel}}{e_{\parallel}+e_{\perp}} \in[0,1]. \tag*{(16)}

A≈0.5A \approx0.5 is isotropic error; A→1A \to1 is error almost entirely along the motion axis, the signature of a moving-object consensus; and A→0A \to0 is error across it, which that mechanism cannot produce. Error without a preferred direction need not give A=0.5A = 0.5 at four corners, however: a small rotation about the frame center displaces each corner perpendicular to its diagonal and a small scale change along it, so on a 16:9 frame with u\mathbf{u} along an image axis they give AA of about 0.36 and 0.64. The level of AA is therefore read against the identity transform, whose error is the platform’s own motion, and the evidence for capture is an increase of AA without masking over AA with masking on the same pairs, not its level.

The bounded form is used instead of the bare ratio e∥/e⊥e_{\parallel}/e_{\perp}, which diverges exactly in the case of interest (e⊥=0e_{\perp}=0) and would drop the strongest evidence from every aggregate. At a crossroads with two busy perpendicular approaches the doubled angles cancel and κ\kappa falls toward zero, so the decomposition is suppressed when κ<κmin⁡=0.3\kappa< \kappa_{\min} = 0.3 rather than reporting an artifact. Eq. (15) also needs oriented annotations, since an HBB carries no orientation.

Measuring capture directly

The direction of the error is a signature of capture read at the four frame corners, far from the objects that cause it. Capture can also be measured where it happens. The reference frame’s objects are found again in frame kk by the procedure the case selection uses (§ 7.1): each box is compared by appearance with the boxes of similar size in frame kk and accepted only as a mutual best match. With yj\mathbf{y}_j, the center of the match of reference box jj, its displacement with the platform’s motion removed by the reference estimator is

mj=H~k→ref⋅yj−qˉj.(17)\mathbf{m}_j = \widetilde{\mathbf{H}}^{k\to\mathrm{ref}}\cdot\mathbf{y}_j-\bar{\mathbf{q}}_j. \tag*{(17)}

A box has moved when ∥mj∥\|\mathbf{m}_j\| exceeds a tenth of the median object width: clear of the apparent motion of a stationary object, which comes from box placement and the reference estimator’s error, and less than a car covers between consecutive frames at urban speeds. Moved objects move together with moved object ii when their displacements differ from mi\mathbf{m}_i by at most a tenth of the median object width or a quarter of ∥mi∥\|\mathbf{m}_i\|, whichever is larger: the members of one stream drift apart in proportion to how far it moved, while opposing or crossing streams never agree. The largest set by area that moves together with one of its members is the group, GG, the structure a single wrong homography could fit. The capture fraction of a candidate is the share of the group’s displacement it follows,

λ=median⁡j∈G−(H~k→ref⋅yj−H~k→ref⋅yj)⋅mj∥mj∥2.(18)\lambda= \operatorname{median}_{j\in G}\frac{-\left(\widetilde{\mathbf{H}}^{k\to\mathrm{ref}}\cdot\mathbf{y}_j-\widetilde{\mathbf{H}}^{k\to\mathrm{ref}}\cdot\mathbf{y}_j\right)\cdot\mathbf{m}_j}{\|\mathbf{m}_j\|^2}. \tag*{(18)}

An estimate that agrees with the reference estimator where the group lies has λ=0\lambda= 0, one that registers the group back onto where it stood in the reference frame, which is what a homography fitted to the group does, has λ=1\lambda= 1; values between are a compromise fit, pulled toward the group without registering it. A measurement is captured when λ≥0.1\lambda\geq0.1: the estimate has moved, where the group lies, by at least a tenth of the group’s displacement. That lies above what a transform reaches without following the group: the identity, which uses no features, reaches 0.07 from the platform’s motion alone, and the reference estimator’s audited error (§ 7.2) amounts to 0.08 on the smallest displacement of any group, 5 px (Table 7). Capture is reported descriptively, with the counts at λ≥0.25\lambda\geq0.25, 0.5 and 1 beside it (files offline_masking_capture.csv and realtime_masking_capture.csv; the CSV files named in this paper are released with its data (Fonod and Geroliminis, 2026a})). λ\lambda is defined on pairs whose group holds at least four objects, and only where the pipeline returned an estimate. Beside it, every measurement records where the estimate’s inliers lie: the share on annotated objects and the share on objects that moved. The second undercounts what moves with the objects, since a vehicle’s cast shadow lies outside its box; λ\lambda does not depend on it. These quantities are diagnostics, like the inlier statistics they are read against, and never rank a pipeline.

PipelineMaskingCapturedShareCasesλmax⁡\lambda_{\max}CTE50\mathrm{CTE}_{50} [px]AuditedCTE50GCP\mathrm{CTE}^{\mathrm{GCP}}_{50} [px]InliersInlier rankRatioRatio rankOn moving
ORBnone289.2%56.0318.844.32900.580.160.5813%
SIFTnone20.7%20.110.921.06420.500.870.500%
RootSIFTnone20.7%10.120.911.05640.690.850.440%
BRISKnone62.0%10.171.611.24640.810.880.693%

Table 7. Part II (real-time regime at downsample ratio 0.18 and 1000 features): consensus capture over the 9 cases, the configurations captured on at least one pair. A pair counts when at least four of the reference frame’s objects moved together; an estimate on it is captured when it follows at least 10% of that group’s displacement (λ≥0.1\lambda\ge0.1). Cases: cases with a capture; CTE50\mathrm{CTE}_{50}: median corner error of the captured against the reference estimator, and CTE50GCP\mathrm{CTE}^{\mathrm{GCP}}_{50} against the ground control points on the Audited frames; Inliers and Ratio: median inlier count and ratio of the captured, with their median percentile among the same configuration’s estimates at the same case and offset bin (0.5 typical, 1 the most); On moving: median share of their inliers on objects that moved. Descriptive; nothing is bolded. The other 26 configurations are captured on no pair, with a largest λ\lambda of 0.09. No masked estimate exceeds λ=0.05\lambda= 0.05. The identity transform reaches λ=0.07\lambda= 0.07, and the smallest displacement of any group is 5 px. Every configuration, per offset bin: realtime_masking_capture_by_offset. Every row counts 304 such pairs.

Treatment of failures

A registration pipeline can fail outright: too few matches survive filtering, or the robust estimator returns a degenerate transform. Implementations commonly substitute the identity, the stabilization library evaluated here among them [35], which keeps a campaign running but hides the failure and biases every metric, because the identity is a plausible answer for a nearly stationary drone. We therefore admit ⊥\bot explicitly in Eq. (4) and detect it from what the estimator returns: an absent inlier count, which every fallback leaves, or a missing or non-finite matrix. An estimate exactly equal to the identity is not a failure, since between near-identical frames the estimator returns exactly the identity with thousands of inliers, and that answer is correct. A finite estimate that drives the third homogeneous coordinate of Eq. (1) to zero at a corner yields non-finite coordinates, which Eqs. (8) and (9) score as a miss; clamping that division, as some implementations do, would score a collapsing estimate as merely inaccurate.

The reference-free measures need a different treatment, because an undefined value drops out of a mean: a pipeline that failed on a third of its pairs would otherwise report its background alignment over the two thirds it solved. A failed pair is therefore scored with the identity warp, the transform the pipeline actually delivered, and so is a pair whose evaluation region is too small to be read; declining to answer costs exactly what delivering the identity costs.

A configuration that exhausts the benchmark machine’s memory is not a failure. It is reported as not available and its partial measurements are discarded, because a sparser scene needs less memory and is also easier to register, so the subset that fit would flatter it. Counting it as a failure would make a score depend on the machine’s memory.

Placing pipelines on equal footing

A benchmark that compares ten pipelines at one shared set of matcher settings measures the settings as much as the pipelines. This is the central methodological finding of the benchmarks reviewed in § 2, and we adopt their remedies.

Two hyperparameters are tuned per pipeline. The ratio-test threshold and the inlier threshold of the robust estimator are both known to be decisive and to have method-specific optima: the first thresholds a descriptor distance distribution, a property of the descriptor, and the second a reprojection residual, a property of keypoint localization accuracy. Efe et al. [27] obtain their entire result by varying the first; Jin et al. [50] find its optimum differs per feature. They are swept jointly rather than sequentially (the inlier threshold alone where the matcher applies no ratio test: LightGlue and LoFTR), following Barath et al. [9]: a looser ratio test admits more outliers, which moves the inlier threshold that best separates them, so the sequential optimum need not be the joint one. Selection uses Eq. (9), the quantity the benchmark ranks on, because the threshold maximizing correspondence-level accuracy is not the one maximizing homography accuracy: a robust estimator turns a surplus of looser matches into a better fit (Efe et al., [27]). It is also conservative: a pipeline keeps the values its preset carries unless the best grid point beats them on mAA by more than the equivalence margin of 0.01 (§ 4.9), because where the optimum is flat the best grid point only moves the values within a set the data cannot tell apart; Part II starts from Part I’s values in the same way.

The remaining estimator parameters are fixed, on published grounds. Of the three parameters every robust estimator here exposes, the confidence level, the inlier threshold and an iteration cap, only the threshold is tuned. Jin et al. [50] “find the confidence value to be the least sensitive” of the three and fix it at 1−10−61 - 10^{-6}, the value used here. The iteration cap is a time budget rather than an accuracy parameter, and the benchmarks we follow treat it as one: Barath et al. [9] fix it at 1000 while grid-searching the thresholds and sweep it only to draw a time–accuracy curve, and Jin et al. [50] set it from a per-pair time budget. We fix it at 5000, where it is not the binding constraint, and report wall-clock time so that no pipeline gains accuracy through unreported computation. All remaining parameters are held at documented values across pipelines (Appendix § 10).

Tuning and evaluation scenes are disjoint. The sweep runs on scenes held out from every reported result, a split the benchmark’s tooling enforces: a threshold chosen on the data a number is reported on measures how much tuning freedom each pipeline was granted, and Barath et al. [9] name the lack of such a split as a drawback of earlier homography datasets. The split is by site, not by frame: the Part I set holds eight intersections recorded by more than one drone, near-duplicate views of the same road layout, markings and parked vehicles, so a frame-wise split would leak the evaluation scenes into the tuning half. Every third site in alphabetical order goes to tuning, which reproduces the split from the file names alone. Of the 29 scenes, 9 from 7 sites tune and 20 from 13 sites evaluate. Part II keeps the split: its evaluation cases are drawn from evaluation sites and its own tuning cases from tuning sites. Tuning is run with masking disabled and the same parameters are used under every masking mode, so no mode is tuned into its own best configuration; because the deployed configuration is masked, this is the conservative choice for the masking comparison, and any advantage measured for masking is measured with parameters tuned against it.

The compute budget is fixed before anything is tuned. Two further parameters, the downsampling ratio and the feature budget, decide how much work a pipeline does rather than how it uses it, and the shipped presets do not hand every pipeline the same amount: the learned models’ ratios are memory bounds. Jin et al. [50] report every method at matched keypoint budgets for this reason, and Barath et al. [9] report accuracy against runtime. We therefore sweep both parameters over every pipeline on the tuning scenes first, under masking disabled and under OBB masking, and read off a single matched operating point: the lowest budget at which every pipeline that can reach it is within the equivalence margin of its own saturation under both masking conditions. Masking removes the highest-contrast content in the frame, so the saturating budget need not be the same with and without it. If no budget on the grid meets the criterion, every pipeline runs at the largest one and its shortfall from saturation is reported. The thresholds are then selected at that operating point, so that what is tuned is what is run; a pipeline that cannot reach it on the benchmark machine is reported as such rather than scored at a lower budget. Part II runs at two other points of the same grid, one for offline and one for real-time registration (§ 7.1); its thresholds are tuned again at each and stated in pixels of the downsampled frame, so that they keep their size relative to keypoint localization as the frame shrinks.

Auditing the reference estimator

Part II has no true homography, so a strong pipeline is used as a reference estimator H~k→ref\widetilde{H}^{k\to\mathrm{ref}} (§ 7.2). This is pseudo-ground truth: every Part II number is a comparison against an estimate that has its own error, and one drawn from the same family as the candidates cannot be assumed neutral between them (Brachmann et al. [15]). We measure that error against independent control points rather than assuming it negligible.

A set of unambiguous ground-plane GCPs is chosen by hand in FrefF^{\mathrm{ref}} and located in a subset of sampled frames, as described below. For a label ℓ\ell present in both, with pℓref\mathbf{p}^{\mathrm{ref}}_{\ell} and pℓk\mathbf{p}^{k}_{\ell} its annotated positions, the residual is

rℓ=∥H~k→ref⋅pℓk−pℓref∥(19)r_{\ell} = \left\lVert\widetilde{H}^{k\to\mathrm{ref}}\cdot\mathbf{p}^{k}_{\ell}-\mathbf{p}^{\mathrm{ref}}_{\ell}\right\rVert \tag*{(19)}

and the upper percentile of its distribution pooled over frames is a resolution floor

εfloor=Q0.95({rℓ}).(20)\varepsilon_{\mathrm{floor}} = Q_{0.95}\left(\{r_{\ell}\}\right). \tag*{(20)}

Differences between candidates smaller than εfloor\varepsilon_{\mathrm{floor}} lie within the reference estimator’s own uncertainty and are not interpreted as evidence.

Because the GCPs depend on no pipeline, Eq. (19) applies equally to a candidate H^k→ref\widehat{H}^{k\to\mathrm{ref}}, and this is the comparison the masking result rests on. The reference runs with OBB masking, the configuration expected to be most accurate, so a candidate run under the same masking shares its feature population (the same background pixels, the same exclusions, the same movers the detector missed), and its error is positively correlated with the reference’s: its disagreement with the reference understates its true error, while an unmasked candidate shares less and carries more of the reference’s error on top of its own. A masking comparison read off reference-relative numbers is therefore biased toward the reference’s own masking mode, and for the reference’s own detector under OBB masking it is circular by construction; Eq. (20) measures the reference’s absolute error on the audited points but says nothing about that covariance. The GCPs break the dependence. On every audited frame we fit a homography to the points themselves,

HGCPk→ref=arg⁡min⁡H∈H ∑ℓ∥H⋅pℓk−pℓref∥2,(21)H_{\mathrm{GCP}}^{k\to\mathrm{ref}} = \underset{H\in\mathcal{H}}{\arg\min}\ \sum_{\ell}\left\lVert H\cdot\mathbf{p}^{k}_{\ell}-\mathbf{p}^{\mathrm{ref}}_{\ell}\right\rVert^{2}, \tag*{(21)}

by the normalized direct linear transform with a nonlinear refinement of the reprojection error (Hartley and Zisserman, 2004), from at least 12 points spread to the frame periphery: the number and placement of control points govern accuracy in UAV photogrammetry (Sanz-Ablanedo et al. [79]), and a homography fitted to central points is poorly determined exactly where the CTE is read. Its error depends on no pipeline. Every candidate is scored against HGCPk→refH_{\mathrm{GCP}}^{k\to\mathrm{ref}} on the audited frames with the metrics of § 4.2, and those scores, with the reference-free measures, are the primary evidence for the masking comparison; the reference-relative scores over all pairs remain evidence for behavior with temporal distance and for ranking detectors within one masking mode, where whatever the reference’s OBB masking favors is common to every candidate compared. The tables of these scores flag SIFT and RootSIFT, which share the reference’s keypoint detector and whose reference-relative scores are therefore the least independent of it.

Placement of the points. Points are placed in a magnified panel with an exact pixel mapping. In FrefF^{\mathrm{ref}} they are chosen and annotated manually, on the road surface, clear of every object and of the excluded structure. In each audited frame every point is first proposed by an image fit that shares nothing with the candidates or the reference estimator: the point’s 81 × 81-pixel neighborhood in FrefF^{\mathrm{ref}}, with the objects of both frames filled with the background mean, is aligned to the frame by an affine enhanced correlation coefficient fit [30] started from the platform motion of the case selection (§ 7.1), and the proposal stands only if the fit correlates at 0.8 or better and the fit back from it returns within half a pixel of the reference point. The annotator sees each proposal beside the point’s reference neighborhood and accepts it or places the point manually instead; a dense local fit and a sparse whole-frame homography fail in different ways, so their agreement is evidence about the estimator, and the review guards against the failures of the fit itself (a neighboring feature, an occlusion, a point that left the frame). A point without a standing proposal is placed by hand in a panel that opens near where H~k→ref\widetilde{\mathbf{H}}^{k\to\mathrm{ref}} predicts it, displaced by a hidden offset of a few pixels, so that a click following the panel rather than the feature can only inflate the residual. On two cases the audited frames were also placed entirely by hand (Table C.6): pooled over both, the unaided clicks lie a median of 0.44 px (0.82 px at the 95th percentile) from the proposals and 0.46 px (0.91 px) from H~GCPk→ref\widetilde{\mathbf{H}}_{\mathrm{GCP}}^{k\to\mathrm{ref}}’s predictions, while the proposals lie 0.06 px (0.32 px) from the predictions; on each case the proposals lie closer to the predictions than either lies to the clicks, which separates the click noise from the estimator’s error. The GCP repeatability is measured by annotating the audited frames of two cases a second time in a separate session without proposals, the reference frame included; it is the annotation floor, reported beside Eq. (20) so the two are not conflated. The residual is reported per temporal-offset bin as well as pooled, because the reference’s error plausibly grows with offset as shadows move and content turns over; the first of the four audit bins holds offsets of one to three frames, at which a moving group is still shared with the reference, so that a captured estimate can be scored against the GCPs as well.

Finally, the evaluation region of Eq. (11) places the current frame’s objects with H~k→ref\widetilde{\mathbf{H}}^{k\to\mathrm{ref}}, so a reference error of ε\varepsilon misplaces the object exclusion by ε\varepsilon. The dilation τ\tau absorbs that only while τ>εfloor\tau> \varepsilon_{\mathrm{floor}}, so the reference-free family uses the reference only to place the exclusion, which still covers the objects while that holds; the implementation checks the audited residual against τ\tau and warns when the condition fails. A candidate’s own error is not absorbed: a misregistered frame carries its objects off their exclusion, and their pixels enter its own score.

Analysis plan

The plan below was fixed before the campaigns it governs ran, and the implementation applies it when it renders a table, so that no number is bolded, ranked, or called a difference by a rule chosen after the outcome was seen. Like the comparisons of learning methods over several data sets [21], it treats the independent unit, not the measurement, as the sample, and uses paired nonparametric tests over it.

Units and clusters. In Part I one measurement is a (scene, trial) pair. The perturbation depends on the scene and the trial only, never on the pipeline, so pipelines are compared on paired per-trial differences. The independent unit is the site, of which the evaluation set has 13: two frames of one intersection from two drones show the same static scene and are not independent events. In Part II one measurement is a frame pair under one pipeline and one masking mode. Masking is a within-pair factor, since all three modes see the same pair, so the masking effect is a paired difference per pair. The independent unit is the case, of which there are nine, and frame pairs within a case are never treated as independent. The nine lie at seven intersections, two of them hovered over in two sessions; a case is defined by the traffic at its reference frame, which two sessions do not share, so the case rather than the intersection is the unit.

Outcomes. Part I ranks on mAA[0, 5] of Eq. (9). The Part II masking claim has one primary contrast, OBB masking against none for the average pipeline (per case, the mean paired benefit over the pipelines measured on every outcome in every case), on five outcomes: the CTE against H~GCPk→ref\widetilde{\mathbf{H}}_{\mathrm{GCP}}^{k\to\mathrm{ref}} on the audited frames, the reference-free ZNCC and gradient-orientation agreement over the evaluation region, and intersection over union (IoU) and OCE against H~GCPk→ref\widetilde{\mathbf{H}}_{\mathrm{GCP}}^{k\to\mathrm{ref}} on the audited frames. The contrast is claimed when at least one outcome favors OBB masking after a Holm adjustment [48] across the five and none favors no masking at the unadjusted level; the second condition can only withhold a claim. The outcomes differ in sensitivity: those against H~GCPk→ref\widetilde{\mathbf{H}}_{\mathrm{GCP}}^{k\to\mathrm{ref}} see eight audited frames per case, the reference-free ones every pair, so requiring all five to agree would tie the claim to the least sensitive. Secondary are the same contrast per pipeline, and horizontal against none and oriented against horizontal, both for the average pipeline and per pipeline. The mechanism claim has three pre-specified contrasts, each formed within a case and then read across cases: the oriented-against-none benefit increases with the coverage of the current frame; it is smaller in pairs whose foreground did not move than in pairs whose foreground did, at matched coverage (§ 7.1); and the motion alignment A of Eq. (16) is larger without masking than with OBB masking. Each is read for the average pipeline, as the primary contrast is, with a Holm adjustment across its outcomes (ZNCC and gradient-orientation agreement for the first two, A for the third), and is supported when an outcome’s interval lies above zero at an adjusted pp of at most 0.05; the per-pipeline readings are secondary. Capture itself is reported descriptively, per pipeline, masking mode and offset bin: the share of pairs with a moving group on which estimate is captured (Eq. (18)), the cases in which that happens, and the inlier count and ratio of captured estimates beside those of the others. It depends on the traffic each case happens to carry at its reference frame, so it describes these cases and is not tested as a population rate.

Estimates and intervals. Every comparison reports the paired difference as an effect size: the Hodges–Lehmann estimate [47] of the per-cluster paired differences, with a 95 % percentile bootstrap interval [20] obtained by resampling sites in Part I and cases in Part II, with 10000 draws. Every outcome, mAA included, is computed per cluster and differenced there, so an interval on mAA is an interval on the cluster-level paired difference. The test is an exact sign-flip permutation [43] of the per-cluster paired differences: 2132^{13} flips in Part I and 29=5122^9 = 512 in Part II, so the smallest two-sided pp attainable in Part II is 2/512=0.00392/512 = 0.0039. After the Holm adjustment across five outcomes, a masking effect is detectable at the 0.05 level only if it has the same sign in all nine cases, or in eight with the dissenting case the smallest in magnitude (4/512×5=0.0394/512 \times5 = 0.039). The design detects a consistent effect and nothing weaker. The 2×22 \times2 over moving foreground and coherence, with two cases per cell, is reported descriptively, without intervals or an interaction test; the within-case mechanism contrasts test its prediction.

Criterion for a difference. A difference is claimed only when the bootstrap interval excludes zero and the point estimate exceeds an equivalence margin δ\delta, the smallest difference worth acting on [58]. In Part I the ground truth is exact and δ=0.01\delta= 0.01 in mAA, one percent of its [0,1][0, 1] range, so that two thousand paired trials cannot declare a difference no reader would act on. In Part II, reference-relative pixel metrics use the audited residual, δ=εfloor\delta= \varepsilon_{\mathrm{floor}}, which for a comparison pooled over the cases is the largest of the per-case residuals; pixel metrics against HGCPk→ref\mathbf{H}_{\mathrm{GCP}}^{k\to\mathrm{ref}} use the 95th percentile of the GCP repeatability, the larger of the two repeat sessions’; and IoU and the reference-free measures use no margin, since their units carry no floor, and are reported with intervals alone.

Multiplicity and presentation. Part I has one primary outcome, mAA, and Part II one primary contrast. Secondary contrasts are reported with intervals and Holm-adjusted pp-values (per pipeline, across the pipelines) and mark the differences that pass the rule above, but carry no claim. Every table bolds the best value in a column and every value not distinguishable from it under the rule above, and says so in its caption; columns that are diagnostics are never bolded. Bolding applies the rule without a multiplicity adjustment and is descriptive: it establishes neither equivalence within the bold set nor an adjusted difference outside it. The estimate, interval and adjusted pp behind each of those decisions are reported in Appendix § 13. The correlation between inlier ratio and alignment quality (Figure 10) is computed within each pipeline and case and its distribution is shown, never a single pooled coefficient, because estimators differ systematically in how many inliers they report at equal accuracy [9] and a pooled correlation would measure that mix.

Figure 10. Inlier ratio against the disagreement with the reference estimator, offline (top) and in real time (bottom). In each row, left, the density of frame pairs and pipelines, one panel per masking mode, darker where more of them fall, on a logarithmic scale of the count; right, the Spearman coefficient between the two within each pipeline and case, one point per group, with the median printed beside it and the number of positive coefficients under each masking mode. A negative coefficient means that a higher inlier ratio goes with a smaller error.

Masking configurations and their annotations

Three configurations are evaluated, and the complete benchmark is repeated for each: no masking, HBB masks, and OBB masks. Under the latter two, the annotated boxes of both frames are rasterized into a binary mask that the feature detector receives, so that no keypoint is detected on an annotated object; an HBB is rasterized as an axis-aligned rectangle and an OBB as a quadrilateral. Each box is first scaled about its center by 1+m1 + m with the mask margin m=0.15m = 0.15, to absorb annotation slack, shadow and motion blur at the object boundary. That margin belongs to the pipeline and is a hyperparameter of it, distinct from the constant dilation τ\tau of the evaluation region in Eq. (11), which belongs to the protocol. The HBB is the detector’s and the OBB is fitted to the segmented vehicle, so the two differ in extent as well as in shape and neither need contain the other; the suppressed-area gap between the two geometries is reported for every scene set.

Rationale for two box geometries. Whether the shape of a mask matters for registration is not obvious in advance, and plausible mechanisms point both ways. An HBB around a vehicle turned off the image axes also covers the road beside it, two wedges of asphalt that at an intersection often hold lane markings, stop lines and curb edges, the static texture the estimator most needs; the wedges grow with the angle, to about twice the vehicle’s own area for a 40×10 box at 45°, and in a queue of turning vehicles adjacent HBBs merge over the markings between them. OBBs keep that texture. Conversely, what moves with a vehicle is not confined to its body: its cast shadow, its motion blur and the reflections off its windows travel with it, lie partly outside a tight OBB, and supply correspondences that agree with its motion, and the extra area of an HBB covers part of them. Which effect dominates, if either, is an empirical question, so the box geometry is a controlled axis of both parts, and the margin sweep of § 7.5 compares the two geometries at equal suppressed area, so that a difference can be told apart from the area each removes. OBBs are also what the motion axis of § 4.4 and the object-level errors of § 4.2 are read from, whatever the masking.

The annotations are the same for both parts and are produced outside the benchmark, once per image set, by the detector released with geo-trax [32] for the HBBs and by the conversion of Appendix § 14 for the oriented ones. Neither model runs inside the benchmark loop: a campaign re-reads the same frames tens of thousands of times, and two pipelines are only comparable if they were handed identical masks, which a per-run detector call cannot promise. Each label set carries a provenance record naming the command, the settings, the package versions and the checkpoint checksums that produced it, because annotations are not bit-reproducible across accelerators and the published annotations, not the command, are the reproducible artifact.

Oriented annotations and the Songdo Vision OBB release

OBB masks require an oriented annotation for every object in every frame the benchmark touches, orders of magnitude beyond what can be drawn by hand. They are obtained from the horizontal annotations by prompting SAM-family segmentation models [56, 74] with each box and fitting a minimum-area rotated rectangle [41] to the resulting mask [33]; the procedure and its validation against a human-verified oriented reference are given in Appendix § 14. Inside the benchmark the conversion runs over the vehicle annotations of the scenes and cases under test, the only use on which any number in this paper depends. Applied to the whole of Songdo Vision [40] at the selected configuration it yields 274 190 OBBs over 5419 frames, one per HBB and carrying its class and identifier, released as Songdo Vision OBB [37] with the human-verified reference used to select and validate the conversion: 4245 OBBs over 50 frames from six recordings (Appendix E.1). None of those 50 frames appears in Songdo Vision, as checked against all 5419 by checksum and by decoded pixels, but they come from the same six flights. That is a weaker guarantee than the site-level split of § 4.7, and the appropriate one here: the conversion is selected by IoU against a human-verified reference, a quantity that involves neither the pipelines nor their metrics, so scene overlap cannot tune it toward a favorable stabilization result; it can only flatter the reported conversion fidelity, which Appendix § 14 therefore states as a tuned-set estimate.

Fallback boxes. Where the segmentation finds no usable mask inside a box, the source HBB is emitted unchanged, which would dilute any difference between the two mask geometries. At the selected configuration that share is 0 wherever it has been measured: among the 4245 boxes of the reference set, the 274 190 of the released conversion, and read from the per-object confidence written beside each label file (exactly 0 marks a fallback), the 2311 boxes of Part I’s 29 scenes, the 60 213 of Part II’s nine cases and the 44 397 of its five tuning cases (136 frames each).

Part I: static benchmark

Protocol

Figure 2 shows the protocol. Known homographies (translation, rotation about the frame center, uniform scale, and a small projective term) and a chain of photometric degradations (brightness, saturation, blur, haze, sensor noise, and JPEG compression) are applied to static aerial frames, and the pipeline registers the degraded, warped frame back onto the original, so the ground truth is exact. Trials are paired: the perturbation of a (scene, trial) does not depend on the pipeline, so every pipeline faces the identical sequence of perturbations, and every sampled parameter is recorded with its trial. The ranges (Table A.1) are those of the published campaign [38]: translation up to 10 % of each frame dimension, rotation up to 15°, scale within 5 %, and a small projective term. This is wider than the deployed hover regime of § 3, so that operating limits are measured; a second configuration at deployment scale (rotation within 2°, translation within 2 %, scale within 1 %, no projective term, the same photometric chain) is run for the detector axis, and rotation is swept separately beyond the main range.

Protocol for the static benchmark

Figure 2. Part I. A scene frame is warped by a known geometric and photometric perturbation, and the pipeline registers the warped copy back onto the original. The perturbation is the ground truth, so no separate reference estimate is involved. The object annotations enter twice and for different purposes: as the feature mask handed to the pipeline, and as the object boxes whose object-level errors are scored.

The masks for the warped copy are the scene’s boxes carried through the known homography, since the warped frame has no annotation of its own. An OBB stays a quadrilateral; an HBB for the warped frame is the axis-aligned envelope of the carried OBB, which is what a detector would draw around the rotated object, rather than of the carried HBB, which would inflate an envelope a second time by an amount growing with the rotation.

Figure 3 shows one trial. Vehicles are among the most textured content of a road scene, and without masking a large share of a pipeline’s inliers lies on them. In Part I that is harmless, because the vehicles move with the ground; under masking, a detector that applies the mask before its keypoint cap (§ 3.2) spends that share of its budget on the ground instead. In Part II these are the correspondences a moving group can contribute.

Evaluation scene and perturbed copy with OBB outlines, inlier matches, and masked areas

Figure 3. One trial of the detector campaign, registered again with the campaign’s stored parameters and seed. (a) The evaluation scene with the most objects, every OBB outlined. (b) Its perturbed copy in trial 0, with the sampled parameters above it and a random sample of unmasked ORB’s inlier matches drawn between the two frames. (c, d) The dashed crop of (a) with every ORB inlier unmasked and under OBB masks, the masked area (each OBB scaled about its center by 1 + mm, Table A.2) shaded; orange marks an inlier on an annotated object, blue one elsewhere. Nothing in the scene moves, so the inliers on vehicles are correct here; under masking ORB spends that share of its feature budget on the ground.

The campaigns run in a fixed order on the two splits of § 4.7. On the tuning scenes: the compute-budget sweep, under masking disabled and OBB masking, which fixes the matched operating point; then the joint sweep of the ratio-test and inlier thresholds at that point. On the evaluation scenes, at the selected values: the detector axis under the three masking modes, the same axis at the untuned starting thresholds without masking, which measures the effect of tuning on data it was not selected on, the detector axis again at deployment-scale perturbation, the estimator axis at two fixed detectors, and the rotation sweep. Trial counts are 30 per scene for the tuning-side sweeps, 100 for the evaluation campaigns and 50 for the rotation sweep.

Scene set and selection rule. Part I runs on 29.4K frames from the hovering sessions of a multi-drone experiment over the Songdo International Business District, South Korea: the static scene set of Section 3.5.2 of Fonod et al. [38], byte-identical to the frames that campaign ran on, whose intersections are shown in its Figure 1. The annotation is regenerated rather than inherited, and the frames are published for the first time with the accompanying data record [36]. All 29 come from one session and are selected by a positional rule: the first frame of the first hover of every drone-intersection pair flown in it. Each drone alternates between two or three intersections during a session, and eight of the 20 intersections are covered in turn by more than one drone, so the 10 drones record them from 29 viewpoints. The set is a census of the session’s viewpoints rather than a sample chosen for content: no frame was kept for being busy, quiet, hard or easy, none was removed after the fact, and the rule reproduces the set from the file names alone. Table 3 describes the evaluation frames: object count and covered area each vary by a factor of about four, so the masking cost is measured over a range of densities, and the gap between the two area columns is how much more of the frame an HBB mask suppresses than an oriented one.

StatisticObjectsOBB area [%]HBB area [%]
minimum361.631.83
q1542.352.78
median783.323.56
q3924.474.83
maximum1526.606.96

Table 3. Object content of the 20 scenes the campaign ran on, as quantiles over scenes. Area is the fraction of the frame covered by the union of the boxes, for the oriented boxes and for the horizontal boxes they were converted from, before any mask margin; the difference between the two area columns is the additional area a horizontal mask suppresses.

Results

Part I asks which settings of a registration pipeline matter when nothing in the scene moves. The results follow the settings in the order of their effect: the compute budget, the ratio-test and inlier thresholds, and the robust estimator; then the ranking they produce at the operating point and what masking costs. The estimator, deployment and rotation sweeps are given in full in Appendix B.

The compute budget sets the accuracy. The budget sweep crossed four downsampling ratios (0.18, 0.30, 0.50, 1.0) with three feature budgets (1000, 2000, 8000) on the tuning scenes, at the untuned starting thresholds, since it precedes the tuning (Figure 4), and it has the largest effect of the settings in this part. No budget met the criterion of § 4.7, so every pipeline runs at the grid’s ceiling, full resolution with 8000 features, and the pipelines fall into two groups below it. ORB, XFeat, KeyNet with LightGlue, BRISK and AKAZE are still gaining at the ceiling: from half to full resolution at 8000 features their unmasked mAA rises by 0.035, 0.048, 0.023, 0.018 and 0.017, so a lower operating point would rank them below their capacity. RootSIFT, SIFT, RoootSIFT, DISK and DeDoDe with LightGlue have saturated by half resolution and move by less than the equivalence margin beyond it. LoFTR, which has no feature budget, cannot reach the operating point: at full resolution it exhausts the 24 GB of the benchmark GPU (an NVIDIA GeForce RTX 4090) and is reported as N/A throughout Part I; below that it scores an mAA of 0.926 at ratios of 0.50 and 0.30 and 0.879 at its shipped ratio of 0.18. OBB masking changes mAA by at most 0.065 at any grid point. The largest gain is ORB’s gain at a ratio of 0.18 with 1000 features, which shrinks to 0.017 at 2000 features and 0.004 at 8000; the largest loss is XFeat’s, 0.034 at full resolution with 1000 features. Since nothing in these scenes moves, any such gain is a matter of which features the budget keeps, a point § 7.5 returns to. At Part II’s real-time point, six of the ten pipelines (ORB, AKAZE, XFeat, SIFT, RootSIFT and DISK with LightGlue) estimate a 4K frame’s transform within one frame interval (33.4 ms, median over the tuning scenes) on the benchmark machine, with or without masking. The time covers matching, feature extraction under the mask, matching and the robust fit, not decoding, detection, box conversion or warping.

Figure 4. Accuracy against compute on the tuning scenes, at the untuned starting thresholds. Each panel is one pipeline; a line joins the downsampling ratios 0.18, 0.30, 0.50 and 1.0 at one feature budget (color and marker), unmasked (solid) or under OBB masks (dashed), so a split between the two lines is a masking effect. A circle rings Part I’s operating point (1.0, 8000 features), a square Part II’s offline one (0.50, 2000 features) and a diamond its real-time one, the cheapest point of the grid (0.18, 1000 features). Time is the median registration time per frame pair; mAA is drawn on a logarithmic scale of 1 − mAA, so the last hundredths before saturation stay visible. LoFTR has no point at full resolution, which exceeds the benchmark GPU’s memory.

The matcher thresholds have little effect at the operating point. The ratio test and the inlier threshold are the two parameters that earlier benchmarks found decisive [27, 50], and they are swept jointly for every pipeline (§ 4.7), from untuned starting values: the ratio of 0.9 that the stabilization library ships and an inlier threshold of 2 to 3 px per detector, listed with the selected values in Table B.1. At the operating point the surfaces are flat. All 80 grid points of SIFT, RootSIFT, ORB, BRISK and AKAZE lie within the equivalence margin of the best, against 57 of XFeat’s; LightGlue applies no ratio test, so its three pipelines are swept over the threshold alone, and all 8 of their points are tied. Tuning raises mAA over the starting values by 0.0062 for XFeat and by at most 0.0011 for every other pipeline. The incumbents are the values each preset carries, and every pipeline keeps them, since no grid point beats them by more than the margin; the largest gain on offer is 0.0002, for AKAZE at a ratio of 0.8. At the kept ratio the inlier threshold moves mAA by at most 0.0002 for every pipeline, so the data do not identify it. The estimator is the reason (Fig. B.1): MAGSAC++, the operating estimator, marginalizes over the noise scale instead of committing to one threshold. Below a ratio of 0.70 mAA stays flat for SIFT, RootSIFT, AKAZE and BRISK and falls for ORB and XFeat, so the kept ratios lie on the plateau rather than at an edge of it.

The selection carries over to the evaluation scenes. Unmasked, every pipeline’s evaluation-set mAA (Table 4) differs from its tuning-set mAA by at most 0.0025, the largest drop being KeyNet with LightGlue’s, about a quarter of the equivalence margin, and the order of the pipelines is unchanged except that SIFT and RootSIFT, 0.0001 apart on the tuning scenes, swap places. Run at the starting values on the evaluation scenes (file detectors_default_static.csv), the pipelines score lower by 0.0034 for XFeat and differ by at most 0.0011 for the other eight, and the only change of rank is that SIFT and RootSIFT, 0.0002 apart, swap places. At this operating point, then, untuned values do not produce the reordering Efe et al. [27] report; the protocol tunes every pipeline regardless, since whether the starting values are adequate cannot be known before the sweep is run.

PipelineMaskingmAACTE50_{50} [px]CTE90_{90} [px]IoUOCE [px]Time [s]
AKAZEHBB0.98450.0700.1290.99830.0350.529
AKAZEOBB0.98450.0710.1280.99830.0340.528
AKAZEnone0.98390.0750.1300.99840.0340.530
RootSIFTnone0.98300.0820.1250.99670.0690.389
RootSIFTOBB0.98280.0830.1260.99670.0690.370
SIFTnone0.98280.0840.1250.99670.0690.308
RootSIFTHBB0.98270.0830.1270.99670.0690.362
SIFTOBB0.98260.0830.1280.99670.0690.318
SIFTHBB0.98250.0840.1290.99670.0690.313
BRISKOBB0.98110.0870.1450.99780.0450.473
BRISKHBB0.98110.0880.1460.99780.0460.468
BRISKnone0.97990.0940.1530.99780.0460.500
DISK+LightGluenone0.97200.1280.2250.99490.1040.357
DISK+LightGlueOBB0.97150.1310.2320.99490.1050.356
DISK+LightGlueHBB0.97150.1300.2320.99490.1050.355
ORBnone0.97030.1410.2170.99560.0910.104
ORBOBB0.96960.1430.2270.99550.0940.118
ORBHBB0.96950.1430.2270.99550.0940.117
DeDoDe+LightGluenone0.96260.1780.2940.99270.1540.842
DeDoDe+LightGlueOBB0.96220.1790.2960.99260.1560.839
DeDoDe+LightGlueHBB0.96210.1790.2980.99260.1560.835
XFeatnone0.94580.2540.4140.99070.2060.112
XFeatOBB0.94340.2660.4250.99040.2110.101
XFeatHBB0.94340.2700.4260.99040.2100.099
KeyNet+LightGlueOBB0.91930.3940.6280.98250.3511.304
KeyNet+LightGluenone0.91890.3960.6320.98210.3581.315
KeyNet+LightGlueHBB0.91860.3980.6380.98240.3531.300
LoFTRallN/AN/AN/AN/AN/AN/A

Table 4. Part I (static benchmark): accuracy under known homographies and photometric degradation. mAA integrates HEAε_{\varepsilon}, the share of trials whose corner transfer error (CTE) stays within ε∈[0,5]\varepsilon\in[0, 5] px, a failed estimate counting as a miss; IoU and OCE (object center error) compare each object’s box with its round trip through the known homography and the estimate. Bold marks the best value of each column and every value not distinguishable from it: a paired difference to the best, over the 13 sites, whose 95% cluster-bootstrap interval includes zero or whose estimate is below the equivalence margin (0.01 for mAA). HEA1_1, HEA3_3 and HEA5_5 are at least 0.999 in every row. No configuration failed on any trial. N/A marks configurations that could not be run on the benchmark machine: memory was exhausted before any measurement was taken, so the run was aborted rather than averaged over the subset of the data that happened to fit.

The estimator matters through its threshold. The estimator axis is swept for ORB and SIFT, one binary and one floating-point descriptor. With the detector fixed, the estimator matters less than the threshold it runs at (Fig. B.1): with SIFT every estimator but RHO reaches an mAA of 0.983 to 0.984 at its best threshold, and MAGSAC++ and LMEDS move by less than 0.0001 over the whole range of θ\theta for both detectors, since neither commits to one threshold. The inlier ratio records the threshold rather than the estimate: for ORB it rises from about 0.17 to 0.81 between θ=0.5\theta= 0.5 px and 4 px whether mAA rises, falls or stays constant.

The ranking at the operating point. Table 4 reports the detector axis on the 20 evaluation scenes at the selected values. AKAZE, RootSIFT, SIFT and BRISK form the leading tie set under all three masking strategies: AKAZE scores highest, at 0.9839 unmasked and 0.9845 under either mask, and the other three lie within the equivalence margin of it (file detectors_comparisons.csv). DISK with LightGlue and ORB follow at 0.972 and 0.970 unmasked, 0.012 and 0.014 behind with intervals clear of zero, and DeDoDe with LightGlue, XFeat and KeyNet with LightGlue trail by 0.022 to 0.066. In pixels the spread at the top is small: with every trial inside the integration interval, mAA over [0, 5] px is one minus the mean CTE divided by 5 px, so the margin of 0.01 corresponds to 0.05 px of mean error, and the unmasked median error of the tie set lies between 0.075 px and 0.094 px. No pipeline that ran fails on any of the 2000 trials or leaves the integration interval under any masking strategy. Speed varies far more than accuracy. The unmasked median registration time spans a factor of about thirteen: ORB and XFeat register a frame in 0.10 s to 0.11 s, SIFT, DISK with LightGlue and RootSIFT in 0.31 s to 0.39 s, BRISK and AKAZE in 0.50 s to 0.53 s, and DeDoDe and KeyNet with LightGlue in 0.84 s to 1.3 s. SIFT reaches the tie set in less than three fifths of AKAZE’s time, and ORB reaches a median error of 0.141 px, against 0.075 px for AKAZE, in a fifth of its time.

Deployment regime and rotation. At deployment-scale perturbation every pipeline scores higher, the leading tie set grows from four pipelines to six and no pipeline moves by more than two places. The upright learned models collapse beyond 30° of in-plane rotation (Fig. B.2), well outside the 15° of the main configuration. Both sweeps are in Appendix B.

Cost of masking on static scenes. The objects in a static scene move with the ground, so masking them removes correct correspondence or, for a detector that refills its budget, exchanges them for others (Figure 3), and a small cost is expected. This is the baseline against which the dynamic results of § 7.5 are attributed to object motion rather than to masking as such. OBB masking costs every pipeline less than the equivalence margin. The largest loss in mAA is 0.0024, for XFeat; for the remaining eight pipelines the change is below 0.0013 in magnitude, and AKAZE, BRISK and KeyNet with LightGlue gain up to 0.0012. HBB masking costs as little: 0.0024 for XFeat and at most 0.0008 for the other eight pipelines, two of which gain. It scores below OBB masking for every pipeline but AKAZE, as the larger mask predicts (a median of 3.56% of the frame against 3.32%, Table 3), but by at most 0.0007, so at these object densities the extra area an HBB removes costs almost nothing, and a difference between the two mask shapes in the dynamic benchmark is not explained by the area they remove alone.

Part II: dynamic benchmark

Protocol

Figure 5 shows the protocol and Table A.3 its constants. A case is one hover, the recording of one drone over one intersection cut to its stable part, which lasts two to four minutes [38], and every sampled frame of it is registered directly onto one reference frame, as the deployed pipeline does. The reference is a frame the case selection picks inside the hover rather than its first: traffic at an intersection comes in phases, and whether a group of vehicles moving together is in view at the reference decides whether capture can occur at all.

Protocol for the dynamic benchmark

Figure 5. Part II. Every candidate registers the same frame pairs under each masking mode in turn, and the reference estimator registers them once, under OBB masks. The reference estimator supplies pseudo-ground truth and its own error is measured against GCPs; that residual is the equivalence margin of the reference-relative pixel metrics. On the audited frames a homography fitted to the same points is an independent reference of its own, and the identity transform is scored on every pair as the unstabilized baseline. The object annotations enter as the feature mask and, placed by the reference estimator, as the evaluation region and as the object motion that the capture fraction is read against.

Temporal offsets run up to 3600 frames (2 min at 29.97 frames per second) in twelve bins one octave wide, 1, 2–3, 4–7, 8–15, and so on to 1024–2047 and 2048–3600 frames, with 15 pairs drawn in each bin that holds that many and every offset of the four shortest, 135 pairs per case, each weighing the same. The spacing follows the mechanism. Capture needs objects present in both frames that moved together, and both conditions decay fast: in the Songdo Traffic trajectories [39], over all 20 intersections and 4 days, about 90 % of the vehicles in view at a given instant are still in view 4 s later, 64 % after 16 s, 45 % after 32 s, 22 % after 64 s and 5 % after 120 s (lower bounds, since the ends of a hover cut some stays short), and the members of a moving group drift apart within seconds. Evenly spaced offsets would spend most pairs on frames that share no moving group with the reference and average the effect away; the octaves resolve the shortest offsets, where a group is still shared and still moves as one, and the results are read per bin.

Every pipeline of Part I is a candidate, so that a ranking that changes between static and dynamic footage can be observed rather than assumed away, and every candidate runs under all three masking modes on every pair, so masking is a within-pair factor. SIFT, the reference estimator’s own detector, is among them, which makes the one pairing that is circular by construction § 4.8 explicit. The candidates run in two regimes, both points of Part I’s budget grid (Figure 4). The offline regime, for registering a recorded video after the flight, is half resolution with 2000 features, the operating point of the campaign this work extends [38]. The real-time regime is the grid’s cheapest point, a downsampling ratio of 0.18 with 1000 features, at which six of the ten pipelines estimate a frame’s transform within one frame interval (33.4 ms) at the median on the benchmark machine. In both, the inlier threshold is given in pixels of the downsampled frame, so that it keeps its size relative to keypoint localization. The regimes share the cases, the pairs, the reference estimator and the GCPs; the mask-margin sweep and the repeated runs are made in the offline regime only. Two further rows are scored on every pair without any pipeline: the identity, which is the unstabilized frame and the transform a failed pipeline delivers, and the reference estimator itself.

Thresholds are tuned again in each regime. The ratio-test and inlier thresholds are not carried over from Part I, where they were selected under synthetic warps of static scenes: moving objects change which correspondences are outliers, and the inlier threshold in particular decides whether matches on a moving group join the consensus. They are tuned in each regime by Part I’s procedure and selection rule, with Part I’s values as the incumbents (a joint grid widened to 14 ratios from 0.4 to 0.99 and 13 inlier thresholds from 0.1 to 16 px, selected on mAA, masking disabled), scored against the reference estimator on five tuning cases that the survey selects, one per cell and one control, from the intersections of Part I’s tuning scenes, so that no tuning case shares an intersection with an evaluation case. Offline, SIFT, BRISK and AKAZE move to the strictest corner of the grid, r=0.40r = 0.40 and θ=0.1\theta= 0.1 px, and XFeat to r=0.45r = 0.45 and θ=8\theta= 8 px, gaining 0.0102 to 0.0156 mAA on the tuning cases; in real time ORB, AKAZE, XFeat, DISK+LightGlue and DeDoDe+LightGlue move, gaining 0.0100 to 0.1249; the others keep Part I’s values (Tables C.1 and C.2). The selections at an end of the grid are not followed past it: the offline ones agree with the reference estimator to a median of 0.08 px to 0.20 px, and the margin of 0.01 mAA corresponds to about 0.05 px of error, both well below the audited floor (§ 4.8), so a stricter setting could only gain agreement with the reference that the audit cannot resolve. The same holds for AKAZE’s θ=0.1\theta= 0.1 px in real time (0.45 px); ORB’s real-time r=0.99r = 0.99 lies next to r=1r = 1, where the ratio test no longer rejects anything. The thresholds selected without masking are then used under all three masking modes, so a measured masking advantage is not inflated by the tuning.

Cases are selected from the video alone. The hypothesis predicts when capture is severe: where more of the foreground moves, and moves coherently. Testing that needs footage spanning those conditions, so the nine cases are selected by design, on the amount and the coherence of the moving foreground as measured from the video. The candidates come from the hovers of one recording day at the evaluation sites of Part I’s split; the next day in a fixed order is added only if a cell cannot be filled. Every hover is read at points 30 s apart, clear of its first and last seconds and with the full 3600 frames after them, and each point is a candidate reference frame; a hover supplies at most one case. Each candidate is read the way the benchmark read it: as pairs of the candidate reference with one later frame in each of the five shortest offset bins (1, 2, 5, 11 and 22 frames, the bins’ geometric centers), since every benchmark pair shares the reference and a moving group moves as one only briefly. In each pair the platform’s motion is removed by an affine enhanced correlation coefficient fit to the background (Evangelidis and Psarakis, 2008), with each frame’s objects filled with the background mean, and every object annotated in the reference frame is sought among the later frame’s detections by appearance: its patch, resampled along its OBB, is compared by ZNCC, and only a mutual best match between boxes of similar size is accepted. A reference object is then found where it was, found displaced (moved by more than a tenth of the median object width), or not found, and the largest set of displaced objects moving together is the largest structure a single wrong homography could fit, both as defined for the capture measure (§ 4.5). Averaged over the offsets, the displaced share of the frame is how many votes the moving foreground can cast, and the share of it moving as one structure, the coherence, is how many it can cast for one model; opposing or crossing streams have low coherence however aligned their boxes are. Distances are measured in object widths, so the procedure does not depend on ground resolution, and nothing is tracked: six annotated frames per candidate suffice. The design is a 2 × 2 split at the pool medians (Table 5): a candidate is high moving if its displaced share of the frame is at or above the median and low moving otherwise, and coherent or mixed by its coherence in the same way. Each cell holds two cases, and their margin from the split boundaries, with at most one hover per site in a cell; an unfilled cell is reported, never padded from its neighbor.

CaseCellReferenceMoving [%]CoherenceStationarityObjects
2022-10-06_D10_PM5_U1high moving / coherent11701.940.800.2567
2022-10-06_D3_PM3_F2high moving / coherent2591.430.720.1740
2022-10-06_D9_AM3_S2high moving / mixed3001.890.220.4876
2022-10-06_D1_PM1_B2high moving / mixed2401.480.220.5070
2022-10-06_D7_PM1_P1low moving / coherent3000.170.740.9044
2022-10-06_D3_AM1_M1low moving / coherent2370.180.590.7918
2022-10-06_D7_PM5_P2low moving / mixed12000.240.230.8962
2022-10-06_D8_PM3_M1low moving / mixed11620.260.250.8438
2022-10-06_D9_AM1_R1stationary control21000.00–1.0034

Table 5. Part II: the cases, selected from 831 candidate reference frames in 370 hovers of 2022-10-06. Candidates are points every 900 frames inside each hover, clear of its first and last seconds, and a case’s reference frame is its selected point; a hover supplies at most one case. Each candidate is read on pairs of its reference frame with the frames 1, 2, 5, 11 and 22 frames later. Moving: the share of the frame covered by reference-frame objects displaced by more than 0.1 median object widths; Coherence: the share of that foreground moving with one of them (within 0.1 widths or 25% of its displacement, whichever is larger); Stationarity: the share of the found foreground that stayed in place; each averaged over the offsets. Cells split at the pool medians (moving 0.67%, coherence 0.42); coherence is undefined below 4 displaced objects, which left 72 candidates unclassified, and the control comes from the top decile of stationarity. Reference: the reference frame’s index in its recording; Objects: its detections. Every cell was filled.

The selection reads video alone, which keeps it applicable to footage without trajectories, but the trajectories published for the same recordings (Fonod et al., 2025b) allow its descriptors to be checked. Applying the survey’s definitions to them, in ground coordinates where the platform’s motion is absent by construction, gives an independent value of every descriptor for every candidate (Table C.3): the static, moving and coherently moving foreground correlate with their trajectory values at r=0.97r = 0.97, coherence at r=0.90r = 0.90 and stationarity at r=0.93r = 0.93; the two place 81% of the candidates in the same cell and 97% of the quarter farthest from the split boundaries, where the selection draws, and the nine selected cases all fall in the cells the trajectories assign.

The stationary control. A queue at a red light is much foreground with little motion. If masking helps there as much as in a flowing case, the effect comes from removing texture rather than moving objects, and the proposed mechanism does not hold. The design therefore includes a stationary control case, a candidate whose reference foreground mostly stays where it was (the unremoved share of the found foreground in the pool’s top decile, with at least the pool’s median amount of it). Because one case gives one number, the control is also defined at the level of the frame pair: the occupancy persistence between the reference frame’s objects and the current frame’s objects placed in reference coordinates (the intersection over union of the two footprints) is recorded with every measurement, so moving and stationary foreground can be compared within one case, under the same background, which is the comparison the texture confound needs.

Table 5 lists the nine cases with the descriptors they were placed on, Figure 6 shows each at its reference frame, and file offline_masking_scenes.csv characterizes their frames in the terms of Table 3.

Figure 6. The nine cases of Table 5 at their reference frames, in cell order. Every object annotated in the reference frame is outlined by its OBB and colored by what it did 22 frames later, the longest offset the survey reads: displaced with the largest group that moved together, displaced otherwise, found where it was, or not found. Objects are found again as the survey finds them, with the reference estimator’s transform as the platform motion, as for the capture measure (§ 4.5); arrows are displacements at true scale, and each panel counts the objects found, displaced and moving together; the low-moving cases show few displaced objects by definition. Each panel is one pair, whereas the survey’s descriptors average five offsets and remove the platform motion by an affine fit to the background.

Reference estimator and its audited error

A reference estimator is built from the leading pipelines of Part I under OBB masking (Table 4): AKAZE, RootSIFT, SIFT and BRISK cannot be told apart on mAA, and SIFT, the most widely used of the four, is taken. It runs at full resolution with a feature budget of 20 000 keypoints, a larger mask margin and a stricter estimator (Table A.2), and its ratio-test and inlier thresholds are tuned as every Part I pipeline is, on the static tuning scenes against their known homographies, at these settings. The surface is flat, all 70 grid points within the equivalence margin of the best, so SIFT’s values, r=0.70r = 0.70 and θ=0.5\theta= 0.5 px, are kept, at an mAA of 0.9856 (Table C.4). Its homographies H~k→ref\tilde{\mathbf{H}}^{k\to\mathrm{ref}} are cached once, so every candidate is scored against identical geometry. It runs masked because that is the configuration expected to be most accurate; § 4.8 states the consequence, which is why the masking result is read primarily off the GCPs and the reference-free measures. Choosing the reference on Part I’s ranking assumes what the paper questions elsewhere, that static accuracy transfers to dynamic scenes; it is admissible only because the reference’s accuracy on the dynamic footage is then measured against independent control points rather than inferred.

Ground-plane GCPs are annotated in the reference frame and in eight sampled frames per case, two in each of four offset bins (1–3, 4–31, 32–511 and 512–3600 frames), the first at the offsets where a moving group is still shared with the reference, and the residual of H~k→ref\tilde{\mathbf{H}}^{k\to\mathrm{ref}} against them is reported per offset bin and pooled, with the GCP repeatability beside it.

Over 1064 points on the 72 audited frames of the nine cases, the reference estimator’s residual has a median of 0.06 px and a 95th percentile of 0.38 px (2.1 cm on the ground), ranging over the cases from 0.12 to 0.65 px (Table 6 and Figure 7).

CaseCellFramesPointsRes.₅₀ [px]Res.₉₅ [px]Res.₉₅ 1–3Res.₉₅ 4–31Res.₉₅ 32–511Res.₉₅ 512–3600Rep.₉₅ [px]<0.5 px
2022-10-06_D1_PM1_B2high moving / mixed81200.080.290.130.170.240.46–67%
2022-10-06_D10_PM5_U1high moving / coherent81190.050.650.620.550.470.730.9362%
2022-10-06_D3_AM1_M1low moving / coherent81190.050.380.070.110.170.81–33%
2022-10-06_D3_PM3_F2high moving / coherent81190.050.160.070.340.120.18–100%
2022-10-06_D7_PM1_P1low moving / coherent81180.110.240.190.150.230.34–50%
2022-10-06_D7_PM5_P2low moving / mixed81170.030.120.040.070.090.28–100%
2022-10-06_D8_PM3_M1low moving / mixed81160.050.400.130.260.290.81–33%
2022-10-06_D9_AM1_R1stationary control81190.050.220.170.340.160.231.0071%
2022-10-06_D9_AM3_S2high moving / mixed81170.120.450.390.250.460.59–80%
all cases7210640.060.380.280.260.270.531.0061%

Table 6. Part II: the reference estimator against ground control points, per case. Frames: the audited frames besides the reference frame; Points: the ground control points matched in them; Res.₅₀ and Res.₉₅: median and 95th percentile of the distance between a point’s annotated position and its reference-frame point carried by the reference estimator, over the case and per offset bin (frames); Rep.₉₅: 95th percentile of the GCP repeatability, where a second session exists; < 0.5 px: the share of the points placed by hand (Table C.5) within half a pixel of the reference estimator’s prediction, near which the annotation panel opens under a hidden offset. The last row pools every case; the largest per-case Res.₉₅ is the equivalence margin of the pooled reference-relative comparisons.

Audit of the reference estimator against GCPs

Figure 7. Audit of the reference estimator against GCPs. (a) Reference frame of case 2022-10-06_D9_AM3_S2 (Table 6) with its GCPs, the static structure excluded from the reference-free measures (hatched), every object’s OBB as produced by hbb2obb (dashed), the reference estimator’s feature mask (solid: each OBB scaled about its center by 1+m1 + m, Table A.2) and the OBB dilated by τ\tau, which the reference-free measures exclude (shaded); the inset magnifies one object at the frame’s resolution. Each arrow is a point’s residual at the case’s longest audited offset, drawn 250 times its length. (b) The points with the smallest, median and largest of those residuals in 21×2121 \times21 px windows: the reference frame beside the audited frame resampled into reference coordinates by Hek→ref\mathbf{H}_{\mathrm{ek}\to\mathrm{ref}}, with the reference point (+) and the carried point (×), whose distance is the residual. (c) Every audited point of every case against its time offset, pooled over the cases at each audited offset (box: quartiles and median; whiskers: 5th and 95th percentiles), the four audit offset bins shaded alternately; the dashed line is the equivalence margin εfloor\varepsilon_{\mathrm{floor}}.

The largest of these is εfloor\varepsilon_{\mathrm{floor}}, the equivalence margin of every reference-relative pixel metric pooled over the cases, and each case’s own value starts its mAA[εfloor,5]\mathrm{mAA}[\varepsilon_{\mathrm{floor}}, 5]. Per offset bin the 95th percentile runs from 0.28 px at 1–3 frames to 0.53 px at 512–3600. The GCP repeatability, from the repeat sessions on two cases, is 1.00 px at the 95th percentile, and 61 % of the points placed by hand lie within half a pixel of the hidden prediction, the check of § 4.8 that the clicks did not follow the panel (Table C.5 counts how each point was placed). The residual can lie below the repeatability because each later frame’s point is placed by following the reference point’s neighborhood, so the error of the reference click moves the point and H^k→ref\hat{H}^{k\to\mathrm{ref}}’s prediction alike and drops out; the repeatability is the distance between two unaided clicks of one feature, noting that only the points placed by hand carry. The floor is far below the dilation τ=10\tau= 10 px, so the exclusions placed with the reference still cover the objects. On the audited frames every candidate is also scored against H^GCPk→ref\hat{H}^{k\to\mathrm{ref}}_{\mathrm{GCP}} of Eq. (21), which is the primary geometric evidence for the masking comparison (Figure 11 and Table 10 and Table 11).

Figure 11. Every candidate in the offline (top) and real-time (bottom) regime, pooled over the nine cases with equal weight per case. Left, mAA over [0, 5] px against the reference estimator, on a logarithmic scale of 1 − mAA, the identity dotted; right, the CTE against the homography fitted to the GCPs of the audited frames, the marker at the median and the bar ending at the 90th percentile, with the GCP repeatability dashed. A marker is filled when the configuration is in the column’s tie set under the analysis plan (§ 4.9) and open otherwise. Unmasked ORB’s real-time bar is set by one audited frame on which its estimate broke down (§ 7.3). Full rows in Table 8 to Table 11.

PipelineMaskingCTE50GCP\mathrm{CTE}^{\mathrm{GCP}}_{50} [px]CTE90GCP\mathrm{CTE}^{\mathrm{GCP}}_{90} [px]CTEmax⁡GCP\mathrm{CTE}^{\mathrm{GCP}}_{\max} [px]IoUGCP\mathrm{IoU}^{\mathrm{GCP}}OCEGCP\mathrm{OCE}^{\mathrm{GCP}} [px]
AKAZEHBB0.210.501.380.99530.10
SIFTHBB0.210.501.290.99540.10
AKAZEOBB0.220.501.380.99540.10
RootSIFTOBB0.220.501.270.99540.10
SIFTOBB0.220.501.290.99540.10
RootSIFTHBB0.220.491.270.99540.10
AKAZEnone0.230.531.440.99520.11
SIFTnone0.250.501.270.99520.11
RootSIFTnone0.260.541.440.99490.12
KeyNet+LightGlueHBB0.270.521.160.99380.14
BRISKOBB0.270.541.380.99440.12
BRISKHBB0.270.541.380.99430.12
DISK+LightGlueOBB0.280.581.380.99320.15
KeyNet+LightGlueOBB0.280.511.160.99380.14
DISK+LightGluenone0.290.561.420.99300.15
DISK+LightGlueHBB0.290.561.390.99330.14
LoFTRHBB0.300.541.450.99180.18
LoFTROBB0.300.541.450.99170.18
LoFTRnone0.310.541.460.99160.18
BRISKnone0.310.561.440.99390.13
KeyNet+LightGluenone0.310.551.310.99330.15
XFeatHBB0.370.671.220.99010.21
XFeatOBB0.370.681.220.99010.21
XFeatnone0.380.681.210.99010.21
DeDoDe+LightGlueOBB0.380.611.250.99040.21
DeDoDe+LightGluenone0.390.621.210.99000.22
DeDoDe+LightGlueHBB0.390.631.420.99020.21
ORBOBB0.430.841.620.98750.27
ORBHBB0.460.811.630.98750.27
ORBnone0.571.154.750.98620.31

Table 10. Part II (offline regime at downsample ratio 0.5 and 2000 features) candidates against the homography fitted to each audited frame’s ground control points, which involves no reference estimator: CTE is the corner transfer error, IoU and OCE compare the frame’s object boxes as placed by the candidate and by the fitted transform. CTE50_{50} and CTE90_{90} are per case and averaged over the cases; CTEmax⁡_{\max} is the largest over every audited frame. Every row is scored on 72 audited frames and 1064 ground control points. Bold marks the best value of each column and every value not distinguishable from it: a paired difference to the best, over the 9 cases, whose 95% cluster-bootstrap interval includes zero or whose estimate is below the equivalence margin (1.00 px, the GCP repeatability, for the pixel columns, none for IoU). With 9 cases the smallest attainable sign-flip pp is 0.0039.

Table 11: Part II (real-time regime at downsample ratio 0.18 and 1000 features) candidates against the homography fitted to each audited frame’s ground control points, which involves no reference estimator: CTE is the corner transfer error, IoU and OCE compare the frame’s object boxes as placed by the candidate and by the fitted transform. CTE50_{50} and CTE90_{90} are per case and averaged over the cases; CTEmax⁡_{\max} is the largest over every audited frame. Every row is scored on 72 audited frames and 1064 ground control points. Bold marks the best value of each column and every value not distinguishable from it: a paired difference to the best, over the 9 cases, whose 95% cluster-bootstrap interval includes zero or whose estimate is below the equivalence margin (1.00 px, the GCP repeatability, for the pixel columns, none for IoU). With 9 cases the smallest attainable sign-flip pp is 0.0039.

PipelineMaskingCTE50GCP^{\mathrm{GCP}}_{50} [px]CTE90GCP^{\mathrm{GCP}}_{90} [px]CTEmax⁡GCP^{\mathrm{GCP}}_{\max} [px]IoUGCP^{\mathrm{GCP}}OCEGCP^{\mathrm{GCP}} [px]
AKAZEHBB0.511.011.860.98740.27
AKAZEOBB0.511.001.860.98740.27
AKAZEnone0.541.011.870.98710.28
RootSIFTHBB0.540.921.900.98800.25
RootSIFTOBB0.540.931.900.98800.25
SIFTHBB0.550.931.750.98800.25
RootSIFTnone0.550.981.950.98780.26
SIFTOBB0.560.931.600.98800.25
SIFTnone0.570.951.560.98750.27
LoFTRHBB0.621.001.520.98140.40
LoFTROBB0.621.001.520.98130.40
LoFTRnone0.631.011.510.98100.41
KeyNet+LightGlueHBB0.711.311.940.98060.42
KeyNet+LightGlueOBB0.711.301.880.98050.41
KeyNet+LightGluenone0.761.241.980.98050.42
XFeatnone0.821.523.300.97490.54
XFeatHBB0.831.542.930.97550.53
XFeatOBB0.831.523.160.97530.53
DISK+LightGlueHBB0.831.612.990.98000.42
DISK+LightGlueOBB0.851.642.810.98000.42
DISK+LightGluenone0.871.612.800.97960.43
BRISKnone1.011.872.940.97690.51
BRISKHBB1.021.942.770.97730.49
BRISKOBB1.041.912.820.97730.49
DeDoDe+LightGlueOBB1.061.913.180.97430.55
DeDoDe+LightGlueHBB1.061.883.200.97430.55
DeDoDe+LightGluenone1.081.923.070.97420.56
ORBOBB1.172.072.960.97040.66
ORBHBB1.192.033.020.96900.69
ORBnone1.67152.574469.430.95168.59

Table 11.

Consensus capture

Capture is read four ways: directly, as a capture fraction of at least 0.1 on a pair with a moving group (§ 4.5); as an error of many pixels against the GCPs where an audited frame falls on such a pair; as error concentrated along the direction of motion; and as all three receding under masking.

Capture by pipeline and case. Without masking, ORB is captured on 28 of the 304 pairs that carry a moving group in the real-time regime (9.2%), in five cases, and on 6 of them offline (2.0%), in two (Figure 8 and Table 7; offline in Fig. C.1 and file offline_masking_capture.csv). In real time BRISK (6 pairs), SIFT and RootSIFT (2 each) are captured as well,

Figure 8. The capture fraction λ\lambda (Eq. (18)) of every real-time measurement on a pair with a moving group, against the offset: one panel per pipeline, its title counting the captured measurements, and a marker per masking mode. The dashed line is the capture threshold of 0.1 and the shaded band the survey’s offsets of 1 to 31 frames. The axis is linear within the threshold and logarithmic beyond it, so a partial pull, a complete capture (λ≈1\lambda\approx1) and a breakdown far from zero are all visible. The offline regime is in Fig. C.1, the captured share per offset bin in Fig. C.2.

all at offsets below 16 frames and misaligned by 0.9 to 1.6 px (CTE50_{50} of the captured), with few of their inliers on the objects found to have moved, a share that undercounts (§ 4.5); offline no other pipeline is. No masked configuration of any pipeline is captured in either regime: the largest capture fraction any masked estimate reaches is 0.05 in real time and 0.03 offline, below the identity’s 0.07, against 6.03 and 0.36 for unmasked ORB. A higher threshold reduces the count: at λ≥0.25\lambda\ge0.25 unmasked ORB keeps 19 captures in real time and 4 offline, at λ≥0.5\lambda\ge0.5 11 and none, and at λ≥1\lambda\ge1 4 and none (released files realtime_masking_capture.csv and offline_masking_capture.csv); it drops partial captures first, so that most of what remains at the top are the breakdowns described below.

Capture follows the survey’s moving-foreground axis. A group of four or more objects that moved together occurs only in the four high-moving cases and, on 13 pairs, in the stationary control, never in the four low-moving cases, and unmasked ORB is captured in real time in all five cases that have one (released files offline_masking_capture_by_case.csv and realtime_masking_capture_by_case.csv; a case is named here by the hover that ends its name in Table 5): S2 holds 22 of ORB’s real-time captures and 4 of its offline ones, U1 and F2 two each in real time, B2 and the control one each, and the control the other two offline. The control is still at the survey’s offsets; it holds a group only several seconds after the reference, once part of its standing foreground has started to move together, and its captures lie there. The coherence axis does not order them, since S2 lies in the mixed cell; the survey averages coherence over its offsets, while a capture needs one coherent group at one offset. By offset, 17 of the 28 real-time captures of ORB lie within the survey’s offsets of 1 to 31 frames, 3 of the 4 pairs at one frame among them, and the rest are spread up to the longest bin (Figs. C.2 and 8); offline, four of six lie within them. Within the survey’s offsets every capture is a pull toward the group: its error at the group lies within 17° of the group’s displacement, ORB’s within 11° (5° offline), up to a capture fraction of 0.96 (Figure 1; files realtime_masking_capture_populations.csv and offline_masking_capture_populations.csv), an estimate that registers the group back onto where it stood. Beyond them the real-time captures are of another kind. Apart from one in the control, under a pixel at the group, they are breakdowns: corner errors of 153 to 7085 px on at most 143 inliers, among them the four at λ≥1\lambda\ge1, up to 6.03. A breakdown is a failure of its own: unmasked real-time ORB’s corner error reaches 100 px on 42 of its 1215 pairs, 25 of them without a group, and no other configuration’s does in either regime (files realtime_masking_capture.csv and offline_masking_capture.csv).

Examples of captured estimates. Both examples are unmasked ORB on S2. In real time, at an offset of ten frames (Figure 1), the estimate follows the group almost completely, with a capture fraction of 0.96, and its corner error of 10.0 px against the reference is not confined to the group: the error arrows carry the group’s motion across the whole crop, so every position measured in the frame inherits it. Offline, at nine frames (Figure 9), the pull is partial, a capture fraction of 0.36, and the error 4.1 px, on 781 inliers, 12 % on objects that moved, against 0.8 px on 1279 inliers under masking. In both the masked estimate has more inliers, because ORB spends the share of its budget the mask frees on the ground (§ 3.2).

Offline ORB estimates with no masking and OBB masking, showing moved objects, inliers, and estimate errors

Figure 9. The offline counterpart of Figure 1: the captured unmasked measurement of the offline regime with the largest capture fraction within the survey’s offsets of 1 to 31 frames whose registration reproduces its stored estimate, beside the same pipeline under OBB masking, drawn as in Figure 1.

Error magnitude. No pipeline fails on any pair in either regime. Against the GCPs, every configuration’s median corner error lies between 0.21 and 0.57 px offline, below the GCP repeatability, and its 90th percentile at most 1.15 px, reached by unmasked ORB. In real time the medians lie between 0.51 and 1.67 px; every configuration but ORB stays within 1.94 px at the 90th percentile, ORB reaches 2.03 and 2.07 px with HBB and OBB masks, and 152.57 px without (Table 10 and Table 11). That value is set by one frame: the percentile is taken over each case’s eight audited frames and averaged over the cases, and on one audited frame of S2, a pair without a moving group, unmasked ORB’s estimate broke down by 4469.4 px (CTEmax⁡GCP\mathrm{CTE}_{\max}^{\mathrm{GCP}}). Where a captured estimate falls on an audited frame, the GCPs confirm the capture independently of the reference estimator: the four such estimates of unmasked ORB in real time lie a median of 4.3 px from them, where masked ORB’s median is 1.17 px, and the one offline 4.7 px, against 0.43 px masked (Audited and CTE50GCP\mathrm{CTE}_{50}^{\mathrm{GCP}} in Table 7 and file offline_masking_capture.csv). The captured SIFT, RootSIFT and BRISK estimates on audited frames lie 1.0 to 1.2 px from them, at the repeatability, which cannot resolve a pull of that size. Against the reference estimator, unmasked ORB’s 90th percentile is 163.44 px in real time (Table 9). For scale, the unstabilized frame lies 6.67 px from the reference at the median (Table 8).

Table 8: Part II (offline regime at downsample ratio 0.5 and 2000 features; dynamic benchmark, 9 cases): every candidate against the reference estimator. mAA, integrated over [0, 5] px, and CTE are reference-relative; the largest of the cases’ audited residuals, 0.65 px, is the equivalence margin of the reference-relative pixel columns, below which a difference is not resolvable, and mAAεfloor_{\varepsilon\mathrm{floor}} integrates from each case’s own residual. SSIM, ZNCC, MI and GOS are reference-free, over the static background; IoU and OCE compare the current frame’s boxes as placed by the candidate and by the reference estimator. Align (0.5 for isotropic error) and the inlier ratio are diagnostics, never bolded. Identity is the unstabilized frame, also what a failed pipeline delivers. The RootSIFT and SIFT rows share the reference estimator’s keypoint detector (SIFT, with OBB masking), so their reference-relative columns are the least independent of it. Bold marks the best value of each column and every value not distinguishable from it: a paired difference to the best, over the 9 cases, whose 95% cluster-bootstrap interval includes zero or whose estimate is below the equivalence margin (0.01 for mAA, 0.65 px for the reference-relative pixel columns). With 9 cases the smallest attainable sign-flip p is 0.0039. The reference-free columns are computed over a median 56.6% to 56.7% of the frame. No configuration failed on any pair. Every row pools 9 cases.

PipelineMaskingmAAmAAεfloor_{\varepsilon\mathrm{floor}}CTE50_{50} [px]CTE90_{90} [px]SSIMZNCCMI [bits]GOSIoUOCE [px]AlignInlier ratioTime [s]
Reference–1.00001.00000.000.000.95050.98903.2800.92861.00000.00––0.000
SIFTOBB0.98160.99960.080.160.95030.98903.2770.92820.99830.040.6180.6410.069
SIFTHBB0.98160.99970.080.160.95030.98903.2770.92820.99830.040.6160.6400.071
AKAZEOBB0.98070.99930.080.170.95030.98903.2780.92820.99810.040.6160.6810.100
AKAZEHBB0.98070.99930.080.180.95030.98903.2780.92820.99810.040.6170.6800.101
SIFTnone0.98050.99960.080.160.95030.98903.2760.92810.99820.040.6120.6220.079
RootSIFTHBB0.98010.99950.090.170.95030.98903.2780.92820.99820.040.6280.9140.085
RootSIFTOBB0.98000.99950.090.170.95030.98903.2780.92820.99820.040.6310.9140.087
AKAZEnone0.97680.99850.100.210.95030.98903.2770.92820.99790.050.6100.6900.097
RootSIFTnone0.97500.99820.100.210.95000.98903.2710.92770.99780.050.6200.8910.085
DISK+LightGlueHBB0.96180.99610.160.300.94870.98863.2460.92590.99570.090.5590.9890.071
DISK+LightGlueOBB0.96090.99560.160.330.94860.98853.2450.92570.99560.090.5600.9890.073
BRISKOBB0.96090.99600.190.300.94980.98893.2650.92750.99610.080.6060.1780.303
BRISKHBB0.96090.99600.180.300.94980.98893.2650.92750.99610.080.6050.1780.301
DISK+LightGluenone0.96070.99530.160.340.94860.98853.2440.92570.99550.100.5580.9830.077
LoFTRHBB0.95440.99500.220.330.94820.98843.2280.92510.99370.130.5670.9900.448
LoFTROBB0.95440.99500.220.330.94820.98843.2280.92510.99370.130.5670.9900.449
LoFTRnone0.95430.99500.220.330.94820.98843.2270.92510.99370.130.5680.9830.446
BRISKnone0.95270.99230.220.360.94940.98883.2560.92680.99540.100.5960.1700.314
KeyNet+LightGlueHBB0.95150.99280.230.360.94940.98883.2560.92680.99490.110.6340.9980.141
KeyNet+LightGlueOBB0.95120.99270.230.370.94940.98883.2570.92680.99490.110.6330.9980.143
KeyNet+LightGluenone0.94890.99100.240.380.94910.98873.2510.92630.99460.120.6290.9850.143
DeDoDe+LightGlueHBB0.94220.98540.250.500.94510.98753.1890.92060.99240.160.5650.8760.205
DeDoDe+LightGlueOBB0.94210.98540.250.490.94510.98753.1890.92070.99240.160.5660.8760.206
DeDoDe+LightGluenone0.94190.98500.250.500.94510.98763.1890.92060.99230.160.5630.8680.211
XFeatOBB0.92090.97170.370.620.94410.98743.1730.91880.98980.210.5500.9990.029
XFeatHBB0.92080.97150.360.630.94410.98743.1730.91870.98980.210.5490.9990.026
XFeatnone0.92060.97150.370.620.94410.98743.1720.91880.98980.220.5470.9990.030
ORBHBB0.90890.96040.390.790.93790.98513.1290.90920.98890.240.5810.7870.033
ORBOBB0.90820.95970.390.790.93760.98503.1280.90860.98890.240.5800.7870.035
ORBnone0.87830.93230.541.030.93630.98493.1070.90670.98780.270.5900.7100.031
Identity–0.29570.31456.6722.050.60530.79751.7940.31920.79956.010.478–0.000

Table 8.

PipelineMaskingmAAmAAεfloor_{\varepsilon\mathrm{floor}}CTE50_{50} [px]CTE90_{90} [px]SSIMZNCCMI [bits]GOSIoUOCE [px]AlignInlier ratioTime [s]
Reference–1.00001.00000.000.000.95050.98903.2800.92861.00000.00––0.000
RootSIFTnone0.88560.94090.540.890.93920.98613.1200.91080.98760.260.6200.7600.020
SIFTnone0.88530.94090.550.890.93910.98603.1190.91060.98750.260.6220.7530.020
RootSIFTOBB0.88470.93960.550.910.93880.98593.1160.91010.98730.270.6170.7630.022
RootSIFTHBB0.88460.93950.550.900.93870.98593.1160.91010.98730.270.6170.7630.021
SIFTHBB0.88420.93930.560.910.93880.98593.1160.91020.98720.270.6180.7580.021
SIFTOBB0.88410.93930.560.910.93890.98593.1160.91030.98730.270.6180.7570.022
AKAZEOBB0.87080.92560.631.020.93520.98453.0910.90390.98520.310.5730.1810.020
AKAZEHBB0.87060.92550.631.020.93510.98453.0910.90360.98520.310.5710.1810.019
AKAZEnone0.86730.92230.661.040.93480.98443.0890.90310.98510.310.5780.1820.018
LoFTRHBB0.85920.91550.691.060.92430.98162.9630.88790.97880.450.5840.9810.049
LoFTROBB0.85910.91540.691.060.92430.98162.9620.88770.97870.450.5840.9810.051
LoFTRnone0.85780.91420.691.070.92380.98152.9580.88700.97850.460.5860.9750.057
KeyNet+LightGluenone0.81100.86530.941.420.91800.97922.9290.87590.97630.490.5360.9860.056
KeyNet+LightGlueOBB0.80900.86280.941.430.91700.97882.9280.87380.97580.510.5370.9910.051
KeyNet+LightGlueHBB0.80850.86230.941.440.91660.97872.9260.87310.97570.510.5360.9910.052
DISK+LightGluenone0.79220.84430.991.730.91810.97982.9310.87720.97730.480.5640.9560.024
DISK+LightGlueOBB0.79080.84281.001.740.91790.97982.9320.87670.97740.480.5600.9600.025
DISK+LightGlueHBB0.79050.84231.001.730.91770.97982.9300.87650.97730.480.5600.9610.024
XFeatnone0.77800.82911.091.680.89780.97422.7970.84320.96950.660.5400.8230.011
XFeatOBB0.77700.82771.081.710.89910.97462.8040.84550.97000.650.5420.8210.012
XFeatHBB0.77680.82751.091.720.89880.97442.8030.84490.97000.650.5430.8210.011
BRISKHBB0.75060.80041.202.000.90900.97742.8730.86160.97510.540.5740.9080.083
BRISKOBB0.75040.80021.202.010.90920.97752.8740.86210.97510.540.5740.9080.085
BRISKnone0.74260.79281.212.040.90640.97682.8530.85730.97450.550.5730.9050.086
DeDoDe+LightGlueHBB0.72720.77531.312.220.88980.97252.7750.82950.96930.670.5270.9850.035
DeDoDe+LightGlueOBB0.72700.77521.302.250.88990.97242.7750.82970.96930.670.5280.9850.036
DeDoDe+LightGluenone0.72530.77341.312.250.88900.97232.7690.82810.96940.670.5220.9790.035
ORBHBB0.67870.72381.512.530.86140.96082.6320.77580.96210.830.5420.4490.012
ORBOBB0.67850.72351.532.570.86330.96172.6380.77980.96250.820.5420.4480.013
ORBnone0.58350.62092.42163.440.81750.93102.4600.70400.922631.730.5790.3520.010
Identity–0.29570.31456.6722.050.60530.79751.7940.31920.79956.010.478–0.000

Table 9. Part II (real-time regime at downsample ratio 0.18 and 1000 features; dynamic benchmark, 9 cases): every candidate against the reference estimator. mAA, integrated over [0, 5] px, and CTE are reference-relative; the largest of the cases’ audited residuals, 0.65 px, is the equivalence margin of the reference-relative pixel columns, below which a difference is not resolvable, and mAAεfloor_{\varepsilon\mathrm{floor}} integrates from each case’s own residual. SSIM, ZNCC, MI and GOS are reference-free, over the static background; IoU and OCE compare the current frame’s boxes as placed by the candidate and by the reference estimator. Align (0.5 for isotropic error) and the inlier ratio are diagnostics, never bolded. Identity is the unstabilized frame, also what a failed pipeline delivers. The RootSIFT and SIFT rows share the reference estimator’s keypoint detector (SIFT, with OBB masking), so their reference-relative columns are the least independent of it. Bold marks the best value of each column and every value not distinguishable from it: a paired difference to the best, over the 9 cases, whose 95% cluster-bootstrap interval includes zero or whose estimate is below the equivalence margin (0.01 for mAA, 0.65 px for the reference-relative pixel columns). With 9 cases the smallest attainable sign-flip pp is 0.0039. The reference-free columns are computed over a median 56.5% to 56.7% of the frame. No configuration failed on any pair. Every row pools 9 cases.

Direction of the error. The identity transform’s motion alignment is 0.478. In real time, unmasked ORB’s is 0.579 against 0.542 under OBB masking, and the planned contrast of A without masking against OBB masking is 0.035 for ORB, with a 95 % interval of 0.014 to 0.064 and a positive value in five of the six cases where the axis is defined; Holm’s adjustment over the ten pipelines leaves it short of the rule. For the average pipeline it is 0.005 in real time and −0.003 offline (Fig. C.3; files offline_masking_mechanism.csv and realtime_masking_mechanism.csv). A pools every pair, and capture touches a few percent of them, so it moves little; the capture fraction, which reads the group itself, is the sharper of the two instruments.

Inlier statistics do not reveal capture. Captured estimates are not exposed by their inliers. Pooled over offsets, unmasked ORB’s captured real-time estimates have a median of 290.5 inliers against 504.5 for its other pairs with a group, but inliers fall with offset, and ranked against ORB’s own estimates on the pairs of the same case and offset bin they sit at 0.58 in both count and ratio; BRISK’s rank 0.81 in count and offline ORB’s 0.93 in both count and ratio Table 7; file offline_masking_capture.csv. The sharper comparison is with the same case at the nearest offsets, two on either side, whose estimates are neither captured nor broken down. Against those, unmasked ORB’s pulls carry 1.08 times the inliers at 1.08 times the ratio in real time, and 1.03 and 1.04 times offline, at a median disagreement from the reference of 9.7 and 4.4 px (18.8 and 4.1 px over all captures) where unmasked ORB’s over all pairs is 2.42 and 0.54 px (Table 8 and Table 9): nothing in their inlier statistics sets them apart. The breakdowns carry half (0.51 and 0.50 times), so theirs do. A captured fit is, by construction, the fit the consensus preferred. Within each pipeline and case the inlier ratio does order estimates by accuracy on the whole: the median Spearman coefficient between ratio and disagreement is −0.38 without masking in real time and −0.14 offline, positive in 16 and 24 of the 90 pipeline and case groups (Figure 10). The pulls are the exception their ordering cannot reveal. Across pipelines the inlier ratio records the estimate and its threshold rather than accuracy, as in Part I’s estimator sweep (Fig. B.1), so choosing a pipeline, which is a comparison between pipelines, cannot rest on it even where it tracks accuracy within one.

The two regimes

Offline. SIFT, RootSIFT and AKAZE form the tie set on mAA under every masking mode, led by SIFT with OBB masks at 0.9816 and ending with unmasked RootSIFT at 0.9750; their median disagreements with the reference, 0.08 to 0.10 px, lie far below the audited floor, and their mAA[εfloor,5]mAA[\varepsilon_{\mathrm{floor}}, 5] is 0.998 or more. DISK with LightGlue and BRISK follow at about 0.96, and every pipeline’s median stays below 0.6 px (Table 8). The GCPs separate almost nothing: all thirty configurations are tied against them in corner and center error (Table 10). Three of Part I’s four leaders lead again; BRISK, the fourth, falls level with DISK with LightGlue.

Real time. The same three pipelines form the tie set, from unmasked RootSIFT at 0.8856 to unmasked AKAZE at 0.8673, at median registration times of 18 to 22 ms, inside one frame interval. LoFTR follows at 0.858 to 0.859 at 49 to 57 ms, then KeyNet and DISK with LightGlue, XFeat, BRISK and DeDoDe with LightGlue between 0.811 and 0.725; masked ORB scores 0.679, unmasked ORB 0.584, and the identity 0.296 (Table 9). Against the GCPs every configuration is tied at the median, and at the 90th percentile all but ORB’s (Table 11). SIFT and RootSIFT share the reference’s detector, which makes their reference-relative scores the least independent of it, so their standing is read against the GCPs as well.

Masking

The primary contrast. For the average pipeline, OBB masking improves every outcome in most cases: in real time ZNCC and GOS in eight of nine, the three outcomes against the GCPs in seven; offline the corner error against them in all nine and the others in five or six. No outcome meets the rule of Section 4.9 in either regime, and the primary contrast is not claimed. HBB masks against none read the same way, and OBB against HBB masks differ on no outcome in either regime (Table 12 and Table 13). Four of the nine cases never present a moving group, so an average over cases and pipelines dilutes an effect confined to a few pairs of a few cases and of one pipeline.

PipelineCTE50GCP^{\mathrm{GCP}}_{50} [px]ZNCC [10−310^{-3}]GOS [10−310^{-3}]IoUGCP^{\mathrm{GCP}} [10−310^{-3}]OCEGCP^{\mathrm{GCP}} [px]Claim
OBB against none (primary)
All pipelines+0.02 (9/9)+0.02 (5/9)+0.32 (5/9)+0.36 (6/9)+0.01 (6/9)
ORB+0.11 (9/9)+0.17 (5/9)+1.25 (5/9)+1.50 (7/9)+0.03 (7/9)–
SIFT+0.01 (6/9)+0.00 (6/9)+0.02 (8/9)+0.07 (3/9)+0.00 (6/9)–
RootSIFT+0.02 (7/9)+0.04 (8/9)+0.27 (9/9)+0.34 (6/9)+0.01 (6/9)–
BRISK+0.03 (6/9)+0.09 (9/9)+0.58 (8/9)+0.47 (8/9)+0.01 (8/9)–
AKAZE+0.01 (6/9)+0.01 (6/9)+0.04 (7/9)+0.22 (7/9)+0.01 (6/9)–
XFeat-0.00 (4/9)-0.01 (3/9)-0.02 (4/9)-0.04 (3/9)+0.00 (3/9)–
DISK+LightGlue+0.01 (5/9)-0.01 (4/9)-0.01 (4/9)+0.03 (5/9)+0.00 (5/9)–
DeDoDe+LightGlue+0.00 (5/9)-0.01 (5/9)+0.04 (5/9)+0.30 (4/9)+0.01 (4/9)–
KeyNet+LightGlue+0.02 (6/9)+0.08 (7/9)+0.52 (7/9)+0.55 (5/9)+0.01 (5/9)–
LoFTR+0.00 (6/9)+0.00 (5/9)+0.03 (6/9)+0.13 (5/9)+0.00 (4/9)–
HBB against none (secondary)
All pipelines+0.02 (8/9)+0.03 (5/9)+0.38 (5/9)+0.34 (6/9)+0.01 (6/9)–
ORB+0.10 (8/9)+0.11 (4/9)+1.74 (4/9)+0.55 (5/9)+0.02 (6/9)–
SIFT+0.01 (7/9)+0.00 (6/9)+0.02 (7/9)+0.05 (4/9)+0.00 (6/9)–
RootSIFT+0.02 (6/9)+0.04 (8/9)+0.27 (9/9)+0.34 (6/9)+0.01 (6/9)–
BRISK+0.03 (6/9)+0.09 (9/9)+0.58 (8/9)+0.45 (8/9)+0.01 (7/9)–
AKAZE+0.02 (8/9)+0.01 (6/9)+0.05 (7/9)+0.13 (6/9)+0.01 (7/9)–
XFeat-0.00 (3/9)-0.01 (4/9)-0.03 (4/9)-0.07 (3/9)+0.00 (3/9)–
DISK+LightGlue-0.00 (5/9)+0.01 (5/9)+0.13 (5/9)+0.23 (5/9)+0.00 (4/9)–
DeDoDe+LightGlue+0.00 (5/9)-0.01 (4/9)-0.04 (4/9)+0.32 (5/9)+0.01 (5/9)–
KeyNet+LightGlue+0.05 (6/9)+0.07 (7/9)+0.42 (7/9)+0.47 (6/9)+0.01 (7/9)–
LoFTR+0.00 (6/9)+0.00 (5/9)+0.04 (6/9)+0.15 (5/9)+0.00 (4/9)–
OBB against HBB (secondary)
All pipelines+0.00 (6/9)+0.01 (5/9)+0.02 (5/9)-0.02 (3/9)-0.00 (3/9)–
ORB+0.01 (6/9)+0.03 (5/9)-0.15 (5/9)-0.19 (4/9)-0.00 (4/9)–
SIFT-0.00 (5/9)-0.00 (4/9)-0.00 (4/9)-0.02 (2/9)-0.00 (2/9)–
RootSIFT+0.00 (7/9)+0.00 (5/9)+0.00 (7/9)+0.01 (6/9)-0.00 (4/9)–
BRISK-0.00 (4/9)-0.00 (4/9)-0.00 (5/9)+0.02 (6/9)+0.00 (5/9)–
AKAZE-0.00 (3/9)-0.00 (2/9)-0.00 (3/9)+0.03 (7/9)+0.00 (7/9)–
XFeat-0.00 (1/9)+0.00 (6/9)+0.03 (6/9)+0.00 (5/9)+0.00 (5/9)–
DISK+LightGlue+0.00 (5/9)-0.01 (4/9)-0.09 (4/9)-0.11 (3/9)-0.00 (4/9)–
DeDoDe+LightGlue+0.00 (5/9)+0.01 (5/9)+0.04 (5/9)+0.16 (7/9)+0.00 (6/9)–
KeyNet+LightGlue-0.00 (5/9)+0.01 (7/9)+0.06 (7/9)+0.04 (5/9)+0.00 (5/9)–
LoFTR-0.00 (3/9)+0.00 (6/9)+0.00 (5/9)-0.01 (3/9)-0.00 (3/9)–

Table 12. Part II (offline regime at downsample ratio 0.5 and 2000 features): the planned masking contrasts over 9 cases. Each cell is the Hodges–Lehmann estimate of the per-case paired benefit of the first mode over the second (positive favors the first; pixels for CTE and OCE) and the number of cases in which it is positive. CTE, IoU and OCE are against the ground control points on the audited frames, ZNCC and GOS over every pair. A contrast is read over all pipelines (Holm across the outcomes), then per pipeline (Holm across the pipelines). Bold: a benefit with a 95% cluster-bootstrap interval above zero, an estimate reaching the margin (the GCP repeatability, 1.00 px for CTE and OCE) and a Holm-adjusted p≤0.05p \le0.05; underlined: the same in favor of the second mode at the unadjusted pp. The smallest attainable pp is 0.0039. The primary contrast (OBB against none, all pipelines) is claimed when an outcome is bold and none is underlined. Intervals and pp-values: offline_masking_contrasts.csv.

Table 13: Part II (real-time regime at downsample ratio 0.18 and 1000 features): the planned masking contrasts over 9 cases. Each cell is the Hodges–Lehmann estimate of the per-case paired benefit of the first mode over the second (positive favors the first; pixels for CTE and OCE) and the number of cases in which it is positive. CTE, IoU and OCE are against the ground control points on the audited frames, ZNCC and GOS over every pair. A contrast is read over all pipelines (Holm across the outcomes), then per pipeline (Holm across the pipelines). Bold: a benefit with a 95% cluster-bootstrap interval above zero, an estimate reaching the margin (the GCP repeatability, 1.00 px for CTE and OCE) and a Holm-adjusted p≤0.05p \le0.05; underlined: the same in favor of the second mode at the unadjusted pp. The smallest attainable pp is 0.0039. The primary contrast (OBB against none, all pipelines) is claimed when an outcome is bold and none is underlined. Intervals and pp-values: realtime_masking_contrasts.csv.

PipelineCTE50GCP\mathrm{CTE}^{\mathrm{GCP}}_{50} [px]ZNCC [10−310^{-3}]GOS [10−310^{-3}]IoUGCP\mathrm{IoU}^{\mathrm{GCP}} [10−310^{-3}]OCEGCP\mathrm{OCE}^{\mathrm{GCP}} [px]Claim
OBB against none (primary)
All pipelines+0.02 (7/9)+0.67 (8/9)+4.30 (8/9)+0.82 (7/9)+0.02 (7/9)
ORB+0.18 (7/9)+5.93 (8/9)+37.90 (8/9)+5.14 (8/9)+0.12 (8/9)–
SIFT+0.01 (5/9)-0.09 (2/9)-0.42 (2/9)+0.27 (5/9)+0.01 (5/9)–
RootSIFT+0.02 (5/9)-0.13 (2/9)-0.70 (2/9)+0.17 (4/9)+0.01 (4/9)–
BRISK-0.06 (2/9)+0.75 (8/9)+4.68 (8/9)+0.23 (4/9)+0.01 (4/9)–
AKAZE+0.02 (5/9)+0.07 (7/9)+0.51 (7/9)+0.31 (5/9)+0.01 (6/9)–
XFeat-0.01 (4/9)+0.25 (8/9)+1.37 (8/9)+0.35 (7/9)+0.01 (7/9)–
DISK+LightGlue+0.03 (6/9)-0.04 (4/9)-0.43 (3/9)+0.41 (6/9)+0.01 (6/9)–
DeDoDe+LightGlue+0.02 (5/9)+0.12 (4/9)+0.84 (6/9)+0.11 (5/9)+0.01 (6/9)–
KeyNet+LightGlue+0.01 (4/9)-0.02 (6/9)-0.83 (6/9)-0.02 (4/9)+0.01 (5/9)–
LoFTR+0.00 (5/9)+0.13 (8/9)+0.74 (7/9)+0.29 (6/9)+0.01 (6/9)–
HBB against none (secondary)
All pipelines+0.03 (7/9)+0.58 (8/9)+3.91 (8/9)+0.62 (6/9)+0.02 (5/9)–
ORB+0.20 (8/9)+4.97 (8/9)+34.44 (8/9)+3.32 (7/9)+0.07 (7/9)–
SIFT+0.02 (5/9)-0.10 (2/9)-0.42 (2/9)+0.37 (5/9)+0.01 (5/9)–
RootSIFT+0.01 (5/9)-0.15 (2/9)-0.78 (2/9)+0.24 (5/9)+0.01 (4/9)–
BRISK-0.04 (3/9)+0.68 (7/9)+3.86 (7/9)+0.14 (4/9)-0.00 (4/9)–
AKAZE+0.03 (5/9)+0.07 (7/9)+0.57 (7/9)+0.32 (5/9)+0.01 (6/9)–
XFeat-0.01 (3/9)+0.19 (6/9)+1.31 (7/9)+0.68 (8/9)+0.01 (6/9)–
DISK+LightGlue+0.05 (6/9)-0.06 (5/9)-0.57 (4/9)+0.41 (6/9)+0.01 (7/9)–
DeDoDe+LightGlue+0.01 (6/9)+0.15 (6/9)+1.07 (6/9)-0.09 (4/9)-0.00 (4/9)–
KeyNet+LightGlue+0.02 (5/9)-0.16 (5/9)-1.93 (5/9)+0.24 (4/9)+0.00 (5/9)–
LoFTR+0.00 (6/9)+0.13 (8/9)+0.80 (9/9)+0.32 (8/9)+0.01 (8/9)–
OBB against HBB (secondary)
All pipelines-0.00 (4/9)+0.13 (5/9)+0.63 (6/9)+0.17 (6/9)+0.00 (6/9)–
ORB+0.02 (6/9)+0.81 (5/9)+4.35 (5/9)+1.32 (6/9)+0.03 (6/9)–
SIFT-0.01 (2/9)+0.01 (6/9)+0.02 (5/9)-0.04 (4/9)-0.00 (3/9)–
RootSIFT+0.00 (5/9)+0.00 (5/9)+0.01 (5/9)-0.01 (5/9)-0.00 (4/9)–
BRISK-0.02 (2/9)+0.05 (5/9)+0.07 (5/9)-0.00 (4/9)+0.00 (4/9)–
AKAZE+0.00 (6/9)+0.01 (4/9)+0.06 (4/9)+0.03 (6/9)-0.00 (4/9)–
XFeat-0.00 (4/9)+0.13 (6/9)+0.59 (6/9)-0.29 (3/9)-0.00 (3/9)–
DISK+LightGlue-0.02 (5/9)+0.01 (4/9)+0.39 (5/9)-0.12 (3/9)-0.00 (3/9)–
DeDoDe+LightGlue+0.01 (6/9)-0.00 (4/9)+0.05 (4/9)+0.13 (6/9)+0.00 (6/9)–
KeyNet+LightGlue-0.00 (3/9)+0.09 (7/9)+0.62 (7/9)-0.05 (3/9)+0.00 (5/9)–
LoFTR-0.01 (1/9)-0.02 (3/9)-0.07 (3/9)-0.03 (5/9)-0.00 (3/9)–

Table 13.

Masking benefit by pipeline. The gains concentrate in the captured pipeline. In real time, OBB masking improves ORB over its unmasked runs in eight of nine cases on ZNCC, GOS, IoU and OCE against the GCPs, by 0.0379 in GOS, with bootstrap intervals that exclude zero; after Holm’s adjustment over the ten pipelines none reaches the rule. SIFT and

RootSIFT do not gain: masked, they score lower in ZNCC and GOS in seven of nine cases, by at most 0.0007, and neither change meets the rule. The per-pipeline benefits that meet the rule are small and belong to pipelines that are never captured or only marginally: offline, RootSIFT in GOS and BRISK in ZNCC, by 0.00027 and 0.00009 in all nine cases, and in real time LoFTR under HBB masks in GOS.

Moving objects or the feature budget. Masking can help in two ways. It removes the correspondences on the objects, from which a captured fit is built, and, for a detector that applies the mask before it caps its keypoints (§ 3.2), it hands the removed share of the budget to the ground whether or not the objects moved. The first predicts a benefit concentrated on pairs with a moving group, a captured share that falls under masking, and an error that loses its alignment with the motion. The second predicts a benefit for the detectors that refill their budget, largest at small budgets, that appears in the stationary control and on Part I’s static scenes as well. Both act on ORB, and the two protocols give evidence of each. The budget effect is real: on Part I’s static scenes OBB masking raises ORB’s mAA by 0.065 at the real-time point (§ 6.2), where Part II’s real-time ORB rises from 0.5835 to 0.6785. The captured pairs cannot account for most of that rise. A pair weighs 1/135 of its case and the cases weigh equally, so the 28 captured pairs can move ORB’s mAA by at most 28/(9×135)=0.02328/(9 \times135) = 0.023, and ORB gains in eight of the nine cases although only five ever hold a moving group (Table 13); the rest of the rise is of the size of the budget effect on static scenes. A budget effect cannot make an estimate follow a group, however, and the capture fraction finds that only without masking. The strata agree: in real time the reference-free benefit of the average pipeline is largest in the high-moving mixed cell, which holds S2, and near zero in the low-moving cells and the stationary control (file realtime_masking_benefit_strata.csv). The mechanism contrasts point the same way for ORB in real time, with a positive coverage correlation, a positive persistence gap and a positive alignment contrast, each with an interval above zero and none meeting the rule after Holm’s adjustment. For the average pipeline only the real-time persistence gap in ZNCC has an interval above zero, and offline no contrast points toward capture (files offline_masking_mechanism.csv and realtime_masking_mechanism.csv). In real time the benefit of the average pipeline is positive in every offset bin and both persistence halves, larger for pairs whose foreground moved, and by cell largest in the high-moving cells; offline it is smaller by an order of magnitude and mixed between 16 and 127 frames (files offline_masking_benefit_strata.csv and `realtime_masking_benefit_strata.csv). With two cases per cell the design reads them descriptively; the within-case contrasts, which match coverage and weigh every case equally, are the test.

Box geometry and mask margin. OBB and HBB masks do not differ on any outcome in either regime. In the offline sweep the masks withhold 2.3 to 4.0% of the frame; every geometry and margin gains a few 10−410^{-4} in ZNCC and GOS with intervals that include zero, and OBBs at a larger margin do not beat horizontal ones at equal suppressed area (file offline_margin_benefit.csv). Neither mechanism of § 5 is large enough here to be detected. The survey is flown square to an arterial grid, so most vehicles are aligned with the image axes and their HBB is nearly their OBB; over the frames of the nine cases the HBBs cover a median of 2.17% of the frame against 2.03% for the oriented ones (file offline_masking_scenes.csv), so the road an HBB withholds beyond the vehicle, and the shadow it covers beyond an OBB, are both small. At these densities and orientations the suppressed area, not the box geometry, is what a mask changes.

Behavior with temporal distance

For the average pipeline the disagreement with the reference grows with the offset. In real time it runs from 0.33 px without masking and 0.16 px with OBB masks at one frame to 3.17 and 1.09 px at 2048 to 3600 frames; offline from 0.16 and 0.09 px to 0.30 and 0.27 px, the order of the reference’s own residual (Table 6). Relative to the unmasked median, masking matters most at the shortest offsets, where the captures of Fig. C.2 are densest and the masked median is about half the unmasked one in both regimes; in real time the absolute gap peaks where unmasked ORB breaks down, at 64 to 127 frames (4.53 against 1.09 px) and at the longest offsets (files offline_masking_benefit_strata.csv and realtime_masking_benefit_strata.csv).

Discussion

Interpretation of the capture measurement

Consensus capture is the failure the dynamic protocol was built to measure. In these nine cases it occurs without masking: for ORB at both budgets, in real time in every case that presents a coherently moving group, and by a pixel or two, at the limit of what the GCPs resolve, for BRISK, SIFT and RootSIFT at the real-time budget. Under masking it occurs nowhere. This bears on the pipeline with which Fonod et al. [38] extracted the Songdo Traffic trajectories [39], ORB at the offline budget under HBB masks: unmasked, it is captured on 6 pairs, masked on none, which supports masking at detection for trajectory extraction. Three properties make this a measurement rather than an anecdote. The instruments resolve it: a complete capture misaligns the fit where the moving group lies by a sizable fraction of an object width, far above the audited error of the reference and the repeatability of the GCPs, and a partial capture still raises the capture fraction and the motion alignment. The cases were chosen to contain it: selected from the video alone, at reference points spread over every hover of a recording day, they span the amount and coherence of moving foreground that pool offers. And the tests were fixed before the outcome: the primary and mechanism contrasts were specified in advance and are reported whatever their result. Capture itself is counted descriptively, at a threshold above what the identity transform reaches, with the counts at stricter thresholds beside it. Of the two descriptors the cases were selected on, the moving share orders where capture occurs and coherence does not: the case with most captures lies in a mixed cell, because the survey averages coherence over its offsets while a capture needs one coherent group at one offset (§ 7.3).

What the cases cannot say is how the effect scales with the moving share. Their moving foreground covers 0.17% to

1.94% of the frame outside the control (Table 5), and unmasked ORB keeps on average 949 inliers per pair offline and 513 in real time in the high-moving coherent cell (released files offline_masking_by_scenario.csv and realtime_masking_by_scenario.csv). The estimator weighs a structure by its share of the correspondences, not of the area (§ 4.1), so the feature budget and the detector’s preference for the high-contrast content of vehicles (Figure 3) matter as much as the area the vehicles cover, and a low-altitude view over congested lanes, where moving vehicles can fill a large share of the frame, is a different regime. The protocol carries over unchanged; the conclusion need not.

Joint reading of the two protocols

Part I ranks pipelines against exact truth on static scenes and shows which of their settings matter there: the compute budget, by a wide margin, then the estimator’s threshold for estimators that commit to one, while the matcher thresholds have little effect at the operating point and masking costs almost nothing. Part II asks whether that ranking survives footage in which part of the frame moves. Capture predicts a specific pattern: a pipeline falls without masking, recovers under masking, and has its error along the motion while unmasked, with a capture fraction near one on the pairs that carry a moving group. The data show that pattern for ORB at the real-time budget: its unmasked estimates follow the group with a capture fraction up to 0.96 within the survey’s offsets (Figure 1), and its error along the motion rises; the alignment rises little, since the captures are a few percent of the pairs. Its fall from 0.6785 masked to 0.5835 unmasked is mostly not on the captured pairs, which bound their share of it at 0.023, and the rest is of the size of the budget effect, which masking has on static scenes as well (§ 7.5). For the other pipelines the static ranking carries over: SIFT, RootSIFT and AKAZE lead Part II in both regimes, masked or not. The static masking cost is the control that makes that reading possible: in Part I the annotated objects satisfy the same homography as the background, so suppressing them cannot improve the consensus, only change which correspondences the budget keeps.

Practical guidance

No single pipeline is called best. The leading tie set on mAA is AKAZE, RootSIFT, SIFT and BRISK under the main Part I configuration and grows to six pipelines at deployment-scale perturbation; beyond 30∘30^\circ of rotation the upright learned models drop out (Fig. B.2). In Part II the tie set against the reference is SIFT, RootSIFT and AKAZE under every masking mode in both regimes; against the GCPs every configuration ties at the median in both, and at the 90th percentile all but ORB in real time (Figure 11). Offline, masking offers little a practitioner could act on: every configuration lies within the repeatability of the GCPs, and the masking benefits are tenths of a pixel or less. In real time it makes ORB usable, at a cost on static scenes of less than 0.005 mAA. For a real-time pipeline the choice is open: masked ORB registers in 12 to 13 ms at a median error of 1.17 to 1.19 px against the GCPs, and SIFT, RootSIFT and AKAZE, masked or not, in 18 to 22 ms at 0.51 to 0.57 px; all fit one frame interval at the median, and for the latter three masking changes background alignment by less than 0.001 in either direction, which no contrast resolves (Table 9, Table 11 and Table 13). Compute excludes some pipelines on its own: LoFTR cannot run at Part I’s full resolution on the benchmark GPU, and four of the ten (BRISK, LoFTR, and DeDoDe and KeyNet with LightGlue) need more than one frame interval at the real-time point. HBBs and OBBs do not differ measurably for registration at these densities and orientations, in the masking contrasts or in the margin sweep; a road grid oblique to the image, larger vehicles at a lower altitude or long turning queues would give the geometry more room to matter, and the protocol measures it there unchanged. OBBs remain useful elsewhere, in the motion axis of § 4.4 and in object-level errors that respect an object’s extent. SIFT and the learned detectors apply the mask after selecting their keypoints in the library evaluated here (§ 3.2), so for them masking can only remove keypoints; selecting after the mask is a change to the library, which the protocol would measure unchanged.

Limitations

Coverage of the cases. The cases come from one recording day at one altitude band, and their moving foreground at the reference covers at most 1.94% of the frame, so what is found about capture holds for that range. Each case is registered onto one reference frame, chosen to span the conditions capture depends on; how often capture occurs over all the frames a deployed pipeline might register onto is a different quantity, which the design does not estimate. Part II has nine independent units at seven intersections: a masking effect is detectable only if it has the same sign in all of them, or in eight with the dissenting case the smallest, and the 2×22 \times2 with two cases per cell is descriptive. Taking the intersection as the unit leaves every claim unchanged, but the per-pipeline benefits that meet the rule fall short of it, since seven units cannot pass Holm’s adjustment over ten pipelines, and ORB’s real-time coverage correlation loses its interval above zero. Cases with a larger moving share, from lower altitudes or congested approaches, and more sites and sessions are the most informative extension; the selection procedure is automated so that they can be added without revisiting the design.

Resolution of the ground control points. The GCPs are repeatable to 1.00 px at the 95th percentile. They separate a captured estimate from a correct one and give a masking contrast independent of the reference, but they cannot rank pipelines whose errors differ by less than a pixel, and the reference-relative columns, which can, are limited by the audited residual. With eight audited frames per case, the GCPs see only the few captured estimates that fall on one (§ 7.3); the capture fraction, read against the reference on every pair, is what counts them. The reference estimator is a member of the family under test, run with OBB masking; its absolute error is measured by the audit, but not its covariance with a candidate’s, which is why the masking comparison is read from the GCPs and the reference-free measures.

Budget, resolution and scene. Part II’s two regimes change the resolution and the feature budget together; Part I separates the two on static scenes (Figure 4), and the mask-margin sweep was run in the offline regime only. Part I’s 29 scenes come from one session on one day, so the split prevents tuning on the evaluation scenes but does not make the session representative of other cities, altitudes or lighting. The registration model is a single homography: buildings, raised structures and large tree canopy are excluded from the reference-free measures, but poles, signs and isolated street trees are not, and their parallax grows with the platform’s drift, which within a hover is small and is the same for every candidate. Footage from a translating platform, or with tall structure filling much of the frame, needs a different registration model.

Annotations and object classes. The OBBs come from a segmentation-based conversion (Appendix § 14); its fallback share is zero on this benchmark’s image sets, and where it is not, a comparison between the two box geometries is diluted by it. Most of the benchmark is indifferent to what the objects are, but two parts assume vehicles: the motion axis is recovered from the long axes of OBBs, which holds for vehicles and vessels and not for people, and the coherence descriptor assumes a population that moves as streams. Both degrade to an undefined axis and an unfilled cell rather than to a wrong number.

Conclusion

Coherently moving objects can capture the geometric consensus of a robust estimator, and the statistics that describe a fit’s self-consistency would then describe the wrong fit. This paper turns that possibility into a measurement for the complete registration pipeline used in reference-frame stabilization of hovering-drone video. A static protocol under exact ground truth establishes what each pipeline can do. A dynamic protocol on nine intersection cases, each registered onto a reference frame selected from the video alone, measures what the same pipelines do when part of the frame moves, at an offline and a real-time budget, with a ground-control-point audit that measures the error of the pseudo-ground-truth reference and scores every candidate independently of it.

On static scenes, four tuned classical pipelines (AKAZE, RootSIFT, SIFT, BRISK) lead; the compute budget decides accuracy far more than the matcher thresholds, which MAGSAC++ makes nearly irrelevant, and masking costs every pipeline less than 0.005 mAA. On the dynamic cases unmasked ORB is captured in real time in every case that presents a coherently moving group, on 28 of 304 real-time pairs and 6 offline, and the estimates pulled toward a group have the inlier statistics of the same pipeline at the nearest offsets. Object masking removes capture for every pipeline at both budgets and restores ORB in real time, though the captured pairs can account for at most a quarter of that gain; it leaves the leading pipelines (SIFT, RootSIFT, AKAZE) unchanged, and HBBs and OBBs do not differ at these densities and orientations, where the two withhold nearly the same pixels.

How capture scales with the moving share, from lower altitudes or over congested approaches, is the question the benchmark is built to answer next. The protocol needs only annotated boxes for the objects and a dozen or so GCPs on each of a handful of frames per case. The benchmark implementation, the data every reported number is computed from, the OBB dataset and the human-verified reference that validates its conversion are released so that the comparison can be extended to other pipelines, other platforms and other classes of moving object without changing the protocol.

CRediT authorship contribution statement

Robert Fonod: Writing – review & editing, Writing – original draft, Visualization, Validation, Software, Project administration, Methodology, Investigation, Funding acquisition, Formal analysis, Data curation, Conceptualization. Nikolas Geroliminis: Writing – review & editing, Supervision, Resources, Project administration, Funding acquisition.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

This work was supported by the Open Research Data (ORD) Program of the ETH Board and as a part of NCCR Automation, a National Centre of Competence in Research, funded by the Swiss National Science Foundation (grant number 51NF40_225155). The funders had no role in the study design, conduct, writing, or decision to submit. The authors thank Yanruiqi Yang (EPFL) for revising the boxes and drawing the oriented ones of the human-verified reference (Appendix § 14.§ 1).

Data and code availability

The benchmark implementation is released as stabilo_benchmark (Fonod, 2026c) under the MIT license. It builds on stabilo (Fonod, 2026d) for registration and on hbb2obb (Fonod, 2026b) for object detection and OBB conversion, with the vehicle detector of geo-trax (Fonod, 2026a), whose weights are used under a CC BY 4.0 license.

The data from which every benchmark number in this paper is computed are released under a CC BY 4.0 license [36]: Part I’s 29 annotated scenes; Part II’s evaluation and tuning cases with their annotations, GCPs and cached reference homographies; the survey output the cases were selected from; and the result directory of every campaign, which records its configuration, package versions and hardware (Appendix § 10). The CSV files named in the text and table captions are in the record’s supplementary/ directory. Songdo Vision OBB and the human-verified reference that validates the conversion (Appendix § 14) are a separate record under the same license [37].

Declaration of generative AI and AI-assisted technologies in the manuscript preparation process

During the preparation of this work the authors used Claude (Anthropic) in order to assist with drafting and editing the text and with checking and editing the references. After using this tool, the authors reviewed and edited the content as needed and take full responsibility for the content of the published article.

Benchmark protocol parameters

Every campaign is a configuration file in the released implementation, and every result directory records the configuration, its hash, the resolved run grid, the package versions and the hardware it ran on. The implementation was written with the assistance of Claude Code (Anthropic) and is covered by the test suite released with it. The values below are the ones the manuscript’s campaigns use.

Table A.1: Part I perturbation. A scalar range vv means uniform on [−v,+v][-v,+v] for additive quantities and on [1−v,1+v][1-v,1+v] for multiplicative ones. The rotation sweep fixes the rotation at one magnitude per run, narrows translation to ±5%\pm5\% and scale to ±2%\pm2\% without a projective term, and disables the photometric chain; the deployment-regime configuration narrows the geometric ranges as stated in § 6.1. The geometric ranges are those of the published campaign.

PerturbationRangeWhat it stands for
Translation±10%\pm10\% of WW and of HHwind drift
Rotation±15∘\pm15^\circ about the frame centeryaw
Scale±5%\pm5\%altitude change
Projective±5×10−5\pm5 \times10^{-5} on both projective entriespitch and roll
Brightnessfactor in [0.75,1.25][0.75,1.25]exposure drift, time of day, cloud shadow
Saturationfactor in [0.95,1.05][0.95,1.05], in HSVcolor rendition
BlurGaussian, kernel of 1, 3 or 5 pxdefocus, residual motion blur
Hazewhite veil blended in, density up to 0.10haze, thin cloud, a long air column
Sensor noiseadditive Gaussian, σ\sigma up to 3.0 gray levelssensor noise
JPEGquality in [70,95][70,95]recompression of the video stream

Table A.1.

Two conventions are worth stating, because a reader comparing against published numbers for the same detectors will otherwise be comparing different quantities. First, the inlier threshold is expressed in pixels of the full-resolution *reference frame whatever the processed resolution, which is the convention of the stabilization library: the pipelines estimate current →\to reference, so the reprojection residual is evaluated there, and a threshold quoted in the other frame differs by the scale factor of the transform. Part I’s ranking campaigns and Part II’s reference estimator run at full resolution, where the processed and the full-resolution frame coincide, and Part I’s budget sweep keeps the library’s convention. Part II’s two regimes state the threshold in pixels of the downsampled frame instead, so that Part I’s values, from which their tuning starts, keep their size relative to keypoint localization: 4 px is 8 px at full resolution in the offline regime and about 22 px in the real-time one. Because every threshold is tuned at the operating point it is run at, no reported number depends on a rescaling between resolutions. Second, all reported values are for the implementations and versions recorded in each result manifest; as Barath et al. [9] document, the same nominal algorithm can differ measurably between libraries, so the implementation is part of what is being compared.

ParameterCandidatesReference estimator
Matcherbrute force, ratio test (LightGlue for its detectors; none for LoFTR)brute force, ratio test
Ratio-test thresholdtuned (Table B.1)tuned (Section 7.2)
Inlier thresholdtuned (Table B.1)tuned (Section 7.2)
Robust estimatorMAGSAC++ (OpenCV flag 38), projective modelsame
Iteration cap500020 000
Confidence1−10−61 - 10^{-6}1−10−71 - 10^{-7}
Feature budget8000; Part II: 2000 offline, 1000 in real time20 000
Reference-frame budgettwice the feature budgettwice the feature budget
Downsampling ratio1.0; Part II: 0.50 offline, 0.18 in real time1.0
Mask margin mm0.150.20
Maskingnone, horizontal, oriented (the axis)oriented

Table A.2. Fixed pipeline parameters shared by every preset unless a campaign sweeps them, and the reference estimator’s settings. The candidates’ inlier and ratio-test thresholds are the tuned parameters of Table B.1 in Part I and are tuned again on the tuning cases in Part II (Section 7.1); the reference estimator’s are tuned the same way at its own settings (Section 7.2). The downsampling ratio and the feature budget are those of the matched operating point of Section 4.7 in every Part I ranking campaign, and points of the same budget grid in Part II’s two regimes (Section 7.1). The mask margin mm scales every box about its center by 1+m1 + m before it is rasterized.

Reference framea frame of the hover chosen by the case selection
Candidate referencesevery 900 frames, clear of the hover’s ends, with 3600 frames after them
Selection offsets1, 2, 5, 11, 22 frames
Displacedfarther than 0.1 median object width
Moving togetherwithin 0.1 median object width or 25% of the displacement, whichever is larger
Regimesoffline: ratio 0.50, 2000 features; real-time: ratio 0.18, 1000 features
Inlier thresholdin pixels of the downsampled frame, both regimes
Samplingoffsets up to 3600 frames in twelve octave bins, up to 15 pairs each, 135 pairs
Audit bins1–3, 4–31, 32–511, 512–3600 frames, two frames each
Capturecapture fraction λ≥0.1\lambda\ge0.1, on pairs whose moving group holds ≥4\ge4 objects
Object dilation τ\tau10 px in both frames
Region erosion ρ\rho8 px; the validity term drops a further 2 px fringe
Excluded static structurebuildings, raised structures, large tree canopy; polygons on the reference frame
Minimum region area5% of the frame, else the identity’s score is recorded
SSIM7×77 \times7 window, region eroded by its radius
MI64×6464 \times64 intensity histogram, in bits
Motion-axis concentration κmin⁡\kappa_{\min}0.3
HEA thresholds1, 3, 5, 10 px; mAA over [0, 5] px
GCPs≥12\ge12 per audited frame, ≥8\ge8 audited frames per case
Repeats1 for OpenCV estimators; 3 for the learned pipelines on one case

Table A.3. Part II protocol constants.

Repeated runs. Jin et al. (2021) run each evaluation three times and average the results to reduce the randomness in them, which they found negligible. We report single runs instead, having measured how random the estimators used here are. On the OpenCV build recorded in the manifests, homography estimation does not change its output across different global random seeds, at outlier fractions up to 75%, and does not advance the global random stream either; each estimator seeds an internal generator from a fixed constant. Every method is additionally bit-identical across repeated calls on identical input, with one exception: the parallel LO-RANSAC variant returns one of a small set of matrices depending on thread scheduling, a race that no seed removes. Its spread is at most 1.5×10−31.5 \times10^{-3} px of corner transfer on a 3840×21603840 \times2160 frame, more than two orders of magnitude below the smallest audited resolution floor of Table 6 and therefore not a limitation on any claim made here. For the OpenCV estimators, results are consequently reproducible exactly rather than in distribution. The learned detectors and matchers run under a tensor library whose kernel selection on a GPU is not guaranteed to be bit-exact, so that argument does not extend to them, and the learned pipelines of Part II’s offline regime were repeated three times on one case instead. The repeats agree: every learned pipeline under every masking mode returned the same estimate on all 135 pairs in all three repeats, and no pair changed between success and failure (released file offline_repeats_repeat_spread.csv).

Part I: further results

Table B.1 lists the values every pipeline keeps under the rule of § 4.7 on the 9 tuning scenes at 30 trials each, over ten ratios rr (0.5, 0.6, 0.7, 0.75, 0.8, 0.85, 0.9, 0.93, 0.96, 0.99) and eight inlier thresholds θ\theta (0.25, 0.5, 1, 1.5, 2, 3, 4, 8 px), with the mAA at the untuned starting values and the number of grid points tied with the best. Where every grid point is tied, no point of the grid is better supported than another, which is why the preset’s values are kept. LoFTR has no row, since it cannot run at the operating point. The other tables behind § 6.2 are released as files of the data record: the detector axis at the starting values on the evaluation scenes (detectors_default_static.csv) and at deployment-scale perturbation (detectors_hover_static.csv), the rotation sweep of Fig. B.2 (rotation_sweep.csv) and the estimator sweep of Fig. B.1 (estimators_estimators.csv).

Pipelinerrθ\theta [px]mAAmAAdefaultmAA_{\mathrm{default}}Δ\DeltamAACTE50_{50} [px]GridTied
AKAZE0.7040.9856{sup}§`0.9844{sup}‡`+0.00110.0668080
SIFT0.700.50.9840{sup}§`0.9830+0.00100.0768080
RootSIFT0.700.50.9839{sup}§`0.9829+0.00100.0778080
BRISK0.701.50.9811{sup}§`0.9802{sup}‡`+0.00090.0898080
DISK+LightGlue–20.9708{sup}§`0.9708+0.00000.12688
ORB0.8540.9708{sup}§`0.9706+0.00020.1378080
DeDoDe+LightGlue–10.9625{sup}§`0.9624+0.00010.17788
XFeat0.7020.9470{sup}§`0.9407+0.00620.2548057
KeyNet+LightGlue–40.9214{sup}§`0.9214+0.00000.38288

Table B.1. Part I: the joint sweep of the ratio-test threshold rr and the inlier threshold θ\theta (reference-frame pixels) on the held-out tuning scenes. A pipeline keeps its preset’s values unless a grid point beats them on mAA by more than the margin (0.01); Grid counts the grid points and Tied those within the margin of the best. mAAdefaultmAA_{\mathrm{default}} is the score at the untuned starting values, a ratio of 0.9 and an inlier threshold of 2 px (3 px for RootSIFT and SIFT; 2.5 px for AKAZE and BRISK), and Δ\DeltamAA, never bolded, the gain from tuning. rr reads – for matchers without a ratio test. No configuration failed on any trial. ‡ marks a reference value off the grid, read at its nearest grid point (the better of two equidistant ones), so the gain beside it is a lower bound. § marks incumbent values kept because no grid point beats them by more than the margin.

The estimator sweep. The estimator axis is swept for two detectors rather than ten. The estimator sees only the correspondences, and the two cover both descriptor families the classical pipelines use: ORB, the fastest, with a binary descriptor, and SIFT, the reference estimator’s detector, with a floating-point one. Nine estimators at six thresholds already make 108 runs of 2000 trials each; all ten detectors would make five times as many, the learned ones among the slowest. With the detector fixed and OBB masking, the estimator matters less than the threshold it runs at (Fig. B.1; values in file estimators_estimators.csv). With SIFT, every estimator but RHO reaches an mAA of 0.983 to 0.984 at its best threshold, all within the equivalence margin of one another; RHO peaks at 0.965 and is the only estimator that fails (6 of its 12 000 trials) or leaves the integration interval (9). With ORB, the three LO-RANSAC configurations, GC-RANSAC and PROSAC reach 0.972 to 0.973 at θ=2\theta= 2 px, MAGSAC++ and LMEDS lie within the margin at 0.970 and 0.968, and plain RANSAC and RHO fall outside it at 0.940 and 0.925. MAGSAC++ and LMEDS move by less than 0.0001 over the whole range of θ\theta for both detectors, since neither commits to one threshold (LMEDS has none), whereas with ORB plain RANSAC loses 0.14 and RHO 0.13 between their best threshold and θ=0.5\theta= 0.5 px. The inlier ratio records the threshold rather than the estimate: at a given θ\theta it is nearly the same for every estimator but RHO, it rises from about 0.17 to 0.81 for ORB and from 0.77 to 0.93 for SIFT between θ=0.5\theta= 0.5 px and 4 px whether mAA rises, falls or stays constant, and the highest inlier ratio in the sweep belongs to neither detector’s best configuration.

Figure B.1. mAA against the inlier threshold for every robust estimator, with the detector fixed (ORB, left; SIFT, right: one binary and one floating-point descriptor), OBB masking, on the evaluation scenes. Color follows the algorithm and the line style the OpenCV flag within it, so the three LO-RANSAC configurations share a hue.

The deployment regime. At deployment-scale perturbation (file detectors_hover_static.csv) every pipeline scores higher and the pipelines move closer together: unmasked mAA spans 0.968 to 0.994, against 0.919 to 0.984 in the main configuration. The leading tie set grows from four pipelines to six. AKAZE stays at the top at 0.994, SIFT and RootSIFT follow at 0.993, and DISK with LightGlue, DeDoDe with LightGlue and BRISK lie within the equivalence margin of it, while ORB, KeyNet with LightGlue and XFeat remain outside it. No pipeline moves by more than two places, and no trial fails or leaves the integration interval under any masking strategy. The wider configuration therefore separates the pipelines more than a hovering platform would, without changing which three pipelines lead, which is why the ranking is read from it.

Figure B.2. mAA against in-plane rotation on the evaluation scenes, OBB masking, 50 trials per scene at each of nine magnitudes. LoFTR cannot run at full resolution on the benchmark GPU and is not shown.

Rotation limits of upright models. Several learning-based detectors do not estimate keypoint orientation, and Fig. B.2 (values in file rotation_sweep.csv) shows where that starts to matter. Up to 30∘30^\circ these pipelines lose accuracy gradually, as the classical ones do: at 30∘30^\circ XFeat, DISK and DeDoDe with LightGlue score 0.843, 0.899 and 0.904, against 0.964 for SIFT. Beyond it they collapse. XFeat falls to 0.442 at 45∘45^\circ and 0.003 at 60∘60^\circ; DISK and DeDoDe with LightGlue hold 0.8260.826 and 0.8600.860 at 45∘45^\circ, fall to 0.1030.103 and 0.4800.480 at 60∘60^\circ, and reach 0.0000.000 and 0.0760.076 at 90∘90^\circ. KeyNet with LightGlue, which estimates orientation, degrades steadily instead, to 0.2240.224 at 90∘90^\circ, and SIFT and ORB stay above 0.870.87 over the whole range. The 15∘15^\circ of the main configuration lies well inside the range where no pipeline has collapsed. The limit is a property of the pretrained weights, so it is reported rather than compensated for.

Part II: further results

This appendix gives the tables and figures that the main tables of § 7 summarize. Every one is written by report from the result directories, and the file each caption names holds the rows a table summarizes; the remaining tables are released as files of the data record, written by the same run.

Tuning. Tables C.1 and C.2 list the thresholds selected in each regime on the five tuning cases, with Part I’s values as the incumbents.

Table C.1. Part II (offline regime at downsample ratio 0.5 and 2000 features): the joint sweep of rr and θ\theta (pixels of the downsampled image) on 5 tuning cases from the sites of Part I’s tuning scenes, unmasked, against the reference estimator, each case weighing equally. A pipeline keeps its Part I values unless a grid point beats them on mAA by more than the margin (0.01); Grid counts the grid points and Tied those within the margin of the best. mAAdefault_{\mathrm{default}} is the score at the Part I values and Δ\DeltamAA, never bolded, the gain from tuning. rr reads – for matchers without a ratio test. No configuration failed on any trial. † marks a selected value at an end of its swept range, where the optimum may lie outside the grid. § marks incumbent values kept because no grid point beats them by more than the margin.

Pipelinerrθ\theta [px]mAAmAAdefault_{\mathrm{default}}Δ\DeltamAACTE50_{50} [px]GridTied
SIFT0.40†0.1†0.98120.9710+0.01020.08418285
AKAZE0.40†0.1†0.98120.9681+0.01310.08118265
RootSIFT0.700.50.9716§0.9716+0.00000.13118298
LoFTR–20.9597§0.9597+0.00000.1951313
BRISK0.40†0.1†0.95710.9415+0.01560.19718251
DISK+LightGlue–20.9559§0.9559+0.00000.191138
KeyNet+LightGlue–40.9522§0.9522+0.00000.2301312
DeDoDe+LightGlue–10.9380§0.9380+0.00000.2741313
XFeat0.4580.93360.9197+0.01380.31318259
ORB0.8540.9004§0.9004+0.00000.43518218

Table C.2. Part II (real-time regime at downsample ratio 0.18 and 1000 features): the joint sweep of rr and θ\theta (pixels of the downsampled image) on 5 tuning cases from the sites of Part I’s tuning scenes, unmasked, against the reference estimator, each case weighing equally. A pipeline keeps its Part I values unless a grid point beats them on mAA by more than the margin (0.01); Grid counts the grid points and Tied those within the margin of the best. mAAdefault_{\mathrm{default}} is the score at the Part I values and Δ\DeltamAA, never bolded, the gain from tuning. rr reads – for matchers without a ratio test. No configuration failed on any trial. † marks a selected value at an end of its swept range, where the optimum may lie outside the grid. § marks incumbent values kept because no grid point beats them by more than the margin.

Pipelinerrθ\theta [px]mAAmAAdefault_{\mathrm{default}}Δ\DeltamAACTE50_{50} [px]GridTied
RootSIFT0.700.50.9095§0.9095+0.00000.437182150
SIFT0.700.50.9088§0.9088+0.00000.454182145
AKAZE0.500.1†0.90640.8963+0.01000.45218288
LoFTR–20.8639§0.8639+0.00000.7031313
KeyNet+LightGlue–40.8326§0.8326+0.00000.816134
XFeat0.8060.81750.7669+0.05060.92018249
DeDoDe+LightGlue–40.80180.6769+0.12490.981136
DISK+LightGlue–30.79580.7590+0.03681.000132
BRISK0.701.50.7955§0.7955+0.00000.99818268
ORB0.99†30.67070.6593+0.01141.4881826

Case selection and the reference estimator. Table C.3 checks the survey’s descriptors against the published trajectories, file offline_masking_scenes.csv characterizes the frames of the nine cases, Table C.4 reports the tuning of the reference estimator, Table C.5 how every GCP of the audited frames was placed, and Table C.6 the agreement of the proposals with unaided clicks on two cases.

Table C.3. The survey’s 831 candidate reference frames against the published vehicle trajectories of the same hovers, which the survey never reads. Each descriptor is recomputed from the trajectories with the survey’s own definitions (ground coordinates, thresholds in median vehicle widths, the survey’s offsets). Areas are in square meters there and frame fractions in the survey, so agreement is stated as correlation. Cell agreement is the share of candidates placed in the same moving × coherence cell by both, among those the survey placed, by their margin from the split boundaries; the selection takes the two of largest margin per cell.

DescriptorCandidatesPearson rrSpearman ρ\rho
Static foreground8310.9720.955
Moving (displaced) foreground8310.9740.953
Coherently moving foreground8310.9700.933
Coherence7260.8980.892
Stationarity8310.9260.918
Cell agreementCandidatesShare
all placed7030.81
top 50% by margin3520.94
top 25% by margin1760.97
selected cases99/9

Table C.4. Part I: the joint sweep of the ratio-test threshold rr and the inlier threshold θ\theta (reference-frame pixels) on the held-out tuning scenes. A pipeline keeps its preset’s values unless a grid point beats them on mAA by more than the margin (0.01); Grid counts the grid points and Tied those within the margin of the best. mAAdefault_{\mathrm{default}} is the score at the preset’s own values (it has no separate starting values), and Δ\DeltamAA, never bolded, the gain from tuning. rr reads – for matchers without a ratio test. No configuration failed on any trial. § marks incumbent values kept because no grid point beats them by more than the margin.

Pipelinerrθ\theta [px]mAAmAAdefault_{\mathrm{default}}Δ\DeltamAACTE50_{50} [px]GridTied
Reference estimator0.700.50.9856§0.9856+0.00000.0737070

Table C.5. Part II: how the ground control points of the audited frames were placed, per case. A local image fit proposes each point and the annotator reviews the standing proposals: Accepted counts those kept as they were, Corrected those moved, by the median Corr.₅₀ and the largest Corr.ₘₐₓ distance, and By hand the points without a standing proposal, placed in the annotation panel. The reference frame’s points, all placed by hand, are not counted. The last row pools every case.

CaseCellPointsAcceptedCorrectedCorr.₅₀ [px]Corr.ₘₐₓ [px]By hand
2022-10-06_D1_PM1_B2high moving / mixed1201170––3
2022-10-06_D10_PM5_U1high moving / coherent1198550.350.5029
2022-10-06_D3_AM1_M1low moving / coherent1191130––6
2022-10-06_D3_PM3_F2high moving / coherent11911610.790.792
2022-10-06_D7_PM1_P1low moving / coherent1181160––2
2022-10-06_D7_PM5_P2low moving / mixed1171160––1
2022-10-06_D8_PM3_M1low moving / mixed1161100––6
2022-10-06_D9_AM1_R1stationary control1191120––7
2022-10-06_D9_AM3_S2high moving / mixed11711110.500.505
all cases106499670.350.7961

Table C.6. Click noise against estimator error on 2 cases (2022-10-06_D10_PM5_U1 and 2022-10-06_D9_AM3_S2), whose audited frames were also annotated entirely by hand, each point clicked in a magnified panel opened near the reference estimator’s prediction under a hidden offset. Over the points that have an unaided click, a standing proposal and a prediction, the distance between each pair of positions in current-frame pixels: median and 95th percentile; the last row pools every point.

CasePointsClick – proposal MedianClick – proposal 95thClick – estimator MedianClick – estimator 95thProposal – estimator MedianProposal – estimator 95th
2022-10-06_D10_PM5_U1900.390.750.370.760.040.09
2022-10-06_D9_AM3_S21120.470.980.521.080.110.40
Pooled2020.440.820.460.910.060.32

Capture. Fig. C.1 is the offline counterpart of Figure 8, and file offline_masking_capture.csv that of Table 7; Fig. C.2 gives the captured share per offset bin in both regimes and Fig. C.3 the direction of the registration error.

Figure C.1. The capture fraction of every offline measurement on a pair with a moving group, drawn as Figure 8.

Figure C.2. Share of the pairs with a moving group on which the estimate is captured (capture fraction λ≥0.1\lambda\ge0.1, Eq. (18)), by offset bin, for every configuration captured at least once in either regime, all without masking.

Masking. The tables behind § 7.5 are released as files: for the mechanism contrasts, offline_masking_mechanism.csv and realtime_masking_mechanism.csv; for the benefit by stratum, offline_masking_benefit_strata.csv and realtime_masking_benefit_strata.csv; and for the mask-margin sweep, offline_margin_benefit.csv.

Figure C.3. Direction of the registration error: the share of the corner error lying along the dominant motion axis, AA of Eq. (16), by masking mode, over the frame pairs whose axis is defined (κ≥κmin⁡\kappa\ge\kappa_{\min}), every pipeline pooled. Left, offline regime; right, real-time regime. Capture would move the unmasked distribution toward 1, away from the masked ones.

The analysis plan applied

The rule of Section 4.9 decides what a results table bolds, and the bold is a verdict rather than evidence. The comparisons behind it are released for every bolded column, one row per configuration and outcome: for Part I in file detectors_comparisons.csv and for Part II, one file per regime, in offline_masking_comparisons.csv and realtime_masking_comparisons.csv; the other campaigns’ files are named in their captions. Each row gives the Hodges–Lehmann estimate of the paired difference to the best configuration, its 95 % cluster bootstrap interval, and the exact sign-flip pp adjusted by Holm within the outcome. A reader who disagrees with the equivalence margin can therefore reread every conclusion at a margin of their own, which a table of bolded means alone would not allow.

Horizontal-to-oriented box conversion

The OBBs are converted from the HBBs by hbb2obb ([33]). This appendix summarizes the procedure, selection and accuracy, which Songdo Vision OBB ([37]) documents in full with the commands that reproduce them.

Procedure. Each HBB, its sides moved outward by a scale factor ss of its width and height (each dimension grows by 2s2s), prompts one or more SAM-family models ([56, 74]), and an ensemble’s masks are combined by majority vote. The mask is cropped to the scaled prompt and refined by a signed morphological step ([81]), its square kernel a set fraction of the shorter side of the largest contour’s bounding rectangle: a positive kernel opens it, removing a thin protrusion such as an overhead sign enclosed by the box, and a negative kernel closes it, rejoining a body split by glare or an occluder. A minimum-area rotated rectangle ([41]) is fitted to the largest contour together with any piece holding at least a set fraction of its area. The OBB keeps the class of its HBB; where no valid mask is found, the HBB is emitted unchanged, the fallback whose consequence is stated in Section 5.

Selection. The ensemble, the inference resolution, the prompt scale factor and the signed kernel fraction interact, so they were selected by grid search against the human-verified reference of Appendix § 14.§ 1: eleven model configurations, each at 45 grid points (resolution {896,1024,1152}\{896, 1024, 1152\}, scale factor {−0.01,0,0.01,0.02,0.03}\{-0.01, 0, 0.01, 0.02, 0.03\}, kernel {−0.10,0,+0.10}\{-0.10, 0, +0.10\}), 495 conversions of the 50 reference frames. Mean IoU saturates on tightly fitted boxes: the three leading configurations lie within 0.44 standard errors of one another (Table E.1 and Fig. E.1). Configurations within one standard error of the leader are treated as tied, a heuristic rather than a test, and ties are broken by the share of boxes below IoU 0.75 and the 90th-percentile orientation error, taking the cheapest configuration that leads on both. That is a single sam_1 at resolution 1024, with a scale factor of 0.02 and a closing whose kernel is 0.1 of that side, at 39 % of the inference cost of the leading ensemble. Accuracy. On the reference, the selected configuration reaches a mean IoU of 0.88679 (standard error 0.00143), a median of 0.91759, 92.1 % of boxes above IoU 0.75 and 61.5 % above 0.90, and a 90th-percentile orientation error of 3.50°, measured on the frames it was selected on, with exact prompts (Appendix § 14.§ 1). With perturbed copies it loses at most 0.002 of mean IoU while the expanded prompt still covers the vehicle, 0.027 under a 2 px center jitter and 0.043 once the prompt falls short. On the 1345 vehicles turned off the image axes it raises mean IoU over the HBB itself from 0.6032 to 0.8039 and improves on it for 91.9 % of them; over the whole set the 90th-percentile orientation error falls from 22.69° to 3.50°. Segmentation found a usable mask inside all 4245 reference boxes and all 274 190 boxes of the release, so neither carries a fallback.

Figure E.1. Accuracy against time, one point per model configuration at its best grid point for the score shown, with the Pareto front dashed (the key below the panels names the checkpoints; + joins an ensemble). Left, mean IoU: the shaded band is one standard error of the mean about the leading configuration. Right, the share of boxes at IoU 0.90 or above, which does not saturate and orders the configurations the mean cannot separate. Times are a single pass over the 50 reference frames.

ModelsSizeScaleKernelMean IoUIoU≥0.90Angle90 [°]Time [s]
sam_l + sam_b + sam2_b + sam2.1_b + sam2.1_t11520.03−0.10.88728 ± 0.0015064.1%4.38385.5
sam_l10240.02−0.10.88679 ± 0.0014361.5%3.50149.4
sam_l + sam_b + sam2.1_b10240.02−0.10.88663 ± 0.0014763.2%3.85262.2
sam_l + sam_b + sam2_b + sam2.1_b10240.03−0.10.88566 ± 0.0014861.2%3.96331.2
sam2.1_b11520.03−0.10.88457 ± 0.0015864.0%4.56160.3
sam_l + sam_b + mobile_sam10240.02−0.10.88414 ± 0.0014760.9%3.97257.8
sam_l + sam_b8960.02−0.10.88284 ± 0.0015261.4%4.06203.7
sam_b10240.02−0.10.88104 ± 0.0014959.8%4.18156.9
sam2.1_t11520.02−0.10.87478 ± 0.0017163.3%6.04175.9
sam_l + mobile_sam10240.03−0.10.86716 ± 0.0014143.4%4.03200.7
mobile_sam11520.02−0.10.85988 ± 0.0014941.9%5.63174.1

Table E.1. OBB conversion against the 4245 boxes of the human-verified reference, one row per model configuration at its grid point of highest mean IoU. Models lists the segmentation checkpoints of an ensemble; Size is the inference resolution, Scale the prompt expansion and Kernel the morphological setting (negative closes). Mean IoU is given with its standard error over boxes; configurations within one standard error of the leader are tied with it and set in bold. IoU≥0.90 is the share of boxes at or above that rung, and Angle90 the 90th percentile of the orientation error.

The human-verified reference

The reference is 4245 OBBs over 50 frames from six recordings. A detector supplied the HBBs; one annotator went over every frame and redrew as OBBs the 1345 vehicles not square to the image, leaving the other 2900 on their detected geometry, and the set was restricted to the four vehicle classes, rectified to true rectangles, and reconciled one-to-one with the HBBs that prompt the conversion, each the exact axis-aligned envelope of its OBB. The unrevised boxes are about 5.8 % wider across the vehicle than hand-drawn ones, so on the axis-aligned majority mean IoU mixes reference slack with conversion error. An independent DOTA-pretrained oriented detector (YOLO11x-OBB (Jocher and Qiu, [51])), run tiled and scored class-agnostically, which shares neither architecture nor training data with the conversion, places the axes of these vehicles within a median 0.82° of the reference boxes, the scale against which the conversion’s orientation error is read.

References

  1. [1]Abdel-Aty, M. et al. (2023). “Advances and applications of computer vision techniques in vehicle trajectory generation and surrogate traffic safety indicators”. In: Accident Analysis & Prevention 191, p. 107191. doi: 10.1016/j.aap.2023.107191.DOI
  2. [2]Alcantarilla, P. F., A. Bartoli, and A. J. Davison (2012). “KAZE Features”. In: European Conference on Computer Vision (ECCV). Vol. 7577. Lecture Notes in Computer Science, pp. 214–227. doi: 10.1007/978-3-642-33783-3_16.DOI
  3. [3]Alcantarilla, P. F., J. Nuevo, and A. Bartoli (2013). “Fast Explicit Diffusion for Accelerated Features in Nonlinear Scale Spaces”. In: British Machine Vision Conference (BMVC), pp. 13.1–13.11. doi: 10.5244/C.27.13.DOI
  4. [4]Arandjelović, R. and A. Zisserman (2012). “Three things everyone should know to improve object retrieval”. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2911–2918. doi: 10.1109/CVPR.2012.6248018.DOI
  5. [6]Barath, D. and J. Matas (2018). “Graph-Cut RANSAC”. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6733–6741. doi: 10.1109/CVPR.2018.00704.DOI
  6. [8]Barath, D. et al. (2020). “MAGSAC++, a Fast, Reliable and Accurate Robust Estimator”. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1301–1309. doi: 10.1109/CVPR42600.2020.00138.DOI
  7. [9]Barath, D. et al. (2023). “A Large-Scale Homography Benchmark”. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 21360–21370. doi: 10.1109/CVPR52729.2023.02046.DOI
  8. [10]Barmpounakis, E. and N. Geroliminis (2020). “On the new era of urban traffic monitoring with massive drone data: The pNEUMA large-scale field experiment”. In: Transportation Research Part C: Emerging Technologies 111, pp. 50–71. doi: 10.1016/j.trc.2019.11.023.DOI
  9. [11]Barroso-Laguna, A. et al. (2019). “Key.Net: Keypoint Detection by Handcrafted and Learned CNN Filters”. In: IEEE/CVF International Conference on Computer Vision (ICCV), pp. 5835–5843. doi: 10.1109/ICCV.2019.00593.
  10. [12]Bazargani, H., O. Bilaniuk, and R. Laganière (2018). “A Fast and Robust Homography Scheme for Real-Time Planar Target Detection”. In: Journal of Real-Time Image Processing 15.4, pp. 739–758. doi: 10.1007/s11554-015-0508-4.
  11. [14]Bock, J. et al. (2020). “The InD Dataset: A Drone Dataset of Naturalistic Road User Trajectories at German Intersections”. In: IEEE Intelligent Vehicles Symposium (IV), pp. 1929–1934. doi: 10.1109/IV47402.2020.9304839.DOI
  12. [15]Brachmann, E. et al. (2021). “On the Limits of Pseudo Ground Truth in Visual Camera Re-localisation”. In: IEEE/CVF International Conference on Computer Vision (ICCV), pp. 6198–6208. doi: 10.1109/ICCV48922.2021.00616.DOI
  13. [16]Choi, S., T. Kim, and W. Yu (2009). “Robust video stabilization to outlier motion using adaptive RANSAC”. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1897–1902. doi: 10.1109/IROS.2009.5354240.DOI
  14. [17]Chum, O. and J. Matas (2005). “Matching with PROSAC: Progressive Sample Consensus”. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). Vol. 1, pp. 220–226. doi: 10.1109/CVPR.2005.221.DOI
  15. [18]Chum, O., J. Matas, and J. Kittler (2003). “Locally Optimized RANSAC”. In: Pattern Recognition (DAGM). Lecture Notes in Computer Science. Springer, pp. 236–243. doi: 10.1007/978-3-540-45243-0_31.DOI
  16. [19]Chum, O., T. Werner, and J. Matas (2005). “Two-View Geometry Estimation Unaffected by a Dominant Plane”. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). Vol. 1, pp. 772–779. doi: 10.1109/CVPR.2005.354.DOI
  17. [20]Davison, A. C. and D. V. Hinkley (1997). Bootstrap Methods and their Application. Cambridge: Cambridge University Press. doi: 10.1017/CBO9780511802843.DOI
  18. [21]Demšar, J. (2006). “Statistical Comparisons of Classifiers over Multiple Data Sets”. In: Journal of Machine Learning Research 7, pp. 1–30.DOI
  19. [23]DeTone, D., T. Malisiewicz, and A. Rabinovich (2018). “SuperPoint: Self-Supervised Interest Point Detection and Description”. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). doi: 10.1109/CVPRW.2018.00060.DOI
  20. [24]Ding, J. et al. (2022). “Object Detection in Aerial Images: A Large-Scale Benchmark and Challenges”. In: IEEE Transactions on Pattern Analysis and Machine Intelligence 44.11, pp. 7778–7796. doi: 10.1109/TPAMI.2021.3117983.DOI
  21. [25]Edstedt, J. et al. (2024a). “DeDoDe: Detect, Don’t Describe — Describe, Don’t Detect for Local Feature Matching”. In: International Conference on 3D Vision (3DV), pp. 148–157. doi: 10.1109/3DV62453.2024.00035.DOI
  22. [27]Efe, U., K. G. İnce, and A. A. Alatan (2021). “Effect of Parameter Optimization on Classical and Learning-based Image Matching Methods”. In: IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), pp. 2506–2513. doi: 10.1109/ICCVW54120.2021.00283.DOI
  23. [28]Espadaler-Clapés, J., E. Barmpounakis, and N. Geroliminis (2023). “Traffic congestion and noise emissions with detailed vehicle trajectories from UAVs”. In: Transportation Research Part D: Transport and Environment 121, p. 103822. doi: 10.1016/j.trd.2023.103822.
  24. [29]Espadaler-Clapés, J. et al. (2025). “An accurate safety and congestion monitoring framework with a swarm of drones”. In: Transportation Research Interdisciplinary Perspectives 32, p. 101490. doi: 10.1016/j.trip.2025.101490.
  25. [30]Evangelidis, G. D. and E. Z. Psarakis (2008). “Parametric Image Alignment Using Enhanced Correlation Coefficient Maximization”. In: IEEE Transactions on Pattern Analysis and Machine Intelligence 30.10, pp. 1858–1865. doi: 10.1109/TPAMI.2008.113.DOI
  26. [31]Fischler, M. A. and R. C. Bolles (1981). “Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography”. In: Communications of the ACM 24.6, pp. 381–395. doi: 10.1145/358669.358692.DOI
  27. [32]Fonod, R. (2026a). Geo-trax: A Comprehensive Framework for Georeferenced Vehicle Trajectory Extraction from Drone Imagery. Version 1.3.0. doi: 10.5281/zenodo.21285193. URL: https://github.com/rfonod/geo-trax.
  28. [33]Fonod, R. (2026b). HBB2OBB: Horizontal to Oriented Bounding Box Conversion and Evaluation Tool. Version 1.12.0. DOI: 10.5281/zenodo.23647804. URL: https://github.com/rfonod/hbb2obb.DOI
  29. [35]Fonod, R. (2026d). Stabilo: A Comprehensive Python Library for Video and Trajectory Stabilization with User-Defined Masks. Version 1.4.5. doi: 10.5281/zenodo.23560718. URL: https://github.com/rfonod/stabilo.
  30. [36][dataset] Fonod, R. and N. Geroliminis (2026a). Accompanying data for the paper “Consensus Capture in Reference-Based UAV Video Stabilization: A Benchmark on Dynamic Urban Scenes”. Version v1. Zenodo. DOI: 10.5281/zenodo.23623540.
  31. [37][dataset] Fonod, R. and N. Geroliminis (2026b). Songdo Vision OBB: Oriented Bounding-Box Vehicle Annotations from High-Altitude BeV Drone Imagery in a Smart City. Version v1. Zenodo. DOI: 10.5281/zenodo.15050578.
  32. [38]Fonod, R. et al. (2025a). “Advanced computer vision for extracting georeferenced vehicle trajectories from drone imagery”. In: Transportation Research Part C: Emerging Technologies 178, p. 105205. ISSN: 0968-090X. DOI: 10.1016/j.trc.2025.105205.DOI
  33. [39][dataset] Fonod, R. et al. (2025b). Songdo Traffic: High Accuracy Georeferenced Vehicle Trajectories from a Large-Scale Study in a Smart City. Version v2. Zenodo. DOI: 10.5281/zenodo.17924857.DOI
  34. [40][dataset] Fonod, R. et al. (2026). Songdo Vision: Vehicle Annotations from High-Altitude BeV Drone Imagery in a Smart City. Version v2. Zenodo. DOI: 10.5281/zenodo.21728430.DOI
  35. [41]Freeman, H. and R. Shapira (1975). “Determining the Minimum-Area Encasing Rectangle for an Arbitrary Closed Curve”. In: Communications of the ACM 18.7, pp. 409–413. doi: 10.1145/360881.360919.DOI
  36. [42]Geng, Z. et al. (2025). “Feature-based multimodal remote sensing image matching: Benchmark and state-of-the-art”. In: ISPRS Journal of Photogrammetry and Remote Sensing 229, pp. 285–302. DOI: 10.1016/j.isprsjprs.2025.08.028.
  37. [43]Good, P. I. (2005). Permutation, Parametric, and Bootstrap Tests of Hypotheses. 3rd ed. Springer Series in Statistics. New York: Springer. DOI: 10.1007/b138696.DOI
  38. [44]Grundmann, M., V. Kwatra, and I. Essa (2011). “Auto-directed video stabilization with robust L1 optimal camera paths”. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 225–232. DOI: 10.1109/CVPR.2011.5995525.
  39. [45]Haber, E. and J. Modersitzki (2006). “Intensity Gradient Based Registration and Fusion of Multi-modal Images”. In: Medical Image Computing and Computer-Assisted Intervention (MICCAI). Vol. 4191. Lecture Notes in Computer Science. Springer, pp. 726–733. DOI: 10.1007/11866763_89.DOI
  40. [46]Hartley, R. and A. Zisserman (2004). Multiple View Geometry in Computer Vision. 2nd ed. Cambridge: Cambridge University Press. DOI: 10.1017/CBO9780511811685.DOI
  41. [47]Hodges, J. L. and E. L. Lehmann (1963). “Estimates of Location Based on Rank Tests”. In: The Annals of Mathematical Statistics 34.2, pp. 598–611. doi: 10.1214/aoms/1177704172.DOI
  42. [48]Holm, S. (1979). “A Simple Sequentially Rejective Multiple Test Procedure”. In: Scandinavian Journal of Statistics 6.2, pp. 65–70.DOI
  43. [50]Jin, Y. et al. (2021). “Image Matching Across Wide Baselines: From Paper to Practice”. In: International Journal of Computer Vision 129.2, pp. 517–547. doi: 10.1007/s11263-020-01385-0.DOI
  44. [51]Jocher, G. and J. Qiu (2024). Ultralytics YOLO11. Version 11.0.0. URL: https://github.com/ultralytics/ultralytics (visited on 11/06/2026).DOI
  45. [52]K. Nejadasl, F., B. G. H. Gorte, and S. P. Hoogendoorn (2006). “Optical flow based vehicle tracking strengthened by statistical decisions”. In: ISPRS Journal of Photogrammetry and Remote Sensing 61.3–4, pp. 159–169. DOI: 10.1016/j.isprsjprs.2006.09.007.DOI
  46. [53]Karimi Nejadasl, F. and R. Lindenbergh (2014). “Sequential and Automatic Image-Sequence Registration of Road Areas Monitored from a Hovering Helicopter”. In: Sensors 14.9, pp. 16630–16650. DOI: 10.3390/s140916630.
  47. [54]Kim, S., P. Frossard, and N. Geroliminis (2026). “Lane-Changing Maneuver Prediction With Urban Arterial Traffic Data From Drones”. In: IEEE Transactions on Intelligent Transportation Systems 27.9, pp. 10878–10892. DOI: 10.1109/TITS.2026.3696643.DOI
  48. [55]Kirchhof, M. and U. Stilla (2006). “Detection of moving objects in airborne thermal videos”. In: ISPRS Journal of Photogrammetry and Remote Sensing 61.3–4, pp. 187–196. DOI: 10.1016/j.isprsjprs.2006.09.003.
  49. [56]Kirillov, A. et al. (2023). “Segment Anything”. In: IEEE/CVF International Conference on Computer Vision (ICCV), pp. 3992–4003. DOI: 10.1109/ICCV51070.2023.00371.
  50. [57]Kumar, R. et al. (2001). “Aerial video surveillance and exploitation”. In: Proceedings of the IEEE 89.10, pp. 1518–1539. DOI: 10.1109/5.959344.DOI
  51. [58]Lakens, D. (2017). “Equivalence Tests: A Practical Primer for t Tests, Correlations, and Meta-Analyses”. In: Social Psychological and Personality Science 8.4, pp. 355–362. doi: 10.1177/1948550617697177.DOI
  52. [59]Leutenegger, S., M. Chli, and R. Y. Siegwart (2011). “BRISK: Binary Robust invariant scalable keypoints”. In: International Conference on Computer Vision (ICCV), pp. 2548–2555. doi: 10.1109/ICCV.2011.6126542.DOI
  53. [60]Li, J., Q. Hu, and M. Ai (2020). “RIFT: Multi-Modal Image Matching Based on Radiation-Variation Insensitive Feature Transform”. In: IEEE Transactions on Image Processing 29, pp. 3296–3310. DOI: 10.1109/TIP.2019.2959244.DOI
  54. [61]Lindenberger, P., P.-E. Sarlin, and M. Pollefeys (2023). “LightGlue: Local Feature Matching at Light Speed”. In: IEEE/CVF International Conference on Computer Vision (ICCV), pp. 17581–17592. DOI: 10.1109/ICCV51070.2023.01616.DOI
  55. [62]Liu, S. et al. (2013). “Bundled camera paths for video stabilization”. In: ACM Transactions on Graphics 32.4, pp. 1–10. DOI: 10.1145/2461912.2461995.DOI
  56. [63]Ljubičić, R. et al. (2021). “A comparison of tools and techniques for stabilising unmanned aerial system (UAS) imagery for surface flow observations”. In: Hydrology and Earth System Sciences 25.9, pp. 5105–5132. DOI: 10.5194/hess-25-5105-2021.DOI
  57. [64]Lowe, D. G. (2004). “Distinctive Image Features from Scale-Invariant Keypoints”. In: International Journal of Computer Vision 60.2, pp. 91–110. doi: 10.1023/B:VISI.0000029664.99615.94.DOI
  58. [65]Lyu, Y. et al. (2020). “UAVid: A semantic segmentation dataset for UAV imagery”. In: ISPRS Journal of Photogrammetry and Remote Sensing 165, pp. 108–119. doi: 10.1016/j.isprsjprs.2020.05.009.
  59. [67]Maes, F. et al. (1997). “Multimodality Image Registration by Maximization of Mutual Information”. In: IEEE Transactions on Medical Imaging 16.2, pp. 187–198. doi: 10.1109/42.563664.DOI
  60. [68]Mardia, K. V. and P. E. Jupp (1999). Directional Statistics. Wiley Series in Probability and Statistics. Chichester: John Wiley & Sons. doi: 10.1002/9780470316979.DOI
  61. [69]Novikov, Y. et al. (2024). “Vehicle Geolocalization from Drone Imagery”. In: ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences X-2-2024, pp. 171–178. doi: 10.5194/isprs-annals-X-2-2024-171-2024.
  62. [70]Pluim, J. P. W., J. B. A. Maintz, and M. A. Viergever (2000). “Image Registration by Maximization of Combined Mutual Information and Gradient Information”. In: IEEE Transactions on Medical Imaging 19.8, pp. 809–814. doi: 10.1109/42.876307.DOI
  63. [71]Potje, G. et al. (2024). “XFeat: Accelerated Features for Lightweight Image Matching”. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2682–2691. doi: 10.1109/CVPR52733.2024.00259.DOI
  64. [72]Punzo, V., M. T. Borzacchiello, and B. Ciuffo (2011). “On the assessment of vehicle trajectory data accuracy and application to the Next Generation SIMulation (NGSIM) program data”. In: Transportation Research Part C: Emerging Technologies 19.6, pp. 1243–1262. doi: 10.1016/j.trc.2010.12.007.DOI
  65. [73]Raguram, R. et al. (2013). “USAC: A Universal Framework for Random Sample Consensus”. In: IEEE Transactions on Pattern Analysis and Machine Intelligence 35.8, pp. 2022–2038. doi: 10.1109/TPAMI.2012.257.DOI
  66. [74]Ravi, N. et al. (2025). “SAM 2: Segment Anything in Images and Videos”. In: International Conference on Learning Representations (ICLR). arXiv: 2408.00714 [cs.CV]. URL: https://openreview.net/forum?id=Ha6RTeWMdO.arxiv.org/abs/2408.00714
  67. [75]Reinartz, P. et al. (2006). “Traffic monitoring with serial images from airborne cameras”. In: ISPRS Journal of Photogrammetry and Remote Sensing 61.3–4, pp. 149–158. doi: 10.1016/j.isprsjprs.2006.09.009.DOI
  68. [76]Riba, E. et al. (2020). “Kornia: an Open Source Differentiable Computer Vision Library for PyTorch”. In: IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 3663–3672. doi: 10.1109/WACV45572.2020.9093363.DOI
  69. [77]Rousseeuw, P. J. (1984). “Least Median of Squares Regression”. In: Journal of the American Statistical Association 79.388, pp. 871–880. doi: 10.1080/01621459.1984.10477105.DOI
  70. [78]Rublee, E. et al. (2011). “ORB: An efficient alternative to SIFT or SURF”. In: International Conference on Computer Vision (ICCV), pp. 2564–2571. doi: 10.1109/ICCV.2011.6126544.DOI
  71. [79]Sanz-Ablanedo, E. et al. (2018). “Accuracy of Unmanned Aerial Vehicle (UAV) and SfM Photogrammetry Survey as a Function of the Number and Location of Ground Control Points Used”. In: Remote Sensing 10.10, p. 1606. doi: 10.3390/rs10101606.
  72. [81]Soille, P. (2004). Morphological Image Analysis: Principles and Applications. 2nd ed. Berlin, Heidelberg: Springer. doi: 10.1007/978-3-662-05088-0.DOI
  73. [82]Stewart, C. V. (1997). “Bias in Robust Estimation Caused by Discontinuities and Multiple Structures”. In: IEEE Transactions on Pattern Analysis and Machine Intelligence 19.8, pp. 818–833. doi: 10.1109/34.608280.DOI
  74. [83]Sun, J. et al. (2021). “LoFTR: Detector-Free Local Feature Matching with Transformers”. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8918–8927. doi: 10.1109/CVPR46437.2021.00881.DOI
  75. [87]Tyszkiewicz, M., P. Fua, and E. Trulls (2020). “DISK: Learning local features with policy gradient”. In: Advances in Neural Information Processing Systems (NeurIPS). Vol. 33, pp. 14254–14265.arxiv.org/abs/2006.13566
  76. [88]Viola, P. and W. M. Wells III (1997). “Alignment by Maximization of Mutual Information”. In: International Journal of Computer Vision 24.2, pp. 137–154. doi: 10.1023/A:1007958904918.DOI
  77. [89]Wang, Z. et al. (2004). “Image Quality Assessment: From Error Visibility to Structural Similarity”. In: IEEE Transactions on Image Processing 13.4, pp. 600–612. doi: 10.1109/TIP.2003.819861.DOI
  78. [97]Zhao, M. and Q. Ling (2019). “A Robust Traffic Video Stabilization Method Assisted by Foreground Feature Trajectories”. In: IEEE Access 7, pp. 42921–42933. DOI: 10.1109/ACCESS.2019.2908186.
  79. [98]Zheng, O. et al. (2024). “CitySim: A Drone-Based Vehicle Trajectory Dataset for Safety-Oriented Research and Digital Twins”. In: Transportation Research Record: Journal of the Transportation Research Board 2678.4, pp. 606–621. DOI: 10.1177/03611981231185768.
  80. [100]Zhu, P. et al. (2022). “Detection and Tracking Meet Drones Challenge”. In: IEEE Transactions on Pattern Analysis and Machine Intelligence 44.11, pp. 7380–7399. DOI: 10.1109/TPAMI.2021.3119563.DOI

Paper details

vlnc:W_vcsmpzsw
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents