WorksComputer Science
ALoMa: Affine Correspondences Beyond Patches
Abstract
Affine correspondences (ACs) enable robust and efficient solvers for multiple-view geometry by constraining the local geometry of point correspondences (PCs). Despite their utility in minimal solvers, AC estimation remains understudied. We revisit AC estimation beyond local image patches, extending it to the image and image-pair levels. Rather than estimating the components of the affine transformation separately, we learn the full 4-DoF matrix jointly. Building on this formulation, we introduce ALoMa, an AC estimator using LoMa keypoint descriptions, demonstrating consistent improvements across a wide range of downstream tasks. Code at https://github.com/davnords/loma.
Introduction
Geometric constraints are fundamental to multiple-view geometry [48], with image correspondences providing the basis for many of its core estimation problems. Affine correspondences (ACs) enrich point correspondences (PCs) with the local affine transformation between the corresponding image regions, providing four additional degrees of freedom (DoF) per match. This additional information enables minimal solvers with far fewer correspondences: for example, relative pose estimation requires 5 PCs [67], but only 2 ACs [15]. ACs have similarly been exploited for surface-normal estimation [47], homography estimation [14], epipolar geometry [12, 15, 50, 70, 71], absolute pose [54, 89], and focal length estimation [19, 88].
Despite these advances in using ACs, estimating ACs themselves has received little attention. Traditional approaches rely on hand-crafted affine region detectors, such as Harris-Affine [62] and Hessian-Affine [63], followed by iterative refinement such as Baumberg iteration [23]. Learned approaches later improved this pipeline, most notably AffNet [66]. However, since its publication in 2018, progress in AC estimation has been limited, while PC estimation has seen many innovations.
Modern PC estimators reason over global context within and across views, rather than treating correspondences as independent local patches. Detector-free methods such as DKM [33] and RoMa [36, 38], as well as learned sparse matchers such as SuperGlue [75] and LoMa [68], have substantially improved the quality and robustness of PCs. Yet AC estimation has largely remained disconnected from this progress. To our knowledge, Sun et al. [81] is the only approach that combines a modern matcher with learned affine estimation. However, the affine transformation is still predicted from a small local patch around each match, and its four DoF are not predicted. Instead, one network estimates scale and orientation, and a separate one the affine shape.
This raises a natural question: can AC estimation benefit from the same contextual information that has improved PC estimation? Existing approaches reduce each correspondence to a local patch, discarding the image-wide and cross-view information already available in modern matchers. Moreover, the four components of an AC are coupled, making their joint prediction challenging from local appearance alone. Indeed, previous work has reported difficulties with jointly estimating these components [66, 81], motivating decomposed predictions and multi-stage estimation.
In this work, we revisit AC estimation from a contextual perspective. Instead of treating each correspondence in isolation, we progressively incorporate image-level and cross-view information, and find that expanding the context from a patch to the full image pair sharply reduces affine error. Building on this, we augment a modern matcher with a lightweight affine head that directly predicts the full 4-DoF affine transformation. Joint prediction of scale, orientation, and shape then becomes both feasible and accurate.
Our resulting family of models, A·LoMa, advances the state of the art in AC estimation with a simpler formulation. As shown in Figure 1, it reduces affine estimation error by an order of magnitude compared with AffNet and Sun et al., while improving downstream geometric estimation.
Figure 1. Accurate estimation of ACs. (a) A circle at a keypoint in image 1 is mapped into image 2 by the AC predicted by KeyNet+HardNet+AffNet (AffNet), Sun et al. [81], and A·LoMa-B (solid), versus ground truth (dashed green). (b) Quantitative performance shows A·LoMa-B reduces affine error and improves accuracy on downstream tasks like essential matrix estimation from 2 ACs.
The main contributions are as follows:
Context matters for affine estimation. We study the effect of context on AC prediction and show that image-level and cross-view information substantially reduce affine estimation error (Figure 4).
Joint prediction of scale, orientation, and affine shape is feasible. We show that a context-aware model can directly predict all four affine DoF jointly, avoiding the decomposed and multi-stage estimation used by prior approaches [66, 81].
An order of magnitude lower affine error. A·LoMa, trained directly with ground-truth (GT) affine transformations, achieves an order of magnitude lower affine error than AffNet and Sun et al., as visualized in Figure 1, while improving downstream geometric tasks, e.g., achieving higher rolling-shutter pose accuracy with 7 ACs than with 20 PCs.
Figure 4. Affine estimation benefits from global context. A CNN on the raw image patch around each keypoint (0), trained with our loss and setup in the style of AffNet, is the weakest. Training a head on the keypoint descriptions (I) is only slightly better and comparable to AffNet. As the attention context is extended, first to the same image (II) and then to the other image (III), the error on MegaDepth-1500 is roughly halved at each step. Reading the matcher’s features instead of the descriptor’s (IV) reduces it further.
Background
Affine Correspondences
An AC consists of a pair of matching keypoints together with the local affine transformation mapping the neighborhood of one keypoint onto the neighborhood of the other. Compared to a single PC, an AC provides additional geometric constraints [24, 70] on the camera poses. As a consequence, minimal solvers require only 2 ACs (vs. 5 PCs) to estimate the essential matrix [15, 20], 3 ACs (vs. 7 PCs) for the fundamental matrix [20, 71], and 2 ACs (vs. 4 PCs) for homographies [13, 16], which reduces to a single AC once the epipolar geometry is known [14]. This has motivated their adoption in minimal solvers across many tasks, such as multi-camera and generalized motion [4, 45, 98], absolute pose estimation [88, 89, 90], relative pose estimation [12, 17, 40, 44, 50, 102], planar and vehicle-mounted motion [46, 87], focal length estimation [19, 103], and surface normal estimation [47]. More broadly, ACs are one of several ways to shrink the minimal sample. Alternatives include a known gravity direction [55, 82, 88] and monocular or relative depth [8, 32, 50]. ACs have thus been incorporated into a wide range of minimal solvers. However, estimating them has received far less attention.
Traditionally, ACs were obtained from hand-crafted affine-covariant region detectors [41, 62, 63, 64]. AffNet [66] proposed a learning-based method for estimating the affine shape and remains widely used almost a decade later. The network predicts an affine shape for each keypoint by decomposing the affine transformation as , where is a unit-determinant upper-triangular shape matrix. An affine transformation A has four DoF: two from the shape (), one from the scale (), and one from the orientation (). AffNet acts on local image patches and is trained by optimizing the matching discriminability, applying synthetic affine perturbations with increasing difficulty through a curriculum. Practically, to estimate ACs using AffNet one combines its affine shape estimation with scale and orientation from separate networks. S3Esti [99] is one such network, estimating only the affine scale and orientation of a keypoint, trained using a self-supervised probabilistic covariant loss that treats GT labels as latent variables. Similar to AffNet, S3Esti uses only local patches around its keypoint for its affine estimate. Dai et al. [29] also uses local patch crops but does so around a pair of keypoints rather than a frame per keypoint.
Sun et al. [81] combine AffNet for shape (2 DoF) and S3Esti for scale and orientation (2 DoF), two separate CNNs, with the dense matcher DKM for extracting PCs. For supervision they use the Sampson distance of the affine epipolar constraint. The patches cropped around the keypoints are the only source of information for the affine transformation, while a finetuned DKM is used for PCs.
Three common themes in prior work on learned AC estimation are reliance on local patch information, indirect supervision, and separate networks for estimating scale, orientation, and shape. We instead adopt a simpler, more effective formulation that uses global information, direct supervision, and estimates all degrees of freedom jointly.
Point Correspondences
For a long time, estimating PCs was closely tied to AC estimation. SIFT [60] descriptors, for instance, are computed on patches normalized for orientation and scale, so this local geometry has to be estimated as part of the algorithm. As learning-based approaches overtook PC estimation, the dominant idea instead became that affine invariance should be learned from data. SuperPoint [31] proposed a self-supervised protocol to learn to detect and describe keypoints, and SuperGlue [75] replaced the mutual-nearest-neighbor matcher with a graph neural network. Detector-free matching was popularized by works such as LoFTR [80], DKM [33], and RoMa [36, 38], while further advances have been made for keypoint detection [22, 34, 37, 53, 91], description [25, 35, 43, 83], joint detection and description [72, 86, 93, 104, 105], and matching [58, 68, 69]. We build on the recent sparse matcher LoMa [68] and extend it from PCs to ACs using its keypoint descriptors.
Method
We fix notation (§ 3.1), then detail AC prediction (§ 3.2), the architecture (§ 3.3), and the learning objective (§ 3.4). See Figure 2 for an overview of A-LoMa.
Figure 2. Architecture. A-LoMa inputs an RGB image pair and uses LoMa to obtain PCs together with keypoint descriptors. The keypoint descriptors are fed into an affine head, implemented as a transformer. For each keypoint, a local affine frame is predicted. The frames of each match are composed into an affine transformation , which is supervised directly by the GT derived from depth maps.
Notation
By and we denote a pair of corresponding keypoints in images and . The two images are locally related by a warp between their image domains, i.e., , induced by the 3D scene and generally known only implicitly, e.g., through depth maps. The affine transformation of an AC (§ 2) is then the local behavior of around , given by its Jacobian ,
for a small displacement , i.e., describes how moves under a small displacement of .
Predicting Affine Correspondences
Our goal is a model that estimates ACs. This decomposes into finding matching points and predicting their local affine matrices. We obtain PCs from the frozen LoMa matcher. To obtain a local affine matrix for each detected keypoint, we append an affine head (§ 3.3) to LoMa. In particular, we predict for and , respectively. Each can be read as a local frame mapping a shared canonical patch onto the neighborhood of its keypoint, so composing them maps the neighborhood of onto that of .
Given a PC , the affine transformation of the AC is obtained by composing the two per-keypoint matrices,
which we compare to the GT of Eq. (1) during training (see § 3.4). While could also be regressed directly per match, per-keypoint frames make the estimate inverse-consistent by construction, i.e. the reverse affine from to is exactly , as the model is permutation equivariant.
Note that Eq. (2) only constrains the composition of the two per-keypoint predictions: for any invertible , replacing and leaves , and hence the loss, unchanged. Training therefore never pins down itself, only its relation to that of a matched partner. We study this gauge freedom empirically in § 5.2.
Each per-keypoint matrix must be parametrized to guarantee invertibility, as required by Eq. (2). Following prior work [66, 81], we decompose into a scale, an orientation, and a shape,
where is an isotropic scale (so that ), a planar rotation, and a residual shape with . We evaluate a simpler parametrization based on LU decomposition in § 5.1 and find that it performs almost as well, indicating that our approach is not highly sensitive to the parametrization.
Affine Head Architecture
For every detected keypoint, the affine head takes the output descriptors of LoMa, and passes them through transformer layers of alternating self- and cross-attention. A per-keypoint MLP then regresses five numbers for the four DoF of Eq. (3): scale (1), orientation (2, a direction vector normalized to unit length, which avoids the wraparound of regressing an angle directly), and shape (2, the stretch and the shear ), to compose . To keep , and hence its inverse in Eq. (2), well conditioned, the scale and the stretch are bounded with fixed , while the shear is regressed directly. The final layer of the prediction head is zero-initialized, so every keypoint starts with an identity affine frame. Because the head attends across views, the per-keypoint affine frames are conditioned on the partner image. We analyze this in § 5.2 and propose in the Suppl. (Sec. F) a method to obtain stable per-view affine frames, useful for e.g. Structure-from-Motion (SfM).
Supervision
For each training pair we have a GT projective (-axis) depth map , camera intrinsics , and relative pose , which make the warp of Eq. (1) explicit,
where is in homogeneous coordinates and denotes the perspective division. From we derive the set of GT matches and affine transformations . Rather than estimating a surface normal and differentiating the homography it induces, we differentiate the warp itself, estimating by sampling on a pixel grid of step and taking forward differences,
with the standard basis of and columns indexing the partial derivatives. We use px and express the result in normalized image coordinates (see Suppl., Sec. A).
The GT matches are the mutual nearest neighbors of the depth-induced warp , restricted to matches whose local Jacobian is well defined, i.e., away from occlusion boundaries. We supervise the predicted AC of every directly against the GT affine ,
For we use the robust Charbonnier loss, a special case of the general robust loss of Barron [21],
with smoothing constant . We show in § 5.1 that the plain squared Frobenius loss leads to worse results.
Results
We compare to state-of-the-art techniques for AC estimation on a wide range of benchmarks. We evaluate the accuracy of the ACs directly (§ 4.1), and then their utility inside minimal solvers for relative pose (§ 4.2), visual localization (§ 4.3), and recovering surface normals (§ 4.4). Throughout this section, the best, second, and third ranked results are highlighted. Additional details are given in the Suppl. (Sec. C).
Implementation details. We train A·LoMa for 100K steps using the AdamW [59] optimizer with a fixed learning rate of and a global batch size of 64, similar to LoMa. Training on H200 GPUs at a resolution of takes roughly a day. Varying the LoMa backbone, we release three models, A·LoMa-{B, L, G}, of increasing embedding dimension. The training data, detailed in the Suppl. (Sec. B), is a large mix of 3D datasets with GT depth, camera intrinsics, and poses.
Baselines. We compare to estimators of sparse ACs:
KN+HN+AN: KeyNet [22] keypoints, HardNet [65] descriptors, and affine shape from AffNet [66].
SIFT+HN+S3E: SIFT [60] keypoints, HardNet descriptors, and scale and orientation from S3Esti [99], assuming identity shape, i.e. with in Eq. (3).
Sun et al. [81]:1 Cropping patches around sparse DKM [33] PCs and estimating one affine frame per patch.
LoMa+S3E+AN: LoMa PCs with scale and orientation from S3Esti and shape from AffNet.
Sparse ACs can also be obtained by differentiating a dense matcher’s warp directly (Eq. (5)). We include such ACs from RoMa v2 [38] as a reference in Table 1 and Table 6.
| Method | HPatches | HPatches | Mega1500 | Mega1500 |
| Frobenius | Rot. | Frobenius | Rot. | |
| Differentiated dense warp, at LoMa keypoints | ||||
| RoMa v2 | 2.82 | 0.46° | 5.04 | 0.75° |
| KN+HN+AN | 26.9 | 6.35° | 32.7 | 7.17° |
| SIFT+HN+S3E | 54.4 | 9.52° | 49.3 | 5.50° |
| Sun et al. | 48.2 | 3.82° | 40.1 | 2.09° |
| LoMa+S3E+AN | 51.9 | 5.06° | 42.9 | 2.39° |
| A·LoMa-B | 1.62 | 0.26° | 3.97 | 0.58° |
| A·LoMa-L | 1.59 | 0.23° | 3.78 | 0.55° |
| A·LoMa-G | 1.50 | 0.23° | 3.61 | 0.53° |
Table 1. Accuracy of affine transformations. We report median Frobenius () and rotational errors (lower is better).
Table 6. Normal estimation from ACs. We report area under the curve (AUC), higher is better.
| Method | Map-free [7] 5° | Map-free [7] 10° | Map-free [7] 20° | Mega1500 5° | Mega1500 10° | Mega1500 20° |
| Differentiated dense warp, at LoMa keypoints | ||||||
| RoMa v2 | 5.9 | 14.1 | 27.8 | 4.3 | 12.7 | 29.2 |
| KN+HN+AN | 0.3 | 1.3 | 4.8 | 0.3 | 1.3 | 4.4 |
| SIFT+HN+S3E | 0.2 | 0.7 | 2.8 | 0.1 | 0.5 | 1.9 |
| Sun et al. | 0.1 | 0.5 | 1.9 | 0.2 | 0.9 | 3.3 |
| LoMa+S3E+AN | 0.2 | 0.6 | 2.4 | 0.2 | 0.7 | 2.8 |
| A·LoMa-B | 9.9 | 19.2 | 33.3 | 4.3 | 12.9 | 30.4 |
| A·LoMa-L | 9.7 | 19.2 | 33.5 | 4.4 | 13.6 | 32.0 |
| A·LoMa-G | 9.8 | 19.3 | 33.6 | 5.0 | 14.7 | 33.5 |
Table 6.
Affine Accuracy
We begin by evaluating the accuracy of the predicted ACs directly, similar to Sun et al. [81]. Figure 3 provides a qualitative comparison. GT affines are obtained differently on the two datasets. On HPatches [10] we use the analytic Jacobian of the known homography, which involves neither the depth nor the finite differences of § 3.4, whereas on MegaDepth [57] we differentiate the depth-induced warp as during training. Following Sun et al., we restrict evaluation to matches whose predicted point already agrees with the GT warp (within 3 pixels at a canonical 480-px scale) and whose GT affine is well defined (positive determinant), isolating the affine estimate itself. We additionally report scores that depend on the PCs in the Suppl. (Sec. D.2). For every kept match, we compare to the GT using the Frobenius error and a residual rotation (i.e. the angle of the closest planar rotation to ). We report the Frobenius error multiplied by 100 throughout, which is easier to read. Table 1 reports the median performance on HPatches and MegaDepth-1500 (1500 outdoor pairs from the LoFTR split [80]). A·LoMa outperforms all sparse baselines by an order of magnitude in Frobenius error, and is also more accurate than ACs obtained by differentiating RoMa v2’s dense warp (gray row).
Figure 3. Local affine transformation. For three correspondences, we warp the image patch around the keypoint (cross) with the predicted affine and compare it with the matched patch in the other image. Accurate estimates yield similar warped and matched patches.
Relative Pose from Minimal AC Solvers
We now use the predicted ACs inside minimal solvers for relative pose, under three increasingly general camera models. Each setting substitutes ACs for the PCs a standard solver would otherwise consume, shrinking the minimal sample to 2, 7, and 9 ACs, respectively.
Calibrated two-view (2AC). We consider relative pose estimation between two calibrated cameras through the estimation of the essential matrix [48]. Following Sun et al., we use GC-RANSAC [18] with the 2AC solver [15]. We evaluate on MegaDepth-1500 and ScanNet-1500 [28] (1500 indoor pairs from Sarlin et al. [75]). A·LoMa outperforms existing AC estimators, particularly on ScanNet, where it exceeds Sun et al., who only provide outdoor weights, by 25.5 AUC@. Most of this gap comes from LoMa’s PCs, as LoMa+S3E+AN shows. On the same matches, our affine head gives a consistent but smaller gain.
Rolling shutter (7AC). Rolling-shutter cameras break the single-center projection model and require their own solvers, for both absolute [1, 2, 3, 76] and relative [30, 49] pose. Following Barath [12], we evaluate rolling shutter relative pose estimation accuracy on TUM-RS [77] using stride 20, 5K iterations of SupeRANSAC [11] with MSAC scoring and PROSAC sampling [27], and 2048 ACs. As shown in Table 3, moving from 20 PCs to 7 ACs improves accuracy for every estimator, and A·LoMa-B gives the best result overall (71.6 AUC@). Sun et al. remain competitive here, driven by accuracy of DKM PCs.
| Solver | Method | @5° | @10° | @20° |
| 20PC | KN+HN | 3.0 | 5.1 | 7.8 |
| 20PC | SIFT+HN | 11.1 | 14.6 | 17.2 |
| 20PC | DKM | 41.2 | 47.5 | 52.0 |
| 20PC | LoMa-B | 22.6 | 30.7 | 36.7 |
| 7AC | KN+HN+AN | 36.9 | 46.2 | 53.3 |
| 7AC | SIFT+HN+S3E | 37.6 | 44.4 | 49.6 |
| 7AC | Sun et al. | 60.0 | 65.5 | 69.0 |
| 7AC | LoMa+S3E+AN | 58.4 | 66.3 | 70.9 |
| 7AC | A·LoMa-B | 60.1 | 67.1 | 71.6 |
Table 3. Rolling shutter pose estimation from PCs vs. ACs. We report pose AUC (%) at on TUM-RS [77] comparing the RS-20PC [30] solver and the RS-7AC [12] solver.
Multi-camera rigs (9AC). Following Xu et al. [98],2 we consider the problem of registering two view-graphs of four posed frames each with a generalized relative-pose-and-scale solver. We evaluate on TUM RGB-D [79] and compare the AUC of a 26PC solver against a 9AC solver in Table 4. For a given matcher, both solvers receive identical matches, while the AC solver additionally receives the predicted affine of each match. A·LoMa-B improves accuracy over the point baseline while requiring approximately fewer RANSAC iterations. AffNet, in contrast, needs more iterations with 9 ACs than with 26 PCs.
| Solver | Method | Iters. | No cap | 100 it. |
| 26PC | KN+HN | 930 | 75.7 | 61.2 |
| 26PC | SIFT+HN | 911 | 76.2 | 61.2 |
| 26PC | DKM | 303 | 73.3 | 72.1 |
| 26PC | LoMa-B | 1009 | 78.7 | 64.7 |
| 9AC | KN+HN+AN | 2288 | 56.3 | 19.3 |
| 9AC | SIFT+HN+S3E | 329 | 70.6 | 53.3 |
| 9AC | Sun et al. | 263 | 70.1 | 64.8 |
| 9AC | LoMa+S3E+AN | 292 | 72.3 | 57.1 |
| 9AC | A·LoMa-B | 73 | 79.4 | 76.7 |
Table 4. Generalized relative pose and scale from PCs vs. ACs. We report AUC (%) at 0.25 m/2° on TUM RGB-D, with RANSAC either capped at 100 iterations (100 it.) or uncapped (No cap). Iters. is the mean number of RANSAC iterations when uncapped.
Visual Localization
We evaluate our affine estimates inside the absolute-pose pipeline of Valtonen Örnhag et al. [88], substituting only the predicted local frames and leaving the reconstruction, detector (SuperPoint [31]), matcher (LightGlue [58]), retrieval, and RANSAC configuration untouched. We compare three minimal solvers: UP1SIFT [90], using scale and orientation and assuming a known focal length, UP1PfAC [88], using the full affine frame and also recovering the focal length, and PfAC [89], using the full affine frame with a known focal length. We report results on Cambridge Landmarks [52] in Table 5. We disable local optimization (LO). With LO enabled, the gains for UP1PfAC and PfAC become negligible beyond about 100 iterations and remain at small budgets, while those for UP1SIFT persist (Suppl., Tab. 10 and Figure 6). Replacing S3Esti’s estimates with ours slightly improves accuracy for all solvers.
Table 5. Absolute pose on Cambridge Landmarks. We report the mean over the five scenes, with 95% confidence intervals over 100 RANSAC seeds.
| Solver | Method | 25 cm, 2° | 50 cm, 5° |
| P4Pf | Points only | 41.4 ±0.07 | 63.3 ±0.05 |
| UP1SIFT | S3Esti | 44.8 ±0.17 | 59.6 ±0.16 |
| A·LoMa-B | 49.9 ±0.15 | 65.5 ±0.14 | |
| UP1PfAC | S3Esti+AffNet | 39.5 ±0.15 | 61.5 ±0.13 |
| A·LoMa-B | 40.6 ±0.14 | 62.1 ±0.12 | |
| P1AC | S3Esti | 67.0 ±0.12 | 84.7 ±0.09 |
| S3Esti+AffNet | 67.1 ±0.12 | 84.6 ±0.11 | |
| A·LoMa-B | 67.4 ±0.12 | 84.8 ±0.09 |
Table 5.
Figure 6. Accuracy against the RANSAC budget. Cambridge Landmarks queries localized within 25 cm, 2° (mean over the five scenes, 95% confidence bands) against the maximum number of GC-RANSAC iterations, with LO disabled (left) and enabled (right).
Normal Estimation
Next, we evaluate the accuracy of our ACs through their utility for surface normal estimation. We recover the surface normal from a single AC and the known relative pose based on the optimal solver of Hajder et al. [47]. We keep only correspondences consistent with the known relative pose (Sampson distance px), decoupling normal accuracy from matching precision. We compare the recovered normal to a reference normal obtained by fitting a plane to the GT depth in a small window around the keypoint, using the sign-invariant angle between the two unit normals as the error. Table 6 reports the area under this per-correspondence angular-error curve (AUC). We find that A·LoMa outperforms the dedicated AC estimators, which fail on this benchmark. A·LoMa-L and A·LoMa-G also outperform normals recovered from RoMa v2’s dense warp.
Analysis
In this section, we study our design choices (§ 5.1) and the gauge freedom induced by our supervision (§ 5.2).
Design Choices
Local versus global information. To our knowledge, all prior dedicated AC estimators use local patches. In Figure 4, we increase the context from local image patches (0), to local keypoint descriptors, to global (II), to cross-view (III), each step of increased context roughly halving the affine error on MegaDepth-1500. Moving from the descriptor features to the matcher (IV) reduces it slightly further. Although (I), (II), and (III) have the same input descriptors, the additional attention context substantially decreases the error. Interestingly, both (0), using local patch information, and (I), using isolated keypoint descriptors, manage to achieve AffNet-level performance through direct supervision and jointly predicting scale, shape, and orientation.
Parametrization. AffNet and Sun et al. both conclude that the parametrization of A is critical. While we find that using scale, shape, and orientation works well, in Table 7, we also try a simpler L U decomposition,
Table 7. Supervision and parametrization. Median affine error (Frobenius, ×100) on MegaDepth-1500 for each combination of loss (Eq. (6)) and parametrization of the per-keypoint frames, with . Our default is shaded.
Table 7.
where with regressed directly (unbounded). It can only represent frames whose top-left entry is positive, so, for example, it cannot express a single frame rotated by more than 90°. Eq. (3) has no such limit. Nonetheless, it performs comparably despite requiring no trigonometric parametrization, suggesting that our formulation is less sensitive to parametrization than prior work.
Supervision. In Table 7, we compare using the squared Frobenius loss as in Eq. (6) to using the robust Charbonnier loss of § 3.4, which for our small smoothing constant is essentially the unsquared Frobenius norm. We find that the latter more than halves the affine error.
Architecture. In Table 8, we study the effect of changing the architecture. We begin by jointly training matching and affine estimation from scratch (A), which leads to strong results. Initializing from LoMa and then fine-tuning with matching and affine estimation supervision jointly (B) does not help. While freezing the matcher (C) slightly improves matching accuracy, the linear head is insufficient. However, increasing the capacity of the affine head to a shallow transformer (E) leads to both strong matching and strong affine estimation performance. D slightly outperforms E on affine error but requires training the backbone, slightly deteriorating matching performance. We adopt E, which is ∼4× cheaper to train and leaves the matcher untouched.
| Initialization | Matcher | Affine head | Frob. ↓ | AUC ↑ | |
| A | Scratch | 🔥 | Linear | 4.6 | 83.2 |
| B | LoMa | 🔥 | Linear | 4.8 | 83.2 |
| C | LoMa | ❄ | Linear | 12.2 | 83.6 |
| D | Scratch | 🔥 | Transformer | 3.8 | 83.2 |
| E | LoMa | ❄ | Transformer | 4.0 | 83.6 |
Table 8. Architecture. Affine error (Frobenius, ×100) and relative pose AUC@20° (5 PCs) on MegaDepth-1500. 🔥 and ❄ denote a frozen and a trained matcher. A trained matcher also receives LoMa’s matching loss. Our default is shaded.
Orientation. Estimating the full affine frame includes orientation, which prior local methods predict with a separate network. Training and evaluating on independently rotated pairs, we find that a frozen rotation-invariant matcher (LoMa-R [69]) cannot support orientation estimation, since its features discard exactly the quantity to be predicted, whereas unfreezing it recovers accurate orientation and good matches under rotation (see Sec. D.5).
Gauge Freedom
Our approach supervises the ACs directly, never the keypoint local frames . As noted in § 3.2, this leaves a “gauge freedom” in the local predictions: for any invertible , which may depend on both keypoints and both images. To illustrate the gauge freedom, we show in the Supplementary (Figure 5(b)) how the prediction of varies as we change the partner image . We quantify this by the residual gauge between the frames predicted for the same keypoint under two different partner images and , which is the identity if and only if the two agree. As a rough baseline, we consider that even for a fully resolved gauge the estimates would differ by to for estimation error (to first order, using the error of Table 1 as a proxy). The measured median on the HPatches sequences we evaluate is around ten times larger, indicating that gauge freedom is unresolved.
Figure 5. Gauge freedom. (a) Right-multiplying both frames of a match by a common G leaves and the loss unchanged. (b) Predicted frames of four keypoints of one image under five partner images (colors). The dashed circle is the identity. (c) Median (×100) per HPatches sequence. The band is the range expected for stable frames.
Limitations and Future Work
While the additional context from the image pair improves performance, it couples the local affine frame to the paired image. Although stable per-view affine frames can be obtained when desired, such as in SfM, by averaging the estimates across pairs (see Sec. F), this affects performance. Studying this and the gauge freedom from our supervision constitutes interesting future work.
We use LoMa’s PCs rather than those of RoMa v2, which are often more accurate. Although this yields substantial runtime improvements (Sec. D.1), leveraging dense matchers and feed-forward reconstruction models [94] for AC estimation constitutes interesting future work.
With LO and large iteration budgets, the gains from accurate ACs are modest (see § 4.3). We find ACs to be most beneficial in solvers requiring a large number of PCs, such as rolling-shutter relative pose (Table 3). Identifying problems where ACs are vastly superior to PCs constitutes interesting future work.
Conclusions
We revisit AC estimation and go beyond local patches. Our formulation allows a simpler pipeline that directly regresses the full 4-DoF affine matrix. Systematic ablations show that global context is crucial for affine estimation. Our method, A-LoMa, adds a lightweight affine head to LoMa’s matcher, achieving an order of magnitude lower affine error than AffNet and Sun et al., with consistent downstream gains.
LLM Usage Statement
We used Claude by Anthropic through Claude Code for:
Generating code for the experiments in § 4.
Source code was read by the authors and evaluations were based on previous work.
The core implementation of the model and loss was done without the assistance of LLMs.
Checking for typos.
Assist with editing the manuscript for clarity and writing quality.
Identifying relevant literature (checked by the authors).
Generating the TikZ and Matplotlib code for the figures.
We did not use LLMs for:
Helping develop theoretical models or conceptual frameworks.
Suggesting presentation structure or style.
Designing or providing feedback on research methodology or experiments.
Proposing or refining hypotheses.
Assisting in natural-language translation.
Interpreting results.
Drafting any part of the paper. The authors wrote all sections of the paper and only then received assistance from the LLM to refine it.
Formatting references.
We have reviewed all LLM-assisted work. We take responsibility for the final content of this work, including text, claims or artifacts produced with the aid of generative AI.
Acknowledgements
This work was supported by the Wallenberg Artificial Intelligence, Autonomous Systems and Software Program (WASP), funded by the Knut and Alice Wallenberg Foundation, by the strategic research environment ELLIIT, funded by the Swedish government, and by the National Science Foundation under Award No. 2144822. The computational resources were provided by the National Academic Infrastructure for Supercomputing in Sweden (NAISS) at C3SE, partially funded by the Swedish Research Council through grant agreement no. 2022-06725, and by the Berzelius resource, provided by the Knut and Alice Wallenberg Foundation at the National Supercomputer Centre.
Implementation Details
Architecture. A·LoMa appends an affine head to a frozen LoMa [68] matching pipeline: DaD keypoints [37] (2048 per image), the LoMa descriptor network, and a LightGlue-style [58] matcher of nine alternating self- and cross-attention layers with four heads. A·LoMa-{B, L, G} differ only in the embedding dimension of the backbone (256, 512, 1024). The head applies further self- and cross-attention layers to the final matcher features, using the same keypoint positional encoding as the matcher in self-attention, followed by a per-keypoint MLP shared across both images (one hidden layer of the embedding width, GELU) that outputs five scalars parameterizing the four DoF of Eq. (3) (log-scale and shape exponent bounded by with ). The last layer is zero-initialized, so every keypoint starts at .
Supervision. The GT affine of a match is the Jacobian of the depth-induced warp, computed by forward differences (Eq. (5)) on the dense warp at the training resolution in normalized coordinates, i.e., with the pixel Jacobian and . A Jacobian is kept only if the pixel and its forward neighbors are covisible (the projection agrees with the depth to within 5%), which rejects differences across occlusion boundaries, and rejected entries are masked out of the loss. Matches are the mutual nearest neighbors of the forward and backward warps at the keypoints. The loss of Eq. (6) is applied to the composed relative affine of every match. The per-keypoint frames are never supervised individually, which is the origin of the gauge freedom studied in § 5.2.
Training Data
We train on the mix of Table 9. Each sample draws a dataset with probability proportional to the weight in Table 9, and then a pair by covisibility following standard matching sampling like in LoMa and RoMa. Images are resized to and the intrinsics rescaled accordingly. None of the evaluation scenes are in the training data: MegaDepth scenes 0015 and 0022 are excluded from all MegaDepth-derived sets, the Map-free scenes used for normal estimation (s00015, s00022) are excluded from Map-free, and HPatches, ScanNet-1500, Cambridge Landmarks, TUM RGB-D and TUM-RS are not used for training.
| Datasets | Type / GT Source | Weight |
| AerialMVS [92] | Aerial / MVS | 10 000 |
| ASE [9] | Indoor / Graphics | 1000 |
| BlendedMVS [100] | Aerial / Mesh | 10 000 |
| FlyingThings3D [61] | Synthetic / Graphics | 5000 |
| Hypersim [73] | Indoor / Graphics | 10 000 |
| Map-free [7] | Object-centric / MVS | 10 000 |
| MegaDepth [57] | Outdoor / MVS | 10 000 |
| MegaDepth-X [56] | Outdoor / MVS | 10 000 |
| MegaScenes [85] | Outdoor / MVS | 10 000 |
| MegaSynth [51] | Indoor / Graphics | 1000 |
| MPSD [5] | Driving / MVS | 1000 |
| ScanNet++ v2 [101] | Indoor / Mesh | 10 000 |
| SpatialV ID [95] | Forward-motion / MVS | 100 |
| TartanAir V2 [97] | Synthetic / Graphics | 10 000 |
| UnrealStereo4k [84] | Outdoor / Graphics | 100 |
| Virtual KITTI 2 [26, 42] | Outdoor / Graphics | 100 |
Table 9. Training data. We use a mix of indoor, outdoor and aerial training data, similar to [38, 68, 94, 96].
Evaluation Protocols
Reproducing Sun et al. We evaluate Sun et al. [81] with their public code and pretrained outdoor weights. The released demo had four implementation issues, which we fix in our code and that we will release publicly. First, the quality filter indexes the unsorted matches with positions from a sorted copy. Second, S3Esti’s orientation, predicted in radians, is passed to a function expecting degrees. Third, S3Esti receives a raw grayscale crop instead of its three-scale normalized RGB pyramid. Fourth, the local affine frame (LAF) magnification factor is applied repeatedly. All other hyperparameters remain unchanged. For fairness, we also report the results reported by Sun et al. where available (Table 2).
| Method | Mega1500 5° | Mega1500 10° | Mega1500 20° | ScanNet1500 5° | ScanNet1500 10° | ScanNet1500 20° |
| KN+HN+AN | 46.4 | 58.7 | 67.5 | 6.8 | 13.2 | 20.4 |
| SIFT+HN+S3E | 46.1 | 58.3 | 67.2 | 5.7 | 11.4 | 18.4 |
| Sun et al.† | 61.5 | 73.1 | 81.1 | 16.4 | 30.7 | 44.7 |
| Sun et al.‡ | 65.5 | 77.9 | 86.3 | — | — | — |
| LoMa+S3E+AN | 63.4 | 77.2 | 86.7 | 28.1 | 50.0 | 67.9 |
| A·LoMa-B | 64.8 | 78.1 | 87.2 | 28.4 | 50.4 | 68.3 |
| A·LoMa-L | 66.0 | 78.9 | 87.8 | 29.7 | 51.9 | 69.8 |
| A·LoMa-G | 65.7 | 78.5 | 87.5 | 30.0 | 52.3 | 70.2 |
Table 2. Essential matrix estimation from ACs. We report area under the curve (AUC) of the pose error. For fair comparison against [81] we use the 2AC solver inside GC-RANSAC [18]. †Results reproduced by the official code. ‡Officially reported numbers from the paper [81].
Affine shapes. Affines are compared in the normalized frame of each image, with and the pixel Jacobian. Sec. D.3 repeats Table 1 in the pixel frame. The residual rotation and scale are read off in the same frame. Every method is given a budget of 2048 keypoints and is evaluated on its own matches that pass the 3 px test of § 4.1. The baselines run at a long side of 1024 px and our descriptor at . The HPatches GT is the analytic Jacobian of at the matched pixel on the 56 viewpoint sequences (280 pairs, excluding v_artisans, v_astronautis and v_talent, following LoFTR [80]). On MegaDepth-1500 it is the finite-difference Jacobian of the depth warp with the covisibility check used in training. Medians are taken over all kept matches pooled over the pairs of a dataset.
Visual localization. We use the implementation of [88] with GC-RANSAC [18] on the re-triangulated COLMAP [78] models of Cambridge Landmarks (five scenes, 1024 px). Each query is localized from its 10 NetVLAD [6] references in hloc [74] with SuperPoint (4096 keypoints) and LightGlue matches. Reconstruction, matches, retrieval and RANSAC are identical in all rows, and only the source of the frames differs. For A¨LoMa-B we run the model on every (query, reference) pair at the SuperPoint keypoints, discard LoMa’s own matches, and read out the predicted frames at the LightGlue matches. Our frames depend on the pair, so they are computed per (query, reference) pair instead of once per image. UP1SIFT uses only the scale and orientation of each frame, read out from the same forward pass of the pair, thus requiring only S3Esti, while UP1PfAC and P1AC use the relative affine of each match (Eq. (2)). For these two, the baseline combines S3Esti’s scale and orientation with AffNet’s shape. We report, for each RANSAC seed, the mean over scenes of the per-scene accuracy (scaled by the fraction of localized queries), with 95% confidence intervals over 100 seeds, without LO in Table 5 and with the default configuration in Table 10. The points-only baseline uses the same matches with the P4Pf solver. Figure 6 varies the RANSAC budget: with UP1SIFT our frames are more accurate at every budget, whereas with UP1PfAC they help mostly at small budgets (32.6% against 19.3% at 2 iterations with LO), and with LO enabled the two coincide from about 100 iterations on, so at the default budget the source of the frames does not matter for UP1PfAC.
| Solver | Method | 25 cm, 2° | 50 cm, 5° |
| P4Pf | Points only | 41.4 ±0.04 | 63.7 ±0.02 |
| UP1SIFT | S3Esti | 45.0 ±0.15 | 59.7 ±0.15 |
| UP1SIFT | A¨LoMa-B | 49.8 ±0.15 | 65.5 ±0.16 |
| UP1PfAC | S3Esti+AffNet | 44.7 ±0.07 | 66.0 ±0.06 |
| UP1PfAC | A¨LoMa-B | 44.7 ±0.06 | 66.1 ±0.07 |
| P1AC | S3Esti | 67.1 ±0.13 | 84.7 ±0.10 |
| P1AC | S3Esti+AffNet | 66.8 ±0.13 | 84.7 ±0.09 |
| P1AC | A¨LoMa-B | 67.2 ±0.13 | 84.8 ±0.09 |
Table 10. Absolute pose on Cambridge Landmarks with the default GC-RANSAC configuration (LO enabled). Same protocol as Table 5.
Generalized relative pose and scale
We follow the view-graph registration protocol of [98] on TUM RGB-D [79] (freiburg2.xyz/rpy), with our own implementation of the solvers as the authors’ code is not available. A pair consists of two view-graphs of four posed frames each,
the second in a random similarity frame (rotation, translation in , scale in ) that is to be recovered. PC and AC solvers receive identical matches, the latter also the predicted affine of each match. RANSAC samples 26 PCs or 9 ACs, scores hypotheses with the Sampson error (2 px) and refines the inliers with a null-space solver. AUC is the area under the recall curve of , pooled over both sequences.
Surface normals. We keep correspondences whose Sampson distance to the known relative pose is below 5 px, which decouples normal accuracy from matching precision. The -side keypoint is backprojected with the GT depth, and the predicted affine (in calibrated coordinates) is passed with the GT pose to the optimal solver of Hajder et al. [47]. The reference normal is a plane fit (PCA) to the 3D points of a window around the keypoint, using pixels within 5% of the center depth. On Map-free we use the validation scenes, which are excluded from training. We report the area under the curve of the sign-invariant angle between the two unit normals.
Additional Experiments
Runtime
In Table 11, we compare the matching speed of A·LoMa to Sun et al. and RoMa v2. As matching dominates the wall-clock time in SfM and visual localization, we only consider it.3 Depending on model size, A·LoMa is roughly 9–13 faster than Sun et al. and 8–12 faster than RoMa v2.
| Method | Throughput (pairs / second) |
| Sun et al. | 3.9 |
| RoMa v2 | 4.3 |
| A·LoMa-B | 51.1 |
| A·LoMa-L | 40.5 |
| A·LoMa-G | 35.7 |
Table 11. Runtime. AC estimation speed on an H200 running a single image pair per batch (higher is better).
End-to-End AC Yield
The protocol of § 4.1 keeps only matches whose point is within 3 px of the GT warp, so the errors of Table 1 are conditional on a correct point match and say nothing about how often a method delivers a usable AC. We therefore also report an unconditional measure.
Protocol. Each method gets a budget of keypoints per image and produces at most ACs. An AC is -correct if its point and its affine are both right,
with the GT warp. The AC yield is the fraction of the budget that is correct, so a method cannot gain by matching conservatively. Setting gives the ordinary point recall (Point column). We fix px and use and (roughly and of rotation error).
Results. Table 12 reports the yield. The Point column is nearly identical for A·LoMa-B/L/G and LoMa+S3E+AN, which use the affine components of their LoMa backbones (G for LoMa+S3E+AN), so the comparison against LoMa+S3E+AN isolates the affine head: LoMa+S3E+AN loses most of its yield to the affine criterion (from 29.1% to 2.8% at on Mega1500), whereas A·LoMa-G loses little (to 23.0%). Sun et al. has a higher point recall on Mega1500 (57.1%), but a lower yield at than ours (17.4% against 27.3%).
| Method | HPatches | Mega1500 | ||||
| Point | 0.25 | 0.1 | Point | 0.25 | 0.1 | |
| RoMa v2 | 96.4 | 94.5 | 89.3 | 72.0 | 63.1 | 47.8 |
| KN+HN+AN | 39.1 | 17.7 | 2.8 | 8.7 | 3.0 | 0.5 |
| SIFT+HN+S3E | 75.0 | 13.2 | 2.0 | 12.7 | 3.0 | 0.7 |
| Sun et al. | 66.5 | 11.8 | 2.0 | 57.1 | 17.4 | 4.9 |
| LoMa+S3E+AN | 51.2 | 9.8 | 1.7 | 29.1 | 9.1 | 2.8 |
| A·LoMa-B | 51.1 | 50.9 | 50.1 | 29.0 | 27.0 | 22.5 |
| A·LoMa-L | 51.1 | 50.9 | 49.8 | 29.1 | 27.2 | 22.8 |
| A·LoMa-G | 51.2 | 51.0 | 50.3 | 29.1 | 27.3 | 23.0 |
Table 12. End-to-end AC yield. Percentage of a budget of 2048 keypoints that yields an AC with a correct point ( px) and affine (Eq. (9)). The columns 0.25 and 0.1 give . Point drops the affine criterion. Higher is better.
Affine Accuracy in the Pixel Frame
A·LoMa is trained in the normalized frame , whereas the patch-based baselines operate in pixel space. Table 13 repeats Table 1 in the pixel frame, with of each image at its native resolution, on the same matches. The residual rotation is read off in the same frame. The two frames agree within about 6% for every method and the ranking is unchanged.
| Method | HPatches | Mega1500 | ||
| Frobenius | Rot. | Frobenius | Rot. | |
| KN+HN+AN | 25.8 | 6.03° | 34.3 | 7.66° |
| SIFT+HN+S3E | 51.2 | 8.83° | 48.8 | 5.24° |
| Sun et al. | 47.3 | 3.39° | 40.1 | 2.00° |
| LoMa+S3E+AN | 50.0 | 4.46° | 42.9 | 2.40° |
| A·LoMa-B | 1.5 | 0.25° | 4.1 | 0.61° |
| A·LoMa-L | 1.6 | 0.24° | 4.0 | 0.60° |
| A·LoMa-G | 1.5 | 0.24° | 3.8 | 0.57° |
Table 13. Accuracy of affine transformations in the pixel frame. Median Frobenius () and rotational errors on the matches of Table 1 (lower is better).
Sensitivity to the Ground-Truth Construction
The supervision of § 3.4 is a finite-difference Jacobian (Eq. (5)) of the depth-induced warp, a first-order approximation. The HPatches GT, the closed-form Jacobian of a known homography, uses no depth, no finite differences and no masking, yet A·LoMa is the most accurate method there by an order of magnitude (Table 1). We also measure the truncation error of the step directly: on HPatches, Table 14 gives the Frobenius distance between the forward difference and the analytic Jacobian as a function of the step . The error grows linearly in , and at our step px it is 0.03 (, median), about below the 1.50 error of A·LoMa-G on the same data.
| (px) | 0.25 | 0.5 | 1 | 2 | 4 | 8 |
| Median | 0.007 | 0.014 | 0.027 | 0.054 | 0.108 | 0.217 |
| Mean | 0.010 | 0.021 | 0.042 | 0.083 | 0.167 | 0.334 |
| 95th pct. | 0.031 | 0.062 | 0.124 | 0.248 | 0.496 | 0.993 |
Table 14. Finite-difference step vs. the exact Jacobian on HPatches. Frobenius distance () between the forward difference with step (px) and the analytic Jacobian. Training uses px.
Full Orientation Estimation
Prior work based on local patches has found it difficult to predict orientation, scale and affine shape jointly [66, 81]. Here we show that our setup predicts full affine frames, including orientation. We compare the LoMa-B matcher with the rotation-invariant LoMa-R matcher [69], which has the same architecture, on independently rotated image pairs, following [69]. Table 15 has four rows: frozen LoMa-B without rotation augmentation (the upright baseline), frozen LoMa-R, unfrozen LoMa-R, and unfrozen LoMa-B initialized from A·LoMa-B, the last two being the proposed variants. Frozen LoMa-R gives subpar results, even compared to the upright baseline, because its features are rotation invariant and cannot be used to estimate orientation, although it matches rotated images better than LoMa-B. The two proposed variants give both good affine estimates and good matches and are comparable, so the LoMa-B feature dimension suffices for both rotation-robust matching and orientation estimation.
| Matcher | ❄/🔥 | RA | HPatches | Mega1500 | ||||
| Aff. Prob. | Aff. Prob. | AUC@20° | ||||||
| ↑ | ↻ | ↑ | ↻ | ↑ | ↻ | |||
| LoMa-B | ❄ | ✗ | 1.62 | 3.30 | 3.97 | 7.01 | 83.6 | 81.2 |
| LoMa-R | ❄ | ✓ | 8.00 | 9.01 | 8.74 | 8.96 | 82.8 | 83.2 |
| LoMa-R | 🔥 | ✓ | 1.69 | 1.83 | 4.06 | 4.11 | 83.4 | 82.8 |
| LoMa-B | 🔥 | ✓ | 1.58 | 1.70 | 4.27 | 4.58 | 82.8 | 82.6 |
Table 15. Results under rotation. ❄ and 🔥 denote using a frozen and un-frozen matcher, respectively. RA is short for rotation augmentation during training. We denote the upright evaluation by ↑ and the rotated by ↻.
Gauge Freedom
The loss of Eq. (6) constrains only the composition , so the per-keypoint frames are determined only up to a common right factor (Figure 5(a)). How much of this freedom the trained model uses is an empirical question.
Protocol. Using A·LoMa-G on HPatches viewpoint sequences, we fix the first image of a sequence and pair it with other views, which gives predictions of the frame of each of its keypoints. For every keypoint and pair of partners we measure the residual gauge
which is the identity exactly when both partners induce the same frame, and summarize it by the median of over keypoints and partner pairs. Even for stable frames two estimates differ by their estimation error, so we compare with the affine error of Table 1 on the same data (1.50 for A·LoMa-G on HPatches, ), which gives an expected distance of roughly to , i.e., 2.1 to 3.0.
Results. Figure 5(b,c) shows the frames of one image under five partners and the drift over six HPatches viewpoint sequences. The median drift () is 30 (london), 33 (coffeehouse), 68 (tempera), 80 (wormhole), 84 (calder) and 130 (gardens), an order of magnitude or more above the 2.1 to 3.0 expected for stable frames. The gauge is therefore largely unresolved. This means that a frame predicted for one pair is not a view-independent quantity, which is what the transport (Sec. F) aims to address.
Per-View Frames by Transport
The frames and of a pair-conditioned A·LoMa depend on the partner image (Sec. E), whereas applications, such as SfM, benefit from a frame per keypoint that is independent of any other view. A head that reads a single image gives such a frame, for keypoint in image , but is less accurate than the relative affine of the conditioned model. We combine the two by transport. Let be the image whose frames we want and another view of the scene, with a match between and . The relative affine carries the per-view frame of to
an estimate of the frame of in the convention of the per-view head. We average Eq. (11) over partner views in the parametrization of the head (Eq. (3)). Each transported matrix is decomposed into scale, rotation and shape as in Eq. (3), then the log-scale, log shape exponent and shear are averaged arithmetically and the orientation as a circular mean (the normalized mean of ). A keypoint that no partner supports keeps . On MegaDepth-1500 we use up to random views of the same scene with at least 100 matches to (never the two images of the evaluated pair), and all uses every other view. The relative affines come from A·LoMa-B. The errors of the per-view head in different views are roughly independent and average down, while the accurate relative affines carry each frame between views. With this is the per-view head, and Figure 7 shows how the error decreases with . Averaging roughly halves the error of the single-view head. Eichhardt and Barath [39] similarly correct local affine frames using multiple views, though geometrically.
Figure 7. Averaged frames improve with the number of partners. Affine error (Frobenius) on MegaDepth-1500 against the number of partner views , from the per-view head () to the pairwise A·LoMa-B.
More Qualitative Results
In Figure 8, we provide additional examples of AC estimation on MegaDepth-1500. We compare A·LoMa-B to KN+HN+AN (AffNet) and Sun et al.
Figure 8. Visualizing further affine estimations. Pairs from the two MegaDepth-1500 scenes, 0015 (left) and 0022 (right). Errors are median Frobenius ().
References
- [1]Cenek Albl, Zuzana Kukelova, and Tomas Pajdla. R6P – rolling shutter absolute pose problem. In CVPR, pages 2292–2300, 2015.DOI
- [2]Cenek Albl, Zuzana Kukelova, and Tomas Pajdla. Rolling shutter absolute pose problem with known vertical direction. In CVPR, pages 3355–3363, 2016.DOI
- [3]Cenek Albl, Zuzana Kukelova, Viktor Larsson, and Tomas Pajdla. Rolling shutter camera absolute pose. IEEE TPAMI, 42(6):1439–1452, 2020.DOI
- [4]Khaled Alyousfi and Jonathan Ventura. Multi-camera motion estimation with affine correspondences. In ICIAR, pages 417–431, Cham, 2020. Springer International Publishing.
- [5]Manuel López-Antequera, Pau Gargallo, Markus Hofinger, Samuel Rota Bulo, Yubin Kuang, and Peter Kontschieder. Mapillary planet-scale depth dataset. In ECCV, 2020.DOI
- [6]Relja Arandjelovic, Petr Gronat, Akihiko Torii, Tomas Pajdla, and Josef Sivic. NetVLAD: CNN architecture for weakly supervised place recognition. In CVPR, 2016.
- [7]Eduardo Arnold, Jamie Wynn, Sara Vicente, Guillermo Garcia-Hernando, Aron Monszpart, Victor Prisacariu, Daniyar Turmukhambetov, and Eric Brachmann. Map-free visual relocalization: Metric pose relative to a single image. In ECCV, 2022.arxiv.org/abs/2210.05494
- [8]Jonathan Astermark, Yaqing Ding, Viktor Larsson, and Anders Heyden. Fast relative pose estimation using relative depth. In 3DV, 2024.DOI
- [9]Armen Avetisyan, Christopher Xie, Henry Howard-Jenkins, Tsun-Yi Yang, Samir Aroudj, Suvam Patra, Fuyang Zhang, Duncan Frost, Luke Holland, Campbell Orme, Jakob Engel, Edward Miller, Richard Newcombe, and Vasileios Baltas. Scenescript: Reconstructing scenes with an autoregressive structured language model. In ECCV, 2025.
- [10]Vasileios Baltas, Karel Lenc, Andrea Vedaldi, and Krystian Mikolajczyk. HPatches: A benchmark and evaluation of handcrafted and learned local descriptors. In CVPR, 2017.
- [11]Daniel Barath. SuperRANSAC: One RANSAC to rule them all. arXiv:2506.04803, 2025.arxiv.org/abs/2506.04803
- [12]Daniel Barath. Rolling shutter relative pose estimation made practical. In ECCV, 2026.arxiv.org/abs/2606.26863
- [13]Daniel Barath and Levente Hajder. Novel ways to estimate homography from local affine transformations. In Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, pages 434–445, 2016.
- [14]Daniel Barath and Levente Hajder. A theory of point-wise homography estimation. PRL, 94(C):7–14, 2017.DOI
- [15]Daniel Barath and Levente Hajder. Efficient recovery of essential matrix from two affine correspondences. IEEE TIP, 27:5328–5337, 2018.DOI
- [16]Daniel Barath and Zuzana Kukelova. Homography from two orientation- and scale-covariant features. In ICCV, 2019.arxiv.org/abs/1906.11927
- [17]Daniel Barath and Zuzana Kukelova. Relative pose from SIFT features. In ECCV, 2022. 2arxiv.org/abs/2203.07930
- [18]Daniel Barath and Jiri Matas. Graph-cut RANSAC: Local optimization on spatially coherent structures. IEEE TPAMI, 44(9):4961–4974, 2021. 5, 6, 2DOI
- [19]Daniel Barath, Tekla Toth, and Levente Hajder. A minimal solution for two-view focal-length estimation using two affine correspondences. In CVPR, 2017. 1, 2arxiv.org/abs/1706.01649
- [20]Daniel Barath, Michal Polic, Wolfgang Förstner, Torsten Sattler, Tomas Pajdla, and Zuzana Kukelova. Making Affine Correspondences Work in Camera Geometry Computation. In ECCV, 2020. 2arxiv.org/abs/2007.10032
- [21]Jonathan T Barron. A general and adaptive robust loss function. In CVPR, 2019. 5arxiv.org/abs/1701.03077
- [22]Axel Barroso-Laguna, Edgar Riba, Daniel Ponsa, and Krystian Mikolajczyk. Key.Net: Keypoint detection by hand-crafted and learned CNN filters. In ICCV, 2019. 3, 5
- [23]A. Baumberger. Reliable feature matching across widely separated views. In CVPR, pages 774–781 vol.1, 2000. 1
- [24]Jacob Bentolila and Joseph M. Francos. Conic epipolar constraints from affine correspondences. CVIU, 122, 2014. 2DOI
- [25]Georg Bökman, Johan Edstedt, Michael Felsberg, and Fredrik Kahl. Steerers: A framework for rotation equivariant keypoint descriptors. In CVPR, 2024. 3arxiv.org/abs/2312.02152
- [26]Yohann Cabon, Naila Murray, and Martin Humenberger. Virtual kitti 2. arXiv preprint arXiv:2001.10773, 2020. 1
- [27]Ondrej Chum and Jiri Matas. Matching with prosac-progressive sample consensus. In CVPR, 2005. 6DOI
- [28]Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. ScanNet: Richly-annotated 3D reconstructions of indoor scenes. In CVPR, 2017. 6arxiv.org/abs/1702.04405
- [29]Ji Dai, Shiwei Jin, Junkang Zhang, and Truong Q. Nguyen. Boosting feature matching accuracy with pairwise affine estimation. IEEE TIP, pages 8278–8291, 2020. 2DOI
- [30]Yuchao Dai, Hongdong Li, and Laurent Kneip. Rolling shutter camera relative pose: Generalized epipolar geometry. In CVPR, 2016. 6arxiv.org/abs/1605.00475
- [31]Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich. Superpoint: Self-supervised interest point detection and description. In CVPR, 2018. 3, 6arxiv.org/abs/1712.07629
- [32]Yaqing Ding, Václav Vávra, Snehal Bhayani, Qianliang Wu, Jian Yang, and Zuzana Kukelova. Fundamental matrix estimation using relative depths. In ECCV, 2024. 2DOI
- [33]Johan Edstedt, Ioannis Athanasiadis, Mårten Wadenbäck, and Michael Felsberg. DKM: Dense kernelized feature matching for geometry estimation. In CVPR, 2023. 2, 3, 5arxiv.org/abs/2202.00667
- [34]Johan Edstedt, Georg Bökman, and Zhenjun Zhao. Dedode v2: Analyzing and improving the dedode keypoint detector. In CVPRW, 2024. 3arxiv.org/abs/2404.08928
- [35]Johan Edstedt, Georg Bökman, Mårten Wadenbäck, and Michael Felsberg. DeDoDe: Detect, don’t describe – describe, don’t detect for local feature matching. In 3DV, 2024. 3arxiv.org/abs/2308.08479
- [36]Johan Edstedt, Qiyu Sun, Georg Bökman, Mårten Wadenbäck, and Michael Felsberg. RoMa: Robust dense feature matching. In CVPR, 2024. 2, 3DOI
- [37]Johan Edstedt, Georg Bökman, Mårten Wadenbäck, and Michael Felsberg. DaD: Distilled reinforcement learning for diverse keypoint detection. arXiv preprint arXiv:2503.07347, 2025. 3, 1arxiv.org/abs/2503.07347
- [38]Johan Edstedt, David Nordström, Yushan Zhang, Georg Bökman, Jonathan Astermark, Viktor Larsson, Anders Heyden, Fredrik Kahl, Mårten Wadenbäck, and Michael Felsberg. RoMa v2: Harder better faster denser feature matching. In ECCV, 2026. 2, 3, 5, 1arxiv.org/abs/2511.15706
- [39]Ivan Eichhardt and Daniel Barath. Optimal multi-view correction of local affine frames. In BMVC, 2019. 5arxiv.org/abs/1905.00519
- [40]Iván Eichhardt and Dmitry Chetverikov. Affine correspondences between central cameras for rapid relative pose estimation. In ECCV, pages 488–503. Springer, 2018. 2DOI
- [41]Per-Erik Forssén and David G Lowe. Shape descriptors for maximally stable extremal regions. In ICCV. IEEE, 2007. 2
- [42]Adrien Gaidon, Qiao Wang, Yohann Cabon, and Eleonora Vig. Virtual worlds as proxy for multi-object tracking analysis. In CVPR, 2016. 1arxiv.org/abs/1605.06457
- [43]Hugo Germain, Guillaume Bourmaud, and Vincent Lepetit. S2DNet: learning image features for accurate sparse-to-dense matching. In ECCV, 2020. 3DOI
- [44]Banglei Guan, Ji Zhao, Zhang Li, Fang Sun, and Friedrich Fraundorfer. Minimal solutions for relative pose with a single affine correspondence. In CVPR, 2020. 2DOI
- [45]Banglei Guan, Ji Zhao, and Laurent Kneip. A complete solution to generalized relative pose estimation from affine correspondences. IEEE TPAMI, 48(7):8598–8612, 2026. 2DOI
- [46]Levente Hajder and Daniel Barath. Relative planar motion for vehicle-mounted cameras from a single affine correspondence. In ICRA, pages 8651–8657, 2020. 2arxiv.org/abs/1912.06465
- [47]Levente Hajder, Lajos Lóczi, and Daniel Barath. Fast globally optimal surface normal from an affine correspondence. In ICCV, pages 3367–3378, 2023. 1, 2, 7, 3DOI
- [48]Richard Hartley and Andrew Zisserman. Multiple View Geometry in Computer Vision. Cambridge University Press, 2 edition, 2004. 1, 5DOI
- [49]Petr Hruby and Marc Pollefeys. Single-scanline relative pose estimation for rolling shutter cameras. In ICCV, 2025. 6arxiv.org/abs/2506.22069
- [50]Petr Hruby, Marc Pollefeys, and Daniel Barath. Semi-calibrated relative pose from an affine correspondence and monodepth. In CVPR, 2024. 1, 2
- [51]Hanwen Jiang, Zexiang Xu, Desai Xie, Ziwen Chen, Haian Jin, Fujun Luan, Zhixin Shu, Kai Zhang, Sai Bi, Xin Sun, Jiuxiang Gu, Qixing Huang, Georgios Pavlakos, and Hao Tan. MegaSynth: Scaling up 3D scene reconstruction with synthesized data. In CVPR, 2025. 1arxiv.org/abs/2412.14166
- [52]Alex Kendall, Matthew Grimes, and Roberto Cipolla. PoseNet: A convolutional network for real-time 6-DoF camera relocalization. In ICCV, 2015. 6arxiv.org/abs/1505.07427
- [53]Shinjeong Kim, Marc Pollefeys, and Daniel Barath. Learn-ing to make keypoints sub-pixel accurate. In ECCV, 2024. 3arxiv.org/abs/2407.11668
- [54]Kevin Köser and Reinhard Koch. Differential spatial resection-pose estimation using a single local image feature. In ECCV, pages 312–325. Springer, 2008. 1DOI
- [55]Zuzana Kukelova, Martin Bujnak, and Tomas Pajdla. Closed-form solutions to minimal absolute pose problems with known vertical direction. In ACCV, pages 216–229, 2010. 2DOI
- [56]Yuan Li, Yuanbo Xiangli, Hadar Averbuch-Elor, Noah Snavely, and Ruojin Cai. Long-tail internet photo reconstruction. In CVPR, 2026. 1arxiv.org/abs/2604.22714
- [57]Zhengqi Li and Noah Snavely. Megadepth: Learning single-view depth prediction from internet photos. In CVPR, 2018. 5, 1arxiv.org/abs/1804.00607
- [58]Philipp Lindenberger, Paul-Edouard Sarlin, and Marc Pollefeys. LightGlue: Local Feature Matching at Light Speed. In ICCV, 2023. 3, 6, 1arxiv.org/abs/2306.13643
- [59]Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In ICLR, 2019. 5
- [60]David G Lowe. Distinctive image features from scale-invariant keypoints. IJCV, 2004. 3, 5DOI
- [61]Nikolaus Mayer, Eddy Ilg, Philip Hausser, Philipp Fischer, Daniel Cremers, Alexey Dosovitskiy, and Thomas Brox. A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation. In CVPR, 2016. 1DOI
- [62]Krystian Mikolajczyk and Cordelia Schmid. An affine invariant interest point detector. In ECCV, 2002. 1, 2DOI
- [63]Krystian Mikolajczyk and Cordelia Schmid. Scale & affine invariant interest point detectors. IJCV, 60, 2004. 1DOI
- [64]Krystian Mikolajczyk, Tinne Tuytelaars, Cordelia Schmid, Andrew Zisserman, Jiri Matas, Frederik Schaffalitzky, Timor Kadir, and Luc Van Gool. A comparison of affine region detectors. IJCV, 65(1–2):43–72, 2005. 2DOI
- [65]Anastasiia Mishchuk, Dmytro Mishkin, Filip Radenovic, and Jiri Matas. Working hard to know your neighbor's margins: Local descriptor learning loss. In NeurIPS, 2017. 5arxiv.org/abs/1705.10872
- [66]Dmytro Mishkin, Filip Radenovic, and Jiří Matas. Repeatability is not enough: Learning affine regions via discriminability. In ECCV, 2018. 1, 2, 4, 5arxiv.org/abs/1711.06704
- [67]David Nistér. An efficient solution to the five-point relative pose problem. IEEE TPAMI, 26(6), 2004. 1
- [68]David Nordström, Johan Edstedt, Georg Bökman, Jonathan Astermark, Anders Heyden, Viktor Larsson, Mårten Wadenbäck, Michael Felsberg, and Fredrik Kahl. LoMa: Local feature matching revisited. In ECCV, 2026. 2, 3, 1arxiv.org/abs/2604.04931
- [69]David Nordström, Johan Edstedt, Georg Bökman, and Fredrik Kahl. Who handles orientation? Investigating invariance in feature matching. In CVPRW, 2026. 3, 8, 4arxiv.org/abs/2604.11809
- [70]Michal Perdoch, Jiri Matas, and Ondrej Chum. Epipolar geometry from two correspondences. In ICPR, pages 215–219, 2006. 1, 2DOI
- [71]Carolina Raposo and Joao P. Barreto. Theory and practice of structure-from-motion using affine correspondences. In CVPR, 2016. 1, 2DOI
- [72]Jerome Revaud, Cesar De Souza, Martin Humenberger, and Philippe Weinzaepfel. R2D2: Reliable and repeatable detector and descriptor. In NeurIPS, 2019. 3
- [73]Mike Roberts, Jason Ramapuram, Anurag Ranjan, Atulit Kumar, Miguel Angel Bautista, Nathan Paczan, Russ Webb, and Joshua M. Susskind. Hypersim: A photorealistic synthetic dataset for holistic indoor scene understanding. In ICCV, 2021. 1arxiv.org/abs/2011.02523
- [74]Paul-Edouard Sarlin, Cesar Cadena, Roland Siegwart, and Marcin Dymczyk. From coarse to fine: Robust hierarchical localization at large scale. In CVPR, 2019. 2arxiv.org/abs/1812.03506
- [75]Paul-Edouard Sarlin, Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich. SuperGlue: Learning feature matching with graph neural networks. In CVPR, pages 4938–4947, 2020. 2, 3, 6arxiv.org/abs/1911.11763
- [76]Olivier Sauer, Marc Pollefeys, and Gim Hee Lee. A minimal solution to the rolling shutter pose estimation problem. In IROS, pages 1328–1334, 2015. 6
- [77]David Schubert, Nikolaus Demmel, Lukas von Stumberg, Vladyslav Usenko, and Daniel Cremers. Rolling-shutter modelling for direct visual-inertial odometry. In IROS, page 2462–2469. IEEE, 2019. 6DOI
- [78]Johannes Lutz Schönberger and Jan-Michael Frahm. Structure-from-Motion Revisited. In CVPR, 2016. 2DOI
- [79]Jürgen Sturm, Nikolas Engelhard, Felix Endres, Wolfram Burgard, and Daniel Cremers. A benchmark for the evaluation of RGB-D SLAM systems. In IROS, pages 573–580, 2012. 6, 2DOI
- [80]Jiaming Sun, Zehong Shen, Yuang Wang, Hujun Bao, and Xiaowei Zhou. LoFTR: Detector-free local feature matching with transformers. In CVPR, 2021. 3, 5, 1arxiv.org/abs/2104.00680
- [81]Pengju Sun, Banglei Guan, Zhenbao Yu, Yang Shang, Qifeng Yu, and Daniel Barath. Learning affine correspondences by integrating geometric constraints. In CVPR, 2025. 1, 2, 4, 5, 6arxiv.org/abs/2504.04834
- [82]Chris Sweeney, John Flynn, Benjamin Nuernberger, Matthew Turk, and Tobias Höllerer. Efficient computation of absolute pose for gravity-aware augmented reality. In ISMAR, pages 19–24, 2015. 2DOI
- [83]Yurun Tian, Xin Yu, Bin Fan, Fuchao Wu, Huub Heijnen, and Vassileios Baltas. Sosnet: Second order similarity regularization for local descriptor learning. In CVPR, 2019. 3
- [84]Fabio Tosi, Yiyi Liao, Carolin Schmitt, and Andreas Geiger. Smd-nets: Stereo mixture density networks. In CVPR, 2021. 1arxiv.org/abs/2104.03866
- [85]Joseph Tung, Gene Chou, Ruojin Cai, Guandao Yang, Kai Zhang, Gordon Wetzstein, Bharath Hariharan, and Noah Snavely. Megascenes: Scene-level view synthesis at scale. In ECCV, 2024. 1arxiv.org/abs/2406.11819
- [86]Michał Tyszkiewicz, Pascal Fua, and Eduard Trulls. DISK: Learning local features with policy gradient. In NeurIPS, 2020. 3arxiv.org/abs/2006.13566
- [87]Marcus Valtonen Örnhag and Alberto Jaenal. Leveraging scale- and orientation-covariant features for planar motion estimation. In ECCV, 2024. 2DOI
- [88]Marcus Valtonen Örnhag, Alberto Jaenal, and Stefan Adalbjörnsson. Gravity-aware partially calibrated absolute pose estimation from affine- or rotation-covariant features. In ECCV, 2026. 1, 2, 6DOI
- [89]Jonathan Ventura, Zuzana Kukelova, Torsten Sattler, and Daniel Barath. PIAC: Revisiting absolute pose from a single affine correspondence. In ICCV, pages 19751–19761, 2023. 1, 6
- [90]Jonathan Ventura, Zuzana Kukelova, Torsten Sattler, and Daniel Barath. Absolute pose from one or two scaled and oriented features. In CVPR, 2024. 2, 6DOI
- [91]Yannick Verdie, Kwang Yi, Pascal Fua, and Vincent Lepetit. Tilde: A temporally invariant learned detector. In CVPR, 2015. 3DOI
- [92]Khiem Vuong, Anurag Ghosh, Deva Ramanan, Srinivasa Narasimhan, and Shubham Tulsiani. Aerialmegadepth: Learning aerial-ground reconstruction and view synthesis. In CVPR, 2025. 1arxiv.org/abs/2504.13157
- [93]Bing Wang, Changhao Chen, Zhaopeng Cui, Jie Qin, Chris Xiaoxuan Lu, Zhengdi Yu, Peijun Zhao, Zhen Dong, Fan Zhu, Niki Trigoni, and Andrew Markham. P2-net: Joint description and detection of local features for pixel and point matching. In ICCV, 2021. 3arxiv.org/abs/2103.01055
- [94]Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. VGGT: Visual geometry grounded transformer. In CVPR, 2025. 8, 1arxiv.org/abs/2503.11651
- [95]Jiahao Wang, Yufeng Yuan, Rujie Zheng, Youtian Lin, Jian Gao, Lin-Zhuo Chen, Yajie Bao, Yi Zhang, Chang Zeng, Yanxi Zhou, et al. SpatialVID: A large-scale video dataset with spatial annotations. arXiv preprint arXiv:2509.09676, 2025. 1arxiv.org/abs/2509.09676
- [96]Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. DUSt3R: Geometric 3D vision made easy. In CVPR, 2024. 1arxiv.org/abs/2312.14132
- [97]Wenshan Wang, Delong Zhu, Xiangwei Wang, Yaoyu Hu, Yuheng Qiu, Chen Wang, Yafei Hu, Ashish Kapoor, and Sebastian Scherer. TartanAir: A dataset to push the limits of visual slam. In IROS, 2020. 1arxiv.org/abs/2003.14338
- [98]Wanting Xu, Xinyue Zhang, Marc Pollefeys, Daniel Barath, and Laurent Kneip. Generalized relative pose and scale from affine correspondences. IJCV, pages 1–17, 2025. 2, 6DOI
- [99]Pei Yan, Yihua Tan, Shengzhou Xiong, Yuan Tai, and Yansheng Li. Learning soft estimator of keypoint scale and orientation with probabilistic covariant loss. In CVPR, 2022. 2, 5
- [100]Yao Yao, Zixin Luo, Shiwei Li, Jingyang Zhang, Yufan Ren, Lei Zhou, Tian Fang, and Long Quan. Blendedmvs: A large-scale dataset for generalized multi-view stereo networks. In CVPR, 2020. 1arxiv.org/abs/1911.10127
- [101]Chandan Yeshwanth, Yueh-Cheng Liu, Matthias Nießner, and Angela Dai. ScanNet++: A high-fidelity dataset of 3D indoor scenes. In ICCV, 2023. 1arxiv.org/abs/2308.11417
- [102]Zhenbao Yu, Banglei Guan, Shunkun Liang, Zhang Li, Shirong Ye, and Qifeng Yu. Globally optimal relative pose estimation using affine correspondences with known vertical direction. IEEE TIM, 72:1–12, 2023. 2DOI
- [103]Zhenbao Yu, Shirong Ye, Ronghe Jin, Shunkun Liang, Zibin Liu, Huiyun Zhang, and Banglei Guan. A minimal solver for relative pose estimation with unknown focal length from two affine correspondences. IEEE RA-L, 11(2): 2290–2297, 2026. 2
- [104]Xiaoming Zhao, Xingming Wu, Jinyu Miao, Weihai Chen, Peter CY Chen, and Zhenguo Li. ALike: Accurate and lightweight keypoint detection and descriptor extraction. IEEE TMM, 2022. 3arxiv.org/abs/2112.02906
- [105]Xiaoming Zhao, Xingming Wu, Weihai Chen, Peter C. Y. Chen, Qingsong Xu, and Zhenguo Li. Aliked: A lighter keypoint and descriptor extraction network via deformable transformation. IEEE TIM, 72, 2023. 3arxiv.org/abs/2304.03608







