betaFeedback

WorksComputer Science

OmniRay: Camera-Agnostic Monocular 3D Reconstruction with Composite Ray Fields

Abstract

Monocular models recover 3D geometry from a single image in one pass, but most assume a pinhole camera and produce distorted geometry on the wide-angle imagery produced by fisheye and panoramic lenses. Camera-aware models remove this assumption by predicting a per-pixel ray field. However, they represent different camera geometries using a single fixed basis to capture both the overall camera projection and its finer variations. We argue that camera geometry can be viewed as having a hierarchy of complexity for representing rays. We present OmniRay, a feed-forward model whose Ray Module is composed of three components: a monotone radial base that captures radially symmetric structure, a spherical-harmonic residual for smooth deviations from it, and a free-form residual for remaining local variation. These components are composed as successive rotations on the sphere, allowing the representation to accommodate diverse camera geometries. Given a single image, OmniRay predicts a ray direction and an up-to-scale radial distance for every pixel, from which a 3D point cloud is recovered without needing camera information at inference time. Extensive experiments across different camera configurations demonstrate that OmniRay establishes a new state-of-the-art in camera-agnostic monocular 3D reconstruction.

Project Page: https://nam1410.github.io/omniray/

Introduction

Monocular 3D reconstruction recovers a 3D point for each pixel in an image. We consider central cameras, in which all viewing rays pass through a single optical center, so each 3D point splits into a ray direction and a distance along that ray. The ray direction depends only on the camera geometry, while the distance depends on the scene under observation. Recent feed-forward models perform well on monocular 3D estimation [33, 31], but their architectures or training data focus on pinhole cameras. When applied to wide-angle images, these assumptions often lead to distorted reconstructed geometry. Wide-angle cameras are common in robotics [34], augmented and virtual reality [4], and 360° capture [15], making this a practical limitation. One way to handle these images is to calibrate the camera and rectify them to a pinhole view before reconstruction, but a pinhole cannot represent rays at or beyond 90° from its optical axis. As such, undistortion typically leads to cropping away part of the field of view or strongly stretching the image near its periphery, leading to low-confidence predictions from conventional depth estimation or 3D foundation models [7].

Camera-aware models instead account for camera geometry directly [3, 26, 21]. DeepFocal [32] and DeepCalib [3] first use convolutional neural networks (CNNs) to estimate focal length and distortion of fixed camera models. Follow-up works generalize camera parameter estimation to arbitrary camera models by estimating ray fields directly. In particular, UniK3D [21] predicts a per-pixel ray field using a spherical-harmonic (SH) basis and a radial distance along each ray. It can therefore reconstruct pinhole, fisheye, and panoramic images without camera metadata. In this formulation, the same basis must describe both the overall camera projection and its finer variations. We argue that there exists a hierarchy of complexity in camera geometry estimation, which can lead to more efficient and performant design. Many camera projections have a simple structure: for approximately radially symmetric lenses, a smooth radial function is sufficient to describe how viewing directions change across the image. Spherical harmonics can be used to complement radial functions.

We therefore propose OmniRay – that takes a single image and predicts a dense, up-to-scale 3D reconstruction without intrinsics or camera-type input (Figure 1). A shared image encoder feeds two modules. The Ray Module predicts a monotone radial base and the two residuals, each of which refines the preceding ray field. The Distance Module predicts the radial distance of each pixel, and the corresponding 3D point is obtained by scaling its predicted ray. We train OmniRay on cameras ranging from pinhole to panorama and evaluate it on held-out scenes, datasets, and camera configurations. An oracle analysis examines which ray fields each representation can express, while benchmark experiments and ablations test the performance of the proposed components.

OmniRay reconstructs 3D geometry from a single image across diverse camera geometries

Figure 1. OmniRay reconstructs 3D geometry from a single image across diverse camera geometries. A single model predicts an up-to-scale, camera-frame point cloud without camera intrinsics or camera-type input. Columns show inputs and reconstructions for pinhole, GoPro, fisheye, catadioptric, and panoramic cameras, respectively. Note that the GoPro and catadioptric camera configurations are withheld from training.

In summary, our contributions are:

  1. A composite ray representation built from a monotone radial base, a spherical-harmonic residual, and a free-form residual.

  2. OmniRay, a feed-forward model for single-image 3D reconstruction across central camera geometries, without needing intrinsics or camera-type input.

  3. An oracle analysis of ray representations, and experiments across seen and held-out scenes, datasets, and camera configurations show that OmniRay improves over the evaluated baselines in ray field estimation and monocular 3D reconstruction.

Related Work

Camera calibration. Camera calibration determines the mapping from image pixels to rays in the camera frame. Classical methods parameterize this mapping with a low-dimensional model whose parameters are estimated from geometric observations [35, 24]. Different camera geometries are handled by different parametric models: pinhole model with radial and tangential distortion for conventional lenses, Kannala & Brand (2006) model for wide-angle optics, and unified camera models [6, 18, 14, 27] for omnidirectional cameras. Each of these models fixes a particular functional form, so a lens whose behavior departs from that form introduces residual error. Generic models relax these assumptions with more degrees of freedom and capture complex lens behavior [25] at the cost of a higher-dimensional calibration problem.

Learning-based methods offer an alternative to this high-dimensional calibration process by learning camera geometry directly from images without input calibration. DeepCalib [3] regresses camera parameters, while Perspective Fields [11], GeoCalib [29], and AnyCalib [26] move toward dense geometric cues: a per-pixel up-vector and latitude field [11, 29], or per-pixel rays [26], each of which describes the camera more generally than a parameter vector. Neural Ray Surfaces [28] take it further by learning a free-form ray surface from video observations and directly representing the pixel-to-ray mapping. These calibration approaches shift from explicit parameter estimation toward learning the pixel-to-ray mapping, but leave open how this mapping should be represented to remain both compact and expressive simultaneously.

Monocular 3D. Recent foundation models have improved depth estimation through large-scale training and distillation [33], but do not explicitly model the camera geometry. Depth Anything 3 [16] instead predicts depth with per-pixel rays by modeling camera geometry alongside scene structure. Camera-aware models specifically account for camera geometry in monocular depth estimation. UniK3D [21] represents a per-pixel ray field with spherical-harmonics and predicts radial distance along each ray to handle diverse camera geometries without camera metadata. Depth Any Camera [8] and UniDAC [5] instead transform images into a common equirectangular representation to process different camera types before depth estimation, but require input camera intrinsics for the transformation. CAM3R (Gurupur̄asad et al., 2026) and Wid3R [12] extend the spherical-harmonic ray representation to multi-view feed-forward 3D reconstruction. Together, these works highlight the value of explicitly accounting for the pixel-to-ray mapping for camera-agnostic 3D geometry, while leaving open whether a single ray representation is necessary for all camera geometries.

Method

Given a single image I∈RH×W×3I \in\mathbb{R}^{H \times W \times3} from an arbitrary central camera, with no intrinsics, distortion parameters or camera-type label, OmniRay predicts for every pixel u=(ux,uy)u = (u_x, u_y) a unit ray d(u)∈S2\mathbf{d}(u) \in S^2 and a radial distance ρ(u)>0\rho(u) > 0.

A DINOv2 backbone (Oquab et al. [20]) encodes the image, and its features drive two branches (Figure 2): the Ray Module composes d\mathbf{d} from radial, spherical-harmonic, and free-form components, while the Distance Module predicts ρ\rho. Their combination gives the 3D point in the camera frame:

OmniRay overview showing reconstruction overview and composite ray representation

Figure 2. OmniRay Overview. Given a single image, a DINOv2 ViT-L/14 encoder produces global and dense features that drive two branches. The Ray Module progressively builds a camera-agnostic ray field from a monotone radial base, a spherical-harmonic residual, and a free-form residual. The Distance Module predicts a positive radial distance for each pixel. Combining the final ray dd and distance ρ\rho yields the local pointmap P=ρdP = \rho d, recovering 3D geometry up to scale without camera intrinsics or a camera-type label.

P(u)=ρ(u)d(u).(1)\mathbf{P}(u) = \rho(u)\mathbf{d}(u). \tag*{(1)}

We predict radial distance ρ\rho rather than perpendicular depth ZZ, related by ρ=Z/dz\rho= Z/d_z, since dz→0d_z \to0 as the ray approaches 90∘90^\circ from the optical axis and changes sign beyond it; ρ\rho therefore remains well-defined across wide fields of view.

Notation. The DINOv2 backbone produces a dense feature map FF and a pooled global token g\mathbf{g}. We use the normalized image coordinate m(u)=(u−cimg)/D\mathbf{m}(u) = (u-\mathbf{c}_{\mathrm{img}})/D, where cimg\mathbf{c}_{\mathrm{img}} is the image center and DD is the image half-diagonal in pixels. This normalization places the image corners at ∥m∥=1\lVert\mathbf{m}\rVert= 1 for any resolution and aspect ratio. Image scale is provided separately to g\mathbf{g} through log⁡(W/H)\log(W/H) and the logarithm of the image diagonal. Ground-truth quantities are marked with a star, d⋆\mathbf{d}^{\star} and ρ⋆\rho^{\star}, and VV denotes the set of pixels at which they are defined.

Ray Module

The module comprises three stages, d0→d1→d\mathbf{d}_0 \to\mathbf{d}_1 \to\mathbf{d}, in which the radial base and the spherical-harmonic residual are predicted from the global token g\mathbf{g}, and the free-form residual from the dense feature map FF.

Radial base models the approximately radially symmetric structure of conventional central cameras with a monotone mapping from image radius to incidence angle.

Monotonicity restricts the base to injective radius-to-angle maps, so no two radii in the modeled range receive the same incidence angle, and the profile is evaluated directly rather than inverted numerically as a parametric projection model would require. A head maps g\mathbf{g} to K+8K + 8 values (Figure 2b): KK increments s∈RK\mathbf{s} \in\mathbb{R}^{K} between fixed knot radii 0=r0<⋯<rK0 = r_0 < \cdots< r_K, with a principal-point offset c\mathbf{c}, an axis-angle rotation ω\boldsymbol{\omega}, and three parameters forming an upper-triangular map A(a)A(\mathbf{a}) for anisotropic scaling and shear. These give the transformed coordinate m′=A(a)(m−c)\mathbf{m}' = A(\mathbf{a})(\mathbf{m}-\mathbf{c}), with radius rr and azimuth φ\varphi. The increments are passed through a softplus to ensure positivity, accumulated into knot heights, and linearly interpolated,

θk=∑j≤ksoftplus⁡(sj),θ0=0,θ(r)=lerp⁡(clip⁡(r;r0,rK);{rk},{θk}),(2)\theta_k = \sum_{j \le k} \operatorname{softplus}(s_j), \qquad\theta_0 = 0, \qquad\theta(r) = \operatorname{lerp}\left(\operatorname{clip}(r; r_0, r_K); \{r_k\}, \{\theta_k\}\right), \tag*{(2)}

so that θ(r)\theta(r) is strictly increasing within the knot span and constant outside it. We use K=16K = 16 knots spanning r∈[0,1.3]r \in[0, 1.3] in half-diagonal units. The corresponding ray is

d0=R(ω)[sin⁡θ(r)cos⁡φsin⁡θ(r)sin⁡φcos⁡θ(r)],(3)\mathbf{d}_0 = R(\boldsymbol{\omega}) \begin{bmatrix} \sin\theta(r)\cos\varphi\\ \sin\theta(r)\sin\varphi\\ \cos\theta(r) \end{bmatrix}, \tag*{(3)}

where R(ω)R(\boldsymbol{\omega}) is the rotation matrix of ω\boldsymbol{\omega}. The largest angle the base can produce is the sum of its increments, so nothing caps θ\theta at 90∘90^\circ, and the base already represents lenses whose rays pass the equator.

Spherical-harmonic residual. The radial base restricts the incidence angle to a monotone function of image radius, which becomes insufficient when the image covers a large portion of the viewing sphere (Figure 3b). A spherical-harmonic residual supplies an angular correction to it (Figure 2b). A head maps g\mathbf{g} to a single coefficient matrix W1∈R(L+1)2×3W_1 \in\mathbb{R}^{(L+1)^2 \times3} for a real spherical-harmonic basis of maximum degree LL, shared across all pixels. Thus, the correction field is represented by 3(L+1)23(L+1)^2 predicted coefficients rather than an independent vector at each pixel. Each pixel evaluates the basis at an address on S2S^2 obtained from its normalized image coordinate through the inverse-stereographic lift,

Why combine the radial construction with the residual corrections?

Figure 3. Why combine the radial construction with the residual corrections? (a) The radial representation gives the same incidence angle to pixels that lie at the same radius in the base coordinates. (b) In the centered 2:1 equirectangular projection, point A is closer to the center of the image than point B, yet has a higher incidence angle. A monotone radial function cannot accommodate such ordering under fixed coordinate transformation and orientation. (c) The spherical-harmonic and free-form residuals improve upon the base direction via iterative correction on the unit sphere.

b(m)=(2m, 1−∥m∥2)1+∥m∥2,w1(u)=W1TΦL(b(m(u))),(4)\mathbf{b}(\mathbf{m}) = \frac{(2\mathbf{m},\,1-\lVert\mathbf{m}\rVert^2)}{1+\lVert\mathbf{m}\rVert^2}, \qquad\mathbf{w}_1(\mathbf{u}) = \mathbf{W}_1^{\mathsf{T}}\Phi_L(\mathbf{b}(\mathbf{m}(\mathbf{u}))), \tag*{(4)}

where ΦL\Phi_L stacks the (L+1)2(L+1)^2 basis values. We use L=7L=7, giving 192 coefficients per image.

The correction is applied through the exponential map on S2S^2. For any ray d\mathbf{d}, only the component of w\mathbf{w} orthogonal to d\mathbf{d} is a tangent direction, so the remaining component is discarded, leaving t=w−⟨w,d⟩d\mathbf{t}=\mathbf{w}-\langle\mathbf{w},\mathbf{d}\rangle\mathbf{d}. Its norm n=∥t∥n=\lVert\mathbf{t}\rVert is the rotation angle in radians, and the ray is transported that far along the great circle through d\mathbf{d} in the direction of t\mathbf{t},

exp⁡d(w)=cos⁡n d+sin⁡nnt.(5)\exp_{\mathbf{d}}(\mathbf{w})=\cos n\,\mathbf{d}+\frac{\sin n}{n}\mathbf{t}. \tag*{(5)}

Free-form residual. The spherical-harmonic residual provides a compact global correction but remains limited by the finite-order basis. The free-form residual captures the remaining local variation using a lightweight convolutional decoder on the dense feature map FF. The resulting correction map is interpolated at each pixel to obtain w2(u)\mathbf{w}_2(\mathbf{u}), so its spatial detail is limited by the resolution of FF. The two residuals are applied in turn through equation 5, giving the composite ray field

d1=exp⁡d0(w1),d=exp⁡d1(w2),(6)\mathbf{d}_1=\exp_{\mathbf{d}_0}(\mathbf{w}_1), \qquad\mathbf{d}=\exp_{\mathbf{d}_1}(\mathbf{w}_2), \tag*{(6)}

with d0\mathbf{d}_0 given by equation 3, and with t1\mathbf{t}_1 and t2\mathbf{t}_2 the tangential parts of w1\mathbf{w}_1 at d0\mathbf{d}_0 and of w2\mathbf{w}_2 at d1\mathbf{d}_1. Both residual heads are zero-initialized, so the composition starts from the radial base, and the norms ∥t1∥\lVert\mathbf{t}_1\rVert and ∥t2∥\lVert\mathbf{t}_2\rVert are the rotation angles each residual applies, measuring how far a camera departs from the representation beneath it and letting the representation adapt without model selection or routing.

Distance Module

The Ray Module determines the direction d\mathbf{d} along which each pixel observes the scene, but not how far along that ray the surface lies. The Distance Module predicts this remaining quantity: the up-to-scale radial distance ρ(u)\rho(\mathbf{u}) from the optical center. It reuses the same encoder pass, fusing four intermediate blocks with a Dense Prediction Transformer (DPT) decoder Ranftl et al. [22] into a dense prediction sampled at each pixel, where a softplus gives a positive distance ρ(u)=softplus⁡(fρ(u))>0\rho(\mathbf{u})=\operatorname{softplus}(f_\rho(\mathbf{u}))>0. Scaling each predicted ray by its distance as in equation 1 produces the camera-frame point cloud {P(u)}\{\mathbf{P}(\mathbf{u})\} in a single forward pass.

Training objective

Angular ray loss supervises the final ray field and the two intermediate fields with the mean cosine residual ℓ(x)=1∣V∣∑i∈V(1−⟨xi,di∗⟩)\ell(\mathbf{x})=\frac{1}{|V|}\sum_{i\in V}(1-\langle\mathbf{x}_i,\mathbf{d}_i^*\rangle),

Lray=ℓ(d)+γ0ℓ(d0)+γ1ℓ(d1)+Lregion(7)\mathcal{L}_{\mathrm{ray}}=\ell(\mathbf{d})+\gamma_0\ell(\mathbf{d}_0)+\gamma_1\ell(\mathbf{d}_1)+\mathcal{L}_{\mathrm{region}} \tag*{(7)}

with stage weights γ0,γ1>0\gamma_0, \gamma_1 > 0. This deep supervision of d0d_0 and d1d_1 keeps each stage responsible for the structure it represents, so a correct final ray cannot be reached through an arbitrary base.

Lregion\mathcal{L}_{\mathrm{region}} weights the same per-pixel error more heavily where it is largest.

With eˉ(S)\bar{e}(S) denoting the mean error over a pixel set SS, it is λceˉ(C)+λpeˉ(Q)+λteˉ(T)\lambda_c \bar{e}(C) + \lambda_p \bar{e}(Q) + \lambda_t \bar{e}(T) over an inner region CC, an outer region QQ, and tail TT of the 5% of pixels with the largest error, with λp\lambda_p the largest of the three since error concentrates at the periphery for wide fields of view. The regions are thresholds on the image radius ∥m∥\lVert m\rVert relative to its maximum, so the split follows the imaged region of a circular fisheye as well as a rectangular frame and does not move with the predicted principal point.

point cloud loss supervises rays and distances jointly. Each cloud is normalized by its own mean point norm, zpred=1∣V∣∑i∈V∥ρidi∥z_{\mathrm{pred}} = \frac{1}{|V|}\sum_{i \in V}\lVert\rho_i d_i\rVert for the prediction and zgt=1∣V∣∑i∈V∥ρi∗di∗∥z_{\mathrm{gt}} = \frac{1}{|V|}\sum_{i \in V}\lVert\rho_i^* d_i^*\rVert for the ground truth, which removes the global scale that a single image leaves undetermined,

Lpc=1∣V∣∑i∈V∥ρidizpred−ρi∗di∗zgt∥22(8)\mathcal{L}_{\mathrm{pc}} = \frac{1}{|V|}\sum_{i \in V}\left\lVert\frac{\rho_i d_i}{z_{\mathrm{pred}}} - \frac{\rho_i^* d_i^*}{z_{\mathrm{gt}}} \right\rVert_2^2 \tag*{(8)}

It is evaluated on P=ρdP = \rho d with the ray attached, so a ray error displaces the point even for an exact distance and the gradient reaches both modules.

Overall objective. L=λrayLray+λpcLpc\mathcal{L} = \lambda_{\mathrm{ray}} \mathcal{L}_{\mathrm{ray}} + \lambda_{\mathrm{pc}} \mathcal{L}_{\mathrm{pc}}. Both modules and the shared encoder are trained end-to-end under this objective. Sec. A.1 gives the head architectures and the full training schedule.

Experiments

Oracle test: representation ceilings

Before training anything, we measure what each candidate representation can express at all, by fitting its parameters directly to analytic ground-truth rays; Sec. A.2 gives the fitting protocol. Table 1 shows that compact parametric models succeed only on the geometries their assumptions match, and that increasing the number of knots in the monotone radial profile reduces the catadioptric error but leaves the equirectangular error at 22.8°, the ceiling identified in § 3.1. The spherical-harmonic field removes that ceiling but remains two orders of magnitude behind the radial profile on the fisheye. Among the compact representations, only the composition is below 0.25° on all three geometries, and since it reduces to the radial base when the residual vanishes, its capacity is bounded below by that of the base.

Representation#ParamsFisheyeEquirectangularCatadioptric
Pinhole65.73148.08050.434
Double sphere80.02122.93241.876
Radial (K=16K = 16)240.00222.8020.104
Radial (K=32K = 32)400.00222.8010.023
SH (L=7L = 7)1950.2760.6410.295
Radial₁₆ + SH₇ residual (ours)2160.0750.2170.174
Free field76.8k0.0050.0130.015

Table 1. Oracle representation fitting. Mean angular error in degrees after fitting each representation directly to ground-truth rays; lower is better. #Params denotes parameters per camera; the SH row adds three rotation parameters to the 3(L+1)2=1923(L + 1)^2 = 192 coefficients for the base direction, which the residual inherits from the radial base. Best results among compact representations are bold. The free field is shown separately as a high-capacity reference.

Datasets and Camera Types

We train on two panoramic datasets, 2D3DS ([1]), of indoor offices and corridors, and 360Loc ([10]), of larger indoor and outdoor environments, rendering perspective, fisheye, and panoramic views from each panorama; the 2D3DS perspective images are the real captures provided with the dataset. We evaluate on held-out scenes of both, and on two unseen datasets, PanoCity (Guo et al. [7]), of outdoor street-level captures, and CO3Dv2 (Reizenstein et al. [23]), of object-centric images. Two camera configurations are withheld from training entirely, a GoPro fisheye and a Nayar catadioptric camera (Nayar (1997); Baker & Nayar [2]), both satisfying our central-camera assumption. Sec. A.2 gives the splits and the camera specifications.

Random-lens Augmentation. During training we augment each panorama with randomly sampled synthetic central cameras, each defined by a viewing direction, a monotone radial profile drawn from positive spline increments, and an imaged sensor region. This single family covers filled and annular domains, from pinhole to wide-angle lenses. Draws whose profile falls close to a held-out geometry are rejected, so neither withheld camera nor its neighborhood appears in training. Sec. A.2 gives the sampling ranges and the rejection criteria, and Figure 8 shows sampled examples.

Random-lens augmentation

Figure 8. Random-lens augmentation. Images rendered from a single panorama under cameras drawn from the sampling distribution of § 4.2. A camera is a viewing direction, a monotone radial profile and an imaged sensor region, so one family spans narrow perspective crops, wide-angle and fisheye views, off-center and anisotropic sensors, and annular domains. The projection varies continuously from sample to sample, so no fixed map from image position to ray direction fits more than one of them. Draws falling close to either held-out geometry are rejected and never enter training.

Benchmark and Ablation

Baselines. UniK3D (Piccinelli et al. [21]) and Depth Anything 3 (DA3) (Lin et al. [16]) jointly predict camera geometry and scene structure and are evaluated on all metrics; we fine-tune UniK3D on the 2D3DS and 360Loc panoramas so that it sees the same source imagery we train on. GeoCalib (Veicht et al. [29]) and AnyCalib (Tirado-Garín & Civera [26]) calibrate the camera without predicting scene geometry. We convert their estimated cameras to dense ray fields for evaluation so all methods are scored identically.

Metrics. We evaluate the predicted ray fields, radial distance, and 3D point clouds using the following metrics: Ray accuracy is the mean angular error arccos⁡(d,d⋆)\operatorname{arccos}(\mathbf{d},\mathbf{d}^{\star}) in degrees, and radial distance is evaluated with absolute relative error (AbsRel) and δ<1.25\delta< 1.25. For point clouds, accuracy and completeness are the median nearest-neighbor distances normalized by the scene’s mean point norm, and their mean is the Chamfer distance (CD). Predictions in different depth gauges are aligned per image by s=median⁡(ρ⋆)/median⁡(ρ)s = \operatorname{median}(\rho^{\star}) / \operatorname{median}(\rho), and perpendicular depth is converted to radial distance by ρ=Z/dz\rho= Z/d_z using the predicted rays. All metrics are computed over pixels with valid ground truth.

Results. Table 2 summarizes performance across camera configurations and domains. On the seen camera types, OmniRay achieves a mean ray error of 0.14∘0.14^\circ and relative Chamfer distance of 0.01790.0179, compared with 6.34∘6.34^\circ and 0.07600.0760 for UniK3D and 33.03∘33.03^\circ and 0.19990.1999 for Depth Anything 3. On the held-out configurations, for the Nayar catadioptric camera, all baselines exceed 75∘75^\circ while ours achieves 1.12∘1.12^\circ and 0.01310.0131 CD; on GoPro, ours achieves 5.53∘5.53^\circ versus 6.34∘6.34^\circ for UniK3D. On PanoCity, ours achieves 1.35∘1.35^\circ ray error versus 21.22∘21.22^\circ and 50.53∘50.53^\circ for UniK3D and DA3, respectively. On CO3Dv2, our 2.97∘2.97^\circ ray error remains within a degree of both baselines.

MethodSeen cameras: Aggregate Ang. (∘^\circ) ↓\downarrowSeen cameras: Aggregate CD ↓\downarrowHeld-out cameras: Catadioptric†^\dagger Ang. (∘^\circ) ↓\downarrowHeld-out cameras: Catadioptric†^\dagger CD ↓\downarrowHeld-out cameras: GoPro†^\dagger Ang. (∘^\circ) ↓\downarrowHeld-out cameras: GoPro†^\dagger CD ↓\downarrowHeld-out domains: PanoCity†^\dagger Ang. (∘^\circ) ↓\downarrowHeld-out domains: PanoCity†^\dagger CD ↓\downarrowHeld-out domains: CO3Dv2†^\dagger Ang. (∘^\circ) ↓\downarrowHeld-out domains: CO3Dv2†^\dagger CD ↓\downarrow
GeoCalib [29]30.997–119.662–14.126–47.132–3.713–
AnyCalib [26]28.655–107.129–10.031–42.335–2.711–
DA3 [16]33.0280.1999121.5661.041216.6370.109750.5330.36772.0410.0288
UniK3D [21]6.3360.076075.7020.59486.3420.045421.2200.21202.6760.0277
OmniRay w/o AUG1.3850.059230.8670.34407.9650.06416.3520.09842.8250.0330
OmniRay + RG0.3560.02051.1180.01135.1800.03751.5920.02772.9350.0411
OmniRay0.1400.01791.1200.01315.5330.03921.3500.02402.9670.0380

Table 2. Main benchmark. Results on seen cameras, held-out cameras, and held-out domains. Ang. denotes mean ray error in degrees; CD denotes median relative Chamfer distance, normalized by each scene’s mean point norm. Metrics are macro-averaged over test images, and all methods use per-image median scale alignment. Lower is better; best results are bold. AUG denotes random-lens augmentation, and RG denotes ray guidance. †^\dagger denotes exclusion from our training data; a dash indicates that the method does not predict scene geometry. Full per-camera and per-dataset results and additional metrics are provided in Sec. A.3.

The w/o AUG row isolates the random-lens sampling of § 4.2. Without it the seen cameras remain strong at 1.39∘1.39^\circ, but the held-out catadioptric and GoPro errors rise to 30.87∘30.87^\circ and 7.97∘7.97^\circ; restoring it recovers 1.12∘1.12^\circ and 5.53∘5.53^\circ and improves the seen cameras to 0.14∘0.14^\circ. Figure 5a shows the same effect on a catadioptric example, where the unaugmented model exhibits larger ray-field and geometric errors. These results indicate that random-lens sampling is important for extending the composite representation beyond the camera configurations encountered during training. Adding ray guidance to each DPT scale following Lin et al. [17] (+ RG) does not change the metrics consistently. The injection is zero-initialized, so the guided head could exploit the prompt if it were beneficial, and we therefore omit it from the final model.

Ablation of ray representations for held-out catadioptric and panoramic cameras

Figure 5. Ablation. (a) Held-out catadioptric camera: predicted ray field and point cloud for UniK3D, OmniRay trained without random-lens augmentation, and the full model, against the ground truth. (b) Panoramic image: the radial base alone covers only part of the sphere (orange) and leaves the reconstruction torn, while the spherical-harmonic and free-form residuals complete the field (blue) and close the geometry.

Figure 4 compares ray fields and camera-frame point clouds on unseen PanoCity and CO3Dv2 images, with each ray field colored by its angular error. On the panorama, OmniRay keeps the road continuous while UniK3D breaks it across the image and DA3 fails to recover the ground structure. On the fisheye and GoPro rows, OmniRay keeps the road, pavement, and lane markings straight where UniK3D bends them and DA3 also distorts the surrounding buildings. On the held-out catadioptric camera both baselines produce degraded reconstructions. On the CO3Dv2 pinhole example, OmniRay remains comparable to the baselines, demonstrating that the additional flexibility for wide-angle cameras does not come at the expense of pinhole imagery. On in-the-wild images, OmniRay also recovers coherent 3D structure, as shown in Figure 6.

Qualitative reconstructions on unseen datasets

Figure 4. Qualitative reconstructions on unseen datasets. We compare OmniRay with Depth Anything 3 and UniK3D on PanoCity and CO3Dv2 across five camera configurations. Small visualizations show viewing-ray fields; larger renderings show the corresponding point clouds, with ground truth in the rightmost column. OmniRay better preserves scene structure in the panoramic and catadioptric examples without camera intrinsics or camera-type input.

In-the-wild reconstructions

Figure 6. In-the-wild reconstructions. Qualitative reconstructions on photographs collected from the internet, shown in order as a panorama, a fisheye image, and a pinhole image. Although these camera families are represented during training, the images come from different camera instances with unknown intrinsics and lens parameters. OmniRay requires no camera metadata at inference.

Ablation. Table 3 isolates and analyzes different ray representations. The radial base performs well on fisheye and pinhole cameras (1.761∘1.761^\circ and 0.788∘0.788^\circ) but gives 21.825∘21.825^\circ on panoramas, where a monotone radial profile assigns one incidence angle per image radius, an ordering that a panoramic projection does not follow (Figure 3b). The spherical base reduces panoramic error to 5.088∘5.088^\circ, while the free-form field performs best among single representations on panoramas, fisheye, and pinhole, but reaches 11.335° on the held-out catadioptric configuration. These results motivate combining a structured radial base with flexible residuals.

Ray representationTrainingPanoramaFisheyePinholeCatadioptricGoPro
Radial base–21.8251.7610.7886.0395.377
Spherical base–5.0881.5400.79013.3275.962
Free-form field–0.7831.5080.62111.3355.191
Radial + SH residualJoint0.7961.7840.5811.8796.566
Radial + SH residualStaged0.6101.5450.6751.6803.393
Radial + free-formJoint1.0181.3940.4033.0383.667
Radial + free-formStaged1.1111.6050.3861.7974.248
SH + free-formJoint1.1141.7471.2502.5683.739
SH + free-formStaged1.0262.5670.5532.3165.142
Radial + SH + free-formJoint0.7281.9630.6882.5253.636
Radial + SH + free-formJoint, γ0=γ1=0\gamma_0=\gamma_1=00.4511.8640.7051.8383.604
Radial + SH + free-formStaged, VGGT init0.4861.6300.3302.5765.096
Radial + SH + free-formStaged0.3991.3220.4181.5383.462

Table 3. Ray representation ablation. Mean angular ray error in degrees (↓) at native resolution. Seen camera families are evaluated on held-out configurations; † denotes configurations excluded from training. Only the Ray Module is trained and evaluated. Best results are bold.

Adding an SH residual to the radial base improves panoramic and held-out accuracy; staged training achieves 0.610° and 1.680°, respectively, and the best GoPro result (3.393°). Adding the free-form residual further improves the panorama (0.399°), fisheye (1.322°), and catadioptric results (1.538°) in the full staged model, which also outperforms both jointly trained three-term variants across all five configurations.

On pinhole cameras, the full model remains competitive at 0.418°. The additional spherical residual offers little benefit for this narrow-field geometry, while potentially providing greater flexibility across wide-field cameras. The VGGT-initialized variant (Wang et al. [30]) illustrates a similar trade-off: it achieves the best pinhole error (0.330°) but performs worse than the full model on the other four configurations. Since VGGT is pretrained for multi-view geometry on perspective imagery, its initialization may favor narrow-field projections over the broader range of camera geometries considered here. As the backbone is fine-tuned end-to-end in both settings, this comparison suggests an effect of initialization rather than a limitation in model capacity.

Overall, these results support the staged radial → SH → free-form composition as a balance between geometric structure and representational flexibility.

Conclusion

We began with two questions: how the pixel-to-ray mapping can remain both compact and expressive, and whether a single representation is necessary for every camera geometry. OmniRay explores both by composing the ray field. A monotone radial base captures the structure of conventional lenses, a spherical-harmonic residual models smooth deviations that a radial profile cannot express, and a free-form residual accounts for remaining local variation. Each component is applied as a rotation on S2S^2, allowing the representation to adapt to different camera geometries without camera-type input or model selection. A shared distance module scales the predicted rays into a point cloud in a single forward pass, and sampling lenses continuously during training helps the composition generalize to held-out camera configurations.

Limitation. A qualitative zero-shot example reveals a limitation on panoramic views that differ from the training distribution. The training images are captured near ground level, with horizons approximately aligned across images. As shown in Figure 7, on an in-the-wild aerial panorama with the horizon near the top of the frame, the predicted rays cover only a bounded, forward-facing region rather than the full viewing sphere. This example suggests that generalization depends not only on camera-projection diversity but also on the distribution of viewing altitudes represented during training.

Aerial panorama, predicted rays, and predicted pointcloud

Figure 7. Limitation: zero-shot in-the-wild reconstruction of an aerial panorama sourced from the internet. With the horizon near the top of the image, the predicted rays cover a bounded, forward-facing region.

Future work. A natural next step is to extend OmniRay to multi-view reconstruction and broaden the training distribution to include views with more varied camera orientations and altitudes.

Acknowledgments

This research is based upon work supported by the Office of the Director of National Intelligence (ODNI), Intelligence Advanced Research Projects Activity (IARPA), via IARPA R&D Contract No. 140D0423C0076. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of the ODNI, IARPA, or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright annotation thereon.

AI Use Statement

In this work, we used generative AI tools to help with the implementation code and to polish grammar in the paper to improve readability. We have reviewed all AI-assisted work. We reviewed LLM-generated text to ensure it was consistent with our findings. We take responsibility for the final content of this work, including text, claims, or artifacts produced with the aid of generative AI.

Appendix

Implementation and training

Architecture. The encoder is DINOv2 ViT-L/14 [20] with 1024-dimensional features. The radial head is a two-layer multilayer perceptron (MLP) with width 256 operating on the global token augmented with (log aspect, log diagonal), while the spherical head is a one-layer MLP of the same width on the same input. The radial head predicts K+8K + 8 values with K=16K = 16 knots spanning r∈[0,1.3]r \in[0, 1.3] in half-diagonal units; the spherical head predicts (L+1)2×3(L+1)^{2\times3} coefficients with L=7L = 7. These coefficients are shared across all pixels of an image. The free head is a convolutional decoder over the dense feature map. The Distance Module uses a width-256 DPT decoder [22] over encoder blocks {4,11,17,23}\{4, 11, 17, 23\} to predict the radial distance. Both residual heads are zero-initialized, while the radial head is initialized to a canonical camera, so training starts from a single radial lens.

Training. Images range from 896×1792896 \times1792 equirectangular panoramas to 5122512^{2} lens crops. We train with a token cap of 8192 using a sampler for camera-type balancing. Within each image, we supervise 4096 pixels drawn round-robin over the occupied cells of a 32×3232 \times32 grid, ensuring coverage across the image. Training proceeds in two phases. We first train the Ray Module alone, activating its three components in order, and then attach the Distance Module at the final stage and train the full model end-to-end. In this second phase we use AdamW with weight decay 0.01 and cosine decay after a warm-up, with learning rates of 5×10−65 \times10^{-6} for the encoder and ray heads and 5×10−55 \times10^{-5} for the distance decoder. The encoder runs in bfloat16, while the heads and losses remain in full precision. Training uses 8 NVIDIA A40 GPUs with distributed data parallelism.

Experimental Details

Oracle fitting protocol. Each representation is fitted to the analytic ground-truth rays of a single camera by minimizing the mean cosine residual in § 3.3, with a multi-start sweep over the axis sign, roll and, where the representation uses one, the focal length. Every row is an independent fit under a fixed optimization budget, so each entry upper-bounds that representation’s ceiling. The fitted error is the representation’s approximation capacity for that camera and lower-bounds the error of any predictor built on it.

Splits and Camera sampling. For 2D3DS we train on areas 1-4 and evaluate on areas 5-6; for 360Loc we train on the Atrium and Piatrium scenes and evaluate on Concourse and Hall. Each synthetic camera samples a viewing orientation, an aspect ratio, a principal-point offset, an optional central blind spot, a rectangular or circular sensor boundary, and inner and outer incidence angles; panoramic cameras instead sample a vertical field of view, an azimuth and a small latitude stretch, with the horizon kept gravity-aligned as in the source captures. Draws near a held-out geometry are rejected: for the GoPro, any profile within δ=15∘\delta= 15^\circ of its incidence-angle curve at every radius; for the catadioptric camera, any sample with an outer angle in [25∘,55∘][25^\circ, 55^\circ] and a rim angle in [140∘,170∘][140^\circ, 170^\circ]. Note: The Nayar catadioptric camera (Nayar, 1997; Baker & Nayar, 1999) is a paraboloid mirror with orthographic projection, imaging a 40∘40^\circ–155∘155^\circ annulus about its axis; the mirror and camera share a single viewpoint. For PanoCity and CO3Dv2 we use the calibration provided with each dataset to obtain ground-truth rays. Ground-truth rays and distances follow analytically from each sampled camera. Figure 8 shows images rendered under cameras drawn from this distribution.

Additional Results

Table 4, Table 5 and Table 6 report every metric of § 4.3 broken out per camera type, per dataset, and over the aggregate seen and held-out splits. Figure 9 and Figure 10 show reconstructions on the 2D3DS and 360Loc test scenes, which share the camera families used in training but none of their scenes, complementing the unseen-dataset comparison of Figure 4. Figure 11 provides additional visualizations of OmniRay on in-the-wild examples.

Qualitative reconstructions on seen dataset: 2D3DS

Figure 9. Qualitative reconstructions on seen dataset: 2D3DS. Ray field: predicted rays on the unit sphere, coloured by angular error to the ground-truth ray (turbo, θ=0°–30°); the ground-truth ray field is coloured by image RGB. Crops: enlarged view of the dashed region, same processing for every column.

Qualitative reconstructions on seen dataset: 360Loc

Figure 10. Qualitative reconstructions on _seen_ dataset: 360Loc

Additional in-the-wild reconstructions

Figure 11. Additional in-the-wild reconstructions.

CameraMethodRay ang∘^\circ ↓\downarrowRadial Dist. AbsRel↓\downarrowδ1↑\delta_1 \uparrowCD↓\downarrowAcc↓\downarrowComp↓\downarrowF@10cm↑\uparrow
pinholeGeoCalib2.196––––––
AnyCalib1.043––––––
DA36.6040.15080.8620.07010.08620.05400.300
UniK3D3.6190.13710.8710.10580.18020.03150.356
ours0.1550.07690.9250.02420.03550.01300.533
fisheyeGeoCalib30.001––––––
AnyCalib22.134––––––
DA328.7270.32250.5620.18880.11110.26650.066
UniK3D7.0550.21550.7680.05430.05690.05170.198
ours0.1540.09060.9240.01590.01970.01210.531
panoramicGeoCalib60.795––––––
AnyCalib62.788––––––
DA363.7520.33520.5470.34080.19560.48590.009
UniK3D8.3360.21540.7740.06780.07230.06330.174
ours0.1100.05930.9490.01370.01630.01110.585
catadioptric†^\daggerGeoCalib119.662––––––
AnyCalib107.129––––––
DA3121.5660.16170.7711.04120.72321.35920.000
UniK3D75.7020.25230.6560.59480.50470.68490.004
ours1.1200.03960.9690.01310.01300.01310.633
GoPro†^\daggerGeoCalib14.126––––––
AnyCalib10.031––––––
DA316.6370.16570.8400.10970.08110.13830.079
UniK3D6.3420.16650.8570.04540.04490.04580.235
ours5.5330.08420.9320.03920.03540.04290.215

Table 4. Per-camera results, all metrics. Macro-averaged over test images. †^\dagger geometry withheld from training. Baselines are median-scale-aligned per image. Acc, Comp and CD are medians of the relative distance, divided by each scene’s mean point norm. δ1\delta_1 is δ<1.25\delta< 1.25. F@10cm is the point-cloud F-score at a 10 cm distance threshold, the harmonic mean of precision and recall over nearest-neighbor distances. A dash means the method predicts no scene geometry.

ScenesMethodRay ang°↓Radial Dist. AbsRel↓δ1\delta_1↑CD↓point cloud Quality Acc↓Comp↓F@10cm↑
2D3DSGeoCalib40.713––––––
2D3DSAnyCalib37.520––––––
2D3DSDA343.7750.21240.7470.36140.17440.54830.224
2D3DSUniK3D19.4320.12720.8640.14870.09810.19940.411
2D3DSours1.6190.05680.9580.02360.02260.02460.767
360LocGeoCalib48.223––––––
360LocAnyCalib42.021––––––
360LocDA348.0640.28720.6290.32120.21210.43030.048
360LocUniK3D19.9790.27090.6760.16010.14550.17470.154
360Locours1.2740.07130.9470.01600.01600.01600.590
PanoCity†GeoCalib47.132––––––
PanoCity†AnyCalib42.335––––––
PanoCity†DA350.5330.18190.7740.36770.33180.40370.001
PanoCity†UniK3D21.2200.19400.8150.21200.27190.15220.015
PanoCity†ours1.3500.08230.9140.02400.03330.01470.141
CO3Dv2†GeoCalib3.713––––––
CO3Dv2†AnyCalib2.711––––––
CO3Dv2†DA32.0410.05730.9500.02880.01990.03770.282
CO3Dv2†UniK3D2.6760.04780.9630.02770.02330.03220.293
CO3Dv2†ours2.9670.08890.9140.03800.02910.04690.173

Table 5. Per-dataset results, all metrics. Macro-averaged over test images. † unseen in training. Conventions as in Table 4. CO3Dv2 contains perspective and fisheye imagery only, so its zero-shot axis is the domain rather than the camera, and its object-masked COLMAP ground truth covers roughly 12% of pixels.

Table 6: Aggregate results over the 4500-sample benchmark. Conventions as in Table 4. Seen cameras are perspective, fisheye and panoramic; the held-out geometries are the catadioptric mirror and the GoPro fisheye.

SplitMethodRay ang∘^\circ ↓\downarrowRadial Dist. AbsRel↓\downarrowδ1↑\delta_1 \uparrowCD↓\downarrowpoint cloud Quality Acc↓\downarrowComp↓\downarrowF@10cm↑\uparrow
allGeoCalib45.356––––––
AnyCalib40.625––––––
DA347.4570.22720.7160.35010.23940.46080.091
UniK3D20.2110.19740.7850.17360.17180.17540.193
ours1.4140.07010.9400.02120.02400.01840.499
seenGeoCalib30.997––––––
AnyCalib28.655––––––
DA333.0280.26950.6570.19990.13100.26880.125
UniK3D6.3360.18930.8040.07600.10310.04880.242
ours0.1400.07560.9330.01790.02380.01200.550
held-out†^\daggerGeoCalib66.894––––––
AnyCalib58.580––––––
DA369.1020.16370.8060.57540.40210.74880.040
UniK3D41.0220.20940.7560.32010.27480.36540.120
ours3.3260.06190.9510.02610.02420.02800.424

Table 6.

References

  1. [1]Iro Armeni, Sasha Sax, Amir R Zamir, and Silvio Savarese. Joint 2d-3d-semantic data for indoor scene understanding. arXiv preprint arXiv:1702.01105, 2017.arxiv.org/abs/1702.01105
  2. [2]Simon Baker and Shree K Nayar. A theory of single-viewpoint catadioptric image formation. International journal of computer vision, 35(2):175–196, 1999.DOI
  3. [3]Oleksandr Bogdan, Viktor Eckstein, Francois Rameau, and Jean-Charles Bazin. Deepcalib: A deep learning approach for automatic intrinsic calibration of wide field-of-view cameras. In Proceedings of the 15th ACM SIGGRAPH European conference on visual media production, pp. 1–10, 2018.
  4. [4]Jakob Engel, Kiran Somasundaram, Michael GoeSele, Albert Sun, Alexander Gamino, Andrew Turner, Arjang Talattot, Arnie Yuan, Bilal Souti, Brighid Meredith, et al. Project aria: A new tool for egocentric multi-modal ai research. arXiv preprint arXiv:2308.13561, 2023.arxiv.org/abs/2308.13561
  5. [5]Girish Chandar Ganesan, Yuliang Guo, Liu Ren, and Xiaoming Liu. Unidac: universal metric depth estimation for any camera. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 26953–26963, 2026.
  6. [6]Christopher Geyer and Kostas Daniilidis. A unifying theory for central panoramic systems and practical implications. In European conference on computer vision, pp. 445–461. Springer, 2000.DOI
  7. [7]Yijing Guo, Mengjun Chao, Luo Wang, Tianyang Zhao, Haizhao Dai, Yingliang Zhang, Jingyi Yu, and Yujiao Shi. Panovggt: Feed-forward 3d reconstruction from panoramic imagery. arXiv preprint arXiv:2603.17571, 2026.arxiv.org/abs/2603.17571
  8. [8]Yuliang Guo, Sparsh Garg, S Mahdi H Miangoleh, Xinyu Huang, and Liu Ren. Depth any camera: Zero-shot metric depth estimation from any camera. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 26996–27006. IEEE, 2025.arxiv.org/abs/2501.02464
  9. [10]Huajian Huang, Changkुन Liu, Yipeng Zhu, Hui Cheng, Tristan Braud, and Sai-Kit Yeung. 360loc: A dataset and benchmark for omnidirectional visual localization with cross-device queries. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 22314–22324, 2024.arxiv.org/abs/2311.17389
  10. [11]Linyi Jin, Jianming Zhang, Yannick Hold-Geoffroy, Oliver Wang, Kevin Blackburn-Matzen, Matthew Sticha, and David Fouhey. Perspective fields for single image camera calibration. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 17307–17316. IEEE, 2023.arxiv.org/abs/2212.03239
  11. [12]Dongki Jung, Jaehoon Choi, Adil Qureshi, Somi Jeong, Dinesh Manocha, and Suyong Yeon. Wid3r: Wide field-of-view 3d reconstruction via camera model conditioning. In European Conference on Computer Vision, pp. 665–684. Springer, 2026.
  12. [14]Bogdan Khomutenko, Gaëtan Garcia, and Philippe Martinet. An enhanced unified camera model. IEEE Robotics and Automation Letters, 1(1):137–144, 2015.
  13. [15]Yiyi Liao, Jun Xie, and Andreas Geiger. Kitti-360: A novel dataset and benchmarks for urban scene understanding in 2d and 3d. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45 (3):3292–3310, 2022.DOI
  14. [16]Haotong Lin, Sili Chen, Junhao Liew, Donny Y Chen, Zhenyu Li, Guang Shi, Jiashi Feng, and Bingyi Kang. Depth anything 3: Recovering the visual space from any views. arXiv preprint arXiv:2511.10647, 2025a.arxiv.org/abs/2511.10647
  15. [17]Haotong Lin, Sida Peng, Jingxiao Chen, Songyou Peng, Jiaming Sun, Minghuan Liu, Hujun Bao, Jiashi Feng, Xiaowei Zhou, and Bingyi Kang. Prompting depth anything for 4k resolution accurate metric depth estimation. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 17070–17080. IEEE, 2025b.arxiv.org/abs/2412.14015
  16. [18]Christopher Mei and Patrick Rives. Single view point omnidirectional camera calibration from planar grids. In Proceedings 2007 IEEE International Conference on Robotics and Automation, pp. 3945–3950. IEEE, 2007.DOI
  17. [20]Maxime Oquab, Timothée Darcet, Théó Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaeddin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023.arxiv.org/abs/2304.07193
  18. [21]Luigi Piccinelli, Christos Sakaridis, Mattia Segu, Yung-Hsu Yang, Siyuan Li, Wim Abbeloos, and Luc Van Gool. Unik3d: Universal camera monocular 3d estimation. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1028–1039. IEEE, 2025.arxiv.org/abs/2503.16591
  19. [22]René Ranftl, Alexey Bochkovskiy, and Vladlen Koltun. Vision transformers for dense prediction. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 12159–12168. IEEE, 2021.arxiv.org/abs/2103.13413
  20. [23]Jeremy Reizenstein, Roman Shapovalov, Philipp Henzler, Luca Sbordone, Patrick Labatut, and David Novotny. Common objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 10901–10911, 2021.arxiv.org/abs/2109.00512
  21. [24]Davide Scaramuzza, Agostino Martinelli, and Roland Siegwart. A toolbox for easily calibrating omnidirectional cameras. In 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 5695–5701. IEEE, 2006.
  22. [25]Thomas Schops, Viktor Larsson, Marc Pollefeys, and Torsten Sattler. Why having 10,000 parameters in your camera model is better than twelve. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2535–2544, 2020.arxiv.org/abs/1912.02908
  23. [26]Javier Tirado-Garín and Javier Civera. Anycalib: On-manifold learning for model-agnostic single-view camera calibration. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 8044–8055. IEEE, 2025.
  24. [27]Vladyslav Usenko, Nikolaus Demmel, and Daniel Cremers. The double sphere camera model. In 2018 International Conference on 3D Vision (3DV), pp. 552–560. IEEE, 2018.DOI
  25. [28]Igor Vasiljevic, Vitor Guizilini, Rares Ambrus, Sudeep Pillai, Wolfram Burgard, Greg Shakhnarovich, and Adrien Gaidon. Neural ray surfaces for self-supervised learning of depth and ego-motion. In 2020 International Conference on 3D Vision (3DV), pp. 1–11. IEEE, 2020.arxiv.org/abs/2008.06630
  26. [29]Alexander Veicht, Paul-Edouard Sarlin, Philipp Lindenberger, and Marc Pollefeys. Geocalib: Learning single-image calibration with geometric optimization. In European Conference on Computer Vision, pp. 1–20. Springer, 2024.arxiv.org/abs/2409.06704
  27. [30]Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Visual geometry grounded transformer. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5294–5306. IEEE, 2025a.arxiv.org/abs/2503.11651
  28. [31]Ruicheng Wang, Sicheng Xu, Cassie Dai, Jianfeng Xiang, Yu Deng, Xin Tong, and Jiaolong Yang. Moge: Unlocking accurate monocular geometry estimation for open-domain images with optimal training supervision. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5261–5271. IEEE, 2025b.arxiv.org/abs/2410.19115
  29. [32]Scott Workman, Connor Greenwell, Menghua Zhai, Ryan Baltenberger, and Nathan Jacobs. Deep-focal: A method for direct focal length estimation. In 2015 IEEE International Conference on Image Processing (ICIP), pp. 1369–1373. IEEE, 2015.DOI
  30. [33]Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10371–10381. IEEE, 2024.arxiv.org/abs/2401.10891
  31. [34]Senthil Yogamani, Ciarán Hughes, Jonathan Horgan, Ganesh Sistu, Padraig Varley, Derek O’Dea, Michal Uricár, Stefan Milz, Martin Simon, Karl Amende, et al. Woodscape: A multi-task, multi-camera fisheye dataset for autonomous driving. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 9308–9318, 2019.arxiv.org/abs/1905.01489
  32. [35]Zhengyou Zhang. A flexible new technique for camera calibration. IEEE Transactions on pattern analysis and machine intelligence, 22(11):1330–1334, 2000.DOI

Paper details

vlnc:W_5ywt7h93
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

generative AI tools

View statement
Looking for related papers…
Contents