betaFeedback

WorksComputer Science

OpenVIO: A Versatile Open-Source Visual-Inertial Odometry System

Abstract

Visual-inertial odometry (VIO) is a fundamental topic in robotics research. Beyond research, VIO has been adopted in a wide range of products. In this manuscript, we present OpenVIO, an open-source VIO system for research and practical applications. It is released under the MIT license. OpenVIO supports pinhole and fisheye cameras and extends this support to stitched 360° images. We describe each part of the system together with its design considerations. We also report benchmark results with other open-source VIO systems on public datasets. OpenVIO is available as open source at https://github.com/93won/openvio.

Introduction

Visual-inertial odometry (VIO) combines camera images with measurements from an inertial measurement unit (IMU) to estimate ego-motion. This cost-effective sensor combination supports reliable motion estimation and has made VIO useful in products such as mobile robots, drones, augmented reality (AR) devices and autonomous cars [4, 5, 6, 7].

In this manuscript, we present OpenVIO, an open-source VIO system with stable performance across diverse camera models. It supports pinhole and fisheye cameras, as well as stitched 360° images with a full spherical field of view (Figure 1). Under the MIT license, it can be used in both research and products. We compare OpenVIO with nine open-source VIO systems on EuRoC and TUM-VI in monocular and stereo configurations. We also evaluate OpenVIO on the 17 flights of PanoAir.

Example images for the camera models supported by OpenVIO: (a) ERP 360°, PanoAir; (b) fisheye, TUM-VI; (c) pinhole, EuRoC

Figure 1. Example images for the camera models supported by OpenVIO. (a) Stitched 360° equirectangular (ERP) image from PanoAir [1]. (b) Fisheye image from TUM-VI [2]. (c) Pinhole image from EuRoC [3].

The rest of this manuscript is organized as follows. Section II reviews related work. Section III describes the OpenVIO pipeline, including the front end, initialization, filtering and extensions for 360° cameras. Section IV reports the benchmark results on EuRoC, TUM-VI and PanoAir. Section V concludes the manuscript.

Related Work

Over the past decade, many open-source VIO systems have been released (Table I). Based on their state estimation methods, these systems fall into two main groups: filter-based and optimization-based approaches. Filter-based systems, such as MSCKF-VIO [8], ROVIO [9] and OpenVINS [14], combine IMU propagation with visual measurement updates in an extended Kalman filter. Optimization-based systems, such as OKVIS [10], VINS-Mono [11], VINS-Fusion [12], SVO Pro [13], Basalt [15] and Kimera-VIO [16], solve a nonlinear least-squares problem over a sliding window of keyframes. These open-source systems have supported further research and the development of practical applications.

SystemLicenseCamera modelsConfiguration
MSCKF-VIO [8]Penn researchpinhole, fisheyestereo
ROVIO [9]BSD-3pinhole, fisheyemono
OKVIS [10]BSD-3pinhole, fisheyemono, stereo
VINS-Mono [11]GPL-3.0pinhole, fisheyemono
VINS-Fusion [12]GPL-3.0pinhole, fisheyemono, stereo
SVO Pro [13]GPL-3.0pinhole, fisheyemono, stereo
OpenVINS [14]GPL-3.0pinhole, fisheyemono, stereo
Basalt [15]BSD-3pinhole, fisheyestereo
Kimera-VIO [16]BSD-2pinhole, fisheyemono, stereo
OpenVIOMITpinhole, fisheye, 360° ERPmono, stereo

Table I. Open-source VIO systems: licenses and supported cameras

OpenVIO uses a filter-based approach. It tracks features and updates a sliding window of past IMU poses with an extended Kalman filter.

OpenVIO

This section describes the OpenVIO pipeline, its main components, and the extensions for stitched 360° cameras.

S. Choi, [email protected]. The code is available under the MIT license at https://github.com/93won/openvio.

Overview

OpenVIO processes monocular or stereo images and IMU samples (Figure 2). The front end tracks features for initialization and subsequent filter updates. OpenVIO outputs the IMU pose at each image exposure. When a pose leaves the filter window, OpenVIO exports it as a keyframe with its feature observations.

Figure 2. Processing of one image in OpenVIO. The dashed camera-1 and stereo-matching boxes indicate stereo-only steps. The gray dashed feedback arrow carries predicted pixels to landmark search. Pale dashed borders group processing stages.

Each camera model converts a pixel into a unit ray, a unit-length direction vector in the camera frame. The filter uses rays for geometric operations and the camera model’s projection for image residuals. This interface supports directions behind the camera and lets the same estimator handle all camera models in Table I.

Front End

The front end tracks features by pyramidal Lucas–Kanade optical flow [17], [18] with a forward-backward check, which rejects a match when tracking it back gives a pixel too far from its original position. When auto exposure changes the brightness, the previous image is scaled by the ratio of the two median intensities before optical flow.

A RANSAC [19] rotation fit between unit rays [20] provides a further outlier check. The fit checks angular agreement across tracks for every camera model. The filter subsequently estimates both rotation and translation. Each grid cell keeps a limited number of features, with priority given to longer tracks. This preserves observation histories while preventing features from concentrating in a small image region. New corners [21] fill empty cells.

A landmark that the tracker loses can be found again. Each landmark stores a patch from its most recent accepted observation. The front end matches this patch by normalized cross-correlation around the pixel predicted by the filter. The patch is sampled on the tangent plane of the landmark’s ray to account for lens distortion and ERP image stretching. For ERP images, this sampling continues across the left and right image edges. A recovered match retains the landmark’s identity and must still pass the filter’s measurement check.

For stereo, optical flow also matches each feature into the second image. The initial pixel is predicted by rotating and projecting its ray into the second camera. This assumes a point at infinity, so the prediction uses only rotation. Optical flow then finds the actual correspondence, including its disparity. The second observation enters the filter with the calibrated translation between the cameras.

Initialization

Initialization supplies the IMU orientation, velocity, biases and covariance needed by the filter. OpenVIO buffers feature tracks and IMU samples, and selects stationary or dynamic initialization based on motion estimated by feature tracking. The initializer uses tracks from the first camera (camera 0) in both monocular and stereo modes. It compares the mean feature displacement in the older and newer halves of the buffer. A stationary interval followed by motion triggers stationary initialization; a moving interval uses dynamic initialization. If both halves remain stationary, it waits for motion.

Stationary initialization. The accelerometer variation must also confirm the transition from rest to motion. The mean gyroscope reading of the still interval gives the gyroscope bias. The mean accelerometer reading defines the initial gravity direction, and its magnitude relative to the known gravity magnitude gives the accelerometer bias component along that direction. Position and velocity are initialized to zero at the end of the still interval. Buffered IMU samples then propagate this state through the subsequent motion.

Dynamic initialization. OpenVIO first solves the linear initialization problem described in [22]. The unknown vector z\mathbf{z} contains the feature positions {pFj}\{\mathbf{p}_{F_j}\}, initial velocity v0\mathbf{v}_0 and gravity vector g0\mathbf{g}_0, expressed in the first IMU frame. Unit-ray observations and IMU preintegration form A\mathbf{A} and b\mathbf{b}, giving

z^=arg min⁡z∥Az−b∥2,subject to ∥g0∥=g,(1)\hat{\mathbf{z}} = \underset{\mathbf{z}}{\operatorname{arg\,min}}\left\|\mathbf{A}\mathbf{z} - \mathbf{b}\right\|^2,\qquad\text{subject to }\left\|\mathbf{g}_0\right\| = g, \tag*{(1)}

where gg is the known gravity magnitude. The camera–IMU extrinsics and bias priors are fixed in this linear step. The initializer requires enough tracked features, IMU samples and accumulated rotation.

The linear solution seeds a joint refinement of the poses, velocities, IMU biases and feature positions, collected in X\mathcal{X}. Ceres [23] minimizes

X^=arg min⁡X[∥rprior∥2+∑k∥rIMU,k∥2+∑k,jρ(∥rimage,kj∥2)].(2)\hat{\mathcal{X}} = \underset{\mathcal{X}}{\operatorname{arg\,min}}\left[\left\|\mathbf{r}_{\mathrm{prior}}\right\|^2 + \sum_k \left\|\mathbf{r}_{\mathrm{IMU},k}\right\|^2 + \sum_{k,j}\rho\left(\left\|\mathbf{r}_{\mathrm{image},kj}\right\|^2\right)\right]. \tag*{(2)}

The residuals are normalized by their uncertainty. They comprise a prior on the first pose and biases, IMU constraints between consecutive poses, and reprojection errors of feature jj at pose kk. The prior sets the position and yaw reference, while the Cauchy loss ρ\rho limits the influence of image outliers. After convergence, the newest IMU state and its marginal covariance initialize the filter. Selected covariance blocks are inflated to avoid an overconfident start. If refinement or covariance recovery fails, OpenVIO collects more data and tries again.

Startup calibration. Before normal filter updates, OpenVIO uses the buffered data to seed the enabled time-offset states. Camera rotations from feature tracks are compared with gyroscope rotations over time-shifted intervals, with a constant gyroscope bias accounted for. For a two-lens ERP camera, the difference between the two estimated offsets gives the back-lens exposure offset (§ 1.2-E). A time estimate is accepted only when the data constrain it sufficiently; otherwise, the correction starts at zero and can be estimated online.

When the recording starts at rest, an optional correction further refines the accelerometer bias component along gravity. This correction combines the mean accelerometer readings over the buffered interval with the initialized vertical velocity.

Filter

OpenVIO combines a multi-state constraint Kalman filter (MSCKF) [24] with persistent landmarks in its state [25]. This uses constraints from completed tracks and repeated observations of selected landmarks. Grouped by variable type, the state is

x=[xIT,td,δtb,xcalT,xC1T,…,xCnT,xL1T,…,xLmT]T,(3)\mathbf{x} = \left[\mathbf{x}_{I}^{\mathsf{T}}, t_d, \delta t_b, \mathbf{x}_{\mathrm{cal}}^{\mathsf{T}}, \mathbf{x}_{C_1}^{\mathsf{T}}, \ldots, \mathbf{x}_{C_n}^{\mathsf{T}}, \mathbf{x}_{L_1}^{\mathsf{T}}, \ldots, \mathbf{x}_{L_m}^{\mathsf{T}}\right]^{\mathsf{T}}, \tag*{(3)}

with the IMU state xI\mathbf{x}_I (orientation, position, velocity and the gyroscope and accelerometer biases), the time offset tdt_d, the exposure delay δtb\delta t_b of the back lens of a 360° camera, the calibration xcal\mathbf{x}_{\mathrm{cal}} (camera–IMU extrinsics and the stitching error kk of the ERP model), the clones xCi\mathbf{x}_{C_i} of past IMU poses and the landmarks xLj\mathbf{x}_{L_j} in inverse depth [26]. Figure 3a shows the order of addition; Eq. (3) groups variables by type.

Figure 3. State and window. (a) Clones and landmarks appear in their order of addition. Orange marks two-lens 360° camera states; the dashed box marks the online ERP parameter. (b) Lost tracks and tracks reaching the oldest clone update the window. Accepted long tracks become landmarks, updated at each observation. The oldest clone leaves as a keyframe.

The filter propagates the IMU state with fourth-order Runge–Kutta integration and the covariance with the closed-form method of [27], using first-estimate Jacobians [28]. At each image stamped tt, the IMU pose at its exposure t+tdt + t_d is added as a clone [29]. The clone preserves the pose and its correlations with the rest of the state, so later observations can constrain the motion between image exposures. Image residuals use current estimates, while the associated Jacobians use the retained first estimates.

Tracks are triangulated from unit rays, with the point closest to all rays as the initial estimate. For inverse-depth refinement [24], the anchor camera frame and each observation’s camera frame are rotated so that their respective measured rays align with the local +z+z axis. This parameterization handles rays in any direction, including those behind an ERP camera.

A track that is lost or reaches back to the oldest clone updates the window (Figure 3b). Its point is triangulated, and nullspace projection removes the point’s contribution from the linearized residuals [24]. Tracks that pass a χ2\chi^2 gate are combined into one update and compressed by QR decomposition. The resulting constraint updates the state without retaining the triangulated point as a state variable. Once consumed, those track observations are removed to prevent their reuse.

A track seen in more images than the window holds is a candidate for a persistent landmark. It must pass triangulation and the measurement gate, and space must remain in the landmark budget. An optional depth-uncertainty test also prevents poorly constrained points from entering the state; such tracks can still contribute a window update. Each accepted landmark retains its inverse depth and correlations with the other states, allowing new observations to constrain motion after the original clones leave the window. Shorter tracks that are still growing wait for more observations.

After the update, the oldest clone leaves as a keyframe when the window exceeds its limit. The released configurations keep up to 20 clones and 150 landmarks. The time offset and camera–IMU extrinsics are estimated online.

A landmark whose track is lost stays in the state for up to 5 s, so that the front end can find it again. It is removed when its inverse depth is not positive, when the gate rejects it twice in a row, or when its inverse depth departs too far from its first estimate, to limit scale errors from inaccurate first-estimate Jacobians.

360° Cameras

OpenVIO takes an ERP image already stitched by a two-lens 360° camera. The two back-to-back fisheye lenses cover the sphere, and the ERP image maps the longitude of a ray to its column and the latitude to its row:

u=W(12+atan2⁡(x,z)2π),v=H(12+arcsin⁡(y/∥p∥)π).(4)u = W\left(\frac{1}{2} + \frac{\operatorname{atan2}(x,z)}{2\pi}\right), \qquad v = H\left(\frac{1}{2} + \frac{\arcsin(y/\lVert\mathbf{p}\rVert)}{\pi}\right). \tag*{(4)}

Here, WW and HH are the image width and height in pixels. The ray p=(x,y,z)T\mathbf{p} = (x,y,z)^{\mathsf{T}} uses camera axes pointing right (+x+x), down (+y+y) and forward (+z+z). The front lens sees the rays with z>0z > 0 and the back lens the others. For such a camera OpenVIO estimates two more states, the stitching error kk and the exposure delay δtb\delta t_b of the back lens (Figure 3a). The first corrects the pixel-to-ray geometry, while the second determines which pose predicts a back-lens observation. Both are estimated within the same filter as the trajectory.

Stitching error. The stitching relies on the camera’s own calibration, and a small error in it places rays at slightly wrong angles from their lens axis. For a ray at angle θ\theta from that axis, OpenVIO models the error with one parameter kk (Figure 4).

Figure 4. Stitching error of a two-lens ERP image. (a) A ray at the angle θ\theta from its lens axis is placed at θ′=θ+kg(θ)\theta^\prime= \theta+ k g(\theta) (kk exaggerated). (b) The error for k=0.0063k = 0.0063, the prior on PanoAir; the pixel equivalent is at the ERP equator.

θ′=θ+k g(θ),g(θ)=θ(1−(2θπ)2).(5)\theta^\prime= \theta+ k\,g(\theta), \qquad g(\theta) = \theta\left(1-\left(\frac{2\theta}{\pi}\right)^2\right). \tag*{(5)}

The same angular correction is applied about each lens axis without changing the ray’s azimuth. The correction k g(θ)k\,g(\theta) vanishes at the lens axis and at the seam, so these directions remain fixed. Near the axis, it grows like kθk\theta, as a focal-length error would. Thus, one state captures a smooth angular mismatch in the stitched image. For k=0.0063k = 0.0063, the prior on PanoAir, the correction peaks at 0.22∘0.22^\circ at θ=52∘\theta= 52^\circ, equivalent to 0.780.78 px along the ERP equator at an image width of 1280 pixels (Figure 4b). The filter estimates kk online.

Exposure delay. A stitched image can combine two lens exposures at different times (Figure 5). During motion, each half then corresponds to a different pose. The common camera–IMU offset tdt_d sets the front exposure t+tdt + t_d, and the relative offset δtb\delta t_b sets the back exposure t+td+δtbt + t_d + \delta t_b. OpenVIO retains the front-exposure pose as a clone and predicts the back-exposure pose with constant linear velocity between exposures,

Figure 5. Timing of one stitched image. The front and back lenses are exposed at t+tdt + t_d and t+td+δtbt + t_d + \delta t_b, respectively. The diagram shows a positive δtb\delta t_b; back-lens rays use the pose at their own exposure.

Ri,b=ΔRiTRi,pi,b=pi+viδtb,(6)R_{i,b} = \Delta R_i^{\mathsf{T}}R_i, \qquad\mathbf{p}_{i,b} = \mathbf{p}_i + \mathbf{v}_i\delta t_b, \tag*{(6)}

Here, RiR_i maps global vectors into the IMU frame at the front exposure, while pi\mathbf{p}_i and vi\mathbf{v}_i are expressed in the global frame. The gyroscope-derived ΔRi\Delta R_i maps vectors from the back-exposure IMU frame into the front-exposure IMU frame. The filter refines δtb\delta t_b online, allowing either sign.

Benchmark

Setup

We compare OpenVIO with the nine open-source VIO systems in Table 1 on EuRoC MAV [3] and TUM-VI [2], using the available monocular and stereo configurations. EuRoC contains 11 drone sequences recorded with a stereo pinhole camera. We use the 20 indoor TUM-VI sequences, recorded with a hand-held stereo fisheye camera at 512×512512 \times512 pixels. Each baseline uses its authors’ configuration with loop closure disabled. We omit a system from a dataset when its authors provide no configuration for it. OpenVIO uses its released configurations, with the parameter variations described below.

Figure 6. Median of the per-sequence ATE values over the sequences that all systems in each panel complete. The underlying values are the medians across runs in Tables II, III and IV. Parentheses show completed sequences out of all evaluated sequences.

SequenceOpenVIOOpenVINSVINS-MonoVINS-FusionSVO ProROVIOKimera-VIO
MH_010.1680.0960.1920.1860.4380.4551.352
MH_020.0760.1520.1670.0880.094×0.280
MH_030.1180.1240.2170.1358.4880.4860.471
MH_040.1100.2230.3780.217×1.0035.282
MH_050.4290.6240.3540.3260.2161.1646.148
V1_010.0610.0690.0790.0590.0530.1182.555
V1_020.0640.0580.0980.0860.0600.1840.100
V1_030.0600.0660.1470.1600.1310.2010.213
V2_010.0760.1040.085×0.080×0.290
V2_020.0670.0670.1220.1190.1310.7430.198
V2_030.1110.1780.2890.2080.2480.2200.450
Mean (8 seq.)0.1350.1600.1870.1601.2210.4461.436
Completed11/1111/1111/1110/1110/119/1111/11

Table II. EuRoC, monocular. ATE RMSE [m] after SE(3) alignment, median of five runs. ×: at least half the runs failed. The mean uses the eight sequences completed by all systems. Best per row in bold.

We report absolute trajectory error (ATE) as the root-mean-square error (RMSE) of positions after rigid SE(3) alignment [30]. For TUM-VI, only poses with motion-capture ground truth enter the alignment and error calculation. On sequences that leave and return to the room, this measures accumulated drift at the observed portions rather than error along the unobserved route.

Each table entry is the median of five runs for EuRoC and TUM-VI room sequences, or three runs for the remaining TUM-VI sequences. A run fails if no valid ATE is available, its ATE exceeds 10 m10\,\mathrm{m}, or its evaluated time span covers less than half the sequence duration. Failed runs enter the median as an infinite error. A sequence is counted as completed when more than half its runs succeed.

OpenVIO is deterministic, so its repeated runs use small parameter variations. The first three runs use the released configuration and two variants that scale both gyroscope and accelerometer noise densities by 0.9 or 1.1. For the five-run sets, two further runs retain the default IMU parameters and change the optical-flow termination tolerance from 0.01 to 0.0099 and 0.0098. The recorded sensor measurements remain the same in every run.

Accuracy on EuRoC and TUM-VI

Tables II, III and IV report the per-sequence results. For each dataset and camera configuration, aggregate errors use only sequences completed by all compared systems. Figure 6 shows the median across these sequences, while the table summaries show the mean. Completion counts cover all evaluated sequences.

OpenVIO completes all EuRoC and TUM-VI sequences in both monocular and stereo modes under the criterion above. On EuRoC, its median ATE is close to OpenVINS monocular and to Basalt and OpenVINS stereo. On TUM-VI, OpenVIO has the lowest median ATE over the common sequences in both configurations. These results show consistent sequence completion with competitive accuracy across the two datasets.

SequenceOpenVIOOpenVINSVINS-FusionBasaltOKVISMSCKF-VIOSVO ProKimera-VIO
MH_010.0910.0700.2530.0660.213×0.1200.199
MH_020.0690.1110.2170.0550.1200.1780.0650.197
MH_030.0750.1440.3260.0620.1350.2260.0770.321
MH_040.0910.1550.4340.1140.1800.2230.176×
MH_050.2100.4410.3070.1450.2970.3450.1820.257
V1_010.0490.0620.1130.0430.0460.0860.0560.058
V1_020.0420.0480.1080.0450.0540.1120.0770.058
V1_030.0490.0580.1120.0530.1100.1880.0750.090
V2_010.0680.0500.1320.0390.0560.0840.0910.059
V2_020.0560.0350.1240.0490.0800.1520.1350.126
V2_030.1250.0850.3180.229×1.9400.202×
Mean (8 seq.)0.0770.1190.1800.0610.1120.1710.0950.146
Completed11/1111/1111/1111/1110/1110/1111/119/11

Table III. EuRoC, stereo. ATE RMSE [m] after SE(3) alignment, median of five runs. ×: At least half the runs failed. The mean uses the eight sequences completed by all systems. Best per row in bold.

SequenceOpenVIOOpenVINSVINS-MonoOpenVIOOpenVINSBasalt
MonocularStereo
room10.0330.0480.0560.0340.0560.106
room20.0440.0820.0560.0410.0880.076
room30.0790.0750.1490.0640.0800.133
room40.0340.0270.0440.0280.0270.052
room50.0650.0750.2160.0680.0960.180
room60.052×0.0770.049×0.023
corridor10.6790.8170.9870.2410.7960.207
corridor20.2120.2441.2480.3410.4030.412
corridor30.3461.0921.4350.3811.0530.493
corridor40.0820.3910.2300.1210.4720.199
corridor50.227×0.4990.262×0.384
slides10.261×0.4860.3250.9010.338
slides21.214×1.0520.496×0.949
slides30.446×0.6281.1492.1921.769
magistrale12.143×3.5822.441×1.720
magistrale21.3134.0802.4091.2302.9621.274
magistrale32.577×0.6682.3540.8121.964
magistrale41.2612.4294.3742.3111.7731.352
magistrale51.520×0.2251.392×0.955
magistrale62.139×2.7632.0843.6223.737
Mean (common seq.)0.3770.8511.0190.7181.0220.819
Completed20/2011/2020/2020/2015/2020/20

Table IV. TUM-VI 512, monocular and stereo. ATE RMSE [m] after SE(3) alignment, median of the runs (5 per room, 3 otherwise); ×: Half or more of the runs failed. Best per row and configuration in bold. The means are over the sequences that all systems of a configuration complete (monocular 11, stereo 15).

360° Camera

The comparison systems in Table I do not support stitched 360° ERP input, so we evaluate OpenVIO alone on PanoAir [1]. It contains 17 drone flights totaling 15.5 km under cloudy, sunny and night conditions, with RTK-GNSS ground truth. Images from a two-lens camera have an ERP resolution of 1280×6401280 \times640 at 30 Hz. We use one run per flight with the released configuration and exclude the first 12.5 s from evaluation. OpenVIO produces a successful result on all 17 flights, with a mean ATE of 1.14 m (Table V).

CloudySunnyNightMean
Flight1234567891011121314151617
Length [m]248802114312061591792500839170036447240746483190313551883
ATE0.290.611.081.902.990.390.360.581.711.390.810.740.531.201.611.471.741.14
k×103k \times10^{3}6.316.676.346.176.466.433.626.396.736.615.885.598.765.786.055.506.58
tdt_d [ms]−1.8-1.8−4.1-4.1−4.2-4.2−3.8-3.8−4.7-4.7−6.3-6.3−3.3-3.3−5.1-5.1−6.8-6.8−39.0-39.0−4.3-4.3−3.5-3.5−3.6-3.6−5.2-5.2−4.6-4.6−5.4-5.4−6.9-6.9
δtb\delta t_b [ms]0.310.270.150.150.410.160.060.220.1535.00−0.17-0.17−0.18-0.18−0.15-0.15−0.13-0.13−0.16-0.160.02−0.20-0.20

Table V. PanoAir: ATE RMSE [m] of OpenVIO and the online calibration at the end of each flight. One run per flight with the released configuration; the first 12.5 s of each flight are excluded from evaluation. Calibration: final online estimates of the stitching parameter kk (prior (6.3±1.5)×10−3(6.3 \pm1.5) \times10^{-3}), the camera time offset tdt_d and the back-lens delay δtb\delta t_b. The kk prior uses earlier online estimates from this dataset.

Table V also reports the final online calibration estimates. No reference values are available for these calibration parameters. The stitching parameter kk lies between 0.0055 and 0.0068 on 15 flights, with flights 7 and 13 outside this range. The estimated back-lens delay has magnitude below 0.5 ms on every flight except flight 10. For that flight, the estimate is 35 ms, approximately one frame interval at 30 Hz.

Conclusion

OpenVIO is an MIT-licensed visual-inertial odometry system for pinhole, fisheye and stitched 360° ERP images. A shared camera interface supports these models within one filter, with online correction of stitching error and inter-lens exposure delay for two-lens 360° cameras. OpenVIO meets the benchmark success criterion on all evaluated EuRoC and TUM-VI sequences in monocular and stereo modes, and on all 17 PanoAir flights. Loop closure and map reuse are not included; exported keyframes and feature observations provide an interface for such extensions.

References

  1. [1]Y. Wu, X. Zhang, Y. Du, T. Zhang, C. Li, S. Chen, G. Zhang, and X. Xu, “PanoAir: A panoramic visual-inertial SLAM with cross-time real-world UAV dataset,” arXiv:2604.00852, 2026.arxiv.org/abs/2604.00852
  2. [2]D. Schubert, T. Goll, N. Demmel, V. Usenko, J. Stückler, and D. Cremers, “The TUM VI benchmark for evaluating visual-inertial odometry,” in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS), 2018, pp. 1680–1687.
  3. [3]M. Burri, J. Nikolic, P. Gohl, T. Schneider, J. Rehder, S. Omari, M. W. Achtelik, and R. Siegwart, “The EuRoC micro aerial vehicle datasets,” Int. J. Robot. Res., vol. 35, no. 10, pp. 1157–1163, 2016.DOI
  4. [4]K. J. Wu, C. X. Guo, G. Georgiou, and S. I. Roumeliotis, “VINS on wheels,” in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), 2017, pp. 5155–5162.DOI
  5. [5]Apple Inc., “Understanding world tracking,” ARKit documentation, https://developer.apple.com/documentation/arkit/understanding-world-tracking, accessed Oct. 2026.
  6. [6]Google, “ARCore fundamental concepts,” https://developers.google.com/ar/develop/fundamentals, accessed Oct. 2026.
  7. [7]X. Jiang, F. Wang, R. Zheng, H. Liu, Y. Huo, J. Peng, L. Tian, and E. Barsoum, “VIPS-Odom: Visual-inertial odometry tightly-coupled with parking slots for autonomous parking,” arXiv:2407.05017, 2024.
  8. [8]K. Sun, K. Mohta, B. Pfanner, M. Watterson, S. Liu, Y. Mulgaonkar, C. J. Taylor, and V. Kumar, “Robust stereo visual inertial odometry for fast autonomous flight,” IEEE Robot. Autom. Lett., vol. 3, no. 2, pp. 965–972, 2018.
  9. [9]M. Bloesch, S. Omari, M. Hutter, and R. Siegwart, “Robust visual inertial odometry using a direct EKF-based approach,” in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS), 2015, pp. 298–304.
  10. [10]S. Leutenegger, S. Lynen, M. Bosse, R. Siegwart, and P. Furgale, “Keyframe-based visual-inertial odometry using nonlinear optimization,” Int. J. Robot. Res., vol. 34, no. 3, 2015.DOI
  11. [11]T. Qin, P. Li, and S. Shen, “VINS-Mono: A robust and versatile monocular visual-inertial state estimator,” IEEE Trans. Robot., vol. 34, no. 4, 2018.
  12. [12]T. Qin, J. Pan, S. Cao, and S. Shen, “A general optimization-based framework for local odometry estimation with multiple sensors,” arXiv:1901.03638, 2019.
  13. [13]C. Forster, Z. Zhang, M. Gassner, M. Werlberger, and D. Scaramuzza, “SVO: Semidirect visual odometry for monocular and multicamera systems,” IEEE Trans. Robot., vol. 33, no. 2, pp. 249–265, 2017.
  14. [14]P. Geneva, K. Eckenhoff, W. Lee, Y. Yang, and G. Huang, “OpenVINS: A research platform for visual-inertial estimation,” in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), 2020.
  15. [15]V. Usenko, N. Demmel, D. Schubert, J. Stückler, and D. Cremers, “Visual-inertial mapping with non-linear factor recovery,” IEEE Robot. Autom. Lett., vol. 5, no. 2, pp. 422–429, 2020.
  16. [16]A. Rosinol, M. Abate, Y. Chang, and L. Carlone, “Kimera: an open-source library for real-time metric-semantic localization and mapping,” in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), 2020, pp. 1689–1696.
  17. [17]B. D. Lucas and T. Kanade, “An iterative image registration technique with an application to stereo vision,” in Proc. Int. Joint Conf. Artif. Intell. (IJCAI), 1981, pp. 674–679.
  18. [18]J.-Y. Bouguet, “Pyramidal implementation of the affine Lucas Kanade feature tracker: Description of the algorithm,” Intel Corporation, Tech. Rep., 2001.
  19. [19]M. A. Fischler and R. C. Bolles, “Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM, vol. 24, no. 6, pp. 381–395, 1981.DOI
  20. [20]K. S. Arun, T. S. Huang, and S. D. Blostein, “Least-squares fitting of two 3-D point sets,” IEEE Trans. Pattern Anal. Mach. Intell., vol. PAMI-9, no. 5, pp. 698–700, 1987.
  21. [21]J. Shi and C. Tomasi, “Good features to track,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 1994, pp. 593–600.DOI
  22. [22]P. Geneva and G. Huang, “Revisiting visual-inertial structure-from-motion for odometry and SLAM initialization,” IEEE Robot. Autom. Lett., vol. 8, no. 4, 2023.
  23. [23]S. Agarwal, K. Mierle, and The Ceres Solver Team, “Ceres Solver,” http://ceres-solver.org.
  24. [24]A. I. Mourikis and S. I. Roumeliotis, “A multi-state constraint Kalman filter for vision-aided inertial navigation,” in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), 2007.
  25. [25]M. Li and A. I. Mourikis, “Optimization-based estimator design for vision-aided inertial navigation,” in Proc. Robot.: Sci. Syst. (RSS), 2012.
  26. [26]J. Civera, A. J. Davison, and J. M. M. Montiel, “Inverse depth parametrization for monocular SLAM,” IEEE Trans. Robot., vol. 24, no. 5, 2008.
  27. [27]Y. Yang, B. P. W. Babu, C. Chen, G. Huang, and L. Ren, “Analytic combined IMU integration (ACI²) for visual inertial navigation,” in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), 2020.
  28. [28]M. Li and A. I. Mourikis, “Online temporal calibration for camera-IMU systems: Theory and algorithms,” Int. J. Robot. Res., vol. 33, no. 7, 2014.
  29. [29]——, “Online temporal calibration for camera-IMU systems: Theory and algorithms,” Int. J. Robot. Res., vol. 33, no. 7, 2014.
  30. [30]Z. Zhang and D. Scaramuzza, “A tutorial on quantitative trajectory evaluation for visual-inertial odometry,” in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS), 2018, pp. 7244–7251.

Paper details

vlnc:W_67ua3b7t
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 6

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents