betaFeedback

WorksComputer Science

When to Ask for Help: Uncertainty-Triggered Aerial Refinement for Off-Road Navigation

Abstract

Heterogeneous air-ground teams can use aerial sensing to address the limited range and viewpoint of ground-vehicle sensors, but deciding when the UGV needs assistance and what information the UAV should collect remains a challenge. This paper presents a planner-conditioned air-ground assistance method that evaluates how uncertain map information affects the UGV's current MPPI rollouts. When navigation remains degraded, we score the same rollouts under resolved map hypotheses, select the map layer with the largest useful improvement, request the corresponding aerial measurement, fuse the returned information, and replan. Unlike approaches driven by fixed support roles, task structure, or uncertainty alone, our method conditions assistance on whether resolving uncertainty improves predicted ground navigation. Across 180 simulation trials on three off-road routes, the method achieves 98.9% goal attainment while reducing mean UAV flight time by 87.7% relative to continuous greedy aerial mapping. Aggregated across the routes, each second of estimated UAV flight time saves 2.94 s of UGV navigation time. Field experiments spanning approximately 2 km of UGV operation demonstrate uncertainty resolution and successful navigation recovery.

Introduction

Ground vehicles operating alone in outdoor, unstructured environments are limited by their sensing perspective, mobility, and onboard perception. Ground-level sensors have limited range, suffer from occlusion, and can provide incomplete or ambiguous information about terrain, obstacles, and vehicle state. An unmanned ground vehicle (UGV) may therefore fail to complete an otherwise achievable mission because it cannot obtain the information needed to navigate safely and reliably. Heterogeneous robot teams address this by combining complementary sensing and mobility. An unmanned aerial vehicle (UAV) can observe the environment and the ground vehicle from viewpoints unavailable to the UGV and provide information that improves the ground robot’s navigation state. However, UAV assistance is limited by flight endurance and cannot be used continuously throughout long missions. The key problem is therefore deciding when the UGV needs assistance and what information the UAV should provide.

This paper presents a heterogeneous air–ground teaming approach for recovering degraded UGV navigation caused by incomplete or uncertain information in off-road environments. The UGV maintains separate navigation map layers together with their associated uncertainty. When navigation degrades, we evaluate the current MPPI rollout population [1] against counterfactual versions of these map layers and score how the resulting map changes affect predicted ground motion. We

then select the corresponding UAV sensing action for the map layer with the largest useful improvement. The UAV collects the requested aerial information, such as an elevation map or semantic map, and communicates it to the UGV. The UGV fuses the information into its navigation maps, updates the planner representation, and replans.

Existing air–ground systems use heterogeneous sensing for cooperative localization, mapping, exploration, and navigation. Many approaches assign the UAV a fixed support role, invoke aerial sensing according to predefined task logic, or trigger additional sensing from uncertainty in a perception output [2]. These sensing criteria do not directly determine whether the uncertainty affects the UGV’s current navigation decision. Uncertainty can be large in regions that have no effect on the current route, while localized uncertainty can prevent the UGV from identifying or executing a safe path. We therefore evaluate uncertainty through its effect on ground navigation and request UAV assistance only when resolving uncertainty is predicted to improve the UGV’s ability to make progress.

Our main contribution, illustrated in Figure 1, is a planner-conditioned air–ground assistance method that determines when the UAV should sense by evaluating resolved map hypotheses against current UGV rollouts and selecting an aerial sensing action based on the change in navigation score. Across 180 simulation trials, our method achieves 98.9% goal attainment with 87.7% less mean UAV flight time than continuous greedy mapping. Across three routes, each estimated UAV flight second saves 2.94 s of UGV navigation time. Field experiments spanning approximately 2 km of autonomous UGV operation through unstructured routes demonstrate repeated resolution of map uncertainty and resumed navigation following aerial assistance.

Planner-conditioned air–ground assistance method

Figure 1. The presented planner-conditioned air–ground assistance method that enables a UGV to navigate highly uncertain off-road environments by requesting targeted aerial assistance to augment its own sensing.

Field Autonomy. Autonomous navigation in unstructured outdoor environments requires a ground robot to reason about obstacles, terrain, vehicle state, and long-range route structure from incomplete onboard observations. Early off-road autonomy systems identified limited sensing lookahead and terrain interpretation as fundamental constraints on ground navigation [3, 4]. Recent work has improved long-range planning through learned cost maps [5], terrain–vehicle interaction models [6], and increasingly capable local planners evaluated under physical field conditions [7]. These approaches improve the information available to the ground-navigation stack, but the sensing geometry and mobility of the UGV continue to limit what can be observed from the ground. We consider failures that arise when these information limitations prevent an otherwise capable navigation stack from identifying or executing useful motion.

Air–Ground Teaming. Air–ground teams use the complementary sensing and mobility of aerial and ground platforms to extend the capabilities of either platform operating independently. Early systems used UAVs as forward scouts to detect hazards beyond the sensing horizon of an off-road UGV [8, 9, 10], while subsequent work has considered heterogeneous teams for perception, exploration, localization, mapping, and navigation [11]. Recent approaches increasingly adapt aerial sensing to the state of the task. ColAG triggers aerial assistance when perception-limited UGVs predict collision risks arising from localization uncertainty or unknown regions along their planned trajectories [12]. PACE proactively schedules aerial perception at information-rich frontiers under timing constraints imposed by the motion of the UGV [13]. While some prior approaches gather aerial information before deploying a UGV [9, 14], our approach selectively invokes aerial assistance during ongoing ground navigation based on its predicted benefit to ground motion, enabling more efficient UAV utilization.

Collaborative Navigation and Mapping. A substantial body of air–ground research uses aerial sensing to improve the information available to a ground-navigation system. UAV imagery has been used to construct obstacle maps and guide UGV motion in large outdoor environments [2, 15], while cooperative mapping systems fuse aerial and ground observations to improve environmental coverage [16]. Related approaches use inter-platform measurements to improve localization, including active motion strategies for positioning geometry [17, 18]. Recent work couples aerial perception more directly to UGV navigation. Fortin et al. estimate off-road terrain properties from aerial imagery to construct traversability information for subsequent UGV path planning [19]. Counterfactual reasoning has also been applied to navigation: CREStE rewards navigation aligned with human preferences using counterfactual trajectory annotations [20], and CoINS reasons about hypothetical object removal to decide when physical intervention is needed to create a traversable path [21]. Our approach instead evaluates hypothetical resolutions of map uncertainty by their effect on the ground planner, an optimistic form of value of information [22] applied to deciding when a robot should ask for help [23], with a UAV rather than a human as the helper. The UGV scores its current planner rollouts under counterfactual resolutions of each uncertain map layer and requests aerial sensing only when a resolution yields sufficient improvement in predicted ground motion. This lets one navigation loop select among multiple aerial sensing roles and leaves the UAV idle when unresolved uncertainty does not constrain the current decision.

Planner-Conditioned Aerial Assistance

Aerial sensing can provide information that is unavailable from the UGV’s ground-level viewpoint, but uncertainty alone does not determine when that sensing is useful. Large uncertainty may occur in regions that do not affect the current navigation problem, while a small uncertain region may prevent the planner from finding viable motion. The assistance decision should therefore depend on whether unresolved information affects the motions currently available to the UGV and whether resolving that could change the navigation outcome.

We condition assistance on the current planning state. As seen in Figure 2, the UGV monitors the current MPPI rollouts [1] and detects when navigation degrades. When degradation is detected, the system evaluates the uncertain map sources against the same rollout population. For each uncertainty source, it constructs a plausible resolved map, rescores the current rollouts under that counterfactual, and measures the resulting change in navigation score. If a source exceeds the intervention threshold, meaning it would improve the UGV route if it were resolved, the UGV sends the corresponding sensing request to the UAV. The UAV acquires the requested information, returns the measurement, and the UGV updates its navigation representation and replans. If no source produces sufficient improvement, no aerial request is made.

UGV mapping, counterfactual evaluation, and UAV mapping workflow

Figure 2. The UGV consumes LiDAR, RGB-D, and IMU data to build layers of maps (obstacles, semantic traversability) that serve as a basis for a fused costmap for motion planning. During planning time, while the projected MPPI plans stay below an uncertainty threshold, the nominal MPPI plan is used. Otherwise, counterfactual diagnosis is used to request the best aerial map that would resolve the uncertainty.

Problem Formulation. We consider a UGV navigating an outdoor, unstructured environment with a receding-horizon local planner. At time tt, let xtx_t denote the UGV state used by the planner and gtg_t the current local planning goal. The UGV maintains LL navigation map layers Mt={mt1,…,mtL}M_t = \{m_t^1,\ldots,m_t^L\}, where each layer mtim_t^i contains information used for navigation, such as obstacle occupancy, terrain semantics, or geometric traversability. Each map layer mtim_t^i has an associated uncertainty representation utiu_t^i, collected as Ut={ut1,…,utL}U_t = \{u_t^1,\ldots,u_t^L\}. Different map layers may represent uncertainty differently, and we do not assume a common uncertainty model across all map layers.

The navigation system combines the map layers into a planner cost representation Ct=F(Mt)C_t = F(M_t), where FF denotes the map-fusion and cost-generation procedure. The uncertainty representations UtU_t are retained separately for assistance diagnosis. Given the state xtx_t, goal gtg_t, and planner representation CtC_t, the local planner Π\Pi generates KK candidate trajectories,

Tt=Π(xt,gt,Ct)={τt(1),…,τt(K)}.(1)T_t = \Pi(x_t, g_t, C_t) = \{\tau_t^{(1)}, \ldots, \tau_t^{(K)}\}. \tag*{(1)}

The rollout population TtT_t contains the motions evaluated by the planner at time tt. We summarize the current planner state with a scalar navigation score St=S(Tt,Ct)S_t = S(T_t, C_t), which indicates whether the current rollout population supports useful ground motion. A degraded value of StS_t can initiate diagnosis, but does not by itself cause an aerial request.

Let A={a1,…,aN}\mathcal{A} = \{a_1, \ldots, a_N\} denote the aerial sensing actions. At an assistance decision, the system selects at∗∈A∪{∅}a_t^* \in\mathcal{A} \cup\{\emptyset\}, where at∗=∅a_t^* = \emptyset means that no aerial assistance is requested.

The problem is to determine whether prolonged uncertainty in any map layer affecting the current rollout population TtT_t can be resolved in a way that improves the navigation score StS_t. If so, the goal is to select the aerial sensing action at∗a_t^* associated with the map layer whose resolution produces the largest predicted improvement.

Navigation Degradation Detection. We begin assistance diagnosis only when degraded navigation persists. Let Γt∈{0,1}\Gamma_t \in\{0, 1\} denote the degradation trigger,

Γt={1,St−j<Smin⁡ for j=0,…,Np−1,0,otherwise,(2)\Gamma_t = \begin{cases} 1, & S_{t-j} < S_{\min}\ \text{for } j = 0, \ldots, N_p - 1, \\ 0, & \text{otherwise,} \end{cases} \tag*{(2)}

where Smin⁡S_{\min} is the minimum acceptable navigation score and NpN_p is the number of consecutive planning cycles required to trigger diagnosis. The persistence requirement prevents temporary changes from starting the assistance procedure.

When Γt=1\Gamma_t = 1, we pass the current map layers MtM_t, uncertainty representations UtU_t, and rollout population TtT_t to the counterfactual diagnosis. At this stage, the system has only established that navigation remains degraded. The following counterfactual evaluation determines whether resolving any map uncertainty is expected to improve the navigation score.

Planner-Conditioned Counterfactual Diagnosis. A degraded navigation score does not by itself show that additional sensing will help. We therefore evaluate uncertainty in each map layer against the current rollout population TtT_t. For map layer mtim_t^i with uncertainty representation utiu_t^i, let Ri\mathcal{R}_i denote a set of plausible resolved hypotheses. Each r∈Rir \in\mathcal{R}_i defines a counterfactual map layer m~ti,r\tilde{m}_t^{i,r} by resolving the uncertain portion of mtim_t^i that affects the current rollouts. All other map layers, the current state xtx_t, the local goal gtg_t, and the rollout population TtT_t remain unchanged. For an occupancy layer, for example, Ri\mathcal{R}_i may contain free and occupied hypotheses over uncertain cells specified by the rollout footprints.

Replacing mtim_t^i with m~ti,r\tilde{m}_t^{i,r} gives the counterfactual map set M~ti,r={mt1,…,m~ti,r,…,mtL}\tilde{M}_t^{i,r} = \{m_t^1, \ldots, \tilde{m}_t^{i,r}, \ldots, m_t^L\} and planner representation C~ti,r=F(M~ti,r)\tilde{C}_t^{i,r} = F(\tilde{M}_t^{i,r}). We then score the same rollout population as S~ti,r=S(Tt,C~ti,r)\tilde{S}_t^{i,r} = S(T_t, \tilde{C}_t^{i,r}). Holding TtT_t fixed isolates the effect of the modified map information from changes caused by replanning.

For map layer ii, we define the counterfactual navigation improvement, ΔSti\Delta S_t^i, as the largest improvement in navigation score across the resolved hypotheses evaluated for map layer mtim_t^i and is the quantity used for assistance selection

ΔSti=max⁡r∈RiS~ti,r−St.(3)\Delta S_t^i = \max_{r \in\mathcal{R}_i} \tilde{S}_t^{i,r} - S_t. \tag*{(3)}

Assistance Selection. Let ϵ\epsilon denote the minimum counterfactual improvement required to justify an aerial request. We first consider only map layers satisfying ΔSti≥ϵ\Delta S_t^i \ge\epsilon, then select the layer with the largest counterfactual improvement,

i∗=arg⁡max⁡iΔSti.(4)i^* = \arg\max_i \Delta S_t^i. \tag*{(4)}

If no map layer satisfies ΔSti≥ϵ\Delta S_t^i \ge\epsilon, no aerial request is issued. Each map layer is associated with a sensing action that can reduce its uncertainty. We denote this mapping by gg and request at∗=g(i∗)a_t^* = g(i^*). When the sensing action requires a spatial target, the request also includes the uncertain map region used in the counterfactual evaluation.

Aerial Refinement and Replanning. For the selected action at∗a_t^*, let yty_t denote the measurement returned by the UAV. We update the affected map and its uncertainty using an action-dependent refinement operator,

(Mt′,Ut′)=Φat∗(Mt,Ut,yt).(5)(M_t', U_t') = \Phi_{a_t^*}(M_t, U_t, y_t). \tag*{(5)}

Depending on the sensing action, Φat∗\Phi_{a_t^*} may replace uncertain map values, fuse the aerial measurement with existing ground observations, or update the uncertainty associated with a map.

The UGV then rebuilds the planner representation as Ct′=F(Mt′)C'_t = F(M'_t). Using the current state xtx_t and local goal gtg_t, the planner generates a new rollout population Tt′=Π(xt,gt,Ct′)T'_t = \Pi(x_t, g_t, C'_t) and computes the updated navigation score St′=S(Tt′,Ct′)S'_t = S(T'_t, C'_t).

The UGV then reevaluates the degradation condition using the updated planning state. If the degradation condition clears, the UGV resumes nominal receding-horizon control. If degradation persists, we repeat the diagnosis using the updated maps Mt′M'_t, uncertainty representations Ut′U'_t, and rollout population Tt′T'_t. This allows later assistance requests to address uncertainty that remains after the first aerial measurement.

Alternative Diagnostics and Non-Map Uncertainty. Not all map representations admit simple map counterfactuals. Discrete maps such as occupancy grids can often be modified by assigning uncertain cells to alternative states. This is less natural for continuous maps such as elevation, roughness, or geometric cost maps. For these sources, we instead measure how much uncertain information intersects the current planning problem. For source ii, let XtiX^i_t denote the uncertain region of map layer mtim^i_t intersected by the current rollout population TtT_t, and let F(Tt)\mathcal{F}(T_t) denote the set of map cells intersected by the swept rollout footprints. We define the fraction of the rollout footprint intersecting uncertain information from source ii as Rti=∣Xti∣/∣F(Tt)∣R^i_t = |X^i_t|/|\mathcal{F}(T_t)|, which measures the exposure of the current candidate motions to uncertainty from that source.

The same assistance logic could also be applied to non-map uncertainty. For example, localization drift can become important during long GPS-denied operation even when the local environment maps remain usable. Localization uncertainty could therefore be treated as another information source with its own diagnostic and sensing action. If that diagnostic indicates that localization is limiting navigation, the UAV could provide an external position measurement ytlocy^{\mathrm{loc}}_t as an additional observation to the UGV’s state estimator.

System Implementation

UGV Sensing and Planning. LiDAR onboard the UGV is used to build a probabilistic obstacle occupancy grid, while a front-facing RGB-D camera provides terrain and traversability information from semantic segmentation. The obstacle and semantic maps use 0.25 m cells over a 40×4040 \times40 m local region. Obstacle occupancy stores per-cell occupancy probability and evidence statistics, terrain quantities store means and variances, and semantic cost is computed from the per-cell class distribution with corresponding uncertainty.

Given a route consisting of a sequence of GPS anchors, the MPPI controller samples K=128K = 128 candidate control sequences over a 4.0 s horizon and evaluates them using continuous route tracking, speed, control, obstacle, and active terrain or semantic costs. The resulting rollout trajectories are retained for assistance diagnosis. For rollout kk, let qk,jq_{k,j} denote the maximum occupancy probability intersected by the vehicle footprint over trajectory segment jj. We define the experimental navigation score as St=1K∑k=1K1kuseful∏j(1−qk,j)S_t = \frac{1}{K}\sum_{k=1}^{K} 1^{\mathrm{useful}}_k \prod_j(1-q_{k,j}), where 1kuseful=11^{\mathrm{useful}}_k = 1 when the rollout remains inside the evaluated map and produces at least 0.25 m of terminal displacement. A confirmed hard collision sets the rollout survival to zero. We declare navigation degraded when St<0.20S_t < 0.20. For obstacle assistance, this condition must persist for 2.0 s before the counterfactual diagnosis is evaluated.

Assistance Diagnostic Implementation. We implement obstacle uncertainty using the counterfactual test in (3). At each diagnosis event, we hold the current MPPI rollout population TtT_t fixed and identify uncertain occupancy cells intersected by the swept rollout footprints. We evaluate three occupancy interpretations: the current map, a free-space hypothesis in which the intersected uncertain cells are assigned occupancy probability 0.05, and an occupied-space hypothesis in which they are assigned occupancy probability 0.95. Known occupancy cells are left unchanged.

We score the same rollout population under each interpretation to obtain StbaseS^{\mathrm{base}}_t, StfreeS^{\mathrm{free}}_t, and StoccupiedS^{\mathrm{occupied}}_t. We then compute

ΔStobs=max⁡(Stfree,Stoccupied)−Stbase(6)\Delta S^{\mathrm{obs}}_t = \max\left(S^{\mathrm{free}}_t,S^{\mathrm{occupied}}_t\right)-S^{\mathrm{base}}_t \tag*{(6)}

and the separation between the two resolved hypotheses, Dtobs=∣Stfree−Stoccupied∣D^{\mathrm{obs}}_t = \left|S^{\mathrm{free}}_t-S^{\mathrm{occupied}}_t\right|. The rollout scores use the obstacle survival or viability metric rather than rerunning the full MPPI optimization. An obstacle request is issued only when the counterfactual test also satisfies the implemented obstacle-crossing, uncertainty-exposure, viability, separation, and persistence conditions.

We implement semantic and geometric uncertainty using rollout-conditioned exposure rather than explicit resolved-map hypotheses. For each map layer mtim^i_t, we project the current rollout footprints into its uncertainty grid and compute the exposure Rti=∣Xti∣/∣F(Tt)∣R^i_t = |X^i_t|/|\mathcal{F}(T_t)|. We also record the uncertainty variance, evidence strength, and fraction of rollouts that intersect uncertain cells. A semantic request can be issued when this exposure persists above its source-specific threshold and the semantic layer contributes to the nominal MPPI cost.

UAV Assistance and Measurement Fusion. UAV assistance is coordinated through an asynchronous map-request service. Each request specifies a region of interest and the required aerial action: a canopy-derived obstacle occupancy map or a semantic segmentation-derived traversability estimate. For each request, the UAV captures an image of the desired ROI, and packages a response containing the geographically registered outputs of the requested mapping pipeline.

Returned maps are registered against the local inertial frame and fused into the corresponding UGV layers. Each layer maintains an estimate, variance, effective observation support, and age. Occupancy uses Bernoulli evidence fusion, with source contributions limited by their support and variance. Semantic costs use inverse-variance-weighted Gaussian fusion. After fusing the map layers, the UGV waits for a fixed fusion-settling interval before releasing the navigation hold. MPPI then evaluates new rollouts using the updated layers and continues with the navigation task.

Simulation Evaluation and Results

We evaluate the assistance strategy in a Gazebo simulation environment using three unique navigation routes. The simulation evaluation addresses mission performance, aerial resource use, and whether the selected sensing action resolves information that constrains UGV navigation.

Simulation Setup and Evaluation Protocol. The simulation uses Gazebo Harmonic with a four-wheel drive, front-steered Ackermann rover, a 16-channel LiDAR, and front-facing depth camera for perception. A 512 m×512 m512\,\mathrm{m} \times512\,\mathrm{m} scene was constructed with LiDAR-derived elevation GeoTIFFs from the USGS 3D Elevation dataset spanning the same area as our physical unstructured experiment site [24]. Grass, bushes, and trees are procedurally generated from hand-painted spatial masks, with seeded variation in position, scale, and orientation.

The simulation adapts the autonomy implementation from Section IV, using ground truth for localization and aerial maps. UAV flight time is calculated analytically from the aerial maps required by each assistance strategy, allowing for a representative comparison of aerial resources without simulating actual UAV flight dynamics. Along with metrics for goal completion, total mission time, and UAV flight time, we also measure the number of interventions performed to recover the UGV from a stuck state.

Three routes of 99 m, 224 m, and 563 m, modeled after real paths at the field site, are traversed in both directions and include elevation changes, vegetation, and physical obstacles. These routes represent EASY, MEDIUM, HARD outdoor navigation tasks. We evaluate four navigation policies: (1) Ground Only, which uses onboard sensing without UAV assistance; (2) UAV Counterfactual, our proposed method, which requests targeted maps when counterfactual analysis predicts improved navigation, making UAV requests only when necessary; (3) Greedy, which continuously maps during UGV navigation; and (4) Explore then Drive, which maps the full route before UGV navigation begins.

Mission Performance and Aerial Resource Use. Table I compares policies over 30 seeds per directional scenario. Trials are matched by route, direction, and random seed, with vehicle, perception, and planner parameters fixed across strategies.

RoutePolicyGoal reachedInterventionsUGV time (s)UAV flight (s)Total mission time (s)Path (m)
Route-1 (EASY)Ground Only60/600.62 ± 0.49127.8 ± 33.2—127.8 ± 33.2131.7 ± 32.9
UAV Counterfactual†60/600.30 ± 0.4697.0 ± 18.711.3 ± 4.9108.4 ± 19.5113.0 ± 11.4
Greedy59/600.47 ± 1.3377.7 ± 14.077.7 ± 14.077.7 ± 14.097.8 ± 2.1
Explore then Drive60/600.32 ± 0.4779.3 ± 14.949.6 ± 0.0129.0 ± 14.998.3 ± 2.0
Route-2 (MEDIUM)Ground Only59/601.28 ± 0.85234.8 ± 79.6—234.8 ± 79.6267.1 ± 76.8
UAV Counterfactual†58/601.42 ± 2.35205.7 ± 47.717.1 ± 7.4222.7 ± 51.3235.7 ± 43.4
Greedy60/601.47 ± 0.60189.4 ± 21.5189.4 ± 21.5189.4 ± 21.5216.2 ± 3.7
Explore then Drive60/601.45 ± 0.72190.6 ± 23.386.6 ± 0.0277.2 ± 23.3216.1 ± 3.3
Route-3 (HARD)Ground Only48/605.32 ± 1.72782.1 ± 185.4—782.1 ± 185.4848.7 ± 260.4
UAV Counterfactual†60/602.87 ± 1.29567.8 ± 62.065.7 ± 12.0633.4 ± 66.6629.8 ± 36.5
Greedy60/602.90 ± 1.07498.2 ± 49.1498.2 ± 49.1498.2 ± 49.1570.6 ± 5.6
Explore then Drive60/602.93 ± 1.21496.0 ± 45.2248.6 ± 0.0744.6 ± 45.2570.0 ± 7.4
Aggregate Statistics Across Runs
Ground Only167/180 (92.8%)2.41353.7—353.7385.6
UAV Counterfactual†178/180 (98.9%)1.53291.131.5322.6327.2
Greedy179/180 (99.4%)1.61256.1256.1256.1295.9
Explore then Drive180/180 (100%)1.57255.3128.3383.6294.8

Table I. Route-level and aggregate comparison over 30 runs in each direction. Time, path, and intervention values are means over trials. Route-level entries are written as mean ± one standard deviation; the aggregate rows report means only. Total mission time includes sequential UAV mapping time.

UAV Counterfactual reaches the goal in 178/180 trials (98.9%), compared with 167 (92.8%) for Ground Only, 179 (99.4%) for Greedy, and 180 (100%) for Explore then Drive, supporting the contribution of aerial mapping to reliable navigation. Relative to Ground Only, counterfactual assistance reduces mean UGV navigation time by 17.7% (353.7 s to 291.1 s) and traveled distance by 15.1% (385.6 m to 327.2 m). Figure 3 shows total mission time against route length: in paired comparisons with Ground Only, counterfactual assistance completes missions 10.4% faster, while Explore then Drive is 10.9% slower due to its initial full-route survey.

Mission Time vs. Route Length

Figure 3. Total mission time vs. route length. Relative to Ground Only (paired): Counterfactual 10.4% faster, Greedy 38.3% faster, Explore-then-Drive 10.9% slower. Greedy’s flight is concurrent and adds no mission time; Explore-then-Drive surveys before the rover moves. Bands: bootstrap 95% CI.

On the HARD route, mean interventions decrease from 5.32 to 2.87 per trial and goal attainment increases from 48/60 to 60/60. The observed standard deviations of UGV time and distance also decrease from 185.4 to 62.0 s and from 260.4 to 36.5 m, respectively, suggesting more consistent navigation in the reported trials.

As seen in Figure 4, aggregated across the three routes, each second of estimated UAV flight time is associated with 2.94 s of UGV navigation time saved relative to Ground Only. This result quantifies the navigation benefit obtained from selectively acquiring aerial information using the proposed counterfactual method.

Return on Aerial Time by Route

Figure 4. UGV navigation time saved for each aerial second spent, referenced off the matching Ground Only trials. Aggregate ratios are 2.94 for Counterfactual, 0.95 for Explore then Drive, and 0.47 for Greedy. The horizontal line marks one second of UGV time saved per aerial second. Bars: 95% bootstrap CI.

The proposed method maintains high goal attainment while substantially reducing aerial resource use. Among trials that reach the goal, mean estimated UAV flight time is 31.5 s, an 87.7% reduction relative to Greedy and a 75.4% reduction relative to Explore then Drive. These savings accompany just one and two fewer goal-reaching trials out of 180, respectively. Mean total mission time is also 15.9% lower than Explore then Drive (322.6 s versus 383.6 s). Although Greedy achieves the shortest mean mission time through concurrent mapping, it requires approximately 8.1 times as much UAV flight time. Together, these results show that counterfactual assistance retains the reliability benefits of aerial mapping with substantially lower aerial resource demand. Reported time, distance, and flight-time means are conditioned on goal attainment and therefore reflect different subsets of trials across policies.

Field Evaluation and Results

To evaluate mission outcomes, aerial resource use, and navigation recovery under real-world uncertainty, we deployed the proposed counterfactual-assisted framework at an unstructured rural field site with hilly terrain, scattered shrubs and trees, tall grass, and pedestrian and vehicle trails.

Field Autonomy. The UGV, labeled in Figure 5, is an autonomous off-road research platform built on a Traxxas X-Maxx chassis. A motor equipped with a Hall-effect sensor and electronic speed controller provide propulsion and telemetry. The sensing suite includes a MicroStrain 3DM-GX5-25 IMU, an Intel RealSense D435i RGB-D camera, and a

The Traxxas X-Maxx UGV and 650mm quadrotor UAV used in the field trials, with onboard sensing and compute labeled

Figure 5. The Traxxas X-Maxx UGV and 650mm quadrotor UAV used in the field trials, with onboard sensing and compute labeled.

Livox Mid-360 LiDAR with an integrated IMU. We run ROS 2 on an NVIDIA Jetson Thor to process sensor data and control the UGV using the methods described in Section IV. We retain a radio-control interface for manual operation and safety intervention. The UAV, labeled in Figure 5, is a quadcopter equipped with a Pixhawk 6X flight controller running PX4, a downward-facing Arducam HQ camera, and an ARK L1/L5 RTK GNSS module. An NVIDIA Jetson Orin Nano Super handles communication with the UGV, commands the flight controller during autonomous navigation, and generates the requested map layers.

The robots, along with an additional ground station, carry

a Morse Micro MM8108 radio that supports communication in the 900 MHz band using IEEE 802.11ah Wi-Fi HaLoW [25]. The three devices connect over a mobile ad hoc network using the B.A.T.M.A.N. mesh-routing protocol [26], enabling long-range, multi-hop communication between network nodes.

For semantic mapping, the UGV uses CLIPSeg [27] on images from its downward-facing camera. Text embedding is grounded on terrain classes such as vegetation, trails, rocks, and water to provide traversability estimation. When the UGV requests obstacle information, the UAV uses the DINOv3-based CHMv2 model to estimate the height above ground at each aerial image pixel [28, 29]. Pixels with estimated height above ground exceeding 0.5 m are registered on the obstacle maps. For both maps, the UAV geo-registers RGB input against an NAIP orthophoto using SuperPoint and SuperGlue correspondences followed by a RANSAC homography [30, 31, 32] before sending them to the UGV.

Field Site and Evaluation Protocol. The UGV navigates the same predefined routes from § 5. These routes cover changes in terrain, elevations, and physical obstacles. Similar to the simulation, we evaluate two navigation policies: (i) Ground Only, which uses onboard sensing without UAV assistance; and (ii) UAV Counterfactual, our proposed method.

We record the number of times the UGV engages in potentially hazardous behavior (attempting dangerous maneuvers, for example), the number of manual interventions (either radio control takeover or relocating the UGV), and the number of times it requests aerial assistance. Figure 6 shows three trials of the Ground Only policy on the ≈563\approx563 m long route, and marks the locations where counterfactual assistance would have been requested. In the UAV Counterfactual runs, we record the number of times the UAV successfully responds with a map, and the number of times the aerial map resolves the uncertainty and UGV resumes navigation.

Ground Only baseline of the 563 m route by the autonomous UGV

Figure 6. Ground Only baseline of the 563 m route by the autonomous UGV. The ground track is shown in red with markers indicating where the current uncertainty policy requested aerial refinement for obstacle uncertainty (orange) or semantic uncertainty (purple), and where manual intervention was required. The three trials produced 9, 5, and 5 candidate requests, respectively.

Unassisted and Assisted Performance. Across all field trials, the UGV autonomously covered approximately 2 km through challenging, unstructured terrain containing large bushes and dense grass. These conditions limited ground-level perception and produced uncertain map cells along candidate UGV motions. Ground only runs recorded three instances of risky behavior and required ten manual interventions. Offline evaluation identified 19 locations where the assistance policy would have requested aerial evidence, comprising 18 obstacle-map requests and one semantic-map request. During UAV Counterfactual runs, the system issued eight requests and successfully received six aerial maps; all six updates enabled the UGV to resume navigation. The two dropped requests were caused by network connectivity issues. Figure 7 illustrates how aerial assistance resolved one such information deficit. Vegetation along the route produced high semantic uncertainty, exceeding the 0.75 request threshold. The returned aerial semantic map provided additional information about the surrounding terrain, updated the fused navigation costs, and reduced the uncertainty diagnostic to 0.19. With this refined representation, the UGV could continue navigation safely. These events demonstrate repeated use of aerial information to resolve navigation uncertainty in challenging field conditions.

Semantic map refinement following counterfactual assistance selection

Figure 7. Semantic map refinement following counterfactual assistance selection. The leftmost column shows the route segment where the counterfactual evaluation identified a need for aerial semantic mapping. The remaining three columns show the semantic map, its uncertainty, and the fused navigation cost map before and after incorporating the UAV-derived semantic map.

Field Challenges. Hardware reliability was the primary challenge in conducting consistent long-range navigation trials. Communication was affected by interference among radios operating in the 900 MHz band and by antenna placement and orientation. We mitigated these issues by integrating a higher-gain omnidirectional antenna and adjusting its placement and alignment. Communications equipment also overheated in direct sunlight, requiring additional cooling and improved airflow. Although the UGV could traverse the rough terrain, grass fragments and dust accumulated in the onboard computer’s cooling fans during repeated operation, restricting airflow and causing overheating; mesh filters were installed to reduce debris ingress. Maintaining UAV connectivity further highlighted the importance of antenna coverage in both azimuth and elevation. Future experiments would benefit from a fixed, elevated antenna installation with a radiation pattern suited to the UAV’s operating altitude. Note that these failures occur on the hardware level rather than in counterfactual evaluations.

Conclusion

This paper presented a planner-conditioned air–ground assistance method that uses counterfactual map evaluations to decide when a UGV should request aerial sensing and what information the UAV should collect. Across 180 simulation trials of the proposed method, it achieved 98.9% goal attainment while reducing mean UAV flight time by 87.7% relative to continuous greedy aerial mapping. Aggregated across the three routes, each second of estimated UAV flight time saved 2.94 s of UGV navigation time relative to the ground only baseline, compared with 0.47 s for the greedy baseline. Across approximately 2 km of unstructured autonomous UGV field operation, subsequent deployments demonstrated resolution of map uncertainty and navigation error. Together, these findings show that targeted aerial sensing can preserve high goal attainment in simulation with substantially reduced aerial resource demand, while resolving information deficits encountered during navigation. Future work will pursue long-range field missions through improved communication, recovery, and coordination mechanisms that increase reliability and reduce human intervention.

References

  1. [1]G. Williams, P. Drews, B. Goldfain, J. M. Rehg, and E. A. Theodorou, “Aggressive driving with model predictive path integral control,” in 2016 IEEE international conference on robotics and automation (ICRA). IEEE, 2016, pp. 1433–1440.DOI
  2. [2]A. Grgic, J. Valente, D. Zapata, and A. Barrientos, “An aerial-ground robotic system for navigation and obstacle mapping in large outdoor areas,” Sensors, vol. 13, no. 1, pp. 1247–1267, 2013.
  3. [3]A. Howard, M. Turmon, L. Matthies, B. Tang, A. Angelova, and E. Mjolsness, “Towards learned traversability for robot navigation: from underfoot to the far field,” J. Field Robot., vol. 23, no. 11–12, pp. 1005–1017, 2006.DOI
  4. [4]E. M. Bajracharya, A. Howard, L. H. Matthies, B. Tang, and M. Turmon, “Autonomous off-road navigation with end-to-end learning for the LAGR program,” J. Field Robot., vol. 26, no. 1, pp. 3–25, 2009.DOI
  5. [5]K. Viswanath, F. Sanchez, T. Overbye, J. M. Gregory, and S. Saripalli, “Trailblazer: learning offroad costmaps for long range planning,” 2025.arxiv.org/abs/2505.09739
  6. [6]F. Tu, Z. Zhan, Z. Zhao, Y. Du, S. Xu, X. Lin, E. T. Esfahani, K. Dantu, S. Chowdhury, and C. Wang, “AnyNav: visual neuro-symbolic friction learning for off-road navigation,” 2025.arxiv.org/abs/2501.12654
  7. [7]C. D. Majhor and J. P. Bos, “Performance characterization of a point-cloud-based path planner for off-road terrain,” J. Field Robot., vol. 43, no. 2, pp. 695–716, 2026.DOI
  8. [8]A. Stentz, A. Kelly, H. Herman, P. Rander, O. Amidi, and R. Manduchi, “Integrated air/ground vehicle system for semi-autonomous off-road navigation,” in Proc. AUVSI Unmanned Syst. Symp., Jul. 2002.
  9. [9]J. Delmerico, E. Mueggler, J. Nitsch, and D. Scaramuzza, “Active autonomous aerial exploration for ground robot path planning,” IEEE Robot. Autom. Lett., vol. 2, no. 2, pp. 664–671, 2017.DOI
  10. [10]S. Zhang, X. Zhang, T. Li, J. Yuan, and Y. Fang, “Fast active aerial exploration for traversable path finding of ground robots in unknown environments,” IEEE Trans. Instrum. Meas., vol. 71, 2022.DOI
  11. [11]J. Zhou, “A survey of the evolution of SLAM techniques for autonomous robots (2016–2026),” Appl. Comput. Eng., vol. 241, pp. 175–180, 2026.DOI
  12. [12]Z. Li, R. Mao, N. Chen, C. Xu, F. Gao, and Y. Cao, “CoLaG: a collaborative air-ground framework for perception-limited UGVs’ navigation,” 2023, arXiv:2310.13324.arxiv.org/abs/2310.13324
  13. [13]Y. Jiao, T. Jiang, Z.-L. Huang, C. Fu, and J. Huang, “PACE: proactive UAV-assisted collaborative motion planning for UGV navigation enhancement,” IEEE Robot. Autom. Lett., vol. 11, pp. 5238–5245, 2026.DOI
  14. [14]F. M. Rockenbauer, J. Lim, M. G. Müller, R. Siegwart, and L. Schmid, “Traversing Mars: cooperative informative path planning to efficiently navigate unknown scenes,” 2024, arXiv:2406.05313. [Online]. Available: https://arxiv.org/abs/2406.05313
  15. [15]J. H. Kim, I. Bae, C.-H. Quan, S. H. Lee, P. W. Son, J. H. Rhee, S. Kim, and J. Seo, “Collaborative navigation of UAV and UGV using vision and LIDAR sensors,” in Proc. ION Pacific PNT Meeting, Honolulu, HI, USA, Apr. 2013, pp. 762–768.
  16. [16]P. Kim, L. C. Price, J. Park, and Y. K. Cho, “UAV-UGV cooperative 3D environmental mapping,” in Proc. ASCE Int. Conf. Comput. Civil Eng. (i3CE), 2019, pp. 384–392.DOI
  17. [17]V. O. Sivaneri and J. N. Gross, “UGV-to-UAV cooperative ranging for robust navigation in GNSS-challenged environments,” Aerosp. Sci. Technol., vol. 71, pp. 245–255, 2017.
  18. [18]——, “Flight-testing of a cooperative UGV-to-UAV strategy for improved positioning in challenging GNSS environments,” Aerosp. Sci. Technol., vol. 82–83, pp. 575–582, 2018.DOI
  19. [19]J.-M. Fortin, O. Gamache, W. Fecteau, E. Daum, W. Larivière-Hardy, F. Pomerleau, and P. Giguère, “UAV-assisted self-supervised terrain awareness for off-road navigation,” 2025, arXiv:2409.18253v2.arxiv.org/abs/2409.18253
  20. [20]A. Zhang, H. Sikich, A. Zhang, and J. Biswas, “CREStE: scalable map-less navigation with Internet scale priors and counterfactual guidance,” 2025.DOI
  21. [21]K. Zhou, Z. Wen, Z. Zhuo, Z. Yan, P. Wu, I. H. U, S. Li, H. Gao, K. Ding, W. Cao, P. Pan, and C. Liu, “CoINS: counterfactual interactive navigation via skill-aware VLM,” 2026.
  22. [22]R. A. Howard, “Information value theory,” IEEE Transactions on systems science and cybernetics, vol. 2, no. 1, pp. 22–26, 1966.DOI
  23. [23]S. Tellex, R. Knepper, A. Li, D. Rus, and N. Roy, “Asking for help using inverse semantics,” 2014.
  24. [24]L. J. Sugarbaker, E. W. Constance, H. K. Heidemann, A. L. Jason, V. Lukas, D. L. Saghy, and J. M. Stoker, “The 3D Elevation Program initiative: a call for action,” US Geological Survey, Tech. Rep., 2014.DOI
  25. [25]T. Adame, A. Bel, B. Bellalta, J. Barcelo, and M. Oliver, “IEEE 802.11AH: the Wi-Fi approach for M2M communications,” IEEE Wireless Communications, vol. 21, no. 6, pp. 144–152, Dec. 2014.arxiv.org/abs/1402.4675
  26. [26]A. Neumann, C. Aichele, M. Lindner, and S. Wunderlich, “Better approach to mobile ad-hoc networking (BATMAN),” IETF draft, pp. 1–24, 2008.
  27. [27]T. Lüddecke and A. Ecker, “Image Segmentation Using Text and Image Prompts,” in 2022 IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2022, pp. 7076–7086.arxiv.org/abs/2112.10003
  28. [28]J. Brandt, S. Yi, J. Tolan, X. Li, P. Potapov, J. Ertel, J. Spore, H. V. Vo, M. Ramamonjisoa, P. Labatut, P. Bojanowski, and C. Couprie, “CHMv2: Improvements in Global Canopy Height Mapping using DINOv3,” Scientific Data, Jul. 2026.
  29. [29]O. Simeoni, H. V. Vo, M. Seitzer, F. Baldassarre, M. Oquab, C. Jose, V. Khalidov, M. Szafraniec, S. E. Yi, M. Ramamonjisoa, F. Massa, D. HAZIZA, L. Wehrstedt, J. Wang, T. Darcet, T. Moutakanni, L. Sentana, C. Roberts, A. Vedaldi, J. Tolan, J. Brandt, C. Couprie, J. Mairal, H. Jegou, P. Labatut, and P. Bojanowski, “DINOv3,” Transactions on Machine Learning Research, 2026.
  30. [30]D. DeTone, T. Malisiewicz, and A. Rabinovich, “SuperPoint: Self-Supervised Interest Point Detection and Description,” in 2018 IEEE/CVF Conf. Comput. Vis. Pattern Recognit. Workshops (CVPRW), Jun. 2018, pp. 337–33712.arxiv.org/abs/1712.07629
  31. [31]P.-E. Sarlin, D. DeTone, T. Malisiewicz, and A. Rabinovich, “SuperGlue: Learning Feature Matching With Graph Neural Networks,” in 2020 IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2020, pp. 4937–4946.arxiv.org/abs/1911.11763
  32. [32]M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Communications of the ACM, vol. 24, no. 6, pp. 381–395, Jun. 1981.

Paper details

vlnc:W_6qphettm
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 1

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents