WorksComputer Science
When to Ask for Help: Uncertainty-Triggered Aerial Refinement for Off-Road Navigation
Abstract
Heterogeneous air-ground teams can use aerial sensing to address the limited range and viewpoint of ground-vehicle sensors, but deciding when the UGV needs assistance and what information the UAV should collect remains a challenge. This paper presents a planner-conditioned air-ground assistance method that evaluates how uncertain map information affects the UGV's current MPPI rollouts. When navigation remains degraded, we score the same rollouts under resolved map hypotheses, select the map layer with the largest useful improvement, request the corresponding aerial measurement, fuse the returned information, and replan. Unlike approaches driven by fixed support roles, task structure, or uncertainty alone, our method conditions assistance on whether resolving uncertainty improves predicted ground navigation. Across 180 simulation trials on three off-road routes, the method achieves 98.9% goal attainment while reducing mean UAV flight time by 87.7% relative to continuous greedy aerial mapping. Aggregated across the routes, each second of estimated UAV flight time saves 2.94 s of UGV navigation time. Field experiments spanning approximately 2 km of UGV operation demonstrate uncertainty resolution and successful navigation recovery.
Introduction
Ground vehicles operating alone in outdoor, unstructured environments are limited by their sensing perspective, mobility, and onboard perception. Ground-level sensors have limited range, suffer from occlusion, and can provide incomplete or ambiguous information about terrain, obstacles, and vehicle state. An unmanned ground vehicle (UGV) may therefore fail to complete an otherwise achievable mission because it cannot obtain the information needed to navigate safely and reliably. Heterogeneous robot teams address this by combining complementary sensing and mobility. An unmanned aerial vehicle (UAV) can observe the environment and the ground vehicle from viewpoints unavailable to the UGV and provide information that improves the ground robot’s navigation state. However, UAV assistance is limited by flight endurance and cannot be used continuously throughout long missions. The key problem is therefore deciding when the UGV needs assistance and what information the UAV should provide.
This paper presents a heterogeneous air–ground teaming approach for recovering degraded UGV navigation caused by incomplete or uncertain information in off-road environments. The UGV maintains separate navigation map layers together with their associated uncertainty. When navigation degrades, we evaluate the current MPPI rollout population [1] against counterfactual versions of these map layers and score how the resulting map changes affect predicted ground motion. We
then select the corresponding UAV sensing action for the map layer with the largest useful improvement. The UAV collects the requested aerial information, such as an elevation map or semantic map, and communicates it to the UGV. The UGV fuses the information into its navigation maps, updates the planner representation, and replans.
Existing air–ground systems use heterogeneous sensing for cooperative localization, mapping, exploration, and navigation. Many approaches assign the UAV a fixed support role, invoke aerial sensing according to predefined task logic, or trigger additional sensing from uncertainty in a perception output [2]. These sensing criteria do not directly determine whether the uncertainty affects the UGV’s current navigation decision. Uncertainty can be large in regions that have no effect on the current route, while localized uncertainty can prevent the UGV from identifying or executing a safe path. We therefore evaluate uncertainty through its effect on ground navigation and request UAV assistance only when resolving uncertainty is predicted to improve the UGV’s ability to make progress.
Our main contribution, illustrated in Figure 1, is a planner-conditioned air–ground assistance method that determines when the UAV should sense by evaluating resolved map hypotheses against current UGV rollouts and selecting an aerial sensing action based on the change in navigation score. Across 180 simulation trials, our method achieves 98.9% goal attainment with 87.7% less mean UAV flight time than continuous greedy mapping. Across three routes, each estimated UAV flight second saves 2.94 s of UGV navigation time. Field experiments spanning approximately 2 km of autonomous UGV operation through unstructured routes demonstrate repeated resolution of map uncertainty and resumed navigation following aerial assistance.

Figure 1. The presented planner-conditioned air–ground assistance method that enables a UGV to navigate highly uncertain off-road environments by requesting targeted aerial assistance to augment its own sensing.
Related Work
Field Autonomy. Autonomous navigation in unstructured outdoor environments requires a ground robot to reason about obstacles, terrain, vehicle state, and long-range route structure from incomplete onboard observations. Early off-road autonomy systems identified limited sensing lookahead and terrain interpretation as fundamental constraints on ground navigation [3, 4]. Recent work has improved long-range planning through learned cost maps [5], terrain–vehicle interaction models [6], and increasingly capable local planners evaluated under physical field conditions [7]. These approaches improve the information available to the ground-navigation stack, but the sensing geometry and mobility of the UGV continue to limit what can be observed from the ground. We consider failures that arise when these information limitations prevent an otherwise capable navigation stack from identifying or executing useful motion.
Air–Ground Teaming. Air–ground teams use the complementary sensing and mobility of aerial and ground platforms to extend the capabilities of either platform operating independently. Early systems used UAVs as forward scouts to detect hazards beyond the sensing horizon of an off-road UGV [8, 9, 10], while subsequent work has considered heterogeneous teams for perception, exploration, localization, mapping, and navigation [11]. Recent approaches increasingly adapt aerial sensing to the state of the task. ColAG triggers aerial assistance when perception-limited UGVs predict collision risks arising from localization uncertainty or unknown regions along their planned trajectories [12]. PACE proactively schedules aerial perception at information-rich frontiers under timing constraints imposed by the motion of the UGV [13]. While some prior approaches gather aerial information before deploying a UGV [9, 14], our approach selectively invokes aerial assistance during ongoing ground navigation based on its predicted benefit to ground motion, enabling more efficient UAV utilization.
Collaborative Navigation and Mapping. A substantial body of air–ground research uses aerial sensing to improve the information available to a ground-navigation system. UAV imagery has been used to construct obstacle maps and guide UGV motion in large outdoor environments [2, 15], while cooperative mapping systems fuse aerial and ground observations to improve environmental coverage [16]. Related approaches use inter-platform measurements to improve localization, including active motion strategies for positioning geometry [17, 18]. Recent work couples aerial perception more directly to UGV navigation. Fortin et al. estimate off-road terrain properties from aerial imagery to construct traversability information for subsequent UGV path planning [19]. Counterfactual reasoning has also been applied to navigation: CREStE rewards navigation aligned with human preferences using counterfactual trajectory annotations [20], and CoINS reasons about hypothetical object removal to decide when physical intervention is needed to create a traversable path [21]. Our approach instead evaluates hypothetical resolutions of map uncertainty by their effect on the ground planner, an optimistic form of value of information [22] applied to deciding when a robot should ask for help [23], with a UAV rather than a human as the helper. The UGV scores its current planner rollouts under counterfactual resolutions of each uncertain map layer and requests aerial sensing only when a resolution yields sufficient improvement in predicted ground motion. This lets one navigation loop select among multiple aerial sensing roles and leaves the UAV idle when unresolved uncertainty does not constrain the current decision.
Planner-Conditioned Aerial Assistance
Aerial sensing can provide information that is unavailable from the UGV’s ground-level viewpoint, but uncertainty alone does not determine when that sensing is useful. Large uncertainty may occur in regions that do not affect the current navigation problem, while a small uncertain region may prevent the planner from finding viable motion. The assistance decision should therefore depend on whether unresolved information affects the motions currently available to the UGV and whether resolving that could change the navigation outcome.
We condition assistance on the current planning state. As seen in Figure 2, the UGV monitors the current MPPI rollouts [1] and detects when navigation degrades. When degradation is detected, the system evaluates the uncertain map sources against the same rollout population. For each uncertainty source, it constructs a plausible resolved map, rescores the current rollouts under that counterfactual, and measures the resulting change in navigation score. If a source exceeds the intervention threshold, meaning it would improve the UGV route if it were resolved, the UGV sends the corresponding sensing request to the UAV. The UAV acquires the requested information, returns the measurement, and the UGV updates its navigation representation and replans. If no source produces sufficient improvement, no aerial request is made.

Figure 2. The UGV consumes LiDAR, RGB-D, and IMU data to build layers of maps (obstacles, semantic traversability) that serve as a basis for a fused costmap for motion planning. During planning time, while the projected MPPI plans stay below an uncertainty threshold, the nominal MPPI plan is used. Otherwise, counterfactual diagnosis is used to request the best aerial map that would resolve the uncertainty.
Problem Formulation. We consider a UGV navigating an outdoor, unstructured environment with a receding-horizon local planner. At time , let denote the UGV state used by the planner and the current local planning goal. The UGV maintains navigation map layers , where each layer contains information used for navigation, such as obstacle occupancy, terrain semantics, or geometric traversability. Each map layer has an associated uncertainty representation , collected as . Different map layers may represent uncertainty differently, and we do not assume a common uncertainty model across all map layers.
The navigation system combines the map layers into a planner cost representation , where denotes the map-fusion and cost-generation procedure. The uncertainty representations are retained separately for assistance diagnosis. Given the state , goal , and planner representation , the local planner generates candidate trajectories,
The rollout population contains the motions evaluated by the planner at time . We summarize the current planner state with a scalar navigation score , which indicates whether the current rollout population supports useful ground motion. A degraded value of can initiate diagnosis, but does not by itself cause an aerial request.
Let denote the aerial sensing actions. At an assistance decision, the system selects , where means that no aerial assistance is requested.
The problem is to determine whether prolonged uncertainty in any map layer affecting the current rollout population can be resolved in a way that improves the navigation score . If so, the goal is to select the aerial sensing action associated with the map layer whose resolution produces the largest predicted improvement.
Navigation Degradation Detection. We begin assistance diagnosis only when degraded navigation persists. Let denote the degradation trigger,
where is the minimum acceptable navigation score and is the number of consecutive planning cycles required to trigger diagnosis. The persistence requirement prevents temporary changes from starting the assistance procedure.
When , we pass the current map layers , uncertainty representations , and rollout population to the counterfactual diagnosis. At this stage, the system has only established that navigation remains degraded. The following counterfactual evaluation determines whether resolving any map uncertainty is expected to improve the navigation score.
Planner-Conditioned Counterfactual Diagnosis. A degraded navigation score does not by itself show that additional sensing will help. We therefore evaluate uncertainty in each map layer against the current rollout population . For map layer with uncertainty representation , let denote a set of plausible resolved hypotheses. Each defines a counterfactual map layer by resolving the uncertain portion of that affects the current rollouts. All other map layers, the current state , the local goal , and the rollout population remain unchanged. For an occupancy layer, for example, may contain free and occupied hypotheses over uncertain cells specified by the rollout footprints.
Replacing with gives the counterfactual map set and planner representation . We then score the same rollout population as . Holding fixed isolates the effect of the modified map information from changes caused by replanning.
For map layer , we define the counterfactual navigation improvement, , as the largest improvement in navigation score across the resolved hypotheses evaluated for map layer and is the quantity used for assistance selection
Assistance Selection. Let denote the minimum counterfactual improvement required to justify an aerial request. We first consider only map layers satisfying , then select the layer with the largest counterfactual improvement,
If no map layer satisfies , no aerial request is issued. Each map layer is associated with a sensing action that can reduce its uncertainty. We denote this mapping by and request . When the sensing action requires a spatial target, the request also includes the uncertain map region used in the counterfactual evaluation.
Aerial Refinement and Replanning. For the selected action , let denote the measurement returned by the UAV. We update the affected map and its uncertainty using an action-dependent refinement operator,
Depending on the sensing action, may replace uncertain map values, fuse the aerial measurement with existing ground observations, or update the uncertainty associated with a map.
The UGV then rebuilds the planner representation as . Using the current state and local goal , the planner generates a new rollout population and computes the updated navigation score .
The UGV then reevaluates the degradation condition using the updated planning state. If the degradation condition clears, the UGV resumes nominal receding-horizon control. If degradation persists, we repeat the diagnosis using the updated maps , uncertainty representations , and rollout population . This allows later assistance requests to address uncertainty that remains after the first aerial measurement.
Alternative Diagnostics and Non-Map Uncertainty. Not all map representations admit simple map counterfactuals. Discrete maps such as occupancy grids can often be modified by assigning uncertain cells to alternative states. This is less natural for continuous maps such as elevation, roughness, or geometric cost maps. For these sources, we instead measure how much uncertain information intersects the current planning problem. For source , let denote the uncertain region of map layer intersected by the current rollout population , and let denote the set of map cells intersected by the swept rollout footprints. We define the fraction of the rollout footprint intersecting uncertain information from source as , which measures the exposure of the current candidate motions to uncertainty from that source.
The same assistance logic could also be applied to non-map uncertainty. For example, localization drift can become important during long GPS-denied operation even when the local environment maps remain usable. Localization uncertainty could therefore be treated as another information source with its own diagnostic and sensing action. If that diagnostic indicates that localization is limiting navigation, the UAV could provide an external position measurement as an additional observation to the UGV’s state estimator.
System Implementation
UGV Sensing and Planning. LiDAR onboard the UGV is used to build a probabilistic obstacle occupancy grid, while a front-facing RGB-D camera provides terrain and traversability information from semantic segmentation. The obstacle and semantic maps use 0.25 m cells over a m local region. Obstacle occupancy stores per-cell occupancy probability and evidence statistics, terrain quantities store means and variances, and semantic cost is computed from the per-cell class distribution with corresponding uncertainty.
Given a route consisting of a sequence of GPS anchors, the MPPI controller samples candidate control sequences over a 4.0 s horizon and evaluates them using continuous route tracking, speed, control, obstacle, and active terrain or semantic costs. The resulting rollout trajectories are retained for assistance diagnosis. For rollout , let denote the maximum occupancy probability intersected by the vehicle footprint over trajectory segment . We define the experimental navigation score as , where when the rollout remains inside the evaluated map and produces at least 0.25 m of terminal displacement. A confirmed hard collision sets the rollout survival to zero. We declare navigation degraded when . For obstacle assistance, this condition must persist for 2.0 s before the counterfactual diagnosis is evaluated.
Assistance Diagnostic Implementation. We implement obstacle uncertainty using the counterfactual test in (3). At each diagnosis event, we hold the current MPPI rollout population fixed and identify uncertain occupancy cells intersected by the swept rollout footprints. We evaluate three occupancy interpretations: the current map, a free-space hypothesis in which the intersected uncertain cells are assigned occupancy probability 0.05, and an occupied-space hypothesis in which they are assigned occupancy probability 0.95. Known occupancy cells are left unchanged.
We score the same rollout population under each interpretation to obtain , , and . We then compute
and the separation between the two resolved hypotheses, . The rollout scores use the obstacle survival or viability metric rather than rerunning the full MPPI optimization. An obstacle request is issued only when the counterfactual test also satisfies the implemented obstacle-crossing, uncertainty-exposure, viability, separation, and persistence conditions.
We implement semantic and geometric uncertainty using rollout-conditioned exposure rather than explicit resolved-map hypotheses. For each map layer , we project the current rollout footprints into its uncertainty grid and compute the exposure . We also record the uncertainty variance, evidence strength, and fraction of rollouts that intersect uncertain cells. A semantic request can be issued when this exposure persists above its source-specific threshold and the semantic layer contributes to the nominal MPPI cost.
UAV Assistance and Measurement Fusion. UAV assistance is coordinated through an asynchronous map-request service. Each request specifies a region of interest and the required aerial action: a canopy-derived obstacle occupancy map or a semantic segmentation-derived traversability estimate. For each request, the UAV captures an image of the desired ROI, and packages a response containing the geographically registered outputs of the requested mapping pipeline.
Returned maps are registered against the local inertial frame and fused into the corresponding UGV layers. Each layer maintains an estimate, variance, effective observation support, and age. Occupancy uses Bernoulli evidence fusion, with source contributions limited by their support and variance. Semantic costs use inverse-variance-weighted Gaussian fusion. After fusing the map layers, the UGV waits for a fixed fusion-settling interval before releasing the navigation hold. MPPI then evaluates new rollouts using the updated layers and continues with the navigation task.
Simulation Evaluation and Results
We evaluate the assistance strategy in a Gazebo simulation environment using three unique navigation routes. The simulation evaluation addresses mission performance, aerial resource use, and whether the selected sensing action resolves information that constrains UGV navigation.
Simulation Setup and Evaluation Protocol. The simulation uses Gazebo Harmonic with a four-wheel drive, front-steered Ackermann rover, a 16-channel LiDAR, and front-facing depth camera for perception. A scene was constructed with LiDAR-derived elevation GeoTIFFs from the USGS 3D Elevation dataset spanning the same area as our physical unstructured experiment site [24]. Grass, bushes, and trees are procedurally generated from hand-painted spatial masks, with seeded variation in position, scale, and orientation.
The simulation adapts the autonomy implementation from Section IV, using ground truth for localization and aerial maps. UAV flight time is calculated analytically from the aerial maps required by each assistance strategy, allowing for a representative comparison of aerial resources without simulating actual UAV flight dynamics. Along with metrics for goal completion, total mission time, and UAV flight time, we also measure the number of interventions performed to recover the UGV from a stuck state.
Three routes of 99 m, 224 m, and 563 m, modeled after real paths at the field site, are traversed in both directions and include elevation changes, vegetation, and physical obstacles. These routes represent EASY, MEDIUM, HARD outdoor navigation tasks. We evaluate four navigation policies: (1) Ground Only, which uses onboard sensing without UAV assistance; (2) UAV Counterfactual, our proposed method, which requests targeted maps when counterfactual analysis predicts improved navigation, making UAV requests only when necessary; (3) Greedy, which continuously maps during UGV navigation; and (4) Explore then Drive, which maps the full route before UGV navigation begins.
Mission Performance and Aerial Resource Use. Table I compares policies over 30 seeds per directional scenario. Trials are matched by route, direction, and random seed, with vehicle, perception, and planner parameters fixed across strategies.
| Route | Policy | Goal reached | Interventions | UGV time (s) | UAV flight (s) | Total mission time (s) | Path (m) |
| Route-1 (EASY) | Ground Only | 60/60 | 0.62 ± 0.49 | 127.8 ± 33.2 | — | 127.8 ± 33.2 | 131.7 ± 32.9 |
| UAV Counterfactual† | 60/60 | 0.30 ± 0.46 | 97.0 ± 18.7 | 11.3 ± 4.9 | 108.4 ± 19.5 | 113.0 ± 11.4 | |
| Greedy | 59/60 | 0.47 ± 1.33 | 77.7 ± 14.0 | 77.7 ± 14.0 | 77.7 ± 14.0 | 97.8 ± 2.1 | |
| Explore then Drive | 60/60 | 0.32 ± 0.47 | 79.3 ± 14.9 | 49.6 ± 0.0 | 129.0 ± 14.9 | 98.3 ± 2.0 | |
| Route-2 (MEDIUM) | Ground Only | 59/60 | 1.28 ± 0.85 | 234.8 ± 79.6 | — | 234.8 ± 79.6 | 267.1 ± 76.8 |
| UAV Counterfactual† | 58/60 | 1.42 ± 2.35 | 205.7 ± 47.7 | 17.1 ± 7.4 | 222.7 ± 51.3 | 235.7 ± 43.4 | |
| Greedy | 60/60 | 1.47 ± 0.60 | 189.4 ± 21.5 | 189.4 ± 21.5 | 189.4 ± 21.5 | 216.2 ± 3.7 | |
| Explore then Drive | 60/60 | 1.45 ± 0.72 | 190.6 ± 23.3 | 86.6 ± 0.0 | 277.2 ± 23.3 | 216.1 ± 3.3 | |
| Route-3 (HARD) | Ground Only | 48/60 | 5.32 ± 1.72 | 782.1 ± 185.4 | — | 782.1 ± 185.4 | 848.7 ± 260.4 |
| UAV Counterfactual† | 60/60 | 2.87 ± 1.29 | 567.8 ± 62.0 | 65.7 ± 12.0 | 633.4 ± 66.6 | 629.8 ± 36.5 | |
| Greedy | 60/60 | 2.90 ± 1.07 | 498.2 ± 49.1 | 498.2 ± 49.1 | 498.2 ± 49.1 | 570.6 ± 5.6 | |
| Explore then Drive | 60/60 | 2.93 ± 1.21 | 496.0 ± 45.2 | 248.6 ± 0.0 | 744.6 ± 45.2 | 570.0 ± 7.4 | |
| Aggregate Statistics Across Runs | |||||||
| Ground Only | 167/180 (92.8%) | 2.41 | 353.7 | — | 353.7 | 385.6 | |
| UAV Counterfactual† | 178/180 (98.9%) | 1.53 | 291.1 | 31.5 | 322.6 | 327.2 | |
| Greedy | 179/180 (99.4%) | 1.61 | 256.1 | 256.1 | 256.1 | 295.9 | |
| Explore then Drive | 180/180 (100%) | 1.57 | 255.3 | 128.3 | 383.6 | 294.8 |
Table I. Route-level and aggregate comparison over 30 runs in each direction. Time, path, and intervention values are means over trials. Route-level entries are written as mean ± one standard deviation; the aggregate rows report means only. Total mission time includes sequential UAV mapping time.
UAV Counterfactual reaches the goal in 178/180 trials (98.9%), compared with 167 (92.8%) for Ground Only, 179 (99.4%) for Greedy, and 180 (100%) for Explore then Drive, supporting the contribution of aerial mapping to reliable navigation. Relative to Ground Only, counterfactual assistance reduces mean UGV navigation time by 17.7% (353.7 s to 291.1 s) and traveled distance by 15.1% (385.6 m to 327.2 m). Figure 3 shows total mission time against route length: in paired comparisons with Ground Only, counterfactual assistance completes missions 10.4% faster, while Explore then Drive is 10.9% slower due to its initial full-route survey.

Figure 3. Total mission time vs. route length. Relative to Ground Only (paired): Counterfactual 10.4% faster, Greedy 38.3% faster, Explore-then-Drive 10.9% slower. Greedy’s flight is concurrent and adds no mission time; Explore-then-Drive surveys before the rover moves. Bands: bootstrap 95% CI.
On the HARD route, mean interventions decrease from 5.32 to 2.87 per trial and goal attainment increases from 48/60 to 60/60. The observed standard deviations of UGV time and distance also decrease from 185.4 to 62.0 s and from 260.4 to 36.5 m, respectively, suggesting more consistent navigation in the reported trials.
As seen in Figure 4, aggregated across the three routes, each second of estimated UAV flight time is associated with 2.94 s of UGV navigation time saved relative to Ground Only. This result quantifies the navigation benefit obtained from selectively acquiring aerial information using the proposed counterfactual method.

Figure 4. UGV navigation time saved for each aerial second spent, referenced off the matching Ground Only trials. Aggregate ratios are 2.94 for Counterfactual, 0.95 for Explore then Drive, and 0.47 for Greedy. The horizontal line marks one second of UGV time saved per aerial second. Bars: 95% bootstrap CI.
The proposed method maintains high goal attainment while substantially reducing aerial resource use. Among trials that reach the goal, mean estimated UAV flight time is 31.5 s, an 87.7% reduction relative to Greedy and a 75.4% reduction relative to Explore then Drive. These savings accompany just one and two fewer goal-reaching trials out of 180, respectively. Mean total mission time is also 15.9% lower than Explore then Drive (322.6 s versus 383.6 s). Although Greedy achieves the shortest mean mission time through concurrent mapping, it requires approximately 8.1 times as much UAV flight time. Together, these results show that counterfactual assistance retains the reliability benefits of aerial mapping with substantially lower aerial resource demand. Reported time, distance, and flight-time means are conditioned on goal attainment and therefore reflect different subsets of trials across policies.
Field Evaluation and Results
To evaluate mission outcomes, aerial resource use, and navigation recovery under real-world uncertainty, we deployed the proposed counterfactual-assisted framework at an unstructured rural field site with hilly terrain, scattered shrubs and trees, tall grass, and pedestrian and vehicle trails.
Field Autonomy. The UGV, labeled in Figure 5, is an autonomous off-road research platform built on a Traxxas X-Maxx chassis. A motor equipped with a Hall-effect sensor and electronic speed controller provide propulsion and telemetry. The sensing suite includes a MicroStrain 3DM-GX5-25 IMU, an Intel RealSense D435i RGB-D camera, and a

Figure 5. The Traxxas X-Maxx UGV and 650mm quadrotor UAV used in the field trials, with onboard sensing and compute labeled.
Livox Mid-360 LiDAR with an integrated IMU. We run ROS 2 on an NVIDIA Jetson Thor to process sensor data and control the UGV using the methods described in Section IV. We retain a radio-control interface for manual operation and safety intervention. The UAV, labeled in Figure 5, is a quadcopter equipped with a Pixhawk 6X flight controller running PX4, a downward-facing Arducam HQ camera, and an ARK L1/L5 RTK GNSS module. An NVIDIA Jetson Orin Nano Super handles communication with the UGV, commands the flight controller during autonomous navigation, and generates the requested map layers.
The robots, along with an additional ground station, carry
a Morse Micro MM8108 radio that supports communication in the 900 MHz band using IEEE 802.11ah Wi-Fi HaLoW [25]. The three devices connect over a mobile ad hoc network using the B.A.T.M.A.N. mesh-routing protocol [26], enabling long-range, multi-hop communication between network nodes.
For semantic mapping, the UGV uses CLIPSeg [27] on images from its downward-facing camera. Text embedding is grounded on terrain classes such as vegetation, trails, rocks, and water to provide traversability estimation. When the UGV requests obstacle information, the UAV uses the DINOv3-based CHMv2 model to estimate the height above ground at each aerial image pixel [28, 29]. Pixels with estimated height above ground exceeding 0.5 m are registered on the obstacle maps. For both maps, the UAV geo-registers RGB input against an NAIP orthophoto using SuperPoint and SuperGlue correspondences followed by a RANSAC homography [30, 31, 32] before sending them to the UGV.
Field Site and Evaluation Protocol. The UGV navigates the same predefined routes from § 5. These routes cover changes in terrain, elevations, and physical obstacles. Similar to the simulation, we evaluate two navigation policies: (i) Ground Only, which uses onboard sensing without UAV assistance; and (ii) UAV Counterfactual, our proposed method.
We record the number of times the UGV engages in potentially hazardous behavior (attempting dangerous maneuvers, for example), the number of manual interventions (either radio control takeover or relocating the UGV), and the number of times it requests aerial assistance. Figure 6 shows three trials of the Ground Only policy on the m long route, and marks the locations where counterfactual assistance would have been requested. In the UAV Counterfactual runs, we record the number of times the UAV successfully responds with a map, and the number of times the aerial map resolves the uncertainty and UGV resumes navigation.

Figure 6. Ground Only baseline of the 563 m route by the autonomous UGV. The ground track is shown in red with markers indicating where the current uncertainty policy requested aerial refinement for obstacle uncertainty (orange) or semantic uncertainty (purple), and where manual intervention was required. The three trials produced 9, 5, and 5 candidate requests, respectively.
Unassisted and Assisted Performance. Across all field trials, the UGV autonomously covered approximately 2 km through challenging, unstructured terrain containing large bushes and dense grass. These conditions limited ground-level perception and produced uncertain map cells along candidate UGV motions. Ground only runs recorded three instances of risky behavior and required ten manual interventions. Offline evaluation identified 19 locations where the assistance policy would have requested aerial evidence, comprising 18 obstacle-map requests and one semantic-map request. During UAV Counterfactual runs, the system issued eight requests and successfully received six aerial maps; all six updates enabled the UGV to resume navigation. The two dropped requests were caused by network connectivity issues. Figure 7 illustrates how aerial assistance resolved one such information deficit. Vegetation along the route produced high semantic uncertainty, exceeding the 0.75 request threshold. The returned aerial semantic map provided additional information about the surrounding terrain, updated the fused navigation costs, and reduced the uncertainty diagnostic to 0.19. With this refined representation, the UGV could continue navigation safely. These events demonstrate repeated use of aerial information to resolve navigation uncertainty in challenging field conditions.

Figure 7. Semantic map refinement following counterfactual assistance selection. The leftmost column shows the route segment where the counterfactual evaluation identified a need for aerial semantic mapping. The remaining three columns show the semantic map, its uncertainty, and the fused navigation cost map before and after incorporating the UAV-derived semantic map.
Field Challenges. Hardware reliability was the primary challenge in conducting consistent long-range navigation trials. Communication was affected by interference among radios operating in the 900 MHz band and by antenna placement and orientation. We mitigated these issues by integrating a higher-gain omnidirectional antenna and adjusting its placement and alignment. Communications equipment also overheated in direct sunlight, requiring additional cooling and improved airflow. Although the UGV could traverse the rough terrain, grass fragments and dust accumulated in the onboard computer’s cooling fans during repeated operation, restricting airflow and causing overheating; mesh filters were installed to reduce debris ingress. Maintaining UAV connectivity further highlighted the importance of antenna coverage in both azimuth and elevation. Future experiments would benefit from a fixed, elevated antenna installation with a radiation pattern suited to the UAV’s operating altitude. Note that these failures occur on the hardware level rather than in counterfactual evaluations.
Conclusion
This paper presented a planner-conditioned air–ground assistance method that uses counterfactual map evaluations to decide when a UGV should request aerial sensing and what information the UAV should collect. Across 180 simulation trials of the proposed method, it achieved 98.9% goal attainment while reducing mean UAV flight time by 87.7% relative to continuous greedy aerial mapping. Aggregated across the three routes, each second of estimated UAV flight time saved 2.94 s of UGV navigation time relative to the ground only baseline, compared with 0.47 s for the greedy baseline. Across approximately 2 km of unstructured autonomous UGV field operation, subsequent deployments demonstrated resolution of map uncertainty and navigation error. Together, these findings show that targeted aerial sensing can preserve high goal attainment in simulation with substantially reduced aerial resource demand, while resolving information deficits encountered during navigation. Future work will pursue long-range field missions through improved communication, recovery, and coordination mechanisms that increase reliability and reduce human intervention.
References
- [1]G. Williams, P. Drews, B. Goldfain, J. M. Rehg, and E. A. Theodorou, “Aggressive driving with model predictive path integral control,” in 2016 IEEE international conference on robotics and automation (ICRA). IEEE, 2016, pp. 1433–1440.DOI
- [2]A. Grgic, J. Valente, D. Zapata, and A. Barrientos, “An aerial-ground robotic system for navigation and obstacle mapping in large outdoor areas,” Sensors, vol. 13, no. 1, pp. 1247–1267, 2013.
- [3]A. Howard, M. Turmon, L. Matthies, B. Tang, A. Angelova, and E. Mjolsness, “Towards learned traversability for robot navigation: from underfoot to the far field,” J. Field Robot., vol. 23, no. 11–12, pp. 1005–1017, 2006.DOI
- [4]E. M. Bajracharya, A. Howard, L. H. Matthies, B. Tang, and M. Turmon, “Autonomous off-road navigation with end-to-end learning for the LAGR program,” J. Field Robot., vol. 26, no. 1, pp. 3–25, 2009.DOI
- [5]K. Viswanath, F. Sanchez, T. Overbye, J. M. Gregory, and S. Saripalli, “Trailblazer: learning offroad costmaps for long range planning,” 2025.arxiv.org/abs/2505.09739
- [6]F. Tu, Z. Zhan, Z. Zhao, Y. Du, S. Xu, X. Lin, E. T. Esfahani, K. Dantu, S. Chowdhury, and C. Wang, “AnyNav: visual neuro-symbolic friction learning for off-road navigation,” 2025.arxiv.org/abs/2501.12654
- [7]C. D. Majhor and J. P. Bos, “Performance characterization of a point-cloud-based path planner for off-road terrain,” J. Field Robot., vol. 43, no. 2, pp. 695–716, 2026.DOI
- [8]A. Stentz, A. Kelly, H. Herman, P. Rander, O. Amidi, and R. Manduchi, “Integrated air/ground vehicle system for semi-autonomous off-road navigation,” in Proc. AUVSI Unmanned Syst. Symp., Jul. 2002.
- [9]J. Delmerico, E. Mueggler, J. Nitsch, and D. Scaramuzza, “Active autonomous aerial exploration for ground robot path planning,” IEEE Robot. Autom. Lett., vol. 2, no. 2, pp. 664–671, 2017.DOI
- [10]S. Zhang, X. Zhang, T. Li, J. Yuan, and Y. Fang, “Fast active aerial exploration for traversable path finding of ground robots in unknown environments,” IEEE Trans. Instrum. Meas., vol. 71, 2022.DOI
- [11]J. Zhou, “A survey of the evolution of SLAM techniques for autonomous robots (2016–2026),” Appl. Comput. Eng., vol. 241, pp. 175–180, 2026.DOI
- [12]Z. Li, R. Mao, N. Chen, C. Xu, F. Gao, and Y. Cao, “CoLaG: a collaborative air-ground framework for perception-limited UGVs’ navigation,” 2023, arXiv:2310.13324.arxiv.org/abs/2310.13324
- [13]Y. Jiao, T. Jiang, Z.-L. Huang, C. Fu, and J. Huang, “PACE: proactive UAV-assisted collaborative motion planning for UGV navigation enhancement,” IEEE Robot. Autom. Lett., vol. 11, pp. 5238–5245, 2026.DOI
- [14]F. M. Rockenbauer, J. Lim, M. G. Müller, R. Siegwart, and L. Schmid, “Traversing Mars: cooperative informative path planning to efficiently navigate unknown scenes,” 2024, arXiv:2406.05313. [Online]. Available: https://arxiv.org/abs/2406.05313
- [15]J. H. Kim, I. Bae, C.-H. Quan, S. H. Lee, P. W. Son, J. H. Rhee, S. Kim, and J. Seo, “Collaborative navigation of UAV and UGV using vision and LIDAR sensors,” in Proc. ION Pacific PNT Meeting, Honolulu, HI, USA, Apr. 2013, pp. 762–768.
- [16]P. Kim, L. C. Price, J. Park, and Y. K. Cho, “UAV-UGV cooperative 3D environmental mapping,” in Proc. ASCE Int. Conf. Comput. Civil Eng. (i3CE), 2019, pp. 384–392.DOI
- [17]V. O. Sivaneri and J. N. Gross, “UGV-to-UAV cooperative ranging for robust navigation in GNSS-challenged environments,” Aerosp. Sci. Technol., vol. 71, pp. 245–255, 2017.
- [18]——, “Flight-testing of a cooperative UGV-to-UAV strategy for improved positioning in challenging GNSS environments,” Aerosp. Sci. Technol., vol. 82–83, pp. 575–582, 2018.DOI
- [19]J.-M. Fortin, O. Gamache, W. Fecteau, E. Daum, W. Larivière-Hardy, F. Pomerleau, and P. Giguère, “UAV-assisted self-supervised terrain awareness for off-road navigation,” 2025, arXiv:2409.18253v2.arxiv.org/abs/2409.18253
- [20]A. Zhang, H. Sikich, A. Zhang, and J. Biswas, “CREStE: scalable map-less navigation with Internet scale priors and counterfactual guidance,” 2025.DOI
- [21]K. Zhou, Z. Wen, Z. Zhuo, Z. Yan, P. Wu, I. H. U, S. Li, H. Gao, K. Ding, W. Cao, P. Pan, and C. Liu, “CoINS: counterfactual interactive navigation via skill-aware VLM,” 2026.
- [22]R. A. Howard, “Information value theory,” IEEE Transactions on systems science and cybernetics, vol. 2, no. 1, pp. 22–26, 1966.DOI
- [23]S. Tellex, R. Knepper, A. Li, D. Rus, and N. Roy, “Asking for help using inverse semantics,” 2014.
- [24]L. J. Sugarbaker, E. W. Constance, H. K. Heidemann, A. L. Jason, V. Lukas, D. L. Saghy, and J. M. Stoker, “The 3D Elevation Program initiative: a call for action,” US Geological Survey, Tech. Rep., 2014.DOI
- [25]T. Adame, A. Bel, B. Bellalta, J. Barcelo, and M. Oliver, “IEEE 802.11AH: the Wi-Fi approach for M2M communications,” IEEE Wireless Communications, vol. 21, no. 6, pp. 144–152, Dec. 2014.arxiv.org/abs/1402.4675
- [26]A. Neumann, C. Aichele, M. Lindner, and S. Wunderlich, “Better approach to mobile ad-hoc networking (BATMAN),” IETF draft, pp. 1–24, 2008.
- [27]T. Lüddecke and A. Ecker, “Image Segmentation Using Text and Image Prompts,” in 2022 IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2022, pp. 7076–7086.arxiv.org/abs/2112.10003
- [28]J. Brandt, S. Yi, J. Tolan, X. Li, P. Potapov, J. Ertel, J. Spore, H. V. Vo, M. Ramamonjisoa, P. Labatut, P. Bojanowski, and C. Couprie, “CHMv2: Improvements in Global Canopy Height Mapping using DINOv3,” Scientific Data, Jul. 2026.
- [29]O. Simeoni, H. V. Vo, M. Seitzer, F. Baldassarre, M. Oquab, C. Jose, V. Khalidov, M. Szafraniec, S. E. Yi, M. Ramamonjisoa, F. Massa, D. HAZIZA, L. Wehrstedt, J. Wang, T. Darcet, T. Moutakanni, L. Sentana, C. Roberts, A. Vedaldi, J. Tolan, J. Brandt, C. Couprie, J. Mairal, H. Jegou, P. Labatut, and P. Bojanowski, “DINOv3,” Transactions on Machine Learning Research, 2026.
- [30]D. DeTone, T. Malisiewicz, and A. Rabinovich, “SuperPoint: Self-Supervised Interest Point Detection and Description,” in 2018 IEEE/CVF Conf. Comput. Vis. Pattern Recognit. Workshops (CVPRW), Jun. 2018, pp. 337–33712.arxiv.org/abs/1712.07629
- [31]P.-E. Sarlin, D. DeTone, T. Malisiewicz, and A. Rabinovich, “SuperGlue: Learning Feature Matching With Graph Neural Networks,” in 2020 IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2020, pp. 4937–4946.arxiv.org/abs/1911.11763
- [32]M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Communications of the ACM, vol. 24, no. 6, pp. 381–395, Jun. 1981.