WorksComputer Science
CostNav: A Navigation Benchmark for Real-World Economic-Cost Evaluation of Physical AI Agents
Abstract
Current navigation benchmarks focus on task success but do not capture the economic constraints essential for commercializing autonomous delivery systems. We introduce CostNav, an Economic Navigation Benchmark that evaluates physical AI agents on a cost-revenue and break-even analysis, pairing Isaac Sim's collision and cargo dynamics with industry-standard data such as Securities and Exchange Commission (SEC) filings and Abbreviated Injury Scale (AIS) injury reports. To our knowledge, CostNav is the first physics-grounded economic benchmark to use regulatory and financial data to quantify the gap between navigation metrics and commercial deployment, revealing that high task-success rates alone do not ensure economic viability. Evaluating seven baselines (two rule-based and five imitation-learning methods), we find no method economically viable: all yield negative contribution margins. CANVAS, using only an RGB camera and GPS, attains the highest task success and the least-negative margin among methods with non-zero Service-Level Agreement (SLA) compliance (-$28.40/run), outperforming LiDAR-equipped Nav2 w/ GPS (-$37.34/run). A sim-trained policy evaluated on a real delivery robot yields SLA compliance close to its simulation result, indicating that policy performance in CostNav's simulation transfers to real-world deployment. We challenge the community to achieve economic viability on CostNav, which scores methods by cost-revenue outcomes. All resources are available at https://github.com/worv-ai/CostNav.
✓ Fully supported, ▲ Partially supported, ✗ Not supported.
Introduction
The deployment of autonomous navigation systems has transitioned from research laboratories to real-world commercial applications. Delivery robots now navigate sidewalks in university campuses and urban areas [1, 2, 3], autonomous vehicles transport goods in warehouses [4], and mobile robots perform tasks in construction sites [5] and hospitals [6]. The adoption of these systems depends on their economic viability; a recent survey finds that decreased operating cost is the principal factor driving the implementation of autonomous delivery robots [7]. However, existing academic navigation benchmarks [8, 9, 10] [11, 12] evaluate embodied physical AI agents primarily on task-oriented metrics such as success rate, collision rate, path length, and navigation time. While these metrics are valuable, they remain disconnected from the cost-revenue dimensions of real-world deployment, leaving a critical practical question unanswered. Which navigation approach minimizes cost and maximizes revenue on the path to profitability?
In this work, we introduce CostNav, an Economic Navigation Benchmark designed to evaluate embodied physical AI agents through the lens of economic viability. CostNav bridges the gap between navigation research and commercial deployment by providing a cost-revenue framework that models the economic lifecycle of autonomous navigation systems. Our key insight is that real-world navigation performance cannot be captured by task-oriented metrics alone because they ignore the economic cost of operation. Evaluating navigation for real-world deployment, therefore, requires an economic-cost metric, and we propose profit per run, which incorporates both delivery revenue and the cost of operation. CostNav is initially instantiated in autonomous food delivery on a complex urban sidewalk map using the Segway E1 delivery robot, a commercially deployed last-mile platform, and makes several key contributions:
High-Fidelity Physics Simulation with Sim-to-Real Evidence. CostNav grounds its cost metrics in Isaac Sim’s high-fidelity physics, deriving them from physical quantities rather than binary collision outcomes. It maps collision impulse and its variation to the Abbreviated Injury Scale (AIS) [13] and crash reports [14, 15], and captures jerk-induced food spoilage (Figure 1) and pedestrian injury liability, operational costs not typically captured by navigation simulation. A real-world check supports this grounding, with a sim-trained CANVAS policy evaluated on a physical Segway E1 along an outdoor urban sidewalk route showing SLA compliance close to simulation (§ 4.3), indicating that policy performance in CostNav’s simulation transfers to real-world deployment.

Figure 1. A motivational example highlighting the core idea behind the CostNav benchmark. Traditional metrics like success rate or collision rate overlook navigation behaviors that can lead to costly outcomes. For instance, overly sharp turning can spill delivery food and cause unnecessary expenses. This gap motivates CostNav, which evaluates navigation through an economic lens. This scene is captured directly from CostNav’s simulation environment, with lighting enhancements.
Real-World referenced Cost-Revenue Model with Break-Even Point Analysis. We develop an economic model that captures the major cost factors in robot navigation, spanning pre-run capital expenditure (CAPEX) and per-run operating expenditure (OPEX), and enables Break-Even Point (BEP) analysis that directly answers how long a deployment takes to become profitable. CostNav integrates revenue modeling based on actual delivery service pricing, modulated by success rate and Service-Level Agreement (SLA) compliance. Every cost and revenue parameter is drawn from public sources such as SEC filings [3], AIS injury reports, and commercial delivery pricing [1], rather than simulator-defined values.
Open-Source Benchmark. We fully open-source our benchmark, including simulation scenarios, real-world referenced cost models, baseline methods, training code and dataset, and evaluation code to facilitate future research at the intersection of physical AI and economic viability.
Related Work
Table 1 shows a comprehensive comparison of our CostNav benchmark with other benchmarks.
Navigation Benchmarks and Systems. Embodied AI simulation has produced indoor benchmarks [10] [16, 17], outdoor robotics simulators [8] [18, 19], and recent urban-sidewalk and delivery-oriented suites [9] [11, 12] that mostly score agents on task-oriented metrics such as success rate, SPL, and collision-related measures. Navigation methods relevant to this space span classical navigation/localization stacks [20, 21] and modern learning-based controllers [22, 23, 24, 25, 26], but most benchmarks lack deployment-economics evaluation. Among these benchmarks, DeliveryBench [12] scores per-episode VLM-courier profit under simulator-defined cost values, but it does not model autonomous-robot CAPEX/OPEX using industry-referenced operating-cost parameters. CostNav closes this gap by combining collision dynamics and cargo dynamics with industry-grounded cost references.
Cost-Aware Robotics. Existing work optimizes isolated cost components such as energy-aware path planning across ground and aerial platforms [27, 28, 29, 30], while domain-specific reviews document deployment economics across warehouse [4], hospital [6], last-mile delivery [7], and construction [5] robotics. Industry filings [3] [31, 32] disclose limited operational figures but keep cost breakdowns proprietary, and none unify these signals into a simulation-driven economic model.
Economic Evaluation of AI. Recent digital-agent benchmarks evaluate general interactive capability [33, 34, 35] and software-engineering capability [36], while newer evaluations attach monetary or labor-automation value to software and remote-work tasks [37, 38]. CostNav extends this value-oriented evaluation perspective from digital tasks to embodied navigation, grounding cost–revenue analysis in physics-based robot interactions and real-world business metrics.
Simulation Platforms for Robotics. Robotics simulators range from game engines such as Unity [39] and Unreal [40], to physics-focused platforms like Gazebo [41], MuJoCo [42], and Isaac Lab [43], with recent hybrid approaches combining rendering and physics engines [44]. CostNav adopts NVIDIA Isaac Sim, which provides unified high-fidelity physics (PhysX 5 and Newton engine) and rendering within a single platform, enabling collision dynamics that directly translate to cost estimation.
CostNav: Cost-Aware Navigation Benchmark for Embodied Agents
We introduce CostNav, a comprehensive benchmark for evaluating embodied navigation systems through the lens of economic viability. While prior navigation benchmarks primarily focus on geometric efficiency or task success, CostNav models the cost–revenue lifecycle of a delivery robot deployment. This enables principled comparison of navigation methods based on their real-world business impact. Figure 2 illustrates the end-to-end process of CostNav. This section presents our economic model (§ 3.1) and simulation environment (§ 3.2).

Figure 2. End-to-end process of the CostNav benchmark, from simulation environments to break-even analysis. Simulation logs capture key operational signals that characterize robot behavior in realistic delivery scenarios, which are then integrated with real-world cost and revenue models to compute profit curves and identify each method’s break-even point. By translating navigation behavior into economic outcomes, CostNav evaluates embodied agents on financial performance, complementing conventional task-centric metrics.
Economic Model
CostNav models the complete economic lifecycle of an autonomous delivery robot, spanning capital expenditure, operational expenditure, and delivery revenue. Our framework explicitly connects simulator-measured quantities to real-world monetary outcomes, enabling direct comparison of navigation methods based on their economic viability. Further details on cost parameters, simulation measurements, and cost-formula derivations are provided in Appendices § 8, § 10, and § 11.
Cost Framework Overview
We define the total profit of a navigation method as:
where denotes cumulative revenue, CAPEX represents capital expenditure incurred before deployment, OPEX captures per-delivery operational costs, and is the number of completed deliveries.
Equivalently, the contribution margin per delivery is:
where is the revenue per delivery (base delivery fee SLA compliance; zero if timeout), and is the total operational cost per run.
The BEP represents the number of deliveries required to recover capital expenditure:
where is the Contribution Margin (profit per delivery). When Contribution Margin is negative (), the BEP is undefined and the navigation system is not economically viable—each delivery increases cumulative losses. Lower BEP indicates faster path to profitability, enabling direct comparison of navigation approaches: Which method recovers its upfront investment fastest?
Capital Expenditure (CAPEX)
Capital expenditure represents fixed upfront investments incurred prior to deployment.
Hardware Cost. The upfront hardware investment is:
where is the robot hardware cost, including onboard sensors, compute hardware, chassis, battery, and supporting electronics.
Data Collection Cost. For learning-based navigation, we model the cost of collecting training data:
where is the hourly wage for data collection personnel and is the total data collection time. Rule-based methods require occupancy maps and environment surveys; these costs are left for future extensions.
Operational Expenditure (OPEX)
Operational costs are variable expenses incurred on a per-delivery basis. These costs are directly influenced by navigation behavior, making them a critical link between low-level control decisions and economic outcomes. We organize OPEX into three categories: direct costs (electricity, repair), service costs (customer compensation), and liability costs (pedestrian and property damage).
Electricity Cost. Energy consumption is computed from simulation and converted to monetary cost:
where is the commercial electricity rate, is the average power draw measured from simulation, is the grid-to-wheel efficiency factor, and is the average run time in simulation.
Repair Cost. Robot repairs arise from physical damage during operation, particularly physical-assistance events—collisions that tip the robot over or accumulate impulse beyond a recoverable threshold, requiring a human operator to physically recover it:
where is the robot hardware cost, is the expected lifetime deliveries per robot, is the annual repair rate as a fraction of hardware cost, and represents the ratio of simulated physical assistance events to the real-world baseline rate.
Service Compensation Cost. Failed deliveries incur compensation costs for customer refunds:
where is the food spoilage rate (delivery arrived but cargo damaged), is the average food cost requiring refund, is the timeout rate, is the rate of physical-assistance events, and is the delivery fee refunded for timeout failures. This cost reflects real-world refund policies where late or damaged deliveries receive full or partial compensation.
Pedestrian Safety Cost. We model pedestrian injury costs using the AIS levels [13], measured from using from [14], with economic costs per AIS severity from [15]:
where is the probability of AIS severity given impact speed, is the economic cost per incident AIS severity level, and is an adjustment factor to adjust for differences in vehicle weight between the delivery robot and the reference vehicle in the crash report. See Appendix § 8 for the scope and limitations of this proxy.
Property Damage Cost. Collisions with urban infrastructure incur repair or replacement costs:
where is a count for contact with property type , and is the associated repair or replacement cost. The property types considered are , selected for their prevalence on urban sidewalks and vulnerability to robot collisions, with costs derived from municipal replacement rates.
Simulation Environment
To evaluate the economic performance of each navigation method, CostNav simulates a Segway E1 delivery robot on complex urban sidewalks in NVIDIA Isaac Sim, chosen for its high-fidelity physics. The map is a sidewalk network spanning distinct zones such as construction sites, work areas, and standard sidewalks, providing diverse and complex environments and scenarios. It is populated with pedestrians at configurable densities and with the urban infrastructure obstacles our cost model prices (Appendix § 8), so pedestrian and property contacts are produced as the agent navigates, grounding the pedestrian-injury and property-damage costs of §3.1.3 in its actual behavior.
We evaluate every navigation method on a single Segway E1 platform, whose sensor options span the configurations these methods require. We adopt it because it is a commercially deployed last-mile delivery robot whose hardware specifications and itemized component pricing are both publicly documented [45, 46]. The published specifications enable us to reproduce its physical and sensing characteristics in simulation, while the listed prices for the base robot and its optional LiDAR and GPS modules ground per-configuration capital expenditure in real market values.
Results
Evaluation Details
We evaluate each method over 100 popcorn delivery episodes on the urban sidewalk environment described in §3.2 (Figure 1), comparing seven baselines in two categories. The rule-based baselines run Nav2 [20] under two localization configurations, AMCL [21] and GPS-only. The learning-based baselines are five end-to-end policies: GNM [22], ViNT [23], NoMaD [24], NavDP [25], and CANVAS [26], all finetuned on our 40.71 hr CostNav teleoperation dataset. Appendix § 9 summarizes the per-baseline configurations in Table 6 and provides their full details. To accommodate more complex avoidance maneuvers, we modify CANVAS to predict command velocities end-to-end through the FAST tokenizer [47], replacing its original waypoint tokens and the separate PD controller.
| Rule-Based | Learning-Based | ||||||
| Nav2 w/ AMCL | Nav2 w/ GPS | GNM | ViNT | NoMaD | NavDP | CANVAS | |
| Sensor Inputs | LiDAR, GPS | LiDAR, GPS | RGB | RGB | RGB | RGB, Depth | RGB, GPS |
| Pretraining Data Size | – | – | 70 hr (real) | 80 hr (real) | 100+ hr (real) | 363 km (sim) | 48 hr / 219 km (sim) |
| Finetuned on CostNav | ✗ | ✗ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Goal Conditioning | Point | Point | Topo. Image | Topo. Image | Topo. Image | Image + Point | Sketch |
Table 6. Baseline Configurations for Simulation Evaluation. Sensor inputs, pretraining data, additional finetuning on our 40.71 hr CostNav teleoperation dataset, and goal conditioning for the seven evaluated baselines in the simulation experiments.
Simulation Results
We report the simulation outcomes in two tables. Table 2 reports the raw measurements aggregated across all evaluation episodes, covering the physical, safety, and outcome quantities defined in § 3.1; full details are in Appendix § 10. Table 3 converts these measurements into the economic metrics of CAPEX, OPEX components, revenue, and contribution margin, using the real-world cost parameters listed in Appendix § 8; full per-method derivations are in Appendix § 11.
| Metric | Nav2 w/ AMCL | Nav2 w/ GPS | GNM | ViNT | NoMaD | NavDP | CANVAS |
| Episode Outcomes (/run) | |||||||
| SLA Compliance | 0.43 | 0.46 | 0.00 | 0.10 | 0.01 | 0.00 | 0.70 |
| Spoiled | 0.12 | 0.29 | 0.00 | 0.00 | 0.00 | 0.00 | 0.10 |
| Timeout | 0.12 | 0.03 | 0.84 | 0.54 | 0.78 | 0.91 | 0.00 |
| Physical Assistance | 0.33 | 0.22 | 0.16 | 0.36 | 0.21 | 0.09 | 0.20 |
| Physical Metrics | |||||||
| Avg Velocity (m/s) | 1.1704 | 1.2625 | 0.1933 | 0.3622 | 0.1427 | 0.0667 | 1.1003 |
| Avg Mech. Power (kW) | 0.0212 | 0.0229 | 0.0035 | 0.0065 | 0.0025 | 0.0011 | 0.0200 |
| Avg Collision Impulse (N·s) | 116.35 | 82.42 | 50.10 | 175.61 | 579.84 | 31.40 | 149.36 |
| Avg Collision (m/s) | 0.8300 | 0.5800 | 0.3200 | 0.8500 | 0.2700 | 0.2600 | 0.4100 |
| Safety Events (/run) | |||||||
| Pedestrian Injury Cost ($) | 29.32 | 20.58 | 11.64 | 29.89 | 9.76 | 9.30 | 14.38 | |||||||
| Mailbox Contact | 0 | 0.02 | 0.00 | 0 | 0 | 0 | 0.00 |
| Trash Bin Contact | 0 | 0 | 0.00 | 0 | 0 | 0 | 0.00 |
| Building Glass Contact | 0.0 | 0 | 0.01 | 0.02 | 0.04 | 0 | 0.02 |
| Bollard Contact | 0 | 0 | 0.01 | 0 | 0 | 0 | 0.00 |
| Performance | |||||||
| Avg Runtime (hr/run) | 0.0260 | 0.0197 | 0.0360 | 0.0294 | 0.0368 | 0.0477 | 0.0287 |
| Avg Distance (km/run) | 0.0498 | 0.0862 | 0.0233 | 0.0368 | 0.0185 | 0.0109 | 0.0902 |
Table 2. Simulation Evaluation Results from 100 evaluation episodes on urban sidewalk scenarios. Full details in Appendix § 10.
| Metric | Nav2 w/ AMCL | Nav2 w/ GPS | GNM | ViNT | NoMaD | NavDP | CANVAS |
| CAPEX ($) | | | | | | | | |||||||
| Hardware | 13000 | 13000 | 8000 | 8000 | 8000 | 8000 | 10000 |
| Data Collection | 0 | 0 | 2165.79 | 2165.79 | 2165.79 | 2165.79 | 2165.79 |
| Total CAPEX Cost | 13000 | 13000 | 10165.79 | 10165.79 | 10165.79 | 10165.79 | 12165.79 |
| OPEX ($/run) | | | | | | | | |||||||
| Electricity | 0.0002 | 0.0002 | 0.0000 | 0.0001 | 0.0000 | 0.0000 | 0.0002 |
| Service Compensation | 5.4021 | 10.1322 | 3.4900 | 3.1410 | 3.4551 | 3.4900 | 3.8910 |
| Pedestrian Safety | 29.3200 | 20.5800 | 11.6400 | 29.8900 | 9.7600 | 9.3000 | 14.3800 |
| Property Damage | 0.0000 | 1.0000 | 3.6500 | 6.0000 | 12.0000 | 0.0000 | 6.0000 |
| Repair | 10.8493 | 7.2329 | 5.2603 | 11.8356 | 6.9041 | 2.9589 | 6.5753 |
| Total OPEX Cost | 45.5716 | 38.9452 | 24.0403 | 50.8667 | 32.1192 | 15.7489 | 30.8466 |
| Profitability | |||||||
| Revenue ($/run) | 1.5007 | 1.6054 | 0.0000 | 0.3490 | 0.0349 | 0.0000 | 2.4430 | |||||||
| Contribution Margin ($/run) | −44.07 | −37.34 | −24.04 | −50.52 | −32.08 | −15.75 | −28.40 | |||||||
| BEP (runs) | – | – | – | – | – | – | – |
Table 3. Overall Economic Performance on urban sidewalk scenarios. Full details in Appendix § 11.
Economic Viability. All seven methods yield negative contribution margins, so none reach break-even. Under our current cost model, the largest modeled OPEX component is pedestrian safety cost, which ranges from $9.30/run (NavDP) to $29.89/run (ViNT) across baselines, with CANVAS at $14.38/run. In terms of overall contribution margin, ViNT performs worst at −$50.52/run. NavDP’s least negative margin at −$15.75/run does not indicate profitable navigation: the policy barely moves, producing 91% timeout and 0% SLA compliance, so both revenue and OPEX stay near zero, and the resulting margin is slightly negative.
Observations on Nav2 and CANVAS. We analyze each of the three highest-performing baselines individually: Nav2 w/ AMCL (43% SLA), Nav2 w/ GPS-only (46%), and CANVAS (70%). Nav2 pairs a prebuilt occupancy map with a classical planner and a LiDAR-based costmap for obstacle avoidance, which is robust on the static sidewalk geometry. CANVAS instead learns an end-to-end policy that produces velocity commands directly from RGB observations and a sketch goal. Training on human teleoperation demonstrations collected under the explicit instruction to drive smoothly so that the food being delivered does not spill has two effects: the policy learns the velocity required to reach the destination, attaining a timeout rate of 0%, the lowest among all baselines; and at that same velocity, it learns to drive without spilling the food. Both Nav2 configurations use a DWB local planner whose critic weights are tuned to maintain forward velocity in order to avoid timeouts, so encountering a pedestrian causes the planner to sample trajectories that hold forward speed and avoid laterally instead of decelerating; the resulting abrupt evasive maneuvers raise the food-spoilage rate to 29% in the GPS-only variant.
Failure Analysis. GNM, NoMaD, and NavDP achieve near-zero SLA compliance (0%, 1%, 0%) with timeout rates of 84%, 78%, and 91%, indicating that these policies fail to reach the goal, getting stuck on or colliding with obstacles. ViNT, by contrast, attains 10% SLA compliance, but its 36% physical-assistance rate indicates frequent collisions requiring operator intervention. GNM, ViNT, and NoMaD condition on topological sub-goal images sampled along the demonstrated path. When the robot deviates from this path to avoid a crossing pedestrian or to make a sharp turn, the current observation falls out of distribution and image-based localization fails, leaving the policy unable to recover. NavDP, separately, conditions on a single image goal plus a point goal; this conditioning provides no intermediate guidance between the current pose and the destination and therefore fails to scale to long-horizon routes.
Sim-to-Real Performance Experiment
To assess whether CostNav-trained policies transfer to the real world, we deployed the same CANVAS checkpoint used in our simulation evaluation on a Segway E1 platform along an outdoor sidewalk route in an urban area, using onboard LiDAR for localization (Table 4). Across 8 delivery scenarios, the sim-trained policy achieved 63% SLA compliance, similar to the 70% rate observed in simulation. Failures involved contact with vegetation or a fence; one vegetation contact also spoiled cargo. Per-scenario raw values are reported in Appendix § 13. The close match between simulated and real-world SLA compliance shows that CostNav’s physics is adequate to reproduce benchmark performance on physical hardware, supporting the simulation benchmark as a proxy for real-world deployment: economic metrics computed in simulation reflect behavior observable on physical hardware.
Table 4: Sim-to-Real Performance Comparison on the CANVAS policy. See Appendix § 13 for the breakdown by scenario.
| Metric | Sim | Real |
| Episode Outcomes (/run) | ||
| SLA Compliance | 0.70 | 0.625 |
| Spoiled | 0.10 | 0.125 |
| Timeout | 0.00 | 0.00 |
| Physical Assistance | 0.20 | 0.25 |
| Physical Metrics | ||
| Avg Velocity (m/s) | 1.1003 | 0.9358 |
| Avg Mech. Power (kW) | 0.0200 | 0.1383 |
| Avg Coll. Impulse (N·s) | 149.36 | 66.67 |
| Avg Coll. Δv (m/s) | 0.41 | 1.3333 |
| Safety Events (/run) | ||
| Pedestrian Injury ($) | 14.38 | 0 | ||
| Static obstacles1 | 0.02 | 0 |
| Plant Contact | – | 0.25 |
| Fence Contact | – | 0.125 |
| Performance | ||
| Avg Runtime (hr/run) | 0.0287 | 0.0297 |
| Avg Distance (km/run) | 0.0902 | 0.1163 |
| Metric | Sim | Real |
| CAPEX ($) | | | ||
| Hardware | 10000 | 11000 |
| Data Collection | 2165.79 | 2165.79 |
| Total CAPEX | 12165.79 | 13165.79 |
| OPEX ($/run) | | | ||
| Electricity | 0.0002 | 0.0015 |
| Service Compensation | 3.8910 | 4.8638 |
| Pedestrian Safety | 14.3800 | 0 |
| Property Damage | 6.0000 | 8.75 |
| Repair | 6.5753 | 8.22 |
| Total OPEX | 30.8466 | 21.83 |
| Profitability | ||
| Revenue ($/run) | 2.4430 | 2.1813 | ||
| Contribution Margin ($/run) | −28.40 | −19.65 | ||
| BEP (runs) | – | – |
Table 4. Sim-to-Real Performance Comparison on the CANVAS policy. See Appendix § 13 for the breakdown by scenario.
Conclusion
We introduced CostNav, an Economic Navigation Benchmark that evaluates physical AI agents through cost-revenue outcomes. CostNav connects simulator-measured behavior to real-world referenced costs and revenue, enabling contribution-margin and break-even analysis for autonomous delivery. Across seven rule-based and learning-based navigation baselines, CostNav shows that high task success does not imply economic viability: every evaluated method yields a negative contribution margin, even the method with the highest task success. CostNav thereby serves as a benchmark for navigation systems that optimize economic outcomes rather than conventional metrics.
Limitations
Simulation and cost transfer. Although CostNav uses real-world referenced cost parameters, its event counts per episode are computed from Isaac Sim and only partially validated against real-world deployment (§ 4.3), so methods with similar totals can differ by only a few events, and close rankings should not be over-interpreted. For pedestrian safety, we approximate the price by rescaling passenger vehicle injury costs by mass ratio, with robot-scale crash test evidence providing partial context for this transfer in Appendix § 8. Even setting this term to zero leaves every evaluated method with a negative margin, so our viability conclusion is robust to it.
Economic model and failure modes. Our economic model captures the dominant cost dynamics of delivery robot operations, but several components remain outside its current scope. CAPEX does not yet include training compute or rule-based mapping costs, revenue does not account for demand or dynamic pricing, and episode outcomes do not yet cover failure modes such as battery depletion.
A single controlled evaluation setup. We evaluate only the Segway E1 platform, popcorn cargo, an urban sidewalk map, and daytime conditions. A subset of learning baselines reach 10% SLA or below. We used a common integration and training setup, but further method-specific optimization may improve performance. Our results should therefore be interpreted as performance under a common CostNav setup, not as each method’s absolute ceiling.
These limitations can be addressed by collecting more real-world deployment data to broaden sim-to-real validation and refine the pedestrian-injury cost, and by expanding the economic model to include omitted components such as training compute and demand effects. Stronger baselines and method-specific optimization can further improve evaluated performance.
Acknowledgments
We thank Yoonseok Kang, Yumin Jung, and Jaehyun Kim for their assistance with the literature review on cost functions, simulation environment setup, and data collection, respectively. We are also grateful to all those who supported this work. This work was supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. RS-2026-25522885, Development of a World Foundation Model for Training and Deployment of Physical AI Systems).
Cost Parameter Details
Detailed breakdown of the cost parameters used in the CostNav benchmark.
| Category | Variable | Symbol | Value | Unit | Reference/Rationale |
| CAPEX | Robot Cost | 8000 | $/robot | Retail price of commercial sidewalk delivery robots [46] (Figure 3). | ||
| CAPEX | LiDAR Cost | 3000 | $/robot | Retail price of LiDAR sensors for commercial sidewalk delivery robots [45] (Figure 4). | ||
| CAPEX | GPS Cost | 2000 | $/robot | Retail price of GPS/GNSS positioning hardware for commercial sidewalk delivery robots [45] (Figure 5). | ||
| CAPEX | Useful Life of Robots | 2 | years | Expected operational lifespan per SEC filings [48] (Figure 6). | |
| CAPEX | Useful Delivery Run Count | 18250 | run/robot | Derived: . | |
| CAPEX | Data Collector Wage | 33 | $/hour | U.S. median hourly wage for data collection personnel [49] (Figure 7). | ||
| OPEX | Market Robot Delivery Price | 3.49 | $/run | Consumer delivery fee for campus robot services [1] (Figure 8). | ||
| OPEX | Avg Market Food Price | 31.93 | $/run | Derived: . | ||
| OPEX | Avg Refund Price | 35.42 | $/run | Industry average order value for delivery failures [50] (Figure 9). | ||
| OPEX | Operator:Robot Ratio | 4 | robot/operator | Level 4 autonomy supervision ratio per SEC filings [3] (Figure 10). | |
| OPEX | Delivery Runs per day | 25 | runs/day | Reported daily throughput per robot [3] (Figure 11). | |
| OPEX | Delivery Failure Rate | 0.06 | %/run | Operational failure rate per SEC filings [3] (Figure 12). | |
| OPEX | Remote Assistance Rate | 80 | %/run | Proportion of deliveries requiring teleoperator intervention [32] (Figure 13). | |
| OPEX | Physical Assistance Rate | 0.002 | %/run | Proportion of deliveries requiring on-site intervention [32] (Figure 14). | |
| OPEX | Repair Rate | 10–20 | %/year | Annual maintenance cost as percentage of based on industry data [51, 52, 53] (Figure 15–Figure 17). | |
| OPEX | Robot Operator Wage | 24 | $/hour | U.S. median hourly wage for robot operators [54] (Figure 18). | ||
| OPEX | Electricity Rate (US) | 0.2704 | $/kWh | U.S. average retail electricity price [55] (Figure 19). | ||
| Category | Variable | Symbol | Value | Unit | Rationale/Source |
| OPEX | Avg Pedestrian Damage (AIS0) | 380 | $/event | Economic cost per MAIS pedestrian incident [15, 13] (Figure 20–Figure 21). | ||
| OPEX | Avg Pedestrian Damage (AIS1) | 8487 | $/event | Same as above. | ||
| OPEX | Avg Pedestrian Damage (AIS2) | 60464 | $/event | Same as above. | ||
| OPEX | Avg Pedestrian Damage (AIS3) | 261200 | $/event | Same as above. | ||
| OPEX | MAIS Injury Report Passenger Vehicle Weight | 4536 | kg | Derived: 10, 000 lbs = 4536 kg. Passenger Vehicle Definition in MAIS Injury Report [14] (Figure 22). | |
| OPEX | Mailbox Damage Cost | 50 | $/event | Municipal replacement cost for mail box damage [56] (Figure 23). | ||
| OPEX | Trash Bin Damage Cost | 50 | $/event | Municipal replacement cost for trash bin damage [57] (Figure 24). | ||
| OPEX | Building Glass Damage Cost | 300 | $/event | Estimated repair cost for building glass damage [58] (Figure 25). | ||
| OPEX | Bollard Damage Cost | 65 | $/event | Estimated replacement cost per bollard [59] (Figure 26). | ||
| OPEX | Plant Damage Cost | 10 | $/event | Lower end of the $5–$20 per-bush shrub/bush trimming cost [60] (Figure 27). | ||
| OPEX | Fence Damage Cost | 50 | $/event | Repair cost per linear foot for aluminum fencing at the upper end of the $20–$50/LF range [61] (Figure 28). | ||
| ETC | Robot Speed | 6.00 | km/h | Maximum operational velocity per manufacturer specifications [62] (Figure 29). | |
| ETC | Robot Max Speed | 12.80 | km/h | Maximum nominal speed of Segway E1 Robot [45] (Figure 30). | |
| ETC | Rolling Resistance Coefficient | 0.0371 | unitless | Coefficient of rolling resistance [63] (Figure 32). | |
| ETC | Rolling Resistance Force | 18.179 | N | Derived: [63] (Figure 31). | |
| ETC | Robot Weight | 50 | kg | Maximum Off-load Weight of Segway E1 Robot [45] (Figure 33). | |
| ETC | Avg Delivery Time | 0.3667 | hr/run | Mean delivery duration from operational deployments [64] (Figure 34). | |
| ETC | Max Delivery Time | 1 | hr/run | Maximum guaranteed delivery time [65] [31] (Figure 35–Figure 36). | |
| ETC | Avg Delivery Distance | 4 | km/run | Mean delivery distance per SEC filings [3] (Figure 37). | |
| Category | Variable | Symbol | Value | Unit | Rationale/Source |
| ETC | Max Delivery Distance | 6 | km/run | Maximum operational delivery radius [2] (Figure 38). | |
| ETC | Charge Efficiency | 0.95 | unitless | Estimated Charge Efficiency [66] (Figure 39). | |
| ETC | Battery Roundtrip Efficiency | 0.95 | unitless | Estimated Battery Roundtrip Efficiency [67] (Figure 40). | |
| ETC | Inverter Efficiency | 0.99 | unitless | Estimated Inverter Efficiency [68] (Figure 41). | |
| ETC | Motor Efficiency | 0.82 | unitless | Estimated Motor Efficiency [69] (Figure 42). | |
| ETC | Electro-Mechanical Efficiency | 0.73 | unitless | Derived: . |
Table 5. Detailed Cost Parameters.

Figure 3. Source data for Robot Cost ()

Figure 5. Source data for GPS Cost ()

Figure 6. Source data for Useful Life of Robots ()

Figure 7. Source data for Data Collector Wage ()

Figure 8. Source data for Price for Market Robot Delivery ()

Figure 9. Source data for Avg Refund Price ()

Figure 10. Source data for Operator:Robot Ratio ()

Figure 11. Source data for Delivery Runs per day ()

Figure 12. Source data for Delivery Failure Rate ()

Figure 13. Source data for Remote Assistance Rate ()

Figure 14. Source data for Physical Assistance Rate ()

Figure 15. Source data for Repair Rate 1 (Standard Bots)

Figure 17. Source data for Repair Rate 3 (Ignus)

Figure 18. Source data for Robot Operator Wage ()

Figure 19. Source data for Electricity Rate (US) ()
Figure 20. Source data for Avg Pedestrian Damage Cost 1 ()

Figure 21. Source data for Avg Pedestrian Damage Cost 2 ()

Figure 22. Source data for Vehicle Weight ()

Figure 23. Source data for Mailbox Damage Cost ()

The cart is city property, and we expect residents to take reasonable care. It must not be painted, mutilated, altered or modified in any way. If the collection truck damages or destroys your cart, it will be replaced at no cost to you. If the responsibility for the damage or loss of a cart cannot be determined, the city will replace it at public expense. If the damage is caused by negligence of the resident, the container will be replaced at a $50.00 charge. Lost or stolen carts will be replaced once at no charge. Any subsequent losses will be the responsibility of the resident. Lost or stolen carts should be reported to the Police and Solid Waste departments.
Figure 24. Source data for Trashbin Damage Cost ()

| Exterior Door Replacement | N/A | $500 |
| Condor Door Replacement | N/A | $300 |
| Window Glass Replacement-Single Pane | $300 | $300 | |
| Window Glass Replacement-Specialty | $500 | $500 | |
| Window Screen Replacement | $150 | $150 |
Table 12.
Figure 25. Source data for Building Glass Damage Cost ()

Bollard Type: The cost depends on whether it is a fixed, removable, or retractable bollard model. The simplest fixed bollards can cost as little as $65, while crash-rated heavy-duty bollards can cost up to $10,000.
Material Selection: The costs can be assorted based on the materials that have been used. Costs typically range from $125 to $1,800 for steel, concrete, and illuminated bollards, depending on durability and design. Depending on the strength and making.
Customization & Features: Optional products to work, color or lighting also increase the expense. For example, bollards with lights and good terminations raise the costs as they are functional and stylish, and elegant.
Location of Installation: Location and complexity can affect urban areas or rural areas or areas that require some form of preparation thus increasing cost.
Installation Costs: Labour depends on individual site and type of bollard, the average range is $300-$3000.
Figure 26. Source data for Bollard Damage Cost ()

| Trimming charges | Average cost |
| Per hour | $40 – $75 |
| Per bush or shrub (2’ to 6’ tall) | $5 – $20 |
| Per bush or shrub (taller than 6’) | $10 – $35 |
| Per 10’ hedge length | $25 – $60 |
Table 27. Shrub and bush trimming cost
Figure 27. Source data for Plant Damage Cost ()

| Fence type | Repair cost per linear foot | Average total repair cost (10 – 50 feet) |
| Wood | $20 – $35 | $200 – $1,800 |
| Chain-link | $10 – $20 | $100 – $1,000 |
| Vinyl | $25 – $45 | $250 – $2,300 |
| Aluminum | $20 – $50 | $200 – $2,500 |
| Bamboo | $15 – $40 | $150 – $2,000 |
| Wrought Iron | $30 – $60 | $300 – $3,000 |
| Lattice | $20 – $45 | $200 – $2,300 |
Table 28. Fence repair cost per foot by material
Figure 28. Source data for Fence Damage Cost ()

Figure 29. Source data for Robot Speed ()

Figure 30. Source data for Robot Max Speed ()
Figure 31. Source data for Rolling Resistance Force ()

Figure 32. Source data for Rolling Resistance Coefficient ()

Figure 33. Source data for Robot Weight ()

Figure 34. Source data for Avg Delivery Run Time ()

Figure 35. Source data for Max Delivery Run Time 1 ()

Figure 36. Source data for Max Delivery Run Time 2 ()

Figure 37. Source data for Avg distance per Run ()
Figure 38. Source data for Max Delivery Distance ()

Figure 39. Source data for Energy Convert () :

Figure 40. Source data for Energy Convert () :

Figure 41. Source data for Energy Convert () :

Figure 42. Source data for Energy Convert ( : )
Pedestrian injury-cost proxy. For each simulated contact, the pedestrian safety term computes an expected injury cost from the measured and accumulates it across the run; the simulated contact rate itself is not calibrated against real-world deployment data. AIS/NHTSA injury-cost tables provide the monetary reference, while robot-scale crash-test evidence bounds how far those numbers transfer to sidewalk robots. Crash tests with mobile service robots and personal mobility devices weighing 60 to 133 kg report low injury risk at 1.0 to 1.5 m/s, with higher risk at larger impact speeds and for vulnerable pedestrians [70].
Source Data Figures

Figure 4. Source data for LiDAR Cost ()
While in the field, we estimate that Serve robots are able to navigate over 80% of their environment autonomously. They are supervised through mobile connectivity and video streaming by remote human operators who can assist robots when necessary, such as at intersection crossings or when robots are unable to navigate certain conditions (e.g., blocked paths, construction zones, etc.). In less frequent occasions, if a robot requires physical assistance, such as when a robot is too low on battery to return home or if it has been damaged, a nearby employee is dispatched to repair or return the robot. As of January 2024, fewer than 1 out of every 500 of our robotic deliveries require physical assistance.
Throughout the day, each robot receives a series of delivery orders from partnered merchants and delivery platforms. Upon acceptance of any such order, the robot navigates to the pick-up location, waits outside, and notifies merchant staff, often through their existing delivery tablets or point-of-sale devices. Once the merchant staff load the package into the robot, it navigates to its drop-off destination. A similar sequence of events results in customers meeting the robot at the curb, unlocking its cargo using their delivery app or on-screen instructions, and retrieving their package.
At night, robots return to their central hub to be recharged, maintained, upgraded when necessary, and prepared for next morning's deployment. Also, various robot components are occasionally replaced due to wear and tear (e.g., tires, motors or batteries). Other components such as computers and key sensors like LIDAR have multi-year lifespans and are rarely replaced. Overall, the useful life of a robot is measured in years, as calculated based on the average lifespan of individual components weighted by their cost.
Sidewalk delivery robots have inherent limitations on how much cargo they can carry and how quickly they can move given the constraints of the sidewalk environment. Current Serve robots carry up to 50 lbs and can move at a maximum speed of 7 mph. This imposes limits on addressable deliveries, travel time, and rate of utilization of robots. For example, due to their speed, delivery robots are best suited for short-distance deliveries, though the majority of restaurant deliveries in the United States are under 25 miles and thus within the delivery range of sidewalk robots. Additionally, given the expected customer wait time of 30 minutes, food delivery robots operating 12 hours per day could perform a maximum of 20 to 30 deliveries per day, depending on the rate of batched deliveries.
Furthermore, our robots are equipped with a number of security features to prevent their cargo and their associated data and IP from unauthorized access. The robots’ secure cargo compartment can only be unlocked through the delivery app’s interface or using a security code entered on the touch screen. Moreover, using encrypted communication protocols and data storage as well as secure authentication methods, the data and software onboard each robot is protected from unauthorized access. Finally, robots are equipped with redundant real-time location tracking systems as well as alarm and communication features to deter attempts at vandalism and enable quick recovery of assets in the unlikely event of theft.

Figure 16. Source data for Repair Rate 2 (Lucas Warehouse)
Downtime and repair costs
While robots can work at much longer intervals than humans, they still require downtime for charging, maintenance, and other updates. Of these, maintenance is the most costly. Humanoid robot budgets should therefore include ongoing maintenance, especially in industries where wear and tear is more likely.
For standard service robots, yearly maintenance costs are about 10–20% of the robot’s purchase price. More advanced robots like humanoids will have additional costs on top of this if a specialized technician is needed to perform maintenance.
have higher response rates for emergency services.
For this report costs specific to police-reported and unreported crashes have been developed. The results of this analysis are presented in Tables 1-4, to 1-7. The differences seem negligible at the more severe injury levels due to the overwhelming costs of factors such as lost productivity and medical care that do not vary by reporting status, except through the shift in injury profile. However, at lower severity levels the unit costs are significant. Injury severity is categorized using the maximum injury level recorded for each victim under the Abbreviated Injury Scale (MAIS). For PDO vehicles and MAIS0 (uninjured) cases, police-reported crashes have costs that are three times those of unreported crashes. For minor (MAIS1) injuries, reported crashes cost 80 percent more than unreported crashes. These ratios decline as injury severity increases to only 16 percent for MAIS2 injuries and 7 percent for MAIS3 injuries. Note that for MAIS4s, MAIS5s, and fatalities, property damage costs are identical under both reported and unreported cases. Virtually all injuries at these levels are believed to be reported to police, and the original property damage cost estimate is thus assumed to represent police-reported cases. These same costs are thus listed under both scenarios.
Figure 1-C shows the proportion of each cost category that is accounted for by police-reported crashes. For most categories, the portions vary due to the differing proportions incidence across the various injury levels. For congestion, property damage, and emergency services, differing
CDS is a nationally representative sample of passenger vehicle (PV) crashes. Passenger vehicles (PVs) are vehicles with a gross vehicle weight rating less than or equal to 10,000 lbs. PVs include passenger cars, sport utility vehicles, and light trucks and vans. The database includes crashes where at least one PV was towed due to damage. CDS is a stratified, multistage, unequal selection probability design sample (Zhang, 2013). CDS collects detailed, in-depth crash, vehicle, and occupant information in addition to those recorded in the police crash report by investigating motor vehicle crashes through interviews, medical records, vehicle inspection, and scene inspection. Since 2009 CDS recorded AIS (MAIS also) injury only for occupants in PVs 10 years and newer when crashes occurred. Therefore, all occupants for the analysis were from relatively new vehicles in each successive calendar year. The data sample for the logistic regression analysis was filtered by MAIS, occupant treatment status, and delta V’s. Delta V used in this analysis is the total delta V (SAS variable: DVTOTAL) from the GV files under the SAS database structure (NHTSA, 2011a to 2016b).
Shrub and bush trimming cost
Shrub and bush trimming costs $5 to $20 per bush on average or $40 to $75 per hour. Shrub trimming service costs $10 to $35 per bush for plants taller than 6’. Minimum service fees are $50 to $200. Total costs depend on the plant type, size, and job complexity.
Figure 4 (PDF p. 26). Motion of the skid-steering mobile robot wheel element on the lab surface (firm ground) from position (1) to (2).
In eq:16, the rolling resistance is denoted as . We can obtain the resistance force, such that:
Dynamic Simulation
This section describes the dynamic simulation that has been used to get the relationship between the treads’ ICRs and path. This model has been simulated for computing the treads’ ICR positions for the Pioneer P3-AT robot. We set all of the key parameters for the model as listed in Table 1.
| Benchmark | Sim. | Delivery Robot | Collision Dynamics | Delivery Package Dynamics | Energy Cost | Pedestrian Safety Cost | Property Damage Cost | Economic Cost Model | Real-World Cost Ref. | BEP Analysis |
| UnrealZoo | Unreal | ▲ | ▲ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ |
| OpenBench | Gazebo | ▲ | ▲ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ |
| Arena-Rosnav | Gazebo | ▲ | ▲ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ |
| Urban-Sim | Isaac Lab | ✓ | ▲ | ✗ | ▲ | ✗ | ✗ | ✗ | ✗ | ✗ |
| DeliveryBench | Unreal | ▲ | ▲ | ✗ | ▲ | ✗ | ▲ | ✗ | ✗ | ▲ |
| CostNav | Isaac Sim | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
Table 1. Comparison of Robot Navigation Simulation Benchmarks across Physics Fidelity and Economic Cost Modeling.
| Key Parameters | Symbol | Value |
| Mass of robot (kg) | 31 | |
| Width of robot (m) | 0.40 | |
| Length of robot (m) | 0.31 | |
| Length of C (m) | 0.24 | |
| Radius of tire (m) | 0.11 | |
| Width of wheel (m) | 0.05 | |
| Shear deformation modulus (m) | 0.00054 | |
| Coefficient of rolling resistance | 0.0371 | |
| Coefficient of friction, of | 0.4437 | |
| Coefficient of friction, of | 0.3093 |
Table 1 (PDF p. 26). The parameters for Pioneer P3-AT robot and terrain dynamic model.
Baseline Details
Table 6 summarizes the sensor inputs, pretraining data, CostNav finetuning, and goal conditioning for the seven evaluated baselines: the rule-based Nav2 [20] variants with AMCL [21] and GPS-only localization, and the learning-based GNM [22], ViNT [23], NoMaD [24], NavDP [25], and CANVAS [26] policies. The remaining subsections detail each configuration.
Rule-Based Navigation Baseline
Our rule-based baseline uses Nav2 [20], which performs classical path planning over a prebuilt 2D occupancy map and uses LiDAR-based costmaps for online obstacle avoidance. We evaluate it under two localization configurations: AMCL [21], which fuses LiDAR scan matching against the prior map with GPS for global drift correction, and a GPS-only variant that omits the scan-matching step and relies solely on GPS measurements.
Learning-Based Baselines: Pretraining Source Datasets
Table 7 lists the source datasets used to pretrain each learning baseline. GNM, ViNT, and NoMaD share many of the same real-world multi-robot navigation datasets, while NavDP and CANVAS each use a single self-collected simulation dataset.
| Source Dataset | GNM | ViNT | NoMaD | NavDP | CANVAS |
| GoStanford | ✓ | ✓ | ✓ | – | – |
| RECON | ✓ | ✓ | ✓ | – | – |
| CoryHall | ✓ | ✓ | ✓ | – | – |
| Berkeley | ✓ | ✓ | ✓ | – | – |
| SCAND-S/J | ✓ | ✓ | ✓ | – | – |
| Seattle | ✓ | ✓ | ✓ | – | – |
| TartanDrive | ✓ | ✓ | ✓ | – | – |
| NeBula | ✓ | ✓ | ✓ | – | – |
| SACSoN | – | ✓ | ✓ | – | – |
| BDD | – | ✓ | – | – | – |
| NavDP Sim | – | – | – | ✓ | – |
| COMMAND | – | – | – | – | ✓ |
| Total Coverage | 70 hr (real) | 80 hr (real) | 100+ hr (real) | 363 km (sim) | 48 hr / 219 km (sim) |
Table 7. Pretraining Source Datasets per Learning Baseline. ✓ indicates that the source dataset is included in the baseline’s pretraining corpus, as reported in the original publications.
Finetuning Procedure
All five learning baselines are finetuned on our 40.71 hr CostNav teleoperation dataset starting from each baseline’s official pretrained checkpoint. The training code, hyperparameter configurations, and dataset preprocessing scripts are provided in the supplementary materials.
Goal Conditioning Specifications
Each baseline accepts a distinct goal modality:
Nav2 (AMCL, GPS-only). A point goal expressed as in the map frame.
GNM, ViNT, NoMaD. An image goal sampled from a topological map of the environment along the shortest path to the destination. The policy is conditioned on a sequence of intermediate sub-goal images that guides long-horizon navigation.
NavDP. An image goal paired with an point goal in the local frame.
CANVAS. A sketch rendered on an orthographic top-down view of the environment, similar to a satellite map, tracing the simulation shortest path to the destination.
Simulation Details
The following table presents the complete simulation evaluation metrics, including symbols, simulated values, and units.
| Variable | Symbol | AMCL | GPS | GNM | ViNT | NoMaD | NavDP | CANVAS | Common Unit | Rationale |
| Physical Metrics | ||||||||||
| Avg Velocity | 1.1704 | 1.2625 | 0.1933 | 0.3622 | 0.1427 | 0.0667 | 1.1003 | m/s | Measured from simulation evaluation episodes | |
| Avg Mechanical Power | 0.0212 | 0.0229 | 0.0035 | 0.0065 | 0.0025 | 0.0011 | 0.0200 | kW | ||
| Avg Collision Impulse | 116.35 | 82.42 | 50.10 | 175.61 | 579.84 | 31.40 | 149.36 | N·s | – | |
| Avg Collision | 0.83 | 0.58 | 0.32 | 0.85 | 0.27 | 0.26 | 0.41 | m/s | – | |
| Episode Arrival Rates | ||||||||||
| SLA Compliance | 0.43 | 0.46 | 0.00 | 0.10 | 0.01 | 0.00 | 0.70 | /run | Safe delivery with intact food | |
| Food Spoiled | 0.12 | 0.29 | 0.00 | 0.00 | 0.00 | 0.00 | 0.10 | /run | Delivery Completed but food spoiled | |
| Timeout | 0.12 | 0.03 | 0.84 | 0.54 | 0.78 | 0.91 | 0.00 | /run | Delivery not completed within timeout limit | |
| Physical Assistance | 0.33 | 0.22 | 0.16 | 0.36 | 0.21 | 0.09 | 0.20 | /run | Bad Robot Orientation or Collision Impulse Health Depletion derived from | |
| Safety & Contact Events | Continued on next page | |||||||||
| Variable | Symbol | AMCL | GPS | GNM | ViNT | NoMaD | NavDP | CANVAS | Common Unit | Rationale |
| MAIS injury adjustment | 0.0110 | Derived: | ||||||||
| Pedestrian Injury Cost | 29.32 | 20.58 | 11.64 | 29.89 | 9.76 | 9.30 | 14.38 | $/run | Derived: , where from simulation, from [14] | ||
| Property Contact | event/run | Count of collisions greater than 100 N · s | ||||||||
| – Bollard | 0 | 0 | 0.01 | 0 | 0 | 0 | 0.00 | event/run | – | |
| – Building Glass | 0.0 | 0 | 0.01 | 0.02 | 0.04 | 0 | 0.02 | event/run | – | |
| – Mail Box | 0 | 0.02 | 0.00 | 0 | 0 | 0 | 0.00 | event/run | – | |
| – Trash Bin | 0 | 0 | 0.00 | 0 | 0 | 0 | 0.00 | event/run | – | |
| Map & Performance Parameters | ||||||||||
| Max Delivery Robot Speed | 7.2 km/h | Based on actual robot speed | ||||||||
| Eval Timeout per delivery | 0.067 hr/run | Derived: | ||||||||
| Avg Runtime per delivery | 0.0260 | 0.0197 | 0.0360 | 0.0294 | 0.0368 | 0.0477 | 0.0287 | hr/run | Measured from simulation evaluation episodes | |
| Avg Delivery Distance | 0.04977 | 0.08623 | 0.02327 | 0.03681 | 0.01848 | 0.0109 | 0.09015 | km/run | Measured from simulation evaluation episodes |
Table 8. Simulation Evaluation Metrics
Continued on next page
| Variable | Symbol | AMCL | GPS | GNM | ViNT | NoMaD | NavDP | CANVAS | Common Unit | Rationale |
| Max Delivery Distance | 0.4 km/run | Maximum taxi distance of 200 m 200 m map | ||||||||
| Robot Max Momentum | 640 kgkm/h | |||||||||
| Data Collection Parameters | ||||||||||
| Data Collector Working Time | 65.63 hr | |||||||||
| Collected Data Time | 40.71 hr | |||||||||
| Data Collection Efficiency | 0.6203 unitless |
Table 8 (continued). from previous page
Aggregated Cost Formula
| Category | Variable | Symbol | AMCL | GPS | GNM | ViNT | NoMaD | NavDP | CANVAS | Unit | Reference/Rationale |
| CAPEX | Hardware Cost | 13000 | 13000 | 8000 | 8000 | 8000 | 8000 | 10000 | $ | Nav2: ; CANVAS: ; Others: | ||
| CAPEX | Data Collection Cost | – | – | 2165.79 | 2165.79 | 2165.79 | 2165.79 | 2165.79 | $ | | ||
| OPEX | Electricity Cost | 0.0002 | 0.0002 | 0.0000 | 0.0001 | 0.0000 | 0.0000 | 0.0002 | $/run | | ||
| OPEX | Service Compensation Cost | 5.40 | 10.13 | 3.49 | 3.14 | 3.46 | 3.49 | 3.89 | $/run | | ||
| OPEX | Pedestrian Safety Cost | 29.32 | 20.58 | 11.64 | 29.89 | 9.76 | 9.30 | 14.38 | $/run | | ||
| OPEX | Property Damage Cost | 0.00 | 1.00 | 3.65 | 6.00 | 12.00 | 0.00 | 6.00 | $/run | | ||
| OPEX | Repair Cost | 10.85 | 7.23 | 5.26 | 11.84 | 6.90 | 2.96 | 6.58 | $/run | | ||
| Total CAPEX | 13000 | 13000 | 10165.79 | 10165.79 | 10165.79 | 10165.79 | 12165.79 | $ | | |||
| Category | Variable | Symbol | AMCL | GPS | GNM | ViNT | NoMaD | NavDP | CANVAS | Unit | Reference/Rationale |
| OPEX Per Run | OPEX Per Run | 45.57 | 38.95 | 24.04 | 50.87 | 32.12 | 15.75 | 30.85 | $/run | components | ||
| Revenue Per Run | Revenue Per Run | 1.50 | 1.61 | 0.00 | 0.35 | 0.03 | 0.00 | 2.44 | $/run | | ||
| Contribution Margin | Contribution Margin | $/run | | |||||||||
| Break Even Point | Break Even Point | – | – | – | – | – | – | – | run | ||
| Break Even Point (days) | Break Even Point (days) | – | – | – | – | – | – | – | days |
Table 9. Aggregated Cost Formula
Per-Component Break-Even Thresholds
For each OPEX component, we report the rate above which its cost alone exceeds 3.49$/run revenue, assuming all other components are zero.
Pedestrian-safety cost . From § 3.1.3, each 1% of contacts at AIS severity adds 0.94$/run (), 6.67$/run (), and 28.79$/run (), so the policy is unprofitable above AIS 1 , AIS 2 , or AIS 3 .
Spoilage rate . Each 1% costs 0.35$/run (food refund plus lost delivery-fee revenue), so the break-even rate is .
Combined timeout and physical-assistance rate . Each 1% costs 0.0698$/run (refund + lost revenue), so the break-even rate is .
Property contact rate . The mean damage across mailbox, trash bin, bollard, and building glass is 116.25$/contact, so a contact rate above exhausts the budget.
Physical-assistance rate for repair alone. Repair scales as per unit PA, so PA exhausts the budget through repair.
Sim-to-Real Experiment: Per-Scenario Details
We deployed the sim-trained CANVAS policy on the physical platform shown in Figure 43 along an outdoor sidewalk route in an urban area. 8 scenarios were attempted; outcomes and physical measurements are summarized below.

Figure 43. Real-world experimental platform. A Segway E1 sidewalk delivery robot with a Livox MID-360 hemispherical LiDAR streams sensor data over a wired LAN tether to an off-board laptop that runs the sim-trained CANVAS policy. The laptop carries an Intel Core Ultra 9 275HX 24-core CPU, 64 GB RAM, 1 TB storage, and an NVIDIA RTX 5090 Laptop GPU. Right inset: cargo bay with a popcorn-cup payload.
Distances are computed from the LiDAR-localized pose stream. Contact occurred only in S3, S5, and S7. The reported impulse for S3 is the directly measured /odom velocity drop multiplied by the 50 kg robot mass. For S5 and S7 a human safety operator intervened to halt the robot before the full collision had run its course, so the raw /odom signal only captures the pre-intervention deceleration and underestimates how severe the un-prevented impact would have been. We therefore use physics-based estimates of the would-be impulse—the impulse the collision would have produced had the operator not stopped the robot. For S5 we adopt the same impulse as S3’s plant hit (50 N·s). For S7 we use 100 N·s, corresponding to an upper-bound near-elastic collision of a 50 kg robot at m/s ( N·s). Average mechanical power (0.1383 kW = 138.3 W) is the mean battery draw over a separate 10-minute teleop run on the same robot.
| Scenario | Outcome | Contact | Runtime (s) | Distance (m) | Avg V (m/s) | Avg Impulse (N·s) | Avg (m/s) |
| S1 | S | – | 147.72 | 176.93 | 0.9573 | – | – |
| S2 | S | – | 97.20 | 91.73 | 0.9278 | – | – |
| S3 | F (physical) | Plant | 78.07 | 95.92 | 0.9511 | 50 | 1.0 |
| S4 | S | – | 129.54 | 151.93 | 0.9772 | – | – |
| S5 | F (spoiled) | Plant (+spilled food) | 132.58 | 117.52 | 0.8328 | 50 | 1.0 |
| S6 | S | – | 124.55 | 141.51 | 0.9852 | – | – |
| S7 | F (physical) | Fence | 57.30 | 57.83 | 0.9525 | 100 | 2.0 |
| S8 | S | – | 87.69 | 96.82 | 0.9023 | – | – |
| Mean | 106.83 | 116.28 | 0.9358 | 66.67 | 1.3333 |
Table 10. Per-Scenario Real-World Results (CANVAS, sim-trained). S=Success, F=Fail. Contact column lists the obstacle struck. Distance from LiDAR-localized pose. Impulse uses 50 kg robot mass.
Aggregated Variable Reference. Table 11 lists all aggregated R* macros defined for the real-world CANVAS run (mirrors the simulation S* schema in §A). These values feed the comparison in Table 4 and are derived from the raw values for each scenario above.
Table 11: Real-World CANVAS Aggregated Variables. All values are means over 8 scenarios. R* prefix denotes real-world (parallel to S* for simulation).
| Category | Variable | Symbol | Value | Unit | Reference/Rationale |
| Episode Outcomes | Episodes | 8 | episodes | S1–S8 | |
| Episode Outcomes | SLA Compliance | 0.625 | rate | S1, S2, S4, S6, S8 | |
| Episode Outcomes | Spoiled | 0.125 | rate | S5 | |
| Episode Outcomes | Timeout | 0.00 | rate | – | |
| Episode Outcomes | Physical Assist | 0.25 | rate | S3, S7 | |
| Physical Metrics | Avg Velocity | 0.9358 | m/s | Mean of linear speed | |
| Physical Metrics | Avg Mechanical Power | 0.1383 | kW | , 10-min teleop | |
| Physical Metrics | Avg Collision Impulse | 66.67 | N·s | ||
| Physical Metrics | Avg Collision | 1.3333 | m/s | See Table 10 | |
| Safety Events | Pedestrian Injury | 0 | $/run | Pedestrians present, no collisions | ||
| Safety Events | Plant Contact | 0.25 | events/run | S3, S5 | |
| Safety Events | Fence Contact | 0.125 | events/run | S7 | |
| Performance | Avg Runtime | 0.0297 | hr/run | Mean episode duration | |
| Performance | Avg Distance | 0.1163 | km/run | From LiDAR-localized pose | |
| CAPEX | Hardware | 11000 | $ | | ||
| CAPEX | Data Collection | 2165.79 | $ | | ||
| CAPEX | Total CAPEX | 13165.79 | $ | | ||
| OPEX | Electricity | 0.0015 | $/run | | ||
| OPEX | Service Compensation | 4.8638 | $/run | | ||
| OPEX | Pedestrian Safety | 0 | $/run | | ||
| OPEX | Property Damage | 8.75 | $/run | | ||
| OPEX | Repair | 8.22 | $/run | | ||
| OPEX | Total OPEX | 21.83 | $/run | components | ||
| Profitability | Revenue | 2.1813 | $/run | | ||
| Profitability | Contribution Margin | −19.65 | $/run | | ||
| Profitability | BEP | – | runs | (undefined; ) |
Table 11.

Figure 44. Sim-to-Real CANVAS — all 8 scenarios on the outdoor sidewalk route. Top: overview showing all eight planned and actual paths on the same map. Bottom row, from left to right: zoomed views for (S1, S3), (S2, S8), and (S4, S5, S6, S7). Solid lines: planned waypoint paths. Dashed lines: actual robot path (LiDAR-localized pose). Each scenario’s label sits next to its start position. Markers (in the scenario’s own color): ■ start, ◯ successful arrival, × crash (S3 plant, S5 plant with spoiled cargo, S7 fence). S3, S5, S7 paths terminate at the impact location rather than at the planned destination.
References
- [1]University of South Carolina. Grubhub and starship delivery. USC Dining Services page, 2024. Accessed: 2025.
- [2]Starship Technologies. Starship dimensions. Dimensions spec page, 2024. Accessed: 2025.
- [3]Serve Robotics Inc. Serve robotics inc. form 10-k for fiscal year 2023. U.S. Securities and Exchange Commission, 2023. Available at SEC EDGAR database.
- [4]K. Russell and H. M. La. Review of autonomous mobile robots for the warehouse environment. arXiv preprint arXiv:2406.08333, 2024. URL https://arxiv.org/abs/2406.08333.
- [5]L. Zeng, S. Guo, J. Wu, and B. Markert. Autonomous mobile construction robots in built environment: A comprehensive review. Developments in the Built Environment, 2024.
- [6]G. Fragapane, H.-H. Hvolby, F. Sgarbossa, and J. O. Strandhagen. Autonomous mobile robots in hospital logistics. In Advances in Production Management Systems. Towards Smart and Digital Manufacturing, pages 672–679. Springer International Publishing, 2020. doi: 10.1007/978-3-030-57993-7_76.DOI
- [7]E. Alverhed, S. Hellgren, H. Isaksson, L. Olsson, H. Palmqvist, and J. Flodén. Autonomous last-mile delivery robots: a literature review. European Transport Research Review, 16(4):4, 2024. doi:10.1186/s12544-023-00629-7. URL https://doi.org/10.1186/s12544-023-00629-7.
- [8]F. Zhong, K. Wu, C. Wang, H. Chen, H. Ci, Z. Li, and Y. Wang. Unrealzoo: Enriching photo-realistic virtual worlds for embodied ai. In Int. Conf. Comput. Vis., 2025.arxiv.org/abs/2412.20977
- [9]J. Wang, D. Huo, Z. Xu, Y. Shi, Y. Yan, Y. Wang, C. Gao, Y. Qiao, and G. Zhou. Openbench: A new benchmark and baseline for semantic navigation in smart logistics. In Int. Conf. Robotics and Automation, 2025.arxiv.org/abs/2502.09238
- [10]V. Shcherbyna, L. Kästner, D. Diaz, H. G. Nguyen, M. H.-K. Schreff, T. Lenz, J. Kreutz, A. Martban, H. Zeng, and H. Soh. Arena 4.0: A comprehensive ros2 development and benchmarking platform for human-centric navigation using generative-model-based environment generation. In Int. Conf. Robotics and Automation, 2025.arxiv.org/abs/2409.12471
- [42]E. Todorov, T. Erez, and Y. Tassa. Mujoco: A physics engine for model-based control. In IEEE/RSJ Int. Conf. Intelligent Robots and Systems, pages 5026–5033, 2012. doi:10.1109/IROS.2012.6386109.DOI
- [43]NVIDIA, ., M. Mittal, P. Roth, J. Tigue, A. Richard, O. Zhang, P. Du, A. Serrano-Muñoz, X. Yao, R. Zubriǵg, N. Rudin, L. Wawrzyniak, M. Rakhsha, A. Denzler, E. Heiden, A. Borovicka, O. Ahmed, I. Akinola, A. Anwar, M. T. Carlson, J. Y. Feng, A. Garg, R. Gasoto, L. Gulich, Y. Guo, M. Gussert, A. Hansen, M. Kulkarni, C. Li, W. Liu, V. Makoviychuk, G. Malczyk, H. Mazhar, M. Moghani, A. Murali, M. Noseworthy, A. Poddubny, N. Ratliff, W. Rehberg, C. Sch warke, R. Singh, J. L. Smith, B. Tang, R. Thaker, M. Trepte, K. V. Wyk, F. Yu, A. Millane, V. Ramasamy, R. Steiner, S. Subramanian, C. Volk, C. Chen, N. Jawale, A. V. Kurrutukulam, M. A. Lin, A. Mandlekar, K. Patzwaldt, J. Welsh, H. Zhao, F. Anes, J.-F. Lafleche, N. Moënne-Loccoz, S. Park, R. Stepinski, D. V. Gelder, C. Amevor, J. Carius, J. Chang, A. H. Chen, P. de Heras Ciechomski, G. Daviet, M. Mohajerani, J. von Muralt, V. Reutsky, M. Sauter, S. Schirm, E. L. Shi, P. Terdiman, K. Vilella, T. Widmer, G. Yeoman, T. Chen, S. Grizan, C. Li, L. Li, C. Smith, R. Wiltz, K. Alexis, Y. Chang, D. Chu, L. J. Fan, F. Farshidian, A. Handa, S. Huang, M. Hutter, Y. Narang, S. Pouya, S. Sheng, Y. Zhu, M. Macklin, A. Moravanszky, P. Reist, Y. Guo, D. Hoeller, and G. State. Isaac lab: A gpu-accelerated simulation framework for multi-modal robot learning, 2025. URL https://arxiv.org/abs/2511.04831.
- [44]J. Embley-Riches, J. Liu, S. Julier, and D. Kanoulas. Unreal robotics lab: A high-fidelity robotics simulator with advanced physics and rendering. arXiv preprint arXiv:2504.14135, 2025.
- [45]Segway Robotics. Segway e1 robot specifications (e1 delivery parameters). Specification PDF, 2025. Accessed: 2025.
- [46]Multirotors.store. Segway outdoor delivery robot. Store Listing, 2025. Accessed: 2025.
- [47]K. Pertsch, K. Stachowicz, B. Ichter, D. Driess, S. Nair, Q. Vuong, O. Mees, C. Finn, and S. Levine. FAST: Efficient action tokenization for vision-language-action models. In Robotics: Science and Systems (RSS), 2025. URL https://arxiv.org/abs/2501.09747.
- [48]Serve Robotics Inc. Serve robotics inc. form 10-q. U.S. Securities and Exchange Commission, 2024. Available at SEC EDGAR database.
- [49]Salary.com. Data collector salary information. Salary.com listing, 2025. Accessed: 2025.
- [50]The Restaurant HQ. Food delivery statistics. The Restaurant HQ article, 2024. Accessed: 2025.
- [51]Standard Bots. How much do robots cost? 2026 price breakdown. Standard Bots article, 2026. Service contracts typically cost 10-15% of robot price annually.
- [52]Lucas. The roi of autonomous mobile robots in your dc. Lucasware article, 2020. February 2020.
- [53]Jared Worth. Maximizing humanoid robot longevity: the power of maintenance-free components. igus Toolbox article, 2024. September 27, 2024.
- [54]Salary.com. Robot operator salary information. Salary.com listing, 2025. Accessed: 2025.
- [55]U.S. Energy Information Administration. Electric power report. EIA electricity data page, 2025. November 2025.
- [56]Town of Wrentham, MA. Mailbox damage claims (news flash). Town of Wrentham article, 2026. Posted January 27, 2026.
- [57]City of Hot Springs. Frequently asked questions. Solid Waste FAQ, 2026. Accessed: 2026.
- [58]Wittenberg University. Wittenberg university damage charge schedule 2024-2025. Damage Charge Schedule PDF, 2024. Accessed: 2026.
- [59]We Are Bollards. How much does it cost to install a bollard? save up to 80%. Blog post, 2024. Accessed: 2026.
- [60]HomeGuide. How much does shrub and bush trimming cost? Cost guide, 2026. Accessed: 2026; $5–$20 per bush or $10–$35 per bush taller than 6 ft.
- [61]HomeGuide. How much does fence repair cost? Cost guide, 2026. Accessed: 2026; aluminum fence repair $20–$50 per linear foot.
- [62]Starship Technologies. Starship robot technical specifications. Starship Deliveries industry page, 2024. Accessed: 2025.
- [63]T. Wang, Y. Wu, J. Liang, C. Han, J. Chen, and Q. Zhao. Analysis and experimental kinematics of a skid-steering wheeled robot based on a laser scanner sensor. Sensors, 15(5):9681–9702, 2015. doi:10.3390/s150509681. URL https://www.mdpi.com/1424-8220/15/5/9681.DOI
- [64]Starship Technologies. Starship technologies celebrates one year of robot deliveries in finland. Starship press release, 2024. Press release, Accessed: 2025.
- [65]AGV Network. Starship technologies faqs. FAQ page, 2024. Accessed: 2025.
- [66]STMicroelectronics. GaN-based Li-ion Battery Charger Demo. STMicroelectronics seminar PDF, 2023. APEC 2023 presentation.
- [67]Battery Design. Round-trip efficiency of lithium-ion batteries. Battery Design article, 2024. Accessed: 2025.
- [68]Texas Instruments. TIDA-010056: 48V 3-Phase Inverter with Shunt-Based In-Line Motor Phase Current Sensing Reference Design. TI TIDA design page, 2024. Reference design for 3-phase BLDC motor driver.
- [69]ZAU. M250W Hub Motor Specifications. ZAU motor spec page, 2024. Hub motor datasheet.
- [70]D. Paez-Granados and A. Billard. Crash test-based assessment of injury risks for adults and children when colliding with personal mobility devices and service robots. Scientific Reports, 12(1):5285, 2022. ISSN 2045-2322. doi:10.1038/s41598-022-09349-9. URL https://doi.org/10.1038/s41598-022-09349-9.
