betaFeedback

WorksComputer Science

Caging Vision-Language Navigation for Legged Robots via Barrier-Certified Safety Architecture

Abstract

By harnessing the inherent commonsense capabilities of pretrained Large Language Models (LLMs) and Vision-Language Models (VLMs), current Vision-Language Action(VLA) models for legged robots excel at interpreting language-based navigation commands but lack the physical grounding and explicit safety assurance required for real-world operation. Therefore, this paper aims to cage Legged-Vision-Language Navigation (VLN) through a safety architecture that provides physical grounding and ensures navigation safety. The key idea is not to abandon the powerful capabilities of end-to-end VLA planning, but rather to augment and cage them. In particular, we integrate a high-level VLA planner with a low-level physical-grounding layer that leverages efficient state estimation, in-situ 3D spatial perception, and an Signed Distance Field (SDF) representation to generate executable trajectories. We provide a navigation planner with provable collision-avoidance guarantees under the stated assumptions by combining a low-dimensional Bezier-parameterized MPPI planner with Discrete Exponential Barrier Function (dEBF) constraints. The proposed Legged-VLN system is modular, making it compatible with various VLM/VLA models and agnostic to the robot platform. The proposed system is validated with extensive real-world experiments on humanoid and quadrupedal robots, demonstrating versatile, efficient, and safe vision-language driven navigation capabilities.

Introduction

Legged robots, including humanoids and quadrupeds, are being engineered to work hand-in-hand with humans and perform practical, real-world tasks with precision and purpose (e.g. [1]). A fundamental prerequisite for accomplishing such tasks is the ability to navigate safely to locations where subsequent actions can be performed. Traditionally, navigation missions are specified as sequences of goal poses through a navigation interface, which typically requires either a trained operator or an expert-authored behavior tree [2]. Recent advances in VLMs/VLAs [3] have enabled VLN, allowing users to specify navigation missions through intuitive natural-language instructions rather than manually defined waypoints. However, to deploy these robots alongside humans, it is imperative that VLN systems are demonstrably safe—generating motion that are both semantically correct and collision free. This work directly addresses this critical challenge of ensuring physical safety in Legged-VLN.

The core issue is that while pretrained models offer powerful visual-semantic reasoning, their navigation outputs [4, 5, 6] typically ignore the robot’s kinematics, local obstacle configurations, and formal safety constraints. Consequently, an end-to-end VLA planner might propose an instruction-following navigation command that are physically unexecutable or prone to collisions. Therefore, rather than executing these outputs directly, they must be treated as high-level navigation proposals that are physically grounded and verified. We refer to this verification process as caging: the interposition of a safety-assured physical-grounding layer between the VLM/VLA planner and the robot’s locomotion controller, preserving the reasoning capabilities of pretrained models while providing provable collision-avoidance guarantees under the stated assumptions.

To this end, we propose a dual-layer caging architecture for Legged-VLN that integrates high-level semantic reasoning with low-level, safety-assured motion planning. We evaluate the complete framework in real-world indoor and outdoor environments using a Unitree G1 humanoid and a Ghost Robotics Vision 60 quadruped.

Specifically, the main contributions of this work are:

  • A dual-layer caging architecture that decouples semantic reasoning from physical execution by treating pretrained VLM/VLA outputs as high-level proposals and mapping them to locally feasible goal poses with open-vocabulary terminal alignment.

  • A safety-assured motion planner utilizing an efficient, low-dimensional Bézier-parameterized Model Predictive Path Integral (MPPI) formulation with SDF-based dEBF constraints, providing provable collision-avoidance guarantees under stated assumptions.

  • Extensive real-world validation on both humanoid and quadrupedal platforms, demonstrating cross-platform applicability and reliable, safe execution of natural-language navigation instructions.

Vision-language navigation for legged robots. VLN was formalized by the Room-to-Room benchmark [7] and later extended from discrete navigation graphs to continuous environments through R2R-CE [8]. Recent VLAs directly map visual and language observations to robot actions. Of particular relevance, QUAR-VLA targets quadruped control [9], while NaVILA performs language-conditioned navigation on legged robots [4]. These models provide strong semantic action proposals, but their learned outputs do not

This work was partially supported by the University of Delaware (UD) College of Engineering, Delaware NASA/EPSCoR Seed Grant, NSF (IIS-2410019), and Toyota Motor Engineering & Manufacturing North America.

The authors are with the Robot Perception and Navigation Group (RPNG), University of Delaware, Newark, DE 19716, USA. Email: {amkulka, saimouli, yuzedu, ghuang}@udel.edu.

Zijun Sha is with the Frontier Research Center, Toyota Motor Corporation, Tokyo, Japan. Email: [email protected]. explicitly encode the robot footprint, the current local obstacle geometry, or a formal closed-loop safety condition. Recent safety-aware VLA methods address this gap either by aligning the policy through constrained learning, as in SafeVLA [10], or by correcting manipulation actions with a plug-and-play Control Barrier Function (CBF) layer, as in VLSA/AEGIS [11]. Our work instead targets legged navigation: it leaves the semantic model unchanged and cages its proposals using online geometric perception and horizon-level motion certification. This separation allows the front end and robot platform to change without transferring responsibility for navigation safety to the learned model.

Semantic and geometric grounding. Methods such as 3D-LLM [12] and Chat-Scene [13] enrich language models with 3D scene structure for spatial reasoning. Open-vocabulary detection and segmentation models, including Grounding-DINO [14] and SAM2 [15], additionally enable language-specified objects to be localized without a fixed task vocabulary. Splat-Nav [16] more tightly couples these domains by using a language-embedded Gaussian-splat map to specify goals, construct safe polytope corridors, and generate Bézier trajectories. Our architecture instead uses the learned front end and open-vocabulary perception to generate semantic goals and refine terminal alignment, while an incrementally updated environmental SDF grounds every proposed motion in locally observed free space. Safety is therefore evaluated from geometric clearance rather than semantic confidence.

Safety in sampling-based predictive control. For legged robots, sequential MPC over convex safe corridors enables reactive bipedal planning in cluttered environments [17]. MPPI supports derivative-free trajectory optimization with nonlinear and nonsmooth costs [18]; low-dimensional trajectory parameterizations further improve sampling efficiency in cluttered environments [19]. Discrete-time barrier conditions provide a route to forward-invariant safety [20, 21], and recent sampling-based controllers incorporate barrier functions through shielded refinement [22], barrier-rate guidance with learned SDFs [23], or discrete barrier states [24]. In contrast, our physical-grounding layer combines Bézier-parameterized

MPPI, value-only queries to an incrementally updated environmental SDF, soft dEBF-guided sampling, and hard rollout certification with a safe backup. This construction cages interchangeable VLM/VLA proposals and is deployed unchanged at the planning level on both quadrupedal and humanoid platforms.

System Overview

We address the problem of language-driven, safety-critical navigation for legged robots operating in cluttered, unknown environments. That is, given (i) a natural-language instruction describing a high-level task and optionally a target object, (ii) onboard sensing (e.g., vision, LiDAR, IMU, leg kinematics), and (iii) platform actuation limits, the objective is to synthesize a control sequence {uk}k=0T−1\{u_k\}_{k=0}^{T-1} that steers the robot from an initial pose x0x_0 to a goal region or an object-relative pose while being caged to safely execute the commands. Formally, the robot must: (i) ground the language into a sequence of high-level, spatially referenced subgoals; (ii) convert these into feasible, collision-free subgoals; and (iii) ensure safety at all times. The solution should be modular (compatible with different legged platforms and VLM/VLA models), and real-time capable. Success is achieved when the robot reaches the goal region or establishes a valid object-relative pose (for open-vocabulary targets), without violating safety constraints throughout the trajectory.

Robot Dynamics and Environment Models

We represent the robot pose in 2D as x=[px,py,ψ]T∈R3\mathbf{x} = [p_x, p_y, \psi]^\mathsf{T} \in\mathbb{R}^3, where (px,py)(p_x,p_y) is the global position and ψ\psi the yaw. The control input is the body-frame twist uk=[vx,vy,ω]T\mathbf{u}_k = [v_x, v_y, \omega]^\mathsf{T} with sampling time Δt\Delta t, and we adopt a constant-velocity, discrete-time dynamic model:

xk+1=xk+R(ψk)ukΔt(1)\mathbf{x}_{k+1} = \mathbf{x}_k + \mathbf{R}(\psi_k)\mathbf{u}_k\Delta t \tag*{(1)}

This planar body-twist model is suitable for motion planning for both the humanoid and quadrupedal robots.

The local environment is encoded by a SDF ϕ:R2→R\phi: \mathbb{R}^2 \to\mathbb{R}. For any point p∈R2\mathbf{p} \in\mathbb{R}^2, ϕ(p)\phi(\mathbf{p}) returns the minimum Euclidean distance to the nearest obstacle surface; ϕ(p)>0\phi(\mathbf{p}) > 0 indicates free space, ϕ(p)=0\phi(\mathbf{p}) = 0 lies on an obstacle boundary, and ϕ(p)<0\phi(\mathbf{p}) < 0 is inside an obstacle.

The Proposed Caging Framework

Our framework employs a hierarchical control architecture that translates high-level semantic instructions into safety-assured motion (see Figure 1). The upper layer converts visual and language inputs into spatial navigation goals. The lower layer converts these goals into feasible control commands while enforcing safety constraints. This separation allows the semantic planner and the robot platform to be changed independently, while preserving a common safety-assured execution layer.

Legged-VLN architecture showing semantic grounding and safety-assured physical grounding

Figure 1. The proposed Legged-VLN architecture separates semantic grounding (top) from safety-assured physical grounding (bottom).

1) Semantic Grounding Layer: This layer converts visual observations and natural-language instructions into a sequence of high-level navigation actions. A VLM processes the user’s command together with the current scene and predicts discrete actions, such as turning left or right in 15∘15^\circ increments up to 45∘45^\circ, or moving forward in 2525 cm increments up to 7575 cm. The Kinematic Goal Projector maps each action to a planar goal pose xgoal\mathbf{x}^{\mathrm{goal}} in the local map and verifies that the resulting waypoint is reachable and collision-free. After the mission-level action sequence is completed, an open-vocabulary object-centric grounding module may be activated to define the robot’s final approach relative to a specified object. The semantic grounding components are detailed in Section IV.

2) Physical Grounding Layer: This layer converts the goal poses xgoal\mathbf{x}^{\mathrm{goal}} provided by the semantic grounding layer into executable control commands while enforcing the prescribed safety constraints. For the humanoid platform, which is equipped with high-quality LiDAR, we use FastLIVO2 [25] to obtain robust LiDAR–inertial odometry. For the quadruped platform, an invariant EKF [26] fuses leg-kinematic and IMU measurements. The local environment is represented by an incrementally updated SDF [27], which provides obstacle-clearance information directly to the motion planner. To enable real-time reactive planning with formal safety guarantees under the stated assumptions, we develop an efficient low-dimensional MPPI planner augmented with dEBF constraints as discussed in § 4. Additionlly an optional JPS global planner [28] can be included to reduce the susceptibility to local minima.

High-Level VLA Planning

The Kinematic Goal Projector converts discrete high-level action commands generated by the VLA model (e.g., turn left, turn right, move forward) into obstacle-free goal poses in the map. To achieve this, the module integrates odometry, and an SDF representation of the environment. The SDF grid is queried for the signed distance ϕ(p)\phi(\mathbf{p}) at candidate position p\mathbf{p}, which is obstacle free if ϕ(p)≥dclear\phi(\mathbf{p}) \geq d_{\mathrm{clear}}, with dclear=0.3d_{\mathrm{clear}} = 0.3 m as the minimum clearance threshold. If the primary position pd\mathbf{p}_d is occupied, we select a valid alternative within a local radius which is after sampling candidate poses on the radius. Candidate poses are evaluated with a cost function that prioritizes proximity to the desired pose, alignment with the robot’s heading, and obstacle clearance:

Ggoal(p)=∥p−pd∥⏟proximity+wψ∣Δψ∣−wϕϕ(p)(2)G_{\mathrm{goal}}(\mathbf{p}) = \underbrace{\lVert\mathbf{p} - \mathbf{p}_d \rVert}_{\text{proximity}} + w_\psi|\Delta\psi| - w_\phi\phi(\mathbf{p}) \tag*{(2)}

where Δψ\Delta\psi is the angular deviation from the forward direction and wψ,wϕw_\psi, w_\phi are weighting parameters. The valid candidate with the lowest cost is dispatched to the local planner. This design ensures the navigation module always receives goals in safe regions. Moreover, this also reduces the risk of local minima and provides more reliable final goal locations for local planners.

The open vocabulary object-centric grounding module is activated after the mission-level language navigation is complete. Its purpose is to semantically ground the final stage of the task: detecting and approaching a user-specified target object. To achieve this, we integrate Grounding-DINO [14] for open-vocabulary object detection and SAM2 [15] for precise segmentation, producing both bounding boxes and fine-grained masks. The segmentation mask or bounding box centroid is then projected into 3D using the robot’s lidar, odometry and camera intrinsics, yielding a target position in the global frame. To reject spurious detections, the target position pt\mathbf{p}_t must satisfy dmin⁡≤∥pt−pr∥≤dmax⁡d_{\min} \leq\lVert\mathbf{p}_t - \mathbf{p}_r \rVert\leq d_{\max}, with dmin⁡=0.5d_{\min} = 0.5 m and dmax⁡=3.0d_{\max} = 3.0 m.

The navigation state transitions through SEARCHING, APPROACHING, and CLOSE, depending on detection confidence and the measured target distance. When the distance falls below a threshold dstopd_{\mathrm{stop}} (set to 0.500.50 m in our experiments), the robot halts in front of the target. This ensures the robot reaches semantically meaningful endpoints, rather than merely completing geometric navigation. As will be shown in our experiments, enabling this module allows the robot to consistently approach user-specified objects, thereby extending the system from pure language-conditioned motion planning to semantically grounded, object-aware interaction.

Physical Grounding Layer

The semantic layer specifies where to move but does not account for robot geometry, obstacles, or closed-loop safety. The Physical Grounding Layer in Figure 1 therefore acts as an execution cage, converting each semantic goal xgoal\mathbf{x}^{\mathrm{goal}} into a dynamically feasible and safe motion. This separation allows the VLM/VLA front end to remain independent of the mechanisms responsible for physical safety.

The planner combines: (i) low-dimensional MPPI, with horizon controls parameterized by Bézier control points; (ii) an SDF-based dEBF cost that biases sampling toward collision-free trajectories; and (iii) a hard rollout filter with a one-step-safe backup policy. The cost improves sampling efficiency, while the filter and backup policy establish safety under the stated modeling, mapping, and tracking assumptions. Unlike collision penalties, which may trade safety for goal progress, the proposed method explicitly certifies each executed command. It also avoids numerical differentiation of an incrementally updated SDF, as required by many gradient-based Model Predictive Control (MPC) formulations [21] and also avoids storage and querying of a separate gradient map. We instead evaluate it along sampled rollouts using only SDF value queries, accommodating nonsmooth costs, nonlinear constraints, and online map updates.

Low-Dimensional Barrier-Certified MPPI

At each control cycle, MPPI samples NN control sequences, rolls out their trajectories, and updates the sampling mean by exponential cost weighting [18]. For T(n)={xk(n)}k=0H\mathcal{T}^{(n)} = \{\mathbf{x}^{(n)}_k\}_{k=0}^{H},

w(n)∝exp⁡(−J(n)/λ),w^{(n)} \propto\exp(-J^{(n)}/\lambda),

where λ>0\lambda> 0 controls concentration around low-cost samples.

To reduce dimension and suppress non-smooth commands, each sequence is represented by a degree-dd Bézier curve [19]:

u(s)=∑j=0d(dj)(1−s)d−jsjPj,s∈[0,1],(3)\mathbf{u}(s) = \sum_{j=0}^{d} \binom{d}{j}(1-s)^{d-j}s^j\mathbf{P}_j,\qquad s \in[0,1], \tag*{(3)}

where {Pj}j=0d\{\mathbf{P}_j\}_{j=0}^{d} are control points. The optimization thus uses d+1d+1 control points instead of one control vector per horizon step and produces smooth commands by construction. After certification, only the first command is executed before replanning. The rollout objective is

J(n)=Cgoal(n)+Cctl(n)+Csafety(n),(4)J^{(n)} = C_{\mathrm{goal}}^{(n)} + C_{\mathrm{ctl}}^{(n)} + C_{\mathrm{safety}}^{(n)}, \tag*{(4)}

where the first two terms rank goal progress and command quality, while Csafety(n)C_{\mathrm{safety}}^{(n)} shapes sampling but does not provide the safety guarantee.

Goal-Tracking and Control Costs

Let pk\mathbf{p}_k and pgoal\mathbf{p}^{\mathrm{goal}} be the current and desired planar positions, with ℓpos(xk)=∥pk−pgoal∥2\ell_{\mathrm{pos}}(\mathbf{x}_k) = \lVert\mathbf{p}_k-\mathbf{p}^{\mathrm{goal}}\rVert_2. Yaw tracks motion direction far from the goal and terminal yaw ψgoal\psi^{\mathrm{goal}} near it

αψ=σ(∥pk−pgoal∥2−dswitchsswitch),(5)\alpha_\psi= \sigma\left(\frac{\lVert\mathbf{p}_k-\mathbf{p}^{\mathrm{goal}}\rVert_2-d_{\mathrm{switch}}}{s_{\mathrm{switch}}}\right), \tag*{(5)}

where dswitchd_{\mathrm{switch}} and sswitchs_{\mathrm{switch}} set the transition location and sharpness, and

ℓyaw(xk)=αψ∣ψk−ψkhead∣+(1−αψ)∣ψk−ψgoal∣,(6)\ell_{\mathrm{yaw}}(\mathbf{x}_k) = \alpha_\psi\lvert\psi_k-\psi_k^{\mathrm{head}}\rvert+(1-\alpha_\psi)\lvert\psi_k-\psi^{\mathrm{goal}}\rvert, \tag*{(6)}

with ψkhead\psi_k^{\mathrm{head}} induced by predicted motion. Thus,

Cgoal=∑k=0H−1(wposℓpos,k+wyawℓyaw,k).(7)C_{\mathrm{goal}} = \sum_{k=0}^{H-1}(w_{\mathrm{pos}}\ell_{\mathrm{pos},k}+w_{\mathrm{yaw}}\ell_{\mathrm{yaw},k}). \tag*{(7)}

Command magnitude and componentwise limit violations are penalized by

Cctl=∑k=0H−1(wctl∥uk∥2+∑q=13wlim[∣uk,q∣−uq,max]+),(8)C_{\mathrm{ctl}} = \sum_{k=0}^{H-1}\left(w_{\mathrm{ctl}}\lVert\mathbf{u}_k\rVert_2+\sum_{q=1}^{3}w_{\mathrm{lim}}[\lvert u_{k,q}\rvert-u_{q,\mathrm{max}}]_+\right), \tag*{(8)}

where [z]+=max⁡(z,0)[z]_+ = \max(z,0). Together with the Bézier representation, this discourages aggressive commands and encourages compliance with platform velocity limits.

SDF-Based dEBF Safety

For barrier function h:Rn→Rh:\mathbb{R}^n\to\mathbb{R}, define the safe set

S≜{x∈Rn∣h(x)≥0}.(9)\mathcal{S} \triangleq\{\mathbf{x}\in\mathbb{R}^n\mid h(\mathbf{x})\geq0\}. \tag*{(9)}

The discrete exponential barrier condition [20]

h(xk+1)≥γh(xk),(10)h(\mathbf{x}_{k+1})\geq\gamma h(\mathbf{x}_k), \tag*{(10)}

with γ∈[0,1)\gamma\in[0,1), limits the rate of approach to the boundary: as h(xk)→0h(\mathbf{x}_k)\to0, the allowable next-step decrease vanishes. Repeated satisfaction preserves nonnegative barriers and makes γ\gamma an interpretable conservatism parameter, with larger values retaining more clearance.

We approximate the robot by MM body-fixed spheres of radii ri≥0r_i\geq0 (Figure 2). The map gi(xk)g_i(\mathbf{x}_k) gives sphere center pi,k∈R2\mathbf{p}_{i,k}\in\mathbb{R}^2, and the SDF ϕ\phi gives clearance ci(xk)≜ϕ(pi,k)−ric_i(\mathbf{x}_k)\triangleq\phi(\mathbf{p}_{i,k})-r_i. Setting hi(xk)≜ci(xk)h_i(\mathbf{x}_k)\triangleq c_i(\mathbf{x}_k) yields

Figure 2. Multi-point safety models for (a) Ghost Robotics Vision 60 and (b) Unitreee G1. Each point has a radius representing the local footprint.

S≜⋂i=1MSi,Si≜{x∣hi(x)≥0}.\mathcal{S}\triangleq\bigcap_{i=1}^{M}\mathcal{S}_i,\qquad\mathcal{S}_i\triangleq\{\mathbf{x}\mid h_i(\mathbf{x})\geq0\}.

The robot is collision-free iff all barriers are nonnegative and both platforms are captured with few SDF queries.

During optimization, predicted dEBF violations are penalized to favor candidates likely to pass hard certification:

Csafety(T(n))=∑k=0H−1∑i=1Mwi[γhi(xk(n))−hi(xk+1(n))+εbar]+,(11)C_{\mathrm{safety}}(\mathcal{T}^{(n)})=\sum_{k=0}^{H-1}\sum_{i=1}^{M}w_i\left[\gamma h_i(\mathbf{x}^{(n)}_k)-h_i(\mathbf{x}^{(n)}_{k+1})+\varepsilon_{\mathrm{bar}}\right]_+, \tag*{(11)}

where wiw_i weights geometric points and εbar>0\varepsilon_{\mathrm{bar}}>0 adds a planning margin. This cost guides sampling; safety is enforced by explicit constraint checking.

The controller first uses (11) to bias sampling toward safe regions, then executes the lowest-cost rollout satisfying (10) for every geometric point and predicted transition. If none is admissible, it applies a one-step-safe backup κ(xk)\kappa(\mathbf{x}_k) satisfying

hi(xk+R(ψk)κ(xk)Δt)≥γhi(xk),∀i.(12)h_i(\mathbf{x}_k+\mathbf{R}(\psi_k)\kappa(\mathbf{x}_k)\Delta t)\geq\gamma h_i(\mathbf{x}_k),\qquad\forall i. \tag*{(12)}

Theorem 1 (Safety of dEBF-Certified MPPI). Assume: (A1) the state follows (1); (A2) valid robot geometry and the SDF define S=⋂i=1M{x∣hi(x)≥0}\mathcal{S}=\bigcap_{i=1}^{M}\{\mathbf{x}\mid h_i(\mathbf{x})\geq0\}; (A3) for every xk∈S\mathbf{x}_k\in\mathcal{S}, a one-step backup κ(xk)\kappa(\mathbf{x}_k) exists and satisfies (12); (A4) the controller executes the first sampled command only if its transition satisfies (10) for every barrier; otherwise apply the backup; and (A5) x0∈S\mathbf{x}_0\in\mathcal{S}. Then, for k∈N0k\in\mathbb{N}_0 and γ∈[0,1)\gamma\in[0,1),

hi(xk)≥γkhi(x0)≥0,∀i∈{1,…,M},(13)h_i(\mathbf{x}_k)\geq\gamma^k h_i(\mathbf{x}_0)\geq0,\qquad\forall i\in\{1,\ldots,M\}, \tag*{(13)}

so S\mathcal{S} is forward invariant.

Algorithm 1 Barrier-Certified Low-Dimensional MPPI
1: Input: $x_0$, $\lambda$, $N_{\mathrm{samples}}$, $N_{\mathrm{iters}}$, $\kappa(\cdot)$
2: Initialize Bézier control points $P$
3: for $m = 1$ to $N_{\mathrm{iters}}$ do
4:   Sample $P^{(j)} \sim \mathcal{N}(P, \Sigma)$, $j = 1, \ldots, N_{\mathrm{samples}}$
5:   for $j = 1$ to $N_{\mathrm{samples}}$ do
6:     Construct $u^{(j)}$ using {xref}`(3)`
7:     Roll out {xref}`(1)` from $x_0$
8:     Evaluate $J^{(j)}$ and $\widetilde{w}^{(j)} \leftarrow \exp(-J^{(j)}/\lambda)$
9:   end for
10:  $w^{(j)} \leftarrow \widetilde{w}^{(j)}/\sum_{\ell=1}^{N_{\mathrm{samples}}}\widetilde{w}^{(\ell)}$
11:  $P \leftarrow \sum_{j=1}^{N_{\mathrm{samples}}}w^{(j)}P^{(j)}$
12: end for
13: $\mathcal{I} \leftarrow \operatorname{argsort}_j J^{(j)}$
14: for $j \in \mathcal{I}$ do
15:   if {xref}`(10)` holds $\forall i,k$ along rollout $j$ then
16:     return first command $u_0^{(j)}$
17:   end if
18: end for
19: return $\kappa(x_0)$

The proof is in Appendix I. For (1), zero body velocity is a valid backup from any safe state, making the planning-level certification recursively feasible. Unlike collision penalty-only MPPI, the dEBF couples consecutive clearances and regulates approach rate rather than only penalizing penetration and thus resulting a longer lookahead with a shorter horizon [21]. Unlike gradient-based SDF-constrained MPC, it requires no map derivatives; unlike CBF-in-MPPI methods with a separate refinement stage [22], it certifies sampled rollouts directly. The margin ϵˉ\bar{\epsilon} improves candidate robustness, while the hard filter and backup enforce the constraint. The resulting front-end-agnostic safety cage reacts to newly observed geometry and its evaluations are presented § 5.

Experimental Validation

We structure the evaluation around three questions. (Q1) How does the proposed dEBF affect obstacle clearance and object-approach behavior relative to standard MPPI? (Q2) Does dEBF reduce observed collision incidence both without a global path planner and when JPS-based global replanning [28] is available? (Q3) Does the complete physical-grounding cage remain effective across environment scales, semantic front ends, and legged platforms? We first characterize dEBF in simulation, then isolate its effect on a Unitree G1 humanoid, and finally evaluate end-to-end navigation on the G1 and a Ghost Robotics Vision 60 quadruped.

Experimental Setup and Metrics

Simulation. We use Gazebo to simulate a Vision 60, LiDAR, and ground-truth odometry. The mapping and perception pipeline mirrors the hardware implementation. The goal is held fixed so that the experiments isolate the local planner under different dEBF parameters and obstacle configurations.

Hardware. The G1 and Vision 60 carry an RGB camera, IMU, and LiDAR for online state estimation, mapping, and geometric grounding. Platform-specific locomotion controllers track the commanded body-frame velocity, while an NVIDIA A4500 executes the mapping and motion-planning stack. The semantic front ends are executed locally or accessed through their respective interfaces.

Planner parameters. For all experiments, the dEBF decay parameter is set to γ=0.95\gamma= 0.95. At each control update, MPPI uses 200 samples and 10 optimization iterations. Each rollout is discretized with a 0.05 s step over a horizon of 50 time steps, corresponding to a 2.5 s prediction horizon.

The G1 is the primary controlled-evaluation platform. We conduct physical grounding layer ablations in a 4 m ×\times 4 m arena, an end-to-end study in a 6 m ×\times 6 m static-clutter environment, and a system-level caging study in a 12 m ×\times 12 m environment. The controlled studies compare MPPI with and without dEBF across ChatGPT, Claude, Gemini, and NaVILA. The moderate-scale study uses 25 instructions, each repeated three times under both planner configurations, yielding 75 trials per configuration. The large-scale study evaluates 15 instructions once with and once without the complete physical-grounding cage for each of the four semantic front ends, yielding 120 executions. The Vision 60 experiments assess cross-platform deployment in real indoor and outdoor environments.

Metrics. Success Rate (SR) is the fraction of trials that complete the instructed navigation task. Collision Rate (CR) is the fraction in which the robot contacts an obstacle or violates the predefined collision boundary. End-to-End Success Rate (E2E SR) requires both task completion and collision-free execution. For object-centric navigation, Object SR denotes successful localization and approach, and Navigation Error (NE) is the final distance to the desired object-relative terminal position, following standard VLN practice [4]. We report only metrics annotated for each study and do not infer unrecorded task success from collision outcomes.

Simulation Characterization of dEBF

Figure 3 illustrates how dEBF shapes local motion. Increasing the decay parameter γ\gamma requires a predicted trajectory to retain a larger fraction of its current barrier value (10) and therefore produces more conservative motion around an obstacle (Figure 3(a)). In a narrow passage, the standard collision-penalty MPPI maintains a relatively high speed until it approaches the obstacle boundary, whereas MPPI+dEBF slows as predicted clearance decreases (Figure 3(b)). The object-approach example similarly shows a smooth reduction in speed during the final approach (Figure 3(c)).

Qualitative effects of dEBF on clearance and approach-rate regulation

Figure 3. Qualitative effects of dEBF on clearance and approach-rate regulation. (a) Effect of γ\gamma on obstacle clearance. (b) Speed modulation in a narrow passage for collision-penalty MPPI and MPPI+dEBF. (c) Smooth final approach in a object approach.

These behaviors arise from the temporal coupling in the discrete barrier condition. A conventional collision penalty evaluates individual predicted states and may trade obstacle proximity against goal progress. In contrast, dEBF constrains the permitted decrease in clearance between consecutive predicted states, regulating both intersection with the safe-set boundary and the rate at which a rollout approaches it. Making dEBF+MPPI highly suitable for safe and smooth navigation through cluttered and narrow spaces.

Controlled dEBF Ablations on G1

We conduct two complementary studies in the 4 m×4 m4\ \mathrm{m} \times4\ \mathrm{m} G1 arena. In the first, JPS is disabled and a fixed obstacle interferes with the nominal direction to a target object. Thus, avoidance is determined by local MPPI, with or without dEBF. In the second, JPS is enabled in both configurations and a previously unobserved obstacle is suddenly introduced after the robot begins moving. The obstacle is incorporated into the online map and triggers global replanning. This paired design evaluates whether dEBF adds short-horizon protection during perception, map updating, and replanning rather than comparing local and global planning. Initial pose, obstacle and target placement, velocity limits, planner parameters, and timeout are held fixed within each comparison.

Tables I and II show a consistent collision reduction in both settings. Without JPS, the aggregate CR decreases from 80% to 0%; with JPS shared by both configurations, it decreases from 100% to 0%. No collisions were observed in any of the 40 dEBF-enabled controlled trials. The aggregate Object SR changes from 35% to 40% in the static study and from 55% to 70% in the sudden-obstacle-introduction study. The corresponding aggregate E2E SR increases from 15% to 40% and from 0% to 70%, respectively. JPS and dEBF serve complementary roles. JPS supplies global route guidance around mapped obstacles, reducing local MPPI’s susceptibility to local minima; dEBF constrains short-horizon rollouts during map-update and replanning transitions.

The consistent collision reduction shows that dEBF improves physical execution across all four tested semantic models. The proposed Legged-VLN architecture physically grounds the model’s navigation proposals but does not modify its semantic reasoning. Task completion therefore remains dependent on the underlying model; for example, the Claude trials in the no-JPS setting are collision-free with dEBF yet do not complete the object-approach task. Thus, these results demonstrate improved collision avoidance without implying improved semantic reasoning or target selection.

End-to-End Scaling and Cross-Platform Transfer

We next evaluate longer language-conditioned missions. On G1, the 6 m×6 m6\ \mathrm{m} \times6\ \mathrm{m} environment contains multiple static obstacles and tasks ranging from one- or two-action instructions to longer sequences involving multiple landmarks and turns. Each of 25 instructions is executed three times under standard MPPI and MPPI+dEBF, yielding 75 trials per configuration. On Vision 60, we use the same planning-level interface in real indoor and outdoor environments. State estimation, sensor coverage, footprint geometry, and low-level locomotion remain platform specific; in particular a platform-specific multi-point footprint is used for SDF queries as in Fig 2.

As shown in Table III, dEBF reduces observed CR on both platforms. On G1, CR decreases from 0.33 to 0.00 for simple tasks and from 0.50 to 0.00 for complex tasks, while SR remains 0.75. The unchanged SR indicates that the reduction was not obtained by suppressing a larger fraction of task executions. On Vision 60, dEBF reduces CR from 0.67 to 0.20 for simple tasks and from 0.91 to 0.08 for complex tasks while increasing the recorded SR.

MethodSimple SR ↑Simple CR ↓Complex SR ↑Complex CR ↓
MPPIUnitree G10.750.330.750.50
MPPI+dEBF0.750.000.750.00
MPPIGhost Robotics Vision 600.470.670.460.91
MPPI+dEBF0.730.200.540.08

Table III. End-to-end G1 and Vision 60 evaluation with MPPI and MPPI+dEBF. Results use 75 G1 trials per planner and the available Vision 60 category-level rates.

The remaining Vision 60 collisions are primarily associated with incomplete environmental representation, including thin wires, table edges, narrow poles, and geometry in sensor blind spots due to an older LiDAR VLP16. Since the barrier is evaluated against the current SDF, absent geometry cannot be certified. Overall, introduction of the proposed caging architecture enhances the safety of the commands proposed by VLM/VLA models.

System-Level Caging Across Semantic Front Ends

The final G1 safety study uses a 12 m×12 m12\ \mathrm{m} \times12\ \mathrm{m} indoor environment and 15 instructions ranging from single-stage commands to multi-stage tasks with multiple landmarks and spatial constraints. For each front end, the same instructions are executed with the complete physical-grounding cage disabled and enabled. Caging enables MPPI, JPS, and dEBF jointly. In uncaged they are turned off to clearly demonstrate the benefits of the proposed caging architecture. Table IV summarizes 120 executions. All of the 60 trials without the proposed caging architecture results in collisions for these longer tasks, whereas no collision is observed with the caging architecture. The same trend occurs within every semantic frontend VLM/VLA model, showing that caging is a necessity for all the models tested with longer tasks.

Front EndCaging OffCaging On
ChatGPT 5.6 Luna15/15 (100%)0/15 (0%)
Gemini 3.5 Flash15/15 (100%)0/15 (0%)
Claude Fable 515/15 (100%)0/15 (0%)
NaVILA15/15 (100%)0/15 (0%)
Aggregate60/60 (100%)0/60 (0%)

Table IV. System-level G1 caging ablation. Each entry reports collisions/trials (CR).

Front EndNNCR Off/OnObj. SR Off/OnE2E SR Off/On
Gemini5100/0%40/80%0/80%
ChatGPT580/0%0/40%0/40%
Claude560/0%80/0%40/0%
NaVILA580/0%20/40%20/40%
Aggregate2080/0%35/40%15/40%

Table I. Static-obstacle G1 evaluation with JPS disabled. Results compare dEBF Off/On using five trials per front end and 20 aggregate trials per planner configuration.

Representative language-conditioned execution on Unitree G1Representative language-conditioned execution on Vision 60

Figure 4. (a) Unitree G1: “Move forward, step onto the black mat, and stop at the TV monitor in front of the window on the right.” (b) Vision 60: “Go out the door, turn left, and stop at the bike.” Representative language-conditioned executions on the humanoid and quadruped platforms.

Object-Centric Grounding and Evaluation Scope

Collision-aware execution does not ensure that a mission terminates at the intended semantic object. We therefore evaluate an auxiliary terminal-grounding stage on Vision 60. After mission-level navigation, Grounding-DINO and SAM2 detect and segment the language-specified target. Its estimated position is projected into the global frame, and the robot performs a final object-relative approach. Table V reports a decrease in mean endpoint error from 1.50 m to 0.29 m with object-centric grounding. This layer is beneficial for missions that require approaching an object.

GroundingEvaluated EndpointsMean NE (m) ↓
Disabled51.50
Enabled30.29

Table V. Object-centric terminal grounding on Vision 60.

Discussion and Limitations

The controlled ablations in § 5-C and end-to-end evaluation in § 5-D show that uncaged collision risk increases with task length and complexity. This trend culminates in the long-horizon G1 ablation (§ 5-E), where all 60 uncaged trials collide, whereas caging reduces the aggregate CR to 0% across four semantic models. JPS provides global route guidance around mapped obstacles, reducing the local planner’s susceptibility to local minima, but does not certify local execution; with JPS enabled, adding dEBF reduces CR from 100% to 0% (Table II). Similar improvements on Vision 60 support portability across the tested task scales, environments, and platforms.

Front EndNNCR Off/OnObj. SR Off/OnE2E SR Off/On
Gemini5100/0%100/100%0/100%
ChatGPT5100/0%60/100%0/100%
Claude5100/0%40/60%0/60%
NaVILA5100/0%20/20%0/20%
Aggregate20100/0%55/70%0/70%

Table II. G1 with JPS enabled and a suddenly introduced obstacle. Results compare dEBF Off/On using 5 trials per front end and 20 aggregate trials per planner configuration.

These results remain subject to the assumptions of Theorem 1. Certification uses the current SDF and cannot account for unobserved geometry. The planar body-twist and multi-point models also exclude state-estimation error, latency, and tracking error. Object-centric grounding (§ 5-F) improves terminal alignment, but the cage cannot correct erroneous or unreachable semantic goals.

Conclusions and Future Work

We presented a modular caging architecture for legged VLN that separates semantic reasoning from physically grounded execution. High-level VLM/VLA outputs are treated as navigation proposals and converted into geometric goals, while a low-dimensional Bézier-parameterized MPPI planner uses the online SDF and dEBF constraints to generate executable motion. This separation permits different semantic front ends and legged platforms to share the same planning-level safety interface. We prove collision avoidance for dEBF-constrained MPPI under the stated assumptions. We extensively evaluated the proposed Legged-VLN architecture on a Unitree G1 humanoid and a Ghost Robotics Vision 60 quadruped. The real-world experiments demonstrate modularity across platforms and substantially reduced observed collision incidence, while open-vocabulary object grounding improves semantic terminal alignment under the tested conditions. Future work will incorporate uncertainty in state estimation and mapping, as well as moving obstacles, into the safety analysis to narrow the gap between the formal model and hardware execution. We will also extend the capabilities of VLA models beyond navigation to richer humanoid skills, including grasping, object handover, and fine-grained target interaction. Finally, integrating multimodal memory will support longer-horizon reasoning toward general-purpose robotic assistants.

Proof of Safe Motion Plan

Theorem 1. By induction on the control step kk.

Base case: By Assumption (A5), x0∈Sx_0 \in S, so hi(x0)≥0h_i(x_0) \ge0 for every ii. Since γ0=1\gamma^0 = 1, we have hi(x0)=γ0hi(x0)≥0h_i(x_0) = \gamma^0 h_i(x_0) \ge0. Thus, (13) holds at k=0k = 0.

Inductive step: Suppose that, at some step kk,

hi(xk)≥γkhi(x0)≥0,∀i.h_i(x_k) \ge\gamma^k h_i(x_0) \ge0,\qquad\forall i.

By Assumption (A4), the controller applies one of two inputs. If a certified MPPI candidate exists, it executes a first command whose resulting transition satisfies (10) for every ii. Otherwise, it applies the backup input uk=κ(xk)u_k = \kappa(x_k), which satisfies the same one-step condition by Assumption (A3) and (12). Hence, in either case,

hi(xk+1)≥γhi(xk),∀i.h_i(x_{k+1}) \ge\gamma h_i(x_k),\qquad\forall i.

Applying the inductive hypothesis for all ii gives

hi(xk+1)≥γhi(xk)≥γ(γkhi(x0))=γk+1hi(x0)≥0,h_i(x_{k+1}) \ge\gamma h_i(x_k) \ge\gamma\left(\gamma^k h_i(x_0)\right) = \gamma^{k+1}h_i(x_0) \ge0,

where the final inequality follows from γ∈[0,1)\gamma\in[0,1) and hi(x0)≥0h_i(x_0) \ge0. Therefore, (13) holds at step k+1k+1. By induction, every barrier remains nonnegative for all k∈N0k \in\mathbb{N}_0. Thus xk∈Sx_k \in S for all time, and the safe set is forward invariant.

References

  1. [1]Unitree Robotics, “Unitree A2, a Quadruped Robotic Dog with Industrial Applications [Video],” YouTube video, 2025, launched in Hangzhou on August 5, 2025.
  2. [2]S. Macenski, F. Martín, R. White, and J. G. Clavero, “The marathon 2: A navigation system,” in 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020, pp. 2718–2725.DOI
  3. [3]R. Sapkota, Y. Cao, K. I. Roumeliotis, and M. Karkee, “Vision-language-action models: Concepts, progress, applications and challenges,” arXiv preprint arXiv:2505.04769, 2025.arxiv.org/abs/2505.04769
  4. [4]A.-C. Cheng, Y. Ji, Z. Yang, Z. Gongye, X. Zou, J. Kautz, E. Biyik, H. Yin, S. Liu, and X. Wang, “Navila: Legged robot vision-language-action model for navigation,” in Robotics: Science and Systems (RSS), June 2025.arxiv.org/abs/2412.04453
  5. [5]Q. Bu, Y. Yang, J. Cai, S. Gao, G. Ren, M. Yao, P. Luo, and H. Li, “Univla: Learning to act anywhere with task-centric latent actions,” arXiv preprint arXiv:2505.06111, 2025.arxiv.org/abs/2505.06111
  6. [6]X. Chen, H. Wei, P. Zhang, C. Zhang, K. Wang, Y. Guo, R. Yang, Y. Wang, X. Xiao, L. Zhao, J. Chen, and J. Bian, “villa-x: Enhancing latent action modeling in vision-language-action models,” arXiv preprint arXiv:2507.23682, 2025.arxiv.org/abs/2507.23682
  7. [7]P. Anderson, Q. Wu, D. Teney, J. Bruce, M. Johnson, N. Sünderhauf, I. Reid, S. Gould, and A. Van Den Hengel, “Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments,” in CVPR, 2018.arxiv.org/abs/1711.07280
  8. [8]J. Krantz, E. Wijmans, A. Majumdar, D. Batra, and S. Lee, “Beyond the nav-graph: Vision-and-language navigation in continuous environments,” in ECCV, 2020.arxiv.org/abs/2004.02857
  9. [9]P. Ding, H. Zhao, Z. Wang, Z. Wei, S. Lyu, and D. Wang, “Quar-vla: Vision-language-action model for quadruped robots,” in ECCV, 2024.arxiv.org/abs/2312.14457
  10. [10]B. Zhang, Y. Zhang, J. Ji, Y. Lei, J. Dai, Y. Chen, and Y. Yang, “SafeVLA: Towards safety alignment of vision-language-action model via constrained learning,” in Advances in Neural Information Processing Systems, vol. 38, 2025.arxiv.org/abs/2503.03480
  11. [11]S. Hu, Z. Liu, S. Liu, J. C. L. Chen, Z. Meng, S. Wang, X. Li, and X. He, “VIsa: Vision-language-action models with plug-and-play safety constraint layer,” arXiv, 2026.arxiv.org/abs/2512.11891
  12. [12]Y. Hong, H. Zhen, P. Chen, S. Zheng, Y. Du, Z. Chen, and C. Gan, “3dllm: Injecting the 3d world into large language models,” in NeurIPS, 2023.arxiv.org/abs/2307.12981
  13. [13]H. Huang, Z. Wang, R. Huang, L. Liu, X. Cheng, Y. Zhao, T. Jin, and Z. Zhao, “Chat-scene: Bridging 3d scene and large language models with object identifiers,” arXiv preprint, 2024.arxiv.org/abs/2312.08168
  14. [14]S. Liu, Z. Zeng, T. Ren, F. Li, H. Zhang, J. Yang, C. Li, J. Yang, H. Su, J. Zhu, et al., “Grounding dino: Marrying dino with grounded pre-training for open-set object detection,” arXiv:2303.05499, 2023.arxiv.org/abs/2303.05499
  15. [15]N. Ravi, V. Gabeur, Y.-T. Hu, R. Hu, C. Ryali, T. Ma, H. Khedr, R. Rädle, C. Rolland, L. Gustafson, E. Mintun, J. Pan, K. V. Alwala, N. Carion, C.-Y. Wu, R. Girshick, P. Dollár, and C. Feichtenhofer, “Sam 2: Segment anything in images and videos,” 2024.arxiv.org/abs/2408.00714
  16. [16]T. Chen, O. Shorinhwa, J. Bruno, A. Swann, J. Yu, W. Zeng, K. Nagami, P. M. Dames, and M. Schwager, “Splat-nav: Safe real-time robot navigation in gaussian splatting maps,” IEEE Transactions on Robotics, vol. 41, pp. 2765–2784, Mar. 2025.arxiv.org/abs/2403.02751
  17. [17]K. S. Narkhede, A. M. Kulkarni, D. A. Thanki, and I. Poulakakis, “A sequential mpc approach to reactive planning for bipedal robots using safe corridors in highly cluttered environments,” IEEE Robotics and Automation Letters, vol. 7, no. 4, pp. 11831–11838, 2022.DOI
  18. [18]G. Williams, A. Aldrich, and E. A. Theodorou, “Information-theoretic model predictive control: Theory and applications to autonomous driving,” IEEE Transactions on Robotics, vol. 34, no. 6, pp. 1603–1622, 2018.arxiv.org/abs/1707.02342
  19. [19]A. M. Kulkarni, I. Poulakakis, and G. Huang, “Learning neural observer-predictor models for limb-level sampling-based locomotion planning,” in 2026 IEEE International Conference on Robotics and Automation (ICRA), 2026, pp. 21 515–21 522.arxiv.org/abs/2510.22789
  20. [20]A. Agrawal and K. Sreenath, “Discrete control barrier functions for safety-critical control of discrete systems with application to bipedal robot navigation,” in Proc. Robot. Sci. Syst., 2017.DOI
  21. [21]J. Zeng, B. Zhang, and K. Sreenath, “Safety-critical model predictive control with discrete-time control barrier function,” in Proc. Amer. Control Conf., 2021, pp. 3882–3889.arxiv.org/abs/2007.11718
  22. [22]J. Yin, C. Dawson, C. Fan, and P. Tsiotras, “Shield model predictive path integral: A computationally efficient robust mpc approach using control barrier functions,” arXiv:2302.11719, 2023.arxiv.org/abs/2302.11719
  23. [23]H. Parwana, T. Kim, K. Long, B. Hoxha, H. Okamoto, G. Fainekos, and D. Panagou, “Br-mppi: Barrier rate guided mppi for enforcing multiple inequality constraints with learned signed distance field,” arXiv:2506.07325, 2025.arxiv.org/abs/2506.07325
  24. [24]F. Wang, Y. Cheng, and C. Tao, “Mppi-dbas: Safe trajectory optimization with adaptive exploration,” arXiv:2502.14387, 2025.arxiv.org/abs/2502.14387
  25. [25]C. Zheng, W. Xu, Z. Zou, T. Hua, C. Yuan, D. He, B. Zhou, Z. Liu, J. Lin, F. Zhu, Y. Ren, R. Wang, F. Meng, and F. Zhang, “Fast-livo2: Fast, direct lidar–inertial–visual odometry,” IEEE Transactions on Robotics, vol. 41, pp. 326–346, 2025.arxiv.org/abs/2408.14035
  26. [26]A. Barrau and S. Bonnabel, “The invariant extended kalman filter as a stable observer,” IEEE Transactions on Automatic Control, vol. 62, no. 4, pp. 1797–1812, 2017.arxiv.org/abs/1410.1465
  27. [27]L. Han, F. Gao, B. Zhou, and S. Shen, “Fiesta: Fast incremental euclidean distance fields for online motion planning of aerial robots,” in 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), IEEE, 2019, pp. 4423–4430.
  28. [28]D. Harabor and A. Grastien, “Online graph pruning for pathfinding on grid maps,” in Proceedings of the Twenty-Fifth AAAI Conference on Artificial Intelligence, vol. 25, no. 1, 2011, pp. 1114–1119.DOI

Paper details

vlnc:W_tj8q4gz7
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 19

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents