betaFeedback

WorksComputer Science

UNITE: Consolidating Embodied VLM Specialists via Shared-Anchor One-Bit Factorization

Abstract

Embodied capabilities such as visual grounding, spatial reasoning, navigation, and driving understanding are often developed in separate vision-language specialists fine-tuned from a common pretrained model.Deploying the specialists separately multiplies storage, whereas merging them into one dense model forces conflicting task updates to share parameters and loses part of each specialist's expertise.We introduce UNITE, a framework that consolidates such specialists into a compact shared-backbone mixture of experts. UNITE keeps the feed-forward block of each specialist as a routable expert and represents the expert bank through a shared-anchor one-bit factorization: a shared floating-point anchor plus expert-specific, row-scaled binary deviations.The anchor is optimized jointly with the scales and signs through closed-form alternating updates, using only source weights and no calibration data.Router adaptation then adjusts token-level expert selection with the backbone and experts frozen, updating fewer than 0.60.6M parameters (<0.01%<0.01\% of the 8.88.8B active parameters). Built from four Qwen3-VL-8B specialists, the complete UNITE model occupies 20.320.3 GB, 1.16×1.16\times the storage of one dense specialist and 59.6%59.6\% below the uncompressed MoE. Across 2222 benchmarks and R2R Val-Unseen, UNITE matches six static merging methods in mean retention of best-specialist performance, leads all of them on Scan2Cap, ScanQA, and all four navigation metrics, and surpasses every source specialist on all five 3D scene understanding benchmarks (up to +19.1+19.1 on Scan2Cap) and on R2R navigation. Relative to the uncompressed MoE, factorization is nearly lossless on 2121 of 2222 benchmarks; the exception is VSI-Bench, where frame-count experiments suggest impaired use of cross-frame evidence.UNITE thus consolidates complementary embodied capabilities near single-model storage.

Paper details

vlnc:W_ezwex2b6
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
arXiv nonexclusive license
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…