betaFeedback

WorksEngineering

What Makes Good Datasets for Intelligent Multi-Modal Sensing-Communication Integration?

Abstract

Intelligent multi-modal sensing-communication integration, namely Synesthesia of Machines (SoM), represents an AI-native paradigm for enabling embodied intelligence and future wireless networks. Multi-modal sensing and communication foundation models developed for SoM learn cross-modal associations and generalizable environmental representations through large-scale pretraining, making the quality of pretraining datasets critical. However, existing methods lack unified evaluation of heterogeneous multi-modal data and thus struggle to meet the requirements for multi-modal pretraining dataset construction and selection. To bridge this gap, we introduce a comprehensive data evaluation framework for intelligent multi-modal sensing-communication integration. Specifically, we establish two complementary heterogeneous multi-modal data evaluation dimensions, including intrinsic data quality and validity, and model-based data utility. Furthermore, we present case studies to validate the effectiveness of the two evaluation dimensions. We also extend our analysis to investigate the impact of data quality on pretraining scaling laws and explore the value of integrating measurement and simulation data. Finally, we outline promising directions for future research on data evaluation.

Paper details

vlnc:W_r4gqv6yh
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
arXiv nonexclusive license
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Engineering

Not recorded for this paper

View statement
Looking for related papers…