ST

Shengbang Tong

cs.CVcs.LGcs.AIcs.CLcs.MMclosed-loop transcriptioncs.CYcs.ROcs.SEeess.IV

On Valency

published · living versions

JUICE: Unified Multimodal Modeling in Pretrained Representation Spaces

with Jiahan Li, Huangyuan Su, Shusheng Yang, Ruihan Guo +7

1 version

W_9td973rjv1 · current

Preprints & journals

36 papers in the corpus · 2021–2026

JUICE: Unified Multimodal Modeling in Pretrained Representation Spaces

Jiahan Li, Huangyuan Su, Shengbang Tong et al.

0 citations

2610.28801v12026arXiv

Towards Physics of Multimodal Pretraining: Knowledge Flow, Modality Synergy, Early Unification, and Recipes

Junlin Han, Shengbang Tong, David Fan et al.

0 citations

2608.05000v22026arXiv

WorldBench: A Challenging and Visually Diverse Multimodal Reasoning Benchmark

Yida Yin, Harish Krishnakumar, Chung Peng Lee et al.

0 citations

2606.06538v12026arXiv

VisionFoundry: Teaching VLMs Visual Perception with Synthetic Images

Guanyu Zhou, Yida Yin, Wenhao Chai et al.

0 citations

2604.09531v22026arXiv

Beyond Language Modeling: An Exploration of Multimodal Pretraining

Shengbang Tong, David Fan, John Nguyen et al.

0 citations

2603.03276v12026arXiv

Asymmetric Idiosyncrasies in Multimodal Models

Muzi Tao, Chufan Shi, Huijuan Wang et al.

0 citations

2602.22734v12026arXiv

Reliable and Responsible Foundation Models: A Comprehensive Survey

Xinyu Yang, Junlin Han, Rishi Bommasani et al.

0 citations

2602.08145v12026arXiv

Scaling Text-to-Image Diffusion Transformers with Representation Autoencoders

Shengbang Tong, Boyang Zheng, Ziteng Wang et al.

0 citations

2601.16208v12026arXiv

Cambrian-S: Towards Spatial Supersensing in Video

Shusheng Yang, Jihan Yang, Pinzhi Huang et al.

0 citations

2511.04670v12025arXiv

Diffusion Transformers with Representation Autoencoders

Boyang Zheng, Nanye Ma, Shengbang Tong et al.

0 citations

2510.11690v12025arXiv

Learning to See Before Seeing: Demystifying LLM Visual Priors from Language Pre-training

Junlin Han, Shengbang Tong, David Fan et al.

0 citations

2509.26625v12025arXiv

From Intention to Execution: Probing the Generalization Boundaries of Vision-Language-Action Models

Irving Fang, Juexiao Zhang, Shengbang Tong et al.

0 citations

2506.09930v12025arXiv

Thinking vs. Doing: Agents that Reason by Scaling Test-Time Interaction

Junhong Shen, Hao Bai, Lunjun Zhang et al.

0 citations

2506.07976v22025arXiv

Seeing from Another Perspective: Evaluating Multi-View Understanding in MLLMs

Chun-Hsiao Yeh, Chenyu Wang, Shengbang Tong et al.

3 citations

2504.15280v22025arXiv

Scaling Language-Free Visual Representation Learning

David Fan, Shengbang Tong, Jiachen Zhu et al.

5 citations

2504.01017v12025arXiv

SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training

Tianzhe Chu, Yuexiang Zhai, Jihan Yang et al.

7 citations

2501.17161v22025arXiv

MetaMorph: Multimodal Understanding and Generation via Instruction Tuning

Shengbang Tong, David Fan, Jiachen Zhu et al.

18 citations

2412.14164v12024arXiv

Connecting Joint-Embedding Predictive Architecture with Contrastive Self-supervised Learning

Shentong Mo, Shengbang Tong

11 citations

2410.19560v12024arXiv

MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmark

Xiang Yue, Tianyu Zheng, Yuansheng Ni et al.

30 citations

2409.02813v32024arXiv

Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs

Shengbang Tong, Ellis Brown, Penghao Wu et al.

68 citations

2406.16860v22024arXiv

Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning

Yuexiang Zhai, Hao Bai, Zipeng Lin et al.

35 citations

2405.10292v32024arXiv

Ctrl123: Consistent Novel View Synthesis via Closed-Loop Transcription

Hongxiang Zhao, Xili Dai, Jianan Wang et al.

0 citations

2403.10953v22024arXiv

Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs

Shengbang Tong, Zhuang Liu, Yuexiang Zhai et al.

189 citations

2401.06209v22024arXiv

White-Box Transformers via Sparse Rate Reduction: Compression Is All There Is?

Yaodong Yu, Sam Buchanan, Druv Pai et al.

6 citations

2311.13110v42023arXiv

Investigating the Catastrophic Forgetting in Multimodal Large Language Models

Yuexiang Zhai, Shengbang Tong, Xiao Li et al.

24 citations

2309.10313v42023arXiv

Emergence of Segmentation with Minimalistic White-Box Transformers

Yaodong Yu, Tianzhe Chu, Shengbang Tong et al.

6 citations

2308.16271v12023arXiv

Mass-Producing Failures of Multimodal Systems with Language Models

Shengbang Tong, Erik Jones, Jacob Steinhardt

15 citations

2306.12105v22023arXiv

Image Clustering via the Principle of Rate Reduction in the Age of Pretrained Models

Tianzhe Chu, Shengbang Tong, Tianjiao Ding et al.

1 citation

2306.05272v52023arXiv

White-Box Transformers via Sparse Rate Reduction

Yaodong Yu, Sam Buchanan, Druv Pai et al.

65 citations

2306.01129v12023arXiv

EMP-SSL: Towards Self-Supervised Learning in One Training Epoch

Shengbang Tong, Yubei Chen, Yi Ma et al.

12 citations

2304.03977v12023arXiv

Closed-Loop Transcription via Convolutional Sparse Coding

Xili Dai, Ke Chen, Shengbang Tong et al.

2 citations

2302.09347v12023arXiv

Unsupervised Manifold Linearizing and Clustering

Tianjiao Ding, Shengbang Tong, Kwan Ho Ryan Chan et al.

7 citations

2301.01805v22023arXiv

Unsupervised Learning of Structured Representations via Closed-Loop Transcription

Shengbang Tong, Xili Dai, Yubei Chen et al.

3 citations

2210.16782v12022arXiv

Revisiting Sparse Convolutional Model for Visual Recognition

Xili Dai, Mingyang Li, Pengyuan Zhai et al.

29 citations

2210.12945v12022arXiv

CTRL: Closed-Loop Transcription to an LDR via Minimaxing Rate Reduction.

Dai, Xili, Tong, Shengbang, Li, Mingyang et al.

20 citations

354551202022Entropy (Basel, Switzerland). 2022;24(4)

Incremental Learning of Structured Memory via Closed-Loop Transcription

Shengbang Tong, Xili Dai, Ziyang Wu et al.

6 citations

2202.05411v32022arXiv

Career total: 40 works. 36 are in this corpus.

Profile built from the corpus for this byline.

Author records are still filling in while the Hub is in beta. If this is your page, you'll be able to claim it soon. Spot a mistake? Tell us.