ZG
Zhe Gan
cs.CVcs.CLcs.LGcs.AIstat.MLcs.MMcs.NEcs.ROcs.SDstat.AP
On Valency
published · living versionsW_eure52qd·v1 · currentpublished
OpenVLHarness: Real World Visual Agentic Tool-Use with Multimodal Memory
with Yu Zhou, Cheng-Fu Yang, Rui Sun, Di Wu +6
1 version
Preprints & journals
141 papers in the corpus · 2015–2026OpenVLHarness: Real World Visual Agentic Tool-Use with Multimodal Memory2610.26860v1 · Yu Zhou, Cheng-Fu Yang, Rui Sun et al.2026 · 0 citationsarXivon Valency
Visualizing Distribution Coverage in Generative Diffusion Models2609.38853v1 · Yifei Wang, Xiaoyu Wu, Tsu-Jui Fu et al.2026 · 0 citationsarXiv
Taming Outlier Tokens in Diffusion Transformers2605.05206v1 · Xiaoyu Wu, Yifei Wang, Tsu-Jui Fu et al.2026 · 0 citationsarXiv
Length Value Model: Scalable Value Pretraining for Token-Level Length Modeling2604.27039v2 · Zhen Zhang, Changyi Yang, Zijie Xia et al.2026 · 0 citationsarXiv
SO-Bench: A Structural Output Evaluation of Multimodal LLMs2511.21750v3 · Di Feng, Kaixin Ma, Feng Nan et al.2025 · 0 citationsarXiv
PRISM-Bench: A Benchmark of Puzzle-Based Visual Tasks with CoT Error Detection2510.23594v4 · Yusu Qian, Cheng Wan, Chao Jia et al.2025 · 0 citationsarXiv
Pico-Banana-400K: A Large-Scale Dataset for Text-Guided Image Editing2510.19808v1 · Yusu Qian, Eli Bocek-Rivele, Liangchen Song et al.2025 · 0 citationsarXiv
UltraCUA: A Foundation Model for Computer Use Agents with Hybrid Action2510.17790v3 · Yuhao Yang, Zhen Yang, Zi-Yi Dou et al.2025 · 0 citationsarXiv
DeepMMSearch-R1: Empowering Multimodal LLMs in Multimodal Web Search2510.12801v1 · Kartik Narayan, Yang Xu, Tian Cao et al.2025 · 0 citationsarXiv
Ferret-UI Lite: Lessons from Building Small On-Device GUI Agents2509.26539v1 · Zhen Yang, Zi-Yi Dou, Di Feng et al.2025 · 0 citationsarXiv
MANZANO: A Simple and Scalable Unified Multimodal Model with a Hybrid Vision Tokenizer2509.16197v1 · Yanghao Li, Rui Qian, Bowen Pan et al.2025 · 0 citationsarXiv
Apple Intelligence Foundation Language Models: Tech Report 20252507.13575v3 · Ethan Li, Anders Boesen Lindbo Larsen, Chen Zhang et al.2025 · 0 citationsarXiv
UniVG: A Generalist Diffusion Model for Unified Image Generation and Editing2503.12652v2 · Tsu-Jui Fu, Yusu Qian, Chen Chen et al.2025 · 2 citationsarXiv
From Multimodal LLMs to Generalist Embodied Agents: Methods and Lessons2412.08442v1 · Andrew Szot, Bogdan Mazoure, Omar Attia et al.2024 · 8 citationsarXiv
Multimodal Autoregressive Pre-training of Large Vision Encoders2411.14402v1 · Enrico Fini, Mustafa Shukor, Xiujun Li et al.2024 · 24 citationsarXiv
Ferret-UI 2: Mastering Universal User Interface Understanding Across Platforms2410.18967v2 · Zhangheng Li, Keen You, Haotian Zhang et al.2024 · 0 citationsarXiv
Improve Vision Language Model Chain-of-thought Reasoning2410.16198v1 · Ruohong Zhang, Bowen Zhang, Yanghao Li et al.2024 · 14 citationsarXiv
MM-Ego: Towards Building Egocentric Multimodal LLMs for Video QA2410.07177v2 · Hanrong Ye, Haotian Zhang, Erik Daxberger et al.2024 · 0 citationsarXiv
Revisit Large-Scale Image-Caption Data in Pre-training Multimodal Foundation Models2410.02740v1 · Zhengfeng Lai, Vasileios Saveris, Chen Chen et al.2024 · 0 citationsarXiv
Contrastive Localized Language-Image Pre-Training2410.02746v2 · Hong-You Chen, Zhengfeng Lai, Haotian Zhang et al.2024 · 2 citationsarXiv
MM1.5: Methods, Analysis & Insights from Multimodal LLM Fine-tuning2409.20566v1 · Haotian Zhang, Mingfei Gao, Zhe Gan et al.2024 · 1 citationarXiv
SlowFast-LLaVA: A Strong Training-Free Baseline for Video Large Language Models2407.15841v2 · Mingze Xu, Mingfei Gao, Zhe Gan et al.2024 · 5 citationsarXiv
Understanding Alignment in Multimodal LLMs: A Comprehensive Study2407.02477v1 · Elmira Amirloo, Jean-Philippe Fauconnier, Christoph Roesmann et al.2024 · 1 citationarXiv
MIA-Bench: Towards Better Instruction Following Evaluation of Multimodal LLMs2407.01509v5 · Yusu Qian, Hanrong Ye, Jean-Philippe Fauconnier et al.2024 · 1 citationarXiv
Ferret-v2: An Improved Baseline for Referring and Grounding with Large Language Models2404.07973v1 · Haotian Zhang, Haoxuan You, Philipp Dufter et al.2024 · 0 citationsarXiv
Ferret-UI: Grounded Mobile UI Understanding with Multimodal LLMs2404.05719v1 · Keen You, Haotian Zhang, Eldon Schoop et al.2024 · 42 citationsarXiv
MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training2403.09611v4 · Brandon McKinzie, Zhe Gan, Jean-Philippe Fauconnier et al.2024 · 15 citationsarXiv
InfoVisDial: An Informative Visual Dialogue Dataset by Bridging Large Multimodal and Language Models2312.13503v1 · Bingbing Wen, Zhengyuan Yang, Jianfeng Wang et al.2023 · 0 citationsarXiv
Pre-trained Language Models Do Not Help Auto-regressive Text-to-Image Generation2311.16201v2 · Yuhui Zhang, Brandon McKinzie, Zhe Gan et al.2023 · 1 citationarXiv
VeCLIP: Improving CLIP Training via Visual-enriched Captions2310.07699v3 · Zhengfeng Lai, Haotian Zhang, Bowen Zhang et al.2023 · 23 citationsarXiv
Ferret: Refer and Ground Anything Anywhere at Any Granularity2310.07704v1 · Haoxuan You, Haotian Zhang, Zhe Gan et al.2023 · 59 citationsarXiv
Compressing LLMs: The Truth is Rarely Pure and Never Simple2310.01382v2 · Ajay Jaiswal, Zhe Gan, Xianzhi Du et al.2023 · 4 citationsarXiv
Guiding Instruction-based Image Editing via Multimodal Large Language Models2309.17102v2 · Tsu-Jui Fu, Wenze Hu, Xianzhi Du et al.2023 · 17 citationsarXiv
Multimodal Foundation Models: From Specialists to General-Purpose Assistants2309.10020v1 · Chunyuan Li, Zhe Gan, Zhengyuan Yang et al.2023 · 207 citationsarXiv
MOFI: Learning Image Representations from Noisy Entity Annotated Images2306.07952v3 · Wentao Wu, Aleksei Timofeev, Chen Chen et al.2023 · 4 citationsarXiv
An Empirical Study of Multimodal Model Merging2304.14933v2 · Yi-Lin Sung, Linjie Li, Kevin Lin et al.2023 · 18 citationsarXiv
Diagnostic Benchmark and Iterative Inpainting for Layout-Guided Image Generation2304.06671v3 · Jaemin Cho, Linjie Li, Zhengyuan Yang et al.2023 · 8 citationsarXiv
Generalized Decoding for Pixel, Image, and Language2212.11270v1 · Xueyan Zou, Zi-Yi Dou, Jianwei Yang et al.2022 · 196 citationsarXiv
GRiT: A Generative Region-to-text Transformer for Object Understanding2212.00280v1 · Jialian Wu, Jianfeng Wang, Zhengyuan Yang et al.2022 · 54 citationsarXiv
ReCo: Region-Controlled Text-to-Image Generation2211.15518v1 · Zhengyuan Yang, Jianfeng Wang, Zhe Gan et al.2022 · 125 citationsarXiv
Exploring Discrete Diffusion Models for Image Captioning2211.11694v2 · Zixin Zhu, Yixuan Wei, Jianfeng Wang et al.2022 · 14 citationsarXiv
Prompting GPT-3 To Be Reliable2210.09150v2 · Chenglei Si, Zhe Gan, Zhengyuan Yang et al.2022 · 61 citationsarXiv
Vision-Language Pre-training: Basics, Recent Advances, and Future Trends2210.09263v1 · Zhe Gan, Linjie Li, Chunyuan Li et al.2022 · 185 citationsarXiv
Non-Contrastive Learning Meets Language-Image Pre-Training2210.09304v1 · Jinghao Zhou, Li Dong, Zhe Gan et al.2022 · 23 citationsarXiv
An Empirical Study of End-to-End Video-Language Transformers with Masked Visual Modeling2209.01540v5 · Tsu-Jui Fu, Linjie Li, Zhe Gan et al.2022 · 57 citationsarXiv
NUWA-Infinity: Autoregressive over Autoregressive Generation for Infinite Visual Synthesis2207.09814v2 · Chenfei Wu, Jian Liang, Xiaowei Hu et al.2022 · 32 citationsarXiv
LAVENDER: Unifying Video-Language Understanding as Masked Language Modeling2206.07160v1 · Linjie Li, Zhe Gan, Kevin Lin et al.2022 · 70 citationsarXiv
GIT: A Generative Image-to-text Transformer for Vision and Language2205.14100v5 · Jianfeng Wang, Zhengyuan Yang, Xiaowei Hu et al.2022 · 237 citationsarXiv
Injecting Semantic Concepts into End-to-End Image Captioning2112.05230v2 · Zhiyuan Fang, Jianfeng Wang, Xiaowei Hu et al.2021 · 140 citationsCVPR 2022
MLP Architectures for Vision-and-Language Modeling: An Empirical Study2112.04453v1 · Yixin Nie, Linjie Li, Zhe Gan et al.2021 · 5 citationsarXiv
Career total: 172 works. 141 are in this corpus.Showing the 50 most recent.
Profile built from the corpus for this byline.
Author records are still filling in while the Hub is in alpha. If this is your page, you'll be able to claim it soon. Spot a mistake? Tell us.