MD
Mostafa Dehghani
cs.LGcs.CLcs.AIcs.CVcs.IRstat.MLcs.HCcs.NEcs.ITeess.IV
On Valency
published · living versionsW_8dhkmaf9·v1 · currentpublished
MetNet: A Neural Weather Model for Precipitation Forecasting
with Casper Kaae Sønderby, Lasse Espeholt, Jonathan Heek, Avital Oliver +4
1 version
Preprints & journals
63 papers in the corpus · 2016–2024Frozen Feature Augmentation for Few-Shot Image Classification2403.10519v2 · Andreas Bar, Neil Houlsby, Mostafa Dehghani et al.2024 · 26 citationsarXiv
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context2403.05530v5 · Gemini Team Google: Petko Georgiev, Ving Ian Lei, Ryan Burnell et al.2024 · 302 citationsarXiv
Fractal Patterns May Illuminate the Success of Next-Token Prediction2402.01825v2 · Ibrahim Alabdulmohsin, Vinh Q. Tran, Mostafa Dehghani2024 · 2 citationsarXiv
Gemini: A Family of Highly Capable Multimodal Models2312.11805v5 · Gemini Team Google: Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac et al.2023 · 988 citationsarXiv
Low-Rank Adaptation for Multilingual Summarization: An Empirical Study2311.08572v2 · Chenxi Whitehouse, Fantine Huot, Jasmijn Bastings et al.2023 · 12 citationsarXiv
How (not) to ensemble LVLMs for VQA2310.06641v2 · Lisa Alazraki, Lluis Castrejon, Mostafa Dehghani et al.2023 · 0 citationsarXiv
The Impact of Group Membership Bias on the Quality and Fairness of Exposure in Ranking2308.02887v2 · Ali Vardasbi, Maarten de Rijke, Fernando Diaz et al.2023 · 3 citationsarXiv
Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolution2307.06304v1 · Mostafa Dehghani, Basil Mustafa, Josip Djolonga et al.2023 · 45 citationsarXiv
PaLI-X: On Scaling up a Multilingual Vision and Language Model2305.18565v1 · Xi Chen, Josip Djolonga, Piotr Padlewski et al.2023 · 42 citationsarXiv
PaLM 2 Technical Report2305.10403v3 · Rohan Anil, Andrew M. Dai, Orhan Firat et al.2023 · 158 citationsarXiv
End-to-End Spatio-Temporal Action Localisation with Video Transformers2304.12160v1 · Alexey Gritsenko, Xuehan Xiong, Josip Djolonga et al.2023 · 32 citationsarXiv
Scaling Vision Transformers to 22 Billion Parameters2302.05442v1 · Mostafa Dehghani, Josip Djolonga, Basil Mustafa et al.2023 · 112 citationsarXiv
Dual PatchNorm2302.01327v3 · Manoj Kumar, Mostafa Dehghani, Neil Houlsby2023 · 0 citationsarXiv
Adaptive Computation with Elastic Input Sequence2301.13195v2 · Fuzhao Xue, Valerii Likhosherstov, Anurag Arnab et al.2023 · 7 citationsarXiv
DSI++: Updating Transformer Memory with New Documents2212.09744v3 · Sanket Vaibhav Mehta, Jai Gupta, Yi Tay et al.2022 · 24 citationsarXiv
Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpoints2212.05055v2 · Aran Komatsuzaki, Joan Puigcerver, James Lee-Thorp et al.2022 · 15 citationsarXiv
Karyotype AI for Precision Oncology2211.14312v5 · Zahra Shamsi, Isaac Reid, Drew Bryant et al.2022 · 2 citationsarXiv
Transcending Scaling Laws with 0.1% Extra Compute2210.11399v2 · Yi Tay, Jason Wei, Hyung Won Chung et al.2022 · 32 citationsarXiv
Scaling Instruction-Finetuned Language Models2210.11416v5 · Hyung Won Chung, Le Hou, Shayne Longpre et al.2022 · 1,440 citationsarXiv
-DARTS: Mitigating Performance Collapse by Harmonizing Operation Selection among Cells2210.07998v2 · Sajad Movahedi, Melika Adabinejad, Ayyoob Imani et al.2022 · 5 citationsarXiv
Intersection of Parallels as an Early Stopping Criterion2208.09529v1 · Ali Vardasbi, Maarten de Rijke, Mostafa Dehghani2022 · 4 citationsarXiv
Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling?2207.10551v1 · Yi Tay, Mostafa Dehghani, Samira Abnar et al.2022 · 43 citationsarXiv
Confident Adaptive Language Modeling2207.07061v2 · Tal Schuster, Adam Fisch, Jai Gupta et al.2022 · 91 citationsarXiv
Beyond Transfer Learning: Co-finetuning for Action Localisation2207.03807v1 · Anurag Arnab, Xuehan Xiong, Alexey Gritsenko et al.2022 · 4 citationsarXiv
Simple Open-Vocabulary Object Detection with Vision Transformers2205.06230v2 · Matthias Minderer, Alexey Gritsenko, Austin Stone et al.2022 · 96 citationsarXiv
UL2: Unifying Language Learning Paradigms2205.05131v3 · Yi Tay, Mostafa Dehghani, Vinh Q. Tran et al.2022 · 89 citationsarXiv
Retrieval-Enhanced Machine Learning2205.01230v1 · Hamed Zamani, Fernando Diaz, Mostafa Dehghani et al.2022 · 53 citationsarXiv
Transformer Memory as a Differentiable Search Index2202.06991v3 · Yi Tay, Vinh Q. Tran, Mostafa Dehghani et al.2022 · 128 citationsarXiv
VUT: Versatile UI Transformer for Multi-Modal Multi-Task User Interface Modeling2112.05692v1 · Yang Li, Gang Li, Xin Zhou et al.2021 · 10 citationsarXiv
PolyViT: Co-training Vision Transformers on Images, Videos and Audio2111.12993v1 · Valerii Likhosherstov, Anurag Arnab, Krzysztof Choromanski et al.2021 · 32 citationsarXiv
Discrete Representations Strengthen Vision Transformer Robustness2111.10493v2 · Chengzhi Mao, Lu Jiang, Mostafa Dehghani et al.2021 · 16 citationsarXiv
The Efficiency Misnomer2110.12894v2 · Mostafa Dehghani, Anurag Arnab, Lucas Beyer et al.2021 · 41 citationsarXiv
SCENIC: A JAX Library for Computer Vision Research and Beyond2110.11403v1 · Mostafa Dehghani, Alexey Gritsenko, Anurag Arnab et al.2021 · 33 citationsarXiv
Exploring the Limits of Large Scale Pre-training2110.02095v1 · Samira Abnar, Mostafa Dehghani, Behnam Neyshabur et al.2021 · 39 citationsarXiv
Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers2109.10686v2 · Yi Tay, Mostafa Dehghani, Jinfeng Rao et al.2021 · 63 citationsarXiv
The Benchmark Lottery2107.07002v1 · Mostafa Dehghani, Yi Tay, Alexey A. Gritsenko et al.2021 · 48 citationsarXiv
TokenLearner: What Can 8 Learned Tokens Do for Images and Videos?2106.11297v4 · Michael S. Ryoo, AJ Piergiovanni, Anurag Arnab et al.2021 · 61 citationsNeurIPS 2021
Gradual Domain Adaptation in the Wild:When Intermediate Distributions are Absent2106.06080v2 · Samira Abnar, Rianne van den Berg, Golnaz Ghiasi et al.2021 · 5 citationsarXiv
Parameter-efficient Multi-task Fine-tuning for Transformers via Shared Hypernetworks2106.04489v1 · Rabeeh Karimi Mahabadi, Sebastian Ruder, Mostafa Dehghani et al.2021 · 153 citationsarXiv
Are Pre-trained Convolutions Better than Pre-trained Transformers?2105.03322v2 · Yi Tay, Mostafa Dehghani, Jai Gupta et al.2021 · 33 citationsarXiv
ViViT: A Video Vision Transformer2103.15691v2 · Anurag Arnab, Mostafa Dehghani, Georg Heigold et al.2021 · 2,760 citationsarXiv
OmniNet: Omnidirectional Representations from Transformers2103.01075v1 · Yi Tay, Mostafa Dehghani, Vamsi Aribandi et al.2021 · 18 citationsarXiv
Long Range Arena: A Benchmark for Efficient Transformers2011.04006v1 · Yi Tay, Mostafa Dehghani, Samira Abnar et al.2020 · 241 citationsarXiv
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale2010.11929v2 · Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov et al.2020 · 21,702 citationsarXiv
Efficient Transformers: A Survey2009.06732v3 · Yi Tay, Mostafa Dehghani, Dara Bahri et al.2020 · 1,264 citationsarXiv
IDF++: Analyzing and Improving Integer Discrete Flows for Lossless Compression2006.12459v2 · Rianne van den Berg, Alexey A. Gritsenko, Mostafa Dehghani et al.2020 · 40 citationsarXiv
Transferring Inductive Biases through Knowledge Distillation2006.00555v3 · Samira Abnar, Mostafa Dehghani, Willem Zuidema2020 · 30 citationsarXiv
MetNet: A Neural Weather Model for Precipitation Forecasting2003.12140v2 · Casper Kaae S\onderby, Lasse Espeholt, Jonathan Heek et al.2020 · 295 citationsarXivon Valency
HiTR: Hierarchical Topic Model Re-estimation for Measuring Topical Diversity of Documents1810.05436v1 · Hosein Azarbonyad, Mostafa Dehghani, Tom Kenter et al.2018 · 1 citationarXiv
Universal Transformers1807.03819v3 · Mostafa Dehghani, Stephan Gouws, Oriol Vinyals et al.2018 · 440 citationsarXiv
Career total: 113 works. 63 are in this corpus.Showing the 50 most recent.
Profile built from the corpus for this byline.
Author records are still filling in while the Hub is in alpha. If this is your page, you'll be able to claim it soon. Spot a mistake? Tell us.