JS
Jacob Steinhardt
cs.LGcs.AIcs.CLstat.MLcs.CVcs.CRcs.CYmath.STstat.THcs.GT
On Valency
published · living versionsW_y3274kcu·v1 · currentpublished
The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation
with Miles Brundage, Shahar Avin, Jack Clark, Helen Toner +21
1 version
Preprints & journals
95 papers in the corpus · 2007–2026Understanding In-context Learning of Addition via Activation Subspaces2505.05145v4 · Xinyan Hu, Kayo Yin, Michael I. Jordan et al.2025 · 0 citationsarXiv
LLM Layers Immediately Correct Each Other2609.07876v1 · Arjun Patrawala, Jiahai Feng, Erik Jones et al.2026 · 0 citationsarXiv
Norm Enforcement for AI Agents: Robustly Shaping Behavior in Multi-Agent Systems2607.09766v1 · Yaowen Ye, Jacob Steinhardt2026 · 0 citationsarXiv
Language Model Circuits Are Sparse in the Neuron Basis2601.22594v2 · Aryaman Arora, Zhengxuan Wu, Jacob Steinhardt et al.2026 · 0 citationsarXiv
Log analysis is necessary for credible evaluation of AI agents2605.08545v1 · Peter Kirgis, Sayash Kapoor, Stephan Rabanser et al.2026 · 0 citationsarXiv
ADAG: Automatically Describing Attribution Graphs2604.07615v1 · Aryaman Arora, Zhengxuan Wu, Jacob Steinhardt et al.2026 · 0 citationsarXiv
LatentQA: Teaching LLMs to Decode Activations Into Natural Language2412.08686v2 · Alexander Pan, Lijie Chen, Jacob Steinhardt2024 · 0 citationsarXiv
Training Language Models to Explain Their Own Computations2511.08579v3 · Belinda Z. Li, Zifan Carl Guo, Vincent Huang et al.2025 · 0 citationsarXiv
Learning a Generative Meta-Model of LLM Activations2602.06964v1 · Grace Luo, Jiahai Feng, Trevor Darrell et al.2026 · 0 citationsarXiv
Eliciting Latent Predictions from Transformers with the Tuned Lens2303.08112v6 · Nora Belrose, Igor Ostrovsky, Lev McKinney et al.2023 · 27 citationsarXiv
Safety versus performance: How multi-objective learning reduces barriers to market entry.41091760 · Jagadeesan, Meena, Jordan, Michael I, Steinhardt, Jacob2025 · 1 citationProceedings of the National Academy of Sciences of the United States of America. 2025;122(42):e2510004122
Extractive Structures Learned in Pretraining Enable Generalization on Finetuned Facts2412.04614v3 · Jiahai Feng, Stuart Russell, Jacob Steinhardt2024 · 1 citationarXiv
Which Attention Heads Matter for In-Context Learning?2502.14010v1 · Kayo Yin, Jacob Steinhardt2025 · 0 citationsICML 2025
VibeCheck: Discover and Quantify Qualitative Differences in Large Language Models2410.12851v7 · Lisa Dunlap, Krishna Mandal, Trevor Darrell et al.2024 · 0 citationsarXiv
Uncovering Gaps in How Humans and LLMs Interpret Subjective Language2503.04113v1 · Erik Jones, Arjun Patrawala, Jacob Steinhardt2025 · 0 citationsarXiv
Interpreting the Second-Order Effects of Neurons in CLIP2406.04341v3 · Yossi Gandelsman, Alexei A. Efros, Jacob Steinhardt2024 · 0 citationsarXiv
Eliciting Language Model Behaviors with Investigator Agents2502.01236v1 · Xiang Lisa Li, Neil Chowdhury, Daniel D. Johnson et al.2025 · 0 citationsarXiv
Iterative Label Refinement Matters More than Preference Optimization under Weak Supervision2501.07886v1 · Yaowen Ye, Cassidy Laidlaw, Jacob Steinhardt2025 · 0 citationsarXiv
Explaining Datasets in Words: Statistical Models with Natural Language Parameters2409.08466v2 · Ruiqi Zhong, Heng Wang, Dan Klein et al.2024 · 0 citationsarXiv
Monitoring Latent World States in Language Models with Propositional Probes2406.19501v2 · Jiahai Feng, Stuart Russell, Jacob Steinhardt2024 · 0 citationsarXiv
Language Models Learn to Mislead Humans via RLHF2409.12822v3 · Jiaxin Wen, Ruiqi Zhong, Akbir Khan et al.2024 · 4 citationsarXiv
The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation1802.07228v2 · Miles Brundage, Shahar Avin, Jack Clark et al.2018 · 495 citationsarXivon Valency
What Do Learning Dynamics Reveal About Generalization in LLM Reasoning?2411.07681v2 · Katie Kang, Amrith Setlur, Dibya Ghosh et al.2024 · 0 citationsarXiv
Safety vs. Performance: How Multi-Objective Learning Reduces Barriers to Market Entry2409.03734v1 · Meena Jagadeesan, Michael I. Jordan, Jacob Steinhardt2024 · 0 citationsarXiv
Adversaries Can Misuse Combinations of Safe Models2406.14595v2 · Erik Jones, Anca Dragan, Jacob Steinhardt2024 · 1 citationarXiv
Covert Malicious Finetuning: Challenges in Safeguarding LLM Adaptation2406.20053v1 · Danny Halawi, Alexander Wei, Eric Wallace et al.2024 · 1 citationarXiv
Incentivizing High-Quality Content in Online Recommender Systems2306.07479v3 · Xinyan Hu, Meena Jagadeesan, Michael I. Jordan et al.2023 · 1 citationarXiv
Feedback Loops With Language Models Drive In-Context Reward Hacking2402.06627v3 · Alexander Pan, Erik Jones, Meena Jagadeesan et al.2024 · 2 citationsarXiv
How do Language Models Bind Entities in Context?2310.17191v2 · Jiahai Feng, Jacob Steinhardt2023 · 1 citationarXiv
Describing Differences in Image Sets with Natural Language2312.02974v2 · Lisa Dunlap, Yuhui Zhang, Xiaohan Wang et al.2023 · 16 citationsarXiv
Interpreting CLIP's Image Representation via Text-Based Decomposition2310.05916v4 · Yossi Gandelsman, Alexei A. Efros, Jacob Steinhardt2023 · 4 citationsarXiv
Overthinking the Truth: Understanding how Language Models Process False Demonstrations2307.09476v3 · Danny Halawi, Jean-Stanislas Denain, Jacob Steinhardt2023 · 6 citationsarXiv
Protein language models are biased by unequal sequence sampling across the tree of life10.1101/2024.03.07.584001v1 · Ding, F., Steinhardt, J. N.2024 · 65 citationsbioRxiv
Discovering Latent Knowledge in Language Models Without Supervision2212.03827v2 · Collin Burns, Haotian Ye, Dan Klein et al.2022 · 46 citationsarXiv
Approaching Human-Level Forecasting with Language Models2402.18563v1 · Danny Halawi, Fred Zhang, Chen Yueh-Han et al.2024 · 5 citationsarXiv
Improved Bayes Risk Can Yield Reduced Social Welfare Under Competition2306.14670v3 · Meena Jagadeesan, Michael I. Jordan, Jacob Steinhardt et al.2023 · 1 citationarXiv
Testing Robustness Against Unforeseen Adversaries1908.08016v4 · Max Kaufmann, Daniel Kang, Yi Sun et al.2019 · 75 citationsarXiv
Progress measures for grokking via mechanistic interpretability2301.05217v3 · Neel Nanda, Lawrence Chan, Tom Lieberum et al.2023 · 52 citationsarXiv
Do Models Explain Themselves? Counterfactual Simulatability of Natural Language Explanations2307.08678v1 · Yanda Chen, Ruiqi Zhong, Narutatsu Ri et al.2023 · 5 citationsarXiv
Are Neurons Actually Collapsed? On the Fine-Grained Structure in Neural Representations2306.17105v1 · Yongyi Yang, Jacob Steinhardt, Wei Hu2023 · 2 citationsarXiv
Auditing Visualizations: Transparency Methods Struggle to Detect Anomalous Behavior2206.13498v2 · Jean-Stanislas Denain, Jacob Steinhardt2022 · 2 citationsarXiv
Automatically Auditing Large Language Models via Discrete Optimization2303.04381v1 · Erik Jones, Anca Dragan, Aditi Raghunathan et al.2023 · 16 citationsarXiv
Reward Learning as Doubly Nonparametric Bandits: Optimal Design and Scaling Laws2302.12349v1 · Kush Bhatia, Wenshuo Guo, Jacob Steinhardt2023 · 0 citationsarXiv
Aligning AI With Shared Human Values2008.02275v6 · Dan Hendrycks, Collin Burns, Steven Basart et al.2020 · 100 citationsarXiv
Learning Equilibria in Matching Markets from Bandit Feedback2108.08843v2 · Meena Jagadeesan, Alexander Wei, Yixin Wang et al.2021 · 5 citationsarXiv
Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small2211.00593v1 · Kevin Wang, Alexandre Variengien, Arthur Conmy et al.2022 · 53 citationsarXiv
Unsolved Problems in ML Safety2109.13916v5 · Dan Hendrycks, Nicholas Carlini, John Schulman et al.2021 · 85 citationsarXiv
Describing Differences between Text Distributions with Natural Language2201.12323v2 · Ruiqi Zhong, Charlie Snell, Dan Klein et al.2022 · 8 citationsarXiv
Scaling Out-of-Distribution Detection for Real-World Settings1911.11132v4 · Dan Hendrycks, Steven Basart, Mantas Mazeika et al.2019 · 176 citationsarXiv
PixMix: Dreamlike Pictures Comprehensively Improve Safety Measures2112.05135v3 · Dan Hendrycks, Andy Zou, Mantas Mazeika et al.2021 · 103 citationsarXiv
Career total: 148 works. 95 are in this corpus.Showing the 50 most recent.
Profile built from the corpus for this byline.
Author records are still filling in while the Hub is in alpha. If this is your page, you'll be able to claim it soon. Spot a mistake? Tell us.