NN
Neel Nanda
cs.LGcs.AIcs.CLcs.CYstat.MLmath.RT
On Valency
published · living versionsW_cr3nzw48·v1 · currentpublished
Designing Effective Monitor-Based Interventions for Mitigating Reward Hacking During RL
with Aria Wong, Joshua Engels
1 version
Preprints & journals
68 papers in the corpus · 2021–2026Model Forensics: Investigating Whether Concerning Behavior Reflects Misalignment2606.26071v2 · Aditya Singh, Gerson Kroiz, Senthooran Rajamanoharan et al.2026 · 0 citationsarXiv
How Transparent is DiffusionGemma?2606.20560v2 · Joshua Engels, Callum McDougall, Bilal Chughtai et al.2026 · 0 citationsarXiv
Building Better Activation Oracles2606.02609v2 · Jan Bauer, Celeste De Schamphelaere, Adam Karvonen et al.2026 · 0 citationsarXiv
How Well Do Models Follow Their Constitutions?2605.24229v1 · Arya Jakkli, Senthooran Rajamanoharan, Neel Nanda2026 · 0 citationsarXiv
Simple LLM Baselines are Competitive for Model Diffing2602.10371v1 · Elias Kempf, Simon Schrodi, Bartosz Cywi'nski et al.2026 · 0 citationsarXiv
Emergent Misalignment is Easy, Narrow Misalignment is Hard2602.07852v1 · Anna Soligo, Edward Turner, Senthooran Rajamanoharan et al.2026 · 0 citationsarXiv
What's the plan? Metrics for implicit planning in LLMs and their application to rhyme generation and question answering2601.20164v2 · Jim Maar, Denis Paperno, Callum Stuart McDougall et al.2026 · 0 citationsarXiv
Building Production-Ready Probes For Gemini2601.11516v4 · J'anos Kram'ar, Joshua Engels, Zheng Wang et al.2026 · 0 citationsarXiv
Interpretable Embeddings with Sparse Autoencoders: A Data Analysis Toolkit2512.10092v2 · Nick Jiang, Xiaoqing Sun, Lisa Dunlap et al.2025 · 1 citationarXiv
Too Late to Recall: Explaining the Two-Hop Problem in Multimodal Knowledge Retrieval2512.03276v1 · Constantin Venhoff, Ashkan Khakzar, Sonia Joseph et al.2025 · 0 citationsarXiv
Difficulties with Evaluating a Deception Detector for AIs2511.22662v2 · Lewis Smith, Bilal Chughtai, Neel Nanda2025 · 0 citationsarXiv
Thought Branches: Interpreting LLM Reasoning Requires Resampling2510.27484v2 · Uzay Macar, Paul C. Bogdan, Senthooran Rajamanoharan et al.2025 · 0 citationsarXiv
Narrow Finetuning Leaves Clearly Readable Traces in Activation Differences2510.13900v3 · Julian Minder, Cl'ement Dumas, Stewart Slocum et al.2025 · 0 citationsarXiv
Base Models Know How to Reason, Thinking Models Learn When2510.07364v4 · Constantin Venhoff, Iv'an Arcuschin, Philip Torr et al.2025 · 1 citationarXiv
Eliciting Secret Knowledge from Language Models2510.01070v2 · Bartosz Cywi'nski, Emil Ryd, Rowan Wang et al.2025 · 0 citationsarXiv
RelP: Faithful and Efficient Circuit Discovery in Language Models via Relevance Patching2508.21258v2 · Farnoush Rezaei Jafari, Oliver Eberle, Ashkan Khakzar et al.2025 · 0 citationsarXiv
Steering Out-of-Distribution Generalization with Concept Ablation Fine-Tuning2507.16795v2 · Helena Casademunt, Caden Juang, Adam Karvonen et al.2025 · 0 citationsarXiv
Reasoning-Finetuning Repurposes Latent Representations in Base Models2507.12638v1 · Jake Ward, Chuqiao Lin, Constantin Venhoff et al.2025 · 0 citationsarXiv
Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety2507.11473v2 · Tomek Korbak, Mikita Balesni, Elizabeth Barnes et al.2025 · 3 citationsarXiv
Simple Mechanistic Explanations for Out-Of-Context Reasoning2507.08218v2 · Atticus Wang, Joshua Engels, Oliver Clive-Griffin et al.2025 · 0 citationsarXiv
Thought Anchors: Which LLM Reasoning Steps Matter?2506.19143v4 · Paul C. Bogdan, Uzay Macar, Neel Nanda et al.2025 · 2 citationsarXiv
Understanding Reasoning in Thinking Language Models via Steering Vectors2506.18167v4 · Constantin Venhoff, Iv'an Arcuschin, Philip Torr et al.2025 · 1 citationarXiv
Model Organisms for Emergent Misalignment2506.11613v1 · Edward Turner, Anna Soligo, Mia Taylor et al.2025 · 0 citationsarXiv
Convergent Linear Representations of Emergent Misalignment2506.11618v2 · Anna Soligo, Edward Turner, Senthooran Rajamanoharan et al.2025 · 0 citationsarXiv
Because we have LLMs, we Can and Should Pursue Agentic Interpretability2506.12152v1 · Been Kim, John Hewitt, Neel Nanda et al.2025 · 1 citationarXiv
Inference-Time Decomposition of Activations (ITDA): A Scalable Approach to Interpreting Large Language Models2505.17769v2 · Patrick Leask, Neel Nanda, Noura Al Moubayed2025 · 0 citationsarXiv
Towards eliciting latent knowledge from LLMs with mechanistic interpretability2505.14352v1 · Bartosz Cywi'nski, Emil Ryd, Senthooran Rajamanoharan et al.2025 · 0 citationsarXiv
Scaling sparse feature circuit finding for in-context learning2504.13756v1 · Dmitrii Kharlapenko, Stepan Shabalin, Fazl Barez et al.2025 · 0 citationsarXiv
Overcoming Sparsity Artifacts in Crosscoders to Interpret Chat-Tuning2504.02922v4 · Julian Minder, Cl'ement Dumas, Caden Juang et al.2025 · 0 citationsarXiv
Learning Multi-Level Features with Matryoshka Sparse Autoencoders2503.17547v1 · Bart Bussmann, Noa Nabeshima, Adam Karvonen et al.2025 · 0 citationsarXiv
SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability2503.09532v4 · Adam Karvonen, Can Rager, Johnny Lin et al.2025 · 0 citationsarXiv
Chain-of-Thought Reasoning In The Wild Is Not Always Faithful2503.08679v6 · Iv'an Arcuschin, Jett Janiak, Robert Krzyzanowski et al.2025 · 6 citationsarXiv
Are Sparse Autoencoders Useful? A Case Study in Sparse Probing2502.16681v1 · Subhash Kantamneni, Joshua Engels, Senthooran Rajamanoharan et al.2025 · 0 citationsarXiv
Sparse Autoencoders Do Not Find Canonical Units of Analysis2502.04878v1 · Patrick Leask, Bart Bussmann, Michael Pearce et al.2025 · 0 citationsarXiv
Open Problems in Mechanistic Interpretability2501.16496v1 · Lee Sharkey, Bilal Chughtai, Joshua Batson et al.2025 · 11 citationsarXiv
BatchTopK Sparse Autoencoders2412.06410v1 · Bart Bussmann, Patrick Leask, Neel Nanda2024 · 6 citationsarXiv
Evaluating Sparse Autoencoders on Targeted Concept Erasure Tasks2411.18895v1 · Adam Karvonen, Can Rager, Samuel Marks et al.2024 · 0 citationsarXiv
Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Models2411.14257v2 · Javier Ferrando, Oscar Obeso, Senthooran Rajamanoharan et al.2024 · 1 citationarXiv
Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 22408.05147v2 · Tom Lieberum, Senthooran Rajamanoharan, Arthur Conmy et al.2024 · 38 citationsarXiv
Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoders2407.14435v3 · Senthooran Rajamanoharan, Tom Lieberum, Nicolas Sonnerat et al.2024 · 4 citationsarXiv
Interpreting Attention Layer Outputs with Sparse Autoencoders2406.17759v1 · Connor Kissane, Robert Krzyzanowski, Joseph Isaac Bloom et al.2024 · 3 citationsarXiv
Confidence Regulation Neurons in Language Models2406.16254v2 · Alessandro Stolfo, Ben Wu, Wes Gurnee et al.2024 · 2 citationsarXiv
Refusal in Language Models Is Mediated by a Single Direction2406.11717v3 · Andy Arditi, Oscar Obeso, Aaquib Syed et al.2024 · 74 citationsarXiv
Transcoders Find Interpretable LLM Feature Circuits2406.11944v2 · Jacob Dunefsky, Philippe Chlenski, Neel Nanda2024 · 16 citationsarXiv
Towards Principled Evaluations of Sparse Autoencoders for Interpretability and Control2405.08366v3 · Aleksandar Makelov, George Lange, Neel Nanda2024 · 4 citationsarXiv
Improving Dictionary Learning with Gated Sparse Autoencoders2404.16014v2 · Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith et al.2024 · 2 citationsarXiv
How to use and interpret activation patching2404.15255v1 · Stefan Heimersheim, Neel Nanda2024 · 6 citationsarXiv
AtP*: An efficient and scalable method for localizing LLM behaviour to components2403.00745v1 · J'anos Kram'ar, Tom Lieberum, Rohin Shah et al.2024 · 0 citationsarXiv
Explorations of Self-Repair in Language Models2402.15390v2 · Cody Rushing, Neel Nanda2024 · 0 citationsProceedings of the 41st International Conference on Machine Learning, PMLR 235:42836-42855, 2024
Summing Up the Facts: Additive Mechanisms Behind Factual Recall in LLMs2402.07321v1 · Bilal Chughtai, Alan Cooney, Neel Nanda2024 · 2 citationsarXiv
Career total: 75 works. 68 are in this corpus.Showing the 50 most recent.
Profile built from the corpus for this byline.
Author records are still filling in while the Hub is in alpha. If this is your page, you'll be able to claim it soon. Spot a mistake? Tell us.