AC
Arman Cohan
cs.CLcs.AIcs.LGcs.IRcs.SEcs.CVcs.CYcs.DLcs.PLArtificial intelligence
On Valency
published · living versionsW_sz2b2m5q·v2 · currentpublished
How Good Are Frontier Models at Physics? Expert Re-Grading Reveals Broken Evaluations and Near-Saturation of Leading Benchmarks
with Ali Ansari, Haoran Sun, Andy Zeyi Liu, Mark Jabbour +46
2 versions
Preprints & journals
156 papers in the corpus · 2016–2026StepSafe: Learning Timely Intervention with Context-Aware Safety Reasoning2610.20960v1 · Siyi Gu, Kunhan Gao, Chao Peter Yang et al.2026 · 0 citationsarXiv
IdeaAnchor: Teaching LLMs to Turn Literature into Research Ideas2610.08781v1 · Ziyu Chen, Yilun Zhao, Jiashuo Sun et al.2026 · 0 citationsarXiv
Balancing Memory Pathways: Analyzing and Improving Memory Utilization in Hybrid LMs2610.06750v1 · Hyunji Lee, Joykirat Singh, Zaid Khan et al.2026 · 0 citationsarXiv
BackTrend: Evaluating Scientific Weak-Signal Prediction via Backward Reconstruction2609.24921v1 · Xiao Zhou, Yilun Zhao, Owen Jiang et al.2026 · 0 citationsarXiv
How Good Are Frontier Models at Physics? Expert Re-Grading Reveals Broken Evaluations and Near-Saturation of Leading Benchmarks2609.13009v2 · Ali Ansari, Haoran Sun, Andy Zeyi Liu et al.2026 · 0 citationsarXivon Valency
IdeaAMBIG: Benchmarking Implementation-Critical Gaps in Research-Idea Specifications2609.10539v1 · Yiling Ma, Yilun Zhao, Sihong Wu et al.2026 · 0 citationsarXiv
ActReview: Rebuttal-Guided Training Data and Rubric Rewards for Actionable Peer Review Generation2609.09076v1 · Yiling Ma, Yilun Zhao, Sihong Wu et al.2026 · 0 citationsarXiv
Small Language Models as Judges for Rubric-Based Reinforcement Learning2608.30005v1 · Fengyu Xie, Yilun Zhao, Bingsen Chen et al.2026 · 0 citationsarXiv
Erratum: MedicalAgentsBench for complex medical reasoning: Comparing internalized reasoning models versus externalized agent-based frameworks.42746258 · Shao, Yanjun, Tang, Xiangru, Sohn, Jiwoong et al.2026 · 2 citationsPatterns (New York, N.Y.). 2026;7(9):101658
Can Scientific Claims Be Removed from Large Language Models? A Systematic Evaluation of Claim-Level Unlearning2608.20960v1 · Snigdha Paul, Manasi Patwardhan, Arman Cohan2026 · 0 citationsarXiv
Metacognition in LLMs: Foundations, Progress, and Opportunities2607.11881v1 · Gabrielle Kaili-May Liu, Areeb Gani, Jacqueline Lu et al.2026 · 0 citationsarXiv
Measuring the Gap Between Human and LLM Research Ideas2607.01233v1 · Ziyu Chen, Yilun Zhao, Arman Cohan2026 · 0 citationsarXiv
Reinforcement Learning with Metacognitive Feedback Elicits Faithful Uncertainty Expression in LLMs2606.32032v2 · Gabrielle Kaili-May Liu, Avi Caciularu, Gal Yona et al.2026 · 0 citationsarXiv
Every Eval Ever: A Unifying Schema and Community Repository for AI Evaluation Results2606.14516v1 · Jan Batzner, Sree Harsha Nelaturu, Damian Stachura et al.2026 · 0 citationsarXiv
Benchmarking AI Agents for Addressing Scientific Challenges Across Scales2606.12736v1 · Tianyu Liu, Allen Xin Wang, Antonia Panescu et al.2026 · 1 citationarXiv
Quantifying Faithful Confidence Expression in Large Reasoning Models2606.03969v1 · Areeb Gani, Asal Meskin, Gabrielle Kaili-May Liu et al.2026 · 0 citationsarXiv
Can LLMs Use Linguistic Uncertainty Markers to Reliably Reflect Intrinsic Confidence?2605.28778v1 · Gabrielle Kaili-May Liu, Arman Cohan2026 · 0 citationsarXiv
OpenComputer: Verifiable Software Worlds for Computer-Use Agents2605.19769v1 · Jinbiao Wei, Qianran Ma, Yilun Zhao et al.2026 · 0 citationsarXiv
Time to REFLECT: Can We Trust LLM Judges for Evidence-based Research Agents?2605.19196v1 · Leyao Wang, Yanan He, Peng Chen et al.2026 · 0 citationsarXiv
BPI26-028: Hippocrates-Karkinos-o1: An Enhanced Model for Oncology Decision Support and Guideline-Aware Reasoning With Self-Refinement and Guideline-Informed Retrieval.42061427 · Aguirre, Nicole, Wang, Benlu, Schaefer, Elizabeth et al.2026 · 0 citationsJournal of the National Comprehensive Cancer Network : JNCCN. 2026;24(3.5)
Examining Reasoning LLMs-as-Judges in Non-Verifiable LLM Post-Training2603.12246v1 · Yixin Liu, Yue Yu, DiJia Su et al.2026 · 0 citationsarXiv
MedTutor: A Retrieval-Augmented LLM System for Case-Based Medical Education2601.06979v2 · Dongsuk Jang, Ziyao Shangguan, Kyle Tegtmeyer et al.2026 · 4 citationsProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 319-353
Hippocrates-o1: A Guideline-Aware, Orchestrated, Self-Refining Protocol for Specialty-Specific Clinical Reasoning10.64898/2025.12.04.25341678v1 · Wang, B., Schaefer, E., Aguirre, N. et al.2025 · 0 citationsmedRxiv
Evaluating Legal Reasoning Traces with Legal Issue Tree Rubrics2512.01020v2 · Jinu Lee, Kyoung-Woon On, Simeng Han et al.2025 · 0 citationsarXiv
On Evaluating LLM Alignment by Evaluating LLMs as Judges2511.20604v1 · Yixin Liu, Pengfei Liu, Arman Cohan2025 · 0 citationsarXiv
AlphaResearch: Accelerating New Algorithm Discovery with Language Models2511.08522v2 · Zhaojian Yu, Kaiyue Feng, Yilun Zhao et al.2025 · 0 citationsarXiv
Rethinking Retrieval-Augmented Generation for Medicine: A Large-Scale, Systematic Expert Evaluation and Practical Insights2511.06738v1 · Hyunjae Kim, Jiwoong Sohn, Aidan Gilson et al.2025 · 2 citationsarXiv
Measuring what Matters: Construct Validity in Large Language Model Benchmarks2511.04703v1 · Andrew M. Bean, Ryan Othniel Kearns, Angelika Romanou et al.2025 · 7 citationsarXiv
From Scores to Steps: Diagnosing and Improving LLM Performance in Evidence-Based Medical Calculations.41799784 · Wang, Benlu, Xia, Iris, Zhang, Yifan et al.2026 · 1 citationProceedings of the Conference on Empirical Methods in Natural Language Processing. Conference on Empirical Methods in Natural Language Processing. 2025;2025:10809-10833
LimRank: Less is More for Reasoning-Intensive Information Reranking2510.23544v1 · Tingyu Song, Yilun Zhao, Siyue Zhang et al.2025 · 1 citationarXiv
FinTrust: A Comprehensive Benchmark of Trustworthiness Evaluation in Finance Domain2510.15232v1 · Tiansheng Hu, Tongyan Hu, Liuyang Bai et al.2025 · 2 citationsarXiv
Investigating Retrieval-Augmented Generation Systems on Unanswerable, Uncheatable, Realistic, Multi-hop Queries2510.11956v3 · Gabrielle Kaili-May Liu, Bryan Li, Arman Cohan et al.2025 · 0 citationsarXiv
MRMR: A Realistic and Expert-Level Multidisciplinary Benchmark for Reasoning-Intensive Multimodal Retrieval2510.09510v2 · Siyue Zhang, Yuan Gao, Xiao Zhou et al.2025 · 0 citationsarXiv
FinLFQA: Evaluating Attributed Text Generation of LLMs in Financial Long-Form Question Answering2510.06426v1 · Yitao Long, Tiansheng Hu, Yilun Zhao et al.2025 · 0 citationsarXiv
PuzzlePlex: Benchmarking Foundation Models on Reasoning and Planning with Puzzles2510.06475v1 · Yitao Long, Yuru Jiang, Hongjun Liu et al.2025 · 0 citationsarXiv
Risks of AI scientists: prioritizing safeguarding over autonomy.40968279 · Tang, Xiangru, Jin, Qiao, Zhu, Kunlun et al.2025 · 53 citationsNature communications. 2025;16(1):8317
MSRS: Evaluating Multi-Source Retrieval-Augmented Generation2508.20867v1 · Rohan Phanse, Yijie Zhou, Kejian Shi et al.2025 · 0 citationsarXiv
AI Scientists for Building Virtual-Cell Models2508.02276v3 · Xiangru Tang, Jiapeng Chen, Daniel Shao et al.2025 · 4 citationsarXiv
AbGen: Evaluating Large Language Models in Ablation Study Design and Evaluation for Scientific Research2507.13300v1 · Yilun Zhao, Weiyuan Chen, Zhijian Xu et al.2025 · 4 citationsarXiv
Can Multimodal Foundation Models Understand Schematic Diagrams? An Empirical Study on Information-Seeking QA over Scientific Papers2507.10787v1 · Yilun Zhao, Chengye Wang, Chuhan Li et al.2025 · 3 citationsarXiv
Ref-Long: Benchmarking the Long-context Referencing Capability of Long-context Language Models2507.09506v2 · Junjie Wu, Gefei Gu, Yanan Zheng et al.2025 · 0 citationsarXiv
Can LLMs Identify Critical Limitations within Scientific Research? A Systematic Evaluation on AI Research Papers2507.02694v1 · Zhijian Xu, Yilun Zhao, Manasi Patwardhan et al.2025 · 4 citationsarXiv
SciVer: Evaluating Foundation Models for Multimodal Scientific Claim Verification2506.15569v1 · Chengye Wang, Yifei Shen, Zexi Kuang et al.2025 · 4 citationsarXiv
SUCEA: Reasoning-Intensive Retrieval for Adversarial Fact-checking through Claim Decomposition and Editing2506.04583v1 · Hongjun Liu, Yilun Zhao, Arman Cohan et al.2025 · 1 citationarXiv
MetaFaith: Faithful Natural Language Uncertainty Expression in LLMs2505.24858v2 · Gabrielle Kaili-May Liu, Gal Yona, Avi Caciularu et al.2025 · 1 citationarXiv
MIR: Methodology Inspiration Retrieval for Scientific Research Problems2506.00249v1 · Aniketh Garikaparthi, Manasi Patwardhan, Aditya Sanjiv Kanade et al.2025 · 0 citationsarXiv
Judging with Many Minds: Do More Perspectives Mean Less Prejudice? On Bias Amplifications and Resistance in Multi-Agent Based LLM-as-Judge2505.19477v3 · Chiyu Ma, Enpei Zhang, Yilun Zhao et al.2025 · 0 citationsarXiv
Diffusion vs. Autoregressive Language Models: A Text Embedding Perspective2505.15045v1 · Siyue Zhang, Yilun Zhao, Liyuan Geng et al.2025 · 2 citationsarXiv
Advancing AI Research Assistants with Expert-Involved Learning2505.04638v5 · Tianyu Liu, Simeng Han, Hanchen Wang et al.2025 · 1 citationarXiv
IRIS: Interactive Research Ideation System for Accelerating Scientific Discovery2504.16728v2 · Aniketh Garikaparthi, Manasi Patwardhan, Lovekesh Vig et al.2025 · 1 citationarXiv
Career total: 217 works. 156 are in this corpus.Showing the 50 most recent.
Profile built from the corpus for this byline.
Author records are still filling in while the Hub is in alpha. If this is your page, you'll be able to claim it soon. Spot a mistake? Tell us.