AI Can Conduct Research, But Still Struggles to Make Scientific Breakthroughs
· 50-word read
In 50 words
An agentic AI system successfully conducted literature reviews, ran hundreds of experiments and produced research papers, but struggled with deeper scientific reasoning. When evaluated by authors of the original research it attempted to replicate, the AI received low scores, suggesting fully autonomous, open-ended AI research remains out of reach.
Why it matters
AI agents are becoming capable research assistants, handling experiments, coding and literature analysis with limited human intervention. But the study suggests today’s systems still struggle with crucial scientific skills: choosing promising hypotheses, abandoning weak approaches and critically evaluating their own conclusions.
What's next
Researchers are developing tougher evaluation methods such as “shadow evaluation,” where AI-generated research is assessed by experts who originally investigated the same problem. Future agentic systems may need stronger self-criticism, hypothesis exploration and decision-making before they can independently produce major discoveries.
Key dates
- 2024: Sakana AI introduces The AI Scientist, designed to automate much of the scientific research process.
- March 2026: Results from an improved AI Scientist are published in Nature.
- July 2026: Researchers post a new study testing autonomous AI research through “shadow evaluation.”
- Results: The agent scores only 2/6 and 1/6 from authors of the original research despite performing well on engineering and experimentation tasks.
- August 2026: Nature reports the findings and limitations.
Ask BangkokScoop AI about this story
Sources
- nature.comView original →
