UBC Computer Science
From generating a project idea and running experiments, to writing a scientific paper and evaluating their own work, an AI scientist can now carry out a research experiment from start to finish — all without human assistance.
In a new published in the magazine Natureresearchers from UBC Computer Science, Sakana AI, Vector Institute and University of Oxford have found a way to automate the entire investigation processpotentially accelerating scientific discovery.
“Although AI has been used to help them in specific taskssuch as predicting the structure of proteins or analyzing medical images, this is the first time that it has been demonstrated that it can go through the entire scientific research process on its own,” he says Jeff Cluneprofessor of Computer Science and lead author of the article, at UBC.
“It’s amazing to see what you’ve been able to do so far.but even more extraordinary to consider what is coming in the very near future”, he adds.
The new AI scientist succeeds generate new ideasconsult the literature to check whether they are truly newwrite code and fix your own mistakes in experiments, analyze data and create graphs, write the manuscript and perform your own review.
Researchers developed the AI scientist using fundamental models such as ChatGPT, which are trained on large datasets to perform a wide variety of tasks.
To assess the quality of the work, researchers submitted the article scientific entirely generated by AI to a workshop at a major machine learning conference — the International Conference on Learning Representations.
The article passed the peer review process. Well, it’s not that you can exactly say that they were his peers; they were only human…
The researchers also created a automated evaluator to evaluate AI-generated articles and found that it can accurately predict the conference acceptance decisionss, producing review scores similar to those provided by human reviewers.
Using the automated evaluator, researchers demonstrated that they can improve the quality of generated itemss, either by improving the underlying models used by the AI scientist, or simply by providing him with more computational resources.
“One of the most exciting directions this work points in is the possibility that the AI scientist self-improve“, states Shengran HuPhD student in the UBC Computer Science department and co-author of the article.
“The AI Scientist opens doors to recursive self-improvement, in which the AI system not only discovers new scientific knowledge, but uses these discoveries to become better at making more discoveries”, says the researcher.
“It’s about um type of scientific progress qualitatively different from anything we’ve seen so far,” adds Hu.
Although researchers have demonstrated that the AI scientist is capable of completing all steps of scientific discovery, they have documented some limitations: sometimes produces undeveloped ideas or generates inaccurate citations. Here, there are no surprises…
And, at least currently, the AI scientist can only carry out research in computer Sciencebut researchers believe that the technology could carry out independent research in other areas as well.
“With additional research, this system could be used to create entire scientific communities of AI agents“, says Clune. “That’s where we’ll see the next great scientific revolution.”
Will AI scientists say “eureka”?
