A new AI model has just achieved a human level of “general intelligence”

An ARC-AGI test translates into the ability to accurately solve previously unknown or new problems from limited data samples. An AI already does this as well as we do.

On December 20th, OpenAI’s o3 system scored 85% in , well above the previous best AI result of 55% and along with the average human result. He also scored well on a very difficult math test.

An ARC-AGI test examines an AI system’s “sample efficiency” in adapting to something new — basically, how many examples of a new situation the system needs to see to understand how it works, explains .

The ARC-AGI benchmark tests efficient sample fitting using small square grid problems like the one below. AI has to discover the pattern that transforms the left grid into the right grid.
Each question gives three examples to learn from.

The AI ​​system then needs to discover the rules that “generalize” from the three examples to the fourth. They are very similar to IQ tests that sometimes take place in schools.

It’s unclear how OpenAI achieved 85% in the test, but the results suggest that the o3 model is highly adaptable. From just a few examples, you find rules that can be generalized.

It is also known that OpenAI started with a general-purpose version of the o3 model (which differs from most other models in that you can spend more time “thinking” about difficult questions) and then trained him specifically for the ARC-AGI test.

French AI researcher François Cholletwho designed the benchmark, believes that o3 searches through different “chains of thought” that describe the steps to solving the task. Then, I would choose the “best” according to a rule or “heuristic” vaguely defined.

According to The Conversation, only when the o3 is finally released will we have a better idea of ​​whether it’s approximately as adaptable as an ordinary human being.

If so, it could have a huge and revolutionary economic impact, ushering in a new era of accelerated, self-improving intelligence. It will be necessary new benchmarks for AGI itself and serious reflection on how it should be governed.