Top AI Models Struggle in General Intelligence Test

Quick Summary

• A new AI benchmark, ARC-AGI-2, designed to measure progress toward Artificial General Intelligence (AGI), has shown that leading AI models currently score poorly on the test.
• AGI refers to an AI’s ability to perform cognitive tasks comparable to humans across diverse domains.
• ARC Prize foundation launched the test following its previous iteration, ARC-AGI-1, where OpenAI’s o3 model performed well but fell short on the newer version.
• Scores for current models on ARC-AGI-2 remain in single digits out of 100, despite humans solving all questions in under two attempts.
• The benchmark shifts focus from raw performance (as measured in ARC-AGI-1) to adaptability and cost-efficiency for problem-solving tasks.Such as:

– Humans cost $17/task; OpenAI’s model costs $200/task for similar output.
– Current top-scoring models are highly resource-intensive and lack efficiency when compared with humans.

Indian Opinion analysis

The introduction of ARC‑AGI‑2 represents a pivotal move toward evaluating Artificial Intelligence beyond conventional accuracy metrics. By emphasizing cost-effectiveness alongside performance metrics,it addresses pragmatic concerns about sustainability within burgeoning AI technologies-a significant consideration given India’s prioritization of affordability and scalability across technological adoption processes domestically ~