Report Overview
Welcome to the seventh edition of the AI Index report. The 2024 Index is our most comprehensive to date and arrives at an important moment when AI's influence on society has never been more pronounced. This year, we have broadened our scope to more extensively cover essential trends such as technical advancements in AI, public perceptions of the technology, and the geopolitical dynamics surrounding its development.
Key Insight: The AI Index report tracks, collates, distills, and visualizes data related to artificial intelligence. Our mission is to provide unbiased, rigorously vetted, broadly sourced data in order for policymakers, researchers, executives, journalists, and the general public to develop a more thorough and nuanced understanding of the complex field of AI.
Key Data Points
Top 10 Takeaways
AI beats humans on some tasks, but not on all
AI has surpassed human performance on several benchmarks, including some in image classification, visual reasoning, and English understanding. Yet it trails behind on more complex tasks like competition-level mathematics, visual commonsense reasoning and planning.
Industry dominates frontier AI research
In 2023, industry produced 51 notable machine learning models, while academia contributed only 15. There were also 21 notable models resulting from industry-academia collaborations in 2023, a new high.
Frontier models get way more expensive
According to AI Index estimates, the training costs of state-of-the-art AI models have reached unprecedented levels. For example, OpenAI's GPT-4 used an estimated $78 million worth of compute to train, while Google's Gemini Ultra cost $191 million for compute.
The United States leads in top AI models
In 2023, 61 notable AI models originated from U.S.-based institutions, far outpacing the European Union's 21 and China's 15.
Standardized evaluations for LLM responsibility are lacking
New research from the AI Index reveals a significant lack of standardization in responsible AI reporting. Leading developers test their models against different responsible AI benchmarks, complicating systematic comparison.
Generative AI investment skyrockets
Despite a decline in overall AI private investment last year, funding for generative AI surged, nearly octupling from 2022 to reach $25.2 billion.
Content Overview
Report Chapters
Chapter 1: Research and Development
This chapter studies trends in AI research and development. It begins by examining trends in AI publications and patents, and then examines trends in notable AI systems and foundation models. It concludes by analyzing AI conference attendance and open-source AI software projects.
Key Finding: Industry continues to dominate frontier AI research. In 2023, industry produced 51 notable machine learning models, while academia contributed only 15. There were also 21 notable models resulting from industry-academia collaborations in 2023, a new high.
Chapter 2: Technical Performance
The technical performance section of this year's AI Index offers a comprehensive overview of AI advancements in 2023. It starts with a high-level overview of AI technical performance, tracing its broad evolution over time.
The chapter examines the current state of a wide range of AI capabilities, including language processing, coding, computer vision (image and video analysis), reasoning, audio processing, autonomous agents, robotics, and reinforcement learning.
Key Finding: AI beats humans on some tasks, but not on all. AI has surpassed human performance on several benchmarks, including some in image classification, visual reasoning, and English understanding. Yet it trails behind on more complex tasks like competition-level mathematics, visual commonsense reasoning and planning.
Chapter 3: Responsible AI
This chapter explores the responsible AI landscape, examining issues such as AI incidents, political deepfakes, model vulnerabilities, and concerns about AI's impact on elections and copyright.
The research reveals a significant lack of standardization in responsible AI reporting, with leading developers testing their models against different responsible AI benchmarks.
Key Finding: Political deepfakes are easy to generate and difficult to detect. Political deepfakes are already affecting elections across the world, with recent research suggesting that existing AI deepfake methods perform with varying levels of accuracy.
Chapter 4: Economy
This chapter examines AI's impact on the economy, including investment trends, job markets, and business adoption of AI technologies.
Despite a decline in overall AI private investment, funding for generative AI surged dramatically. The data also shows that AI makes workers more productive and leads to higher quality work.
Key Finding: Generative AI investment skyrockets. Despite a decline in overall AI private investment last year, funding for generative AI surged, nearly octupling from 2022 to reach $25.2 billion.
Chapter 5: Science and Medicine
This chapter explores how AI is accelerating scientific discovery and advancing medical applications.
In 2023, significant science-related AI applications were launched, from AlphaDev which makes algorithmic sorting more efficient, to GNoME which facilitates the process of materials discovery.
Key Finding: Scientific progress accelerates even further, thanks to AI. In 2022, AI began to advance scientific discovery. 2023, however, saw the launch of even more significant science-related AI applications.
Chapter 6: Education
This chapter examines trends in AI education, including CS graduate numbers, the migration of AI PhDs to industry, and the internationalization of AI education.
The data shows that the migration of AI PhDs to industry continues at an accelerating pace, with 70.7% joining industry after graduation compared to 20.0% entering academia.
Key Finding: The migration of AI PhDs to industry continues at an accelerating pace. In 2011, roughly equal percentages of new AI PhDs took jobs in industry (40.9%) and academia (41.6%). However, by 2022, a significantly larger proportion (70.7%) joined industry after graduation.
Chapter 7: Policy and Governance
This chapter analyzes AI policy and governance developments around the world, including regulatory trends and legislative activity.
The number of AI regulations in the United States has risen significantly, with 25 AI-related regulations in 2023, up from just one in 2016.
Key Finding: The number of AI regulations in the United States sharply increases. The number of AI-related regulations has risen significantly in the past year and over the last five years. In 2023, there were 25 AI-related regulations, up from just one in 2016.
Chapter 8: Diversity
This chapter examines diversity trends in AI education and the workforce, including ethnic and gender representation.
While U.S. and Canadian CS students continue to grow more ethnically diverse, substantial gender gaps persist in European informatics, CS, CE, and IT graduates at all educational levels.
Key Finding: U.S. and Canadian bachelor's, master's, and PhD CS students continue to grow more ethnically diverse. While white students continue to be the most represented ethnicity, the representation from other ethnic groups continues to grow.
Chapter 9: Public Opinion
This chapter explores global public perceptions of AI, including awareness, concerns, and expectations about AI's impact.
People across the globe are more cognizant of AI's potential impact—and more nervous. A survey shows that 66% think AI will dramatically affect their lives in the next three to five years, up from 60% the previous year.
Key Finding: People across the globe are more cognizant of AI's potential impact—and more nervous. A survey from Ipsos shows that, over the last year, the proportion of those who think AI will dramatically affect their lives in the next three to five years has increased from 60% to 66%.
Note: The above is only a summary of the report content. The complete document contains extensive data, charts, and detailed analysis. We recommend downloading the full PDF for in-depth reading.