Title page 1

Contents 13

Report Highlights 14

CHAPTER 1: Research and Development 27

Overview 29

Chapter Highlights 30

1.1. Publications 31

Overview 31

Total Number of AI Publications 31

By Type of Publication 32

By Field of Study 33

By Sector 34

AI Journal Publications 36

AI Conference Publications 37

1.2. Patents 38

AI Patents 38

Overview 38

By Filing Status and Region 39

1.3. Frontier AI Research 45

General Machine Learning Models 45

Overview 45

Sector Analysis 46

National Affiliation 47

Parameter Trends 49

Compute Trends 50

Highlight: Will Models Run Out of Data? 52

Foundation Models 56

Model Release 56

Organizational Affiliation 58

National Affiliation 61

Training Cost 63

1.4. AI Conferences 66

Conference Attendance 66

1.5. Open-Source AI Software 69

Projects 69

Stars 71

CHAPTER 2: Technical Performance 73

Overview 76

Chapter Highlights 77

2.1. Overview of AI in 2023 78

Timeline: Significant Model Releases 78

State of AI Performance 81

AI Index Benchmarks 82

2.2. Language 85

Understanding 86

HELM: Holistic Evaluation of Language Models 86

MMLU: Massive Multitask Language Understanding 87

Generation 88

Chatbot Arena Leaderboard 88

Factuality and Truthfulness 90

TruthfulQA 90

HaluEval 92

2.3. Coding 94

Generation 94

HumanEval 94

SWE-bench 95

2.4. Image Computer Vision and Image Generation 96

Generation 96

HEIM: Holistic Evaluation of Text-to-Image Models 97

Highlighted Research: MVDream 98

Instruction-Following 99

VisIT-Bench 99

Editing 100

EditVal 100

Highlighted Research: ControlNet 101

Highlighted Research: Instruct-NeRF2NeRF 103

Segmentation 105

Highlighted Research: Segment Anything 105

3D Reconstruction From Images 107

Highlighted Research: Skoltech3D 107

Highlighted Research: RealFusion 108

2.5. Video Computer Vision and Video Generation 109

Generation 109

UCF101 109

Highlighted Research: Align Your Latents 110

Highlighted Research: Emu Video 111

2.6. Reasoning 112

General Reasoning 112

MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI 112

GPQA: A Graduate-Level Google-Proof Q&A Benchmark 115

Highlighted Research: Comparing Humans, GPT-4, and GPT-4V on Abstraction and Reasoning Tasks 116

Mathematical Reasoning 117

GSM8K 117

MATH 119

PlanBench 120

Visual Reasoning 121

Visual Commonsense Reasoning (VCR) 121

Moral Reasoning 122

MoCa 122

Causal Reasoning 124

BigToM 124

Highlighted Research: Tübingen Cause-Effect Pairs 126

2.7. Audio 127

Generation 127

Highlighted Research: UniAudio 128

Highlighted Research: MusicGEN and MusicLM 129

2.8. Agents 131

General Agents 131

AgentBench 131

Highlighted Research: Voyageur 133

Task-Specific Agents 134

MLAgentBench 134

2.9. Robotics 135

Highlighted Research: PaLM-E 135

Highlighted Research: RT-2 137

2.10. Reinforcement Learning 138

Reinforcement Learning from Human Feedback 138

Highlighted Research: RLAIF 139

Highlighted Research: Direct Preference Optimization 140

2.11. Properties of LLMs 141

Highlighted Research: Challenging the Notion of Emergent Behavior 141

Highlighted Research: Changes in LLM Performance Over Time 143

Highlighted Research: LLMs Are Poor Self-Correctors 145

Closed vs. Open Model Performance 146

2.12. Techniques for LLM Improvement 148

Prompting 148

Highlighted Research: Graph of Thoughts Prompting 148

Highlighted Research: Optimization by PROmpting (OPRO) 150

Fine-Tuning 151

Highlighted Research: QLoRA 151

Attention 152

Highlighted Research: Flash-Decoding 152

2.13. Environmental Impact of AI Systems 154

General Environmental Impact 154

Training 154

Inference 156

Positive Use Cases 157

CHAPTER 3: Responsible AI 158

Overview 160

Chapter Highlights 161

3.1. Assessing Responsible AI 163

Responsible AI Definitions 163

AI Incidents 164

Examples 164

Risk Perception 166

Risk Mitigation 167

Overall Trustworthiness 168

Benchmarking Responsible AI 169

Tracking Notable Responsible AI Benchmarks 169

Reporting Consistency 170

3.2. Privacy and Data Governance 172

Current Challenges 172

Privacy and Data Governance in Numbers 173

Academia 173

Industry 174

Featured Research 175

Extracting Data From LLMs 175

Foundation Models and Verbatim Generation 177

Auditing Privacy in AI Models 179

3.3. Transparency and Explainability 180

Current Challenges 180

Transparency and Explainability in Numbers 181

Academia 181

Industry 182

Featured Research 183

The Foundation Model Transparency Index 183

Neurosymbolic Artificial Intelligence (Why, What, and How) 185

3.4. Security and Safety 186

Current Challenges 186

AI Security and Safety in Numbers 187

Academia 187

Industry 188

Featured Research 191

Do-Not-Answer: A New Open Dataset for Comprehensive Benchmarking of LLM Safety Risks 191

Universal and Transferable Attacks on Aligned Language Models 193

MACHIAVELLI Benchmark 195

3.5. Fairness 197

Current Challenges 197

Fairness in Numbers 197

Academia 197

Industry 198

Featured Research 199

(Un)Fairness in AI and Healthcare 199

Social Bias in Image Generation Models 200

Measuring Subjective Opinions in LLMs 201

LLM Tokenization Introduces Unfairness 203

3.6. AI and Elections 205

Generation, Dissemination, and Detection of Disinformation 205

Generating Disinformation 205

Dissemination of Fake Content 207

Detecting Deepfakes 208

LLMs and Political Bias 210

Impact of AI on Political Processes 211

CHAPTER 4: Economy 213

Overview 215

Chapter Highlights 216

4.1. What's New in 2023: A Timeline 218

4.2. Jobs 223

AI Labor Demand 223

Global AI Labor Demand 223

U.S. AI Labor Demand by Skill Cluster and Specialized Skill 224

U.S. AI Labor Demand by Sector 228

U.S. AI Labor Demand by State 229

AI Hiring 232

AI Skill Penetration 234

AI Talent 236

Highlight: How Much Do Computer Scientists Earn? 240

4.3. Investment 242

Corporate Investment 242

Startup Activity 243

Global Trends 243

Regional Comparison by Funding Amount 247

Regional Comparison by Newly Funded AI Companies 251

Focus Area Analysis 254

4.4. Corporate Activity 258

Industry Adoption 258

Adoption of AI Capabilities 258

Adoption of Generative AI Capabilities 266

Use of AI by Developers 269

Preference 269

Workflow 270

AI's Labor Impact 272

Earnings Calls 277

Aggregate Trends 277

Specific Themes 278

Highlight: Projecting AI's Economic Impact 279

4.5. Robot Installations 283

Aggregate Trends 283

Industrial Robots: Traditional vs. Collaborative Robots 285

By Geographic Area 286

Country-Level Data on Service Robotics 290

Sectors and Application Types 292

China vs. United States 294

CHAPTER 5: Science and Medicine 296

Overview 298

Chapter Highlights 299

5.1. Notable Scientific Milestones 300

AlphaDev 300

FlexiCubes 301

Synbot 303

GraphCast 304

GNoME 305

Flood Forecasting 306

5.2. AI in Medicine 307

Notable Medical Systems 307

SynthSR 307

Coupled Plasmonic Infrared Sensors 309

EVEscape 310

AlphaMissence 312

Human Pangenome Reference 313

Clinical Knowledge 314

MedQA 314

Highlighted Research: GPT-4 Medprompt 315

Highlighted Research: MediTron-70B 317

Diagnosis 318

Highlighted Research: CoDoC 318

Highlighted Research: CT Panda 319

Other Diagnostic Uses 320

FDA-Approved AI-Related Medical Devices 321

Administration and Care 323

Highlighted Research: MedAlign 323

CHAPTER 6: Education 325

Overview 327

Chapter Highlights 328

6.1. Postsecondary CS and AI Education 329

United States and Canada 329

CS Bachelor's Graduates 329

CS Master's Graduates 331

CS PhD Graduates 333

CS, CE, and Information Faculty 336

Europe 344

Informatics, CS, CE, and IT Bachelor's Graduates 344

Informatics, CS, CE, and IT Master's Graduates 347

Informatics, CS, CE, and IT PhD Graduates 351

AI-Related Study Programs 355

Total Courses 355

Education Level 356

Geographic Distribution 357

6.2. K-12 CS and AI Education 359

United States 359

State-Level Trends 359

AP Computer Science 361

Highlight: Access Issues 363

Highlight: ChatGPT Usage Among Teachers and Students 364

CHAPTER 7: Policy and Governance 366

Overview 368

Chapter Highlights 369

7.1. Overview of AI Policy in 2023 370

7.2. AI and Policymaking 376

Global Legislative Records on AI 376

Overview 376

By Geographic Area 378

By Relevance 379

By Approach 380

By Subject Matter 381

U.S. Legislative Records 382

Federal Level 382

State Level 383

AI Mentions 385

Overview 385

U.S. Committee Mentions 388

7.3. National AI Strategies 391

By Geographic Area 391

7.4. AI Regulation 393

U.S. Regulation 393

Overview 393

By Relevance 394

By Agency 395

By Approach 396

By Subject Matter 397

EU Regulation 398

Overview 398

By Relevance 399

By Agency 400

By Approach 401

By Subject Matter 402

7.5. U.S. Public Investment in AI 403

Federal Budget for AI R&D 403

U.S. Department of Defense Budget Requests 405

U.S. Government AI-Related Contract Spending 406

AI Contract Spending 406

Microelectronics and Semiconductor Spending 409

CHAPTER 8: Diversity 411

Overview 413

Chapter Highlights 414

8.1. AI Postsecondary Education 415

North America 415

CS Bachelor's Graduates 415

CS Master's Graduates 417

CS PhD Graduates 419

Disability Status of CS, CE, and Information Students 421

CS, CE, and Information Faculty 422

Europe 425

Informatics, CS, CE, and IT Bachelor's Graduates 425

Informatics, CS, CE, and IT Master's Graduates 425

Informatics, CS, CE, and IT PhD Graduates 425

8.2. AI Conferences 429

Women in Machine Learning (WiML) NeurIPS Workshop 429

Workshop Participants 429

Demographic Breakdown 430

8.3. K-12 Education 432

AP Computer Science: Gender 432

AP Computer Science: Ethnicity 433

CHAPTER 9: Public Opinion 435

Overview 437

Chapter Highlights 438

9.1. Survey Data 439

Global Public Opinion 439

AI Products and Services 439

AI and Jobs 444

AI and Livelihood 446

Attitudes on ChatGPT 448

AI Concerns 451

U.S. Public Opinion 452

9.2. Social Media Data 454

Dominant Models 454

Highlight: AI-Related Social Media Discussion in 2023 456

Appendix 458

Chapter 1: Research and Development 460

Chapter 2: Technical Performance 465

Chapter 3: Responsible AI 472

Chapter 4: Economy 478

Chapter 5: Science and Medicine 488

Chapter 6: Education 491

Chapter 7: Policy and Governance 495

Chapter 8: Diversity 500

Chapter 9: Public Opinion 501

Figures 31

Figure 1.1.1. Number of AI publications in the world, 2010-22 31

Figure 1.1.2. Number of AI publications by type, 2010-22 32

Figure 1.1.3. Number of AI publications by field of study (excluding Other AI), 2010-22 33

Figure 1.1.4. AI publications (% of total) by sector, 2010-22 34

Figure 1.1.5. AI publications (% of total) by sector and geographic area, 2022 35

Figure 1.1.6. Number of AI journal publications, 2010-22 36

Figure 1.1.7. Number of AI conference publications, 2010-22 37

Figure 1.2.1. Number of AI patents granted, 2010-22 38

Figure 1.2.2. AI patents by application status, 2010-22 39

Figure 1.2.3. AI patents by application status by geographic area, 2010-22 40

Figure 1.2.4. Granted AI patents (% of world total) by region, 2010-22 41

Figure 1.2.5. Granted AI patents (% of world total) by geographic area, 2010-22 42

Figure 1.2.6. Granted AI patents per 100,000 inhabitants by country, 2022 43

Figure 1.2.7. Percentage change of granted AI patents per 100,000 inhabitants by country, 2012 vs. 2022 44

Figure 1.3.1. Number of notable machine learning models by sector, 2003-23 46

Figure 1.3.2. Number of notable machine learning models by geographic area, 2023 47

Figure 1.3.3. Number of notable machine learning models by select geographic area, 2003-23 47

Figure 1.3.4. Number of notable machine learning models by geographic area, 2003-23 (sum) 48

Figure 1.3.5. Number of parameters of notable machine learning models by sector, 2003-23 49

Figure 1.3.6. Training compute of notable machine learning models by sector, 2003-23 50

Figure 1.3.7. Training compute of notable machine learning models by domain, 2012-23 51

Figure 1.3.8. Projections of ML data exhaustion by stock type: median and 90% CI dates 52

Figure 1.3.9. A demonstration of model collapse in a VAE 53

Figure 1.3.10. Convergence of generated data densities in descendant models 54

Figure 1.3.11. An example of MAD in image-generation models 55

Figure 1.3.12. Assessing FFHQ syntheses: FID, precision, and recall in synthetic and mixed-data training loops 55

Figure 1.3.13. Foundation models by access type, 2019-23 56

Figure 1.3.14. Foundation models (% of total) by access type, 2019-23 57

Figure 1.3.15. Number of foundation models by sector, 2019-23 58

Figure 1.3.16. Number of foundation models by organization, 2023 59

Figure 1.3.17. Number of foundation models by organization, 2019-23 (sum) 60

Figure 1.3.18. Number of foundation models by geographic area, 2023 61

Figure 1.3.19. Number of foundation models by select geographic area, 2019-23 61

Figure 1.3.20. Number of foundation models by geographic area, 2019-23 (sum) 62

Figure 1.3.21. Estimated training cost of select AI models, 2017-23 64

Figure 1.3.22. Estimated training cost of select AI models, 2016-23 64

Figure 1.3.23. Estimated training cost and compute of select AI models 65

Figure 1.4.1. Attendance at select AI conferences, 2010-23 66

Figure 1.4.2. Attendance at large conferences, 2010-23 67

Figure 1.4.3. Attendance at small conferences, 2010-23 68

Figure 1.5.1. Number of GitHub AI projects, 2011-23 69

Figure 1.5.2. GitHub AI projects (% of total) by geographic area, 2011-23 70

Figure 1.5.3. Number of GitHub stars in AI projects, 2011-23 71

Figure 1.5.4. Number of GitHub stars by geographic area, 2011-23 72

Figure 2.1.1. (Omit) 78

Figure 2.1.2. (Omit) 78

Figure 2.1.3. (Omit) 78

Figure 2.1.4. (Omit) 78

Figure 2.1.5. (Omit) 79

Figure 2.1.6. (Omit) 79

Figure 2.1.7. (Omit) 79

Figure 2.1.8. (Omit) 79

Figure 2.1.9. (Omit) 79

Figure 2.1.10. (Omit) 79

Figure 2.1.11. (Omit) 80

Figure 2.1.12. (Omit) 80

Figure 2.1.13. (Omit) 80

Figure 2.1.14. (Omit) 80

Figure 2.1.15. (Omit) 80

Figure 2.1.16. Select AI Index technical performance benchmarks vs. human performance 81

Figure 2.1.17. A selection of deprecated benchmarks from the 2023 AI Index report 82

Figure 2.1.18. Year-over-year improvement over time on select AI Index technical performance benchmarks 83

Figure 2.1.19. New benchmarks featured in the 2024 AI Index report 84

Figure 2.2.1. A sample output from GPT-4 85

Figure 2.2.2. Gemini handling image and audio inputs 85

Figure 2.2.3. HELM: mean win rate 86

Figure 2.2.4. Leaders on individual HELM sub-benchmarks 86

Figure 2.2.5. A sample question from MMLU 87

Figure 2.2.6. MMLU: average accuracy 87

Figure 2.2.7. A sample model response on the Chatbot Arena Leaderboard 88

Figure 2.2.8. LMSYS Chatbot Arena for LLMs: Elo rating 89

Figure 2.2.9. Sample TruthfulQA questions 90

Figure 2.2.10. Multiple-choice task on TruthfulQA: MC1 91

Figure 2.2.11. A generated hallucinated QA example and a human-labeled ChatGPT response for a user query 92

Figure 2.2.12. HaluEnal hallucination classification accuracy 93

Figure 2.3.1. Sample HumanEval problem 94

Figure 2.3.2. HumanEval: Pass@1 94

Figure 2.3.3. A sample model input from SWE-bench 95

Figure 2.3.4. SWE-bench: percent resolved 95

Figure 2.4.1. Which face is real? 96

Figure 2.4.2. Midjourney generations over time: "a hyper-realistic image of Harry Potter" 96

Figure 2.4.3. Image-text alignment: human evaluation 97

Figure 2.4.4. Model leaders on select HEIM sub-benchmarks 97

Figure 2.4.5. Sample generations from MVDream 98

Figure 2.4.6. Quantitative evaluation on image synthesis quality 98

Figure 2.4.7/Figure 2.4.8. A sample VisIT-Bench instruction set 99

Figure 2.4.8/Figure 2.4.9. VisIT-Bench: Elo rating 99

Figure 2.4.9/Figure 2.4.10. A sample VisIT-Bench instruction set 100

Figure 2.4.10/Figure 2.4.11. EditVal automatic evaluation: editing accuracy 100

Figure 2.4.11/Figure 2.4.12. Sample edits using ControlNet 101

Figure 2.4.12/Figure 2.4.13. Average User Ranking (AUR): result quality and condition fidelity 102

Figure 2.4.13/Figure 2.4.14. A demonstration of Instruct-NeRF2NeRF in action 103

Figure 2.4.14/Figure 2.4.15. Evaluating text-image alignment and frame consistency 104

Figure 2.4.15/Figure 2.4.16. Various segmentation masks created by Segment Anything 105

Figure 2.4.16/Figure 2.4.17. SAM vs. RITM: mean IoU 106

Figure 2.4.17/Figure 2.4.18. Objects from the 3D reconstruction dataset 107

Figure 2.4.18/Figure 2.4.19. Skoltech3D vs. the most widely used multisensor datasets 107

Figure 2.4.19/Figure 2.4.20. Sample generations from RealFusion 108

Figure 2.4.20/Figure 2.4.21. Object reconstruction: RealFusion vs. Shelf-Supervised 108

Figure 2.5.1. Sample frames from UCF101 109

Figure 2.5.2. UCF101: FVD16 109

Figure 2.5.3. High-quality generation of milk dripping into a cup of coffee 110

Figure 2.5.4. Video LDM vs. LVG: FVD and FID 110

Figure 2.5.5. Sample Emu Video generations 111

Figure 2.5.6. Emu Video vs. prior works: human-evaluated video quality and text faithfulness win rate 111

Figure 2.6.1. Sample MMMU questions 113

Figure 2.6.2. MMMU: overall accuracy 114

Figure 2.6.3. MMMU: subject-specific accuracy 114

Figure 2.6.4. A sample chemistry question from GPQA 115

Figure 2.6.5. GPQA: accuracy on the main set 115

Figure 2.6.6. A sample ARC reasoning task 116

Figure 2.6.7. ConceptARC: accuracy on minimal tasks over all concepts 116

Figure 2.6.8. Sample problems from GSM8K 117

Figure 2.6.9. GSM8K: accuracy 118

Figure 2.6.10. A sample problem from the MATH dataset 119

Figure 2.6.11. MATH word problem-solving: accuracy 119

Figure 2.6.12. GPT-4 vs. I-GPT-3 on PlanBench 120

Figure 2.6.13. A sample question from the Visual Commonsense Reasoning (VCR) challenge 121

Figure 2.6.14. Visual Commonsense Reasoning (VCR) task: Q→AR score 121

Figure 2.6.15. A moral story from MoCa 122

Figure 2.6.16. Zero-shot alignment with human judgments on the moral permissibility task: discrete agreement 123

Figure 2.6.17. Sample BigToM scenario 124

Figure 2.6.18. Forward action inference with initial belief: accuracy 125

Figure 2.6.19. Backward belief inference with initial belief: accuracy 125

Figure 2.6.20. Forward belief inference with initial belief: accuracy 125

Figure 2.6.21. Sample cause-effect pairs from the Tübingen dataset 126

Figure 2.6.22. Performance on the Tübingen Cause-Effect Pairs dataset: accuracy 126

Figure 2.7.1. UniAudio vs. selected prior works in the training stage: objective evaluation metrics 128

Figure 2.7.2. Evaluation of MusicGen and baseline models on MusicCaps 130

Figure 2.8.1. Description of the AgentBench benchmark 131

Figure 2.8.2. AgentBench across eight environments: overall score 132

Figure 2.8.3. Voyager in action 133

Figure 2.8.4. Voyager's performance improvements over prior state of the art in Minecraft 133

Figure 2.8.5. MLAgentBench evaluation: success rate of select models across tasks 134

Figure 2.9.1. PaLM-E in action 136

Figure 2.9.2. Performance of select models on TAMP environment: success rate 136

Figure 2.9.3. Select models on mobile manipulation environment tests: failure detection 136

Figure 2.9.4. Evaluation of RT-2 models and baselines on seen and unseen tasks: success rate 137

Figure 2.10.1. Number of foundation models using RLHF, 2021-23 138

Figure 2.10.2. RLHF usage among foundation models 138

Figure 2.10.3. RLAIF and RLHF vs. SFT baseline: win rate 139

Figure 2.10.4. Harmless rate by policy 139

Figure 2.10.5. Comparison of different algorithms on TL;DR summarization task across different sampling temperatures 140

Figure 2.11.1. Emergence score over all Big-bench tasks 142

Figure 2.11.2. Performance of the March 2023 and June 2023 versions of GPT-4 on eight tasks 144

Figure 2.11.3. GPT-4 on reasoning benchmarks with intrinsic self-correction 145

Figure 2.11.4. Score differentials of top closed vs. open models on select benchmarks 146

Figure 2.11.5. Performance of top closed vs. open models on select benchmarks 147

Figure 2.12.1. Graph of Thoughts (GoT) reasoning flow 148

Figure 2.12.2. Number of errors in sorting tasks with ChatGPT-3.5 149

Figure 2.12.3. Sample OPRO prompts and optimization progress 150

Figure 2.12.4. Accuracy difference on 23 BIG-bench Hard (BBH) tasks using PaLM 2-L scorer 150

Figure 2.12.5. Model competitions based on 10,000 simulations using GPT-4 and the Vicuna benchmark 151

Figure 2.12.6. Flash-Decoding operation process 152

Figure 2.12.7. Performance comparison of multihead attention algorithms across batch sizes and sequence lengths 153

Figure 2.13.1. CO₂ equivalent emissions (tonnes) by select machine learning models and real-life examples, 2020-23 154

Figure 2.13.2. CO₂equivalent emissions (tonnes) and number of parameters by select machine learning models 155

Figure 2.13.3. nvironmental impact of select models 155

Figure 2.13.4. Carbon emissions by task during model inference 156

Figure 2.13.5. Positive AI environmental use cases 157

Figure 3.1.1. Responsible AI dimensions, definitions, and examples 163

Figure 3.1.2. Number of reported AI incidents, 2012-23 164

Figure 3.1.3. Tesla recognizing pedestrian but not slowing down at a crosswalk 165

Figure 3.1.4. Romantic chatbot generated by DALL-E 165

Figure 3.1.5. Relevance of selected responsible AI risks for organizations by region 166

Figure 3.1.6. Global responsible AI adoption by organizations by region 167

Figure 3.1.7. Average trustworthiness score across selected responsible AI dimensions 168

Figure 3.1.8. Number of papers mentioning select responsible AI benchmarks, 2020-23 169

Figure 3.1.9. Reported general benchmarks for popular foundation models 170

Figure 3.1.10. Reported responsible AI benchmarks for popular foundation models 171

Figure 3.2.1. AI privacy and data governance submissions to select academic conferences, 2019-23 173

Figure 3.2.2. Adoption of AI-related data governance measures by region 174

Figure 3.2.3. Adoption of AI-related data governance measures by industry 174

Figure 3.2.4. Extracting PII From ChatGPT 175

Figure 3.2.5. Recovered memorized output given different repeated tokens 176

Figure 3.2.6. Fraction of prompts discovering approximate memorization 177

Figure 3.2.7. Identical generation of Thanos 178

Figure 3.2.8. Identical generation of toys 178

Figure 3.2.9. Identical generation of Mario 178

Figure 3.2.10. Visualizing privacy-auditing in one training run 179

Figure 3.3.1. AI transparency and explainability submissions to select academic conferences, 2019-23 181

Figure 3.3.2. Adoption of AI-related transparency measures by region 182

Figure 3.3.3. Adoption of AI-related transparency measures by industry 182

Figure 3.3.4. Foundation model transparency total scores of open vs. closed developers, 2023 183

Figure 3.3.5. Levels of accessibility and release strategies of foundation models 184

Figure 3.3.6. Integrating neural network structures with symbolic representation 185

Figure 3.4.1. AI security and safety submissions to select academic conferences, 2019-23 187

Figure 3.4.2. Adoption of AI-related reliability measures by region 188

Figure 3.4.3. Adoption of AI-related reliability measures by industry 188

Figure 3.4.4. Adoption of AI-related cybersecurity measures by region 189

Figure 3.4.5. Adoption of AI-related cybersecurity measures by industry 189

Figure 3.4.6. Agreement with security statements 190

Figure 3.4.7. Harmful responses across different risk caregories by foundation model 191

Figure 3.4.8. Total number of harmful responses across different foundation models 192

Figure 3.4.9. Using suffixes to manipulate LLMs 193

Figure 3.4.10. Attack success rates of foundation models using different prompting techniques 194

Figure 3.4.11. Trade-offs on the MACHIAVELLI benchmark 195

Figure 3.4.12. Mean behavioral scores of AI agents across different categories 196

Figure 3.5.1. AI fairness and bias submissions to select academic conferences, 2019-23 197

Figure 3.5.2. Adoption of AI-related fairness measures by region 198

Figure 3.5.3. Adoption of AI-related fairness measures by industry 198

Figure 3.5.4. Number of runs (out of 5 total runs) with concerning race-based responses by large language model 199

Figure 3.5.5. Midjourney generation: "influential person" 200

Figure 3.5.6. Average image model bias scores for five widely used commercial image generation models 200

Figure 3.5.7. GlobalOpinionQA Dataset 201

Figure 3.5.8. Western-oriented bias in large language model responses 202

Figure 3.5.9. Context window 203

Figure 3.5.10. Variable language tokenization 203

Figure 3.5.11. Tokenization premium using XLM-RoBERTa and RoBERTa models by language 204

Figure 3.6.1. Potential uses of deepfakes 205

Figure 3.6.2. Progressive Slovakia leader Michal Šimečka 206

Figure 3.6.3. AI-based generation and dissemination pipeline 207

Figure 3.6.4. Generalizability of deepfake detectors to unseen datasets 208

Figure 3.6.5. Ethnic and gender distribution in FaceForensics++ training data 209

Figure 3.6.6. Default vs. political ChatGPT average agreement 210

Figure 3.6.7. Key research findings on audio deepfakes 211

Figure 3.6.8. AI usage, risks, and mitigation strategies in electoral processes 212

Figure 3.6.9. Assessments of AI integration and risks in electoral processes 212

Figure 4.1.1. InstaDeep acquired by BioNTech 218

Figure 4.1.2. Microsoft invests $10 billion in ChatGPT maker OpenAI 218

Figure 4.1.3. GitHub Copilot for Business becomes publicly available 218

Figure 4.1.4. Salesforce introduces Einstein GPT 218

Figure 4.1.5. Microsoft announces integration of GPT-4 into Office 365 219

Figure 4.1.6. Bloomberg announces LLM for finance 219

Figure 4.1.7. Adobe launches generative AI tools inside Photoshop 219

Figure 4.1.8. Cohere raises $270 million 219

Figure 4.1.9. Nvidia reaches $1 trillion valuation 220

Figure 4.1.10. Databricks buys MosaicML for $1.3 billion 220

Figure 4.1.11. Thomson Reuters acquires Casetext for $650 million 220

Figure 4.1.12. Inflection AI raises $1.3 billion from Bill Gates and Nvidia, among others 220

Figure 4.1.13. Hugging Face raises $235 million from investors 221

Figure 4.1.14. SAP introduces new generative AI assistant Joule 221

Figure 4.1.15. Amazon and Google make multibillion-dollar investments in Anthropic 221

Figure 4.1.16. Kai-Fu Lee launches OpenSource LLM 221

Figure 4.1.17. Sam Altman, OpenAI CEO, fired and then rehired 222

Figure 4.1.18. Mistral AI closes $415 million funding round 222

Figure 4.2.1. AI job postings (% of all job postings) by geographic area, 2014-23 223

Figure 4.2.2. AI job postings (% of all job postings) in the United States by skill cluster, 2010-23 224

Figure 4.2.3. Top 10 specialized skills in 2023 AI job postings in the United States, 2011-13 vs. 2023 225

Figure 4.2.4. Generative AI skills in AI job postings in the United States, 2023 226

Figure 4.2.5. Share of generative AI skills in AI job postings in the United States, 2023 227

Figure 4.2.6. AI job postings (% of all job postings) in the United States by sector, 2022 vs. 2023 228

Figure 4.2.7. Number of AI job postings in the United States by state, 2023 229

Figure 4.2.8. Percentage of US states job postings in AI, 2023 229

Figure 4.2.9. Percentage of US AI job postings by state, 2023 230

Figure 4.2.10. Percentage of US states' job postings in AI by select US state, 2010-23 230

Figure 4.2.11. Percentage of US AI job postings by select US state, 2010-23 231

Figure 4.2.12. Relative AI hiring rate year-over-year ratio by geographic area, 2023 232

Figure 4.2.13. Relative AI hiring rate year-over-year ratio by geographic area, 2018-23 233

Figure 4.2.14. Relative AI skill penetration rate by geographic area, 2015-23 234

Figure 4.2.15. Relative AI skill penetration rate across gender, 2015-23 235

Figure 4.2.16. AI talent concentration by geographic area, 2023 236

Figure 4.2.17. Percentage change in AI talent concentration by geographic area, 2016 vs. 2023 236

Figure 4.2.18. AI talent concentration by gender, 2016-23 237

Figure 4.2.19. Net AI talent migration per 10,000 LinkedIn members by geographic area, 2023 238

Figure 4.2.20. Net AI talent migration per 10,000 LinkedIn members by geographic area, 2019-23 239

Figure 4.2.21. Median yearly salary by professional developer type, 2023 241

Figure 4.3.1. Global corporate investment in AI by investment activity, 2013-23 242

Figure 4.3.2. Private investment in AI, 2013-23 243

Figure 4.3.3. Private investment in generative AI, 2019-23 244

Figure 4.3.4. Number of newly funded AI companies in the world, 2013-23 245

Figure 4.3.5. Average size of AI private investment events, 2013-23 245

Figure 4.3.6. Number of newly funded generative AI companies in the world, 2019-23 246

Figure 4.3.7. AI private investment events by funding size, 2022 vs. 2023 246

Figure 4.3.8. Private investment in AI by geographic area, 2023 247

Figure 4.3.9. Private investment in AI by geographic area, 2013-23 (sum) 248

Figure 4.3.10. Private investment in AI by geographic area, 2013-23 249

Figure 4.3.11. Private investment in generative AI by geographic area, 2019-23 250

Figure 4.3.12. Number of newly funded AI companies by geographic area, 2023 251

Figure 4.3.13. Number of newly funded AI companies by geographic area, 2013-23 (sum) 252

Figure 4.3.14. Number of newly funded AI companies by geographic area, 2013-23 253

Figure 4.3.15. Private investment in AI by focus area, 2022 vs. 2023 254

Figure 4.3.16. Private investment in AI by focus area, 2017-23 255

Figure 4.3.17. Private investment in AI by focus area and geographic area, 2017-23 257

Figure 4.4.1. Share of respondents who say their organizations have adopted AI in at least one function, 2017-23 258

Figure 4.4.2. Most commonly adopted AI use cases by function, 2023 259

Figure 4.4.3. AI capabilities embedded in at least one function or business unit, 2023 260

Figure 4.4.4. AI adoption by industry and function, 2023 261

Figure 4.4.5. Percentage point change in responses of AI adoption by industry and function, 2022 vs. 2023 262

Figure 4.4.6. AI-related roles that organizations hired in the last year by industry, 2023 263

Figure 4.4.7. Cost decrease and revenue increase from AI adoption by function, 2022 264

Figure 4.4.8. AI adoption by organizations in the world, 2022 vs. 2023 265

Figure 4.4.9. Most commonly adopted generative AI use cases by function, 2023 266

Figure 4.4.10. AI vs. generative AI adoption by function, 2023 267

Figure 4.4.11. Generative AI adoption by organizations in the world, 2023 268

Figure 4.4.12. Most popular AI developer tools among professional developers, 2023 269

Figure 4.4.13. Most popular AI search tools among professional developers, 2023 269

Figure 4.4.14. Top 10 most popular cloud platforms among professional developers, 2023 270

Figure 4.4.15. Adoption of AI tools in development tasks, 2023 270

Figure 4.4.16. Primary benefits of AI tools for professional developers, 2023 271

Figure 4.4.17. Sentiment toward AI tools in development among professional developers, 2023 271

Figure 4.4.18. Trust level in AI tool output accuracy, 2023 271

Figure 4.4.19. Cross-study comparison of task completion speed of Copilot users 272

Figure 4.4.20. Effect of GPT-4 use on a group of consultants 273

Figure 4.4.21. Impact of AI on customer support agents 273

Figure 4.4.22. Effect of GPT-4 use on legal analysis by task 274

Figure 4.4.23. Comparison of AI work performance effect by worker skill category 275

Figure 4.4.24. Effects on job performance of receiving different types of AI advice 276

Figure 4.4.25. Number of Fortune 500 earnings calls mentioning AI, 2018-23 277

Figure 4.4.26. Themes of AI mentions in Fortune 500 earnings calls, 2018 vs. 2023 278

Figure 4.4.27. Anticipated impact of generative AI on revenue by industry, 2023 279

Figure 4.4.28. Expectations about the impact of AI on organizations' workforces in the next 3 years, 2023 280

Figure 4.4.29. Anticipated effect of generative AI on number of employees in the next 3 years by business function, 2023 281

Figure 4.4.30. Estimated impact of AI adoption on annual productivity growth over a ten-year period 282

Figure 4.5.1. Number of industrial robots installed in the world, 2012-22 283

Figure 4.5.2. Operational stock of industrial robots in the world, 2012-22 284

Figure 4.5.3. Number of industrial robots installed in the world by type, 2017-22 285

Figure 4.5.4. Number of industrial robots installed by country, 2022 286

Figure 4.5.5. Number of new industrial robots installed in top 5 countries, 2012-22 287

Figure 4.5.6. Number of industrial robots installed (China vs. rest of the world), 2016-22 288

Figure 4.5.7. Annual growth rate of industrial robots installed by country, 2021 vs. 2022 289

Figure 4.5.8. Number of professional service robots installed in the world by application area, 2021 vs. 2022 290

Figure 4.5.9. Number of professional service robot manufacturers in top countries by type of company, 2022 291

Figure 4.5.10. Number of industrial robots installed in the world by sector, 2020-22 292

Figure 4.5.11. Number of industrial robots installed in the world by application, 2020-22 293

Figure 4.5.12. Number of industrial robots installed in China by sector, 2020-22 294

Figure 4.5.13. Number of industrial robots installed in the United States by sector, 2020-22 295

Figure 5.1.1. AlphaDev vs. human benchmarks when optimizing for algorithm length 300

Figure 5.1.2. Sample FlexiCubes surface reconstructions 301

Figure 5.1.3. Select quantitative results on 3D mesh reconstruction 302

Figure 5.1.4. Synbot design 303

Figure 5.1.5. Reaction kinetics of M1 autonomous optimization experiment, Synbot vs. reference 303

Figure 5.1.6. GraphCast weather prediction 304

Figure 5.1.7. Ten-day z500 forecast skill: GraphCast vs. HRES 304

Figure 5.1.8. Sample material structures 305

Figure 5.1.9. GNoME vs. Materials Project: stable crystal count 305

Figure 5.1.10. GNoME vs. Materials Project: distinct prototypes 305

Figure 5.1.11. Predictions of AI model vs. GloFAS across return periods 306

Figure 5.2.1. SynthSR generations 307

Figure 5.2.2. SynthSR correlation with ground-truth volumes on select brain regions 308

Figure 5.2.3. ImmunoSEIRA detection principle and the setup 309

Figure 5.2.4. Deep neural network predicted vs. actual fibrils percetages in test samples 309

Figure 5.2.5. EVEscape design 310

Figure 5.2.6. EVEscape vs. other models on SARS-CoV-2 RBD mutation prediction 311

Figure 5.2.7. Hemaglobin subunit beta (HBB) 312

Figure 5.2.8. AlphaMissense predictions 312

Figure 5.2.9. Graph genome for the MHC region of the genome 313

Figure 5.2.10. Ensembl mapping pipeline results 313

Figure 5.2.11. MedQA: accuracy 314

Figure 5.2.12. GPT-4 vs. Med-PaLM 2 answering a medical question 315

Figure 5.2.13. Model performance on MultiMedQA sub-benchmarks 316

Figure 5.2.14. Performance of select models on MedQA 317

Figure 5.2.15. CoDoC vs. standalone predictive AI system and clinical readers: sensitivity 318

Figure 5.2.16. PANDA detection 319

Figure 5.2.17. PANDA vs. mean radiologist on multicenter validation (6,239 patients) 319

Figure 5.2.18. PANDA performance on real-world multi-scenario validation (20,530 patients) 319

Figure 5.2.19. Additional research on diagnostic AI use cases 320

Figure 5.2.20. Number of AI medical devices approved by the FDA, 2012-22 321

Figure 5.2.21. Number of AI medical devices approved by the FDA by specialty, 2012-22 322

Figure 5.2.22. MedAlign workflow 323

Figure 5.2.23. Evaluation of model performance: human vs. COMET ranks 324

Figure 6.1.1. New CS bachelor's graduates in the United States and Canada, 2010-22 329

Figure 6.1.2. New international CS bachelor's graduates (% of total) in the United States and Canada, 2010-22 330

Figure 6.1.3. New CS master's graduates in the United States and Canada, 2010-22 331

Figure 6.1.4. New international CS master's graduates (% of total) in the United States and Canada, 2010-22 332

Figure 6.1.5. New CS PhD graduates in the United States and Canada, 2010-22 333

Figure 6.1.6. New international CS PhD graduates (% of total) in the United States and Canada, 2010-22 334

Figure 6.1.7. Employment of new AI PhDs (% of total) in the United States and Canada by sector, 2010-22 335

Figure 6.1.8. Employment of new AI PhDs in the United States and Canada by sector, 2010-22 335

Figure 6.1.9. Number of CS, CE, and information faculty in the United States and Canada, 2011-22 336

Figure 6.1.10. Number of CS faculty in the United States, 2011-22 337

Figure 6.1.11. New CS, CE, and information faculty hires in the United States and Canada, 2011-22 338

Figure 6.1.12. Source of new faculty in American and Canadian CS, CE, and information departments, 2018-22 339

Figure 6.1.13. Reason why new CS, CE, and information faculty positions remained unfilled (% of total), 2011-22 340

Figure 6.1.14. Faculty losses in American and Canadian CS, CE, and information departments, 2011-22 341

Figure 6.1.15. Median nine-month salary of CS faculty in the United States, 2015-22 342

Figure 6.1.16. New international CS, CE, and information tenure-track faculty hires (% of total) in the United States and Canada, 2010-22 343

Figure 6.1.17. New informatics, CS, CE, and IT bachelor's graduates by country in Europe, 2022 344

Figure 6.1.18. Percentage change of new informatics, CS, CE, and IT bachelor's graduates by country in Europe, 2012 vs. 2022 345

Figure 6.1.19. New informatics, CS, CE, and IT bachelor's graduates per 100,000 inhabitants by country in Europe, 2022 346

Figure 6.1.20. Percentage change of new CS, CE, and Information bachelor's graduates per 100,000 inhabitants by country in Europe, 2012 vs. 2022 346

Figure 6.1.21. New informatics, CS, CE, and IT master's graduates by country in Europe, 2022 347

Figure 6.1.22. Percentage change of new informatics, CS, CE, and IT master's graduates by country in Europe, 2012 vs. 2022 348

Figure 6.1.23. New informatics, CS, CE, and IT master's graduates per 100,000 inhabitants by country in Europe, 2022 349

Figure 6.1.24. Percentage change of new informatics, CS, CE, and IT master's graduates per 100,000 inhabitants by country in Europe, 2012 vs. 2022 350

Figure 6.1.25. New informatics, CS, CE, and IT PhD graduates by country in Europe, 2022 351

Figure 6.1.26. Percentage change of new informatics, CS, CE, and IT PhD graduates by country in Europe, 2012 vs. 2022 352

Figure 6.1.27. New informatics, CS, CE, and IT PhD graduates per 100,000 inhabitants by country in Europe, 2022 353

Figure 6.1.28. Percentage change of new informatics, CS, CE, and IT PhD graduates per 100,000 inhabitants by country in Europe, 2012 vs. 2022 354

Figure 6.1.29. Number of AI university study programs in English in the world, 2017-23 355

Figure 6.1.30. AI university study programs in English (% of total) by education level, 2023 356

Figure 6.1.31. Number of AI university study programs in English by geographic area, 2022 vs. 2023 357

Figure 6.1.32. AI university study programs in English per 100,000 inhabitants by geographic area, 2022 vs. 2023 358

Figure 6.2.1. States requiring that all high schools offer a foundational CS course, 2023 359

Figure 6.2.2. Public high schools teaching foundational CS (% of total in state), 2023 359

Figure 6.2.3. Changes over time in state-level US K-12 CS education 360

Figure 6.2.4. Number of AP computer science exams taken, 2007-22 361

Figure 6.2.5. Number of AP computer science exams taken, 2022 362

Figure 6.2.6. Number of AP computer science exams taken per 100,000 inhabitants, 2022 362

Figure 6.2.7. Schools offering foundational CS courses by size, 2023 363

Figure 6.2.8. Schools offering foundational CS courses by geographic area, 2023 363

Figure 6.2.9. ChatGPT usage rate among American K-12 teachers, 2023 364

Figure 6.2.10. ChatGPT usage purposes among American K-12 teachers, 2023 364

Figure 6.2.11. ChatGPT perceptions among educational users, 2023 365

Figure 7.1.1. China introduces regulation on administration of deep synthesis of the internet 370

Figure 7.1.2. U.S. legislators propose AI for National Security Act 370

Figure 7.1.3. U.S. policymakers introduce AI Leadership Training Act 371

Figure 7.1.4. U.S. policymakers propose National AI Commission Act 371

Figure 7.1.5. House of Representatives advances Jobs of the Future Act 371

Figure 7.1.6. U.S. Senate puts forward Artificial Intelligence and Biosecurity Risk Assessment Act 372

Figure 7.1.7. Private AI labs sign voluntary White House AI commitments 372

Figure 7.1.8. U.S. Senate passes Outbound Investment Transparency Act 372

Figure 7.1.9. U.S. Senate proposes CREATE AI Act 373

Figure 7.1.10. China updates cyberspace administration of generative AI measures 373

Figure 7.1.11. U.S. Senate puts forward Protect Elections from Deceptive AI Act 373

Figure 7.1.12. U.K. proposes principles to guide competitive AI markets and protect consumers 374

Figure 7.1.13. President Biden issues Executive Order on Safe, Secure, and Trustworthy AI 374

Figure 7.1.14. Frontier AI taskforce releases second progress report 374

Figure 7.1.15. U.K. hosts AI Safety Summit (2023) 375

Figure 7.1.16. U.K. announces AI Safety Institute 375

Figure 7.1.17. Europeans reach deal on EU AI Act 375

Figure 7.2.1. Number of AI-related bills passed into law by country, 2016-23 376

Figure 7.2.2. Number of AI-related bills passed into law in 128 select countries, 2016-23 377

Figure 7.2.3. Number of AI-related bills passed into law in select countries, 2023 378

Figure 7.2.4. Number of AI-related bills passed into law in select countries, 2016-23 (sum) 378

Figure 7.2.5. Number of AI-related bills passed into law in select countries by relevance to AI, 2016-23 379

Figure 7.2.6. Number of AI-related bills passed into law in select countries by approach, 2016-23 380

Figure 7.2.7. Number of AI-related bills passed into law in select countries by primary subject matter, 2016-23 381

Figure 7.2.8. Number of AI-related bills in the United States, 2016-23 (proposed vs. passed) 382

Figure 7.2.9. Number of AI-related bills passed into law in select US states, 2023 383

Figure 7.2.10. Number of state-level AI-related bills passed into law in the United States by state, 2016-23 (sum) 383

Figure 7.2.11. Number of state-level AI-related bills in the United States, 2016-23 (proposed vs. passed) 384

Figure 7.2.12. Number of mentions of AI in legislative proceedings in 80 select countries, 2016-23 385

Figure 7.2.13. Number of mentions of AI in legislative proceedings by country, 2023 386

Figure 7.2.14. Number of mentions of AI in legislative proceedings by country, 2016-23 (sum) 387

Figure 7.2.15. Mentions of AI in US committee reports by legislative session, 2001-23 388

Figure 7.2.16. Mentions of AI in committee reports of the US House of Representatives for the 118th congressional session, 2023 389

Figure 7.2.17. Mentions of AI in committee reports of the US Senate for the 118th congressional session, 2023 389

Figure 7.2.18. Mentions of AI in committee reports of the US House of Representatives, 2001-23 (sum) 390

Figure 7.2.19. Mentions of AI in committee reports of the US Senate, 2001-23 (sum) 390

Figure 7.3.1. Countries with a national strategy on AI, 2023 391

Figure 7.3.2. AI national strategies in development by country and year 392

Figure 7.3.3. Yearly release of AI national strategies by country 392

Figure 7.4.1. Number of AI-related regulations in the United States, 2016-23 393

Figure 7.4.2. Number of AI-related regulations in the United States by relevance to AI, 2016-23 394

Figure 7.4.3. Number of AI-related regulations in the United States by agency, 2016-23 395

Figure 7.4.4. Number of AI-related regulations in the United States by approach, 2016-23 396

Figure 7.4.5. Number of AI-related regulations in the United States by primary subject matter, 2016-23 397

Figure 7.4.6. Number of AI-related regulations in the European Union, 2017-23 398

Figure 7.4.7. Number of AI-related regulations in the European Union by relevance to AI, 2017-23 399

Figure 7.4.8. Number of AI-related regulations in the European Union by institution and body, 2017-23 400

Figure 7.4.9. Number of AI-related regulations in the European Union by approach, 2017-23 401

Figure 7.4.10. Number of AI-related regulations in the European Union by primary subject matter, 2017-23 402

Figure 7.5.1. US federal NITRD budget for AI, FY 2018-24 403

Figure 7.5.2. US governmental agency NITRD budgets for AI, FY 2021-24 404

Figure 7.5.3. US DoD budget request for AI-specific research, development, test, and evaluation (RDT&E), FY 2020-24 405

Figure 7.5.4. US government spending in AI/ML and autonomy by segment, FY 2018-23 406

Figure 7.5.5. US government spending in AI/ML and autonomy by segment, FY 2022 vs. 2023 407

Figure 7.5.6. Total value of contracts, grants, and OTAs awarded by the US government for AI/ML and autonomy, FY 2018-23 408

Figure 7.5.7. US government spending in microelectronics by segment, FY 2018-23 409

Figure 7.5.8. Total value of contracts, grants, and OTAs awarded by the US government for microelectronics, FY 2018-23 410

Figure 8.1.1. Gender of new CS bachelor's graduates (% of total) in the United States and Canada, 2010-22 415

Figure 8.1.2. Ethnicity of new resident CS bachelor's graduates in the United States and Canada, 2011-22 416

Figure 8.1.3. Ethnicity of new resident CS bachelor's graduates (% of total) in the United States and Canada, 2011-22 416

Figure 8.1.4. Gender of new CS master's graduates (% of total) in the United States and Canada, 2011-22 417

Figure 8.1.5. Ethnicity of new resident CS master's graduates in the United States and Canada, 2011-22 418

Figure 8.1.6. Ethnicity of new resident CS master's graduates (% of total) in the United States and Canada, 2011-22 418

Figure 8.1.7. Gender of new CS PhD graduates (% of total) in the United States and Canada, 2010-22 419

Figure 8.1.8. Ethnicity of new resident CS PhD graduates in the United States and Canada, 2011-22 420

Figure 8.1.9. Ethnicity of new resident CS PhD graduates (% of total) in the United States and Canada, 2011-22 420

Figure 8.1.10. CS, CE, and information students (% of total) with disability accomodations in United States and Canada, 2021 vs. 2022 421

Figure 8.1.11. Gender of CS, CE, and information faculty (% of total) in the United States and Canada, 2011-22 422

Figure 8.1.12. Gender of new CS, CE, and information faculty hires (% of total) in the United States and Canada, 2011-22 423

Figure 8.1.13. Ethnicity of resident CS, CE, and information faculty in the United States and Canada, 2011-22 424

Figure 8.1.14. Ethnicity of resident CS, CE, and information faculty (% of total) in the United States and Canada, 2011-22 424

Figure 8.1.15. Gender of new informatics, CS, CE, and IT bachelor's graduates (% of total) in Europe, 2011-22 426

Figure 8.1.16. Gender of new informatics, CS, CE, and IT master's graduates (% of total) in Europe, 2011-22 427

Figure 8.1.17. Gender of new informatics, CS, CE, and IT PhD graduates (% of total) in Europe, 2011-22 428

Figure 8.2.1. Attendance at NeurIPS Women in Machine Learning workshop, 2010-23 429

Figure 8.2.2. Attendance at NeurIPS Women in Machine Learning workshop (% of total), 2010-23 429

Figure 8.2.3. Continent of residence of participants at NeurIPS Women in Machine Learning workshop, 2022 vs. 2023 430

Figure 8.2.4. Gender breakdown of participants at NeurIPS Women in Machine Learning workshop, 2022 vs. 2023 431

Figure 8.3.1. AP computer science exams taken (% of total) by gender, 2007-22 432

Figure 8.3.2. AP computer science exams taken by female students (% of total), 2022 433

Figure 8.3.3. AP computer science exams taken by race/ethnicity, 2007-22 434

Figure 8.3.4. AP computer science exams taken (% of total responding students) by race/ethnicity, 2007-22 434

Figure 9.1.1. Global opinions on products and services using AI (% of total), 2022 vs. 2023 439

Figure 9.1.2. 'Products and services using AI have more benefits than drawbacks, 'by country (% of total), 2022 vs. 2023 441

Figure 9.1.3. Opinions about AI by country (% agreeing with statement), 2023 442

Figure 9.1.4. Percentage point change in opinions about AI by country (% agreeing with statement), 2022-23 443

Figure 9.1.5. Global opinions on the impact of AI on current jobs, 2023 444

Figure 9.1.6. Global opinions on the impact of AI on current jobs by demographic group, 2023 445

Figure 9.1.7. Global opinions on the potential of AI improving life by country, 2023 446

Figure 9.1.8. Global opinions on the potential of AI improving life by demographic group, 2023 447

Figure 9.1.9. Global awareness of ChatGPT (% of total), 2023 449

Figure 9.1.10. Global usage frequency of ChatGPT (% of total), 2023 450

Figure 9.1.11. Global concerns on the impacts of AI in the next few years, 2023 451

Figure 9.1.12. Americans' feelings toward increased use of AI in daily life (% of total), 2021-23 452

Figure 9.1.13. Americans' opinions of whether AI helps or hurts in specific settings (% of total), 2023 452

Figure 9.1.14. Differences in Americans' view of AI's impact by education level (% of total), 2023 453

Figure 9.2.1. Net sentiment score of AI models by quarter, 2023 454

Figure 9.2.2. Select models' share of AI social media attention by quarter, 2023 455