Title page 1

Contents 2

Abbreviations 3

Acknowledgments 3

1. INTRODUCTION: A JOURNEY OF DISCOVERY 4

Background 4

Context and Driving Forces 5

Objectives and Scope of the Artificial Intelligence Strategy 5

Defining Artificial Intelligence for Evaluation Practice 6

Where We Are Today 6

2. WHERE WE WANT TO GO: STEPS ON THIS JOURNEY OF DISCOVERY 8

Vision Statement 8

Key Values and Guiding Principles 8

Rethinking Evaluators and Evaluation Knowledge Brokers’ Roles and Workflows 10

Maturity Model 10

3. HOW WE WILL GET THERE: MAIN OBJECTIVES 12

Objective: Artificial Intelligence Use in Evaluation Practice 12

Objective: Artificial Intelligence Use in Validation Practice 16

Objective: Artificial Intelligence to Leverage and Broker Independent Evaluation Group Knowledge 18

4. BUILDING BLOCKS: IMPLEMENTING THE STRATEGY 19

Governance 19

Staffing, Capacity, and Resources 25

Data and Information Technology Ecosystems 27

Partnerships and Strategic Engagements 30

References 31

Appendixes 32

Appendix A. Stocktaking, Future State, and Gaps 32

Appendix B. World Bank Group Main Guidelines on the Responsible Use of Artificial Intelligence 36

References 38

Tables 2

Table 1.1. Gaps to Be Addressed by the Artificial Intelligence Strategy 7

Table 2.1. Values and Guiding Principles of Embedding Artificial Intelligence in Evaluation Practice 9

Table 2.2. Maturity Levels 11

Table 4.1. Engagement Spectrum 30

Figures 2

Figure 1.1. Artificial Intelligence Strategy Framework 6

Figure 3.1. Scaling Artificial Intelligence Use 13

Figure 4.1. Artificial Intelligence Tool Ecosystem 29

Appendix Figures 36

Figure B.1. Artificial Intelligence Risk Management 36

Figure B.2. Artificial Intelligence Prohibited Practices 37