Abstract  2

Acknowledgements  3

Executive summary  6

1 Introduction  8

General AI governance frameworks  9

Need for operational level guidance in experimenting with generative AI  12

2 Overview of existing guidance for experimenting with GenAI  14

Existing guidelines for experimenting with generative AI: Common themes and divergence  14

3 Prioritisation, monitoring and evaluation of generative AI experimentation  25

Prioritisation  25

Monitoring and evaluation  26

Developing criteria for prioritisation, monitoring and evaluation  26

4 Factors and conditions for successful experimentation and evaluation  37

(Non-)Material resources and assets  37

Lessons from the experimental practice  43

5 The road towards more systemic experimentation  51

1) Establish public trust in GenAI experiments  52

2) Provide AI literacy training and upskilling opportunities to civil servants  53

3) Prevent fragmentation of projects and AI systems  53

4) Prioritise investments in shared tools, data and secure testing mechanisms  53

5) Limit and mitigate risks  54

6) Invest in experimental AI research  54

7) Make AI experimentation frameworks future-fit  55

8) Ensure efficiency through prioritisation  55

9) Plan experiments and their evaluation early  55

10) Create environments that encourage experimentation  56

References  57