Executive summary i

Key findings ii

Recommendations iv

Table of contents vii

Abbreviations ix

List of figures, tables and boxes xi

List of figures xi

List of tables xii

List of boxes xiii

1. Introduction xiii

1.1. Background 1

1.2. Objective 5

1.3. Scope 5

1.4. Methodology 8

1.5. Structure 13

2. Analysis 14

2.1. Measuring effectiveness: uplift in attack chains 14

2.2. Measuring efficiency: time spent on attack chains 20

2.3. LLM interactions: prompts, refusals and role play 22

3. Discussion and recommendations 26

3.1. Limited evidence of uplift to complete an entire offensive cyber operation 27

3.2. Limited evidence of uplift among novices 27

3.3. More early-stage uplift versus late-stage uplift 27

3.4. Uplift impacted by AI model guardrails 28

3.5. Study limitations and other discussion 29

3.6. Recommendations 30

References 33