Acknowledgement of Country  2

Executive summary  7

1 Introduction  8

1.1 Motivation  8

1.2 Purpose and audience  9

1.3 Scope  9

2 Foundations  11

2.1 LLM-based agents  11

2.2 Multi-agent systems  14

2.3 Reasoning about risk and controls in multi-agent systems  15

2.4 The agent ecosystem today  17

3 Governing multi-agent deployments  20

3.1 Why multi-agent governance is different  20

3.2 Four governance practices under stress  21

3.3 Governance tiers  22

3.4 Structure of this report  24

4 Agents under singular governance  25

4.1 Salient risk factors  26

4.2 Failures in agent interaction and controls  29

4.3 Failures in governance practices and controls  37

4.4 Open problems  44

5 Agents under federated governance  46

5.1 Salient risk factors  49

5.2 Failures in agent interaction and their controls  52

5.3 Failures in governance practices and controls  64

5.4 Open problems  70

6 Agents in open environments  71

Setting 1: Lock down the operating scope  72

Setting 2: General purpose agents participating in polycentric governance  73

6.1 Salient risk factors  76

Risks and controls for multi-agent systems

| industry.gov.au  6

6.2 Failures in agent interaction  77

6.3 Failures in governance practices and their controls  85

6.4 Open problems  89

7 Discussion  92

7.1 Selected cross-cutting themes  92

7.2 Trade-offs and tensions  93

7.3 Scope-adjacent settings  94

8 Conclusion  96

9 References  98

Appendix A: Glossary  105