Tell Me What You Don’t Know: Large Language Models and the Pathologies of Intelligence Analysis

Discussing AI, Automated Systems, and the Future of War Seminar Series

This seminar seeks to offer a warning. Prompted by the likely increase in the use of artificial intelligence (AI) in intelligence analysis, it will raise grave concerns about the prospect of relying on large language models (LLMs), including in high-stakes contexts such as the state-level resort-to-force decision-making that is the subject of this broader Defence-funded project. It will begin by identifying the twin informational pathologies that intelligence analysis is subject to by its very nature: information scarcity and epistemic scarcity. It will then go on to argue that the use of LLMs would compound these pathologies, attributing this risk to the nature of the international information landscape, especially the rise of private actors in data markets and the changed intelligence environment in the years following September 11.

The seminar will conclude by making recommendations for possible responses to the informational risks engendered by the use of LLMs in intelligence analysis, especially in high-stakes contexts.

About the speaker
Sarah Logan is a Senior Lecturer in the Department of International Relations in the Coral Bell School of Asia Pacific Affairs at The Australian National University. Prior to joining academia, she worked in intelligence analysis for the Australian government. Her research interests focus on the impact of information technology on international politics. Her current projects include a study of the relationship between information and political authority and an assessment of the impact of big tech on state capacities to exercise control over political violence. Her sole-authored monograph, Hold Your Friends Close: Countering Radicalisation in Britain and America, was published by Oxford University Press in 2024.

About the chair
Toni Erskine is a Professor of International Politics in the Coral Bell School of Asia Pacific Affairs at The Australian National University (ANU) and Associate Fellow of the Leverhulme Centre for the Future of Intelligence at Cambridge University. She is also Chief Investigator of the ‘Anticipating the Future of War: AI, Automated Systems, and Resort-to-Force Decision Making’ Research Project, funded by the Australian Government, Department of Defence, and recipient of the International Studies Association’s 2024-25 Distinguished Scholar Award in International Ethics.

This seminar series is part of a two-year (2023-2025) research project on Anticipating the Future of War: AI, Automated Systems, and Resort-to-Force Decision Making, generously funded by the Australian Department of Defence and led by Professor Toni Erskine from the Coral Bell School of Asia Pacific Affairs.