Writing an annotation schema for qualitative research involves translating your research aims and theoretical concepts into a clear, structured system for labelling and interpreting data. The process combines conceptual thinking, practical design, and iterative refinement to ensure that annotations are both meaningful and reliable. The following stages outline how to write one effectively: 1. Define your research purpose and scope 2. Ground the schema in theory 3. Identify an

An annotation schema for qualitative research is a structured framework that sets out how researchers label, categorise, and interpret data  such as text, audio, video, or images. Its purpose is to ensure that human annotations—whether thematic codes, discourse features, or behavioural observations—are systematic, interpretable, and reproducible across annotators and research contexts. A well-designed annotation schema begins with a clear conceptual framework that defines the

The Methodological Framework for Analysing Hostile Narratives The Differing Positions of Hate Speech Detection and Hostile Narrative Analysis Hate speech research generally adopts a victim-focused perspective, assessing harm through the eyes of those targeted by speech. While this approach rightly centres on protecting individuals and communities from abuse, it is also highly subjective: what one person experiences as harmful, another might interpret as satire, irony, or even

Hostile narrative analysis seeks to detect and explain how texts legitimise violence through the narratives people tell themselves. It is a human-centric approach that uses natural language processing (NLP) combined with a rigorous theoretical foundation in Peace Studies. It uses Johan Galtung’s theory of cultural violence by measuring a “Self–Other gradient”: the discursive process by which a speaker elevates the ingroup (“Self”) while debasing or devaluing an outgroup (“Oth

Hate speech detection aims to identify language that attacks, dehumanises, or incites hostility against protected groups (e.g. based on race, religion, gender, sexual orientation, disability) or individuals as members of those groups. It uses a combination of quantitative and qualitative methods, combining machine learning with human oversight. In practice, many systems operationalise broader “toxicity” taxonomies (hate, identity attacks, insults, threats, obscenity) to suppo

Sentiment analysis, also known as opinion mining or emotion AI, is a field that uses natural language processing (NLP) to detect and interpret people’s attitudes, emotions and stances in text. It's widely applied to various materials like customer reviews, survey responses, social media, and healthcare data for purposes ranging from marketing to customer service and clinical medicine.  The most common method detects whether a text expresses positive, negative or neutral senti

Personal Experience Narrative (PEN) Analysis is a qualitative methodology that offers profound insights into how individuals construct and rationalise their lived experiences. Originally defined by Labov and Waletzky (1997) as a "verbal technique for recapitulating experience, in particular, a technique of constructing narrative units which match the temporal sequence of that experience," PEN Analysis involves a deep examination of personal accounts, such as interviews, journ

The Turing Trap: Augmentation vs. Replacement Erik Brynjolfsson’s “The Turing Trap: The Promise & Peril of Human-Like Artificial Intelligence” (2022) highlights a critical challenge in AI development: the pursuit of creating AI that is indistinguishable from human intelligence can lead to economic and societal risks if misaligned with human needs. Brynjolfsson explains that AI research focuses on automation by replicating human-like intelligence in machines, as measured by th

Introduction John Searle’s Chinese Room  thought experiment (1980) highlights the promise and limitations of Natural Language Processing (NLP) in Defence and Security contexts, particularly regarding integrating qualitative approaches. In the experiment, Searle imagines himself inside a room, following a rulebook that allows him to manipulate Chinese symbols and produce appropriate responses without understanding their meaning. The responses may appear fluent to an external o

Introductions People are driven by narrative because narrative provides a way to rationalise the world. Hostile narratives tell stories as old as human history; hostile narratives are stories of heroes facing off against villains to save humanity. Analysing these narratives reveals powerful insights into how societies shape identities, beliefs, and actions. They act as interpretative frameworks through which individuals and groups understand lived experiences and the broader

Introduction The increasing integration of artificial intelligence (AI) into security and defence applications necessitates rigorous evaluation frameworks to ensure reliability, fairness, and interpretability. While quantitative benchmarks (e.g., accuracy, F1 score) provide useful performance indicators, they fail to capture the complexities of human interaction with AI. Capturing the complexities of human interaction with AI is particularly critical for human-centric applica

Introduction The increasing adoption of artificial intelligence (AI) within UK policing highlights a profound need for “human-centric applications” that generate insights about people. While not explicitly stated, multiple police strategy and policy documents currently under consultation or in practice implicitly recognise this need. The premise underpinning human-centric AI is that it should synthesise quantitative methods, which enable large-scale data processes driving pre

Introduction A series of MOD policy documents reflect s  a growing recognition that truly human-centric  Artificial Intelligence (AI) requires far more than well-curated quantitative data and powerful machine-learning algorithms. The requirement for more human-centred AI demands a deeper, more qualitative understanding of how humans think, decide, and work for intelligence analysis. Human-centred ways of working with AI in intelligence analysis  is the source document upon wh

The Internet and the World Wide Web have introduced a new frontier for Defence and Security, necessitating qualitative approaches to analyse online behaviours, narratives, and emerging threats effectively. Key requirements include: Digital Ethnography:  Analysts must immerse themselves in online communities, forums, and social media platforms to understand extremist networks, misinformation campaigns, and subcultures that influence radicalisation. Discourse and Sentiment Anal

The Mary's Room Thought Experiment What can an imaginary colour scientist called Mary teach us about applying AI to analysis and decision-making in Defence and Security? Frank Jackson’s thought experiment, commonly known as ‘Mary’s Room,’ is a philosophical thought experiment designed to explore the nature of human intelligence, particularly the difference between physical (explicit) knowledge and experiential (tacit) knowledge (Jackson, 1982). In this thought experiment, Mar

Introduction In the pursuit of gaining intelligence for decision-making, analysts and decision-makers frequently face a critical - but often unobvious - methodological choice: should they rely on qualitative or quantitative research, or indeed a combination of both? This debate extends across sectors, from defence and security to business analytics and social sciences. While quantitative methods offer measurable and replicable insights at scale, qualitative approaches provide

Artificial Intelligence (AI) holds extraordinary potential for enhancing operational capabilities across Defence and Security. Yet, for many, AI remains entangled in hype, misunderstood definitions, and questions over what it can realistically achieve. In response, this explainer aims to clarify three key paradigms of AI – Strong AI, Narrow AI, and Frontier AI – and reflect on the governance needed to ensure their responsible, effective use in Defence and Security contexts. T

The Ministry of Defence (MOD) is embracing the transformative potential of Artificial Intelligence (AI) to enhance national security and economic resilience. Realising this potential requires more than cutting-edge algorithms and data; it demands a human-centric approach to the development and evaluation of AI systems. Human-centric AI involves designing technology around the values, behaviours, and lived experiences of people. As is now explained, achieving human-centricity

The UK government is creating extraordinary opportunities to accelerate the adoption of artificial intelligence (AI) across public services. As the current Government’s AI action plan states, “ AI should become core to how we think about delivering services, transforming citizens’ experiences, and improving productivity ”. Fully realising this potential, however, requires revisiting a long-standing debate: the role of quantitative versus qualitative research in understanding

Defence is at an inflexion point where its continued relevance depends on its ability to adopt Artificial Intelligence. In comments about the revolutionary impact of uncrewed systems in the Ukraine War, the Parliamentary Under-Secretary of State (Minister for the Armed Forces), Al Carns MP, has said, we are in “ a machine gun moment for the Army, a submarine moment for the Navy and a jet engine moment for the Air Force ”. Central to this revolution with uncrewed systems is a