What does the Mary's Room Thought Experiment teach us about applying Artificial Intelligence to intelligence analysis and decision-making?

The Mary's Room Thought Experiment

What can an imaginary colour scientist called Mary teach us about applying AI to analysis and decision-making in Defence and Security? Frank Jackson’s thought experiment, commonly known as ‘Mary’s Room,’ is a philosophical thought experiment designed to explore the nature of human intelligence, particularly the difference between physical (explicit) knowledge and experiential (tacit) knowledge (Jackson, 1982). In this thought experiment, Mary is a scientist who has spent her entire life in a black-and-white room. While in the room, she gains all possible knowledge about the science of colour without ever having experienced colour firsthand. The thought experiment asks: what does Mary learn about colour when she steps out of the room into a colourful world for the first time? In short, she gains a new form of knowledge, what it is like to experience colour.

Mary’s Room has significant implications for Strong AI, which aims to achieve human-like intelligence, known as artificial general intelligence (AGI). The idea of Strong AI arose from Alan Turing’s Turing Test (1950), which involves a human evaluator interacting with a machine and another human without knowing which is which. If the evaluator cannot reliably distinguish between the human and the machine, then the machine is considered to have passed the Turing Test because it demonstrates human-like intelligence. The goal of Strong AI has been to develop an AI system that can replicate human intelligence.

A screenshot from Eliza

How and whether AGI is achievable has since been the subject of many philosophical papers and science fiction. A practical example of assessing a machine’s ability to exhibit AGI is ELIZA, created by Joseph Weizenbaum in 1966. Eliza was an early natural language processing program designed to simulate conversation, most notably through its DOCTOR script, which mimicked a Rogerian psychotherapist (Weizenbaum, 1966). Contrary to the sophistication of modern AI machines, the DOCTOR script relied upon a simple set of rules to generate an output from user inputs.

While ELIZA did not strictly pass the Turing Test, it demonstrated a key milestone in AI by simulating human intelligence (Turing, 1950). Eliza’s reflective conversational style encouraged people to project their feelings onto the system, leading users to develop strong emotional attachments, even though they knew it was a machine. This phenomenon, known as the Eliza Effect, highlights the human tendency to form emotional connections with machines even when they lack genuine empathy or comprehension (Weizenbaum, 1976; Turkle, 1984). In effect, Eliza inverted the Turing test because humans treated the machine as human, even though they knew it was a machine.

In modern AI governance and development, many UK government policy documents cite “human intelligence” in their definitions of AI, which implies the goal of creating AGI.

1. Covenant for Using Artificial Intelligence (AI) in Policing

“Artificial intelligence (AI) refers to a machine that learns, generalises, or infers meaning from input, thereby reproducing or surpassing human performance. The term AI can also be used loosely to describe a machine’s ability to perform repetitive tasks without guidance.”

2. National AI Strategy

“In general, the following definition is sufficient for our purposes: ‘Machines that perform tasks normally requiring human intelligence, especially when the machines learn from data how to do those tasks.’”

3. Policing and AI (Police Foundation)

“There is currently no universally agreed definition of AI. The majority of definitions of AI refer to ‘computers or machines that can perform tasks that require intelligence, especially when the machines learn from data how to do those tasks.’”

4. Human-centred ways of working with AI in Intelligence Analysis (DSTL)

“We’ll say that AI consists of theories, tools, technologies and techniques developed to allow computer systems to perform tasks normally requiring human or biological intelligence.

5. JSP 936 (Part 1)

“For the purposes of this JSP, the MOD characterisation is: ‘AI is a general purpose technology that may enable machines to perform tasks normally requiring human or biological intelligence, especially when the machines learn from data how to do those tasks.’”

6. Defence Artificial Intelligence Strategy

“Defence understands AI as a family of general-purpose technologies, any of which may enable machines to perform tasks normally requiring human or biological intelligence, especially when the machines learn from data how to do those tasks.”

7. Ambitious, Safe and Responsible (AI Policy Statement, 2022)

“Defence understands Artificial Intelligence (AI) as a family of general-purpose technologies, any of which may enable machines to perform tasks normally requiring human or biological intelligence, especially when the machines learn from data how to do those tasks.”

Yet, what does "human intelligence" really mean for these definitions? After all, Mary’s intelligence combines explicit knowledge of colour with the emotional experience triggered by experiencing colour for the first time. This distinction between explicit knowledge (data) and tacit knowledge (experience) is critical to understanding how human intelligence differs from AI.

The Interplay of Tacit and Explicit Knowledge in the Context of Research

Mary’s Room highlights the distinction and interplay of explicit and tacit knowledge. Explicit knowledge refers to information that can be formally articulated, codified, and transferred, such as scientific data, statistical models, and mathematical equations (Polanyi, 1966). Tacit knowledge, on the other hand, encompasses insights gained through experience, intuition, and contextual understanding—forms of knowledge that are difficult to quantify or express in formal terms (Nonaka & Takeuchi, 1995).

In Defence and Security, tacit knowledge is critical in intelligence analysis, decision-making, and strategic operations. Explicit knowledge is gained from deploying intelligence assets that gather data in response to intelligence requirements. While explicit knowledge helps analysts quantify risks and identify patterns, tacit knowledge allows them to interpret ambiguous or incomplete information, make nuanced professional judgments, and understand adversarial intent in complex, unpredictable environments. In the interplay of each, tacit knowledge may also challenge professional knowledge and intuition to gain new insights.

Application to the Qualitative vs. Quantitative Debate

The interplay between tacit and explicit knowledge mirrors the broader debate between qualitative and quantitative research methodologies. Quantitative research is rooted in explicit knowledge, relying on statistical models, measurable variables, and structured data to provide objective insights (Creswell, 2014). This approach is valuable for identifying trends, testing hypotheses, and ensuring replicability in research.

Qualitative research captures tacit knowledge by focusing on lived experiences, social contexts, and interpretative analysis (Denzin & Lincoln, 2018). Just as Mary’s firsthand experience of colour reveals something beyond the data she previously knew, qualitative research uncovers insights into human behaviour and motivations, which are vital elements of intelligence work. For intelligence analysis, qualitative insights might include understanding enemy morale, interpreting strategic narratives, or assessing the socio-political climate—factors that are difficult to quantify.

What can Mary’s Room Teach Us About Intelligence Analysis in Defence and Security?

Mary’s Room provides valuable lessons for intelligence analysis in Defence and Security, where analysts seek to understand people and populations from their intelligence cells. Intelligence analysts work to understand adversaries and anticipate threats, often operating in high-stakes environments. For example, military analysts generate recommendations on overcoming adversaries using quantifiable metrics of military strength. Police analysts use data to identify criminal activity and forecast criminal behaviour. Extending Mary’s Room to intelligence analysis begs the question: if intelligence analysts can gather all possible information about adversaries and populations, what new understanding could they gain through experiential knowledge?

The Russian-Ukraine war provides a real-world answer to this question and insight into the problem of not considering the tacit knowledge of human experiences that drive real-world events. The infographic below compares Ukraine’s and Russia’s fighting power before the February 2022 invasion. The Russian military capability is vastly superior, leading Russia’s President, Vladamir Putin, to reportedly boast he would win the war in two weeks. An AI simulation of the war using these metrics would likely give credence to his boast.

What Putin’s boast and an AI model may not account for, however, is the Ukrainian’s exceptional will to fight and the devastatingly low morale of Russian troops. Against all expectations, the Ukrainian military has fought the Russians for three years at the time of writing. Analysts stepping out of their black-and-white intelligence cells to experience the colour of the battlefield would have learned about their opponent's morale, which has been a critical factor in this conflict. Yet, were they able to walk among their opponent, how would they quantify morale in the model shown above? While these intangible factors of morale are not easily quantified, they have been pivotal in shaping the war’s outcome. The unquantifiable nature of morale in a military force underscores the inherent complexity of human emotions and the need to integrate qualitative insights from an analysts tacit knowledge.

Conclusion

The Mary’s Room thought experiment offers a valuable framework for understanding the interplay between explicit and tacit knowledge when applying AI to intelligence analysis. While AI can process vast amounts of explicit data to generate insights, it cannot replicate the experiential knowledge analysts gain through direct engagement with complex, real-world scenarios. In Defence and Security, integrating tacit and explicit knowledge is essential for understanding the intelligence picture, from predicting enemy behaviour to interpreting the nuances of criminality. The real challenge lies not just in collecting data, but in understanding and applying the tacit knowledge that brings meaning to that data. The pursuit of AGI may one day lead to machines capable of accounting for the more tacit aspects of human intelligence. Until then, human expertise and experience will remain irreplaceable in areas like intelligence analysis and strategic decision-making.

References

• Bryman, A. (2015). Social Research Methods. Oxford University Press.

• Creswell, J. W. (2014). Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. SAGE Publications.

• Denzin, N. K., & Lincoln, Y. S. (2018). The SAGE Handbook of Qualitative Research. SAGE Publications.

• Patton, M. Q. (2015). Qualitative Research & Evaluation Methods. SAGE Publications.

• Tracy, S. J. (2020). Qualitative Research Methods: Collecting Evidence, Crafting Analysis, Communicating Impact. Wiley

• Turing, A. M. (1950). Computing Machinery and Intelligence. Mind, 59(236), 433–460.

• Turkle, S. (1984). The Second Self: Computers and the Human Spirit. Simon & Schuster.

• Weizenbaum, J. (1966). ELIZA – A Computer Program For the Study of Natural Language Communication Between Man and Machine. Communications of the ACM, 9(1), 36-45.

• Weizenbaum, J. (1976). Computer Power and Human Reason: From Judgment to Calculation. W.H. Freeman.

• Covenant for Using Artificial Intelligence (AI) in Policing, Section 1.1 (National Police Chiefs’ Council, Version 1.1).

• National AI Strategy (September 2021), p.16.

• Policing and AI (Police Foundation), Section 2.2 (February 2025).

• Human-centred ways of working with AI in intelligence analysis (Dstl), Foreword (July 2023).

• JSP 936 Part 1 (Directive), Section 1 (November 2024).

• Defence Artificial Intelligence Strategy, p.4 (June 2022).

• Ambitious, Safe and Responsible, p.1 (June 2022).