Categories
Saved and Shared Stories from Michael Novakhov

Michael Novakhov – SharedNewsLinks℠: The Limits of Intuition: Army Intelligence Should Embrace Analytic Tradecraft Standards



Michael_Novakhov
shared this story
from War on the Rocks.

Army intelligence lags behind most of the intelligence community in one key respect. In 2007, the intelligence community established analytic tradecraft standards to provide a common framework for critical and creative thinking when conducting all-source intelligence analysis. The Army, however, has not adopted these standards, leading to an analytic workforce that is less proficient in applying tradecraft than its civilian counterparts. The root cause of this failure is a common misconception that analytic tradecraft standards are applicable only at the strategic level. At higher levels, the value of analytic tradecraft is more obvious because intelligence analysts are focused on supporting senior decision-makers — primarily combatant commanders and civilian policymakers — who often approach problems deliberately and address longer-term issues. Some will argue that analytic tradecraft is incompatible with the rapid pace of operations at lower echelons, in which commanders are focused on preparing for local battles and operations. This misconception is preventing the Army from recognizing an opportunity to help its intelligence analysts overcome the limits of intuition and ensure rigor in their analytic products.

Analytic tradecraft standards are based on universal principles for critical and creative thinking that can be applied in any environment. One of the primary goals of analytic tradecraft is to mitigate cognitive biases, an inherent weakness of intuition that occurs when pre-existing mental models influence how people perceive their environment. Cognitive biases can cause analysts to view complex problems through the narrow filter of their individual experiences and subconsciously ignore ideas that are inconsistent with pre-existing mental models. For example, a 2018 Army War College report concluded that an “analytical bias toward a familiar, hierarchical, Soviet-style enemy” hindered the United States from recognizing a growing insurgency in Iraq after the overthrow of Saddam Hussein in 2003. Cognitive biases are a universal problem because they can occur in any situation involving human thought or perception. In the realm of military intelligence, they are particularly prevalent at lower echelons because analysts tend to default to familiar frameworks when faced with the pressures of a battlefield. Thus, analytic tradecraft standards have wide utility beyond the strategic-level organizations — such as national intelligence agencies and combatant commands — that have already implemented them.

It is important to distinguish between analytic tradecraft standards (which have widespread value) and the manner in which they are applied and enforced (which varies by organization). The failure to make this distinction is behind the tendency to view analytic tradecraft as being relevant only at the strategic level. The intelligence community’s analytic tradecraft standards are broad and reflect universal principles that can be used even outside of an intelligence context. However, these standards have gained acceptance only among national intelligence agencies and the military’s combatant commands, reinforcing the misconception that analytic tradecraft is useful only at the strategic level. Furthermore, many of these organizations employ deliberate processes to apply and enforce standards, such as structured analytic techniques and layered reviews of analytic products. As a result, some may incorrectly conclude that the Army must employ these same deliberate processes if adopting tradecraft standards. This misconception exists because of a failure to visualize the different ways that analytic tradecraft can be tailored and employed at all levels throughout the Army.

Summary of the Intelligence Community’s Analytic Tradecraft Standards

Standard 1: Sourcing

Provide at least basic descriptions of the sources of information that support each analytic conclusion (e.g., “according to a senior foreign government official with firsthand access”); identify key sources that are most important for each analytic conclusion; be transparent about the quality of available sources.

Standard 2: Uncertainty

Explain the level of uncertainty for each analytic conclusion; use approved terms (e.g., “likely” or “very likely”) to express the likelihood that the assessed event or outcome will occur; express a confidence level based on the quality of the overall analytic argument.

Standard 3: Distinctions

Distinguish between underlying evidence, assumptions, and analytic conclusions; consistently use “signaling language” to alert decision-makers as to the type of information they are reading; be transparent about key assumptions that underpin each analytic conclusion.

Standard 4: Alternatives

Identify at least one plausible alternative for every major analytic conclusion to mitigate surprise or alert decision-makers of low-probability/high-impact situations; identify indicators that, if detected, would alert decision-makers that an alternative conclusion is becoming more likely.

Standard 5: Relevance

Ensure analytic products are tailored to the needs of decision-makers; provide deeper insights to decision-makers by addressing second- and third-order impacts; conduct opportunity analysis as appropriate.

Standard 6: Argumentation

Prominently present a main analytic conclusion up front; subordinate analytic conclusions should support the main conclusion; skillfully combine evidence, logic, assumptions, and information gaps to support analytic conclusions.

Standard 7: Analytic Line

Be transparent about how analytic conclusions are consistent with or different from previously published analysis; alert decision-makers if there are significant analytic differences between two or more intelligence organizations.

Standard 8: Accuracy

Ensure clarity of message in all analytic products; do not avoid difficult analytic conclusions in order to minimize the risk of being wrong; the Defense Intelligence Agency further requires its analysts to express absolute, rather than relative, probabilities (e.g., “likely” instead of “more likely”); the agency also requires its analysts to assess events, actions, or behaviors instead of mental states or beliefs.

Standard 9: Visualization

Use visual information to clarify, complement, or enhance the presentation of analysis.

Visualizing Analytic Tradecraft in the Field

What follows are eight hypothetical scenarios that illustrate the benefits of analytic tradecraft standards at all levels, and the consequences of not having such standards.

Scenario 1: Defending an analytic argument

An Army battalion commander disagrees with the unit intelligence officer’s assessment that the enemy main attack will occur in a particular area and asks, “How did you arrive at your analytic conclusion?” The intelligence officer’s familiarity with tradecraft standards makes him well postured to explain the analytic argument to the commander. There are multiple components of an argument: analytic conclusions, evidence (or raw reporting from intelligence collectors), logic that links evidence to analytic conclusions, assumptions, and
information gaps. The intelligence officer knows how to examine these components in a holistic manner to mitigate cognitive biases and logical fallacies. More importantly, the intelligence officer understands how the commander likes to receive information and provides a concise explanation of the reasoning behind the original assessment. The Army’s adoption of a common framework for critical and creative thinking will ensure that intelligence officers across the force are not relying purely on their individual intuition when having these types of conversations.

Scenario 2: Common framework for analytic collaboration in the Army

The commander for a deployed Army brigade combat team asks the intelligence staff for a deep-dive briefing on an emerging issue. It is already 7:00 p.m. and the commander wants the briefing at 6:00 a.m. tomorrow morning. The brigade intelligence staff reviews assessments written by subordinate units. Many of these assessments contain declarative statements without common signaling language to distinguish raw intelligence reporting from an analyst’s assumptions or application of judgement. For example, one subordinate unit writes that “the enemy will be unable to operate its tanks for the next 48 hours due to fuel shortages.” Brigade analysts cannot determine whether this sentence is an assumption, raw reporting from a single source, or an analytic conclusion based on multiple sources, making it difficult to evaluate its importance in the overall intelligence picture. Colin Powell’s famous adage comes to mind: “Tell me what you know. Tell me what you don’t know. Tell me what you think. Always distinguish which is which.” This problem grows exponentially as this brigade collaborates with other units.

Scenario 3: Graphically displaying enemy courses of action

Intelligence analysts in an Army brigade combat team are creating a situation template, a graphical depiction of an enemy’s most likely course of action on a battlefield. The brigade’s analysts realize that tradecraft standard #1 (sourcing) requires them to describe the sources used in their products. However, it may not be possible to describe every source used in the situation template. As a result, the analysts prioritize and describe only the three most important sources. Additionally, the brigade’s analysts adhere to tradecraft standard #6 (argumentation) by clearly identifying the natural hierarchy in their assessments. Their main analytic conclusion is that the enemy will likely conduct a mobile defense. Analysts also include two subordinate analytic conclusions that characterize the doctrinal components of a mobile defense: a fixing force that will prevent the opposing side from escaping the area and a striking force that will destroy the opposition. In this example, analytic tradecraft enabled the underlying critical thinking process and helped analysts organize their thoughts before creating the situation template.

Scenario 4: Being open to alternative analytic conclusions

The commanding general of a deployed Army division asks the intelligence staff for an assessment on a complex issue. In their search for plausible answers, the division’s analysts rely on their intuition as experienced military intelligence professionals. While valuable, intuition also has pitfalls that must be acknowledged and mitigated. Division analysts rely on their past experiences in countering insurgencies in Iraq and Afghanistan to help frame their thinking on the current problem, ignoring fundamental cultural and societal differences in each of these cases. This situation leads to cognitive biases in which intuition, over time, causes analysts to perceive patterns based only on what they are familiar with, instead of what is actually occurring. The division’s analysts fail to employ structured analytic techniques to determine plausible alternatives to their main assessment. Furthermore, they do not identify specific indicators that would alert the division commander that their analytic conclusion may be incorrect, missing an opportunity to provide a complete picture of the uncertainty inherent in the situation.

Scenario 5: Transparency about key assumptions

An Army corps preparing for large-scale combat operations is monitoring the adversary’s elite armored divisions. The adversary upgraded some of its tanks, but they were issued only to the elite divisions. Corps analysts make an assumption that this practice will continue. Collectors in the field detect unique signatures consistent with four upgraded tanks as part of a larger movement of forces by the adversary. As a result, corps analysts assess that the adversary is beginning to deploy one of its elite armored divisions into the area, alarming the commander. However, this assessment turns out to be false: The four upgraded tanks are part of a conventional division. The adversary recently began fielding small numbers of upgraded tanks in conventional divisions as infantry support. Corps analysts were not transparent with their logic and did not conduct periodic checks of key assumptions, missing opportunities to employ collectors to confirm or deny the initial assumption.

Scenario 6: Professional standards for intelligence analysis

An Army corps deploys overseas as the core of a joint task force in anticipation of armed conflict as tensions escalate with an adversary nation. The joint task force commander is predominately focused on operational matters, but there are significant national policy implications. Disagreements emerge between the joint task force and a vocal community of scholars and policy advocates over how to define the key issues of the crisis. The commander makes public statements that contradict the views of these scholars and policy advocates. The commander also reassures the American public that his or her intelligence analysts are better postured to understand the crisis. In order for this reassurance to have credibility, the Army must have uniform standards for critical and creative thinking for its intelligence analysts. The Army has its own analytic processes — such as intelligence preparation of the battlefield — but currently has no formal standards to ensure rigor as analysts navigate through these processes. Analysts are largely left to rely on their intuition.

Scenario 7: Interoperability with the intelligence community

The same Army corps from the previous scenario continues serving as the core of a deployed joint task force. What started out as a purely military problem for the joint task force has now become part of a tense national debate involving policymakers and Congress. Given the increased stakes, joint task force intelligence officers begin collaborating with senior analysts from national intelligence agencies. Joint task force analysts do not have prior experience with analytic tradecraft, making it difficult for them to adjudicate disagreements with their national-level counterparts. Many deployed analysts, for example, use anecdotes and assessments from lower-echelon commanders as building blocks in their analytic products. While valuable, battlefield anecdotes are not processed with the same rigor and standardization as raw reporting from national-level intelligence collectors. The lack of common standards for intelligence analysis makes it difficult to produce a unified view of the crisis across the entire chain of command.

Scenario 8: Analytic tradecraft standards during time-sensitive operations

An Army major is assigned as the senior intelligence officer (i.e., the S2) for a brigade combat team after a three-year tour at the Defense Intelligence Agency. The major intends to adopt some of the
agency’s processes and implement them throughout the brigade intelligence staff. The major specifically recalls how the agency applies and enforces tradecraft standards in time-sensitive and crisis environments. For example, agency analysts still adhere to tradecraft standards when producing the Defense Intelligence Digest Note, which is typically one paragraph and is designed to rapidly alert policymakers of new developments without assessing what will occur. Army analysts conduct similar tasks when they rapidly disseminate targeting or force-protection information to commanders. Additionally, the Defense Intelligence Agency relaxes its enforcement of some tradecraft standards when operating a crisis cell, but the underlying standards remain the same. Similarly, the major intends to adopt broad analytic tradecraft standards, while remaining flexible on how leaders apply and enforce these standards under time-sensitive circumstances.

The Way Ahead

The eight scenarios above illustrate how analytic tradecraft standards can improve all-source intelligence analysis at lower echelons just as much as they can at the national level. These standards have universal value because they are broad and allow each organization to apply and enforce them in a tailored manner. The prevailing view that tradecraft standards are applicable only at the strategic level is false. In fact, cognitive biases — one of the primary reasons for adopting analytic tradecraft standards — are arguably most prevalent at lower echelons. As analysts encounter unexpected and time-sensitive events on the battlefield, they must be guided by a common framework for critical and creative thinking that increases the chances of suppressing cognitive biases. Otherwise, their assessments will be undisciplined reactions to chaotic circumstances based solely on their individual intuition. The intelligence community’s nine analytic tradecraft standards provide a proven mechanism to ensure all-source intelligence analysis is conducted with a level of rigor that commanders expect and deserve.

Once these standards are adopted, the Army can immediately begin improving how it develops all-source intelligence analysts without significant disruptions to existing training programs. Tradecraft training does not require specialized equipment or facilities. Furthermore, the Army can leverage its organizational relationships throughout the intelligence community to solicit advice and expertise. The easiest way to initiate improvements to training would involve adjusting evaluation standards to be consistent with the intelligence community’s nine analytic tradecraft standards. For example, during mock briefings to commanders, instructors can ask trainees to explain the confidence level associated with their assessments and evaluate how well trainees understand the components of an analytic argument. Some tradecraft standards have direct parallels with Army doctrine and may already be employed as best practices at lower echelons. However, they are not formal standards that are routinely enforced across the Army, leading to inconsistent or incorrect application.

The Army has an opportunity to be a leader among the military services by being the first to implement analytic tradecraft standards and enhance collaboration with the rest of the intelligence community. As the Department of Defense inspector general reported in December 2018, uniformed analysts assigned to combatant commands are not as proficient in applying analytic tradecraft as their civilian counterparts. This disparity in tradecraft proficiency has exacerbated existing interoperability issues within the Department of Defense all-source analytic community. Army analysts are providing assessments to commanders without using the same framework for critical and creative thinking as their civilian counterparts at national intelligence agencies and combatant commands. This situation creates problems when decision-makers at different levels interact to address common national security issues, each operating based on intelligence assessments developed using different standards (or no standards in some cases). Each military service will require its own approach to implementing analytic tradecraft standards. The first hurdle to overcome is to ensure shared awareness of how the Army can benefit from implementing such standards.

Maj. James Kwoun is an active-duty Army officer currently assigned to the Defense Intelligence Agency. He has previously served in various Army and joint positions throughout the defense intelligence enterprise.

The views in this article are entirely the author’s and do not necessarily reflect the official policy or position of the U.S. Army, Defense Intelligence Agency, the Department of Defense, or the U.S. government.

Image: Cpl. Lucas Hopkins

Michael Novakhov – SharedNewsLinks℠