Intelligence Report
Bayes' Theorem in the Korean War
Canrnt 0
8 No.
Foreword
This reportest to determine the applicability of probability theory in intelligence forecasting. The test simulated0 on the prospect of Chinese Communist intervention in the Korean War.
Testing of the mathematical model will continue, vitb some shift of effort from replications of past history to "live-mode" processing of current evidence
Replication has the advantageeady-made sconario complete to denouement; test participants do not mark time waiting for events to happen. However, it Is difficult to replay the past in full insulation from the complicating factor of hindsight knowledge.
of the
In any case, one approach does not excludework along other lines, and suggestions for future lines of investigation would be appreciated. Comments may be addressedirectorate of Intelligence
lanning Staff,
SECRET
OECHKT
Co ti tents
The Substitute for
The Probability
Korea
The
Basis for the Starting
The Starting
The New
The Revised
Technical Notes
Partitioning
and
Figure 1Probability of Chinese Communist
Intervention in Korea: 0
ii
3ECRET
Summary
Mathematical processing of evidence would have supported an intelligence estimatedds in0 that the Chinoso were about to intervene in Koreaarge scale. The UN Command in Korea launched its "home by Christmas" offensive on Chinoso Communist forces in unexpected strength smashed the offonsive before it got really under way.
dds were reachedathematical simulation8 that required analyst appraisals of0 evidence. On tbe basis of these appraisals the mathematical model applied Bayes' Theorem from probability theory to rate the comparative merits of three hypotheses about Peking'sIntervention, limited intervention, and nonintervention
The effort was made to appraise the accumulating evidence through the eyes of the analyst To this end,8 appraisals were checked with the written record of intelligence thinking
This kind of mental projection backward0 notwithstanding, lt is impossible to be sure that8 simulation was entirely free from the advantage of retrospective knowledge. Tlie indication,is that the massive intervention hypothesis would have scared high, whether or not the limitedhypothesis scored higher. Tho probability figures given by Bayes' Theorem would at the least havea virtual directive for precautionary measures in battlefield strategy.
I. BAYESIAN METHOD
Onorth Korean troops crossedh Parallel to launch their surprise invasion of South Korea. US forces were quickly committed to the fighting. Would the Communists react to the USby escalating their support to North Korea?
The logic for Communist introduction of non-Korean combat troops did not seem especially compelling in the first few weeks of the war. Three days aftor they crossed the Parallel, the North Koreans were in Seoul. The landingN of the first American troops did not stop theI-.riog advance. By the end of July, the Americans and South Koreans had their backs against the aaa,in precarious defense of their Pusan perimeter at the baso of the peninsula.
The chances of early North Korean victory diminished after the rapid build-up of US forces through the port of Pusan. Augustonth of military stalemate; the North Koreans could not break through the perimeter.
Oneptember, American troops made the daring amphibious assault at Inchon in the North Korean rear. The expulsion of the North Koreans from the south was in sight, and American advance acrossh Parallel in prospect.
The turn in tbe tide raised anew the question of Soviet and Chinese reaction. What if any limitation was there on the extent of US military success in Korea before tho Chinese or Russians replied in kind? The buildup of Chinese forces in Manchuria made for especially sharp intelligence focus on Peking's intentions.
The Substitute for Certainty
A mathematical simulation0 intelligence analysis on prospective Chinese intervention wasin The evidence used in thewas only that which was available
Mathematical processing does not concludo with yes-or-no answers. The muthomatics does not eliminate uncertainty. Itasis for rational not
infallible decision. The rational decision in situations of uncertainty is for some combination of insurance against loss and gamble for gain. Tbe preciseis compoundedisk calculus which gives due weight to estimated probabilities about the future. The required intelligence support for rationalis to be read not as prophecy buteighing of the odds, muchambler weighs them bofore placing his beta.
The Korean experience ls instructive. Post-mortem criticisms took issue with the decision to advance American forcos full-speed to the Yalu. Defending briefs, on the other hand, pointed to tho intelligence consensus that the ChineHo wore not about to intervene in force.
Intelligence, for its part, might havo quoted from its estimates to show that it had been far from ruling out the possibility of Chinese intervention in force.
Yet the language of the intelligence estimates may well have conveyed difforont shades of meaning to different readers. Suppose intelligence hadits language of wordsanguage of numbers. Suppose it had said that the chances of large-scale Chinese intervention were less than even, but that the probability nevertheless ran as high asercent orercent. Tho matter is by no means certain, but thisrobability estimate could well havo inclined the American command to another combination of insurance and gamble, not to the chosen strategy of swift advance to the China border by military units well in advance of their main bodies and vulnerable to entrapment.
Intelligence could offerumerical judgment with present methods of analysis. 8 simulation, however,odel for analysis thatumerical judgment grounded in probability mathematics.
The problem-solving model in the simulation did bettor in fact than give an estimate of significant though less than even chance of Chinese intervention. Incorporating Bnyes' Theorem from probability theory, the model came updds in favor of large-scale intervention.
The endeavor was made to incorporate ail the biases0 opinion into the simulation. Did8 analysis free Itself entirely from hindsight knowledge? It is impossible to be sure, and lt therefore remains problematic that the mathematical model would have performed as well0 as it did Theis only that the probability of Chinesewould have come out high enough, at the least, to constitute an injuction for careful hedge against the contingency.
The Probability Scale
Intelligence estimation under Bayesian method beginset of hypotheses. tarting opinion is offered about the merits of each hypothesis. This opinion is expressed as tho odds or probabilities, asertain date, that the particular hypothesis is the true one. The starting opinion could be taken from the last National Intelligence Estimate on tho subject. If so, the date of tho starting odds is the dato of the last NIE.
The analysis draws no further, in principle, on anyone's opinion about the hypotheses. The analysis is instead confined to examination of tho evidence received after tho starting date. Two judgments are made about this evidence.
One judgment rates the reliability of tbesource or technical sensor. The reliability rating is necessary only when the accuracy of Incoming reports
is in question
second judgment about the evidonco rates the diagnostic value of the event reported. This diagnostic Judgment is called the likelihood ratio. ountry be more likely or less likely to follow its current propaganda lino if it were going to make war than if it were going to keep the peace? The likelihood ratio states Just how much more likely or less likely.
The analyst can approach his estimation ofratio In either of two ways. One isapproachevent
reported, the analyst estimates, ib say twice as likely
SEGftET
to happon if the war hypothesis is true than if the peace hypothesis is true.
In the second, the Indirect approach, the analyst addresses himself explicitly to two probabilities. The event reported, be judges, is one that will almost certainly happen whenever the war hypothesis ls true; the event otherwise has only an even-money chance of happening. Do each of these two verbal probability propositionseasonable numerical equivalent? If so, tbe first divided by tbe second is theratio.
The analyst in the Korea simulation used both direct and indirect approaches. He turned the matter over in his mind one way, then the other, until he came to what he felt toair judgment. He drew on the following table of equivalencies for assistance in expressing his judgments numerically.
Form
Equivalent
sure to, no question about almost certainly very probably probably
on balance, somewhat more likely than not
like as not, even money
somewhat less than even chance
probably not
very probably not
almost certainly not
certainly not, impossible
verbal-numerical equivalencies could be defendod, for there is no common standard of word usngo. Some statisticians would think of "almost certainly"orm best reserved for estimates which the analyst expected to see substantiated inases or more outundred.
Whether estimated directly or by way of its component
probabilities, the likelihood ratio in intelligence
analysisersonal or group opinion. The numbers
in Bayesian analysis do not free intelligence from
subjective Judgment. They help channel subjective
judgment to appraisals of evidence, letting estimative
conclusions about tbe alternative hypotheses follow from the mathematical logic.
OKCRliT
Laboratory experiments at the University of Michigan and other centers suggest that this approach hasover traditional method. Whon ho uses traditional method, the analyst makes one subjective leap, so to speak, to judgment about hypotheses from consideration of the evidence as an aggregate. But the analyst, these experiments suggest, has egregious Imperfectionsogical aggregator. He does betterayeslan processor (machine or human) takes his opinions about single items of evidence and then tells him whatconclusion is consistent with those opinions.
The advantages and disadvantages of the Bayesian approach in intelligence analysis are still matters for research. The analyst using the Bayesian approach has his problems in estimating the probability of an event, giving the hypothesis. The analyst using traditional method has his difficulties estimating the probability that an hypothesis is true, given the events reported in his body of evidence. Perhaps the most to be said for Bayesian method in intelligence at this time is that it is one way to evaluate evidence and roach conclusions. Like the different valid ways opentudent forroblem in arithmetic, Bayesian method can be usedross-check on traditional method. When the two methods give disparate results. Intelligence will want to see if the different analyses are reconcilable. On occasion, intelligence may be moved toward conclusions it would not otherwise entertain.
II. THE KOREA SIMUJATION
The analysis is assumed to begin oneptember, when the Inchon landing turned the tables on thein Korea and occasioned renewed speculation about Peking's intentions.
The Hypotheses
The first task in the exorcise Is the formulation of hypotheses. The main concern0 was not simply whether or not Peking would get Involved; tbe transfer of ethnic Koreans from Chinese Communist military units to the Korean armed forces had already made further military cooperation between Peking and Pyongyang seem logical enough. The key question was whether Peking would Intervene with forces largo enough toecisive weight in the military balance.
thereforo,
which weighed only two hypothoses, the Korea simulation begins with three:
Chinese Communist troops will cross the Yalu in largo numbers (more) to engage in full combat on the side of the North Koreans.
Communist China will intervene but not with large numbers of combat troops.
Communist China will not Intervene with its own combat troops in tho Korean War.
The objective is to calculate the probability that each one of these hypotheses Is true. To this ond, the evidence received fromeptember on is to be examined with an eye cocked especially for what are calledin strategic warning parlance. The indications may bo positive or negative. That is to say, they may be signals suggestive of imminent Chinese intervention, or they may suggest Communist policy decisions against intervention.
Basis for tho Starting Opinion
The task following the formulation of the three hypotheses is to estimate starting probabilities. These probabilities express the analyst's opinion about the hypotheses before his item-by-item consideration of the later evidence. This opinion is based on the impression made by evidence before IS September and by the seeming logic of the situation. The following specific considerations enter into the analyst'sof starting probabilities.
The Chinese Communist government in Peking is lessear old. Communist control over the mainland is not yet consolidated; actions are still in progress against anti-Communist guerrilla forces.
Domestic Chinese policy gives great emphasis to economic recovery. Industrial production, agricultural output, and other economic indexes are far below their "pre-liberation" peaks.
Koreaost peripheral feature of Chinese Communist foreign policy before the summerhinese propaganda made no mention of Korea but rather stressed the necessity of "liberating" Taiwan and Tibet. The Chinese Communists did not post their first ambassador to Pyongyang until
Inao Tse-tung and the Central Committee of the Chinese Communist Party issued an order for partial demobilization of tho armed forces. The order instructed the army to "demobilize part of its troops
but only on condition that sufficient forces to liberate Taiwan and Tibet are guaranteed as well as sufficient forces to consolidate the national defense and suppress the."
Soviet Union is treaty bound to come to
the aid of Communist China if Peking is attacked by Japanountry allied with Japan. The US is operating from bases in Japan. Chinese intervention would therefore bring the USSR closer to the brink of military confrontation with the US.
6. Peking has released to the North Korean army many troops of Korean descent who had beeni serving with the Chinese Communist forces.
borders on China. Peking'sUS policy in Korea were deepened byof President Truman's response to
the North Korean attack. The President directed the Seventh Fleet to intordlct the Taiwan Strait "to prevent Communist attacks on the island and Nationalist forays against the Peking fumed at this US enlargement of the military theater as "armed aggression on Chinese territory" requiring the Chinese people to "act with firm counterblows."
hour visit to Taiwan on
10 may also have suggested to Peking the danger of US spillover from Korea into other spheres of Chinese interest. General MacArthur again rubbed one of Peking's most sensitive nerves onugust, when he called Taiwan part of the island chain from which the US could "dominate with air power every Asiatic port from Vladivostok to Singapore."
are recent intimations in Chinesethat Peking sees its vitallinked to the Communist position Onugust, for example, thedeclared that the US action inthreatens the security ofis impossible to solve the Koreanthe participation of North Korea's defensedefense."
10. Onugust, the US representative in the United Nations strongly intimated the US interest in freeing the whole of the Korean peninsula from Communist control. "The Security Council has set as its first objective the end of the breach of the peace. This objective must be pursued inanner that no opportunity is provided
OEGRET
for another attempt at Tho United Nations must see that the people of Koroa attain complete individual and political Shallart of this country be assured this freedom? hink The United Nations ought to have free and unhampered access to and full freedom to travel within all
parts of We are waiting and while
we wait the strength of the United Nations increases."
Peking responded to this statementable to the US onugust: "Korea is China's neighbor. The Chinese people cannot but bo concerned about solution of the Koroan It must and can be settled peacefully." Two days later, the SovietIn the UN warned: "Any continuation of the Korean War will lead inevitablyidening of the
While Poking has ranted at the US design to turn Koreagangway of aggression" against China, there is no intimation in current propaganda that events aro near the point of requiring Chinese military intervention. Peking foresees no early victory for Pyongyang but expresses faith in North Korean self-sufficiency. "there is no doubt that the Korean peoplehave sufficient strength to defeat imperialist aggression and eventually
to attain national liberation."
are Indications of plans toNorth Korean air arm with large Aerial photography in late Augustof new revetments and ropalrones at major airfields occupied by the
North Koreans.
US reconnaissance aircraft on missions near the Manchurian border in late August were subjected to Chinese AA fire.
BCRBT
16. After the US Seventh Fleet interdicted the Taiwan Strait, Peking began to mute its "liberate Taiwan" propaganda. The propaganda still affirms the theme of eventual liberation, but "we must not neglect our task of national economic recovery."
17.
The Starting Probabilities
On the basis of the foregoing considerations, the analyst oneptember estimates that Peking is clearly concerned about the potential threat to its security from the gathering US military strength in South Korea. Consolidation of domestic control and final defeat of Chiang Kai-shek, however, rate higher in the Chinese Communist scale of priorities than the expansion of Communism on the Korean peninsula. For the'present at least, while US forces are as far as they are from the ManchUrian border, Peking is likely toolicy of watchful waiting.
The analyst feels he can almost certainly exclude imminent intervention on the scale described in hypothesis one. If the Chinese do intervene, the limited scale of intervention described in hypothesis two is probable. The chances seem better than even, however, that the Chinese will not intervene with their own combat forces at all.
Using the verbal-numerical equivalencies suggested in the tabulation on pagehe analyst assignso the probability that hypothesis one (large-scale
OEGRET
intervention) is true. The starting probabilities of hypothesis two (limited intervention) and hypothesis three (nonintervention) arendespectively.
At thisoment's pause is in order for some reflections on these starting probabilities. From the perspectivet is quite apparent that the probability of the nonintervention hypothesis is not overly important. That hypothesis is going to be conclusively disproved in little moreonth. It will then be the respective probabilities ofone (massive intervention) and hypothesis two (small-scale intervention) that will be governing for the crucial US decisions about military strategy. Only these two probabilities will enter into the odds for or against large-scale Chinese intervention.
What the odds are going to be will depend on what events take place to change the odds and on what level of odds was estimated to start with. The starting odds favoring limited over large-scale interventiono 1 (the probability of hypothesis two over theof hypothesis
I Will the
lower starting odds of the Korea simulation unfairly bias the ond result to favor the hypothesis known in hindsight to have been the true one?
This question is best answered by going back to the literature Just what was the thinking of the intelligence community? The feeling of the community in0 is suggested by two authoritative analyses of the time.
One, datedugust observed: "As it became apparent that the North Koreans were being defeated in South Korea, the Chinese might well take up defensive positions north ofh Parallel. The USSR might use Chinese Communist troops at Any stage in the fighting, but their participation would be especially useful ath Parallel where UN members could legally discontinue their support of the US policy."
The second, publishedeptember, concluded: "In view of the momentous repercussions from such overt action (large-scaleappears
probable that the Chinese Communist participation in the Korean conflict will be more indirect, although significant, and will be limited to integrating into the North Korean forces 'Manchurianerhaps including air units as well as ground forces."
The prevailing opinion then did not reject any idea of significant Chinese troop movement right down toh Parallel. Large-scalo intervention was deemed tolausible prospect, although limited intervention was "more probable."
Ifndtarting probabilities are in fact close to what intelligence felt inow should intelligence havo felt later? What specific probabilities should intelligence havoby mid-November? Onovember, the UN military command began its end-the-war drive to the Yalu, only to run head-on into the conclusive evidence of large-scale Chinese intervention.
The New Evidence
These estimated probabilities of intervention and nonintorvention keep changing. The warrant for change is the incoming evidence, some of it presonted below to illustrate analytical method under the Bayesian approach.
Two likelihood ratios aro shown for each unit of evidence appraised in September and October. The first expresses the diagnostic value of the evidence for comparing the massive intervention hypothesis with the nonintervention hypothesis. The second servesto compare the limited intervention andhypotheses.
New Delhi newspapersctoberctober carried articles with Peking datelines stating that major conflict in Korea now looked almost inevitable. The articles reported high Chinese sources as saying that when American forces crossedh Parallel, they would clash with Chinese forces.
(The analyst in hindsight is tempted to assign higher diagnostic value to this unit of evidence. Intelligence thinking at the time, however, gave considerable weight to the idea the Chinese were bluffing.)
4. Success of UN Arms inctober, the North Koreans were in full retreat UN forces had captured the Communist capital of Pyongyang.
decision to intervene might well be contingent on the course of military events. If so, the decision to go in would almost certainly be preceded by such evidence as is now appearing, that the North Koreans cannot hold on without outside assistance.
The evidence (displaying as it does the might of US arms) could alsoeterrent to It is highly probable that themilitary successes of the UN forces would be associatedhinese Communist decision in favor of nonintervention.
Both arguments are good and both considerations may be influencing Chinese policy decisionseking. The deterrent consideration is probably the more influential one.
Likelihood
(The presumed deterrent effect of the UN military advance clearly influenced US analysis during As General MacArthur put it at his meeting with President Truman in mid-October on Wake Island: "Had they interfered in the first or second months, it would have been decisive. Now we are no longer hat inf the Chinese tried to get down to Pyongyang, there would be great slaughter." The consensus of the intelligence community in mid-October also was that the time for effective Chinese Communist intervention had probably passed.)
The first contacts with Chinese troops toward the end of October eliminated the nonintervention hypothesis from further consideration. Only one likelihood ratio is therefore shown after October. This likelihood ratio describee the diagnostic value of the evidence forthe massive intervention hypothesis with the limited intervention hypothesis. Where accuracy of evidence is ineliability rating is also assigned.
5. War Propaganda Aftor the Initialpropaganda on the war, which had diminished just before the entry of Chinese troops into Korea, stepped up again after tbe intervention.
Onctober, People's Daily declared in an editorial: "The ambitions of the US imperialist bandits will not be satisfied with the attack on Korea. Truman will certainly extend his aggressive war to tho borders ofin the footsteps of the Japanese predecessors who also began with aggression against Korea and then the Northeast and the interior of China. But this aggression will not be tolerated by the Chinese people."
Excerpts from other propaganda at this time follow:
"The war in Korea has nowewhina and Korea are separated by one river, with the two countries havingi of common
The Korean people "took an active part in China's revolutions and did not hesitate to shed their blood and sacrifice themselves for ourFBISharp increase during the first week of November in propaganda to convince Chinese domestic audiences that theyblood debt" to Korea.)
"It is vory clear now that American imperialism
ls following the beaten path of Japanese imperialism
wishful thinking of annexing Korea, and then
from there invading our The Chinese
people will notopetition of the history ofears ago. Therefore we must be on the same front as the Korean Rise up in the struggle against the American imperialist agres-sors to aid our heroic Korean brethren."
During the closing week of October and early November, mass rallies were held in every major Chinese city. The rallies staged pledges to defend the fatherland by "volunteers anxious to fight the American imperialists in Korea."
"Resist America, Aid Korea" was the slogan of the propaganda campaign, but the propaganda was not explicit about the scale of intervention envisaged. It was intimated in one Chinese Communist article, however that the Intervention should be massive enough to bog the Americans down in Korea: "There are twone is that the American imperialists will be
forced off the Korean The second
is that after US troops suffer defeat, they will continue to increase reinforcements, ceaselessly expending men and material, becoming mired ever deeper and more helplessly."
Chinese would almost certainly mobilize domestic opinion in thisrobability) if they anticipated fighting in force against the US. However, there are enoughin the propaganda to leave the Chinese the option of support to the Koreans in aof guerilla resistance. The chances are better thanrobability) that aid to North Korea in the form of overt guerrilla support would be preceded by the same propaganda line as would precede all-out intervention.
Likelihood
Improved Communist Militarywere signs in early November that Chinese participa-tion contributed markedly to the stiffening of Communist military resistance.
Chinese units were evidently in close proximity to the regrouped North Koreans. Ono North Korean military documentovember mentioned a
meeting with the "division commander ofh Force of the Chinese People's Liberationther North Korean military documents referred to the "volunteer army."
I captured LnlneBo prisoners reported that theh0 men) had crossed the Yalu onctober.
An increase in air capabilities was also obsorved. MIG aircraft engaged by US pilots flew in from tbe Chinese side of the Yalu. They had noon wings or fuselage.
Chinese ground troops so far known by US intelligence to be in Korea number fewer thanostulated in the all-outhypothesis.
However, the reference to "division commander" is noteworthy.
This reference and the evidence (as yetof an entire army crossing the Yalu suggest that large Chinese military units are perhaps being keptforces are not being infiltrated ln mere battalion or other small-unit strength for say support of guerrilla operations.
The signs are almostrobability) of tbe kind that would now be appearing if all-out intervention is imminent. The signs are
also compatible, however,robability)robing effort that the Chinese will abort when the US demonstrates its clear superiority on the battlofleld or threatens to carry the war to Chinese territory.
Likelihood
Reliability
(Intelligence opinion at the timerave view of this evidence. The JIIC [JointIntelligence Committee, predecessor of the USIB Watch Committee] notedovember that "elements" of four Chinese armies had been identified in Korea and that tho estimated number of troops in these elementsut "from the successes achieved by the North Koreans with Chinese assistance in their counter-offensive in Northwest Korea, it is difficult to believe that considerable more Chinese or North Koreans trained in China are not employed. If the full four Chinese arraios are engaged, then the figure will be in tbe neighborhood. Reports from the American units engaged describe the enemy as the best so far encountered in Korea. They are described as being more vigorous, skilled and betterin night movement and attack.")
The Revised Opinion
The Impact of these and the other units of evidence on the odds ls shown graphically in Figure 1. The weight of evidence does not really begin to tell in favor of the massive intervention hypothesis until the first brushes with Chinese troops rule out the nonintervention hypothesis at the end of October. These first contacts with the Chinese bring the probability of massive intervention toercent. The probability rises thereafter until it stands at overercent ln mid-November.
eek before, the intelligence analyst in tbe simulation would have estimated about an even-money chance of large-scale intervention. The simulation ls an instructive lesson on tho transient value of the intelligence estimate in tho crisis situation, on the imperative inituation of keeping intelligence opinion unfrozen, on tbe necessity during the crisis of staying receptive to every new item of evidence, on the obligation to revise tho odds from day to day or hour to hour.
With tho probability value at aboutercont, the intelligence analyst in the simulation can use his table of verbal-numerical equivalencies (pagoo say that largo-scale intervention is probable. He can say this much verbally and numerically and unequivocally. He would want to supplement his unequivocal statement of the probabilities with tbe kind of well-reasoned estimative conclusion that features good intelligence writing today. The hindsight critics notwithstanding, it was tho mark of good intelligence in0 also. Three examples from the period follow.
The first is from "The Chinesere free to adjust their action in accordance with the development of the situation. If the Chinese Communists wore to succeed inthe effective strength of UN forces in northern Korea, they would pursue their advantage as far as possible. If tbe military situation is stabilized, they may well consider that, with advantageous terrain and the onset of winter, their forces now in Korea are sufficient to accomplish their immediateikely and logical development of the presentls that the opposing sides will build up their combat power in successive increments to checkmate the other until forces of major magnitude are involvod."
The second is from the conclusion of the Joint Indications Intelligence Committee (predecessor of the USIB Watch Committee) after its mooting onovomber. Thereefinite possibility, tho Committee noted, of major Chinese Communist intervention In Korea. Chinese strategy, the Committee report continued, seemed designed to bait the IN advance in Korea and to stall for time while preparations for larger action were completed.
The third Is theovember update of NIE-2. (This was the date the UN offensive drive to the Yalu began; the NIE is based on information as ofhinese military activity so far, observed the estimate, does not demonstrate any plan for major offensive However, if Peking fails to obtain UN withdrawal from Korea by intimidation and diplomatic means, there will be increasing Chinese Communist intervention.
Nobody hit the nail right on tho head, but the analysis in retrospect does not look bad. Somehow the verbalizations did not communicate the full measure of intelligence anxiety to the political and military commands. Bayesian method in intelligence is an endeavor to go beyond the necessary verbalizations. Itsla valid quantification of probabilistic judgments. Its ideal is the union of phrasemaking with unambiguous numerical scoring, so that uncertain information can make ita due contribution to rational decision.
EORET
secret
ft. BftBlc Raves
The formulation of hypotheses was different ln the Korea simulation; they were propositions in tho future tense. Communist China will intervene in the Korean War with large forces, it will Intervene with small forces, or it will not Intervene at all.
There was no assumption In the Korea simulation that the incoming evidence was necessarily derivative of Chinese policy decisions already taken to intervene or not to intervene. The reasoning was simply that certain events are more likely (or less likely) to precede intervention than nonintervention.
Whether the hypotheses are cast in past or present or future tense, the mathematics is straightforward. Let an hypothesis be stated In the future tense: war will break out.
Now let Prob(eW) stand for the probability thatill occur first and that war will break out soon thereafter. This probability is equal to the product of two factors. One ls the probability, represented byhate will occur whether or not war follows.
The other factor isead "given" for the vertical stroke symbol. Prob(wle) ls tho probability that war will break out, givon thatas already occurred.
In concise mathematical formulation, the probability logic can then be written as:
)
Alternatively, Prob(eW) may be expressedroduct of two other factors. Before anything is known or assumed about event e, what Is the probability that war will break out? This probability, represented by
SKCKKT
s one factor. Given that war will break out, what is the probability that it will be preceded by event e? This probability, represented bys the second factor. So,
)
These two equationshird:
f)
Divide both sides of this third equation byhe result is one form of Bayes' Theorem:
roMe.W)
In mathematical parlance,) is called the posterior probability ofprobability in the light of the latest evidence. Prob(K) is the prior probability ofprobability estimated before the latest evidence was received.
Similar notation can serve to show the probabilitys the true one. That is,
rob(N) x
Suppose the task to be the determination of odds favoring the war hypothesis over the no-war hypothesis. These odds simply set the probability of war over the probability that the no-war hypothesis is true. The result is the following equation:
rob(w) Prob(el w)) >
Simplify the notation by substituting one symbol for each fraction in this equation, if the notation used in the Cuba study is borrowed, the equation then reads:
L
R stands for the revised or posterior odds favoring the war hypothesis over the no-war hypothesis. These are the odds after consideration of the latest evidence. P, the prior odds, is carried forward from consideration
A-2
SECRET
of tho previous evidence. If accuracy of evidence is not in question, the main burden on the analyst is to decide on the valuehe so-called likelihood
ratio.
If accuracy of evidence is in question, he is under the additional burden of rating the reliability of the reporting source.
The Intelligence analyst applying Bayes' Theorem does not keep debating with himself or with others tho merits of the hypotheses. Approaching the evidence in the Korea simulation, he does not ask how each new event affects his own or anybody else's previous estimate about prospective Chinese intervention in the Korean War.
He asks himself, instead, how likely the event would be, given that say the massive intervention hypothesis is true. He asks himself also how likely the event would be if tho nonintervention hypothesis is taken as true. His answers to these two questions givo him the probability components theratio.
Or, estimating the likelihood ratio directly, he combines the two questions and asks himself how much more likely the event would be if one hypothesis is true than if the other is true. But he does not ask himself how the event affects the probability that an hypothesis is true. He inverts the question to ask how probable would the event be under the assumption that the hypothesis is true.
The Partitioning Problem
ihe mathematics ofheorem was poker chip experiment. The
oxperiraont typically involves two boxes containing red and blue poker chips in different proportions. The color mix ised-blue in the so-called wared-blue ln the no-war box.
A test subject is given this information, but he cannot tell the boxos apart from their outward He picks one of the boxes at random from tbe nheif. His task ls the determination of tbe odds favoring the war hypothesis (that he has picked the war box) over the no-war hypothesis.
In the beginning, he can give no better than even-money odds. That ls to say, he startsaluen his BayosianL. Then, drawing chips at random from the box in front of him, tho test subject applies bis likelihood ratios to send the odds up or down.
Suppose the test subject replaces each chip after noting its color. The latest event is the drawed chip.
The values determined by tho formula:
The numerator is the probability ofediven the condition that It is drawn from the war box. The denominator is the probability ofediven that it is drawn from the no-war box. The likelihood ratio for this event, the drawed chip, is.
The only conditions the analyst considers when evaluating the event are these "given war" and "given no-war" conditions. The previous evidence, once it has been Incorporated into bis odds, is not considered again. The reason it is not considered again is that oach chip drawn from the box is replaced before another is drawn. Thus the probability ofed or blue chip from tho war box or no-war box never changes; tho probability is not affected by the previous evidence.
B-l
Suppose each chip drawn is not replaced. Then the probability ofed chip or blue chippecified box does not stay the samethe experiment. The probability is determined, not only by the condition that the chip is drawn from the specified box, but also by the number of chips and combination of colors previously drawn. The valueay then be expressed as:
robCelwE) Prob(eJNE)
The numerator again has the vertical stroke to indicate that the probability holds onlyertain condition is taken as given. The condition now is, not only as before that the red chip is drawn from the war box, but also all the previous evidence (denoted by the The denominator similarlyrobability which this time depends, not only on the hypothesis assumed true, but also on the previous pattern of evidence.
The analyst appraising each new unit of evidence in the Korean simulation generally took only hypotheses as given, not previous evidence. In other words, the probabilitynit of evidence was generally held to be no different for appearing in November than in October or September.
The Instinct of the intelligence analyst is to recoil from this supposition. Intelligence doctrine makes much about the significance of patterns. Military deployments officially described as training exercises, for example, take on more significance if preceded by ominous evidence than if preceded by reassuring evidence.nit of evidence in Bayesian analysis to be given the same likelihoodhave the same effect on the odds for war ormatter what the surrounding context of other evidence?
The answerlarification of terms. The necessary distinction to make, an important one in probability theory, is between conditional independence and unconditional independence.
Once again the poker chip experiment helps to clarify the issue. The test subject can give no better than even money that the first chip he draws will be
B-2
red. But If he draws (witherieships, of whichurn out to be red. he can estimate something close6 probability that bis next draw will be red. Tho mathematician would call the events unconditionally dependent--unconditlonally because nothing is postulated about the box the test subject is drawing from, dependent because theofed chip changes as the pattern of previous evidence becomes more conclusive.
He would also say the events are conditionally independent. Given tbe condition that the test subject is drawing (with replacement)pecified box, the probability ofod chip never changes, that is, never depends on the pattern of previous evidence
Selection of hypotheses, in other words, makes It possible to treat many events as independent of the previous evidence. The intelligence analyst says, "of course they arend in one sense, he is right. But if he is applying Bayesian method, he has to know the sense ofword he is using.
If he is applying Bayesian method, one of the burdens on him is the partitioning of evidence to get, as far as possible, counterparts of poker chip drawings with replacement. In practice, itroblem of avoiding serious error rather than oscaping error altogether, for judgments about correct partitioning will vary from person to person.
The rule of reason is to combine all reports on one general subject intounit of evidence. The analystittle training in probability theory may well do better at partitioning than the one relying only on his sense of practical reason. The analyst with some theoretical background may also do better atthe cases where, despite his best efforts, he ls left with some conditionally dependent units of evidence. When such cases arise (as they inevitablye must estimate his likelihood ratio accordingly. Does Chinese propagandaote of dire warning which was also observedreceding unit of evidence? The propaganda will not be appraised as it would bo If it stood alone.
C, Baves and Bias
Every hypothesis is diffuse; it covers more ground than is explicit in the wording. To make the hypothesis more explicit, substitute subhypotheses.
The nonintervention hypothesis, for example, can become two subhypotheses that incorporate alternative propositions about Chinese defensive policy. Onesubhypothesis is that Peking will not intervene but will build up its defenses in Manchuria. The other nonintervention subhypothesis is that Peking will neither intervene nor improve its defensivein Manchuria.
The number of conceivable subhypotheses is myriad. Fortunately they do not all have to be introduced into the analysis. Nor once introduced, do they have to be considered in the appraisal of every unit ofubhypothesis about defensive buildup may be important when weighing evidence about troop deployments. ubhypothesis about hedging against pound devaluation may be important when weighing evidence about drawdown of sterling deposits.
A distinguishing mark of the professional in intelligence analysis is his ability to explainand sometimes to explain itintroducing sub-hypotheses. When he explains evidence away, it is by rejecting one of his alternative subhypotheses. In the Korea simulation, the possibility of noninterventionefensive buildup in Manchuria was rejected as altogether implausible. The Chinese might not intervene, so the feeling went, but they would surely hasten to improve their defensive capabilities in Manchuria.
Given this judgment, the evidence on troop movements to Manchuria did not give intelligence muchearing on Peking's intervention intentions. Untilcale thatuildup past the needs of simple defense, the evidenceikelihood ratio.
Suppose intelligence felt that Chinese nonintervention would very) but not certainlyollcv of defensive buildup in Manchuria. As before, troop
deployments toward Manchuria would be evaluated as necessary and certain if Peking wero about to intervene; they would be equally necessary and certain if Peking were not intervening butolicy of defensive buildup in Mancburla.
However, to complete tha chain of reasoning,would now also have to bring in another opinion about the probability of troop movements tothe probability if Peking wore not intervening and had no policy of defensive buildup. The intelligence opinion would be influenced by the knowledge that Lin Piao's Fourth Field Army was slated in any case to return to its home base in Manchuria after tho Communist conquest of the Chineso mainland. Suppose therefore thatfelt there was an even) of these northward troop movements in September even if Peking were not intent on Improving its Manchurian defenses.
The Mathematical Notes, Topic D, of the,ormula for calculating the likelinuou ratio
when subhypotheses are introduced in this fashion into the analysis. Under this formula, the suppositions given above would have resultedikelihood ratio for the early evidence about troop movements to Manchuria, notatio that was in fact assigned.
Theikelihood ratio was applied in the simulation bocause the intelligence analyst felt so strongly about his subhypotheses. Bayes' Theorem does not eliminate strong feelings. If he is reasoning toward probabilistic conclusions, tbe analyst will find Bayes' Theorem an especially useful rule of logic. Like all other rules of formal logic, it makes the most of antecedont propositions by assuringthat are consistent with those propositions. But it does not free analysis from prepossession and predilection.
C-2
figure
PROBABILITY OF CHINESE COMMUNIST INTERVENTION IN KOREA0
Original document.
Comment about this article, ask questions, or add new information about this topic: