THE DEFINITION OF SOME ESTIMATIVE EXPRESSIONS

Created: 9/1/1964

OCR scan of the original document, errors are possible

STUDIES IN

INTELLIGENCE

A coliocUon ol articles on Iho historical, operational, docirlnai, and theoretical aspects ol intelligence.

>

All suiemenu o( fact, opinion or analysts expressed in Studies in Intelligence arc those of

the authors They do not necessarily reflect official positions or views of lhe Central Intelligence Agency or any other US Government entity, past or present. Nothing in the contents should be construed as asserting or implying US Government endorsement of an article's faciual statemems and inicrprctalions.

...

shows general agreement on the meaning of "probable' and some equivalents, eltewhere much disagreement.

THE DEFINITION OF SOME ESTIMATIVE EXPRESSIONS

Finlslied intelligence, particularly fa making estimative rtatements,umber of mcdifrers likeprobable,'* "unlikely" -possible" that can be thought of asange of oddsiltty, Rad Aesosupplemenled by various other expressions, especially verb forms, conveying the sense of probabilitycertain other words express notbut quantity, imprecisely but perhaps witifa definableome people object to any effort to define the odds or quantities meant by such words. They argue that coobsxt always modifies the rneaning of words and, more broadly, that rtgi^efinidons deprive language of the freedom to adapt to changing

It is possible, however, to state the defmlUons fa quantitative terms without making them artificially precise. And if twothirds of the users and readers of the word probably, for example, feel Itange of oddsuthen it is more useful to give ft this deration than to define it more or less tautcJogicafly fa terns of other words of probability. This would not deny to context Its proper role as the arbiter of value, but only limit the range of its iiiflueoce. Nor would it freeze the language fa perpetuity; as the meanings of the words evolved the quantitative ranges could be changed.

This article describes the resultsurvey undertaken toif such words are indeed tmderstood as measurable quantities and if so to ascertain the extent to which thereonsensus about the quantitative range ofhree-part qucstiomnure on the subject was distributed in the inlcUigence community--to INR/State, the DIA Office of Estimates, and five CIAa simplified version of ft was sent to policy staffs fa the White House, State, and

ions

Estimative Express*

Pentagon. Responses were receivedteUigeoce analysts andobcy officers.

The responsesatisfactory consensus with respect to various usages of likely and prorwMe. phrasos expressing greaterthan these, and modifications ofbetter-than-even, slight There was no satisfactory agreement on tbe meaningide variety of verb forms such as ic* believe and might. There was also little agreement on the non-odds quantitative words such in-< yAiffice^^C^

probabiuties than fnteuigeoce analysts did Correlation betweenassigned in and out of context was good.

"

Part One of the quesbotmaire listedxpressions that might be thought of as Indicating odds and offered the choice oftc.s the percentage probability or en'Trfft outJg-olfled by each. If the respondent believed that no quantitative answer was satisfactory be could mark "Not Applicable" instead. These expressions of course had to be fudged without benefit ofbut in order to check on the validity of such judgments some of them were repeated fn Part Two, where they were included inentences taken from intelligence documents which had beenix different offices of tbe community. Tbe names of all persons and countries in the sen fences were changed to sterilize them against bias. Part Three then listed nine expressions of magnitude notto probability and offered an assortment of ranges for each.

The ideaelative, but for purposes of PartsTwo It was defined asr more of respondentsodds withinoints, pins or minus, of the mostIf the odds or chances most frequently specified forwere SO outundred (a* tbey were)f allhad fallen within tbe rangehe requirementsconsensus on this word would bave been satisfied Only onerecorded for each question: when an answer was rangedseveral adjacent figures, it was recorded as the mean.range ofoor possible would thus haveefknibons were also considered invalidatedc snore"Not Applicable* responses rejecting tbe

Estimativ* EKprenions

The replies were tabulated In four categories in descending order of valid definition, as follows.

Categoryco risen mir more of all re-ipondents.

Categoryconsensusf all

Categorycocsemus, but fewerf respondents marked "Not Applicable,"

"Not Armucable.-Findingr

The following tables summarize tlie findings of the survey. After each expression fron Parts One and Two axe" shown the odds most frequently specised and the percentage of respondents withinoints of that For questions submitted to policy officers as well as analysts, their responses are shown separately. The expressions of magnitude fn Part Three are listed with the percentage of "Not Applicable" responses and the most frequent response for each.

Of thexpressions in Part One three fell Intosuper-consensushirteen Intoeventeen intoC (nond eight intorejected asFrom Part Two five expressions In context fell Into Category B, twelve Into Category C, and three into Category D. All the quantitative phrases fn Part Three were rejected as not measurabler more of tbe respondents except for next /esc years and next year or to. Though rejected byext few yeart found noears,ean. Nest year or aoears to two-thirds of theears to the rest

PART ONE (No Context)

iluufoten WrnoMorrrrt

Aaatyrtorurnsui)

Almort SO

Are

Wfl100

Category BCofoenru.)

-

Eslimalive Expressions

mlfki ibo

o undliteoUM ta-

ix"t -

cMbl*

U-

Weooto&ry

bo*

hange* HWJ

aajfl be lata thanwlfAr Wi-

Category D.bat

ot 01

_oemg maintained and

progressetrig aaade Id reducing the ana of lemakuVigteapoodentjasked to tpeeify the probabdMy thai AlbaMa aad Poland wet* beadeda rapcroctvmroirohafaebey eW the kleologleeJ drScreoce. would be settled.

were asked for the probability that tbe rpeabar believed

arked for the probebfltty that change* would be minor.

were atked for the probability of that to which thaot cccnrrutted." The haHiven on

irjodaL

' The full coGtut ocjiii iMbm wm tbe sentence. 'Although btcUng toe draaaa of vMb by top leeden. tha travel of these <kkgitkio. to Albaalaf tbe AlbulBD. Polishfa beta? maintained and

-

EtfimoKve Expressions

_

(wct^owiht) ocest

Analyst Rrjected)raises tn* ouoftonhey

so _

We do Ht expect them to change _

Cube has oUcpJly SO _

PART THREE (Words of Xi.snln.de)

.-iw

Subtuand

Slrsullcaat

fpvrttoa)

Sevrial

ean

Fe-

Next few yean

Neat year or

The difference between the good cortsensuset of odds tor one expression end no consensusoother showVup dearh/when the odds axe graphed according to how frequently each set was tptxftcf lo the rcstMiu^uestion.f all responses fall withmoints of the most frequent one, the graphteep curvearrow base. The high, narrow peakearly defined consensus,road-based curveingle peak shows less agreementurve with several "peaks reflects dear differences about what the word means.

Steady retrogression from consensus can be seen fn graphs of sample responses from successive categories. Following are these seven from Parts Ooe and Two.

Ootc/ecotext,

Almost Certainly

Serious '

Estimative Expressions

At

C.TXCC-T

The Norm Koreans hive thus far shown marked respect for US power, toddo not eiprci them to change this basic at-tirade" ntvcues what ptobaUttj- that the North Koreaa* wtD

continue provocations against South Korea' D

"Al tbe saane (tea. tba resvrvattoo* coo-eyed la Usecomment Mfffert that tba practical military change* resulting from tba oew Urn may well be less dramatic than the tot* of de Caulk'a nccch mixtaand that to am orait. bat w*iTu* conMti tact,sftV

haveom>weaponww

tto blackhohat ofoUcy The

diUS,Cdde^esS^ peaks ot roost frequent response.

-e;

Conclusion*

Ofuestionnaires ftlumcd, only one indicated (hat no quantitative equivalent was suitable for any of the pcobabillstic ex-pfessions. All others selected sets of odds for at least half of those listed In Part One,id so for two-thirds of them. Evenumber who disapprove of quantitative definitions probably fust did not bother to return their questionnaires, the results appear to Indicate that the vast majority in the intelligence communityit Ir^ilimate to think of such' cspiessicun fn quantitative -

On the other hand, although moref both analysts and policy officer] agreed0 point range on the expressions fnnd H, the results for some offices on the analytical side did not agree with the consensus for all analysts, and there

bn mere were

similar exceptions among tbe policy offices. So when an analystoffice uses the word probably, policy officers and analysts indo not necessarily Interpret the word to mean thend B, however, tbe differences are usually notfollows tbe quantitative definition oiort frequent plusexpressions on which there was found to be a

0

Urdoubtedly

Htahly

Probably

cosa

eo-80

Probably

Good dance

Seems Likely

Better Than Even Chance

lAafy

Not

Tho oui^-context definitions lo Part One were spot-crsecked by

SOiF^WO- COotet.there was no way

ube si,nsJ Wlcdge of -Ihe subfect matter. But despite these limitations, because the most "

EstimolW.

frequent deEtnitioos in and ou! of context agreed withinoints, ft appears that nearly the same meanings were conveyed either way. The comparison appears below.

MOST FREQUENT

amlytt Folic, axuj^rt - 00

M75 to :

cmcej (hat . Appaieedy

:

good 70 Good

the couplingerb of opinion with an expression of odds, as in "We believe the chances areeemed not to affect the meaning of the latter for tbe respondents to the survey questions, this writer agrees with Mr. Cent's purist that the doubling up of probabilistic words Is potentially confusing and should be avoided. The response pattern on tbe Korean question (pagend Graphas an interesting side light In that the probability queried does not follow from the estimative sentence. The questionnaire was not designed to test tbe propensity of arialystf and policy officers to draw unsubstantiated conclusions, but In this one hritaranef the respondents showed they recognized the nonsequitur by"Not Applicable."

were

The survey showed that for expressions on which thereonsensus (and some others) the most frequent response was the samo from policy officers as from analysts withinoints plus or minus But where differences did occur, lhe policyonsistently on the conservative side; see tbe following examples. MOST FREQUENT RESPONSE

Analyst Pobey

100 90

Highly Probable 00 68

90 ao

5 to

The results from Part Three showed there is little consensus on the common expressions of vague magnitude, at least without the guidance of context.

An effort was made to keep the questioemaire as staple toand as short as possible. In Parts One and Three thegenerally successful, but Part Two was neither simple norof the questions in the latter related to specific peopleand there was danger that respondents would permitand knowledge of tbe subject to influence theiraddition, several of the estimative sentences were long andthe hazard of confusion about what they meant andwanted in evaluation of

For pragmatic reasons, administration ol tbe survey had to be informal. It is possible that such things as attitudes of supervisors, office collusion, or misunderstanding of the purpose of the survey could have introducedareful perusal of each of thofatted to turn up any obvious evidence that such factors influenced the findings. But if it were done again the questionnaire should be modified In Part Two and the conditions under which it Is filled out should he controlled and standardized.

Original document.

---

---