STUDIES IN
INTELLIGENCE
A cdlcclion ol articles on Iho historical, opcrattorujl. doctrinal, and theoretical aspects ol Intelligence.
All statements offset, opinion or analysis expressed in Studies in Intelligence are those of
the authors They do not necessarily reflect official positions or views of Ihe Central Intelligence Agency or any other US Govcrriment entity, past or present. Nothing in the contents should be construed as asserting or implying US Govcmmcnl endorsement of an article's factual statements and interpretations
Defense of the inteUigence input to systematic plarming for defense.
OSITIVE VIEW
Julie O. Kertfn
In an article entitled "Economic Intelligence In Defense Wanning"ecent issue of thisolonel Clyde C. Wooten examines aspects of the economico-military research done on the Soviet Union as input to systems analysis. defense needs. He concludes with some exasperation that the data are unusable, the methodology questionable, and the productn this essay we shallthe character of the product and show that in the process an intelligent methodologyogical ordering for data which are Indeed sparse but which can be used to advantage in place of an otherwise unknown, intuitive input into military judgments.plans must reflect, among other factors, judgments concerning the potential enemy; exposing the basis for these judgments and putting the data into logical array improves the plans and provides flexibility through control of changes in the data.
Aggregate Resources and Specific Allocation
The product of military-economic intelligence on the USSR divides naturally into two categories. The first covers general or aggregate aspects of the Soviet economy and its military establishment. An illustration of this typeudgment made2 on the sire of the Soviet gold reserves. By estimating thetechniques analogous to those used for militarya level much lower
'..
a some timet monumental fruitratsOo svith tyitemi analystsethod it bound toW. Quad*iwiyrir for MrtiWry Oeestioiw.
' The methodology Is summarized tn Foul B. Storm's "Estimating the Soviet Cold Position" in Studies VfJ. Iff.ontemporary appraisal see "Soviet Coldeserves Becroiidered" by Keith Bush to Sorter Studiesol Glasgow) Vol XVApril.
than had been accepted on the basis of Informed intuitive estimatej. economic analysts exposed an Important aspect of the Soviets' guns-ot-butlcr problem. Another illustration is the estimates CIA made public of significantly reduced growth rates for Soviet CNP inestern economists and many Soviet economists have since endorsed the general order of that judgment and the predictions made at thatt is because of studies such as these that Colonel Woolen is able to write, "We know that the Soviets have important resource allocation problems."
Tbe relationship between Soviet aggregate military erpenitures and CNP suggested by thend "economic feasibility" can serve, in conjunction with assumptions about theand political environment,road guide loecision maker.ross sense this relationship expresses the marginal cost to the Soviets of various possible defense postures In terms of their economic and political goals, say Irving standards or industrial growth. For this purpose changes in the relationship are more important than absolute levels; consequently the need forin the measurements gives wayess stringent one that they be takenonsistent and reasonable way.
No more detailed exposition of analytic procedures in this macro-economic field will be attempted in the limited space available here.
ecent dBcussWo ol this topic, sec "Latest from InsideS jVewi and Wo/fd Report,. SO.
See LAffaife Aeenbtgyun, lis Economic Hecelattofu, CIA/BB,
Tbe second category, to which this paper is primarily addressed, concerns allocations of military expenditure to particular weaponBecause, as Colonel Wooten points out. it is difficult to isolate military expenditures in the published Soviet budget, the total isby adding up costed estimates of Soviet forces. Costing and production estimates arc also the way to get at the allocation of those military resources among weapon systems, an important ingredient In. military planner's decision on how to allocate his own resources. The kernel of Colonel Wooten's question is, "Can present data and methods quantity Soviet military forces and potential with sufficient accuracy to assist. military planner In thise believe they can and do, with due regard to the statements of uncertainty in which the producers of the estimates, especially when
prognosticating, seek to eommuriicate the nature of their product aoc the care needed in using it.
The following explanations of the costing methodology, the meaning of the results, and the way the process is handled in the community should help answer some of the questions raised by Colonel VVooten It docs not deal, however, with his doubts about the validity ot using ruble costs toeasure of opportunity cost or economic scarcity in the Soviet Union. The academic detail involved would take up ^Disproportionate amount of space, and the question bas been discussed elsewhere by experts on the Soviet economy.1
Methodologu: Ruble-DoUor Ratios
First, dollar prices of Soviet equipment are estimated: these are what It would cost. DoD to procure the Soviet weapons. manufacturers. The technical specifications of the Soviet weapon are given to the manufacturers in the same way they would be. weapon. Some adjustments to. technical environment may be allowed, for example in standard dimensions of materials or incessing techniques the alteration of which to meet Soviet specifications would be expensive without significantly affecting performance. The purpose of the dollar figures is staled, with the necessary caveat,ecent paper:
The dollar vacations of Soviet militaryan provide US planners with an appreciaOon ot the physical magnitude of given Soviet programs and aboseful basis for comparison svith USecause of significant diferaiKes in the price structures of the two countries, however, the dollar valuations of Soviet programs do not necessarily ptossde the most accurate indication of the relative costs of risen programs from the Soviet point of view.'
' For esamplc: Value and Flan by Ciegoey Grossman, Universitv of Californiahe Soviet Price System" by Morris Bornsleen In The American Ecommterice Determination In the USSR by Larar Moeseyevseh Kantor. JPRS translation
7 Slatti TrtmU in Soviet Mihtory Pobey, CIA/BR
Then it is attempted to cost the Soviet military expenditures in rubles, that is in terms of the Soviet ecooomic environment. Idcaliv. this will show what portion of Soviet resources are being diiecled to the military effort and the distribution of this effort among different military programs as seen by the Soviet planners. How close reality
comrs to the Ideal depends not only on how closely the nible reflects relative economicquestion on which the reader has just been referred to expertalso on how good theruble prices are. The Soviets' ruble prices for some military items like aircraft and tanks have become available from time to time. Others must be constructed from infomation concerning the weapon. Inputs to the weapon, and the technical circumstances of its production.*
When the ruble prices must be constructed,. manufacturer is asked to apply his experience to cost the Soviet weapon using Soviet Inputs, Soviet plant, and ruble values. The ruble values of the inputs are drawn from information available on the same orsimilar inputs outside the militarycosts, wages, electronic components, inputs into merchant ships, etc.of the thus estimated ruble priceeapon by the estimated dollar priceuble-dollar ratio for that weapon. This ratio remains valid until there Is significant new (nformation on or changes in the proportions of different inputs or in their ruble or dollar prices.
Because of the very different scarcity rcUtionships in. and Soviet economies, the more specific tbe ruble prices are to each weapon the better they will express their relativeingle ruble-dollar ratio for all military hardware would ignore the different scarcity proportions in different weapons which should be approximately reflected In their ruble prices. At the other extreme one might try to cost in rubles every one of the thousands of pieces of militaryor even all the inputs to them.racticaluble-dollar ratio Is constructed for each broad category of weapon and used to convert to rubles the dollar cost of all weapons in that category.
Production Estimates
* Esamples of this bad of information can be found In Compoistlon and Size of the Military Seel or of the Soviet Electronic* Industry,.
The use of production experience to estimate costs of proposed production, tbe basis for this method of estimating Soviet weapon costs, has been found sufficiently reliable. manufacturers in prognostications about their own new products except with respectnd to the possibility of subsequent changes in the product or
conditions. The uncertainciei of RAD. as explained below, are segre-gated from the costing of individual Soviet weapons The uncertainly of possible subsequent changes remains, but no more than in any future estimating, whether of US or foreign, civilian or mditary pro-ductioo. Estimates are always subject to change in the light of subsequent information. Their reliability depends on the quality of the data base, tho soundness of the producers' judgment, and equally important, the context in which they are used.
Production techniques improve with experience, and consequently costs decline with continued output. This phenomenon hat been studied quantitatively, the rate of decline being representedearningearning curve for each type of production is constructed on the bans. and Soviet experience and of known production conditions. From it can be read the cost per unit at any cumulative production level.'
It is useful to know the independently estimated order of battleheck on production estimates when choosing the appropriate cost along the learning curve. Often it is even necessary to use ordet-of-battle estimates in order to arrive at production estimates. But this use of corroborative or combined sources to arrive at quantities and costs involves no such circular reasoning with respect to fudging economic feasibility as Colonel Woolen seems to think. Crossfeasibility could be judged without reference to production estimates simply by costing the order-of-battlc eslimate.
' Sea Paodotph Payne's "Production at in Aircraft Plant" ta Studiri VI. ST If.
As Colonel Woolen emphasizes,more accurately Research, Development. Test, andprobably the most difficult area not only to cost but even to define. Definitional and statistical efforts.ctivitiesery short history, and such activities are not attributable with precision to particular weapon systems. Consequently no attempt is now being made to cost Soviet RDToE by weapon system for use in aggregate figures. Test vehicle estimates are dropped from production estimates, andigures are separately derived from published Sovietbudgetary "science" figure and "funds for science from otherA nible-dollar ratio is independently estimated for this line. The difficulties involved tn Judging costs of research and development in the Soviet Union, as in the United States, do not, therefore, permeate the military expenditures estimate but are confined to the one item, RDTAE.
now who worry about analogy costing because of difference!. aod Sovietractices should find solace in this egreptton.'"
fn* High-Low Range
'1 hereisundci sin tidingost Important attempt to leBnc the costed range of possible Soviet choices. Colonelistressed byap between some of tbe low and high wumptions made for the level of Soviet forces in the study "Soviet Defense Expenditures" (CtA/RRecause he nterprcts these as defining the range of confidenceingle estimate jf the most probable Soviet outlays. Bui tho high and tow projection lines do not boundonfidenceange within which any singleline is as likely as any other. The inferenceimilar range of uncertainty must be appbed to any estimate on Soviet defense expenditures and the conclusion thai no meaningful measure can be based on such uncertain estimates are therefore completely wrong.
The range is comparable tather to lhatcatter diagram where the high and low occurrences have been liared to band the scalier. Within lhe full band one can expect all occurrences to fall; there% confidence that any event will be in this range. The statistician would then draw narrower bands according to the data, defining lhe range within which,.f the events occur. These bands would not necessarily parallel% bound-artel. If the events were systematically related to the variables against which they were plotted one wouldigh proportion of them lo fallarrow band coveringf the full range.
The high and low assumptions of the analysts might rimilarly be described asomprehensive range of Soviet military force structures or as banding the spread of practical possibilities. Within this range, considering technical, military, economic, and politicalingle most probable force structure is then chosen. While there is of course less confidence in the single estimate than in the whole banded area, there is greater confidence in it than in
"Sea SoiV* Dtftnm HOD FtptndHw.Aeptember MM
any other tingle structure possible. And the degree of confidence is not at all determined by the width of the banded area.
The high-low estimates thusrame within whichapplied to the problem of US. forces can be accommodated to the widest range of circumstances. The dual estimatesuantified means to reduce the appearance of inconlovertibthtyingle estimateasis for introducing contingency insurance into decisions based on itu The defined spread of alternatives makes it possible for policy decisions to allow not only for uncertainties in the data but tor the possibility of unforseen environmental changes that could lead Soviet policy makers to shift their decisions. Ita variety of sensitivity analysis on the policy level, making for greater flexibility in the decisions.
Community and Confidence
An crpaiulon of interagency cooperation has accompanied theefforts to meet DoD requirerncots on Soviet military ei-penditures. The inclusion of more cost analysis in the NIEs hasin htoader participation In the evaluation of cost estimates, the published figures and discussion being an agreed summary of contributions from the community. Much benefit is derived from the distribution of the detailed contributions to participating agencies. Community review in Ihe representatives' meetings helps clardy methodology and communicate degrees of confidence.
In the NIE. confidence levels are indicated in general terms by words like "possible" and "probable" Esplanaticns of methodology, documentation, and specific discussions of confidence are available in the individual contributions and. often in greater detail, Inresearch and analysis on which the contributions are based. To require that each NIE contain all these materials would make it so compendious that there would no doubtequest for the separate publication and distribution of athe NIE
The Intelligence Assumptions for Planning and National Intelh-gencc Protects for Planning papers, also subjected to the USIB review process, do contain discussions of the limitations ofpend iture estimates. Footnotes, in addition, provide an alert to areas ofMachine runs for these estimates are available to readerseed for more detailed input figures.
" See especially "Analysis and Deslrn ol Conflict Systems" by Albert Wohhifner in Analyna /or Mtfliaryh. 7.
SECRET
Military Costs
Another interagency cooperative effort to estimate military erpondi-tures, the CIA/DIA Joint Analysis Croups alternative projections for Soviet forces, uie generally the same procedure) and pricing factors in coiling The first Eve pages of the "Foreword" of their costing study is devotediscussion of cost estimates and their limitations. The Foreword also raplicitjy invites users "to consult with JACthe degree of confidence associated with cost items ofconcern to them and. inform JAC of any projections which appear to be in error." j.^
Coniumers arc everywhere encouraged to consult the military-economic analysts. Users of the NIEs and NIPP have ready access to the producers. Inlose working relationship has been established between personnel of the office of the Assistant Secretary of Defense for Systemsajor user, and cost analysts in CIA. so that they are in continual consultation without any formalities. Other agencies have sirrular relationships when needed.
Tlie increased formal and informal cooperation within thecommunity provides an opportunity for joint improvement of cost estimates as additional input data become available and asIs improved- The concomitant increase in interaction between the producers and consumers of expenditure* eatsrnates enables the producers to keep aware of and respond rapidly to the needs of the policy maker. Including his need to know the [mutations of the productarticular use
The Slate of ihe Art
Do the acknowledged gaps in information and the large role played by human judgment in arriving at estimates make it Impossible for the Intelligence community to produce figures which will meet the needs of the systems analyst? llts needs are lessthan may be realized. Analijtis for Military Decisionshorough review of the systems analysis approach to organizing theof complex questions of choice in the face of uncertainty, questions such as occur in the field of national defense. Itrepeatedly that the systems analyst does not pretend to provide certain or absolute answers as input to military policy. He himself must accept Input data derived from human lodgments to form the basis for other judgments.ecent DoD-sponsored symposium on cost analysis, Alan Enthoven offered hb general impression that the art of systems analysis Is now at about the same state as medicine
was during tho latter half of the nineteenth century; that is. it has reached the point where it can on the average do more good than harm. He added that it would be just as unwise to stop systems analysis now as it would have been to stop medical research then
Under these circumstances producers and consumers should both be continuously alert for errors and tin justified conclusions. One kind of error is illustrated In that correctly cited by Colonel Woolen,4 exti*apolationeclining trend In Soviet economic assistance to "non-aligned" nations- lhe error was not in the economicbut in the implied judgment that the Soviets would not reexamine the political value of expenditures consumingmall portion of the national product This particularshould have been recognized as one especially vulnerable to political considerations.
Errors resulting from gaps In data also occur. Such was theof information on production at Airframe Plant No.nd its consequent omissionstimates. This is the reason for continuous scrutiny of all source data; data gaps must be expected to occur, especially in the more dynamic sectors of militaryTbe error introduced in this case, cumulated for the fivc-ycar period, wasillion rubles, by itself too small to have appreciable influence on aggregate figures given in billion rubles "correct" to one decimal. It does hosveverimitation, one of the uncertainties of which intelligence customers are warned. It should always be assumed that there are gaps in productionmany of them filled by technical Judgments but someunknown and unmeasurablc except by interpolation from order-of-batUe estimates.
Summary Assessment
We conclude, therefore, that the data base, though oot solid, is usable, the methodology logical, and the product rational. Reasonable intelligence estimates, as an element in malting military decisions, must bo quantified; and no alternative method of quantification bas been proposed. There is no question but that many improvements can beindeed this is an important aspect of the work of military intelligence analysts throughout thewithin the framework of the present methodology and organization.
The real lesson to be drawn from the discomfort Colonel Woolen feels with the current product mayeed for quantification of
SECRET
Military Costs
uncertainty through sensitivity analysis, mcnrurlng the effects ofin input. Such analysis, done at the technical input level, would not be suitable for ^corporation into the published NIE, but it could be communicated to users in other ways. Although sensitivity analysis is practicedimited but growing extent in tbe currentit is legitimate to suggest that it and other methods describing confidence levels be made an integral part of all basic work on estimates of Soviet military expenditures.
It would be wrong, however, to expect statistical lechnicrues and descriptive language to eliminate uncertaintyontinuing element ha Soviet military expenditures estimates. Acceptance of uncertainty in military policy formationecessary ingredient in preparation for reality.'*
For an eicetJent dtacustion svhkh tncluoVi references to alternatives foresfunatei of Soviet military poatur* aias tha present and future needs of system] analysts see Dtahng withinatntu Ahoul fh*Some Rand Eipertmea in Syrlettu Anofyf for WdiMry FLmiUng, by F. I. MosmsaaPR.. This itudylear picture of the need for and use of intelligence estimates cm enemy poatusa. It iKowi the method of handling the expected unc* futilities and al Use lame Uuw Indicates areas of hoped-for impiove-ment in quatttty and quality,
Original document.
Comment about this article, ask questions, or add new information about this topic: