Approved For Release 2000/08/10 : CIA-RDP96-00791 R000200300003-1 DRAFT UNCLASSIFIED SUMMARY REPORT STAR GATE OPERATIONAL TASKING AND EVALUATION 1.0 EXECUTIVE SUMMARY From 1986 to the first quarter of FY 1995, the Star Gate program received more than 200 tasks from operational military organizations requesting that the program staff apply a paranormal psychological technique know as "remote viewing" (RV) to attain information unavailable from other sources. The operational tasking comprised "targets" identified with as little specificity as possible to avoid "telegraphing" the desired response. In 1994, the program office created a methodology for obtaining numerical evaluations from the tasking organizations of the accuracy and value of the information provided by the Star Gate program. By May 1, 1995, 40 tasks from five operational organizations had been evaluated under this process by one or more of the three Program Office remote viewers. Ninety-nine accuracy scores and 100 value scores resulted from these evaluations. On a 6-point basis where 1" is the most accurate, accuracy scores cluster around "2's" and "3's" (55 of the entries) with 13 scores of 1". Value scores, on a 5-point basis with 1" the highest, cluster around "3's" and "4's" (80 of the entries); there are no 1's" and 11 scores of "2". The conclusion that can be drawn from an evaluation of these results of the 40 operational tasks is that the value and utility to the Intelligence Community of the information provided by this process cannot be readily discerned. The ambiguous and subjective nature of the process actually creates a need for additional efforts of questionable operational return on the part of the intelligence analyst. Assuming that the subjective nature of the psychic process cannot be eliminated, one must determine whether the information provided justifies the required resource investment. 2.0 GENERIC DESCRIPTION OF OPERATIONAL TASKING Over the period from 1986 to first quarter of FY 1995, the Star Gate program received more than 200 tasks from operational military organizations. These tasks requested that the program staff apply their paranormal psychological technique know as "remote viewing" (RV) in the hope of attaining information unavailable from other sources. The operational tasking comprised "targets" which were "identified" in some manner, normally with as little specificity as possible (see discussion below) to avoid excessively "telegraphing" the desired response. However, until 1994, the results UNCLASSIFIED DRAFT Approved For Release 2000/08/10 : CIA-RDP96-00791 R000200300003-1 Approved For Release 2000/08/10 : CIA-RDP96-00791 R000200300003-1 DRAFT UNCLASSIFIED from this tasking were not evaluated by the tasking organizations by any numerical method that would identify the accuracy and value of the provided information (in a few cases narrative comments were provided by some organizations in prior years). In 1994, this situation changed when the Program Off ice developed a methodology for obtaining numerical evaluations from the tasking organizations of the Star Gate inputs; this methodology is described briefly in Section 3.0. By May 1, 1995, 40 tasks assigned by five operational organizations had been evaluated under this process.' Section 4.0 describes the numerical evaluations performed by evaluators from the tasking organizations. The descriptions presented below regarding the tasking and the related targets refer principally to the operational tasks that were numerically evaluated. The process for a typical tasking, RV response and subsequent evaluation is as follows: - The tasking organization provides information to the Star Gate Program Manager (PM) describing the problem to be addressed. - The PM provides a Tasking Form delineating only the most rudimentary information to one or more of the three Star Gate RV's for their use during the RV session (a typical Tasking Form is presented in Figure 2-1). In addition, the RV's are appraised of the identity of the tasking organization. - Subsequently the RV's hold individual "viewing" sessions recording their comments, observations, feelings, etc. and including line drawings or sketches of things, places, or other items "observed" during the session. - The individual RV inputs are collected and provided to the tasking organization for their review with a request for completing a numerical evaluation of the individual RV inputs for accuracy and for value. - Finally, for those organization who comply with the request, the evaluation scores are returned to the Star Gate Program Office. Twenty-six (26) of the 40 operational tasks originated from two counterdrug Task Forces, JTF-4 and JTF-5, (see Section 4.0). Typical tasking targets from these organizations comprised the name of a person or thing (e.g., vessel) with a generic request to describe the target, his/her/its activities, location, associations, etc. as 1 Evaluation of additional 1994-95 tasks is still proceeding. Three (3) evaluated since 5/1195 were reviewed and cause only marginaJ changes to the statistical information provided in Table 4-1 and do not alter any of the Conclusions and Recommendations in Section 6.0 UNCLASSIFIED DRAFT Approved For Release 2000/08/10 : CIA-RDP96-00791 R000200300003-1 Approved For Release 2000/08/10 CIA'RDPgf6-00791 R000200300003-1 FIGURE 2-1 TASKING SHEET SOURCE NO: 079 DATE: 18 Jul 94 SUSPENSE: 18 Jul 94 1600 Hrs 1. PROJECT NUMBER: 94-252-0 2. METHOD/TECHNIQUE: Method of Choice 3. BACKGROUND: 4. ESSENTIAL ELEMENTS OF INFORMATION: Access and describe target. 5. COMMENTS: Approved For Release 2000/08/10 : CIA-RDP96-00791 RD00200300003-1 Approved For Release 2000/08/10 : CIA-RDP96-00791 R000200300003-1 DRAFT UNCLASSIFIED appropriate. No specific information (e.g., what is the height/weight/age of the target?) was requested in the tasking. As noted above, the identity of the tasking organization also was provided. For the JTF tasks that identification implies an interest in the drug culture and drug operations. Thus, any drug related information provided by the RV's in response to the tasking "could be" relevant; and, therefore, could be interpreted by the evaluators as having some level of "accuracy' and "value" depending upon the information described and the evaluator's interests and beliefs. The tasking provided by the organization denoted as Org. S (see Section 4.0) comprised targets that were "places" visited by "beacons" (i.e., an individual from Org. S who visited and "viewed" the site of interest) to assist the RV in "visualizing" and describing the site. Targets could be a general vista in or around a particular location, a particular facility at a selected location or, perhaps, a particular item at,a location (in the one case where this type of target was used, the item was a particular kind of boat). No specifics regarding the type of target or its location was provided. Tasking by elements of the DIA comprised two generic types of targets that related to military interests/concerns current at the time of the tasking, e.g., North Korean (NK) capabilities and leadership. The first type of target focused upon then-current military concerns while the second type required "precognitive" (predictive) capabilities since it required a prognoses of future intentions and actionS.2 The tasking from the USA FCI Activity was similar in scope, albeit quite different in context, from that of the JTF's noted earlier, i.e., describe a person, his activities, SG1 B location, etc.. 3.0 EVALUATION MEASURES The numerical evaluation measures that were given to the evaluators of the tasking organizations to score the accuracy and value of the Star Gate inputs were extracted 2 Some operational tasks from the period Oct. 1990 to Jan 1991 regarding Middle East issues were of a similar types, albeit these were not numerically evaluated. They would provide some data for an after-the- fact check of the accuracy of the RV predictions - see Section 6.0 for a discussion of this possibility. UNCLASSIFIED DRAFT Approved For Release 2000/08/10 : CIA-RDP96-00791 R000200300003-1 Approved For Release 2000/08/10 : CIA-RDP96-00791 R000200300003-1 DRAFT UNCLASSIFIED from the Defense Intelligence Agency Manual (DIAM) 58-13. These measures are shown in Table 3-1. Most of the stipulated measures include modifiers such as "may", "possibly", "high", "low", etc. which are subjective and open to individual interpretation by each evaluator. The DIAM 58-13 definitions for the ratings under "Value" are presented in Table 3-2; whether the individual evaluators reviewed these definitions prior to their scoring is unknown. There was no clarification of what was intended by the generic headings of "Accuracy" and "Value", e.g., in the evaluator's estimation how much of the RVs response to the tasking had to qualify for a particular measure, 1%, 10%, 90%, to be granted the related score? Table 3-1 Numerical Evaluation Measures Category Score Accuracy - Is the information accurate? Yes (true) 1 May be true 2 Possibly true 3 No 4 Possibly not true3 5 Unsure 6 Value - what is the value of the sources' information? Major significance 1 High value 2 Of value 3 Low value 4 No value 5 As noted in Section 2.0, one series of tasks were evaluated by a narrative discussion only. While much of the final narrative evaluation for this series was complimentary, it lacked any real specifics regarding the usefulness or relevance of the Star Gate inputs and much of the narrative was replete with modifiers and other hedges. A sanitized extract from the final evaluation report for these tasks is presented in Appendix A illustrating the subjective, "uncertain" nature of the comments. 3 Note that Accuracy scores 5 and 6 actually rank "higher" than 4 since both imply that there may be something accurate in the information. Changing the scoring order to accommodate this observation causes only marginal changes to both the averages and the standard deviations shown on Table 4-1. UNCLASSIFIED DRAFT Approved For Release 2000/08/10 : CIA-RDP96-00791 R000200300003-1 Approved For Release 2000/08/10 : CIA-RDP96-00791 R000200300003-1 DRAFT UNCLASSIFIED Table 3-2 - Value Rating Definitions from DIAM 58-13 MAJOR SIGNIFICANCE - Intelligence Information Report (11R) provided information which will alter or significantly influence national policy, perceptions, or analysis; or provided unique or timely indications and warning of impending significant foreign military or political actions having a national impact. HIGH VALUE - IIR(s) was best report to date or first report on this important topic, but did not significantly influence policy or change analyses. OF VALUE - lIR(s) provided information which supplements, updates, confirms, or aids in the interpretation of information in data bases, intelligence production, policy research and analysis, or military operations and plans; most DoD HUM INT System reporting falls into this category. LOW VALUE - IIR was not a good report because the information was not reported in a timely manner, or was of poor quality/of little substance. Nevertheless, it satisfied some of the consumer's informational needs. NO VALUE - IIR provided no worthwhile information to support data base maintenance, intelligence production, policy research and analysis, or military operations and planning; or its information had no utility, was erroneous, or misleading. 4.0 EVALUATION SUMMARY AND COMMENTS Thirty-nine (39) of the 40 numerically evaluated, operational tasks were performed in 1994 and one in 1995. The information provided by the Star Gate RV's for each task was evaluated by staff of the tasking organization. The complete compilation of evaluated scores is presented in Table 4-1 which includes an identification of the tasking organization and, where known, the name of the evaluator from that organization. Also presented are the individual and collective scores for Accuracy and Value for each of the three RV's and the related average and standard deviations for the compiled scores. (Note that the total number of scoring entries for either Accuracy or Value is not equal to the maximum of 120, i.e., 3x4O, since all three RV's did not participated in all tasks). Table 4-2 presents the same scoring data by tasking organization. Histograms of the scores from Table 4-1 are shown below. Note that "Accuracy' scores tend to cluster around 2's and 3's (55 of the 99 entries) while "Value" scores cluster around 3's and 4's (80 of the 100 entries). This is not too surprising as the nonspecific, nebulous nature of the individual task/target requests permits the RV to "free associate" and permits the evaluator to pick and choose from the RV commentary anything that he thinks "may" or "possibly" is related to his problem (and score accordingly) regardless of how much of the RV commentary may satisfy the particular UNCLASSIFIED DRAFT Approved For Release 2000/08/10 : CIA-RDP96-00791 R000200300003-1 Approvjjftfi"OW/E8/jOA§K4 Or9ffAW0300003-1 .;D"A NUMERICAL EVALUATIONS A B C D E F GT H I J K L I I I I I i Doc. Date Tasker Evaluator Remote Totals Viewer & Scores 2 1A 1V 2A 2V 3A 3v A 3 250 7/13/94 0 3.0 3.0 2.0 3.0 4.0 5.0 ..... . . . ...... -.- ........... ... ....... .............. -,-- . . ....... ....... -.- .... 4 264 /6/94 2.0 3.0 5.0 4.0 5 270 11/3/94 JTF-4 5.0 4.0 5.0 4.0 . . ......... 6 271 11/3/94 JTF-4 3.0 4.0 5.0 4.0 7 273 11/3/94 JTF-4 4.0 5.0 5.0 4.0 4.0 5.0 . . .......... ....... . ..... . ...................... . ..... 8 -.2.6-7-1-1/3/94 JTF-4 3.0 4.0 - 3.0 4.0 9 268 11/3/94 JTF-4 3.0 4.0 4.0 3.0 5.0 4.0 ' - . . .. - ..................... --- 10 2~9 I JTF 3 3 5 5 l /3/94 4 0 0 0 0 - .11 272 1173/9 JTF -4 .... ........ 4 . ..... - ....... .............................". .......... 12 258 8/3/94 JTF-5 1.0 3.0 2.0 3.0 13 257 8/1/94 JTF-5 3.0 5.0 3.0 5.0 ...... ........... - . ....... ... 141 iS 7/28/94 JTF-5 2.0 3.0 5.0 4.0 6 15 249 7/11/94 DIA/PAN-3E 1.0 4.0 2.0 2.0 2.0 4.0 ................. . ............................................... ................ ...... . ............ ............. --I 16 248 7/6/94 DIA/PAN-3E 3.0 3.0 2.0 2.0 1.0 4.0 - 17 ~45 6/24/94 DIA/PAN-3E 3.0 3.0 1.0 4.0 .. . . .......................... . 18 252 7/18/94 JTF-5 4.0 4.0 2.0 3.0 - 191 i51 7/15/94 JTF-5 2.0 3.0 1.0 3.0 2.0 3.0 ....... ............11 ...... ......... . .... . . .. . . ...... ................. ..... 20 243 5/31/94 JTF-5 3.0 3.0 5.0 4.0 1.0 4.0 21 242 5/25/94 JTF-5 1.5 3.0 1.5 3.0 .... ...... ................-.1- .. ................ ......... .......... 22 244 6/6/94 Or 4.0 5.0 2.0 3.0 1.0 2.0 - 23 ~39 6/12/94 2.0 2.0 1.0 2.0 2.0 2.0 . . ............... ....... ....... ...................... ....... . ... ....... .... ... ... . ............ 24 1 4/1/94 - Orl S 2.0 2.0 2.0 2.0 1.0 2.0 25 230 5/17/94 JTF-5 2~O 3.0 3.0 4.0 240 ---- . ..................................... ....... ... ............. ...... . ............ . ....... 26 235 4/18/94- JTF-5 3.0 4.0 3.0 3.0 3.0 4.0 27 234 4/14/94 JTF-5 2.0 3.0 5.0 3.0 6.0 5.0 . ................... ...................... .. . ........................... .... ............. .. ........ ...... . . . . .. . .. . ... 28 , 4/11/94 -JTF-5 3.0 3.0 3.0 3.0 3.0 -3.0 233- 29 1 3/29/94 JTF-5 2.0 4.0 2.0 4.0 5.0 4.0 229 ........ .. .. ... ........ . ...... . ... ... . . .. .... . .. 30 228 3/28/94 JTF-5 1.0 2.0 3.0 4.0 3.0 3.0 31 227 3/24/94 JTF-5 3.0 3.0 4.0 5.0 3.0 3.0 ........... ....... - ----.. .......... - ............ .. --- 32 226 3/22/94 JTF-5 5.0 4.0 5.0 4.0 2.0 3.0 33 225 3/21/94 JTF-5 2.0 3.0 3.0 3.0 2.0 3.0 .. ................ . ....... . 34 1 4/11/94 USA FC1 2.0 4.0 5.0 4.0 5.0 4.0 232 Act. 35 236 4/26/94 USA FC1 6.0 4.0 6.0 2.0 Act. 36 237 4/26/94 USA FC1 5.0 4.0 5.0 4.0 Act. 37 241 4/27/94 USA FC1 3.0 4.0 2.0 4.0 Act. . .. - ' .. . ....... . ............... - "'- - " - '- - - - -- . - 38 j4 2 - 1.0 3.0 3.0 3.0 3.0 3.0 7 ; I 4 Ej A ~7 G ~ A 9 39 265 7/6/94 61A/PAG-lA 1.0 3.0 2.0 3.0 2.0 4.0 .. .... ....... ......... ... ........ ........... . . . . ............... ............. . . . ....... 40 259 7/15/94 JTF-5 5.0 4.0 2.0 2.0 41 262 8/23/94 JTF-5 6.0 4.0 4.0 5.0 -- -"' ----"' ....... . . ...................... . ..... . . ........... .............. . .............. ..... ... . ..... ... .... .. . 42 287 4/3/95 JTF-5 2.0 4.0 1.0- 4.0 43 S--.- 106.5130 76.0 83.0113.5135.0296 348 ... . ........... ..........-......................... ....... 44 --------------------------------. ....................... 37 3--7-------26 . .................................... 45 . ........ 2.9 ------2-5 . 37 99~ 100 . .. ........ 3.5 1 3.6 3.0 Number 1. 0.7 37 - ----------- of entries 0.8 1.3 3.1 --- -- .8 = - - 1.4 = - - .6 - --I;t d. ------ Deviation 0.9 Approved For Rele%gd 1200qrqgAB:L, L9lf4Rt)P96-00791 R000200300003-1 VcoooocoozoooN ~U00-96d(3fV f 0?4§TOOZ 8SEGIGN J0=1 PGAoiddV ............. tl . J7 .. .. ..... . ...... . . ....... E .. . 1-1 . z .. ......... ..... L . . $ s ............. v ............. ........ 2 JODS AV ........ 0 ------- V 9 . ............ ... 9 ~C 9 S =SaIJIUD . ............0~' '_L 0.'..8..LJO 'ON ....... 0.,-9.z t, Z-L Z.- .. D ........ S s 1:~ =tl J7 _- . .. . ... .... SL 17 L 9 o'c . . ........ t76/6/ -Z L L o's O's WE 0, t-JiF t,6/E/ 69Z t, ....... .... c ............... L L E ... . ....... O'V o's O'E oll O'tlWE -V6/E/ 89Z E L L E - 0"t,WE 0, 0'E tl-ilf -K/C/t Z . .........-_..____................t7 ........ L E . .I----. . .. ......... . . . . .. . ...... O's O'V o'v o's O's 0'-V t7-jir V6/E/ L EZZ L - - L E - -v T t-jir t,6/E/ LZZ 0 t ........... .. C ...........L L E ................... -__.._.._-__.._l.... . . ................ . s o.v 0 0 O's V-JiF t,6/E/ OZZ 17 L L ......... . ..... ...... . . . ........ zz _ _ _ - - L 7 L L cf 'ON 9 ... F ........ . ....... z -- -- .......... . ................. .- -- .- ., . - - .- L ~ ~. ' ' 59 S 5 I [ 0 0 s z ~ 6 9 E S oll o'l O'tlo"Z 9-Jif S6/E/t, z8z t, .................... ... . z ....... O's O'tl O't70'9 s-jir V6/CZ/8 nz z - - - - - - (Tz I Y(fs S_Jir -V6/S L/Z 6S2 Z ........... C . .... ............ . .I_ ............I.......... Z ............. Ft ......... ........ ' -_ .. - f 6 s-jir V6/ L Z/C szz L WE O'Z 0'-Vo's O'V z S_Jir V6/~Z/E 9 2 z O's ....... z o . _- .... 2 .............. -..- - ot 0-E 0-s o.-vo-E 0.2 S - t,6/VZ/E ZZE 6 L WE WE 0'V WE O'Z O'L S_Jir V6/8Z/E ezz 8 . l.__._-. L . .............. .............. ......... oll O's O*tlO'Z O'tl0'Z S_Jir VUH/E UZ Z L WE WE WE 0'E WE WE s-JIF t,6/ L EEZ 9 ... ........-.l._..__- --.-................... ..............L L . -- ............ .. . .... . . . ..... ..... . ................ 0"s 0,9 WE O's 0'2 07 S-Jif t,6/V L vu 3 A, L 0, 0, O's WE 0't,0E 5-jir t76/8 L _Z V .. t, E ...................-....... ........_..... . . L . ..... . -.. . ...... .......... . . . ..... ....... .................. .... . ...... ........ ... ..... . . oll 0.,16 0'6 07 5-JiF t,6/Z US 0 VZ 6 L WE 9' WE S'L 9-JiF VUSZ/g ZtZ Z L ....................... ................... .... . L . . ............ .......... ....... ...... . ---- .. 017 0' 01, 0, WE WE 9-jir -V6/ L D12 L L s C/S L WE 07 WE 0, WE 07 S_~ir VUS L/Z LSZ 0 ................ . ---.1............L --... ... ........... . . .. . ........ L .. ..... - .......... ......... -... ............ -- ...... WE 07 oll 0'-V S_~ir V6/8 L/Z zsz 6 - O'tlO's WE 0, s-dir t,6/BZ/ 8 .. .......... ... .. . . . . . ..... ............ .. . ...... o's O'E S_Jir V6/ L/8 zsz L O-E O-Z WE 0, S_Jir t,6/C/ 9 L Aoua bV E b U I IS 8 J -A AE AZ V?. A VL z V L SlelojL Sajoas J0jenjBA3jJ;qjse_L Die(] '30(] L 12 JamalA 910wg~l _ r I H 9 j 3 Cl G V I 1 F SNoavmVA3 IVOIU3wnN vcooooUqAyotkgjML" C§4gYA/qjXM"LLt"P4qGAojddV dula wa Z-as"101a Approv-,q for ReAqflb2WXf!/fA?kgD%jQf 9 Wkf~O 3 0 0 0 0 3 -1 tj P F- NUMERICAL EVALUATIONS A B C D E F I G I H I I i K I L 46 47 Doc. Date Ta-sker Evaluator Remote Viewer & Scores Totals 48 1 1 A IV 2A 2V 3A 3v A V 49 By Tasking Agency .. . . ...... ........ .... . ...... .......... ............. ..... ..... .......... ................. 51 DTA . ..... . .... .......... 52 1.0 4.0 2.0 2.0 2.0 4. 0 53 7/6/94 DIA/PAN-3E f) A 248 :~o 3.0 ............ 0 ............ -541 245 6/24/94 DIA/PAN-3E 1.0 0 55 247 6/29/94 61A/PAG-1A I. . . . . . . ............. . . 561 265 ...... . ..... 79 4 DIA/PAG'-"l A 1.0 3.0 2.0 1-3.0 2.0 1 4.0 57 9.0 10.0 9.0 L 9.0 45 ....... ....... ..................... ....... ... .. . 58 N 5 5 4 -4 5 14 ~~8 41.9 3.2 59 3 .8 - - - ------- - ------------- -- - --------------- . . ................. ... .... ......... 60 61 Org. 5 62 102 7/13/94 0 3.0 3. Q 2.0 3- r9---S 1 4.0 5 63 lol 9/6/94 0 S 2 n q n Iq 0 4 -- ........... ---- . ................... ..... T-A --. - 70--t 64 82 6/6/94 0r2--S ,~OL -,- 65 81 6/12/94 S 2 66 79 4 1 9 4 Org. S 2.0 2.0 2.0 2.0 1.0 2.0 67 S 11.0 12.0 9.0 13.0 13.0 15.0 33 40 ............ ... .. ...........* 68 N 4 4 5 5 5 5 14 14 69 2.8 3.0 1.8 2.6 2.6 3.0 2.4 2.9 . ................ ............... ........... SA- +-- I- 1 U ct. 7 ... . ........ . 72 232 4/11/94 USA FC1 Act. 2.0 4.0 5.0 4.0 5.0 4.0 73 236 4/26/94 USA FC1 Act. 6.0 4.0 6.0 2.0 . ... . .......-. . ..... . .. . ........ --- ............ .. ..... ...... ...... 74 237 4/26/94 USA FC1 Act. 5.0 4.0 5.0 :1. 75 241 4/27/94 USA FC1 Act. 3.0 4.0 0 76 S ~;-o if; - . - - - - W 39 34 77 N 4 4 2 3 9 9 . ........ . ..... ...... .... ................... . ........ . ... ...... ..... . . .. 1-4 . ... ........... .... ....... 1. .-. 78 4.0 4.0 5.0 3 4.3 3.8 7 9 . .......... . .... ........ . ...... . . ........ . .. ............ 80 81 82 . . ....... ---"Comparison-"----A-1 ra -SE6F6~ Uy' .... .. -0-r, ganiza-fli on 83 Omanization Avera e -icores . . . ........ ............ . . ...... . ........ . . .......... 841 JTF-5 2.8 3.4 3.3 3.5 2.9 3.6 85 JTF-4 3.6 4.2 4.0 3.2 4.5 4.3 86 D IA 1.8 32 2. 2 2.5 1 8 3.8 87 8 3 0 1.8 2 6 2.6 3.0 ..... ....... .. ............. . ....... 5.0-T'- 88 USA FC1 Act 4.0 4.0 3.3 Approved For Release 2000TA41P 4,&?-RDP96-00791 R000200300003-1 Approved For Release 2000/08/10 : CIA-RDP96-00791 R000200300003-1 DRAFT UNCLASSIFIED measure. If the Accuracy of the information is somewhat uncertain, its Value must be vaguer still, i.e., scored lower. This presumption is supported by review of the scored "pairs for all cases, e.g., 1 A and 1 V; only rarely does the "V" score equal or exceed the "A" score for a specific RV and target. Note further that of the 100 "V" scores shown on Table 4-1, there are no 1" scoreS4 while the 99 "A" scores include 13 1's". Regarding the latter, a detailed review of the evaluator comments and/or the tasking suggests that the importance of these 1's is less than the score would imply in all but 4 cases since: - the evaluator of Document 243 stated that the RV 3A score "...though vague, is probably correct." - the tasking and targets for Documents 245,247, 248, 249 and 2655 concern topics widely publicized in the open media during the same period, hence the "source" of the RV 1 A and 3A comments, intended or not, is suspect, and - for Documents 2X 239 and 244, the evaluator's supporting narrative6 is inconsistent with the 1" score (and Org. S evaluators score higher - see comments later in this section). Another comparison between the Accuracy and Value scores can be made by considering the standard deviations for each. In all cases, the normal distribution for Value scores has a much sharper peak than for the Accuracy scores, i.e., the 1 (Y values are substantially smaller. This implies a better, albeit independent, understanding or agreement across all of the evaluators for the scoring of Value. 4 The significance of this omission is further enhanced it one assumes that the evaluators were familiar with the definitions in Table 3-2 since even those 11 instances scored as #2 ("High value") merely require that the input be the "best report to date or first report on this important topic, but [it] did not significantly influence policy or change analyses." 5 The evaluation of Document 265 is actually a second evaluation of the same RV inputs provided for Document 248 but done by a different evaluator and many months after the first evaluation. 6The following were the comments from RV 3 regarding Document 244: "- The site seemed to be crowded. There were people walking down a narrow'ramp' and they were dressed casually and looked serious. These people were outside and it was warm. - There were lots of animals at the site. The animals were outside sifting. - There was one area that was isolated and this area has low rolling green hills with several small structures spread apart sitting on the land." The evaluators comments regarding this input and for which an Accuracy value of 1 was granted were: "Site is crowded. Prison is full of prisoners moving back and forth between buildings. Casually dressed is true of prison dress and a prisoner does look serious and it was a warm day. One prisoner was feeding a cat outside by the fence. It must be noted that there was not a lot of animals in this area. This area is isolated and does have low rolling green hills with several small structures spread apart sitting in different types of confinement locations thru-out the prison complex." UNCLASSIFIED DRAFT Approved For Release 2000/08/10 : CIA-RDP96-00791 R000200300003-1 Approved For Release 2000/08/10 : CIA-RDP96-00791 R000200300003-1 DRAFT UNCLASSIFIED 30- 50- 40- .......... N20- U U30- M M b b20---- elo- e r 10- A 0__1 1 2 3 4 5 6 1 2 3 4 5 Scoring Factor ScodngFactor Histograms of Evaluator Scoring A review of the data is Table 4-2 provides several other observations: -The average scores from the JTF-4 evaluators averaged 0.7 to 1.5 marks lower than those of the JTF-5 evaluators in all but one case (i.e., scores for 2V) even though the targets were similar, albeit in different hemispheres. Since the tasking was very similar, this discrepancy raises the suspicion that the difference is due to different evaluators views of data provided from the Star Gate source and/or different interpretations of the scoring criteria, i.e., the subjective nature of the whole process makes accountability and evaluation difficult (see Section 5.0). - Conversely, the two principal JTF-5 reviewers were quite consistent in their average scoring considering the general uncertainty in the whole process. It would be interesting to know whether these evaluators were provided common guidance regarding how to interpret the measures; it is unfortunate that they never scored the same information. - The highest scores for Accuracy occurred for the DIA tasking (these tasks received 5 of the 13 "1's" for Accuracy), but as noted above, this tasking was directly relevant to information readily available in the open media during the same period which may have, knowingly or unknowingly, biased the RV derived information. - The marginally higher average scores for Org. S may result from either (or both) of two possible causes: 1) the use of "beacons" to support the RV efforts or 2) the differences in the evaluator interpretations of the scoring measures (two of the Org. S evaluators - "We" and "Kn" - score Accuracy and Value from 1 to 3 levels higher than UNCLASSIFIED DRAFT Approved For Release 2000/08/10 : CIA-RDP96-00791 R000200300003-1 Approved For Release 2000/08/10 : CIA-RDP96-00791 R000200300003-1 DRAFT UNCLASSIFIED any other evaluator from any organization) or other subjective causes. - The cause of the exceptionally low scores given by the USA FCI Activity evaluator is unknown but may be due to an honest appraisal of the data or a personal bias against the source by the single evaluator (among other possibilities). 5.0 GENERAL COMMENTS ON THE PROCESS Several general observations were derived by the author from the review of the operational tasking folders, each of which included the information derived by the RV's as well as the subsequent evaluator scoring and comment sheets. As noted earlier, most tasking was nonspecific in terms of the information desired. Most task targets were generic, e.g., a name and nothing else. Knowledge of the identity of the tasking organization may have telegraphed the type of information of interest. The result of this approach, in general, was that much of the RV-provided information was an unstructured discourse on a variety of topics. It frequently had the characteristic that I would associate with "free association or stream of consciousness" The combination of the broad, unspecified nature of the tasking in conjunction with the resulting unstructured information from the RV makes evaluation difficult if not impossible. The evaluation process is subjective and, thereby, influenced by the beliefs, interests, whims, and fancies of each evaluator. Conversely, any "telegraphed" information acquired by the RV7, whether by accident, inference or intention, makes Accuracy scoring suspect since some "accurate" information can be provided based upon that knowledge8, e.g.," the target is involved in the drug culture" is almost certainly true of any targets specified by the JTF's. Correspondence between portions of the RV-derived information and the interests of the evaluator can be completely illusionary since the communication channels between the RV(s) and the evaluator: - has a very narrow information bandwidth, i.e., the RV-derived information cannot be embellished by a dialogue with the evaluator without substantially telegraphing the evaluator's needs and interests, thereby biasing any RV information subsequently derived , and - is extremely "noisy" as a result of the unidentifiable beliefs, intentions, 7 Telegraphed by knowledge of the tasking organization and its interests and concerns or by tasking that relates to highly publicized media information during the same time period. 8 In addition, "Accuracy" is a necessary but not a sufficient condition for determining the utility of the information since "accurate information " may already been know by the tasker or, even though accurate, may not have any utility for the tasker's problem. UNCLASSIFIED DRAFT Approved For Release 2000/08/10 : CIA-RDP96-00791 R000200300003-1 Approved For Release 2000/08/10 : CIA-RDP96-00791 R000200300003-1 DRAFT UNCLASSIFIED knowledge, biases, etc. that reside in the subjective consciousness of the RV(s) and the evaluator . As a result, the potential for self-deception on the part of the evaluator exists, i.e, he/she "reads" into the RV information a degree of validity that in truth is based upon fragmentary, generalized information and which may have little real applicability to his/her problem. The relevant question in the overall evaluation process is who and what is being evaluated, i.e., is the score a measure of the RV's paranormal capabilities or of the evaluators views, beliefs and concepts? One of the RV's expressed a concern to the author that the protocols that were followed in conducting the RV process in response to the operational tasking were not consistent with those that are generally specified for the study of paranormal phenomena. Whether the claimed discrepancy was detrimental to the information derived by the RV's,or to its subsequent evaluation or use is not derivable from the available data. The operational tasking noted earlier concerning activities in North Korea which required precognitive abilities on the part of the RV's provides an opportunity for a post-analysis by comparing the RV predictions against subsequent realities. Additional comparative data of this type is available from operational tasking during the period 11/90 through 1/91 regarding the Middle East situation (this tasking was not numerically evaluated). 6.0 CONCLUSIONS AND RECOMMENDATIONS 6.1 Conclusions The single conclusion that can be drawn from an evaluation of the 40 operational tasks is that the value and utility to the Intelligence Community of the information provided by the process cannot be readily discerned. While, if one believes the validity of parapsychological phenomena, the potential for value exists in principal, there is, none-the-less, an alternative view of the phenomenology that would disavow any such value and, in fact, could claim that the ambiguous and subjective nature of the process actually creates a need for additional efforts with questionable operational return on the part of the intelligence analyst. Normally, much of the data provided by the RV(s) is either wrong or irrelevant although one cannot always tell which is which without further investigation. Whether this reality reduces or eliminates the overall value of the totality of the information can only be assessed by the intelligence analyst. It clearly complicates his/her problem in two ways: 1) it adds to the overburden of unrelated data which every analyst already UNCLASSIFIED DRAFT Approved For Release 2000/08/10 : CIA-RDP96-00791 R000200300003-1 Approved For Release 2000/08/10 : CIA-RDP96-00791 R000200300003-1 DRAFT UNCLASSIFIED receives on a daily basis, i.e., the receipt of information of dubious authenticity and accuracy is not an uncommon occurrence for intelligence analysts, and 2) since the analyst does not normally know which information is wrong or irrelevant, some of it is actually "disinformation" and can result in wasted effort as the analyst attempts to verify or discount those data from other sources. The review of the operational tasking and its subsequent evaluation does not provide any succinct conclusions regarding the validitV of the proc (or the information provided by it). First and foremost, as discussed in Section 5.0, the entire process, from beginning to end, is highly subjective. Further, as noted in Section 3.0, the degree of consistency in applying the scoring measures, any guidance or training provided to the evaluators by any of the tasking organizations and/or the motivation or interest of the evaluators are all unknown. The lack of information regarding these items could account for some of the variability in the scores across organizations noted in Figure 4-2, but this cannot be certified and is, at most, a suspicion. Whether the information provided by the Star Gate source is of sufficient value to overcome the obvious detriment of accommodating the irrelevant information included therein is an open question? More precisely, whether the Star Gate information is of sufficient value to continue this program - vis-a-vis other sources of information and other uses of resources - is an important question for the Intelligence Community to address, irrespective of one's personal views and/or beliefs regarding this field of endeavor, i.e., does the information provided justify the required resource investment? One method that might assist this evaluation is to develop a means for scoring the complete input from the RV process, i.e., evaluate all information and determine how much is truly relevant, how much is of undeterminable value and how much is completely irrelevant. One could then analyze how much information is being handled to achieve the relevant information (along with some measure of the relevancy) and make judgments on its value vis-a-vis the investment in time and money. Other, less technical methods, for adjudicating this issue also exist. 6.2 Recommendations Considering the statements above, the only sensible recommendation in this author's mind is to bring some "scientific method" into this process (if it is continued). As evidenced by more than 20 years of research into paranormal psychology, much of it done by institutions of higher education or others with excellent credentials in related fields, validation of parapsychological phenomena may never be accredited in the sense that is understood in other scientific and technical fields of endeavor . Control in any rigorous scientific sense of the multitude of human and physical variables which could, and probably do, influence this process is difficult - perhaps impossible - for any UNCLASSIFIED DRAFT Approved For Release 2000/08/10 : CIA-RDP96-00791 R000200300003-1 Approved For Release 2000108/10 : CIA-RDP96-00791 R000200300003-1 DRAFT UNCLASSIFIED illusionary, interpretation regarding both accuracy and value. If some specificity regarding the target could be defined such that the relevance and accuracy of the RV- derived data could be evaluated more readily, some of the uncertainties might be eliminated. In this context, note that the cases where targets were more specific, e.g., the NK targets , the resulting scores were generally higher. Finally, it was noted in Section 5.0 that some of the RV information obtained from operational tasks regarding North Korea (and others concerning the Middle East) depended upon the precognitive ability of the RV's in predicting events yet to occur. These data provide an opportunity for a post-analysis of the accuracy of these predictions by making a comparison with subsequent information regarding actual events (some data for this comparison might require access to classified information from other sources). Such a post-analysis would provide data for evaluating the ability of the RV's to perform precognitive tasks and of the related operational value of the predictions. Performance of this post-analysis lies beyond the scope of this paper, but is a topic for a subsequent study if any sponsor is interested. UNCLASSIFIED DRAFT Approved For Release 2000/08/10 : CIA-RDP96-00791 R000200300003-1 SG1 B Approved For Release 2000/08/10 : CIA-RDP96-00791 R000200300003-1 Next 1 Page(s) In Document Exempt Approved For Release 2000/08/10 : CIA-RDP96-00791 R000200300003-1