Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 PfiG-TH- I 068-SL DEFENSE INTELLIGENCE AGENCY PROJECT STHR GHTE RESEHRCHnnD PEER REVIEW PLHn(u) MRY 1994 NOFOEN SECRET LIMDIS STHR GHTE Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 SECRET 10 PROJECT STAR GATE RESEARCH AND PEER REVIEW PLAN (U) This document was prepared by the Technology Assessment and Support Activity Office for Ground Forces Directorate for Military Assessments National Military Intelligence Production Center Defense Intelligence Agency Date of Publication May 1994 REPRODUCTION REQUIRES APPROVAL OF ORIGINATOR OR HIGHER DOD AUTHORITY WARNING NOTICE-INTELLIGENCE SOURCES OR METHODS INVOLVED LIMITED DISSEMINATION FUTHER DISSEMINATION ONLY AS DIRECTED BY DIA/PAG CLASSIFIED BY MULTIPLE SOURCES OR HIGHER DOD AUTHORITY DECLASSIFY ON OADR. SECRET NOT RELEASABLE TO FOREIGN NATIONALS STAR GATE Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 UNCLASSIFIED OUTLINE PAGE EXECUTIVE SUMMARY ................................... 1 I. INTRODUCTION .................................. 2 ii. PLAN OBJECTIVES ............................... 3 III. SIGNIFICANCE OF EFFORT ........................ 3 IV. PLAN OVERVIEW ................................. 5 V. BASIC RESEARCH PLAN FOR ANOMALOUS COGNITION... 6 Vi. BASIC RESEARCH PLAN FOR ANOMALOUS PERTURBATION. 14 VII. APPLIED RESEARCH PLAN FOR ANOMALOUS COGNITION.. 17 SG1B IX. POTENTIAL RESEARCH RETURN ...................... 24 X. PROJECT OVERSIGHT METHODOLOGY .................. 25 Xi. DEVELOPMENT OF EVALUATION CRITERIA ............. 26 XII. BUDGET AND RESOURCE REQUIREMENTS (FYs 95-99) ... 27 APPENDICES A. CONGRESSIONALLY-DIRECTED ACTION, DEFENSE AUTHORIZATION CONFERENCE ...................... A-1 B. TERMINOLOGY AND DEFINITIONS ................... B-1 C. POTENTIAL RESEARCH SUPPORT FACILITIES ......... C-1 D. RESOURCE LITERATURE ........................... D-1 E. CURRENT CONTRACTOR SCIENTIFIC OVERSIGHT COMMITTEE MEMBERSHIP .......................... E-1 F. CURRENT CONTRACTOR INSTITUTIONAL REVIEW BOARD ................................. F-1 G. ACADEMIC STUDIES REGARDING THE SCIENTIFIC VALIDITY OF AMP .............................. G-1 H. AN ASSESSMENT OF THE ENHANCED HUMAN PERFORMANCE PROGRAM .......................... H-1 I. IN-HOUSE STAFFING REQUIREMENTS ............... I-1 UNCLASSIFIED Approved For Release 2003/04/18: CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 SECRET (U) EXECUTIVE SUMMARY: (S/NF/SG/LIMDIS) In compliance with the congressional conferees' request (Appendix A), DIA proposes to develop a multi- year research and development program, subject to rigorous scientific and technical oversight, to demonstrate the scientific validity of.the STAR GATE program, and that results of military and intelligence value can be obtained in a cost-effective manner using anomalous mental phenomena (AMP). (S/NF/SG/LIMDIS) This proposed program, if successfully implemented, will: Identify the underlying mechanisms of AMP. - Establish the limits of operational usefulness of AMP. - Determine the degree to which foreign activities in AMP represents a threat to national security. - Lead to the development of countermeasures to neutralize this threat. - Use research findings to improve operational activities. - Develop data fusion criteria to integrate AMP results with other intelligence sources. (S/NF/SG/LIMDIS) Due to the diversity of the STAR GATE mission/objectives, both external resources and in-house expertise are required. Since this Activity possesses no in- house R&D capability, an absolute need for external R&D support is required to meet Congressional concerns which are addressed in this program plan. A balance will be maintained between external and in-house activities, and every effort will be made to integrate and link these activities where appropriate. The external aspect permits a wide range of expertise covering many disciplines to be focused on this area; this also has the benefit of ensuring peer group review and of facilitating a variety of scientific interactions. In-house personnel with a wide-range of expertise in this phenemenology will need to be retained to make this proposed plan work. (SINF) In order to review the major tenets of the draft program plan, the Defense Intelligence Agency will convene a panel of appropriate scientists to provide recommendations on the plan and the research it achieves. Based on the panel's recommendations, the Defense Intelligo-nce Agency will then submit a budget line item to fund those appr.oved objectives. SECRET NOT RELEASABLE TO FOREIGN NATIONALS STAR GATE LIMDIS Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 SECRET (C) An annual report will document the current operational, technical and administrative status of the program. I. (U) INTRODUCTION: (S/NF/SG/LIMDIS) This program plan was developed in response to a Defense Authorization Conference, Congressionally Directed Action (CDA) to prepare a long-term systematic and comprehensive research and peer review plan in order to investigate anomalous mental phenomena (AMP), and to apply program research results to potential operational activities. This plan also describes key in-house activities along with an appropriately integrated basic and applied external research support effort. (S/NF/SG/LIMDIS) Specifically, this program plan represents DIA's view on how best to proceed with both in-house activities and external research support for the period of FY95 through FY99. Research findings, both domestic and foreign, and results from operational activities may lead to updates of this plan in order to reflect improved phenomena understanding and to pursue follow-on research and/or application directions. (S/NF/SG/LIMDIS) A underlying and fundamental premise governing the implementation of this program plan is that a well- integrated interdisciplinary approach is considered to be the most appropriate strategy for conducting research in this diverse field. Consequently, this plan includes a wide variety of research topics which are based on recent findings from leading- edge pursuits in other disciplines that are suspected of being germane for STAR GATE. other topics are derived from a review of worldwide research, consultations with leading area experts, and on insights gained from previous research and application activities associated with the STAR GATE program. (S/NF/SG/LIMDIS) This program plan also allows for the STAR GATE program to show results that are cost effective and will at the same time satisfy reasonable program performance criteria. The implementation of this program plan will preclude the reoccurrence of the yearly cyclical activity of project start-up, limited progress, followed by anticipated project shut- down which previously inhibited program activity. (S/NF/SG/LIMDIS) In sum, the implementation of this 4W research and peer review plan will allow DIA to successfully accomplish identified R&D activities which, in-turn, will enhance the capability of STAR GATE personnel to engage in operational activities and to assess the work done by potential adversaries, thereby, reducing the risk potential for a technological surprise. SECRET NOT RELEASABLE TO FOREIGN NATIONALS STAR GATE LIMDIS 2 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 SECRET (U) Terminology and definitions are discussed at Appendix B. Ii. (U) PLAN OBJECTIVES: (S/NF/SG/LIMDIS) The objective of this follow-on research and peer review plan is to further develop phenomena understanding and/or validation, in applications understanding, and in operational feasibility evaluation. This continued work will have a direct bearing on DIA's ability to both assess the significance of foreign research and to perform a systematic review of potential applications regarding this phenomena. (S/NF/SG/LIMDIS) Accomplishment of the various activities identified in this plan will further enhance threat assessment of foreign achievements in this area, and will help achieve the potential for U.S. military/intelligence applications on select tasks as a supplement to HUMINT operations. (U) It is anticipated that this plan will assist decision makers in their review and consideration of future directions for this field, and that this plan can begin formal implementation starting in FY95. (S/NF/SG/LIMDIS) In compliance with the congressional conferees' request, DIA recommends that a period of six to nine months be set aside at the beginning of this new program for the purpose of identifying the most promising and cost-effective experiments to be conducted under the program to meet the overall research objectives outlined below. It is further suggested that a series of small working groups consisting of scientific experts from a variety of pertinent disciplines meet during this time period to accomplish this end. III. (U) SIGNIFICANCE OF EFFORT: (S/NF/SG/LIMDIS) STAR GATE is a dynamic approach for pursuing the largely unexplored area of human consciousness and subconsciousness interaction. Its scope is comprehensive; a wide range of phenomenological issues are examined that include psychological, physiological/neurophysiological, physics and other leading-edge scientific areas. Although broad in scope, STAR GATE is well grounded due to its solid independent scientific review base. STAR GATE is based on a dynamic style in all its endeavors, especially in its pursuit of on-going foreign activities in this area. (S/NF/SG/LIMDIS) One of the tasks previously levied on DIA by the FY91 Defense Authorization Act was to develop a long-range comprehensive plan for investigating )@arapsychological phenomena. This task was one of several objectives included in a new program for this phenomenological area that identified DIA as executive SECRET NOT RELEASABLE TO FOREIGN NATIONALS STAR GATE LIMDIS 3 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 SECRET SG1 B agent. Moreover the FY91 Defense Authorization Act authorized for DIA a funding level of $2 million for DIA in order to initiate this new program. As a result, a balanced and integrated plan to include operations, foreign assessment, and research and development was implemented . In addition, a new DIA limited dissemination (LIMDIS) program, codeword STAR GATE, was established in order to accomplish the objectives that were set forth in this plan. (S/NF/SG/LIMDIS) The external research support conducted under monies appropriated to date comes to a close in the June 1994 time-frame. The impact of this is that if research activities utilizing human subjects are interrupted, it has generally been necessary to begin again instead of later resuming activities from the point of termination. Consequently, it is important for the STAR GATE program to remain stable. Research involving human use differs considerably from that involving physical systems. For example, data from human subjects cannot be collected nor analyzed as rapidly, in that additional empirical data is often required to reach analytical conclusions. This type of data analysis utilizing human subjects can only be achieved with an in-place, uninterrupted, multi-year research and development program. Therefore, should it be decided to go forward with this program, it should be done in a timely fashion. (SINF) The funding allocation for external research received by STAR GATE in FY91 and continued through FY 1993 permitted several important research areas to be initiated and continued. It is anticipated that results of this research will assist in clarifying some of the possible future research directions; consequently, not all long-range research possibilities can identified in this plan. However, most all of the major investigation areas can be addressed, and many of the specifics can be identified with reasonable confidence. Figure 1 presents an overview of overall research objectives for both Anomalous Cognition (AC) and Anomalous Perturbation (AP) which will be considered for inclusion in this program. SECRET NOT RELEASABLE TO FOREIGN NATIONALS STAR GATE LIMDIB 4 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 SECRET (SINF) Previous basic research activities from FY91 through PY93 focused on the following; (1) validating findings from previous magnetoencephalograph (MEG) research and initiating new work with a variety of conditions and individuals; (2) performing a variety of anomalous cognition (AC) experiments to determine potential correlations (e.g., target type, environmental factors); (3) developing various theoretical constructs that might be testable and that could help explain the phenomena; (4) examining effects of altered states on data quality; (5) initiating review of and research into the energetics area; and (6) examining various application possibilities (e.g., communication, search). (U) Results from previous basic and applied research activity have been factored into this research and development plan and provide the basis upon which further R&D efforts will be built. IV. (U) PLAN OVERVIE A. (U) BASIC RESEARCH OBJECTIVES conservative approach generalizes to understand the source of AMP and its propagation mechanisms (Figure 1). (S/NF/SG/LIMDIS) The objective of basic research is to understand the fundamental, underlying mechanisms for AMP. To achieve this objective in an efficient way, basic research of the detection mechanism should begin in a conservative direction. That is, assume that a putative "sensorial" system exists for AMP and that it most likely will behave similarly to those common elements which are known through the five senses. This B. (U) APPLIED RESEARCH OBJECTIVES 40 (S/NF/SG/LIMDIS) The objective of applied research is to improve AMP functioning to its maximum possible limit. To realize this objective, it is critical to define AMP output measures that are consistent with either a laboratory setting and/or an operational environment. The approach should also reflect scientific conservatism. In investigating any single variable (e.g., different training methodologies) all other variables should remain as constant as possible (e.g., use the same individuals and known good target systems). C. (U) FOREIGN ASSESSMENT SUPPORT OBJECTIVES (SINF) From a research perspective, the objective of foreign assessment is to determine the degree to which claims from foreign laboratories can be confirmed in a U.S.-based setting. In science, replication is ctitical for understanding. SECRET NOT RELEASABLE TO FOREIGN NATIONALS STAR GATE LIMDIS 5 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003104118 ANWMIWAR002700020001-0 I Anorhal6us Phphor@olia 11 Cogrillion 1.0 Detector 2.0 Transmissio tal Nervous System ant C NeuroN'et Models Autonomlo Responses Inter-*species 06mmunloations Other Animals j @2 AppIte d Other Physiology (Skin) Pets onailty(B ehAv;otAVS OnAe port-a SorVMBT1) Perooptual Modeli psychol4loil Models (MoUvationjErnotIon) Selection (Dire cqCorrelatlo n al) Eny1ronmant-Physloal (GMF) Enyironme ni-Lftycho logical (Set and Setting) EnvItonmehI--P'hysldIogIW (06mfort) ArUflotal Response Type (AudloNld6olLeft Hand) Redundancy (Multiple Pas s/Multiple, Detectors) Communication ,knaly-313 1.3 Mixed internal Noise Source Training (M acroscopiclOp a rant) Session Prolowls 3.0 Source 2.1 Q11216 Informatlonlal (Enttopy/MeanIng) Other Thermodynamlo Vactor/Scalar Potential Soundarles Human Sander Dem&rcatlori (Coordinate s1B e.aoon) External Nolsi Source Inverse ($4rch) Physical Charablerl3ilos (Sizeloomposhlon) T@pq (SIAU64@ArNo) One-th-W (Fprced Cholqe/Blnery Decision Augmehtillon Th"ry Worm Holes (4--Vrri@nsl6ns) Vaetor/Scal&rPotentlal0topt~ttlon 2.0 Macro. 3.0 Mixed 2.1 Basta P102041104-WO Strain Gafige Rest" StMIn Gauge Metal (Bandlhg) Pendulums Nne&r1TbrsIonM$oIog$*q Mochanloal "terns (BaII3Antetferomeieri) 2.1 Apollid Inertial "tan@s Atoms Nuclei (Mossbaper Eff act) Photons 00113 (Algae, Blood) Bacteria (Mutatlo re-9 almon alla.) Quantum Sryslams (Neu&on/Pholon Iriterfatence) OryslaVstructure . Mot ecular Structurs 'OR Spectra of H20) Theory (Qu&ntum Weasuremenl/Zeno) Random Number Generators (Electronlc/Nucleat) Magnello 06mlIns J.n Mixed Electrons 2.1 Applied Mlc@omkchlnes Figure I (U) Research OvervieW Approved For Release 2003/04/18: CIA-RDP96-0 0,2700020001-0 UNCLUffitu III PertUrbation 1.0 Micro Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 SECRET V. (U) BASIC RESEARCH PLAN FOR ANOMALOUS COGNITION: A. (U) BASIC APPROACH (SINF) The link of basic and applied research with other applications investigations or with research activities is shown on Figure 2. The top of the chart shows that for any research or application task, certain conditions must be met (e.g., a reliable calibrated individual is required; proper scientific procedures need to be developed, etc.). Once these basic foundations are laid, then basic/applied research can be initiated with a reasonable expectation of success and with assurance that results will not be ambiguous or fail scientific scrutiny. (SINF) This chart also illustrates the difference between basic and applied research; applied research relates to various methods for collecting, recording, improving and analyzing data output, while basic research is aimed at phenomena understanding. In this chart, the "detector" is the human brain/mind, the "source" is the target or an aspect of the target, and "transmission" refers to notions of how information and/or energy are actually transmitted between source and detector. (U) Figure 3 illustrates the interdisciplinary scope that will be brought to bear on this research problem. Leading- edge researchers in their various fields can provide clues, if not make direct contributions, that will assist in phenomena and applications understanding. Appendix C lists candidate research support facilities that could be involved in this long-range effort. Appendix D outlines pertinent research literature applicable to this field. Final selection will be based on how well the activities if these institutions will fit into specific time-lines and priorities to be established in FY95. Figure 4 lists milestones for the anomalous cognition basic research to be conducted under this plan. B. (U) RESEARCH DETAILS 1. (U) Source. (S/NF/SG/LIMDIS) Source research will address those topics that show promise for understanding the characteristics of the target or target area that may play a role in anomalous cognition (AC) occurrence and data quality. Aspects of the target that can be defined by conventional information theory (involving entropy/information content) will be explored in-depth. A wide variety of targets with a wide range of information content, dynamics, or other parameters will be examined to explore this possible link*. If not successful, other SECRET NOT RELEASABLE TO FOREIGN NATIONALS STAR GATE LIMDIS 7 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 Cl@-RDP96-00789ROO2700020001-0 UNCLIRISSIFED 0" Ca Mqa@"wo3itk :far- Spocifzic .E9-- racalmor Tatvo-(-- u,3x protocx3l :Gor J)zLtav CaElDctlDn 0,3td -hL ODUALtux I>ZxtaL JLfwC==GnI-- v, JntGg-,VaUOM CXF. 1100"ItEr Ebloation IK sdurc* VC nacx3ivax, mL-aining Vir Target- Soloction vt@ DE]t()CtOr w PrOtOC;OIr w * XnalyErin Vt Countarzoasurou Figure 2 (U) Research objectives IJKLIHISSIFIED Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96'00789ROO2700020001-0 UNCUSSIFIED RNaMfILOUS MENML ;9tatust;.1cuisiguz-a. Xn2a-yzriff PHINUMENa Oagu-i-t-i-yo lWycho1osry Figure 3 (U) Integration of Scientific Disciplines UNCLRSSIFIED Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 UNCLASSIFIED TIME FRAME flCTIUITY 1995 1996 1997 1998 1999 Information/Entropy SOURCE - - - - RESEARCH Analysis Various Target Attributes (TARGET) (Size, Form, Content) TRANSMISSION Four-Dimensional Calculations (Relativity Extensions) RESEARCH Unconventional Waves - (MECHANISM) 7 (Laboratory) (Long"Range Tests) Variables (Distance, Shielding, Energyl DETECTOR Neuroscience (EEC7, tlemory, Etc. RESEARCH Environmental Factors (BRAIN) Other Physiology (Electrical, Infrared) - - - - sear7o l R@ h /AniiWa Implications from Kedical ' Physical Sciences (Physics, Statistics, Parallel Processing, Etc.) Psychological Sciences INTEGRATION (Psychology, Anthropology, Cognitive, heni-a-il, Subliminal Perception, Etc.) ftedical (Genetics, Etc.) FIGURE 4 (U) BASIC RESEARCH MILESTONES - ANOMALOUS COGNITON 4i UNCLASSIFIED 10 Approved For Release 2003/04/18: CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 SECRET approaches to investigate the targets' innate nature and its possible link to phenomenon occurrence will be initiated. Definitive data in this area would also have implications for defining those targets which have the highest probability of successful data acquisition in an operational setting, thus establishing operational tasking parameters. 2. (U) Transmission. (SINF) The pursuit of possible transmission mechanisms for AC phenomena is essentially the most significant basic research task and also the most difficult to formulate. In this effort, a theoretical basis will be developed from extensions of current theory in light of recent advanced physics formulations. Some of these formulations permit unusual "information flows" that may, in fact, have relevance for this phenomenon. Testable models/constructs will be developed and evaluated. A variety of other possible explanations involving extensions of gravitation theory, quantum physics or other areas will be constructed and tested where possible. Some of these tests may require close cooperation of leading-edge researchers using equipment in their facility. (CINF) Effort in this area will also focus on integrating diverse aspects of the source, transmission, and detector categories. For example, it will examine how "targeting" occurs. Insight will be drawn from in-depth reviews of various unusual physical effects identified by physical sciences researches. These include distant particle coupling (Bell's theorem), ideas from quantum gravity, possible electrostatic/gravity interactions, unusual quantum physics, observational theories, vacuum "energy" potential, and a variety of other concepts. (SINF) Perhaps the most promising exploratory model of all is one based on little-understood aspects of the fundamental equations for electromagnetic wave propagation (Maxwell's equations). These equations indicate that forms of "wave propagation" could also exist that do not have the conventional electric or magnetic field components (i.e., vector and scalar waves). These waves would not be blocked by matter and therefore could be leading candidates for AC propagation or SG1B for certain aspects of AC phenomenon. Research papers SG1 B indicate that these waves are considered a leading candidate for AC transmissions by their researchers. Pilot study investigations in this area were conducted by PAG-TA in FY92 with promising preliminary results. Future research could couple with other DIA exploratory R&D efforts in this area currently being explored. SECRET NOT RELEASABLE TO FOREIGN NATIONALS STAR GATE LIMDIS 11 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 SECRET (SINFISGILIMDIS) Research on this topic will be closely integrated with research involving the anomalous phenomena (AP) aspect, since findings in the AP area would have direct implications for phenomena transmission mechanisms in general. Findings from the target (or target source) research area would also provide insight into possible transmission mechanisms. For example, different forms of the same target (e.g., target size, 2D vs 3D, holographic representations) may show patterns in the AC data that might provide clues regarding phenomena mechanisms. 3. (U) Detector. (U) The most important and promising aspect of understanding the nature of the AC detection system in humans is through modern advances of the neuroscience. Earlier neurophysiological results obtained from magnetoencephalograph (MEG) measurements begun in FY92 will be validated and expanded. This earlier work indicated MEG correlations between visual evoked responses areas of the brain may exist, and that remote stimuli might also be detectable in MEG data. Some of the specific investigations will examine a variety of near and far- field situations, other sensory modes and different types of individuals in order to search for potential variables. It might be possible, with advanced MEG instrumentation, to actually locate the exact brain areas involved in AC phenomena occurrence. Future research in this area could couple with research currently being explored at the National Laboratory. (U) Other physical/psychophysical aspects of the central nervous system (CNS) will also be explored to look for possible correlates. This would include galvanic skin responses (GSR) or other parameters. (U) Related to this overall area are several investigations that relate to possible environmental interactions with the brain that could affect AC data. This would include possible geomagnetic or electromagnetic influences. (SINF) A spin-off from findings in this basic research area could be for unique communication applications. MEG correlates might exist between remotely located people. If so, the possibility of transmission of remote messages ' (via a type of code) might be possible. Since AC phenomenon is not degraded by distance or shielding, the potential of transmitting basic "messages" to individuals in submarines would exist. Preliminary exploration of this application by PAG-TA has yielded promising results. (SINF) Another potential spin-off benefit from detector research in this program is that new insights into brain memory or parallel processing might be achieved. This could lead SECRET NOT RELEASABLE TO FOREIGN NATIONALS STAR GATE LINDIS 12 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 SECRET to new directions in advanced compute dampInDments involving neural networks. For example, recent _Jindicates that SG1 B "wave-like" brain activity occurs in additiont'd usual neuronal processes. This wave-like phenomenon may have some link to the "phase shift" observed in MEG data from the previous MEG project. Further MEG work involving remote stimuli may help clarify such issues. 4. (U) Integration. (U) The basic research activities will liberally avail itself of the existing research communities that specialize in neuroscience, physics and statistics and the broader psychological/social sciences. Direct support with a variety of university departments, national and international, will be explored. PAG-TA contacts with such national laboratories as Los Alamos, Lawrence Livermore, Oak Ridge, and have indicated an interest on their part in supporting the research efforts. Frequent conferences and data exchanges are anticipated. These data exchanges will insure that a proper interdisciplinary approach is maintained, and that findings from other disciplines will be incorporated in this program where appropriate. This peer group dialogue will greatly benefit research sponsored through this plan, new ideas will be generated, and possibly clues regarding phenomena operation will be easier to identify. (U) Some specific interdisciplinary examples that will benefit this program are as follows: - In 1990 The American Anthropological Association (AAA) formed a new division, the Society for the Anthropology of Consciousness (SAC). This division has established a technical journal to support interdisciplinary, cross-cultural, experimental, and theoretical approaches to the study of consciousness. This group may be able to contribute this program by providing cross-cultural examples. These members might also assist in the assessment of foreign data in this area. - The psychophysiology of vision has already contributed to the earlier program. This plan calls for a collaborative effort with researcher in an attempt to understand how the central nervous system process subliminal stimuli. This should assist in understanding how MEG correlates occur. The relationship between mind and body is currently discussed in the research literature as well as in the popular press. Researcher at the California Institute for Transpersonal Psychology (CITP) have been active in investigating the role of mental attitudes and body chemistry. While there may not be a direct link with AC, and exchange of techniques and experimental designs would be helpful.' SECRET NOT RELEASABLE TO FOREIGN NATIONALS STAR GATE LINDIS 13 Approved For Release 2003/04/18: CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 SECRET - The Journal of Cognitive Neuroscience contains at least one article of interest in each issue. This discipline is where most of the cognitive work with the neuromagnetism is conducted. There is the possibility of joint investigations with researchers performing MEG investigations at the National Institutes of Health (NIH). - Stanford University has been conducting research on internal mental imagery. The manipulation and control of this imagery is extremely important in understanding the source of internal noise during an AC session. A collaborative effort with Stanford should lead to methods for noise reduction. - Neural networks are particularly good at recognizing subtle patterns in complex data, and are being applied in the subjective arena of decision making in business. In order to improve AC analysis, the program will conduct a collaborative effort with scientists who are active in neural network research and with selected individuals who have had success with interpreting highly subjective data. - Statistics is the heart of AC research in that most of the results are usually quoted in statistical terms. Hypothesis testing has traditionally been the primary focus, but there are other possible approaches that should be explored. Statistics researchers at Harvard have already expressed interest in contributing to the research effort. - A major portion of the effort will be a search for a AC evoked response in the brain. sophisticated processing is required in that magnetic signals from the brain can not be easily characterized by standard statistical practices. Several research facilities can contribute. - Classical statistical thermodynamics may be the heart of understanding the nature of an AC source of information. A physical property called entropy may be related to what is sensed by AC. The program intends to collaborate with a variety of university physics departments to calculate the appropriate parameters. (SINF) The specific experiments to be conducted in these research domains will be defined during the first six to nine months of the program utilizing the recommendations of the 40 working groups mentioned above subject to approval by the Scientific oversight Committee. VI. (U) BASIC RESEARCH PLAN FOR ANOMALOUS PERTURBATION: (S/NF) Figure 5 illustrates th 'e@ basic approach for investigations "energetics", or anomalous perturbation (AP) phenomenon. Intelligence reporting indicates that this aspect of SECRET NOT RELEASABLE TO FOREIGN NATIONALS STAR GATE LINDIS 14 Approved For Release 2003/04/18: CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 UNCLI'ISSIFIED RCTIDITY TIME FRRME 1995 1996 199-7 1998 1 ggg DEUELOP EURLURTION CRITERIR PERFORM Historical Data Base flNflLYSIS EHFIMINE Various Technical Targets TflRGET SYSTEMS Laboratory Setting CONDUCT Advanced Sensors Complex Components URLIORTION EHPERIMENTS PURSUE Far-Field Effects (countermeasures) RPPLICHTIONS Solicit Known PERSONNEL Talent Screening/Training (Develop) SELECTION Figure5 (U) Basic Research Milestones - flnomalous Perturbation (To Include Biological Systems) UNCLRSSIFIED 15 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 SECRET SG1 B AMP hould receive attention in tHis research pian --o-pre =1@ical surprise. Thus, beginning in FY95, acceptance criteria will be establish with which to judge the historical literature for potential AP effects. Using those criteria, a detailed review of the literature will begin in mid FY95 and considering the size of that data base will continue through FY95. Knowledge gained from this review may provide insights for the development of new AP target systems or provide data so that particular experiments can be replicated. Given the complexity of most AP experiments, considerable time is needed to plan and conduct them properly. If the results warrant, then application development may begin as early as FY96; however the primary task of basic research of AP is to attempt to validate its existence. Findings from foreign research will be examined and factored into this activity as appropriate. (SINF) The keys to investigating this area will be in appropriate personnel selection and, very likely, in proper selection of the AP test device. Thus, the initial phase of this effort will involve identification and solicitation of individuals known or claimed to have such talents. For example, certain expert martial arts or yoga practitioners might do well in such experiments due to their strong mental conditioning and ability for intense mental focus. After locating such individuals, various instruments, such as microcomputer devices, sensitive electronic/sensor devices, or other unique or sensitive equipment would be used as targets in AP experiments. (SINF) Some of the unique sensor candidates include devices that are highly sensitive to very weak gravitational effects (such as Mossbauer devices or atomic clocks). Perhaps the most promising device is one that involves detection of an unusual non-electromaqnetic wave (A vector/scalar wave). If experiments with such sensors are successful, then significant understanding of AP or.AC phenomenon would occur. Experiments with such a device is a distinct near-term possibility; consequently this will be given high priority in the early part of this long-range program. (S/NF) Should these pilot experiments prove successful, then a near and distant experiments would be developed for a wide variety of devices to evaluate application aspects. Potential applications could include, for example, remote switching (in a communication role) or possibly as a countermeasure to minimize effectiveness of threat systems such as sensitive computer components or sensors. similarly, if these results are successful, they would provide insight regarding potential threats to U.S. systems or security. SECRET NOT RELEASABLE TO FOREIGN NATIONALS STAR GATE 40 LIMDIS 16 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 SECRET (SINF) The specific experiments to be conducted in these research domains will be defined during the first six to nine months of the program utilizing the recommendations of the working groups mentioned above. VII. (U) APPLIED RESEARCH PLAN FOR ANOMALOUS COGNITION: (U) Figure 6 illustrates the overall plan for the applied research portion for several main functional categories. a. (U) SELECTION (C) The most promising potential for selecting individuals is to identify ancillary activity that correlates with AC ability. If such a procedure can be identified, then receiver selection can be incorporated as part of other screening tests (e.g., fighter pilot candidacy), and thus large populations can be used. Among the items that will be examined are physiology (e.g., responses of the brain to external stimuli) and hypnotic susceptibility (i.e., an individuals predisposition for being hypnotized). The results of this effort will be examined continuously; however, a decision to end the investigation will occur in mid FY96. Should the results at that time warrant, then refining of the techniques will continue to the end of FY 1998. The reason the initial research spans several years is that to validate even one psychological finding requires long-term testing of candidate individuals. Current statistical methods require many AC sessions, and experience has shown that only a few sessions can be conducted per week for any single individual. (C) The previous program was able to estimate that approximately one percent of the general population possessed a high-quality, natural AC ability. Because the empirical method (i.e., asking large groups to attempt AC) is labor intensive and very inefficient, it is included in the research plan only as an alternate approach. b. (U) TRAINING (SINF) Training has been a major part of the previous program; however, results of training approaches have been difficult to evaluate and have not been examined systematically. Systematic review of this issue was begun in FY 92. One of the methods that will be examined involves lowering an individual's visual subliminal threshold (i.e., the level below which an individual is not consciously aware of visual material). This could enhance the individual's sensitivity to AC data. other forms of altered states, such as dreaming and hypnosis, will also be evaluated to see if such states can enhance AC data quality. SECRET NOT RELEASABLE TO FOREIGN NATIONALS STAR GATE LINDIS 17 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 UNCLASSIFIED TI ME FRAME ACTI U ITY 1995 1996 1 (ig7 1998 1()gg State Parameters PERSONNEL (Hypnosis, Physiology, Etc.) Psychology SELECTION RESERRCH (Self Report, Behavioral Ileasures, Etc.) Solicit Known Talent Empirical (Mass Screening) State Parameters (Altered States, Subliminal ThLe-Shq1_dt1easures-,.Etc.)- PERSONNEL Empirical Evaluation TRAINING RESEARCH Practical Application Tests (Increasing Project Difficulty) Target Characteristics (Entropy, Size, Etc.) APPLICATION Other Aspects (Target Function, Dynamics, Degree of EURLURTION Importance, Etc.) RESEARCH Operational Conditions (Targets, Feedback, Etc.) PROTOCOL Search/Location Projects DEUELOPMENT New Applications/Procedures Respons Definition (Written, Drawn, Physiological Measures, Etc.) ANALYSIS METHOD Artificial Intellicrence (Fuzzy Sets, Etc.) DEUELOPMENT Neural Network Analociies Combination of Ketbods DATA Intellicience Data Fusion 11ethqd1_ INTEGRATION/ Training/Seminars ASSIMILATION Advanced Training DEUELOPMENT Various Customers Figure 6 (U) Applied Research Milestones - Anomalous Cognition UNCLASSIFIED 18 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 SECRET (U) Results on these issues should be available at the close of FY95. If no progress has been observed and if there have been no positive results from the basic research, the task ends. However, should any of the variables examined appear promising then the task will be continued. (SINF) It is anticipated that all laboratory successes must be validated by simulating operational tasks. These experiments involve identifying the specialty to be tested, the acceptance criteria, and conducting sessions in which the complete target systems are know. This three-year activity runs concurrently with the other tasks but with a one-year offset to allow for planning. C. (U) TARGET/APPLICATION SELECTION (C) Based on earlier research, the most promising approach to target selection appears to be a single physical characteristic called entropy (i.e., a measure of inherent target information). Beginning in FY95, two and one half years have been allocated for the detailed study of this aspect of target properties. Initially, little experimentation is required; rather, a retrospective examination of previous target systems should indicate if this approach is valid. Included in this examination are detailed calculations of the information content of natural target scenes. (SINF) Beginning in mid FY96, other potential intrinsic target properties will be examined. For example, a target may be more readily sensed by AC if the collection of elements at the site (e.g., landmark, buildings, roads) constitute a conceptually coherent unit as opposed to a collage of unrelated items. Quantitative definition of targets will also be developed that include non-physical target parameters such as function, meaning, or relationships. These aspects are highly important in most operational projects and need to be quantified. (SINF) Part of this effort will involve investigations that serve two purposes: (1) add insight into the phenomenon; and (2) help evaluate the feasibility of certain potential applications. For example, long distance experiments could be conducted to or from deep caves or submarines in deep water to test communication potential and transmission theories. Experiments could also be conducted to targets on board space platforms to test distance and gravitational effects. Experiments to or from magnetically shielded rooms or certain earth locations (e.g., the magnetic pole) might indicate if magnetic fields influence the phenomenon. Experiments to opposite sides of the earth might als.6 indicate if a mass or gravity effect can be noted. SECRET NOT RELEASABLE TO FOREIGN NATIONALS STAR GATE LINDIS 19 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 SECRET (S/NF/SG/LIMDIS) This area of investigation will be integrated with a variety of applications in coordination with findings/investigations pursued by the in-house effort. Figure 9 identifies the main application or operational areas. Along with types of data desired. This activity will be integrated, where possible, into in-house pursuits that will explore these areas in a systematic fashion. Initial emphasis will be in counternarcotics and counterterrorism areas. (S/NF/SG/LIMDIS) Specific types of applications that will be explored in-depth include the search problem. Search tasks are expected to remain as high priority operational tasks (e.g., hostage location, lost equipment or system location). Search tasks are complicated by timing issues, especially if the missing target is being moved frequently. Related to this will be examination of predictive capability in order to evaluate feasibility of detecting hostile plans and intentions in advance. Pilot studies of other areas (e.g., code breaking, medical diagnostics, low intensity conflict support) will also be initiated. (S/NF/SG/LIMDIS) Another application area that will be examined is "communications". Previous research indicates that with proper protocols, basic or coded messages can be sent and received via AC procedures. Redundant coding methods can readily enhance probability of success, and new statistical methods can also improve success rates. Communication applications may have significant value for search problems by providing additional information on location of kidnapped or hostage victims. Such techniques might also help in determining hostage or POW state-of-health or other significant issues. d. (U) PROTOCOLS (U) Given the laboratory success of AC experimentation, the protocol task can build upon a substantial literature. Determining optimal, specialty-dependent protocols only require extending current concepts. Several years are required due to the statistical nature of analysis that is required to determine the effects of environment, receiver, target and feedback conditions. Several high-interest application areas (such as search/location) will be examined in detail. A variety of session procedures will be evaluated to determine those that are beneficial to improving data quality. (SINF) Protocol effectiveness may be measured by quality, quantity, and/or usefulness of the AC information elicited by its use. The requirements for protocols that are designed for laboratory settings are considerably more restrictive than those required for operational settings. For example, providing limited information to a receiver while an operational session is in progress (i.e., intermediate feedback) SECRET NOT RELEASABLE TO FOREIGN NATIONALS STAR GATE LIMDIS 20 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 SECRET might facilitate the acquisition of the desired data. This kind of feedback is strictly prohibited, however, in most protocols designed for laboratory experiments. Protocols may also vary depending on nature of the data required. For example, for some search projects, only general data may be adequate. For such cases would not require development of highly specific details and protocols the sessions would not be as complex. (U) A detailed protocol will need to consider a variety of potential session variables such as the individuals, physical environment, mental state and attitude, and how the target or task is designated (e.g., coordinates, abstract terms). other data includes specifics of the session (monitor present or not), type of feedback, type of response data (e.g., predictive), and mode and method of response (e.g., drawings, verbal). (SINF) Concurrently, the only known way to resolve the above issues is to conduct a large number of trials for a given individual with as many of the potential variables as possible held constant. Standard statistical methods can then be used to identify trends, patterns, and operational constraints. e. (U) DATA ANALYSIS (U) This area requires extensive review of leading analysis tools, such as those required for describing imprecise concepts or data (i.e., artificial intelligence techniques, fuzzy sets). This work will be combined with findings from neural network analysis and research, or possibly combinations of other emerging advanced analysis methods. (SINF) Various approaches that are anticipated to directly benefit operational evaluations. one promising technique involves procedures based on an adaptive (frequent data base update) approach. This will permit an individual's progression, and possibly time dependent data variables in an individual's track record, to be identified. (SINF) In addition to the search for new analysis methods, the current methods will also be reexamined. Laboratory requirements differ from those for operational activities in that the target can be controlled and well defined. For operational activities, uncertainties in tasking may arise, especially if operational requirements are changing or if some of the initial "known" data are incorrect. Such uncertainties complicate later analyses. (SINF) Analysis methods will also be developed that can make predictions on data quality for any given task. This will require development of an ektensive track record for each individual based on both controlled and operational projects. SECRET NOT RELEASABLE TO FOREIGN NATIONALS STAR GATE LIMDIS 21 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 SECRET (SINF) These analysis methods will also address certain practical issues. For example, a detailed, high-quality example of AC data may have little value to an intelligence analyst if that information was known from other sources. Likewise, a poor example of AC data might provide a single element as a tip-off for other assets, or provide the missing piece in a complex analysis, and thus be quite valuable. The intelligence utility of AC data may in some cases be only weakly connected to the AC quality. Therefore a data fusion analysis procedure is needed for AC-derived operational data. Methods that permit appropriate data analysis from an accuracy and utility viewpoint will be developed. f. (U) INTEGRATION (U) This activity would be an on-going review/ integration effort in order to identify patterns or clues useful for understanding practical aspects of this phenomenological area. (SINF) Identifying approaches and procedures that permit assimilation of AC data from operational support projects into all-source intelligence analysis procedures will also be part of this support activity. Depending on results of applied research findings and operational pursuits, a basic seminar/ training program for other applications-oriented elements might be established. Such a training/seminar program would focus on basic techniques and would augment possible operational training activity that might become part of the in-house effort. This would require several years to develop and establish. (SINF) The specific experiments to be conducted in these research domains will be defined during the first six to nine months of the program utilizing the recommendations of the SG1B working groups mentioned above. SECRET NOT RELEASABLE TO FOREIGN NATIONALS STAR GATE LIMDIS 22 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 SG1B Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 SG1 B SECRET IX. (U) POTENTIAL RESEARCH RETURN: (S/NF/SG/LIMDIS) The in-house and external research pursuits identified in this overall research and peer review plan have the potential for achieving highly significant results using AMP to address problems of national security by pushing the phenomena to their natural limits. This overall result will be achieved by: - Determining the underlying physical mechanisms of AMP. - Isolating specific brain processes involved in the phenomenon. - Identifying:*unique applications involving energetics" phenomenon (e.g., remote switching). SECRET NOT RELEASABLE TO FOREIGN NATIONALS STAR GATE LINDIS 24 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 SECRET (S/NF/SG/LIMDIS) It is the intention of STAR GATE to pursue all aspects of this area with high intensity, drawing on an experienced and well-qualified staff along with appropriate external assistance, in order to quantify and evaluate all available classified and unclassified research. By so doing, discoveries into how these phenomena work may be achievable. How to identify people with such talent (or potential for it) and how to develop/train selected individuals should also be a natural end-result. STAR GATE also draw heavily from lessons learned in all previous research and application investigations on a worldwide basis. X. (U) PROJECT OVERSIGHT METHODOLOGY: A. (U) PROGRAM MANAGEMENT/OVERSIGHT (SINF) DIA, as executive agent, has management structure that fosters a proactive, creative environment for this activity. Both and in-house activities are centered in one unit the direct supervision of the Director, Office (DIA/PAG). implemented a responsive, and' external research (PAG-TA) under for Ground Forces (SINF) Project oversight for this program will be provided by a Project Review Board (PRB) composed of five senior management individuals selected from areas of DIA outside of the National Military Intelligence Production Center (NMIPC). In addition, a six-member Project Oversight Panel will be established to provide program and technical guidance on all STAR GATE activities. The 28 member DIA Advisory Board has been appraised of the STAR GATE program and their recommendations have been incorporated into project activities. Review/guidance is available from DIA's Executive Director and from the Deputy Director. The General Defense Intelligence Program (GDIP) staff director conducts periodic project reviews and provides guidance. Links with the Intelligence Community help provide a broader management and program review base for this activity. (U) The extensive nature and scope of these various program management and oversight activities will insure that all activities identified in this long-range plan can be appropriately monitored and evaluated on an on-going basis. B. SCIENTIFIC OVERSIGHT (SINF) Oversight for external contract activity is currently provided by a six-member expert Scientific oversight Committee (SOC). A Human Use Review Board has also been established to provide expert guidance/advice regarding contractor adherence to appropriate DOD human use regulation. (U) There is currently in place a contractor SECRET NOT RELEASABLE TO FOREIGN NATIONALS STAR GATE LIMDIS 25 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 SECRET Scientific oversight Committee (SOC) which is tasked with three major responsibilities: a. Review and approve all experimental protocols prior to the collection of experimental data. b. Critically review all experimental final reports as if they were submissions to technical scientific journals. All remarks in writing are included in the final technical reports to DIA. C. Suggest directions for further research. (U), In addition to these responsibilities, the SOC members are encouraged to exercise un-announced drop-in privileges to view experiments in progress. (U) The five voting members of the SOC are respected scientists from the following disciplines: physics, astronomy, statistics, neuroscience, and psychology. See Appendix E for membership data. (U) A contractor Institutional Review Board (IRB) is currently in place with the responsibility of assuring compliance with all U.S. and DoD regulations with regard to the use of humans in experimentation and assuring their safety. The IRB members represent the health, legal, and spiritual professions in accordance with government guidelines. See Appendix F for membership data. (U) It is anticipated that oversight of this program will be conducted by these Committees, if available, or new committees with equivalent scientific credentials. XI. (U) DEVELOPMENT OF EVALUATION CRITERI A. (U) SCIENTIFIC VALIDITY (SINF) The STAR GATE Scientific Advisory Committee has determined that the scientific validity of the STAR GATE program has been satisfactorily demonstrated under the most demanding of experimental protocols. An statistically significant anomaly does exist which cannot be currently explained by conventional means. For example, 77% of academics in the arts, humanities, and education believe that AMP is either an established fact or a likely possibility. Supporting technical evidence contained in technical studies may be found at Appendix G. (SINF) A substantial number of examples dating back to 1972 provide at a minimum prima facia.'.'evidence that AMP can be used in such a way as to provide a "value-added" function to the Intelligence Community. Appendix H is a formal evaluation of the SECRET NOT RELEASABLE TO FOREIGN NATIONALS STAR GATE LIMDIS 26 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 SECRET use of AMP for intelligence gathering purposes conducted'in 1987. The overall findings of this evaluation were that 11 ... the Project Review Group has determined to its satisfaction that the work of the Enhanced Human Performance Group is scientifically sound ... and is providing valuable insight into the nature of an anomaly which have a significant impact on the DoD.11 B. (U) PERFORMANCE (SINF) The ability of the STAR GATE program to produce results that have an intelligence value can only be measured by customer evaluations. AMP provided intelligence data, along with other forms of intelligence, are evaluated, in part, with subjective criteria. STAR GATE will develop feedback mechanisms and procedures for customers that will result in a method of quantifying this subjective feedback and evaluation data so that the value added and cost-effectiveness can be measured. XII. (U) BUDGET AND RESOURCE REQUIREMENTS (FYs 95-99): (S/NF/SG/LIMDIS) Due to the diversity of the STAR GATE mission/objectives, both external resources and in-house expertise are required. Since this Activity possesses no in- house R&D capability, an absolute need for external R&D support is required to meet Congressional concerns which are addressed in this program plan. A balance will be maintained between external and in-house activities, and every effort will be made to integrate and link these activities where appropriate. The external aspect permits a wide range of expertise covering many disciplines to be focused on this area; this also has the benefit of ensuring peer group review and of facilitating a variety of scientific interactions. In-house personnel with a wide-range of expertise in this phenemenology will need to be retained to make this proposed plan work. (SINF) In order to review the major tenets of the draft program plan, the Defense Intelligence Agency will convene a panel of appropriate scientists to provide recommendations on the plan and the research it achieves. Based on the panel's recommendations, the Defense Intelligence Agency will then submit a budget line item to fund those approved objectives. (C) An annual report will document the current operational, technical and administrative status of the program. . SECRET NOT RELEASABLE TO FOREIGN NATIONALS STAR GATE LIMDIS 27 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 SECRET APPENDIX A CONGRESSIONALLY-DIRECTED ACTION DEFENSE AUTHORIZATION CONFERENCE (SINF) REQUEST: "The conferees are concerned that insufficient funds have been spent on research and development to establish the scientific basis for the STAR GATE program. The conferees direct the Director of DIA to prepare a program plan and to submit an appropriate budget request for a research effort, over several years, to determine whether the STAR GATE program can show results that are cost-effective and satisfy reasonable performance criteria. This plan, and any research under this program, should be subject to peer review by neutral scientific experts. The Director of DIA is directed to prepare this research and peer review plan within existing program funds." SECRET NOT RELEASABLE TO FOREIGN NATIONALS STAR GATE LINDIS A-1 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003104118 : CIA-RDP96-00789ROO2700020001-0 SECRET APPENDIX B TERMINOLOGY AND DEFINITIONS (U) PHENOMENA TERMINOLOGY: (U) This phenomenological area has had a variety of descriptive terms over the years, such as paranormal, parapsychological, or as psychical research. Foreign researchers use other terms: "psychoenergetics" in the USSR; "extraordinary human function" in the People's Republic of China (PRC). In general, this field is concerned with a largely unexplored area of human consciousness/subconsciousness interactions associated with unusual or underdeveloped human capabilities. (U) Recently, researchers have shown a preference for terms that are neutral and that emphasizes the anomalous or enigmatic nature of this phenomena. The term anomalous mental phenomena (AMP), is generally preferred. (U) This area has two aspects; information access and energetics influence. Information access refers to a mental ability to describe remote areas or to access concealed data that are otherwise shielded from all known sensory channels. A recent term for this ability is anomalous cognition (AC). This term places emphasis on potential understanding that might be available from advances in sensory/brain functioning research or other related research. Older terms for this aspect have included extra-sensory perception (ESP), remote viewing (RV), and in some cases, precognition. (U) The energetics aspect refers to the ability to influence, via mental volition, physical or biological systems by an as yet unknown physical mechanism. An example of physical system influence would include affecting the output-of sensors or electronic devices; biological systems influence would include affecting physiological parameters of an individual. A recent descriptive term for this ability is anomalous perturbation (AP). older terms for this phenomenon included psychokinesis (PK) or telekinesis. (U) GENERAL DEFINITIONS: (SINF) For this program, basic research is defined to mean any investigation or experiment for determining fundamental SECRET NOT RELEASABLE TO FOREIGN NATIONALS STAR GATE LINDIS B-1 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO27000200 0.1-0 SECRET processes or for uncovering underlying parameters that are involved in this phenomenon. Basic research is primarily oriented toward understanding the physical, physiological , and psychological mechanisms of Anomalous mental phenomena (AMP). (SINF) Applied research refers to any investigation directed toward developing particular applications or for improving data quality and reliability. For anomalous cognition (AC) phenomenon, research is primarily directed toward improving the output quality of AC data. This would include ways to develop/improve utility of AC data for variety of potential application. For example, examination of spatial and temporal relationships of AC data could assist in developing a reliable search capability useful for locating missing people or equipment. SECRET NOT RELEASABLE TO FOREIGN NATIONALS STAR GATE LIMDIS B-2 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 SECRET APPENDIX C POTENTIAL RESEARCH SUPPORT FACILITIES ANOMALOUS MENTAL PHENOMENA Science Applications International Corp. Mind Science Foundation Princeton Engineering Anomalies Laboratory American Society for Psychical Research St. John's University Foundation for Research into the Nature of Man ARE/Atlantic University University of Virginia Psychophysical Research Laboratories Edinburgh University OTHER RELATED DISCIPLINES Psychology Stanford University Cornell University Anthropology University of California University of Arizona Los Altos, CA San Antonio, TX Princeton Univ, NJ New York, NY Long Island, NY Durham, NC Virginia Beach, VA Charlottesville, VA Edinburgh, Scotland Edinburgh, Scotland Stanford, CA Ithaca, NY Berkeley, CA Tucson, AZ Psychophysiology SRI International Menlo Park, CA Langly-Portor Neuropsychiatric Institute San Francisco, CA Menninger Foundation Topeka, KS Psychoimmiinology California Institute for Transpersonal Menlo Park, CA Psychology Cognitive Neuroscience Los Alamos National Laboratory Los Alamos, NM Sandia National Laboratory Albuquerque, NM University of California San Diego, CA SECRET NOT RELEASABLE TO FOREIGN NATIONALS STAR GATE LINDIS C-1 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 @SEWT cognitive Psychology Psychology Department, Princeton Univ Princeton, NJ Psychology Department, City College of New York, NY New York Artificial Intelligence Massachusetts Institute of Technology Stanford University Neural Networks Massachusetts Institute of Technology Science Applications International Corp Statistics/Signal Analysis University of California Harvard University Thermodynamics Rochester University Physics Department, Stanford University Quantum Measurement International Business Machines, Research Laboratories General Relativity California Institute of Technology University of Texas at Austin Electromagnetic/Basic Research Electronetics Corp Battelle Corp Institute for Advanced Study Cambridge, MA Stanford, CA Cambridge, MA Los Altos,'CA Davis, CA Cambridge, MA Rochester, NY Stanford, CA College Park, MD Pasadena, CA Austin, TX Buffalo, NY Columbus, OH Austin, TX SECRET NOT RELEASABLE TO FOREIGN NATIONALS STAR GATE LINDIS C-2 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 Approved For Release 200310 4118 CIA-RDP96-00789ROO2700020001-0 SECRET APPENDIX D RESOURCE LITERATURE 1. A.R.E. Journal 2. Abnormal hypnotic Phenomena 3. American Anthropologist 4. American Ethnologist 5. American Journal of Clinical Hypnosis 6. American Journal of Physiology 7. American Journal of Sociology 8. American Psychologist 9. American Society for Psychical Research 10. Annals of Eugenics 11. Annals of Mathematical Statistics 12. Annales de Sciences Psychiques 13. Archivo, di Psicologica Neurologia. e Psychiatra 14. Association for the Anthropological Study of Consciousness Newsletter 15. Behavioral and Brain Science 16. Behavioral Science 17. Bell System Technical Journal 18. Biological Psychiatry 19. Biological Review 20. British Journal for the Philosophy of Science 21. British Journal of Psychology 22. Bulletin of the American Physical Research 23. Bulletin of the Boston Society for Psychic Research 24. Bulletin of the Los Angeles Neurological Societies 25. Contributions to Asian Studies 26. Electroencephalography and Clinical Neurophysiology 27. Endeavour 28. Ethnology 29. Exceptional Human Experience 30. Experientia 31. Experimental Medicine and surgery 32. Fate 33. Fields within Fields 34. Foundations of Physics 35. Hibbert Journal 36. Human Biology 37. International Journal of Clinical and Experimental Hypnosis 38. International Journal of Comparative Sociology SECRET NOT RELEASABLE TO FOREIGN NATIONALS STAR GATE LIMDIS D-1 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 SECRET 39. International Journal of Neuropsychiatry 40. International Journal of Parapsychology 41. International Journal of Psychoanalysis 42. Journal of Abnormal and Social Psychology 43. Journal of Altered States of Consciousness 44. Journal of Applied Physics 45. Journal of Applied Psychology 46. Journal of Asian and African Studies 47. Journal of Biophysical and Biochemical Cytology 48. Journal of Cell Biology 49. Journal of Communication 50. Journal of Comparative and Physiological Psychology 51. Journal of Consulting Psychology 52. Journal of Existential Psychiatry 53. Journal of Experimental Biology 54. Journal of Experimental Psychology 55. Journal of General Psychology 56. Journal of Genetic Psychology 57. Journal of Mind and Behavior 58. Journal of Nervous and Mental Diseases 59. Journal of Personality 60. Journal of Personality and Social Psychology 61. Journal of Research in PSI Phenomena 62. Journal of Scientific Exploration 63. Journal of the American Academy of Psychoanalysis 64. Journal of the London Mathematical Society 65. Journal of the Royal Anthropological Institute of Great Britain and Ireland 66* Metapsichica 67: Mind-Brain Bulletin 68. Motivation and Emotion 69. Nature 70. Naturwissenschaftliche Rundschau 71. New Horizons 72. New Scientist 73. New Sense bulletin 74. Newsletter of the Parapsychology Foundation 75. Parapsychology Bulletin 76. Parapsychology Abstracts International 77. Parapsychology Review 78. Perceptual and Motor Skills 79. Philosophy of Science 80. Physiology and Behavior 81. Proceedings of the Society for Psychical Research 82. Psychedelic Review 83. Psychic SECRET NOT RELEASABLE TO FOREIGN NATIONALS STAR GATE LIMDIS D-2 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 SECRET .00 84. Psychic Science 85. Psychoanalytic Quarterly 86. Psychoanalytic Review 87. Psychological Bulletin 88. Psychometrika 89. Psychophysiology 90. Physics Today 91. Renti Teyigongneng (EFHB Research) (PRC] 92. Revue Metapsychique, 93. Revue Philosophique 94. Revue Philosophique de la France et de L'Etranger 95. Revue Philosophique, Applique 96. Science 97. Skeptical Inquirer 98. Social Studies of Science 99. Subtle Energies 100. The Humanistic Psychology Institute 101. The Journal of Parapsychology 102. The Journal of the American Society for Psychical Research 103. Theta 104. Tijdschrif voor Parapsychologie 105. Tomorrow 106. Voprosy Filosofi (Questions of Philosophy) [RUSSIA] 107. Western Canadian Journal of Anthropology 108. Zeitschrift fur die Gesamte Neurologie und Psychiatrie 109. Zietschrift fur Parapsychologie und Grenzgebeite der Psychologie 110. Zietschrift fur Tierpsychologie ill. Zietschrift fur Vergleichende Physiologie 112. Zetetic Scholar 113. Zhongguo Shebui Kexue (China Social Sciences) [PRC3 114. Ziran Zazhi (Nature) [PRC) SECRET NOT RELEASABLE TO FOREIGN NATIONALS STAR GATE LIMDIS D-3 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 SECRET APPENDIX E CURRENT CONTRACTOR-SCIENTIFIC OVERSIGHT COMMITTEE MEMBERSHIP Steven A. Hillyard -Professor of Neurosciences, Department of Neurosciences, University of California, San Diego. -Author or coauthor of 118 technical neuroscience publications. -Eighty-two invited presentations at technical conferences. -Ph.D., Yale University, 1968 (Psychology). S. James Press - Professor of Statistics, Department of Statistics, University of California, Riverside. -Author or coauthor of 132 statistics publications. -Author of 12 books and/or monographs. -Ph.D., Stanford University, 1964 (Statistics). Garrison Rapmund - Responsible for facilitating transfer of Strategic Defense Initiative technologies to health care industries. - Major General, USA retired in 1986 as Assistant Surgeon General (R&D) and Commander, Army Medical R & D Command. -M.D., Columbia University, 1953 (Pediatrics). Melvin Schwartz - Associate Director for High Energy and Nuclear Physics, Brookhaven National Laboratory. - Author or coauthor of 40 technical publications in high energy physics, author of "Principles of Electrodynamics.11 - Nobel Prize, Physics (1988). - Ph.D., Columbia University, 1958 (Physics). Yervant Terzian - Professor of Physical sciences, Chairman of the Department of Astronomy, Cornell University. - Author/coauthor of numerous technical publications and books. - Ph.D., Indiana University, 1965 (Astronomy). Phillip G. Zimbardo - Professor of Psychology, Department of Psychology, Stanford University. - Author/coauthor of numerous experimental psychology publications. - Ph.D., Yale University, 1959 (Psychology). SECRET NOT RELEASABLE TO FOREIGN NATIONALS STAR GATE LIMDIS E-1 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 SECRET APPENDIX F CURRENT CONTRACTOR INSTITUTIONAL REVIEW BOARD MEMBERSHIP Byron Wz. Brownp Jr., Ph.D. - Biostatistics, Stanford University Gary R. Fujimoto, M. D. occupational Medicine, Palo Alto Medical Foundation John Hanley, M. D. - Neuropsychiatry, University of California, Los Angeles Robert B. Livingston, M. D. - Neuroscience, University of California, San Diego Robin P. Michelson, M. D. - Otolaryngology, University of California, San Francisco Ronald Y. Nakasone, Ph.D. Buddhist Studies, Institute of Buddhist Studies, Berkeley, CA Garrison Rapmund, M. D. (Chair) - Air Force Science Advisory Board Louis J. Westo M. D. - Neuropsychiatry, University of California, Los Angeles SECRET NOT RELEASABLE TO FOREIGN NATIONALS STAR GATE LIXDIS F-1 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 CIA-RDP96-00789ROO2700020001-0 SECRk APPENDIX G- ACADEMIC STUDIES REGARDING THE SCIENTIFIC VALIDITY OF AMP SECRET NOT RELEASABLE TO FOREIGN NATIONALS STAR GATE LIMDIS G Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 CPYRGHT Approved For Release 2003104118: CIA-RDP96-00789ROO2700020001-0 1's:@chologioal Bulletin (January, 1994) Does Psi Exist? Replicable Evidence for an Anomalous Process of Information'h-ansfer Dary1J. Bem and Charles Honorton Version 4.7 October 1, 1993 Most academic. plychologistado.not yet accept the existenoe of psi, anomalous proce .seen ofin- formation or energy -transfer (such as tolepathyorother forms of extrasensory perception) that are currently unexplained in terms of known physical or biological mechanimns. We believe thatthe replication ratesandeffect,sizes achieved-by one particular experimental method, the ganrfeU procedure. are nowsufficient to warrant bdngini this. body of data to the attention of the wider-psychological.communitv.- Competing ineta4malyses of the ganzield database are re- viewed. 1 by R. Hyman (1985), -a skeptical critic of psi research, and the other by C. Honorton (1985), a parapsychologist -and major contributor to the kanzfeld-database: Nextwthe results of 11 new ganzfeld studies that:comply.with guidelines jointly authored by R. Hyman and C. Honorton (1986) are summarized. Finally, issues ofreplication and -theoretical explanation are discussed. The term psi denotes anomalous:processes of informa- tion or energy transfer, processes such as telepathy or other forms of extrasensoryperception that arevurrently unexplained in terms of known physical or biological @necha 'nisms. 'The term in, purely descriptive: It neither impties that such anomalousphenomena areparanormal nor connotes anything about -their underlying mecha- nisms. Does psi exist? Most academic psychologists don't think so. A survey of more than 1,100 college professors in the United States found that 55% of natural scientists, 66% of social scientists (excluding psychologists), -and 77% of sca - demics in the arts, humanities, and education believed that ESP is either anestabliahed flactor alikelypossibil- ity. The comparable figure for psychologists was only 34%. Moreover, an equal number of psychologists declared ESP to be an impossibility, a view expressed by only 2% of all other respondents (Wagner & -Monnet, 1979). Daryl J. -Bem, Department of PsycholoM Comell'Univeraitr. Charles Honorton, Department or Psychology. University of Ed- inburgh. Edninburgh, Scotland. Sadly, Charies Honorton died of a heart attack on November 4, 1992, 9.days before this article was acceptedfor publication. He was 46. Pampsychology has loot one of its most valued contau- tors. I have lost a valued friend. This collabomtion had its origins in a 1983 visit I made to Honorton!a Psychophysical Research Laboratories (PRL) in Princeton. New Jersey, as one of several outside consultants brought In to examine the design and implementation of the ex- perimental protocols. Prepamtion fthis article was supportec4 -inpart, by grants to Charles Honoxn from the American Society for Psychical Re- search and the Pampsychology Foundation. both of New York City. The work at PRLsummerized in the second halfof this ar- ticle was supported by the James S. McDonnell Foundation of SL Louis, Missouri, and by the John E. Fetzer Foundation of Kala- mazo-0, Michigan. Helpful comments an dmits of this article were received Emm Deborah Delanoy. Edwin May, Donald McCarthy, Robert Morr* John Palmer, Robert Rosenthal, Lee Ross, Jessica Utts, Philip Zimbardo, and two anonymous reviewers. Correspondence concerning this article should be addressed to Daryl J. Bem, Department of Psychology, Uria Kau, Cornell University. Ithaca, New York 14853. (Electronic mail may be sent to dbetnewmelledu). Psychologists are probably more ske tical about psi for several reasons. First, we believe Zat eitraordinary claims requf re extraordinary proof. And although our col- leagues from other disciplines would probably agree with thin dictum, we are more likely to be familiar with the methodological and statistical requirementa for sustaining such claims, as well as with previous claims that failed ei- ther to meet those requirements or to survive the test of succesdul replication. Even for ordinary claims, our con. ventional statistical criteria are conservative. The sacred p = .05 threshold is a constant reminder that it is far more sinful to assert that an effect exists when it does not (the Type I error) thari to. assert that an effect does not exist when It does (the Type Il error). . Second@ most of us distinguish4harply between phe- nomenai whose explanations are merely obscure or contro- versial (eg, hypnosis) and,phenomena. such as psi that would appear to fall outside our current explanatory framework altogether. (Some would characterize thin as the difference between the unexplained and the inexplica- ble.)'In contrast, many laypersons treat all exotic psycho- logical phenomena as epistemologically equivalent; many even consider ddik vu. to be a psychic phenomenon. The blurring 6ftlds critical distinction Is aided and abetted by the mass media, 'new agen books and mind-power courses, and 'psychie entertainers who present both genuine hyp- nosis and fake 'mind reading" in the course of a single performance. Accordingly, most laypersons would not have to revise their conceptual model of reality an radi- cally as we would.to assimilate the existence of psi. For us,.pai is simply mom extraordinary. F!indly, research in cogrutive and social psychology has sensitized us to the'errors and biases that plague intuitive attempts to draw valid inferences from the data of every- day experience (Gilovich, 1991; Nisbett & Rose, 1980; Tversky,& Kahneman, 1971). This leads us to, give virtu- ally no probative weight to anecdotal or journalistic re- ports of psi, the main source cited.by our academic col- leagues. as evidence for their beliefs about psi (Wagner & Monnet, 1979). Ironically, however, psychologists am probably not more farmliar than others with recent experimental research on psi. Uke most psychological research, parapsycholoecal research is reported primarily in specialized journals' . un- like most psychological research, however@ contemporary parapsychological research is not usually reviewed or Approved For Release 2003/04/18 CIA-RDP96-00789ROO2700020001-0 CPYRGHT Approved For Release 2003104118: CIA-RDP96-00789ROO2700020001-0 ANOMALOUS INFORMATION TRANSFER Meta-Analyses of the Gaazfeld Database In 1985 and 1986, the Journal of Parapsychology de- voted two entire issues to a critical examination of the ganzfeld database. The 1985 issue comprised two contri- butions: (a) a meta-analysis and critique by Ray Hyman (1985), a cognitive psychologist and skeptical critic of parapsychological research, and (b) a competing meta- analysf a and rejoinder by Charles Honorton (1985), a parapsychologist and mAjor contributor to the ganzfeld database. The 1986 issue contained four commentaries on the Hyman-Honorton exchange, a joint commun qud by Hyman and Honorton, and six additional commentaries on the joint communiqu6 itself. We summarize the major issues and conclusions here. Replication Rates- Ratee by etudy. Hymans meta-analysis covered 42 psi ganzfeld studies reported in 34 separate reports written or published fi-om 1974 through 1981. One of the first problems he discovered in the database was multiple analysis. As noted earlier, it is possible to calculate sev- eral indexes of psi performance in a ganzfeld experiment and, fuzthez ore, to subject those indexes to several kinds of statistical treatment. Many investigators reported mul- tiple indexes or applied multiple statistical tests without adjusting the criterion significance level for the number of tests conducted. Worse, some may have 'shopped" among the altematives until finding one that yielded a aigriffi- cantly successful outcome. Honorton agreed that t1iis was a problem. Accordingly, Honorton applied a uniform test on a common index across all studies from which the pertinent daturn could be extrmcted@ regardless of how the investiga- tors had analyzed the data in the original reports. He se- lected the proportion of lifts as the common index because it could be calculated for the largest subset of studies: 28 of the 42 studies. The hit rate is also a conservative index because it discards most of th' rating information; a soc- e, ond place ranking--a "near miss@-receiv'es no more credit than a last place ranking. Honorton then calculated the exact binomial probability and its associated z score for each study. Of the 28 studies, 23 (82%) had positive z scores (p 4.6 x 10-4, exact binomial test with p - q = .6). Twelve of the studies (43%) had z scores that were independently significant at the 5% level (p a 3.5 x 10-9, binomial test with 28 studies, p = .05, and q = .95), and 7 of the studies (25%) were independently significant at the 1% level (p = 9.8 x 10-9). The composite Stouffer z score across the 28 studies was 6.60 (p - 2.1 x 10-11).1 A more conservative estimate of significance can be obtained by including 10 additional studies that also used the relevantiudging pro- cedure but did not report lift rates. If these studies am as- signed a mean z score of zero, the Stouffer z across all 38 studies becomes 6.67 (p - 7.3 x 10-9). Thus, whether one considers only the studies for which - the relevant information is available or includes a null es timate for the additional studies for which the information is not available, the aggregate results cannot reasonably lStouffees z is computed by dividing the in im of the r scores for the individual studies by the square root of the number of studies (Rosenthal. 1978). 3 be attributed to chance. And, by design, the cumulative outcome reported here cannot be attributed to the infla- tion of significance levels through multiple analysis. Rates by laboratory. One objection to estimates such as those just described is that studies from a common labora- tory are not independent of one another (Parker, 1978). Thus, it is possible for one or two investigators to be dis- proportionately responsible for a high replication rate whereas other, independent investigators are unable to obtain the effect. The ganzfeld database is vulnerable to this possibility. 7be 28 studies providing lift rate information were con- ducted by investigaWn in 10 different laboratories. One laboratory contributed 9 of the studies, Honorton's own laboratory contributed 6, 2 other laboratories contributed 3 each, 2 contributed 2 each, and the remaining 4 labora- tories each contributed 1. Thus, half of the studies were conducted by only 2 laboratories, I of them Honorton's own. Accordingly, Honorton calculated a separate Stouffer Z score for each laboratory. Significantly positive outcomes were reported by 6 of the 10 laboratories, and the com- bined z score across laboratories was 6.16 (p = 3.6 x 10-10). Even if all of the studies conducted by the 2 most prolific laboratories are discarded from the analysis, the Stouffer z across the 8 other laboratories remains ffignifi- cant (z - 3.67, p = 1.2 x 10-4). Four of these studies are significant at the 1% level (p - 9.2 x 10--6, binomial test with 14 studies, p = .01, and q = .99), and each was con- tributed by a different laboratory. Thus, even though the total number of laboratories in this database is small, moat of them have reported significant studies, and the significance of the overall effect does not depend on just one or two of them. Selective Reporting In recent years, behavioral scientists have become in- creasingly aware of the 'file-drawer" problem: the likeli- hood that successful studies are more likely to be pub- lished than unsuccessful studies, which are more likely to be consigned to the file drawers of their disappointed in- vestigatom (Dozarth & Roberts, 1972; Sterling, 1959). Parapsychologists were among the first to become sensi- tive to the problem, and, in 1975, the Parapsychological Association Council adopted a policy opposing the selec- tive reporting of positive outcomes. As a consequence, negative findings have been routinely reported at the as- sociation's meetings and in its affiliated publications for almost two decades. As has already been shown, more than half of the ganzfeld studies included in the meta- analysis yielded outcomes; whose significance falls short of the conventional .05 level. A variant of the selective reporting problem arises from what Hyrhan (1985) has termed the "retrospective study.' An investigator conducts a small set of exploratory trials. If they yield null results, they remain exploratory and never become part of the official record; if they yield posi- tive results, they are defined as a study after the fact and are submitted for publication. In support of this possibil- ity, Hyman noted that there are more significant studies in the database with fewer than 20 trials than one would expect under the assumption that, all other things being 6qual, statistical power should increase with the square root of the sample size. Although Honorton questioned the Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 CPYRGHT Approved For Release 2003/04/18: CIA-RDP96-00789 ROO 2700020001 -0 ANOMALOUS INFORMATION TRANSFER In psi ganzfeld studies, the hit rate itself provides a straightforward descriptive measure of effect size, but two measure cannot be compared directly across studies be- cause they do not a use a four-stimulus judging set and, hence, do not all have a chance baseline of .25. The next most obvious candidate, the difference in each study be- tween the hit rate observed and the hit rate expected un- der the null hypothesis, is also intuitively descriptive but is not appropriate for statistical analysis because not all differences between proportions that are equal are equally detectable (eg, the power to detect the difference between .55 and .25 is different from the power to detect the differ- ence between .50 and .20). To provide a scale of equal detectability, Cohen (1988) devised the effect size index h, which involves an arcoine transformation on the proportions before calculation of their difference. Cohen's h is @quite general and can assess the difference between any two proportions drawn from independent samples or between a single proportion and any specified hypothetical value. For the 28 studies exam- ined in the meta-analyses, h was .28, with a 95% confi- dence interval from .11 to AS. But because values of h do not provide an intuitively descriptive scale, Rosenthal and Rubin (1989; Rosenthal, 1991) have recently suggested a new index, j; which ap- plies specifically to one-sample, multiple-choice data of the kind obtained in ganzfeld experiments. In particular, Yr expresses all hit rates as the proportion of hits that would have been obtained 'if there had been only two equally likely alternatives-essentially a coin flip. Thus, ir ranges from 0 to 1, with .6 expected under the null hy- pothesia. The formula is P(k - 1) P(k - 2) + 1 where P is the raw proportion of hits and k is the number of alternative choices available. Because Yr has such a straightforward intuitive interpretation, we use it (or its conversion back to an equivalent four-alternative hit rate) throughout this article whenever it is applicable. For the 28 studies examined in the meta-analyses, the mean value of Yr was .62, with a 95% confidence interval from .55 to .69. This corresponds to a four-alternative hit rate of 35%, with a 95% confidence interval from 28% to 43%. Cohen (1988, 1992) has also categorized effect sizes into small, medium, and large, with medium denoting an effect size that should be apparent to the naked eye of a careful observer. For a statistic such as r, which indexes the de- viation of a proportion from .5, Cohen considers .65 to be a medium effect size: A statistically unaided observer should be able to detect the bias of a coin that comes up heads on 65% of the trials. Thus, at .62, the psi ganzfeld effect size falls just short of Cohen's naked-eye criterion. From the phenomenology of the ganzfeld experimenter, the corresponding hit kate of 35% implies that he or she will see a subject obtain a hit approximately every third session rather than every fourth. It is also instructive to compare the psi ganzfeld effect with the results of a recent medical study that sought to determine whether aspirin can prevent heart attacks (Steering Committee of the Physicians' Health Study Re- search Group, 1988). The study was discontinued after 6 years because it was already clear that the aspirin treat- trxent was effective (p < .00001) and it was considered un- ethical to keep the control group on placebo medication. The study was widely publicized as a major medical breakthrough. But despite its undisputed reality and practical importance, the size of the aspirin effect is quite small: Taking aspirin reduces the probability of suffering a heart attack by only .008. The corresponding effect size (h) is .068, about one third to one fourth the size of the psi ganzfeld effect (Atkinson et al, 1993, p. 236; Utts, 1991b). In sum, we believe that the psi ganzfeld effect is large enough to be of both theoretical interest and potential practical importance. Experimental Correlates of the Psi Ganzfeld Effect We showed earlier that the technique of correlating variables with effect sizes across studies can help to as- sess whether methodological flaws might have produced ardfactual positive outcomes. The same technique can be used more affirmatively to explore whether an effect varies systematically with conceptually relevant varia- tions In experimental procedure. The discovery of such correlates can help to establish an effect as genuine, oug- gest ways of increasing replication rates and effect sizes, and enhance the chances of moving beyond the simple demonstration of an effect to its explanation. This strat- egy is only heuristic, however. Any correlates discovered must be considered quite tentative, both because they ernerge from post hoc exploration and because they neces- sorily involve comparisons across heterogeneous studies that differ simultaneously on many interrelated variables, known and unknown. Two such correlates emerged from the meta-analyses of the psi ganzfeld effect. Single- versus multiple-image targets. Although most of the 28 studies in the meta-analysis used single pictures as targets, 9 (conducted by three different investigators) used View Master stereoscopic slide reels that presented multiple images focused on a central theme. Studies using the View Master reels produced significantly higher hit rates than did studies using the single-image targets (60% vs. 34%), t(26) - 2.22, p =.035, two-tailed. Sender-receiver pairing. In 17 of the 26 studies, partici- pants were fi-ee to bring in friends to serve as senders. In 8 studies, only laboratory-aAsigned senders were used. (Tbree studies used no sender.) Unfortunately, there its no record of how many participants in the former studies ac- Wally brought in fhends. Nevertheless, those 17 studies (conducted by six different investigators) had significantly higher hit rates than did the studies that used only labo- ratory-assigned senders (44% vs. 26%), 023) - 2.39, p .025, two-tailed. The Joint Cornmuniqu6 After. their published exchange in 1985, Hyman and Honort6n agreed to contribute a joint communiqu6 to the subsequent discussion that was published in 1986. First they act forth their areas of agreement and disagreement: We agree that there is an overall significant effect in this data base that cannot reasonably be explained by selective reporting or multiple analysis. We continue to differ over the degree to which the effect constitutes evidence for PsL but we agree that the final verdict awaits the outcome of fu- ture experiments conducted by a broader range of investiga- Approved For Release 2003104118: CIA-RDP96-00789ROO2700020001-0 CPYRGHT ANOMAMUS INFORMATION TRANSFER Randomizzation. 'Me random selection of the target and sequencing of the judging not were controlled by a noise- based random number generator interfaced to the com- puter. Extensive testing confirmed that the generator was providing a uniform distribution of values throughout the full target range (1-160). Tests on the actual frequencies observed during the experiments confirmed that targets were, on average, selected uniformly from among the 4 clips within each target set and that the 4 judging se- quences used were uniformly distributed acros's sessions. Additional control features. The receiver's and sender's rooms were sound-ioolated, electrically shielded chambers with single-door access that could be continuously moni- tored by the experimenter. There was two-way Intercom communication between the experimenter and the re- ceiver but only one-way communication into the sender's room; thus, neither the experimenter nor the receiver could monitor events inside the sender's room. The archival record for each session includes an audiotape containing the receiver's mentation during the ganzfeld period and all verbal exchanges between the experimenter and the receiver throughout the experiment. The automated ganzfeld protocol has been examined by several dozen parapsychologists and behavioral re- searchers fi-om other fields, including well-known critics of parapsychology. Many have participated as subjects or observers. All have expressed satisfaction with the han- dling of security Issues and controls. Parapsychologists have often been urged to employ ma- gicians as consultants to ensure that the experimental protocols are not vulnerable either to inadvertent sensory leakage or to deliberate cheating. Two 0@mentalists,' magi- cians who specialize in the simulation of pm*, have exam- ined the autoganzfeld system and protocol. Ford Kroes, a professional mentalist and officer of the mentalisiVe pro- feasional organization, the Psychic Entertainers Associa- tion, provided the following written statement 'In my pro- fessional capacity as a mentalist, I have reviewed Pay- chophysical Research Laboratories' automated ganzfeld system and found it to provide iexcellentsecurity against deception by subjects" (personal communication, May, 1989). Daryl J. Bem has also performed as a mentalist for many years and is a member of the 1%ychic Entertainers Association. As mentioned in the author note, this article had its origins in a 1983 visit he made to Honortores labo. ratory, where he was asked to critically examine the re- search protocol from the perspective of a mentalist, a re- search psychologist, and a subject. Needless to say, this article would not exist if he did not concur with Ford Kross's assessment of the security procedures. Experimental Studies6 Altogether, 100 men and 140 women participated as re- ceivers, in 354 sessions during the research program. The participants ranged in age from 17 to 74 years (iii = 37.8 SD 11.8), with a mean formal education of 15.6 ye@@ (SD 2.0). Eight separate experimenters, including Hon- orton, conducted the studies. 6A recent review of the original computer files uncovered a duplicate record in the autoganzfeld database. This has now been eliminated, reducing by one the number of subjects and sessions. As a result, some of the numbers presented in this article differ slightly from those in Honorton et al. (1990). The experimental program included three Pilot and eight formal studies. Five of the formal studies used novice (first-time) participants who served gis the receiver in one session each. The remaining three formal studies used experienced participants. Pilot studies Sample sizes were not preset in the three pilot studies. Study I comprised 22 sessions and wimcon. ducted during the initial development and testing of the autoganzfeld system. Study 2 comprised 9 sessions testing a procedure in which the experimenter, rather than the receiver, served as the judge at the end of the session. Study 3 comprised 35 sessions and served as practice for participants who had completed the allotted number of sessions in the ongoing formal studies but who wanted additional ganzfeld experience. This study also included several demonstration sessions whenTV film crews were present Novice Studies. Studies 101-104 were each designed to test 50 participants who had had no prior ganzfeld experi. ence; each participant served as the receiver in a single ganzfeld session. Study 104 included 16 of 20 students re- cruited from the Juilliard School in New York City to test an artistically gifted sample. Study 105 was initiated to accommodate the overflow of participants who had been recruited for Study 104, including the four remaining Jul- liard students. The sample size for this study was set to 25, but only 6 sessions had been completed when the labo- ratory dosed. For purposes of exposition, we divided the 56 sessions from Studies 104 and 105 into two parts: Study I04/105(a) comprises the 36 non-Juilliard partici- pants and Study 104/105(b) comprises the 20 Juilliard students. Study 201. This study was designed to retest the most promising participants from the previous studies. The number of trials was set to 20, but only 7 sessions with 3 participants had been completed when the laboratory closed. Study SOL This study was designed to compare static and dynamic targets. The sample size was set to 50 ses- sions. Twenty-five experienced participants each served as the receiver in 2 sessions. Unknown to the participants, the computer control program was modified to ensure that they would each have I session with a static target and 1 session with a dynamic target. Study 302. This study was designed to examine a dy- namic target set that had yielded a particularly high hit rate in the previous studies. The study involved experi- enced participants who had bad no prior experience with Us particular target set and who were unaware that only one target set was being sampled. Each served as the re- ceiver in a single session. The design called for the study to continue until 15 sessions were completed with each of the targets, but only 25 sessions had been completed when the laboratory closed. The 11 studies just descrf bed comprise all sessions con- ducted duting the 6.5 years of the program. There is no -me dra*er-* of unreported sessions. Results Over-all hit rate. As in the earlier meta-analysis, re- ceivers' ratings were analyzed by tallying the proportion of hiteachieved and calculating the exact binomial proba- bility for the observed number of bits compared with the chance expectation of .25, As noted earlier, 240 partici- Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04118 : CIA-RDP96-00789ROO2700020001-0 CPYRGHT @FER ANOWA-1,0US WF--OWJAT t@W ;ON 7W A @ @z 9 Table 2 Study 302. Emeded Hit Rafe and Propodion of Sessbns Inwhich, Each Vsdeo Gfip.was Ranked Fw-9 when it was a TaTet and when it was a Decov Relative Frequency of Ranked First RankodFirst Fashees Firstflaoe Expected. -when -@ when Exact Video Clip as Target Ranking Hit Rate (%) Target :Decoy Weronce P Tidal Wave 28 .24 6.72 ..57 (M2 5) f6W .1417) (2118). Snakes, .12 .112 1.44 ..67 .05 Pr2s) (3MS) (2/3) A6 .62 ..029 Sex Scene A16 -08 1.28 25 .05 20. (4/25) . (Z125) 41r2l) Bugs Bunny .44 .56 24X4 _82 .36 .46 .027 (l IM) .(W25) pill) (5114) Overall 08 ..48 .14 .44 sessions: of a aWdy are more sucicesafiil thari'laterses- sions.,Tf there were such an iffecttheri, studies with fewer sessions would showlarger edect sizes because. they wo,ild endbefore a decline could eket in. To check this pos- sibility, we computedpoint-biserial correlations between hits (1) or misses (6) and the session number within each of the 10 studies. Alllof the correlations hovered around zero,--six-were altive', faur were -negative, andtheoverall . . POM mean was. '01 An Inspeddon of Table I reveals that -the negative corre- lation derivesprimanly fi-om,the two--studies with the largest effect sizes: the 20 sessions withthe Juilliard. stu- dents and the 7 sessions of Study 201,,4be attidy-specifi.; callyidesigned to retest the =oatpronusmg-participants from the.1previous studies. Accordingly, it@peems likely that thelarger effect sizes,of these two studies,-and hence the significant negative-corrilation between the number ofveseions@ and the effect size reflectgenuine performance differences between these two small, highly selected.samples and other autoganzfeld participants. Study 302. All of the studies except Study 302 ran&mly sampled from a pool of 160 static and dynamic targets. Study 302 sampled from a single, dynamic -target set that had yielded a particularly high hit rate in the previous studies. The four film cli in this set consisted of a scene ps ofa tidal wave fi-orn the movie Ckmh ofthe Titans. a high- speed sex. scene from A Clockupork Orange. a scene of crawling snakes from 'a TV dpcumentary@ and a scene fi-om a Bugs Bunny cartoon. The experimental design called for this study to con- tinue until each of the clips had served as the target 16 times. Unfortunately, the premature termination of this study at 25 sessions left an imbalance in the frequency with which each clip had served as the target. This means that the high hit rate observed (64%) could well be in- flated by response biases. As an illustradon, water imagery is frequently reported by receivers in: ganzfeld sessions whereas sexual imagery is rarely reported. (Some participants are probably reluc. tant both to report sexual-imagery,and to give the highest rating to the sex-related clip.) If a video clip containing popular imagery <6uch as waterYliappensto appearps a target more-fiequently, than- aclip edritainingvripopular imji6ry - p1j - ect (such assex),-.a.high hitirate--raight iim ren the -coincidence of those frequencies'-of occurrence -with participants! -response biasea@'And, -as Ahe:sec6fid. column of Table 2 revaala@Ahetidal wave-clip iid,fri'Tact appear more fivquently.as the.-target than did-thweei clip. More generally, the @second and lthird,colu=4 of TWO 2'4how that the frequency with -which each Min clijW" aif 9 ranked first closely matches the frequency with which each ap- peared as the target One can adjustfor this problem by @using the observed fivquencies in these two column :to compute, the hit'rate expected-if there-were no-psi effect. Iri particular, one can multiply,eachproportion -in the second column by the cor- responding,proportion.-in the t1drd-iolumn--yielding the joint probability thatthe -clip'was the target and that it was ranked first --- and then sum across the four clips. As shown in the fourth column of Table 2, this computation yields man overall expected hit rate of 34.08%. When the observed -hitrate of 64% is compared with this.-baseline,.. the effect size -(h-3 is .61. As shown in Table 1, Us is equivalent to a four-alternative hit rate of 64 or a ir I %, value of .78, -and isi-statistically significant (z = 3.04, p .0012). 7he psi effect canbe seen even more clearly in -the re- rns.ining-colur.pris of Table 2, which control for the dffer- la I a 6 Ydiap Y the image in 0 entia popul @h M was the ho w w y fre d en i@ t t ar"ad ow quent y It w ed fi get p h as r in@ wh asr ps (deco 8)!A be 5 can T n @n the jectad th get-rel- i@w ee f ch as se say e e v ly 0 tly ps w th tV hen e re ftequMn when it was e ianw it waft a decoy, a, ' erence thaat is tor ithree of the four clips. On average, a clip was identified as the tar- get 58% of the time when it was the target and only 14% of the time when it was a decoy. Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 CPYRGHT ANOMAWUS INFORMATION TRANSFER ary (z = 2.23, p < .05, two-tailed). You now have cause to run an additional group of 10 subjects. What do you think the probability is that the results will be significant, by a one- tailed test, separately for this group9 (p. 106) The median estimate was .85, with 9 out of 10 respon- dents providing an estimate greater Om .60. The correct answer is approximately A8. As Rosenthal (1990) has warned: sGiven the levels of statistical power at which we normally operate, we have no right to expect the proportion of significant results that we typically do expect, even if in nature there is a ve'ry real and very important effect" (p. 16). In this regard, It. is again instructive to consider the medical study that found a highly significant effect of aspirin on the incidence of heart attacks. The study monitored more than 22,000 subjects. Had the investigators monitored 3,000 subjects, they would have had less than an even chance of finding a conventionally significant effecL Such is Iffe with small ef- fect sizes. Given its larger effect size, the prospects for success- fully replicating the psi ganzfeld effect are not quite so daunting, but they are probably still grimmer than intu- ition would suggeaL If the true hit rate is in fact about 34% when 25% is expected by chance, then an experiment with 30 trials (the -mean for the 28 studies in the original meta-analysis) has only about 1 chance in 6 of finding an effect significant at the .05 level with a one-tailed test. A 50-trial experiment boosts that chance to about I in 3. One must escalate to 100 trials in order to come close to the break even point, at which one has a 60-50 chance of finding a statistically significant affect (Utts, 1986). (Recall that only 2 of the 11 autoganzfeld studies yielded results that were individually significant at the conven- tional .05 level.) Those who require that a psi effect be statistically significant every time before they wiII seri- ously entertain the possibility that an effect really exists know not what they ask. S4gnificance Versus Effect Size il . The preceding discussion is unduly pessimistic, how- ever, because it perpetuates the tradition of worshipping the significance level. Regular readers of this journal am likely to be familiar with recent arguments imploring be- havioral scientists to overcome their slavish dependence on the significance level as the ultimate measure of virtue and instead to focus more of their attention on effect sizes: 'Surely, God loves the .06 nearly as much as the .05* (Roanow & Rosenthal, 1989, p. 1277). Accordingly, we suggest that achieving a respectable effect size with a methodologically tight ganzfeld study would be a perfectly welcome contribution to the replication effort, no matter how untenurable the p level renders the investigator. Career consequences aside, this suggestion may seem quite counterintuitive. Again, Tversky and Kahneman (1971) have provided an elegant demonstration. They asked several of their colleagues to consider an investiga- tor who runs 15 subjects and obtains a significant t value of 2.46. Another investigator attempts to duplicate the procedure with the same number of subjects and obtains a result in the same direction but with a nonsignificant value of t. Tversky and Kahneman then asked their col- leagues to indicate the highest level of t in the replication study they would describe as a failure to replicate. The majority of their colleagues regarded t = 1.70 as a failure to replicate. But if the data fi-ora two such studies Q = 2A6 11 and t w 1.70) were pooled, the t for the combined data would be about 3.00 (assuming equal variances): Thus, we are faced with a Paradoxical state of afTairs, in which the sarae data that would increase our confidence in the finding when viewed as part of the original study, shake our confidence when viewed as an independent study. (Trersky & Kahneman 1971, p. 108) Such is the iron grip of the arbitrary .05. Pooling the data, of course, is what meta-analysis is all about. Ac- oot-dingly, we suggest that two or more laboratories could collaborate in a ganzfeld replication effort by conducting independent studies and then pooling them in meta-ana- lytic fitsbion, what one tafght call real-time meta-analy- sis. (Each investigator could then claim the pooled p level for his or her own curriculum vitae.) Maximizing Effect Size Rather than buying or borrowing larger sample sizes, those who seek to replicate the psi ganzfeld effect might find it more intellectually satisfying to attempt to ma--d- mize the effect size by attending to the variables associ- ated with successful outcomes. Thus researchers who wish to enhance the chances of successful replication should use dynamic rather than static targets. Similarly we ad- vise using participants with the characteristics we have reported to be correlated with successful psi performance. Random college sophomores enrolled in introductory psy- chology do not constitute the optimal subject pool. Finally, we urge ganzfeld researchers to read carefully the detailed description of the warm social ambiance that Honorton etal. (1990) sought to create in the autoganzfeld laboratory. We believe that the social climate created in psi experiments is a critical determinant of their success or failure. The Problem of "Other" Variables This caveat about the social climate of the ganzfeld ex- periment prompted one reviewer of this article to worry that this provided 'an escape clause" that weakens the falsiflability of the psi hypothesis: 'Until Rem and Hon- orton can provide operational criteria for creating a warm social ambiance, the failure of an experiment with otherwise adequate power can always be dismissed as due to a lack of warmth.' Alas, it is true; we devoutly wish it were otherwise. But the operation of unknown variables in moderating the success of replications is a fact of life in all of the sci- ences. Consider, for example, an earlier article in this journal by Spence (1964). He reviewed studies testing the straightforward derivation from Hullian learning theory that high-anxiety subjects should condition more strongly than low-arixiety subjects. This hypothesis was confirmed 9 '4% of the time in Spence's own laboratory at the University of Iowa but only 63% of the time in labo- ratorie' at other universities. In fact, Kimble and hit; as- sociate: at Duke University and the University of North Caroiina*obtained results in the opposite direction in two of three experiments. In searching for a post hoc explanation, Spence (1964) noted that ma deliberate attempt was made in the Iowa studies to provide conditions in the laboratory that might elicit some degree of emotionality. Thus, the experi- menter was instructed to be impersonal and quite formal ... and did not try to put (subjects] at ease or allaY any Approved For Release 2003/04/18 CIA-RDP96-00789ROO2700020001-0 UPYKUHI ANOMALOUS INFORMATION TRANSFER Crae, 1992), which assesses six different facets of the ex- traversion-introversion factor. 77se sender. In contrast to this information about the re- ceiver in psi experiments, virtually nothing is known about the characteristics of a good sender or about the ef- fects of the sender's relationship with the receiver. As has been shown, the inf tial suggestion from the meta-analynis of the original ganzfeld database that psi performance might @be enhanced when the sender and receiver am friends was not replicated at a statistically significant level in the autoganzfeld studies. A number of parapsychologists have entertained the more radical hypothesis that the sender may not even be a necessary element in the psi process. In the terminology of parapsychology, the sender-receiver procedure tests for the existence of telepathy, anomalous communication be- tween two individuals; however if the receiver is somehow picking up the information from the target itself, it would be termed clairvoyance, and the presence of the sender would be irrelevant (except for possible psychological rea- sons such as expectation effects). At the time of his death, Honorton was planning a se- ries of autoganzfeld studies that would systematically compare sender and no-sender conditions while keeping both the receiver and the experimenter blind to the condi- tion of the ongoing session. In preparation, he conducted a meta-analytic review of ganzfeld studies that used no sender. He found 12 studies with a median of 33.5 ses- sions, conducted by seven investigators. The overall effect size (x) was .56, which corresponds to a four-alternative hit rate of 291%. But tWo eff6ct size does not reach statisti- cal significance (Stouffer x - 1.31, p - .095). So far, then, there is no firm evidence for psi in the ganzfeld in the ab- sence of a sender. (There are, however, nonganzfeld stud- ies in the literature that do report significant evidence for clairvoyance, including a classic card-guessing experiment conducted by J. B. Rhine and Pratt (19541.) The Physics of Psi The psychological level of theorizing discussed earlier does not, of course, address the conundrum that makes psi phenomena anomalous in the first place: their presurned incompatibility with our current conceptual model of physical reality. Parapsychologists differ widely from one another in their taste for theorizing at this level, but sev- eral whose training lies in physics or engineering have proposed physical (or biophysical) theories of psi phenom- ena (an extensive review of theoretical parapsychology was provided by Stokes, 1987). Only some of these theo- ries would force a radical revision in our conception of physical reality. Those who follow contemporary debates in modern physics, however. will be aware that several phenomena predicted by quantum theory and confirmed by expen- ment are themselves incompatible with our current con- ceptual model of physical reality. Of these, it is the 1982 empirical confirmation of Bell's theorem that has created the most excitement and controversy among Philosophers and the few physicists who are willing to speculate on such matters (Cusbing & McMullin, 1989; Herbert, 1987). In brief, Bell's theorem states that any model of reality that is compatible with quantum mechanics must be non- local: It must allow for the possibility that the results of observations at two arbitrarily distant locations can be correlated in ways that are incompatible with any pbysi- cally permissible causal mechanism. 13 Several possible models of reality that incorporate non- locality have been proposed by both Philosophers and physicists. Some of these models clearly rule out psi-like information transfer, others permit it, and some actually require it. 7hus, at a grander level of theorizing, some parapsychologists believe that one of the more radical models of reality compatible with both quantum mechan- ice and psi will eventually come to be accepted. If and when that occurs, psi phenomena would cease to be anomalous. But we have learned that all such talk provokes moet of our colleagues in psychology and in physics to roll their eyes and gnaah their teeth. So lees just leave it at that. Skepticism Revisited More generally, we have learned that our colleagues' tolerance for any kind of theorizing about psi is strongly determined by the degree to which they have been con- vinced by'the data that psi has been demonstrated. We have further learned that their diverse reactions to the data themselves are strongly determined by their a priori beliefs about and attitudes toward a number of quite gen- eral issues, some scientific, some not. In fact, several statisticians believe that the traditional hypothesis test- ing methods used in the behavioral sciences should be abandoned in favor of Bayesian analyses, w1dch take into account a person's a priori beliefs about the phenomenon under investigation (eg.. Bayarri & Berger, 1991; Daw- son, 1991). In the final analysis, however, we suspect that both one's Bayesian a prioris and one's reactions to the data are ultimately determined by whether one was more severely punished in childhood for Type I or Type II er- rors. References Atkinson, R., Atkinson, R. C., Smith, E. E., & Bem, D. J. (1990). Introduction to psychology (10th ed.). San Diego, CA: Harcourt Brace Jovanovich. Atkinson, R., Atkinson, R_ C., Smith, E. E., & Rem, D. J. (1993). Introduction to psychology (11th ed.). San Diego, CA_- Harcourt Brace Jovanovich. Avant, L. L. (1966). Vision in the ganzfeld. Psychological Bulletin, 64, 246-258. Bayard, M. J., & Berger, J. (1991). Comment. Statistical Science, 6, 379-382. Blackmore, S. (1980). The extent of selective reporting of ESP Ganzfeld studies. European Journal of Parapsy- chology, 3, 213-219. Bozarth, J. D., & Roberts, R. R. (1972). Signifying signifi- cant of gnificance. American Psychologist, 27,774-775. Braud, W. G., Wood, R, & Braud, L. W. (1975). Free-re- sponse GESP performance during an experimental h3rpngg6gic state induced by visual and acoustic ganzfeld techniques. A Replication and extension. Jour- nal of the American Society for Psychical Research, 69, 105413. Broughton, R. S. (1991). Parapsychology: The controver- sial science. New York: Ballantine Books. Broughton, R. S., Hanthamani, H, & KhiUi, A. (1990). As- sesaing the PRL success model on an independent ganzfeld data base. In L. Henkel & J. Palmer (Edo.), Re- ychology 1989 (pp. 32-35). Metuchen, search in paraps NJ: Scarecrow Press. Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 CPYRGHT a i-or KeieaseZUUJIU4IIU : L;IA-KUVtlb-UU[t$tPKUUZIUUUZUUUI-U ANOMAMUS WFORMATIONTRANSFER Roig, M, Icochea, H, & Cuzzucoli, A. (1991). Coverage of parapsychology in introductory psychology textbooks. Teaching of Psychology, 18, 157-160. Rosenthal, R. (1978). Combining results of independent studies. Psychological Bulletin, 85, i85_193. Rosenthal, R. (1979). The 'file drawer problem" and toler- ance for null results. Psychological Bulletin, 86, 638- 641. Rosenthal, R. (1990). Replication in I>ehavioral research. Journal ofSocial Behavior and Personality, 5, 1-30. Rosenthal, IL (1991). Meta-analytic prooedurce.for social research (Rev. ed.). Newbury Park, CA. Sage. Rosenthal, IL, & Rubin, D. B. (1989). Effect size estima- tion for one-sample multiple-choice-type data: Design, analysis, and meta-analysis. Psychological Bulletin, 106,332-337. Rosnow, R. L, & Rosenthal, R. (1989). Statistical proce- dures and the justification of knowledge in psychologi- cal science. American Psychologist, 44, 1276-1284. Sannwald, G. (1959). Statistische untersuchungen. an SpontanphAnomene [Statistical investigation of sponta- neous phenomena]. Zeitschrift flir Parapsychologie and Grenzgebiete der Psychologic, 3, 69-71. Saunders, D. IL (1985). On Hyman's factor analyses. Journal of Parapsychology, 49, 86-88. Schechter, E. 1. (1984). Hypnotic induction vs. control conditions: Illustrating an approach to the evaluation of replicability in parapsychology. Journal ofthe Anterican Society for Psychical Research, 78, 1-27. Schlitz, M. J., & Honorton, C. (1992). Ganzfeld psi per- formance within an artistically gifted population. Jour. nal of the American Society for Psychical Research, 86, 83-98. Schmeidler, G. R. (1988). Paraps ychology and psychology: Matches and Mismatches. Jefferson, NC: McFarland. Schmidt, H., & Schlitz, M. J. (1989). A large scale pilot PK experiment with prerecorded random events. In L. A. Henkel & R. E. Berger (Eds.), Research in Parapeychol- ogy 1988 (pp. 6-10). Metuchen, NJ: Scarecrow Preset. Spence, Y. W. (1964). Anxiety (drive) level and perfor- mance in eyelid conditioning. Psychological Bulletin, 61, 129-139. Stanford, R. G. (1987). Ganzfeld and hypnotic-induction procedures in ESP research: Toward understanding their success. In S. IKrippner (Ed.), Advances in para- psychological research (Vol. 5, pp. 39-76). Jefferson, NC: McFarland. Steering Committee of the Physicians' Health Study Re- search Group. (1988). Preliminary report: Findings from the aspirin component of the ongoing Physicians' Health Study. New England Journal of Medicine, 318, 262-264. Sterling, T. C. (1959). Publication decisions and their pos- aible effects on inferences drawn fi-orn testa of signifi- cance--or vice versa. Journal of the American Statisti. cal Association, 54, 30-34. Stokes, D. M. (1987). Theoretical parapsychology. In S. Krippner (Ed.), Advances in parapsychological research (Vol. 5, pp. 77-189). Jefferson, NC: McFarland. Swets, J. A., & Bjork, R. A@ (1990). Enhancing human per- formance: An evaluation of "new age* techniques con- Bidered by the U. S. Army. Psychological Science, 1, 85- 96. Tversky, A., & Kahneman, D. (1971). Belief in the law of small numbers. Psychological Bulletin, 2, 105-110. 15 Ullman, M., Kxippner, S, & Vaughan, A. (1973). Dream telepathy. New York- Macmillan. Utts, J. (1986). The ganzfeld debate: A statiatician,a per- spective. Journal of Paraps logy yrho 50,393-402. Utts, J. (1991a). Rejoinder. Statistical Science, 6,396-403. Utts, J. (1991b). Replication and meta-analysis in para- psychology. Statistical Science, 6,363-378. Wagner, M. W, & Monnet, M. (1979). Attitudes of college professors toward extra-sensory perception. Zetetic Scholar, 5, 7.17. Received September 28, 1992 Revision received March 10, 1993 Accepted March 14, 1993 Approved For Release 2003/04/18 CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 CPYRGHT sfta"&'al scim" im. Vol. 6. No. 4. W3-4M Replication and Mqta-Analysis in Pa''raps."yehology Jessica-Utts Abstract. Parapsychology, -the laboratory study of psychic -phenomena, has had its @history: -interwoven with that of statistics. Many -of the controvers .i-es in p.arapsychology have focused,on @statistical issues, and statisticaY models. have played -an integral. role @ -in the experimental work. Recently, - parapsychologists have been'using vaeta-analysis as a tool for -synthesWng -large bodies of work. This paper presents un ,overview of-the "use of statistics in-parapsychology and offers a summary of the meta-analyses that have been conducted. It begins with some anecdotal information about the involvementof statistics and statisti- cians with the e arly history of parapsychology. Next, it is argued that most nonstatisticians do not appreciate the -connection 'between power and "successful" replication of experimental effects. Returningto para- psychology, a particular experimental regime isexamined by summariz- ing an extended debateover the interpretation of the results. A new set of experiments- designed to resolve -the debate is then reviewed.- Finally, meta-analyses tiom several areas of parapsychology -are summarized. It is concluded that th e overall -evidence indicates thatthere is -an anoma- lous effect in need of an explanation. Key words and phrases.- Effect size, psychic research, statistical contro- versies,;randomness, vote-counting. 1. INTRODUCTION In a June 1990 Gallup Poll, 49% of the 1236 respondents claimed to believe in extragensory per- ception -(ESP), and one in four claimed to have had a personal experience involving telepathy (Gallup and Newport, 1991). Other surveys have shown even higher percentages; the University of Chicago's National Opinion Research Center re- cently surveyed 1473 adults, of which 67% claimed that they had experienced ESP (Greeley, 1987). Public,opinion. is a poor arbiter of science, how- ever. and experience is apoor substitute for the scientific. method. For more than a century, small numbers of. scientists have been conducting labora- tory experiments to study phenomena such as telepathy, clairvoyance and precognition, collec- tively known as "psi" abilities. This paper will examine some of that work, as well as some of the statistical controversies ithas generated. Jessica Utts is Associate Professor, Division of Parapsychology, as this field is called, has been a- sourceof controversy throughout its history. Strong beliefs tend to be -resistant to -change even in the face of data, and many people, scientists included, seem to have made up their minds on the question without examinin. any empirical data at all. A critic of parapsychology recently acknowledged that "The level of the debate during the past 130 years has been an embarrassment for anyone who would like to'Wlieve -that scholars and scientists adhere to standards of rationality and fair play" (Hyman, 1985a, page 89). While much of the controversy has focused on poor experimental design and potential fraud, there have been attacks and defenses of. the statisiical methods as well, sometimes calling into question the very foundations of probability and statistical inference. Most of the criticisms have been leveled by psy- chologists. For example, a 1988 report of the U.S. National Academy of Sciences@ concluded that "The fro committee finds no scientific justification- In -of 130; -years for research conducted over a period the exist46nce' "of parapsychological` phenomentl!@, .1 no o "V q%piaanter Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 CPYRGHT REPLICATION IN PARAPSYCHOLOGY One of the first American researchers to use statistical methods in parapsychology was John Edgar Coover, who was the Thomas Welton Stanford Psychical Research Fellow in the Psychol- ogy Department at Stanford University from 1912 to 1937 (Dommeyer, 1975). In 1917, Coover pub- lished a large volume summarizing his work (Coover, 1917). Coover believed that his results were consistent with chance, but others have ar- gued that Coover's definition of significance was too strict (Dommeyer, 1975). For example, in one evaluation of his telepathy experiments, Coover found a two-tailed p-value.1of .0.0062. He concluded, "Since this value, then, ties within the field of chance deviation, although the probability of its occurrence by chance is fairly low, it cannot be accepted as a decisive indication of some cause beyond chance which operated in favor of success in guessing" (Coover, 1917, page 82). On the next page, he made it explicit that he would require a p-value of 0.0000221 to declare that something other than chance was operating. It was during the summer of 1930, with the card-guessing experiments of J. B. Rhine at Duke University, that parapsyc .hology began to take hold as a laboratory science. Rhine's laboratory still exists under the name of the Foundation for Re- search on the Nature of Man, housed at the edge of the Duke University campus. It wasn't long after Rhine published his first book, Extrasensory Perception in 1934, that the attacks on his methodology began. Since his claims were wholly based on statistical analyses of his experiments, the statistical methods were closely scrutinized by critics anxious to find a conventional explanation for Rhine's positive results. The most persistent critic was a psychologist from McGill University named Chester Kellogg (Mauskopf and McVaugh, 1979). Kellogg's main argument was that Rhine was using the binomial distribution (and normal approximation) on a se- ries of trials that were not independent. The experi- ments in question consisted of having a subject guess the order of a deck of 25 cards, with five each of five symbols, so technically Kellogg was correct. By 1937, several mathematicians and statis- ticians had come to Rhine's aid. Mauskopf and McVaugh (1979) speculated that since statistics was itself a young discipline, "a number of statisticians were equally outraged by Kellogg, whose argu- ments they saw as discrediting their profession" (page 258). The major technical work, which ac- knowledged that Kellogg's criticisms were accurate but did little to change the significance of the - . . - . - I 365 and Greenwood, 1937). Stuart, who had been an undergraduate in mathematics at Duke, was one of Rhine's early subjects and continued to work with him --as a researcher until Stuart's death in 1947. Greenwood was a Duke mathematician, who appar- ently converted to a statistician at the urging of Rhine. Another prominent figure who was distressed with Kellogg's attack was E. V. Huntington, a mathematician at, Harvard. After corresponding with.Rhine, Huntington decided that, rather than further confuse the public with a technical reply to Kellogg's arguments, a simple statement should be made to the effect that the mathematical issues in Rhine's work had been resolved. Huntington must have successfully convinced his former student, Burton Camp of Wesleyan, that this was a wise approach. Camp was the 1937 President of IMS. When the annual meetings were held in December of 1937 (jointly with AMS and AAAS), Camp released a statement to the press that read: Dr. Rhine's investigations have two aspects: experimental and statistical. On the exper- imental side mathematicians, of course, have nothing to say. On the statistical side, however, recent mathematical work has established the fact that, assuming that the' experiments have been properly performed, the statistical analysis is essentially valid. If the Rhine investigation is to be fairly attacked, it must be on other than mathematical grounds (Camp, 19371. One statistician who did emerge as a critic was William Feller. In a talk at the Duke Mathemati- cal Seminar on April 24, 1940, Feller raised three criticisms to Rhine's work (Feller, 1940). They had been raised before by others (and continue to be raised even today). The first was that inadequate shuffling of the cards resulted in additional infor- mation from one series to the next. The second was what is now known as the "file-drawer effect," namely, that if one combines the results of pub- lished studies only, there is sure to be a bias in favor of. successful studies. The third was that the results were enhanced by the use of optional stop- ping, that is, by not specifying the number of trials in a *diance. All three of these criticisms were ad- dressed in a rejoinder by Greenwood and Stuart (1940), but Feller was never convinced. Even in its third edition published in 1968, his book An Intro- duction to Probability Theory and Its Applications still contains his conclusion about Greenwood and Stuart: "Both their arithmetic and their expen- U-_ . A;c+;"1_f f;ncrp nf the supernatural" Approved For Release 2003/04/18 CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 CPYRGHT REPLICATION IN PARAPSYCHOLOGY their colleagues at a professional meeting, with ie question: An investigator has reported a result that you consider implausible. He ran 15 subjects, and reported a significant value, t = 2.46. Another investigator has attempted to duplicate his pro- cedure, and he obtained a nonsignificant value of t with the same number of subjects. The direction was the same in both sets of data. You are reviewing'the literature. What is the highest value of t in the second set of data that you would describe as a failure to replicate? [1982, page 281. reporting their results, Tversky and Kahne- an stated: The majority of our respondents regarded t 1.70 as a failure to replicate. If the data of two such studies (t = 2.46 and t = 1.70) are pooled, the value of ( for the combined data is about 3.00 (assuming equal variances). Thus, we are faced with a paradoxical state of affairs, in which the same data that would increase our confidence in the finding when viewed as part of the original study, shake our confidence when viewed as an independent study (1982, page 28). At a recent presentation to the History and Phi- osophy of Science Seminar at the University of .alifornia at Davis, I asked the following, question. @wo scientists, Professors, A and B, each, have a heory they would like to demonstrate. Each plans o run a fixed number of Bernoulli trials.and then est HO: p = 0.25 versus H.: p > 0.25. Professor A Las access to large numbers of students each emester to use as subjects. In his first experiment, ke runs 100 subjects, and there are 33 successes p = 0.04, one-tailed). Knowing the importance of eplication, Professor A runs an additional 100 sub- acts as a second experiment. He finds 36 successes p = 0.009, one-tailed). Professor B only teaches small classes. Each [uarter, she runs an experiment on her students to est her theory. She carries out ten studies this vay, with the results in Table 1. 1 asked the audience by a show of hands to ndicate whether or not they felt the scientists had uccessfully demonstrated their theories. Professor Cs theory received overwhelming support, with .pproximately 20 votes, while Professor B's theory eoeived only one vote. If you aggregate the results of the experiments c)r each professor, you will notice that each con- 367 with 71 as opposed to 69 successful trials. The one-tailed p-values for the combined trials are 0.0017 for Professor A and 0.0006 for Professor 13. To address the question of replication more ex- plicitly, I also posed the following scenario. in December of 1987, it was decided to prematurely terminate a study on the effects of aspirin in reduc. ing heart attacks because the data were so convinc- ing (see, e.g., Greenhouse and Greenhouse, 1988; Rosenthal, 1990a). The physician-subjects had been randomly assigned to take aspirin or a placebo. There were 104 heart attacks among the 11,037 subjects in the aspirin group, and 1-89 heart attacks among the 11,034 subjects in the placebo group (chi-square = 25.01, p < 0.00001). After showing -the results of that study, I pre- sented the audience with two hypothetical experi- ments conducted to try to replicate the original result, with outcomes in Table 2. I asked the audience to indicate which one they thought was a more successful replication. The au- dience.chose -the second one, as would most journal editors, because of the "significant p-value." In fact, the first replication has almost exactly the same proportion of heart attacks in the two groups as the original study and is thus a very close repli- cation of that result. The second replication has TABLE 1 Attempted repkiations for professor B a Number of successes One-tailed p-value 10 4 0.22 15 6 0.15 17 6 0.23 25 8 0.17 30 10 0.20 40 13 0.18 18 7 0.14 10 5 0.08 15 5 0.31 20 7 0.21 TABLE 2 Hypothetical replications of the aspirin 1heari attack study Replication # 1 Replication #2 Heart attack Heart attack Yes No Yes No - Aspirin 11 1156 20 2314 Placebo 19 1090 48 2170 - n AMI Approved For Release 2003/04/18 CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 CPYRGHT REPLICATION IN PARAPSYCHOLOGY target, rather than being forced to make a choice from a small discrete set of possibilities. Various types of target material have been used, including pictures, short segments of movies on video tapes, actual locations and small objects. Despite the more complex target material, the statistical methods used to analyze these experi- ments are similar to those for forced-choice experi- ments. A typical experiment proceeds as -follows. Before conducting any trials, a large pool of poten- tial targets is assembled, usually in packets of four. Similarity of targets within a'packet is kept to a minimum, for reasons made clear below. At the start of an experimerittil session, after the subject is sequestered in an isolated -room, a target is selected at random from the pool. A sender is placed in another room with the target. The subject is asked to provide a verbal or written description of what he or she thinks is in the target, knowing only that it is a photograph, an object, etc. After the subject's description has been recorded a.nd secured against the potential for later alter- ation, a judge (who may or may not be the subject) is given a copy of the subject's description and the four possible targets that were in the packet with the correct target. A properly conducted experi- ment either uses video tapes or has two identical sets of target material and uses the duplicate set for this part of the process, to ensure that clues such as fingerprints don't give away the answer. Based on the subject's description, and of course on a blind basis, the judge is asked to either, rank the four choices from most to least likely to have been the target, or to select the one from the four that seems to best match the subject's description. If ranks are used, the statistical analysis proceeds by summing the ranks over a series of trials and comparing the sum to what would be expected by chance. If the selection method is used, a "direct hit" occurs if the correct target is chosen, and the number of direct hits over a series of trials is compared to the number expected in a binomial experiment with P = 0.25. Note that the subjects' responses cannot be con- sidered to be "random" in any sense, so probability assessments are based on the random selection of the target and decoys. In a correctly designed ex- periment, the probability of a direct hit by chance is 0.25 on each trial, regardless of the response, and the trials are independent. These and other issues related to analyzing free-response experiments are discussed by Utts (1991). 4.2 The Psi Ganzfeld Experiments 369 isolation technique originally developed by Gestalt psychologists for other purposes. Evidence from spontaneous case studies and experimental work had led parapsychologists to a model proposing that psychic functioning may be ina ked by sensory in- put and by inattention to internal states (Honorton, 1977).. The ganzfeld procedure was specifically de- signed to test whether or not reduction of external it noise" would enhance psi performance. In these experiments, the subject is placed in a comfortable reclining chair in an acoustically shielded room. To create a mild form of sensory deprivation, the subject wears headphones through which white =ise is played, and stares into a constant field of red light. This is achieved by taping halved translucent ping-pong balls over the eyes and then illuminating the room with red light. In the psi ganzfeld experiments, the subject speaks into a microphone and attempts to describe the target material being observed by the sender in a distant room. At the 1982 Annual Meeting of the P&rapsycho- logical Association, a -debate took place over the degree to which the results of the psi ganzfeld experiments constituted evidence of psi abilities. Psychologist and critic Ray Hyman and parapsy- chologist :Charles Honorton each analyzed the re- sults of all known psi ganzfeld experiments to date, and they reached strikingly different conclusions (Honorton, 1985b; Hyman, 1985b). The debate con- tinued with the publication of their arguments in separate articles in the March 1985 issue of the Journal of Parapsychology. Finally, in the Decem- ber 1986 issue of the Journal of Parapsychology, Hyman and Honorton (1986) wrote a joint article in which they highlighted their agreements and disagreements and outlined detailed criteria for future experiments. That same issue contained commentaries on the debate by 10 other authors. The data base analyzed by Hyman and Honorton (1986) consisted of results taken from 34 reports written by a total of 47 authors. Honorton counted 42 separate experiments described in the reports, of which 28 reported enough information to determine the number of direct hits achieved. Twenty three of the studies (55%) were classified by Honorton as having achieved statistical significance at 0.05. 4.3 The Vote-Counting Debate Vote-counting is the term commonly used for the technique of drawing inferences about an experi- mental effect by counting the number of significant versus nonsignificant studies of the effect. Hedges and Olkin (1985) give a detailed analysis of the nr t),iq mpthnA , 7 that it is more .,howinL Approved For Release 2003/04/18: CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 CPYRGHT REPLICATION IN PARAPSYCHOLOGY ization, multiple tests used without a4justing the significance level (thus inflating the significance level from the nominal 5%) and'failure to use a duplicate set of targets -for the judging process (thus allowing possible-clues such as fingerprints). Using cluster and factor analyses, the 12 binary flaw variables were combined into three new variables, which Hyman named General Security, Statistics and Controls. Several analyses were then conducted. The one reported with the most detail is a factor analysis utilizing 17 variables for each of S6 studies. Four factors emerged from the7 analysis., From -these, Hyman concluded that security had Increased -over the years, that the significance level tended to be inflated the most for the most complex studies and that both effect size and level of significance were correlated with the existence of flaws. Following his factor analysis, Hyman picked the three flaws that seemed to be most highly corre- lated with success, which were inadequate atten- tion to both randomization and documentation and the potential for ordinary communication between the sender and receiver. A re tion was . gression equa then computed using each of the three flaws as dummy variables, and the effect size for the experi- ment as the dependent variable. From this equa- tion, Hyman concluded that a study without these three flaws would be predicted to have a hit rate of 27%. He concluded that this is "well within the statistical neighborhood of the 25% chance rate" (1985b, page 37), and thus "the ganzfeld psi data base, despite initial impressions, is inadequate ei- ther to support the contention of a repeatable study or to demonstrate the reality of psi" (page 38). Honorton discounted both Hyman's flaw classifi- cation and his analysis. He did not deny that flaws existed, but he objected that Hyman's analysis was faulty and impossible to interpret. Honorton asked psychometrician David Saunders to write an Ap- pendix to his article, evaluating Hyman's analysis. Saunders first criticized Hyman's use of a factor analysis with 17 variables (many of which were dichotomous) and only 36 cases and concluded that "the entire analysis is meaningless" (Saunders, 1985, page 87). He then noted that Hyman's choice of the three flaws to include in his regression anal- ysis constituted a clear case of multiple analysis, since there were 84 possible sets of three that could have been selected (out of nine potential flaws), and Hyman chose the set most highly correlated with effect size. Again, Saunders concluded that "any interpretation drawn from (the regression analysis] must be regarded as meaningless" (1985, page 88). 371 Hyman in his capacity as Chair of the National Academy of Sciences' Subcommittee on Parapsy. chology. Using Hyman's flaw classifications and a multivariate analysis, Harris and Rosenthal con- cluded that "Our analysis of the effects of flaws on study outcome lends no support to the hypothesis that ganzfeld research results are a significant function of the set of flaw variables" (1988b, page-3). Hyman and Honorton were in the process of preparing papers for a second round of debate when they were invited to lunch - together at the 1986 Meetingof the Parapsychological Association. They discovered.that they were in general agreement on several major issues, and they decided to coauthor a "Joint Communiqu6" (Hyman and Honorton, 1986). It is clear from their paper that they both thought it was more important to set the stage for future experimentation than to continue the techni- cal arguments over the current data base. In the abstract to their paper, they wrote: We agree that there is an overall significant effect in this data base that cannot reasonably be explained by selective reporting or multiple analysis. We continue to differ over the degree to which the effect constitutes evidence for psi, but we agree that the final verdict awaits the outcome of future experiments conducted by a broader range of investigators and according to more stringent standards [page 3511. The paper then outlined what these standards should be. They included controls against any kind of sensory leakage, thorough testing and documen- tation of randomization methods used, better re- porting of judging and feedback protocols, control for multiple analyses and advance specification of number of trials and type of experiment. Indeed, any area of research could benefit from such a careful list of procedural recommendations. 4.5 Rosenthal's Meta-Analysis The same issue of the Journal of Parapsychology in which the Joint Communiqu6 appeared also car- ried commentaries on the debate by 10 separate author@. In his commentary, psychologist Robert Rosenthal, one of the pioneers of meta-analysis in psychology, summarized the aspects of Hyman's and Honorton's work that would typically be in- cluded in a meta-analysis (Rosenthal, 1986). It is worth reviewing Rosenthal's results so that they can be used as a basis of comparison for the more recent psi ganzfeld studies reported in Section 5. Rosenthal, like Hyman and Honorton, focused .,1-- 00 r-- , Approved For Release 2003/04/18 CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 CPYRGHT REPLICATION IN PARAPSYCHOLOGY likely to be, selected by the computer's random number generator than any of the others in the- set. The selection of the target @by the computer is the only source of randomness in these experiments. This is an important point,.,and one that is often misunderstood-'(S@ee Utts; 199'1, for elucidation.) Eighty of the targets were "dynaetc," consisting of scenes from movies, documentaries, Andcartoons; 80 were .ic static," consisting of photo: graphs, art printsand-advertisements. The four targets within each set were all.of the same type. Earlier studies indicated that.dynamic targets were more *likely to produce successful results,, ina'.one' of the goals of the new. experiments was -to tesi that the 'ory..' The randomization procedure used to select the target and the. order of presentation for-judging was thoroughly tested before and during the experi- men ts. A detailed description isgiven by Honorton et al. (1990, pages 118-120). Three of the 11 series were pilot series, five were formal series with novice receivers, and three were formal series.with experienced receivers. The last series with experienced receivers was.the,only one that did not use the 166 targets. Instead, it used only one set ,of four dynamic targets in which one target had. previously received several first place ranks and one had never received a first place rank. The receivers, none of whom had had prior exposure tothat target pack, were not aware that only one target pack was being used. They each contributed one sessionvaly tothe'series. This will be called the ""special series"' in what follows. Except -for two of the pilot series, numbers of trials were- planned in advance for eAch series. Unfortunately. three of the formal series were not yet completed when thefunding ran out, including the special series, and one pilot study -with @advance planning was terminated early -when the experi- menter relocated. There were no unreported trials during the,6-year period under review, so there was no "file drawer." Overall, there were 183 Rs who contributed only one trial and 58 who contributed more than one, for a total of 241 participants and 355 trials. Only 23 Rs had previously participated in ganzfeld experi- ments, and 194 Rs (81%) had -never participated in any parapsychological research. 5.2 Results While acknowledging that no probabilistic con- clusions can be drawn from qualitative data, Hon- orton et al. (1990) included several examples of session excerpts that Rs identified as'Providing the basis for their target rating. To give a flavorfor the 373 rank, the first example is reproduced here. The target was a painting by Salvador Dali called "Christ. Crucified." The correct target received a first place rank. The part of the mentation R used to make this -assessment read., ... I think. of guides, like spirit guides, leading me and 1, come into a court with a king. 'It's quiet .... It's like heaven. The king is some- thing like Jesus. Woman. Now rm just sort of su 'mmersaulting through heaven .... Brooding .... Aztecs, the Sun God.... High priest Year .... Graves. Woman. P rayer . . . . Funeral i . . . Dark. Death .... Souls .... Ten Commandments. Moses .... (Honorton et al., 19901. Over all 11 series, there were 122 direct hits in the 355 trials, for a hit -rate of 34.4% (exact bino- mial p-value = 0.00005) when 25% were expected by chance. Cohen's h is 0.20, and a 95% confidence interval for the overall hit rate is from 0.30 to 0.39. Thiscalculation assumes, of course, that the proba- bility of a direct hit is constant and independent across trials, an assumption,that may be question- able except -under the null hypothesis of no psi abilities. Honorton et al. 41990) also calculated effect sizes for each of the 11 series and each of the eight experimenters. All -but one of the series (the first novice series) had positive effect sizes, as did all -of - the experimenters. The special series with experienced Rs had an exceptionally high effect sizewith h = 0.81, corre- sponding to .16 direct hits out of 25 trials (64%), but the remaining series -and the experimenters had relatively homogeneous effect sizes given the amount of variability expected by chance. If the special series is removed, the overall hit rate is 32.1%, h = 0.16. Thus, the positive effects are not due to just one series or one experimenter. Of the 218 trials contributed by novices, 71 were direct hits (32.5%, h = 0.17), compared with 51 hits in the 137 trials by those with prior ganzfeld experie.ncd (37%, h 0.26). The hit rates and effect sizes were 31% (h 0.14) for the combined,pilot series, 32.5% (h = 0.17) for the combined formal novice. series, and 41.5% (h = 0.35) for the com- bined experienced series. The last figure drops to 31.6% if the outlier series is removed. Finally, without the outlier series the hit rate for the com- bined series where all of the planned trials were completed was 31.2% (A = 0.14), while it was 35% (h = 0.22) for the combined series that were termi- Approved For Release 2003/04/18 CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 CPYRGHT REPLICATION IN PARAPSYCHOLOGY scores from zero for the lowest quality, to eight for the highest. They included features such as ade- quate randomization, preplanned analysis and au- tomated recording of the results. The correlation between study quality and effect size was 0.081, indicating a slight tendency for 'nigher quality studies to be more successful, contrary to claims by critics that the opposite would be true. There was a clear relationship between quality and year of publication, presumably because over the years experimenters in parapsychology have responded to suggestions from critics for improving their methodology. File Drawer. Following 'Rosenthal (1984), the authors calculated the "fail-safe N" indicating the number of unreported studies that would have to be sitting in file drawers in order to negate the signifi- cant effect. They found N = 14,268, or a ratio of 46 unreported studies for each one reported. They also followed a suggestion by Dawes, Landman and Williams (1984) and computed the mean z for all studies with z > 1.65. If. such studies were a ran- dom sample from the upper 5% 'tail of a N(O, 1) distribution, the mean z would be 2.06. In this case it was 3.61. They concluded that selective reporting could not explain these results. Comparisons. Four variables were identified that appeared to have a systematic relationship to study outcome. The first was that the 25 studies using subjects selected on the basis of good past performance were more successful, than the .223 using unselected subjects,.` with mean effect -sizes,-,of 0.051 and 0.008, respectively. Second, the 97 stud- ies testing subjects individually were more success- ful than the 105 studies that used group testing; mean effect sizes were 0.021 and 0.004, respec- tively. Timing of feedback was the third moderat- ing variable, but information was only available for 104 studies. The 15 studies that never told the subjects what the targets were had a mean effect size of -0.001. Feedback after each trial produced the best results, the mean ES for the 47 studies was 0.035. Feedback after each set of trials re- sulted in mean ES of 0.023 (21 studies), while delayed feedback (also 21 studies) yielded a mean ES of only 0.009. There is a clear ordering; as the gap between time of feedback and time of the actual guesses decreased, effect sizes increased. The fourth variable was the time interval be- tween the subject's guess and the actual target selection, available for 144 studies. The best results were for the 31 studies that generated targets less than a second after -the guess (mean ES = 0.045), while the worst were for the seven studies that 375 trend, decreasing in order as the time interval increased from minutes to hours to days to weeks to months. 6.2 Attempts to Influence Random Physical Systems Radin and Nelson (1989) examined studies de- signed to test the hypothesis that "The statistical output of an electronic RNG [random number gen- eratorl is correlated with observer intention in ac- cordance with prespecified instructions" (page 1502).,These experiments typically involve RNGs based on radioactive decay, electronic noise or pseu- dorandom number sequences seeded with true ran- dom sources. Usually the subject is instructed to try to influence the results of a string of binary trials by mental intention alone. A typical protocol would ask a subject to press a button (thus starting the collection of a fixed.length sequence of bits), and then try to influence the random source to produce more zeroes or more ones. A run might consist of three successive button presses, one each in which the desired result was more zeroes or more ones, and one as a control with no conscious intention. A z score would then be computed for each button press. T@e 832 studies in the analysis were conducted from 1959 to 1987 and included 235 "control" stud- ies, in which the output of the RNGs were recorded but there was no conscious intention involved. These were usually conducted before and during the,. experimental series, as tests of the RNGs. Results. The effect size measure used was again z / vrn-, where z was positive if more bits of the specified type were achieved. The mean effect size for control studies was not significantly different from zero (-1.0 X 10-6). The mean effect size for the experimental studies was also very small, 3.2 x 10 - 1, but it was significantly higher than the mean ES for the control studies (z = 4-1). Quality. Sixteen quality measures were defined and assigned to each study, under the four general categories of procedures, statistics, data and the RNG device. A score of 16 reflected the highest quality. The authors regressed mean effect size on mean quality for each investigator and found a slope of 2.5 x 10-5 with standard error of 3.2 x 10-'5, indicating little relationship between quality and outcome. They also calculated a weighted mean effect size, using quality scores as weights, and found that it was very similar to the uriweighted mean ES. They concluded that "differences in methodological quality are not significant predictors of effect size" (page 1607). . " . - I %T_1___ .-A CAVArAl Approved For Release 2003/04/18 CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 CPYRGHT REPLICATION INPARAPSYCHOLOGY The correlation between extroversion scoresand ganzfeld rating scores was r = 0.18, -with a 95% confidence interval from 0.05 to 0.30. This is con. sistent with 'the mean correlation.of r = 0.20 for free-response experiments, i determined, -from the meta_an@dysis. 7hese correlations indicate that @ex- troverted subjects can produce --higher -scores 4n free-response ESP tests. 7 CONCLUSIONS Parapsychologists often make a distinction be- tween "proof-oriented research" and"I" process- oriented researcli.`@-Tlie,@'f6rmer is typica jly con- ducted to test the hypothesis that-psiabilities exist, while -the latter is Aesigned to answer -questions about how psychic- functioning works. Proof- oriented research has domi*nated 'the literature in . parapsychology. . Unfortunately, many ,of the studies used. small samples and would thus be nonsignificant even if a moderate-sized effect exists. The recent focus on meta-analysis in parapsy- chology has revealed -that there are small but consistently nonzero effects across -studies, experi- menters and laboratories. The sizesof the effects in forced-choice studies appear to bo comparable to those - reported in Some medical studies that had been heralded as breakthroughs. (See Section 5; also Honorton -and Ferrari, -1989,.page 301.) Free- response studies show effect @sizes of far 'greater magnitude. A promising direction for futureprocess-oriented research is to examine the causes of -individual differences in psychic functioning. The ESP/ex- troversion meta-analysis Is a step in thatAirection. In keeping with the idea of individual differ- ences, Bayes and empirical Bayes methods would appear to make more sense than the classical infer- ence methods commonly used, since they would allow individual abilities and beliefs to be modeled. Jeffreys (1990) reported a Bayesian analysis of some of the RNG experiments and showed that conclu- sions were closely tied toprior beliefs,even though hundreds of thousands of -trials were available. it may be that the nonzero effects observed in the meta-analyses can be explained by something other than ESP, such as shortcomings in our understand- ing of randomness and independence. Nonetheless, there is an anomaly that needs an explanation. As I have argued elsewhere (Utts, 1987), research in parapsychology should receive more 'support, from the -scientific community., If ESP does not . exist, there is little to-be lost by erring-in thedirection,of r__@L --- ------- ro@+ li"t-nuar Afb^@ 377 much to be gained by discovering how to enhance and apply these abilities to important world problems. ACKNOWLEDGMENTS I would like to thank Deborah Delanoy., Charles Honorton, Wesley Johnson, Scott Plous and an anonymousreviewer for their helpfuleomments on an earlierdraft of -this paper,and Rbbertlto@enthal and Charles Honorton -for discussions that helped tans. clarify de -REFERENCES ATKINSON. R. L., ATKINSON, R. C., SMITH, E. E. and BE&I, D. J. 11990). Introduction to Psychology. 10th ed. Harcourt Brace Jovaaovich..,San Diego. BELOFF, J. (1985). Research strategies for dealing with unstable phenomena. In The Repeatability Problem in Parapsychol- ogy (B. Shapin and L Coly. eds.) 1-21. Parapsychology ;Foundation.,New York. BLAcKmoRE. S. J. (1985). Unrepeatability: Parapsychology's only finding. In The Repeatabilily.Problent in Parapsychology (B. Shapin and 1. Coly. eds.) 183-206. Parapsychology Foundation. New York_ BURDICK, D.,S.,and KELLY, -E. F. (1977). Statistical methods in parapsychological.research. In Handbookof Parapsychology (B. B. Wolmnn ed.) 81-130. Van Nostrand Reinholdi New York. CAMP, B. H..(1937). (Statement in Notes Section.) Journal of Parapsychology 1305. COHEN. J. (1990). Things I have learned (so far). American Psychologist 451M-1312. COOVER. J. E. (1917). Ej:periments in Psychical Research at Leland Stanford Junior University. Stanford Univ. DAWES. R. M., LANDMAN, J. and WiLuAms,.J. (1984). Reply to Kurosawa. American Psychologist 39 74-75. DIACONtS, P. -(1978). Statistical problems in ESP research. Sci- ence 201 131-136. DOMMEYER, F. C. (1975). Psychical research at Stanford Univer- sity. Journal of Parapsychology 39 173-205. DRUCKMAN. D. and SwEm J. A., eds. (1988) Enhancing-Human Performance: Issues, Theories, and Techniques. National Academy Press, Washington, D.C. EDGEWORTH. F. Y. (1885). The calculus of probabilities applied to psychical research. In Proceedings of the Society for Psychical Research 3 190-199. EDGEWORTH, F. Y. (1886). The calculus of probabilities applied to psychical research. II. In Proceedings of the Society for Psychical Research 4 189-208. FELLER, W. K. (1940). Statistical aspects of ESP. Journal of Parapsychology 4 271-297. FELLER: W. K. (1968). An Introduction.to Probability Theory and Its Applications 1, 3rd ed. Wiley, New York. FISHER, R. A. (1924). A method of scoring coincidences in tests with playing cards. In Proceedings of the Socie@v for Psychi- cal Research 34 181-185. FISHER. R. A. (1929). The statistical method in psychical re- search. In Proceedings of the Socie4y for Psychical Research 39189-192. GALLUP, G. H., JR., and NEWPORT, F. (1991). Belief in paranor- mal phenomena among adult Americans. Skeptical I@qfdrer .15137-146. - .- . . _. - - -- intervals Approved For Release 2003/04/18 CIA-RDP96-00789ROO2700020001-0 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 CPYRGHT J. UTTS com, meknt m. J. Bayard and James Berger 1. INTRODUCTION There are many fascinating issues discussed in this - paper.'-Several concern parapsychology, %itself and the interpretation of statistical methodology therein.'We are not experts in parapsychology, and so have only one comment concerning such mat- ters: In Section 3 we briefly discuss the need to switch from P-values to Bayes factors in discussing evidence zoncerning parapsychology. : A more general issue raised in -the,paper is that of replication. It is quite illuminating to consider the issue of replication from -a Bayesian perspec- tive, and this is done in Section 2 of our discussion. 2. REPUCATION Many insightfulobservations concerning Xeplica. tion are given inthe article, and-these@ spurred us to determine if they could he quantified within Bayesian reasoning. Quantification requires clear delineation of the possible purposes of replication,- and at least two are obvious. The first is simple reduction of random error, achieved by obtaining more observations from the replication. The second purpose is to search for possible bias in the original experiment, We use "bias" in a loose sense here, to of refer to any bfth6 huge -number ' ways-in which the effects being measured by the experiment can differ from the actual effects of interest. Thus a clinical trial without a placebo can suffer a placebo "bias"; a survey can suffer a "bias" due to the. sampling frame being unrepresentative of the actual population; and possible sources of bias in parapsychological experiments have been extensively discussed. 379 ExAmPLE 1. Consider the example from Tversky and Kahnemarm, 0982), in wWi@h, an experiment results'..inA standardized test statistic of z, - 2.46. (W@ *111 assume normality to keep comix"tions trivial'y The' question is: What isAhe, highest - value Of Z2 in' 6 second set of data that would be,consid-,- ered a -failure to replicate? Two possible precise versions of this question are: Question 1: What is the probability of observing z2 for which the null hypothesis would be rejected in the replicated ex- periment? Question 2: What value of z2 would leave one's overall opinion about the null hypothe- sis. unchanged? Consider the simple case where Z, - N(z, 16, 1) and 0. and suppoise 'that'liniti'' prior -opinion abdat 0 tan le.described by 'the nonintbrniative prior* ir(O) -- i. We consider''ih~'.6'ne-sided't6ding problem with a@ constaiit, .prior in-Ahis secti6n, bHe- cause it is known'that then'the,, "' erior probabil- ity 0 to'be denoted by P(Hojdata), equals the f Hot P-value, allowing us to avoid complications. arising from -differences between Bayesian and classical a answers. After observing z, 2.46, the posterior distribu- tion of 0 is r(6 I z,) N(012.46, 1). Question 1 then has the answer (using predictive Bayesian reasoning) Replication to Reduce Random Error If the sole goal of replication of an experiment is to reduce rundom error, matters are very straight- forward. Reviewing the Bayesian way of studying this .issue is, however, useful and will be done through the following simple example. M. J. Baya'rri is ntular Professor, Department of Statistics and Operations Research, University of Valencia, Avenida Dr. Moliner 50, 46100 Burjassot, - - . '_ .1 - ". 1 -7 2r P(rejecting at level a I z1) 1 1/2(.2_0)2 e 1r(0 I ZI) 40 dZ2 727 Ca c. - 2.46 1-41 ( @ I where t is the standard normal cdf and.Ca is the (one-sided) critical value corresponding to. the level, cc, of the test. For instance, if as 0.05, .. tfien this probability equals 0.71 .78, denionstraim" there is a quite substantial probability-that e second - ;a@rlhri4i&nAo be Approved For Release 2003/04/18 CIA-RDP96-00789ROO2700020001-0 4 Approved For Release 2003/04/18 : CIA-RDP96-00789ROO2700020001-0 CPYRGHT REPLICATION IN PARAPSYCHOLOGY A sensible candidate for the prior density ir(fl) is the Cauchy (0, V) density 1 Ir V V (1 + (O/V)21 Flat-tailed densities, such as this, are well known to have the property that when discordant data is observed (e.g., when (I y - X21 is large), substan- tial mass shifts away from the prior center towards the likelihood center. It is easy to see that a normal prior for 0 can not have the desired behavior. Our first surprise in consideration of these priors was how small V needed to be chosen in order for P(Ho I y, x,) to be unaffected. by the bias. For instance, even with V = 1.54/100 (recall that 1.54 was the standard deviation of Y from the original expenment), computation yields P(Ho I y, xj) = 4.3 x 10 -', compared with the P-value