SAIC - (415) 322-7960 - Created: Wednesday, June 14,1995 22:38 - Page 1 of 2
------- ----------- AW-aved-F~of-Fk4eas,-2W3f6*l87-CiA--RDP967-'OU 7 grMON690-70i §d7(Wj__j - - - - - - - - - - - - - - - - -
The Cognitive Sciences Laboratory
A, -0 330 Cowper Street, Suite 200, Palo Alto, CA 94301
Voice: 415.3 27.2007 - Fax 415.322.7960
An Employee-Owned Company e-mail: may@,hildegard.saicmp.com
SG1 I Date: 14 June 1995
To: I
From: Edwin C.
Re: Project Evaluation
SG1 I ___] faxed to me new tasking with regard to providing support for the project
evaluation. I am asked to find "the 10 primary studies or reports developed under the
subject program ... thatmake the best case for establishing the validity of the paranormal
phenomena known as Remote Viewing." I am happy to do this as soon as possible.
I have a concern, however, about a problem that may be raised that could call any results
into question.
Dr. Robert Rosenthal in the Psychology Department at Harvard and others have defined a
statistical circumstance known as the "file drawer" problem. It is standard practice in
behavioral sciences (including the study of anomalous cognition-remote viewing) to use
a p=0.05 criteria as a measure of statistical significance. That is, statistical significance is
claimed when, given that the null hypothesis (i.e. no RV in this case) is TRUE, there is a
5% chance that a repeated experiment of the same number of trials would yield a
statistical deviation as large as in the original, or larger. We use that criterion in our
laboratory.
The file draw problem is this: Under the null hypothesis 5% of the studies will be
statistically significant If only the significant studies are published (or in our case given
to AIR), it is problematical to assess the validity of the research, because it might be that
for every study published there are 19 others (in the file drawer) that did not reach
statistical significance and were not published. If this were the case, there would be no
evidence for an anomaly. Rosenthal and others provide ways of assessing the potential
magnitude of this problem.
It might be argued that if I am allowed to pick the best 10 studies, and if it turns out that
there were 200 others that were not "good," one could be seriously mislead. This is a
worst case scenario; however, critics would correctly argue that there is an undetermined
file drawer problem with this new approach, and if it is not addressed, the validity of the
evaluation can be questioned. I can assure you that Ray Hyman will raise this point.
Might I suggest an alternative that address the concerns of the time constraint, yet avoids
the file drawer problem. In 1989, SRI published (now downgraded to unclassified) a
meta-analysis of all the SRI work from 1973 through 1989. This study was part of the
Approved For Release 2003109/16 : CIA-RDP96-00791 R000200190041-2 Page 1
Approved For Rey4se 2003/09/1.6: Cl RM96-00791R=Q20019Q041-2
Fax i ransm- Co 'Sneet
SG1 I To: - ORD
From: Edwin C. May, Ph.D., SAIC
Fax Phone Number: (415) 322-7960
Date: Wed, Jun 14,1995 - 22:38
Transmitting (3) pages, Including cover sheet.
If there Is difficulty with this transmission, please call: (415) 322-7960
Note:
Approved For Release 2003/09/16 : CIA-RDP96-00791 R000200190041-2