ARgpcXg0dnfRfjReeIeasRu200n0j /Of  CIARDP9600789RO040P~Al
c1slon me a9P bory u0gou0s tl 39 4
9 on h
Applications
of
Decision Augmentation Theory
by
Edwin C. May, Ph.D
Science Applications International Corporation
Menlo Park, CA
Jessica M. Utts, Ph.D.
University of California, Davis
Division of Statistics
Davis, CA
and
S. James Spottiswoode (Consultant), Christine L. James
Science Applications International Corporation
Menlo Park, CA
Abstract
Decision Augmentation Theory (DAT) provides an informational mechanism for a class of anomalous
mental phenomena which have hitherto been viewed as a causal, forcelike, mechanism. Under the
proper conditions, DAT's predictions for microanomalous perturbation databases are different from
those of all forcelike mechanisms, except for one degenerate case. For large random number genera
tor (RNG) databases, DA T predicts a zero slope for a I east squares fit to the (Z2,n) scatter diagram,
where n is the number of bits resulting from a single run and Z is the resulting Zscore. Wefindaslopeof
(1.73±3.19) X 106(t= 0.543, df = 126, p:!~ 0.295) for the historical binary random number generator
database. In a 2sequence length analysis of a limited set of RNG data from the Princeton Engineering
Anomalies Research laboratory, we find that a forcelike explanation misses the observed data by 8.6cy;
however, the observed data is within 1.1o of the DAT prediction. We also apply DAT to one PRNG
study and find that its predicted slope is not significantly different from the expected value. We review
and comment on six published articles that discussed DAT's earlier formalism (i.e., Intuitive Data Sort
ingIDS). Our DA T analysis of Braud's hemolysis study confirms his finding in favor of a causal model
over a selection one (i.e.,p < 0.023); so far, this is the only studywe have found that supports anomalous
perturbation (AP). We provide six circumstantial arguments, which are based upon experimental out
comes against the perturbation hypothesis. Our anomalous cognition (AC) research suggests that the
quality of AC data is proportional to the total change of Shannon entropy. We demonstrate that the
change of Shannon entropy of a binary sequence from chance is independent of sequence length; thus,
we have provided a fundamental argument in favor of DAT over causal models. In our conclusion, we
suggest that, except for one degenerate case, the physical RNG database cannot be explained by any causal
model, and that Braud's contradicting evidence Should inspire more AP investigations of biological systems
in away that would allow a valid DAT analysis.
Approved For Release 2000/08/08 : CIARDP9600789ROO32002100013
IMRA%qO 8/0~h CIARDP9600789 ROO 3350205210001 3
nfaq ion bory ugust 1994
Introduction
May, Utts, and Spottiswoode (1994) proposed Decision Augmentation Theory (DA7~ as a general
model of anomalous mental phenomena (AMP).* DATholds that anomalous cognition (AC) informa
tion is included along with the usual inputs that result in a final human decision that favours a "desired"
outcome. In statistical parlance, DAT says that a slight, systematic bias is introduced into the decision
process by A C.
This concept has the advantage of being quite general. We know of no experiment that is devoid of at
least one human decision; thus, DA T might be the underlying basis for AMP. Mayetal.(1994)mathe
matically developed this concept and constructed a retrospective test algorithrn than can be applied to
existing databases. In this paper, we SLIMMarize the theoretical predictions of DAT, review the criteria
for valid retrospective tests, and analyze the historical random number generator (RNG) database. In
addition, we summarize the findings from one prospective test ofDAT (Radin and May, 1985) and com
ment on the published criticisms of an earlier formulation, which was then called Intuitive Data Sorting.
We conclude with a discussion of RNG results that provide a strong circumstantial argument against a
causal explanation. As part of this review, we show that one biologicalAP experiment is better de
scribed by an influence model (Braud, 1990).
Review of Decision Augmentation Theory
Since the formal discussion of DATis statistical, we will describe the overall context for the development
of the model from that perspective. Consider a random variable, X, that can take on continuous values
(e.g., the normal distribution). Examples of X might be the hit rate in an RNG experiment when the
number of binary bits in the sequence is large, the swimming velocity of cells, or the mutation rate of
bacteria. Let Y be the average computed over n values of X, where n is the number of items that are
collectively subjected to anAMP influence as the result of a single decisionone trial, and let Z be the
appropriate Zscore corresponding to Y. Often this may be equivalent to a single effort period, but it
also may include repeated efforts. The key point is, that, regardless of the effort style, the average value
of the dependent variable is computed over the n values resulting frorn one decision point. In the exam
ples above, n is the sequence length of a single run in an RNG experiment, the number of swimming cells
measured during the trial, or the number of bacteriacontaining test tubes present during the trial.
Under DAT, we assume that the underlying parent distribution of a physical system remains unper
turbed; however, the measurements of the physical system are systematically biased by anACmediated
informational process. Since the deviations seen in actual experiments tend to be small in magnitude, it
is safe to assume that the measurement biases are small and that the sampling distribution will remain
normal; therefore, we assume the bias appears as small shifts of the mean and variance of the sampling
distribution as:
Z 
The Cognitive Sciences Laboratory has adopted the term anomalous mentalphenomena instead of the more widely knownpsi.
Likewise, we use the terms anomalous cognition and anomalous perturbation for ESP and PK, respectively. We have done so
because we believe that these terms are more naturally descriptive of the observables and are neutral in that they do not imply
mechanisms. These new terms will be used throughout this paper.
Approved For Release 2000/08/08 : CIARDP9600789ROO32002100013 2
ARppr,QvQd Foo Rlease 2000/08/08 : CIARDP9600789ROO32002100013
in
gons a, sion Augmentation Theory V5 25 August 1994
This notation means that Z is distributed as a normal distribution with a mean of y, and a standard
deviation of q, Under the null hypothesis,jt, = 0.0 and a, = 1.0.
Review of a Causal Model
For comparison sake, we summarize a model forAP interactions. We begin with the assumption that a
putative AP force would give rise to a perturbational interaction. What we mean is that given an en
semble of entities (e.g., random binary bits), a small force perturbs, on the average, each member of the
ensemble. We call this type of interaction perturbational AP (PAP).
In the simplestPAP model, the perturbation induces a change in the mean of the parent distribution but
does not effect its variance. We parameterize the mean shift in terms of a multiplier of the initial stan
dard deviation. Thus:
III ~ YO + AP (10,
where ~q and ~O are the means of the perturbed and unperturbed distributions, respectively, and where
~ 0.0038),
where X2 is a goodnessoffit measure in general given by:
V
X2 = 7 7
I (Yi  fi)"
j=1 J
where the aj are the errors associated with data pointyj,fj is the falue of tile fitted function at pointj, and
v is the number of data points.
A "good" fit to a set of data should lead to a nonsignificantX2. The fit is not improved by using higher
order polynomials (i.e., X2 = 170.8, df = 124, X2 = 174. 1, df = 123; for quadratics and cubics, respective
ly). If, however, the AP effect size was tiny monotonic function of n other than the degenerate case
where theAP effect size is exactly proportional to I / V_n, it would manifest as a nonzero slope in the
regression analysis.
Within the limits of this retrospective analysis, we conclude for RNG experiments that we must reject all
causal models of AP which propose a shift of the mean of tile parent distribution.
Approved For Release 2000/08/08 : CIARDP9600789ROO320021000137
AP gov RDP9600789ROO439PR1
atpd F,?s Release 2000/08/qp : CIA
ApR ions o ecision ALIgmentation eory U ROM 15; 4
Princeton Engineering Anomalies Research Laboratory RNG Data
The historical database we analyzed does not include the extensive RNG data from the Princeton Engi
neering Anomalies Research (PEAR) laboratory since their total number of bits exceeds the total
amount in the entire historical database. For example, in a recent report Nelson, Dobyns, Dunne, and
Jahn (1991) analyze 5.6 x 106 trials all at n = 200. In this section, we apply DAT retrospectively to their
published work where they have examined other sequence lengths; however, even in these cases, they
report over five times as much data as in the historical database.
Jahn (1982) reported an initial RNG data set with a single operator at n = 200 and 2,000. Data were
collected both in the automatic mode (i.e., a single button press produced 50 trials at n) and the manual
mode (i.e., a single button press produced one trial at n). From a DAT perspective, data were actually
collect at four values of n (i.e., 200, 2000, 200 x 50 = 10,000, and 2000 x50 = 100,000). Unfortunately
data from these two modes were grouped together and reported only at 200 and 2, 000 bit/trial. It would
seem, therefore, we would be unable to apply DA T to these data. Jahn, however, reports that the differ
ent modes "...give little indication of importance of such factors in the overall performance." Thisqual
itative statement suggests that the P11P model is indeed not a good description for these data, because,
under PAP, we would expect stronger effects at the longer sequence lengths.
Nelson, Jahn, and Dunne (1986) describe an extensive RNG and pseudorandom RNG (PRNG) data
base in the manual mode only (i.e., over 7 x 106 trials); however, whereas Jahn provide the mean and
standard deviations for the hits, Nelson et al. report only tile means. We are unable to apply DAT to
these data, because any assumption about the standard deviations would be highly amplified by the
massive data set.
As part of a cooperative agreement in 1987 between PEAR and the Cognitive Sciences Program at SRI
International, we analyzed a set of RNG data from a single operator.* Since they supplied the raw data
for each button press, we were able to analyze this data at two extreme values of n. We combined the
individual trial Zscores for the high and low aims by inverting the sign of the lowaim scores, because
our analysis is twotailed, in that we examine Z2.
Given that the data sets atn = 200 and 100,000 were independently significant (StOLIffer's Z of 3.37and
2.45, respectively), and given the wide separation between the sequence lengths, we used DAT as a ret
rospective test on these two data points.
Because we are examining only two values of n, we do not compute a bestfit slope. Instead, as outlined
in May, Utts, and Spottiswoode (1994), we compare the PAP prediction to the actual data at a single
value of n.
At n = 200,5918 trials yielded Z = 0.044 ± 1.030 and Z2 = 1.063 ± 0.019. We compute aproposedAP
effect size 7 / V200 = 3. 10 x ]03. With this effect size, we computed what would be expected under
thePAP model at n = 100,000. Using the theoretical expressions in Table 1, we computed 72 = 1.961
0.099. The Isigma error is derived from the theoretical variance divided by the actual number of trials
(597) at n = 100,000. The observed values were Z 0. 100 ± 0.997 and Z2 = 1.002 ± 0.050. A ttest
between the observed and expectvalues of Z2gives t 8.643, df = 1192. Considering thist as equivalent
to a Z, the data at n = 100,000 fails to meet what would be expected under the causal PAP model by
* We thank R. Jahn, B. Dunne, and R. Nelson for providing this raw data for our analysis in 1987.
Approved For Release 2000/08/08 : CIARDP9600789ROO32002100013 8
APproved,F RDP9600789RO04?99R19OuO
Jtjo s On Retean 2000/08/qp : CIA s9j
AP n ecision ugmentation ebry U A4
1c,
&6c. Suppose, however, that the effect size observed at n = 100,000 (3.18 X 10 4) better represents
theAP effect size. We computed the predicted value of 72 = 1.00002 ± 0.018 for n = 200. Usingattest
for the difference between the observed value and this predicted one gives t = 2.398, df=11,834. The
PAP model fails in this direction by more than 2.4a. DAT predicts that Z2 would be statistically equiva
lent at the two sequence lengths, and we find that to be the case (t = 1. 14, df = 6513, p < 0. 12 7).
Jahn (1982) indicates in their 1982 PEAR RNG data that "Traced back to the elemental binary samples,
these values imply directed inversion from chance behavior of about one or one and a half bits in every
one thousand...." Assuming 1. 5 excess bi ts/1000, we calcul ate an AP effect size of 0. 003, which is consis
tent with the observed value in their n = 200 data set. Thus, we are forced to conclude that this data set
from PEAR is inconsistentwith the simple PAP model, and that Jahn's statement is not a good descrip
tion of the anomaly.
We urge caution in interpreting these calculations. As is often the case in a retrospective analysis, some
of the required criteria for a meaningful test are violated. These data were not collected when the oper
ators were blind to the sequence length. Secondly, these data represent only a fraction of PEAR's RNG
database.
A Prospective Test of DAT
In developing a methodology for future tests, Radin and May (1986) worked with two operators who
had previously demonstrated strong AP ability in RNG studies. They used a pseudorandom number
generator (PRNG), which was based on a shiftregister algorithm by Kendell and has been shown to
meet the general criteria for "randomness" (Lewis, 1975), to create the binary sequences so that timing
considerations could be examined.
The operators were blind to which of nine different sequences (i.e., n = 101, 201, 401, 701, 1001, 2001,
4001, 7001, 10001 bits)* were used in any given trial, and the program was such that the trials lasted for a
fixed time period and feedback was presented only after the trial was complete. Thus, the criteria for a
valid test of DAT bad been met, except that the Source of the binary bits was a PRNG.
We reanalyzed the combined data from this experiment with the current Zscore formalism of DAT
For the 200 individual runs (i.e. 10 at each of the sequence lengths for each of the two participants) we
found the best fit line to yield a slope = 4.3 X 108 :E 1.6 X 106 (t = 0.028, df = 8, p :!~~ 0.489) and an
intercept = 1.16 ± 0.06 (t = 2.89, df = 8, p :~~ 0.01). The slope easily encompasses zero and is not
significantly different from zero, the significance level is consistent with what Radin and May reported
earlier.
Since the PRNG seeds and bit streams were saved for each trial, it was possible to determine if the ex
periment sequences exactly matched the ones produced by the shift register algorithm; they did. Since
our UNIXbased Sun Microsystems workstations were synchronized to the system clock, any momen
tary interruption of the clock would "crash" the machine, but 110 Such crashes Occurred. Therefore, we
believe no casual interaction occurred.
To explore the timing aspects of the experiment Radin and May reran each run with PRNG seeds rang
ing from 5 to +5 clock ticks (i.e., 20 ms/tick) from the actual seed used in the run. We plot the resulting
* 'Me original IDS analysis required the sequence lengths to odd because of the logarithmic formalism.
Approved For Release 2000/08/08 : CIARDP9600789ROO32002100013 9
An SFC~6Fe*g~kqc?AWPN9§e, ~IARDP9600789RO0439PRI9.%9I
04
run effect sizes, which we computed from the experimental Fratios (Rosenthal, 1991), for operator 531
in Figure 5. The estimated Istandard errors are the same for each seed shift and equal 0.057.
0.20
0.1 5
ED
0. 10
0.05
_E1
o.oo
6
Relative Seed Position
Figure 5. Seed Timing for Operator 531 (298 Runs).
Radin and May erroneously concluded that the significant differences between zero and adjacent seed
positions was meaningful, and that the DAT ability was effective within 20 milliseconds. In fact, the
situation shown in Figure 5 is expected. Differing from true random number generators in which slight
changes in timing produce essentially the same sequence, PRNGs produced totally different sequences
as a function of single digit seed changes. Thus, itwould lie surprising if the seedshift display produced
anything but a spike at seed shift zero. We will return to this point in Our analysis of some of the pub
lished remarks on our theory.
From this prospective test of DAT, we conclude that for PRNGs it is possible to select a proper entry
point into a bit stream to produce significant deviations from MCE that are independent of sequence
length.
The Literature: Review and Comment
We have identified six published articles that have commented upon the Intuitive Data Sorting (IDS)
theory, the earlier name for DAT In this section, we chronologically summarize and comment on each
report.
Walker  September 1987
In his first of two criticisms of IDS, Walker (1987) suggested that his Monte Carlo simulations did not fit
the predictions of the IDS model. He generated a single deviant set of 100 bits (i.e., Z = 2.33, p :!:~ 0.01),
and he inserted this same sequence as the first 100 bits of 400 otherwise randomly generated sequences
ranging from 100 to 106 bits in length. Walker's analysis of these sequences did not yield a least square's
slope of 0.5 as predicted tinder the IDS formalism. Walker concluded that the IDS theory was incor
rect. Walker's sequences, however, are not the type that are generated inAP experiments or tile type for
which the DATIIDS model is valid.
Approved For Release 2000/08/08 : CIARDP9600789ROO320021 0001 19
APgov d F?6 ReleaRe 2000/08/98 : CIARDP9600789ROO 00
Icat,e
AP Ons o ecision ugmentation heory 439PRIgMU34
May et a]. (1985) were explicit about the character of thesequences that fit the IDS model. Specifically,
Walker quotes May et a]. "Using psiacquired infbrmation, individuals are able to select locally deviant
subsequencesfrom a large random sequence. " (Italics are used in the original May paper.) The very next
sentence on page 249 of the reference says, "Such an ability, if mediated by precognition, would allow
individuals (subjects or experimenters) to initiate a collection unit of continuous samples (this has been
reported as a trial, a block, a run, etc.) in such a way as to optitnize thefinal result. (Italics added here for
emphasis.) Walker continued, "Indeed, the only way the subject can produce results that agree with the
data is to wait for an extrachance run that matches the experimental run length." In the final analysis,
Walker actually supported our contention that individuals select deviant subsequences. Both from our
text and the formalism in our 1.985 paper, it is clear that what we meant by a "large random sequence,"
was large compared to the trial length, n.
In his second criticism of IDS in the same paper, Walker proposed that individuals would have to exhibit
a physiologically impossible control over timing (e.g., when to press a button). As evidence apparently
in favor of such an exquisite timing ability, lie referred to the data presented by Radin and May (1986)
that we have discussed above. (Please see Figure 5.) Walker suggested that Radin and May's result,
therefore, supported his quantum mechanical observer theory. It is beyond the scope of this paper to
critique Walker's quantum mechanical models, but we would hope they do not depend upon his analysis
of Radin and May's results. The enhanced hitting at zero seed and the suppressed values ± one 20 ms
clock tick that we show in Figure 5 is tile expected result based upon the wellunderstood properties of
PRNG's and does not represent the precision of the operator's reaction time.
We must consider how it is possible with normal human reactions to obtain significant scores, which can
only happen in 20 ms windows. In typical visual reaction time measurements, Woodworth and Schlos
berg (1960) found a standard deviation of 30 ms. If we assume these human reactions are typical of
those forAC performance and are normally distributed, we compute the maximum probability of being
within a 20 ms window, which is centered about the mean, of 23.5%. For the worst case, the operators
must "hit" significant seeds less often than 23.561o of tile time. Radin and May do not report tile number
of significant runs, so we provide a worstcase estimate. Given that they quote apvalue of 0. 005 for 500
trials, we find that 39 trials inust be independently sign ifi cant. That is, the accumulated binomial proba
bility is 0.005 for 39 hits in 500 trials with an event probability of 0.05. This corresponds to a hitting rate
(i.e., 391500) of only 7.8%, a value well within the capability of human reaction times. We recognize that
it is not a requirement to hit only on significant seeds; however, all other seeds leading to positive Z
scores are less restrictive then the case we have presented.
The zerocenter "spike" in Figure 5 misled Walker and others into thinking that exceptional timing was
required to produce the observed deviations. As we have shown this is not the case, and, therefore,
Walker's second criticism of the IDS theory is not valid.
Bierman  1988
Bierman (1988) attempted to test the IDS model with a gifted Subject. His experimental design ap
peared to meet most of the criteria for a valid test of tile model; however, Bierman found no evidence
forAMP and stated that no conclusions Could be drawn from his work. We encourage Bierman to con
tinue with this design and to be specific with what lie would expect to see if DATwere the correct mecha
nism compared to if it were not.
Approved For Release 2000/08/08 : CIARDP9600789ROO320021 0001 11
Approved For Release 2000/08/08 : CIARDP9600789ROO32002100013
Applications of Decision Augmentation Theory V5 25 August 1994
Braud and Schlitz  1989
Braud and Schlitz (1989) conducted an electrodermal PK experiment specifically to test the IDS model.
They argued that if the mechanism of the effect were informational, then allowing participants more
opportunities to select locally deviant values of the dependent variable should yield stronger effects. In
their experiment, 12 electrodermal sampling epochs were either initiated individually by a press of a
button, or all 12 were determined as a result of tile first button press. Braud and Schlitz hypothesized
that under IDS, they would expect to see a larger overall effect in the former condition. Theyfoundthat
the single button press data yielded a significant result; whereas tile multiple press data scored at chance
Q,i.g1J31]=2.14,t,.11J31J= 0.53). They concluded, therefore, that their data were more consistent
with an influence mechanism than with an informational one.
In both buttonpress conditions, the dependent variable was averaged over all 12 epochs; therefore, the
formalism discussed in this paper cannot beapplied because the data should have been averaged over at
least two different values. The idea of multiple decision points, however, is still valid. As stated in their
paper, the timing of the epochs wasSUCh that 20 seconds of the 30 second epoch was independent of the
buttonpress condition and COUld not, therefore be subjected to a DATlike selection. To examine the
consequence of this overlap, we computed the effect size for tile Single button ease as 0.359 (Rosenthal,
1991, Page 19, Equation 2.16). Since data for the 20 seconds is the same in each condition, the operator
can only make ACmediated decisions for the first 10 seconds of the data. If we assume that on the
average the remaining 20 seconds meets mean chance expectation and the effect size is constant, then
wewouldexpectaneffectsizeof(O.359+0+0)/3=0.119.* The measured effect size was 0.095, which
is consistent with this prediction.
Braud and Schlitz's idea was good and provides a possible way to use DAT effectively. Because of the
epoch timing and the consistency of the effect sizes, however, we believe they have interpreted their
results in favor of causal mechanism prematurely. Aside from the timing issues, their protocol compli
cates the application of DAT further. To observe an enhanced effect because of multiple decision
points, Zscores should be computed for each decision and combined as a Stouffer's Z where the de
nominator is the square root of the number of decision points. In their protocol, they only combine the
dependent variable.
Vassy 1990
Vassy (1990) used a similar timing argument to refute the IDS model as did Walker (1987). Vassy gener
ated PRNG single bits at a rate of one each 8.7 ms. He argued that if IDS were operating, that a subject
would be more likely to identify bursts of ones rather than single ones given the time between consecu
tive bits. While lie found significant evidence for the primary task of "selecting" individual bits, he
found no evidence that these hits were imbedded in excess clusters of ones.
We compute that the maximum probability of a hit within an & 71nswindow centered on the mean of the
normal reaction curve with a standard deviation of 30 ms (Woodworth and Schlosberg, 1960) is 11.5%.
Vassy quotes an overall Zscore for 100 runs of 2.39. From this, we compute a mean Z of 0.239 for each
run of 36 bits. To obtain this result requires an excess hitting of 0.717 bits, which corresponds to an ex
Asa function of n, DA Tpred icts a 11V_n dependency in the effect size; however, ata fixed n, as in this case, the effect size
should
be constant.
Approved For Release 2000/08/08 : CIARDP9600789ROO3200210001312
r~IARDP9600789 ROO~~99QOO
ld u S9 1 634
cess hitting rate of 2%. Given that 11.5% is the maximum one can expect with normal human reaction
times, Vassy's results easily allow for individual bit selection, and, thus, cannot be used to reject the DAT
model on the basis of timing.
Braud  1990
In a cooperative effort with SRI International, Braud (1990) conducted a biological AP study with hu
man red blood cells as the target system. The study was designed, in part, as a prospective test of DAT, so
all conditions for a valid test were satisfied. Braud found that a significant number of individuals were
independently able to "slow" the rate of hernolysis (i.e., the destruction of red blood cells in saline solu
tion) in what he called the "protect" mode. Using data from the nine significant participants, Braud
found support in favor of PAPoverDAT Figure 6 shows the resultsof ourreanalysisof all of Braud's
raw data using our more modern formalism of DAT



10 


T  T
0 Fffort Data
El Control Data
X Predicted A P
I
Predicted DAT
>E
21
0 2
4
6
a
10
Number
of
Test
'Ribes
Figure6. DAT Analysis of Hernolysis Data.
The solid line indicates the theoretical expected value for MCE. The squares are the mean values of Z2
for the control data, and the error bars indicate the I standard error for the 32 trials in the study. We
notice that the control data with eight test tubes is significantly below MCE (t = 2.79, df = 62, p :!!~
0.996). Compared to the MCE line, the effort data is significant (t = 4.04, df = 31, p:!E~ 7.6 x 105) for
eight test tubes and nearly so for n = 2 (t = 2.06, df = 31, p !!~~ 0. 051). The x at n = 8 indicates the
calculated value of the mean of Z2 assuming that the effect size at n = 2 was entirely because of AP;
similarly, the X atn = 2 indicates the calculated value assuming that the effect size, which was observed
at n = 8, was totally due to AR TheseAP predictions are not significantly different from the observed
data (t = 0. 156, p :!~~ 0. 431, df = 62 and t = 0. 906, p ::]~ 0. 184, df = 6Z at n = 2 and 8, respectively).
Whereas DAT predicts no differences between the data at the end points for n, we find a significant
difference (t = 2.033, p :2~ 0. 023, df = 62). That is, to a statistical degree the data at n = 8, cannot be
explained by selection alone. Thus, we concur with BraLid's original conclusion; these results indicate a
possible causal relationship between mental intent and biological consequences.
It is difficult to conclude from our analysis of a single Study with only 32 trials thatAP is part of nature;
nonetheless, this result is very important. It is the first data we have encountered that supports the PAP
Approved For Release 2000/08/08 : CIARDP9600789ROO320021 0001 Y
Approved OVK.Release 2000108/ph8 : CIARDP9600789R0QWRu1ggPjW 4
e
A ications cision AUgmenta ion eory
hypothesis, which is in direct opposition to the substantial Support against PAP resulting from the analy
sis of the RNG data sets. In addition, May and Vilenskaya (1993) and Vilenskaya and May (1994) re
port that the preponderance ofAMP research in the Former Soviet Union is the study ofAP on biologi
cal systems. Their operators, as do ours, report their internal experiences as being a causal relationship
between them and their biological targets.
Dobyns  1993
Dobyns (1993) presents a method for comparing what lie calls the "influence" and "selection" models,
corresponding to what we have been calling DAT and PAP He uses data from 490 "tripolar sets" of
experimental runs at PEAR. For each set, there was a high aim, a baseline and a low aim condition.
The three values produced were then sorted into which one was actually highest, in the middle, and
lowest for each set. The datawere then summarized into a 3 x 3 matrix,where the rows represented the
three intentions, and the columns represented the actual ordering. If every attempt had been success
ful, the diagonal of the matrix would consist of the number of tripolar sets, namely 490. We present the
data portion of Dobyns'Table frorn page 264 of the reference as our Table 2:
Tab I e 2.
Scoring Data From Dobyns (1993)
Intention
t
l
A
c High Middle Low Total
ua
High 180 167 143 490
Baseline159 156 175 490
Low 151. 167 172 490
Total 490 490
A i
kl
Dobyns computes an aggregate likelihood ratio of his predictions for the DATand PAP models and con
cludes in favor the the influence model with a ratio of 28.9 to one.
However, there are serious problems with the methods used in Dobyns' paper. In this paper we simply
outline only two of the difficulties. To fully explain them would require a level of technical discussion
not suitable for a short summary such as this.
One problem is in the calculation of the likelihood ratio function using Equation 6, which we reproduce
from page 265 of the reference:
"1 "2 `3 ni
P1 P2 P1 r1 2 62 [P 3 "3
12 nj 7_
B(plq) = q'I q2 qI ~ ] 12] 71
wherep and q are the predicted rank frequencies for eachaim Linder the influence and selection models,
respectively, and the n are the observed frequencies for each aim. We agree that this relationship cor
rectly gives the likelihood ratio for comparing the two models for one row of Table 2. However, immedi
ately following that equation, Dobyns writes, "The aggregate I ikelihood of the hypothesis over all three
Approved For Release 2000/08/08 : CIARDP9600789ROO320021 0001 _14
Approved For Release 2000/08/08 : CIARDP9600789ROOMP3100 1A
Applications of Decision Augmentation Theory ugus?
intentions may be calculated by repeating the individual likelihood calculation for each intention, and
the total likelihood will simply be the product of factors such as (6) above for each of the three inten
tions."
That statement is simply incorrect. A combi ned likeli hoo d is fou nd by M 111 ti plying the individual likeli
hoods only if the random variables are independent of each other (DeGroot, 1986, p. 145). Clearly, the
rows of the table are not independent. In fact, if You know any two of the rows, the third is determined
exactly! The correct likelihood ratio needs to build that dependence into the formula.*
A second technical problem with the conclusion that the data support the influence model is that the
method itself strongly supports the influence model. As noted by Dobyns, "In fact, applying the test to
data sets that, by construction, contain no effect, yields strong odds (ranging, in a modest Monte Carlo
database, from 8.5 to over 100) in favor of the influence model (page 268)." The actual data in hispaper
yielded odds of 28.9 to one in favor of the influence model; however, this value is well within the re
ported limits from his " infl ue nce less" Monte Carlo data.
Under DAT it is possible thatACmediated selection might occur at the protocol level, but the primary
way is through timinginitiating a run to capitalize upon a locally deviant Subsequence. Howthismight
work in dynamic RNG devices is clear; wait until Such a deviant sequence is in your immediate future
and initiate the run in time to capture it. With "static" devices, Such as PEAR's random mechanical
cascade (RMC) device, how timing enters in is less obvious. Under closer inspection, however, even
with the RMC device there is a statistical variation among unattended control runs. That is, there is
never a series of control runs that give exactly the same mean. Physical effects, such as Browian motion,
temperature gradients, etc., can account for the observed variance in the absence of human operators.
Thus, when a run is initiated to capture favorable local " envi ron mental" factors, even for "static" de
vices, remains the operative issue with regard to DAT Dobyns does not consider this case at all in his
analysis. If DAT enters in at the protocol selection, as it probably does, it is likely to be a secondorder
contribution because of the limited possibilities from which to select (i.e., six in the tripolar case).
Finally, a major problem with Dobyns' conclusion, which was pointed out when he first presented this
paper at a conference (May, 1990), is that the likelihood ratio supports tile influence model even for
their PRNG data. Dobyns dismisses this finding (page 268) all too easily given the preponderance of
evidence that suggest that no influence occurs during PRNG studies (Radin and May, M6).
Aside from the technical flaws in Dobyns' likelihood ratio arguments, and even ignoring the problem
with the PRNG analysis, we reject his conclusions simplybecause they hold in favor of influence even in
Monte Carloconstructed and unperturbed data.
Circumstantial Evidence Against an AP Model for RNG Data
Experiments with hardware RNG devices are not new. In fact, the title of Schmidt's very first paper on
the topic (1969) portended our conclusion, "Precognition of a Quantum Process." Schmidt listsPKas a
third option after two possible sources for precognition, and remarks, "The experiments done so far do
not permit a distinction (if such a distinction is at all meaningful) between the three possibilities." From
* Dobyns agrees on this pointprivate commurtication.
Approved For Release 2000/08/08 : CIARDP9600789ROO3200210001315
A roved For Release 2000/08/08 : C'ARDP9600789ROM~Ro~u'gouostoll~4
Aperications of Decision Augmentation Theory
1969 onward, the RNG research has been strongly oriented toward a PK model. The term microPK,
itself, embeds the force concept further into the lexicon of RNG descriptions.
In this section, we examine a number of RNG experimental results that provide circumstantial evidence
against the AP hypothesis. Any single piece of evidence could be easily dismissed; however, taken to
gether, they demonstrate a substantial case againstAP.
Internal Complexity of RNG Devices and Source Independence
Schmidt (1974) conducted the first experiment to explore potential dependencies upon the internal
workings of his generators. Since by definitionAP implies a force or influence, it seemed reasonable to
expect that an influence should depend upon the details of the target system. In this study, one genera
tor produced individual binary bits, which were derived from the ~decay of 90Sr, while the other
"binary" output was a majority vote from 100 bits, each of which were derived from a fast electronic
diode. Schmidt reports individually significant effects with both generators, yet does not observe a sig
nificant difference between the generators.
This particular study is interesting, quite aside from the timing and majority vote issues; the binary
streams were derived from fUndarnentally different physical sources. Radioactive Pdecay is governed
by the weak nuclear force, and electronic devices (e.g., noise diodes) are governed by the electromag
netic force. Schematically speaking, the electromagnetic force is approximately 1,000 times as strong as
the weak nuclear force, and modern highenergy physics has shown them to be fundamentally different
after about 1010 seconds after the big bang (Raby, 1985). Thus, a Putative APforce would have to
interact equally with these two forces; and since there is no mechanism known that will cause the elec
tromagnetic and weak forces to interact with each other, it is unlikely thatAP will turn out to be the first
coupling mechanism. The lack of difference between Pdecay and noise diode generators was con
firmed years later by May et a]. (1980).
We have already commented upon one aspect of the timing issue with regard to Radin and May's (1986)
experiment and the papers by Walker (1987) and Vassy (1990). May (1975) introduced a scheme to
remove any firstorder biases in binary generators that also is relevant to the timing issue. The output of
his generator was a match or antimatch between the random bit stream and a target bit. One mode of
the operation of the device, which May describes, included an oscillating target bitone oscillation per
bit at approximately 1 MHz rate.* May and Honorton (1975) and Honorton and May (1975) reported
significant effects with the RNG operating in this mode. Thus, significant effects can be seen even with
devices that operate in the microsecond time domain, which is three orders of magnitude faster than
any known physiological process.
Effects with Pseudorandom Number Generators
Pseudorandom number generators are, by definition, those that depend upon an algorithm, which is
usually implemented on a comp Liter. Radin (1985) analyzed all the PRNGs commonly in useand found
that they require a startingvalLie (i.e., a seed), which is often derived from the computer's system clock.
As we noted above, Radin and May (1986) showed that the bit stream, which proved to be "successful"
in a PRNG study, was bitforbit identical with the strearn, which was generated later, but with the same
* Later, this technique was adopted by Jahn (1982) for use in the RNG devices at PEAR.
Approved For Release 2000/08/08 : CIARDP9600789ROO320021 0001 _316
,roved FP6 ReJease 2000108/9h8 : QIARDP9600789R0qWAjRqPjW4
Arp
A ications of ecision AUgmenta ion eory
seed. With that generator, at least, there wits no change from the expected bit stream. Perhaps it is
possible that the seed generator (i.e,, system clock) was subjected to some A P interaction. We propose
two arguments against this hypothesis:
(1) Even one cycle interruption of a computers' system clock will usually invoke a system crash; an
event not often reported in PRNG experiments.
(2) Computers use crystal oscillators as the basis for their internal clocks. Crystal manufacturers usual
ly quote errors in the stated oscillation frequency of the order of 0.001 percent. That translates to
500cycles for a 50MHz crystal, or to 10 [ts in tirne. Assuming that the quoted error is a 1a estimate,
and that a putative AP interaction acts at within the ± 2o domain, then shifting the clock by this
amount might account for only one seed shift in Radin and May's experiment. By Monte Carlo
methods, we determined that, given a randorn entry into seedspace, the average number of ticks to
reach a "significant" seed is 10; therefore, even if AP could shift the oscillators by 2o, it cannot
account for the observed data.
Since computers in PRNG experiments are not reported as "crashing" often, it is safe to assume that
PRNG results are only due toAC. In addition, since the results of PRNG studies are statistically insepa
rable from those reported with true RNGs, it is also reasonable to assume that the mechanisms are simi
larlyACbased.
Precognitive AC
Using the tools of modern metaanalysis, Honorton reviewed the precognition cardguessing database
(Honorton and Ferarri, 1989). This analysis included 309 separate Studies reported by 62 investigators.
Nearly two million individual trials were contributed by more the 50,000 Subjects. The combined effect
size was _E = 0.020±0.002, which corresponds to an overall combined effect of 11.4o. TWO important
results emerge from Honorton's analysis. First, it is often stated by critics that the best results are from
studies with the least methodological controls. To check this hypothesis, Honorton devised an eight
point quality measure (e.g., automated recording of data, proper randomization techniques) and
scored each study with regard to these measures. There was no significant correlation between study
quality and study score. Second, if researchers improved their experiments over time, one would expect
a significant correlation of study quality with date Of Publication. Honortonfoundr=0.246,df=307,p
< 2 x 10~ In brief, Honorton concludes that a statistical anornaly exists in this data that cannot be
explained by poor study quality or a large variety of other hypotheses; therefore, a potential mechanism
underlying DAT has been verified.
SRI International's RNG Experiment
May, Humphrey, and Hubbard (1980) conducted an extensive RNG study at SRI International in 1979.
They applied stateoftheart engineering and methodology to construct two true RNGs, one based on
the Pdecay of 137pm and the other based on an MD20 noise diode from Texas Instrument,. It is be
yond the scope of this paper to describe, in detail, the intricacies of this experiment; however, we will
discuss those aspects, which are pertinent to this discussion.
Technical Details
Each of the two sources were battery operated and optically Coupled to a Digital Equipment Corpora
tion LSI 11/23 computer. Failsafe circuitry Would disable the sources if critical physical parameters
Approved For Release 2000/08/08 : CIARDP9600789ROO3200210001317
ged Fp6ReleaRe 200P
A npro 108/98: Cl1ARDP9600789ROO;?9P3J#R9Jg;4
"AW c ons o ecislon ugmen ation Heory
a
(e.g., battery voltages and currents, temperature) exceed preset ranges. Both sources were subjected to
environmental testing which included extreme temperature cycles, vibration tests, E&M and nuclear
gamma and neutron radiation tests. Both sources behaved as expected, and the critical parameters,
such as temperature, were monitored and their data stored along with the experimental data.
Asource was sampled at 1 KHz rate. After eight milliseconds, the resulting byte was sent to the comput
erwhile the next byte was being obtained. In this way, a continuous stream of 1 ms data was presented to
the computer, May et al. had specified, in advance, that bit number 4 was the designated target bit.
Thus each byte provided 3 ms of bits prior to the target and 4 ms of bits after the target bit.
A trial was defined as a definitive outcorne from a sequential analysis of bit four from each byte. In
exchange for not specifying the number of samples in advance, sequential analysis requires that the
'Iype I and`l~pe II errors, and the chance and extrachance hitting rate be specified in advance. In May
et al.'s twotailed analysis, (x = P = 0.05 and the chance and extrachance hitting rate was 0.50 and 0.52,
respectively. The expected number of samples to reach a definitive decision was approximately 3,000.
The outcome from a single trial could be in favor of a hitting rate of 0.52,0.48, or at chance of 0.50, with
the usual risk of error in accordance with the specified Type I and Type 11 errors.
Each of seven operators participated in 100 trials of this type. For an operator's data to reach indepen
dently statistical significance, the operator had to produce 16 successes in 100 trials, where a successwas
defined as extrachance hitting (i.e., the exact binomial probability of 16 Successes for 100 trials with an
event probability of 0.10 is 0.04 where one less success is not significant). Two operators produced 16
and 17 successful trials, respectively.
Temporal Analysis
We analyzed the 33 trials from the two independently significant operators from May et al.'s experi
ment. Each of the 33 trials consisted of approximately 3,000 bits of data with 3 bits and +4 bits of I
msibit temporal history surrounding the target bit. We argue that if the significance observed in the
target bits was because of AP, we would expect a large correlation with the target bit's immediate neigh
bors, which are only ± I ins away. As far as we know, there is no known physiological process that can be
cognitively, or in any other way, manipulated within a millisecond. We might even expect a 100% cor
relation under the complete AP model.
We computed the linear correlation coefficients between bits 3 and 4, 4 and 5, and 3 and 5. For binary
data:
NO'  XI(df = 1),
where q5 is the linear correlation coefficient and N is the number of samples. Since we examined three
different correlations for 33 trials, we computed 99 different values of N02. Four of them produced X2s
that were sign ifican twel I within chance expectation. The complete distribution is shown in Figure 7.
We see that there is excellent agreement of the 99 correlations with tile X2 distribution for one degree of
freedom, which is shown as a smooth Curve.
We conclude, therefore, that there was no evidence beyond chance to suggest that the target bit neigh
bors were affected even when the target bit analysis produced significant evidence for an anomaly. So,
Approved For Release 2000/08/08 : CIARDP9600789ROO32002100013 18
Wroved
9% ReleaAs 0  FlARDP9600789ROQA22qWvJQPA19A
cations ecision u$Sen9A(9P'A§or
knowing the physiological limitations of the human systems, we further concluded that tile observed
effects could not have arisen due to a humanmediated force (i.e., AP).
Mathematical Model of the Noise Diode
Because of the unique construction parameters of Texas Instrument's MD20 noise diode, May et a].
were able to construct a quantum mechanical model of the detailed workings of the device. This model
contained all known properties of the material and it's construction parameters. For example, the band
gap energy in Si, the effective mass of an electron or hole in the semiconductor, and the impurity con
centration were among the parameters for the rnodel. The model was successful at calculating the
diode's known and measured behavior as a function of temperature. May et al. were able to simulate
their RNG experiment down to the quantum mechanical details of the noise source. They hoped that
by adjusting the model's parameters so that the computed Output agreed with the experimental one,
that they could gain insight as to where the causal influence "entered" tile device.
May et al. were not able to find a set of model parameters that mimicked their RNG data. For example,
changing the band gap energy for short periods of tirne; increasing or reducing the electron's effect
mass; or redistributing or changing tile impurity content produced no unexpected changes in the device
output. The only device behavior that could be effected was its known function of temperature.
Because of the construction details of the physical RNG, this result could have been anticipated. The
changes that could be Simulated in the model were all in the microsecond domain because of the details
of the device. Both with the RNG and in its model, the diode's multiMHz output was filtered by a
100KHz wide bandwidth filter. Thus, any microsecond changes would not pass through the filter. In
short, because of this filtering, the RNG was particularly insensitive to potential changes of the physical
parameters of the diode.
Yet solid statistical evidence for an anomaly was seen by May et al. Since the diode device was shown
mathematically and empirically to be insensitive to environmental and physical changes, these results
must have been as a result ofA C rather than any caUsalAP. In fact, this observation coupled with the bit
bi 9
Approved For Release 2000/08/08 : CIARDP9600789ROO32002100013
Figure 7. Observed and Theoretical Correlation Distributions.
Wipgr6gsFo? 6 p6gip RgAqq(90q(q§ eb ~11ARDP9600789ROOMRIURA JA 4
timing argument, which we have described above, lead May et al. to question causality in RNG studies
in general.
Summary of Circumstantial Evidence Against AP
We have identified six circumstantial arguments that, when taken together, provide increasingly diffi
cult requirements that Must be met by a putative AP force. In Summary, the RNG database demon
strates that:
(1) Data are independent of internal complexity of the hardware RNG device.
(2) Data are independent of the physical mechanism producing the randomness (i.e., weak nuclear or
electromagnetic).
(3) Effects with pseudorandom generators are statistically equivalent to those observed with true
hardware generators.
(4) Reasonable AP models of mechanism do not fit the data.
(5) In one study, bits which are ± 1 ms from a "perturbed" target bit are themselves unperturbed.
(6) A detailed model of a diode noise source, which includes all known physics of the device, could not
simulate the observed data streams.
In addition, AC, which is a mechanism to describe the data, has been confirmed in nonRNG experi
ments. We conclude, therefore, an AP force that is consistent with the database must
~ Be equally coupled to the electromagnetic and weak nuclear forces.
~ Be mentally mediated in times shorter than one millisecond.
~ Follow a I IV n law.
For these to be true, an AP force would be at odds with an extensive amount of verified physics and
common behavioral observables. We are not saying, therefore, that AP cannot exist; rather, we are sug
gesting that instead of having to force ourselves to invent a whole new science, we should look for ways
in which AP might fit into the present world view. In addition, as DAT tries to accomplish, we should
invent noncausal and testable alternate mechanisms for the experimental observables.
Conclusions
We have shown that DATcan determine whether a causal or informational explanation is more consis
tent with a given set of anomalous statistical data. In applying DAT to the substantial physical RNG
database,we showed that an informational mechanism is strongly favored over a large class of perturba
tional ones. Given that one possible information mechanism (i.e., precognitiveAC) can, and has been,
independently confirmed in the laboratory, and given the weight of the empirical, yet circumstantial,
arguments taken together against AP, we conclude that the anornalOUs results from the RNG studies
arise not because of a mentally mediated force, but rather because of a human ability to be a mental
opportunist by makingACinediated decisions to capitalize on the locally deviant circumstances.
Our recent results in the study of anomalous Cognition (May, Spottiswoode, and James, 1994) suggest
the the quality of AC is proportional to the change in Shannon entropy. Following Vassy (1990), we
compute the change in Shannon entropy for an extrachance, binary sequence of length n. The total
change of entropy is given by:
Approved For Release 2000/08/08 : CIARDP9600789ROO320021000130
)AAMNSA ffPSAWARg A PANMO'rP I ARDP9600789R00AWRuVgPjjA
'JS = SO  S,
where for an unbiased binary sequence of length n, S, = n, and S is given by:
S =  np,log2p,  n(1 P1)'()92(1 _P1)'
Letpj = 0.5 (1 + s) and assume that v, the effect size, is small compared to one (i.e., typical RNG effect
sizes are of the order of 3 x 104). Using the approximation:
In (1 + e) = E  F
21
we find that S is given by:
2
S
n 21n2'
or that the total change of entropy for a biased binary sequence is given by;
AS = So  S = n F2
21n2
Since our analysis of the historical RNG database shows that the effect size is proportional to 1 V_n
(i.e., Zscore is independent of n), the total change of Shannon entropy becomes a constant that is inde
pendent of the sequence length. Thus, if entropy is related to what is being sensed by anomalous cogni
tion, then this argument suggests that informational processes are responsible for the RNG anomaly.
The one exception to this conclusion is Braud's study of 11P on red blood cells. It may be that there is
somethingunique about living systems that can account for this observation. On the other hand, it is the
only biological AP study we Could find that could be analyzed by DATand the perturbation hypothesis is
only weakly favored over the selection one (i.e., p < 0.023). Before we would be willing to declare that
AP is a valid mechanism, more than a single, albeit well designed and.executed, study is needed.
Generally,we suggest that future RNG, PRNG, and biologica]AP studies be designed in such away that
the criteria, as outlined in this paper and in May, Utts, Spottiswoode (1994), are adhered to for a valid
DAT analysis. Our discipline has evolved to the point where we can no longer be satisfied with yet one
more piece of evidence of a statistical anomaly. We must identify the sources of variance as suggested
by May, Spottiswoode, and James ('1994); limit thern as Much as possible; and apply models, such as
DAT, which can begin to shed light on the physical, physiological, and psychological mechanisms of
anomalous mental phenomena.
Approved For Release 2000/08/08 : CIARDP9600789ROO320021 0001
hff6.qy&q PlARDP9600789R00A2AQAjgAqjcA
References
Bierman, D. J. (1988). Testing the IDS model witli a gifted Subject., Theoretical Parapsychology, 6, 3136.
Braud, W G. and Schlitz, M. J. (1989). Possible role Of IntUituve Data Sorting in electrodermal
biological psychokinesis (BioPK). The Journal of'the American Societyfbr Psychical Research, 83,
No. 4, 289302.
Braud, W G. (1990). Distant mental influence of rate of hernolysis of human red blood cells. TheJoumal
of the American Society fbr Psychical Research, 84,No.1,124.
DeGroot, Morris H. (1985). Probability and Statistics, 2nd Edition. Reading, MA: AddisonWesley
Publishing Co.
Dobyns, Y. H. (1993). Selection versus influence in rernote REG anornalies. Joumal of Scientific
Exploration. 7, No. 3, 259270.
Honorton, C. and May, E. C. (1975). Volitional control in a psychokinetic task with auditory and visual
feedback. Research in Parapsychology, 1975, 9091.
Honorton, C. and Ferrari, D. C. (1989) "Future Telling:" A metaanalysis of forcedchoice precognition
experiments, 19351.987. Journal of Parapsychology, 53, 281308.
Jahn, R. G. (1982). The persistent paradox of psychic phenomena: an engineering perspecitve.
Proceedings of the IEEE. 70, No. 2, 136170.
Lewis, T. G. (1975). Distribution Sampling for Computer Simulation. Lexington, MA: Lexington
Books.
May, E. C. (1975). PSIFI: A physiologycoupled, noisedriven random generator to extend PK studies.
Research in Parapsychology, 1975, 2022.
May, E. C. and Honorton, C. (1975). A dynamic PK experiment with Ingo Swann. Research in
Parapsychology, 1975, 8889.
May, E. C., Humphrey, B. S., Hubbard, G. S. (1980). Electronic System Perturbation Techniques. Final
Report. SRI International Menlo Park, CA.
May, E. C., Radin, D. I., Hubbard, G. S., and Humphrey, B. S. (1985). Psi experiments with random
number generators: an informational model. Proceedings of Presented Papers Vol L The
Parapsychological Association 28th Annual Convention, Tufts University, Medford, MA, 237266.
May, E. C. (1990). As chair for the session at the annual meeting of the Society for Scientific
Exploration in which this original work was presented, I pointed Out the problern of the likelihood
ratio for the PRNG data from the floor of the convention.
May, E. C., Spottiswoode, S. James R, and James, C. L. (1994). Shannon entropy as an Intrinsic Target
property: Toward a reductionist model of anomalous cognition. Submitted to The Journal of
Parapsychology.
May, E. C., Utts, J. M., Spottiswoode, S. J. (1994). Decision augmentation theory: Toward a model of
anomalous mental phenomena. Submitted to The Journal of Parapsychology.
May, E. C. and Vilenskaya, L. (1994). Overview of current parapsychology research in the former Soviet
union. Subtle E nergies. 3, No 3. 4567.
Nelson, R. D., Jahn, R. G., and Dunne, B. J. (1986). Operatorrelated anomalies in physical systems and
information processes. Jorunal of'the Societyfor Psychical Research, 53, No. 803, 261285.
Nelson, R. D., Dobyns, Y. H., Dunne, B. J.,and Jahn, R. G., and (1991). Analysis of Variance of REG
Experiments: Operator Intention, Secondary Parameters, Database Structures. PEAR
Laboratory Technical Report 91004, School of Engineering, Princeton University.
Approved For Release 2000/08/08 : CIARDP9600789ROO320021 000, _%2
W[WcRig ffWeFNRt~E~Mr39MWigg%ga'rPIARDP9600789ROQ4NUlfigAig
Raby, S. (1985). Supersyminetry and cosmology. in Supersymmetry, Supergravity, and Related Topics.
Proceedings of the XVth GIFT International Seminar oil Theoretical Physics, Sant Feliu. de
Guixols, Girona, Spain. World Scientific Publishing Co. Pte. Ltd. Singapore, 226270.
Radin, D. 1. (1985). Pseudorandom Number Generators in Psi Research. Journal of Parapsycholog. 49,
No 4, 303328.
Radin, D. 1. and May, E. C. (1986). Testing tile Intuitive Data Sorting mode with pseudorandom
number generators: A proposed method. The Proceedings of Presented Papers of the 29th Annual
Convention of the Parapsychological Association, Sonoma State University, Rohnert Park, CA,
539554.
Radin, D. I. and Nelson, R. D. (1989). Evidence for consciousnessrelated anomalies in random
physical systems. Roundations of Physics. 19, No. 12, 14991514.
Rosenthal, R. (1991). Metaanalysis procedures for social research. Applied Social Research Methods
Series, Vol. 6, Sage Publications, Newbury Park, CA.
Schmidt. H. (1969). Precognition of a quantum process. Journal of Parapsycholog. 33, No. 2, 99108.
Schmidt. H. (1974). Comparison of PK action on two different random number generators. Journal of
Parapsycholog. 38, No. 1, 4755.
Vassy, Z. (1990). Experimental study of precognitive timing: Indications of a radically noncausal
operation. Journal of'ParapsycholoA~,. 54, 299320.
Vilenskaya, L. and May, E. C. (1994). Anomalous mental phenomena research in Russia and the
Former Soviet Union: A follow Lip. Submitted to the 1994 Annual Meeting of the
Parapsychological Association.
Walker, E. H. (1987). A comparison of the intuitive data sorting and quantum mechanical observer
theories. Journal of Parapsychology, 51, 217227.
Woodworth, R.S. and Schlosberg H. (1960). Experimental Psychology. Rev ed. New York Holt. New
York.
Approved For Release 2000/08/08: CIARDP9600789ROO3200210001 J3