ARgpcXg0dnfRfjReeIeasRu200n0j /Of - CIA-RDP96-00789RO040P~Al c1slon me a9P bory u0gou0s tl 39 4 9 on h Applications of Decision Augmentation Theory by Edwin C. May, Ph.D Science Applications International Corporation Menlo Park, CA Jessica M. Utts, Ph.D. University of California, Davis Division of Statistics Davis, CA and S. James Spottiswoode (Consultant), Christine L. James Science Applications International Corporation Menlo Park, CA Abstract Decision Augmentation Theory (DAT) provides an informational mechanism for a class of anomalous mental phenomena which have hitherto been viewed as a causal, force-like, mechanism. Under the proper conditions, DAT's predictions for micro-anomalous perturbation databases are different from those of all force-like mechanisms, except for one degenerate case. For large random number genera- tor (RNG) databases, DA T predicts a zero slope for a I east squares fit to the (Z2,n) scatter diagram, where n is the number of bits resulting from a single run and Z is the resulting Z-score. Wefindaslopeof (1.73±3.19) X 10-6(t= 0.543, df = 126, p:!~ 0.295) for the historical binary random number generator database. In a 2-sequence length analysis of a limited set of RNG data from the Princeton Engineering Anomalies Research laboratory, we find that a force-like explanation misses the observed data by 8.6-cy; however, the observed data is within 1.1-o of the DAT prediction. We also apply DAT to one PRNG study and find that its predicted slope is not significantly different from the expected value. We review and comment on six published articles that discussed DAT's earlier formalism (i.e., Intuitive Data Sort- ing-IDS). Our DA T analysis of Braud's hemolysis study confirms his finding in favor of a causal model over a selection one (i.e.,p < 0.023); so far, this is the only studywe have found that supports anomalous perturbation (AP). We provide six circumstantial arguments, which are based upon experimental out- comes against the perturbation hypothesis. Our anomalous cognition (AC) research suggests that the quality of AC data is proportional to the total change of Shannon entropy. We demonstrate that the change of Shannon entropy of a binary sequence from chance is independent of sequence length; thus, we have provided a fundamental argument in favor of DAT over causal models. In our conclusion, we suggest that, except for one degenerate case, the physical RNG database cannot be explained by any causal model, and that Braud's contradicting evidence Should inspire more AP investigations of biological systems in away that would allow a valid DAT analysis. Approved For Release 2000/08/08 : CIA-RDP96-00789ROO3200210001-3 IMRA%qO 8/0~h- CIA-RDP96-00789 ROO 3350205210001 -3 nfaq ion bory ugust 1994 Introduction May, Utts, and Spottiswoode (1994) proposed Decision Augmentation Theory (DA7~ as a general model of anomalous mental phenomena (AMP).* DATholds that anomalous cognition (AC) informa- tion is included along with the usual inputs that result in a final human decision that favours a "desired" outcome. In statistical parlance, DAT says that a slight, systematic bias is introduced into the decision process by A C. This concept has the advantage of being quite general. We know of no experiment that is devoid of at least one human decision; thus, DA T might be the underlying basis for AMP. Mayetal.(1994)mathe- matically developed this concept and constructed a retrospective test algorithrn than can be applied to existing databases. In this paper, we SLIMMarize the theoretical predictions of DAT, review the criteria for valid retrospective tests, and analyze the historical random number generator (RNG) database. In addition, we summarize the findings from one prospective test ofDAT (Radin and May, 1985) and com- ment on the published criticisms of an earlier formulation, which was then called Intuitive Data Sorting. We conclude with a discussion of RNG results that provide a strong circumstantial argument against a causal explanation. As part of this review, we show that one biological-AP experiment is better de- scribed by an influence model (Braud, 1990). Review of Decision Augmentation Theory Since the formal discussion of DATis statistical, we will describe the overall context for the development of the model from that perspective. Consider a random variable, X, that can take on continuous values (e.g., the normal distribution). Examples of X might be the hit rate in an RNG experiment when the number of binary bits in the sequence is large, the swimming velocity of cells, or the mutation rate of bacteria. Let Y be the average computed over n values of X, where n is the number of items that are collectively subjected to anAMP influence as the result of a single decision-one trial, and let Z be the appropriate Z-score corresponding to Y. Often this may be equivalent to a single effort period, but it also may include repeated efforts. The key point is, that, regardless of the effort style, the average value of the dependent variable is computed over the n values resulting frorn one decision point. In the exam- ples above, n is the sequence length of a single run in an RNG experiment, the number of swimming cells measured during the trial, or the number of bacteria-containing test tubes present during the trial. Under DAT, we assume that the underlying parent distribution of a physical system remains unper- turbed; however, the measurements of the physical system are systematically biased by anAC-mediated informational process. Since the deviations seen in actual experiments tend to be small in magnitude, it is safe to assume that the measurement biases are small and that the sampling distribution will remain normal; therefore, we assume the bias appears as small shifts of the mean and variance of the sampling distribution as: Z - The Cognitive Sciences Laboratory has adopted the term anomalous mentalphenomena instead of the more widely knownpsi. Likewise, we use the terms anomalous cognition and anomalous perturbation for ESP and PK, respectively. We have done so because we believe that these terms are more naturally descriptive of the observables and are neutral in that they do not imply mechanisms. These new terms will be used throughout this paper. Approved For Release 2000/08/08 : CIA-RDP96-00789ROO3200210001-3 2 ARppr,QvQd Foo Rlease 2000/08/08 : CIA-RDP96-00789ROO3200210001-3 in gons a, sion Augmentation Theory V5 25 August 1994 This notation means that Z is distributed as a normal distribution with a mean of y, and a standard deviation of q, Under the null hypothesis,jt, = 0.0 and a, = 1.0. Review of a Causal Model For comparison sake, we summarize a model forAP interactions. We begin with the assumption that a putative AP force would give rise to a perturbational interaction. What we mean is that given an en- semble of entities (e.g., random binary bits), a small force perturbs, on the average, each member of the ensemble. We call this type of interaction perturbational AP (PAP). In the simplestPAP model, the perturbation induces a change in the mean of the parent distribution but does not effect its variance. We parameterize the mean shift in terms of a multiplier of the initial stan- dard deviation. Thus: III ~ YO + -AP (10, where ~q and ~O are the means of the perturbed and unperturbed distributions, respectively, and where ~ 0.0038), where X2 is a goodness-of-fit measure in general given by: V X2 = 7 7 I (Yi - fi)" j=1 J where the aj are the errors associated with data pointyj,fj is the falue of tile fitted function at pointj, and v is the number of data points. A "good" fit to a set of data should lead to a non-significantX2. The fit is not improved by using higher order polynomials (i.e., X2 = 170.8, df = 124,- X2 = 174. 1, df = 123; for quadratics and cubics, respective- ly). If, however, the AP effect size was tiny monotonic function of n other than the degenerate case where theAP effect size is exactly proportional to I / V_n, it would manifest as a non-zero slope in the regression analysis. Within the limits of this retrospective analysis, we conclude for RNG experiments that we must reject all causal models of AP which propose a shift of the mean of tile parent distribution. Approved For Release 2000/08/08 : CIA-RDP96-00789ROO3200210001-37 AP gov -RDP96-00789ROO439PR1 atpd F,?s Release 2000/08/qp : CIA ApR ions o ecision ALIgmentation eory U ROM 15; 4 Princeton Engineering Anomalies Research Laboratory RNG Data The historical database we analyzed does not include the extensive RNG data from the Princeton Engi- neering Anomalies Research (PEAR) laboratory since their total number of bits exceeds the total amount in the entire historical database. For example, in a recent report Nelson, Dobyns, Dunne, and Jahn (1991) analyze 5.6 x 106 trials all at n = 200. In this section, we apply DAT retrospectively to their published work where they have examined other sequence lengths; however, even in these cases, they report over five times as much data as in the historical database. Jahn (1982) reported an initial RNG data set with a single operator at n = 200 and 2,000. Data were collected both in the automatic mode (i.e., a single button press produced 50 trials at n) and the manual mode (i.e., a single button press produced one trial at n). From a DAT perspective, data were actually collect at four values of n (i.e., 200, 2000, 200 x 50 = 10,000, and 2000 x50 = 100,000). Unfortunately data from these two modes were grouped together and reported only at 200 and 2, 000 bit/trial. It would seem, therefore, we would be unable to apply DA T to these data. Jahn, however, reports that the differ- ent modes "...give little indication of importance of such factors in the overall performance." Thisqual- itative statement suggests that the P11P model is indeed not a good description for these data, because, under PAP, we would expect stronger effects at the longer sequence lengths. Nelson, Jahn, and Dunne (1986) describe an extensive RNG and pseudorandom RNG (PRNG) data- base in the manual mode only (i.e., over 7 x 106 trials); however, whereas Jahn provide the mean and standard deviations for the hits, Nelson et al. report only tile means. We are unable to apply DAT to these data, because any assumption about the standard deviations would be highly amplified by the massive data set. As part of a cooperative agreement in 1987 between PEAR and the Cognitive Sciences Program at SRI International, we analyzed a set of RNG data from a single operator.* Since they supplied the raw data for each button press, we were able to analyze this data at two extreme values of n. We combined the individual trial Z-scores for the high and low aims by inverting the sign of the low-aim scores, because our analysis is two-tailed, in that we examine Z2. Given that the data sets atn = 200 and 100,000 were independently significant (StOLIffer's Z of 3.37and 2.45, respectively), and given the wide separation between the sequence lengths, we used DAT as a ret- rospective test on these two data points. Because we are examining only two values of n, we do not compute a best-fit slope. Instead, as outlined in May, Utts, and Spottiswoode (1994), we compare the PAP prediction to the actual data at a single value of n. At n = 200,5918 trials yielded Z = 0.044 ± 1.030 and Z-2 = 1.063 ± 0.019. We compute aproposedAP effect size 7 / V20-0 = 3. 10 x ]0-3. With this effect size, we computed what would be expected under thePAP model at n = 100,000. Using the theoretical expressions in Table 1, we computed 72 = 1.961 0.099. The I-sigma error is derived from the theoretical variance divided by the actual number of trials (597) at n = 100,000. The observed values were Z 0. 100 ± 0.997 and Z2 = 1.002 ± 0.050. A t-test between the observed and expectvalues of Z2gives t 8.643, df = 1192. Considering thist as equivalent to a Z, the data at n = 100,000 fails to meet what would be expected under the causal PAP model by * We thank R. Jahn, B. Dunne, and R. Nelson for providing this raw data for our analysis in 1987. Approved For Release 2000/08/08 : CIA-RDP96-00789ROO3200210001-3 8 APproved,F -RDP96-00789RO04?99R19OuO Jtjo s On Retean 2000/08/qp : CIA s9j AP n ecision ugmentation ebry U A4 1c, &6-c. Suppose, however, that the effect size observed at n = 100,000 (3.18 X 10 -4) better represents theAP effect size. We computed the predicted value of 72 = 1.00002 ± 0.018 for n = 200. Usingat-test for the difference between the observed value and this predicted one gives t = 2.398, df=11,834. The PAP model fails in this direction by more than 2.4-a. DAT predicts that Z2 would be statistically equiva- lent at the two sequence lengths, and we find that to be the case (t = 1. 14, df = 6513, p < 0. 12 7). Jahn (1982) indicates in their 1982 PEAR RNG data that "Traced back to the elemental binary samples, these values imply directed inversion from chance behavior of about one or one and a half bits in every one thousand...." Assuming 1. 5 excess bi ts/1000, we calcul ate an AP effect size of 0. 003, which is consis- tent with the observed value in their n = 200 data set. Thus, we are forced to conclude that this data set from PEAR is inconsistentwith the simple PAP model, and that Jahn's statement is not a good descrip- tion of the anomaly. We urge caution in interpreting these calculations. As is often the case in a retrospective analysis, some of the required criteria for a meaningful test are violated. These data were not collected when the oper- ators were blind to the sequence length. Secondly, these data represent only a fraction of PEAR's RNG database. A Prospective Test of DAT In developing a methodology for future tests, Radin and May (1986) worked with two operators who had previously demonstrated strong AP ability in RNG studies. They used a pseudorandom number generator (PRNG), which was based on a shift-register algorithm by Kendell and has been shown to meet the general criteria for "randomness" (Lewis, 1975), to create the binary sequences so that timing considerations could be examined. The operators were blind to which of nine different sequences (i.e., n = 101, 201, 401, 701, 1001, 2001, 4001, 7001, 10001 bits)* were used in any given trial, and the program was such that the trials lasted for a fixed time period and feedback was presented only after the trial was complete. Thus, the criteria for a valid test of DAT bad been met, except that the Source of the binary bits was a PRNG. We re-analyzed the combined data from this experiment with the current Z-score formalism of DAT For the 200 individual runs (i.e. 10 at each of the sequence lengths for each of the two participants) we found the best fit line to yield a slope = 4.3 X 10-8 :E 1.6 X 10-6 (t = 0.028, df = 8, p :!~~ 0.489) and an intercept = 1.16 ± 0.06 (t = 2.89, df = 8, p :~~ 0.01). The slope easily encompasses zero and is not significantly different from zero, the significance level is consistent with what Radin and May reported earlier. Since the PRNG seeds and bit streams were saved for each trial, it was possible to determine if the ex- periment sequences exactly matched the ones produced by the shift register algorithm; they did. Since our UNIX-based Sun Microsystems workstations were synchronized to the system clock, any momen- tary interruption of the clock would "crash" the machine, but 110 Such crashes Occurred. Therefore, we believe no casual interaction occurred. To explore the timing aspects of the experiment Radin and May reran each run with PRNG seeds rang- ing from -5 to +5 clock ticks (i.e., 20 ms/tick) from the actual seed used in the run. We plot the resulting * 'Me original IDS analysis required the sequence lengths to odd because of the logarithmic formalism. Approved For Release 2000/08/08 : CIA-RDP96-00789ROO3200210001-3 9 An SFC~6Fe*g~kqc?AWPN9§e, ~IA-RDP96-00789RO0439PRI9.%9I 04 run effect sizes, which we computed from the experimental F-ratios (Rosenthal, 1991), for operator 531 in Figure 5. The estimated I-standard errors are the same for each seed shift and equal 0.057. 0.20 0.1 5 --ED- 0. 10 0.05 _E1 o.oo -6 Relative Seed Position Figure 5. Seed Timing for Operator 531 (298 Runs). Radin and May erroneously concluded that the significant differences between zero and adjacent seed positions was meaningful, and that the DAT ability was effective within 20 milliseconds. In fact, the situation shown in Figure 5 is expected. Differing from true random number generators in which slight changes in timing produce essentially the same sequence, PRNGs produced totally different sequences as a function of single digit seed changes. Thus, itwould lie surprising if the seed-shift display produced anything but a spike at seed shift zero. We will return to this point in Our analysis of some of the pub- lished remarks on our theory. From this prospective test of DAT, we conclude that for PRNGs it is possible to select a proper entry point into a bit stream to produce significant deviations from MCE that are independent of sequence length. The Literature: Review and Comment We have identified six published articles that have commented upon the Intuitive Data Sorting (IDS) theory, the earlier name for DAT In this section, we chronologically summarize and comment on each report. Walker - September 1987 In his first of two criticisms of IDS, Walker (1987) suggested that his Monte Carlo simulations did not fit the predictions of the IDS model. He generated a single deviant set of 100 bits (i.e., Z = 2.33, p :!:~ 0.01), and he inserted this same sequence as the first 100 bits of 400 otherwise randomly generated sequences ranging from 100 to 106 bits in length. Walker's analysis of these sequences did not yield a least square's slope of -0.5 as predicted tinder the IDS formalism. Walker concluded that the IDS theory was incor- rect. Walker's sequences, however, are not the type that are generated inAP experiments or tile type for which the DATIIDS model is valid. Approved For Release 2000/08/08 : CIA-RDP96-00789ROO320021 0001 -19 AP-gov d F?6 ReleaRe 2000/08/98 : CIA-RDP96-00789ROO 00 Icat,e AP Ons o ecision ugmentation heory 439PRIgMU34 May et a]. (1985) were explicit about the character of thesequences that fit the IDS model. Specifically, Walker quotes May et a]. "Using psi-acquired infbrmation, individuals are able to select locally deviant subsequencesfrom a large random sequence. " (Italics are used in the original May paper.) The very next sentence on page 249 of the reference says, "Such an ability, if mediated by precognition, would allow individuals (subjects or experimenters) to initiate a collection unit of continuous samples (this has been reported as a trial, a block, a run, etc.) in such a way as to optitnize thefinal result. (Italics added here for emphasis.) Walker continued, "Indeed, the only way the subject can produce results that agree with the data is to wait for an extra-chance run that matches the experimental run length." In the final analysis, Walker actually supported our contention that individuals select deviant subsequences. Both from our text and the formalism in our 1.985 paper, it is clear that what we meant by a "large random sequence," was large compared to the trial length, n. In his second criticism of IDS in the same paper, Walker proposed that individuals would have to exhibit a physiologically impossible control over timing (e.g., when to press a button). As evidence apparently in favor of such an exquisite timing ability, lie referred to the data presented by Radin and May (1986) that we have discussed above. (Please see Figure 5.) Walker suggested that Radin and May's result, therefore, supported his quantum mechanical observer theory. It is beyond the scope of this paper to critique Walker's quantum mechanical models, but we would hope they do not depend upon his analysis of Radin and May's results. The enhanced hitting at zero seed and the suppressed values ± one 20 ms clock tick that we show in Figure 5 is tile expected result based upon the well-understood properties of PRNG's and does not represent the precision of the operator's reaction time. We must consider how it is possible with normal human reactions to obtain significant scores, which can only happen in 20 ms windows. In typical visual reaction time measurements, Woodworth and Schlos- berg (1960) found a standard deviation of 30 ms. If we assume these human reactions are typical of those forAC performance and are normally distributed, we compute the maximum probability of being within a 20 ms window, which is centered about the mean, of 23.5%. For the worst case, the operators must "hit" significant seeds less often than 23.561o of tile time. Radin and May do not report tile number of significant runs, so we provide a worst-case estimate. Given that they quote ap-value of 0. 005 for 500 trials, we find that 39 trials inust be independently sign ifi cant. That is, the accumulated binomial proba- bility is 0.005 for 39 hits in 500 trials with an event probability of 0.05. This corresponds to a hitting rate (i.e., 391500) of only 7.8%, a value well within the capability of human reaction times. We recognize that it is not a requirement to hit only on significant seeds; however, all other seeds leading to positive Z- scores are less restrictive then the case we have presented. The zero-center "spike" in Figure 5 misled Walker and others into thinking that exceptional timing was required to produce the observed deviations. As we have shown this is not the case, and, therefore, Walker's second criticism of the IDS theory is not valid. Bierman - 1988 Bierman (1988) attempted to test the IDS model with a gifted Subject. His experimental design ap- peared to meet most of the criteria for a valid test of tile model; however, Bierman found no evidence forAMP and stated that no conclusions Could be drawn from his work. We encourage Bierman to con- tinue with this design and to be specific with what lie would expect to see if DATwere the correct mecha- nism compared to if it were not. Approved For Release 2000/08/08 : CIA-RDP96-00789ROO320021 0001 -11 Approved For Release 2000/08/08 : CIA-RDP96-00789ROO3200210001-3 Applications of Decision Augmentation Theory V5 25 August 1994 Braud and Schlitz - 1989 Braud and Schlitz (1989) conducted an electrodermal PK experiment specifically to test the IDS model. They argued that if the mechanism of the effect were informational, then allowing participants more opportunities to select locally deviant values of the dependent variable should yield stronger effects. In their experiment, 12 electrodermal sampling epochs were either initiated individually by a press of a button, or all 12 were determined as a result of tile first button press. Braud and Schlitz hypothesized that under IDS, they would expect to see a larger overall effect in the former condition. Theyfoundthat the single button press data yielded a significant result; whereas tile multiple press data scored at chance Q,i.g1J31]=2.14,t,.11J31J= -0.53). They concluded, therefore, that their data were more consistent with an influence mechanism than with an informational one. In both button-press conditions, the dependent variable was averaged over all 12 epochs; therefore, the formalism discussed in this paper cannot beapplied because the data should have been averaged over at least two different values. The idea of multiple decision points, however, is still valid. As stated in their paper, the timing of the epochs wasSUCh that 20 seconds of the 30 second epoch was independent of the button-press condition and COUld not, therefore be subjected to a DAT-like selection. To examine the consequence of this overlap, we computed the effect size for tile Single button ease as 0.359 (Rosenthal, 1991, Page 19, Equation 2.16). Since data for the 20 seconds is the same in each condition, the operator can only make AC-mediated decisions for the first 10 seconds of the data. If we assume that on the average the remaining 20 seconds meets mean chance expectation and the effect size is constant, then wewouldexpectaneffectsizeof(O.359+0+0)/3=0.119.* The measured effect size was 0.095, which is consistent with this prediction. Braud and Schlitz's idea was good and provides a possible way to use DAT effectively. Because of the epoch timing and the consistency of the effect sizes, however, we believe they have interpreted their results in favor of causal mechanism prematurely. Aside from the timing issues, their protocol compli- cates the application of DAT further. To observe an enhanced effect because of multiple decision points, Z-scores should be computed for each decision and combined as a Stouffer's Z where the de- nominator is the square root of the number of decision points. In their protocol, they only combine the dependent variable. Vassy- 1990 Vassy (1990) used a similar timing argument to refute the IDS model as did Walker (1987). Vassy gener- ated PRNG single bits at a rate of one each 8.7 ms. He -argued that if IDS were operating, that a subject would be more likely to identify bursts of ones rather than single ones given the time between consecu- tive bits. While lie found significant evidence for the primary task of "selecting" individual bits, he found no evidence that these hits were imbedded in excess clusters of ones. We compute that the maximum probability of a hit within an & 71nswindow centered on the mean of the normal reaction curve with a standard deviation of 30 ms (Woodworth and Schlosberg, 1960) is 11.5%. Vassy quotes an overall Z-score for 100 runs of 2.39. From this, we compute a mean Z of 0.239 for each run of 36 bits. To obtain this result requires an excess hitting of 0.717 bits, which corresponds to an ex- Asa function of n, DA Tpred icts a 11V_n dependency in the effect size; however, ata fixed n, as in this case, the effect size should be constant. Approved For Release 2000/08/08 : CIA-RDP96-00789ROO3200210001-312 r~IA-RDP96-00789 ROO~~99QOO ld u S9 1 634 cess hitting rate of 2%. Given that 11.5% is the maximum one can expect with normal human reaction times, Vassy's results easily allow for individual bit selection, and, thus, cannot be used to reject the DAT model on the basis of timing. Braud - 1990 In a cooperative effort with SRI International, Braud (1990) conducted a biological AP study with hu- man red blood cells as the target system. The study was designed, in part, as a prospective test of DAT, so all conditions for a valid test were satisfied. Braud found that a significant number of individuals were independently able to "slow" the rate of hernolysis (i.e., the destruction of red blood cells in saline solu- tion) in what he called the "protect" mode. Using data from the nine significant participants, Braud found support in favor of PAPoverDAT Figure 6 shows the resultsof ourre-analysisof all of Braud's raw data using our more modern formalism of DAT ---- - - 10 - - - -T- -------- -T-- 0 Fffort Data El Control Data X Predicted A P I Predicted DAT >E -21 0 2 4 6 a 10 Number of Test 'Ribes Figure6. DAT Analysis of Hernolysis Data. The solid line indicates the theoretical expected value for MCE. The squares are the mean values of Z2 for the control data, and the error bars indicate the I -standard error for the 32 trials in the study. We notice that the control data with eight test tubes is significantly below MCE (t = -2.79, df = 62, p :!!~ 0.996). Compared to the MCE line, the effort data is significant (t = 4.04, df = 31, p:!E~ 7.6 x 10-5) for eight test tubes and nearly so for n = 2 (t = 2.06, df = 31, p !!~~ 0. 051). The x at n = 8 indicates the calculated value of the mean of Z2 assuming that the effect size at n = 2 was entirely because of AP; similarly, the X atn = 2 indicates the calculated value assuming that the effect size, which was observed at n = 8, was totally due to AR TheseAP predictions are not significantly different from the observed data (t = 0. 156, p :!~~ 0. 431, df = 62 and t = 0. 906, p ::]~ 0. 184, df = 6Z at n = 2 and 8, respectively). Whereas DAT predicts no differences between the data at the end points for n, we find a significant difference (t = 2.033, p :2~ 0. 023, df = 62). That is, to a statistical degree the data at n = 8, cannot be explained by selection alone. Thus, we concur with BraLid's original conclusion; these results indicate a possible causal relationship between mental intent and biological consequences. It is difficult to conclude from our analysis of a single Study with only 32 trials thatAP is part of nature; nonetheless, this result is very important. It is the first data we have encountered that supports the PAP Approved For Release 2000/08/08 : CIA-RDP96-00789ROO320021 0001 -Y Approved OVK.Release 2000108/ph8 : CIA-RDP96-00789R0QWRu1ggPjW- 4 e A ications cision AUgmenta ion eory hypothesis, which is in direct opposition to the substantial Support against PAP resulting from the analy- sis of the RNG data sets. In addition, May and Vilenskaya (1993) and Vilenskaya and May (1994) re- port that the preponderance ofAMP research in the Former Soviet Union is the study ofAP on biologi- cal systems. Their operators, as do ours, report their internal experiences as being a causal relationship between them and their biological targets. Dobyns - 1993 Dobyns (1993) presents a method for comparing what lie calls the "influence" and "selection" models, corresponding to what we have been calling DAT and PAP He uses data from 490 "tripolar sets" of experimental runs at PEAR. For each set, there was a high aim, a baseline and a low aim condition. The three values produced were then sorted into which one was actually highest, in the middle, and lowest for each set. The datawere then summarized into a 3 x 3 matrix,where the rows represented the three intentions, and the columns represented the actual ordering. If every attempt had been success- ful, the diagonal of the matrix would consist of the number of tripolar sets, namely 490. We present the data portion of Dobyns'Table frorn page 264 of the reference as our Table 2: Tab I e 2. Scoring Data From Dobyns (1993) Intention t l A c High Middle Low Total ua High 180 167 143 490 Baseline159 156 175 490 Low 151. 167 172 490 Total 490 490 A i kl Dobyns computes an aggregate likelihood ratio of his predictions for the DATand PAP models and con- cludes in favor the the influence model with a ratio of 28.9 to one. However, there are serious problems with the methods used in Dobyns' paper. In this paper we simply outline only two of the difficulties. To fully explain them would require a level of technical discussion not suitable for a short summary such as this. One problem is in the calculation of the likelihood ratio function using Equation 6, which we reproduce from page 265 of the reference: "1 "2 `3 ni P1 P2 P1 r1 2 62 [P 3 "3 12 nj 7_ B(plq) = q'I q2 qI ~ ] 12] 71 wherep and q are the predicted rank frequencies for eachaim Linder the influence and selection models, respectively, and the n are the observed frequencies for each aim. We agree that this relationship cor- rectly gives the likelihood ratio for comparing the two models for one row of Table 2. However, immedi- ately following that equation, Dobyns writes, "The aggregate I ikelihood of the hypothesis over all three Approved For Release 2000/08/08 : CIA-RDP96-00789ROO320021 0001 _14 Approved For Release 2000/08/08 : CIA-RDP96-00789ROOMP3100 1A Applications of Decision Augmentation Theory ugus? intentions may be calculated by repeating the individual likelihood calculation for each intention, and the total likelihood will simply be the product of factors such as (6) above for each of the three inten- tions." That statement is simply incorrect. A combi ned likeli hoo d is fou nd by M 111 ti plying the individual likeli- hoods only if the random variables are independent of each other (DeGroot, 1986, p. 145). Clearly, the rows of the table are not independent. In fact, if You know any two of the rows, the third is determined exactly! The correct likelihood ratio needs to build that dependence into the formula.* A second technical problem with the conclusion that the data support the influence model is that the method itself strongly supports the influence model. As noted by Dobyns, "In fact, applying the test to data sets that, by construction, contain no effect, yields strong odds (ranging, in a modest Monte Carlo database, from 8.5 to over 100) in favor of the influence model (page 268)." The actual data in hispaper yielded odds of 28.9 to one in favor of the influence model; however, this value is well within the re- ported limits from his " infl ue nce- less" Monte Carlo data. Under DAT it is possible thatAC-mediated selection might occur at the protocol level, but the primary way is through timing-initiating a run to capitalize upon a locally deviant Subsequence. Howthismight work in dynamic RNG devices is clear; wait until Such a deviant sequence is in your immediate future and initiate the run in time to capture it. With "static" devices, Such as PEAR's random mechanical cascade (RMC) device, how timing enters in is less obvious. Under closer inspection, however, even with the RMC device there is a statistical variation among unattended control runs. That is, there is never a series of control runs that give exactly the same mean. Physical effects, such as Browian motion, temperature gradients, etc., can account for the observed variance in the absence of human operators. Thus, when a run is initiated to capture favorable local " envi ron mental" factors, even for "static" de- vices, remains the operative issue with regard to DAT Dobyns does not consider this case at all in his analysis. If DAT enters in at the protocol selection, as it probably does, it is likely to be a second-order contribution because of the limited possibilities from which to select (i.e., six in the tripolar case). Finally, a major problem with Dobyns' conclusion, which was pointed out when he first presented this paper at a conference (May, 1990), is that the likelihood ratio supports tile influence model even for their PRNG data. Dobyns dismisses this finding (page 268) all too easily given the preponderance of evidence that suggest that no influence occurs during PRNG studies (Radin and May, M6). Aside from the technical flaws in Dobyns' likelihood ratio arguments, and even ignoring the problem with the PRNG analysis, we reject his conclusions simplybecause they hold in favor of influence even in Monte Carlo-constructed and unperturbed data. Circumstantial Evidence Against an AP Model for RNG Data Experiments with hardware RNG devices are not new. In fact, the title of Schmidt's very first paper on the topic (1969) portended our conclusion, "Precognition of a Quantum Process." Schmidt listsPKas a third option after two possible sources for precognition, and remarks, "The experiments done so far do not permit a distinction (if such a distinction is at all meaningful) between the three possibilities." From * Dobyns agrees on this point-private commurtication. Approved For Release 2000/08/08 : CIA-RDP96-00789ROO3200210001-315 A roved For Release 2000/08/08 : C'A-RDP96-00789ROM~Ro~u'gouostoll-~4 Aperications of Decision Augmentation Theory 1969 onward, the RNG research has been strongly oriented toward a PK model. The term micro-PK, itself, embeds the force concept further into the lexicon of RNG descriptions. In this section, we examine a number of RNG experimental results that provide circumstantial evidence against the AP hypothesis. Any single piece of evidence could be easily dismissed; however, taken to- gether, they demonstrate a substantial case againstAP. Internal Complexity of RNG Devices and Source Independence Schmidt (1974) conducted the first experiment to explore potential dependencies upon the internal workings of his generators. Since by definitionAP implies a force or influence, it seemed reasonable to expect that an influence should depend upon the details of the target system. In this study, one genera- tor produced individual binary bits, which were derived from the ~-decay of 90Sr, while the other "binary" output was a majority vote from 100 bits, each of which were derived from a fast electronic diode. Schmidt reports individually significant effects with both generators, yet does not observe a sig- nificant difference between the generators. This particular study is interesting, quite aside from the timing and majority vote issues; the binary streams were derived from fUndarnentally different physical sources. Radioactive P-decay is governed by the weak nuclear force, and electronic devices (e.g., noise diodes) are governed by the electromag- netic force. Schematically speaking, the electromagnetic force is approximately 1,000 times as strong as the weak nuclear force, and modern high-energy physics has shown them to be fundamentally different after about 10-10 seconds after the big bang (Raby, 1985). Thus, a Putative AP-force would have to interact equally with these two forces; and since there is no mechanism known that will cause the elec- tromagnetic and weak forces to interact with each other, it is unlikely thatAP will turn out to be the first coupling mechanism. The lack of difference between P-decay and noise diode generators was con- firmed years later by May et a]. (1980). We have already commented upon one aspect of the timing issue with regard to Radin and May's (1986) experiment and the papers by Walker (1987) and Vassy (1990). May (1975) introduced a scheme to remove any first-order biases in binary generators that also is relevant to the timing issue. The output of his generator was a match or anti-match between the random bit stream and a target bit. One mode of the operation of the device, which May describes, included an oscillating target bit-one oscillation per bit at approximately 1 MHz rate.* May and Honorton (1975) and Honorton and May (1975) reported significant effects with the RNG operating in this mode. Thus, significant effects can be seen even with devices that operate in the microsecond time domain, which is three orders of magnitude faster than any known physiological process. Effects with Pseudorandom Number Generators Pseudorandom number generators are, by definition, those that depend upon an algorithm, which is usually implemented on a comp Liter. Radin (1985) analyzed all the PRNGs commonly in useand found that they require a startingvalLie (i.e., a seed), which is often derived from the computer's system clock. As we noted above, Radin and May (1986) showed that the bit stream, which proved to be "successful" in a PRNG study, was bit-for-bit identical with the strearn, which was generated later, but with the same * Later, this technique was adopted by Jahn (1982) for use in the RNG devices at PEAR. Approved For Release 2000/08/08 : CIA-RDP96-00789ROO320021 0001 _316 ,roved FP6 ReJease 2000108/9h8 : QIA-RDP96-00789R0qWAjRqPjW4 Arp A ications of ecision AUgmenta ion eory seed. With that generator, at least, there wits no change from the expected bit stream. Perhaps it is possible that the seed generator (i.e,, system clock) was subjected to some A P interaction. We propose two arguments against this hypothesis: (1) Even one cycle interruption of a computers' system clock will usually invoke a system crash; an event not often reported in PRNG experiments. (2) Computers use crystal oscillators as the basis for their internal clocks. Crystal manufacturers usual- ly quote errors in the stated oscillation frequency of the order of 0.001 percent. That translates to 500cycles for a 50MHz crystal, or to 10 [ts in tirne. Assuming that the quoted error is a 1-a estimate, and that a putative AP interaction acts at within the ± 2-o domain, then shifting the clock by this amount might account for only one seed shift in Radin and May's experiment. By Monte Carlo methods, we determined that, given a randorn entry into seed-space, the average number of ticks to reach a "significant" seed is 10; therefore, even if AP could shift the oscillators by 2-o, it cannot account for the observed data. Since computers in PRNG experiments are not reported as "crashing" often, it is safe to assume that PRNG results are only due toAC. In addition, since the results of PRNG studies are statistically insepa- rable from those reported with true RNGs, it is also reasonable to assume that the mechanisms are simi- larlyAC-based. Precognitive AC Using the tools of modern meta-analysis, Honorton reviewed the precognition card-guessing database (Honorton and Ferarri, 1989). This analysis included 309 separate Studies reported by 62 investigators. Nearly two million individual trials were contributed by more the 50,000 Subjects. The combined effect size was _E = 0.020±0.002, which corresponds to an overall combined effect of 11.4o. TWO important results emerge from Honorton's analysis. First, it is often stated by critics that the best results are from studies with the least methodological controls. To check this hypothesis, Honorton devised an eight- point quality measure (e.g., automated recording of data, proper randomization techniques) and scored each study with regard to these measures. There was no significant correlation between study quality and study score. Second, if researchers improved their experiments over time, one would expect a significant correlation of study quality with date Of Publication. Honortonfoundr=0.246,df=307,p < 2 x 10-~ In brief, Honorton concludes that a statistical anornaly exists in this data that cannot be explained by poor study quality or a large variety of other hypotheses; therefore, a potential mechanism underlying DAT has been verified. SRI International's RNG Experiment May, Humphrey, and Hubbard (1980) conducted an extensive RNG study at SRI International in 1979. They applied state-of-the-art engineering and methodology to construct two true RNGs, one based on the P-decay of 137pm and the other based on an MD-20 noise diode from Texas Instrument-,. It is be- yond the scope of this paper to describe, in detail, the intricacies of this experiment; however, we will discuss those aspects, which are pertinent to this discussion. Technical Details Each of the two sources were battery operated and optically Coupled to a Digital Equipment Corpora- tion LSI 11/23 computer. Fail-safe circuitry Would disable the sources if critical physical parameters Approved For Release 2000/08/08 : CIA-RDP96-00789ROO3200210001-317 ged Fp6ReleaRe 200P A npro 108/98: Cl1A-RDP96-00789ROO;?9P3J#R9Jg;4 "AW c ons o ecislon ugmen ation Heory a (e.g., battery voltages and currents, temperature) exceed preset ranges. Both sources were subjected to environmental testing which included extreme temperature cycles, vibration tests, E&M and nuclear gamma and neutron radiation tests. Both sources behaved as expected, and the critical parameters, such as temperature, were monitored and their data stored along with the experimental data. Asource was sampled at 1 KHz rate. After eight milliseconds, the resulting byte was sent to the comput- erwhile the next byte was being obtained. In this way, a continuous stream of 1 ms data was presented to the computer, May et al. had specified, in advance, that bit number 4 was the designated target bit. Thus each byte provided 3 ms of bits prior to the target and 4 ms of bits after the target bit. A trial was defined as a definitive outcorne from a sequential analysis of bit four from each byte. In exchange for not specifying the number of samples in advance, sequential analysis requires that the 'Iype I and`l~pe II errors, and the chance and extra-chance hitting rate be specified in advance. In May et al.'s two-tailed analysis, (x = P = 0.05 and the chance and extra-chance hitting rate was 0.50 and 0.52, respectively. The expected number of samples to reach a definitive decision was approximately 3,000. The outcome from a single trial could be in favor of a hitting rate of 0.52,0.48, or at chance of 0.50, with the usual risk of error in accordance with the specified Type I and Type 11 errors. Each of seven operators participated in 100 trials of this type. For an operator's data to reach indepen- dently statistical significance, the operator had to produce 16 successes in 100 trials, where a successwas defined as extra-chance hitting (i.e., the exact binomial probability of 16 Successes for 100 trials with an event probability of 0.10 is 0.04 where one less success is not significant). Two operators produced 16 and 17 successful trials, respectively. Temporal Analysis We analyzed the 33 trials from the two independently significant operators from May et al.'s experi- ment. Each of the 33 trials consisted of approximately 3,000 bits of data with -3 bits and +4 bits of I msibit temporal history surrounding the target bit. We argue that if the significance observed in the target bits was because of AP, we would expect a large correlation with the target bit's immediate neigh- bors, which are only ± I ins away. As far as we know, there is no known physiological process that can be cognitively, or in any other way, manipulated within a millisecond. We might even expect a 100% cor- relation under the complete AP model. We computed the linear correlation coefficients between bits 3 and 4, 4 and 5, and 3 and 5. For binary data: NO' - XI(df = 1), where q5 is the linear correlation coefficient and N is the number of samples. Since we examined three different correlations for 33 trials, we computed 99 different values of N02. Four of them produced X2s that were sign ifican t-wel I within chance expectation. The complete distribution is shown in Figure 7. We see that there is excellent agreement of the 99 correlations with tile X2 distribution for one degree of freedom, which is shown as a smooth Curve. We conclude, therefore, that there was no evidence beyond chance to suggest that the target bit neigh- bors were affected even when the target bit analysis produced significant evidence for an anomaly. So, Approved For Release 2000/08/08 : CIA-RDP96-00789ROO3200210001-3 18 Wroved 9% ReleaAs 0 - FlA-RDP96-00789ROQA22qWvJQPA19A cations ecision u$Sen9A(9P'A§or knowing the physiological limitations of the human systems, we further concluded that tile observed effects could not have arisen due to a human-mediated force (i.e., AP). Mathematical Model of the Noise Diode Because of the unique construction parameters of Texas Instrument's MD-20 noise diode, May et a]. were able to construct a quantum mechanical model of the detailed workings of the device. This model contained all known properties of the material and it's construction parameters. For example, the band gap energy in Si, the effective mass of an electron or hole in the semiconductor, and the impurity con- centration were among the parameters for the rnodel. The model was successful at calculating the diode's known and measured behavior as a function of temperature. May et al. were able to simulate their RNG experiment down to the quantum mechanical details of the noise source. They hoped that by adjusting the model's parameters so that the computed Output agreed with the experimental one, that they could gain insight as to where the causal influence "entered" tile device. May et al. were not able to find a set of model parameters that mimicked their RNG data. For example, changing the band gap energy for short periods of tirne; increasing or reducing the electron's effect mass; or redistributing or changing tile impurity content produced no unexpected changes in the device output. The only device behavior that could be effected was its known function of temperature. Because of the construction details of the physical RNG, this result could have been anticipated. The changes that could be Simulated in the model were all in the microsecond domain because of the details of the device. Both with the RNG and in its model, the diode's multi-MHz output was filtered by a 100-KHz wide bandwidth filter. Thus, any microsecond changes would not pass through the filter. In short, because of this filtering, the RNG was particularly insensitive to potential changes of the physical parameters of the diode. Yet solid statistical evidence for an anomaly was seen by May et al. Since the diode device was shown mathematically and empirically to be insensitive to environmental and physical changes, these results must have been as a result ofA C rather than any caUsalAP. In fact, this observation coupled with the bit bi 9 Approved For Release 2000/08/08 : CIA-RDP96-00789ROO3200210001-3 Figure 7. Observed and Theoretical Correlation Distributions. Wipgr6gsFo? 6 p6gip RgAqq(90q(q§ eb ~11A-RDP96-00789ROOMRIURA JA 4 timing argument, which we have described above, lead May et al. to question causality in RNG studies in general. Summary of Circumstantial Evidence Against AP We have identified six circumstantial arguments that, when taken together, provide increasingly diffi- cult requirements that Must be met by a putative AP force. In Summary, the RNG database demon- strates that: (1) Data are independent of internal complexity of the hardware RNG device. (2) Data are independent of the physical mechanism producing the randomness (i.e., weak nuclear or electromagnetic). (3) Effects with pseudorandom generators are statistically equivalent to those observed with true hardware generators. (4) Reasonable AP models of mechanism do not fit the data. (5) In one study, bits which are ± 1 ms from a "perturbed" target bit are themselves unperturbed. (6) A detailed model of a diode noise source, which includes all known physics of the device, could not simulate the observed data streams. In addition, AC, which is a mechanism to describe the data, has been confirmed in non-RNG experi- ments. We conclude, therefore, an AP force that is consistent with the database must ~ Be equally coupled to the electromagnetic and weak nuclear forces. ~ Be mentally mediated in times shorter than one millisecond. ~ Follow a I IV n law. For these to be true, an AP force would be at odds with an extensive amount of verified physics and common behavioral observables. We are not saying, therefore, that AP cannot exist; rather, we are sug- gesting that instead of having to force ourselves to invent a whole new science, we should look for ways in which AP might fit into the present world view. In addition, as DAT tries to accomplish, we should invent non-causal and testable alternate mechanisms for the experimental observables. Conclusions We have shown that DATcan determine whether a causal or informational explanation is more consis- tent with a given set of anomalous statistical data. In applying DAT to the substantial physical RNG database,we showed that an informational mechanism is strongly favored over a large class of perturba- tional ones. Given that one possible information mechanism (i.e., precognitiveAC) can, and has been, independently confirmed in the laboratory, and given the weight of the empirical, yet circumstantial, arguments taken together against AP, we conclude that the anornalOUs results from the RNG studies arise not because of a mentally mediated force, but rather because of a human ability to be a mental opportunist by makingAC-inediated decisions to capitalize on the locally deviant circumstances. Our recent results in the study of anomalous Cognition (May, Spottiswoode, and James, 1994) suggest the the quality of AC is proportional to the change in Shannon entropy. Following Vassy (1990), we compute the change in Shannon entropy for an extra-chance, binary sequence of length n. The total change of entropy is given by: Approved For Release 2000/08/08 : CIA-RDP96-00789ROO3200210001-30 )AAMNSA ffPSAWARg A PANMO'rP I A-RDP96-00789R00AWRuVgPjjA 'JS = SO - S, where for an unbiased binary sequence of length n, S, = n, and S is given by: S = - np,log2p, - n(1 -P1)'()92(1 _P1)' Letpj = 0.5 (1 + s) and assume that v, the effect size, is small compared to one (i.e., typical RNG effect sizes are of the order of 3 x 10-4). Using the approximation: In (1 + e) = E - F 21 we find that S is given by: 2 S n 21n2' or that the total change of entropy for a biased binary sequence is given by; AS = So - S = n F2 21n2 Since our analysis of the historical RNG database shows that the effect size is proportional to 1 V_n (i.e., Z-score is independent of n), the total change of Shannon entropy becomes a constant that is inde- pendent of the sequence length. Thus, if entropy is related to what is being sensed by anomalous cogni- tion, then this argument suggests that informational processes are responsible for the RNG anomaly. The one exception to this conclusion is Braud's study of 11P on red blood cells. It may be that there is somethingunique about living systems that can account for this observation. On the other hand, it is the only biological AP study we Could find that could be analyzed by DATand the perturbation hypothesis is only weakly favored over the selection one (i.e., p < 0.023). Before we would be willing to declare that AP is a valid mechanism, more than a single, albeit well designed and.executed, study is needed. Generally,we suggest that future RNG, PRNG, and biologica]AP studies be designed in such away that the criteria, as outlined in this paper and in May, Utts, Spottiswoode (1994), are adhered to for a valid DAT analysis. Our discipline has evolved to the point where we can no longer be satisfied with yet one more piece of evidence of a statistical anomaly. We must identify the sources of variance as suggested by May, Spottiswoode, and James ('1994); limit thern as Much as possible; and apply models, such as DAT, which can begin to shed light on the physical, physiological, and psychological mechanisms of anomalous mental phenomena. Approved For Release 2000/08/08 : CIA-RDP96-00789ROO320021 0001 hff6.qy&q PlA-RDP96-00789R00A2AQAjgAqjcA References Bierman, D. J. (1988). Testing the IDS model witli a gifted Subject., Theoretical Parapsychology, 6, 31-36. Braud, W G. and Schlitz, M. J. (1989). Possible role Of IntUituve Data Sorting in electrodermal biological psychokinesis (Bio-PK). The Journal of'the American Societyfbr Psychical Research, 83, No. 4, 289-302. Braud, W G. (1990). Distant mental influence of rate of hernolysis of human red blood cells. TheJoumal of the American Society fbr Psychical Research, 84,No.1,1-24. DeGroot, Morris H. (1985). Probability and Statistics, 2nd Edition. Reading, MA: Addison-Wesley Publishing Co. Dobyns, Y. H. (1993). Selection versus influence in rernote REG anornalies. Joumal of Scientific Exploration. 7, No. 3, 259-270. Honorton, C. and May, E. C. (1975). Volitional control in a psychokinetic task with auditory and visual feedback. Research in Parapsychology, 1975, 90-91. Honorton, C. and Ferrari, D. C. (1989) "Future Telling:" A meta-analysis of forced-choice precognition experiments, 1935-1.987. Journal of Parapsychology, 53, 281-308. Jahn, R. G. (1982). The persistent paradox of psychic phenomena: an engineering perspecitve. Proceedings of the IEEE. 70, No. 2, 136-170. Lewis, T. G. (1975). Distribution Sampling for Computer Simulation. Lexington, MA: Lexington Books. May, E. C. (1975). PSIFI: A physiology-coupled, noise-driven random generator to extend PK studies. Research in Parapsychology, 1975, 20-22. May, E. C. and Honorton, C. (1975). A dynamic PK experiment with Ingo Swann. Research in Parapsychology, 1975, 88-89. May, E. C., Humphrey, B. S., Hubbard, G. S. (1980). Electronic System Perturbation Techniques. Final Report. SRI International Menlo Park, CA. May, E. C., Radin, D. I., Hubbard, G. S., and Humphrey, B. S. (1985). Psi experiments with random number generators: an informational model. Proceedings of Presented Papers Vol L The Parapsychological Association 28th Annual Convention, Tufts University, Medford, MA, 237-266. May, E. C. (1990). As chair for the session at the annual meeting of the Society for Scientific Exploration in which this original work was presented, I pointed Out the problern of the likelihood ratio for the PRNG data from the floor of the convention. May, E. C., Spottiswoode, S. James R, and James, C. L. (1994). Shannon entropy as an Intrinsic Target property: Toward a reductionist model of anomalous cognition. Submitted to The Journal of Parapsychology. May, E. C., Utts, J. M., Spottiswoode, S. J. (1994). Decision augmentation theory: Toward a model of anomalous mental phenomena. Submitted to The Journal of Parapsychology. May, E. C. and Vilenskaya, L. (1994). Overview of current parapsychology research in the former Soviet union. Subtle E nergies. 3, No 3. 45-67. Nelson, R. D., Jahn, R. G., and Dunne, B. J. (1986). Operator-related anomalies in physical systems and information processes. Jorunal of'the Societyfor Psychical Research, 53, No. 803, 261-285. Nelson, R. D., Dobyns, Y. H., Dunne, B. J.,and Jahn, R. G., and (1991). Analysis of Variance of REG Experiments: Operator Intention, Secondary Parameters, Database Structures. PEAR Laboratory Technical Report 91004, School of Engineering, Princeton University. Approved For Release 2000/08/08 : CIA-RDP96-00789ROO320021 000, _%2 W[WcRig ffWeFNRt~E~Mr39MWigg%ga'rPIA-RDP96-00789ROQ4NUlfigAig Raby, S. (1985). Supersyminetry and cosmology. in Supersymmetry, Supergravity, and Related Topics. Proceedings of the XVth GIFT International Seminar oil Theoretical Physics, Sant Feliu. de Guixols, Girona, Spain. World Scientific Publishing Co. Pte. Ltd. Singapore, 226-270. Radin, D. 1. (1985). Pseudorandom Number Generators in Psi Research. Journal of Parapsycholog. 49, No 4, 303-328. Radin, D. 1. and May, E. C. (1986). Testing tile Intuitive Data Sorting mode with pseudorandom number generators: A proposed method. The Proceedings of Presented Papers of the 29th Annual Convention of the Parapsychological Association, Sonoma State University, Rohnert Park, CA, 539-554. Radin, D. I. and Nelson, R. D. (1989). Evidence for consciousness-related anomalies in random physical systems. Roundations of Physics. 19, No. 12, 1499-1514. Rosenthal, R. (1991). Meta-analysis procedures for social research. Applied Social Research Methods Series, Vol. 6, Sage Publications, Newbury Park, CA. Schmidt. H. (1969). Precognition of a quantum process. Journal of Parapsycholog. 33, No. 2, 99-108. Schmidt. H. (1974). Comparison of PK action on two different random number generators. Journal of Parapsycholog. 38, No. 1, 47-55. Vassy, Z. (1990). Experimental study of precognitive timing: Indications of a radically noncausal operation. Journal of'ParapsycholoA~,. 54, 299-320. Vilenskaya, L. and May, E. C. (1994). Anomalous mental phenomena research in Russia and the Former Soviet Union: A follow Lip. Submitted to the 1994 Annual Meeting of the Parapsychological Association. Walker, E. H. (1987). A comparison of the intuitive data sorting and quantum mechanical observer theories. Journal of Parapsychology, 51, 217-227. Woodworth, R.S. and Schlosberg H. (1960). Experimental Psychology. Rev ed. New York Holt. New York. Approved For Release 2000/08/08: CIA-RDP96-00789ROO3200210001- J3