Bs"d, Sivan 5761 (June 2001).
Designed to Distract
(On McKay and Kalai's Response to our article
"New Statistical Evidence for a Genuine Code in Genesis")
By Doron Witztum
McKay and Kalai's response,
titled "Replication of the famous rabbis experiment a reply
to Doron Witztum" [1], purports to answer our critical article [2] "
New Statistical Evidence for a Genuine Code in Genesis". But it is
merely a failed attempt to distract people from McKay et al's deceit
which we discussed [2] and which they have still not explained.
I.
On the Working Procedures of McKay et al:
In our article [2] we detailed McKay et al's manipulations
concerning their "replication" "according to Dr Emanuel".
Here, we will briefly mention some main points. First, let us discuss
the preparation of "list b" and "list c" which were
intended to "mimic" the original second list of Prof. Havlin.
- McKay et al asked Emanuel to prepare appellations for 35
personalities and he did so and sent them the appellations. Thus McKay
ordered appellations for this group of personalities, which constitute
the total of several possible choices, in order to select only
some of these appellations a posteriori.
- McKay et al indeed chose 33 rabbis from this list
a posteriori.
These two stages (A and B) absolutely contradict the procedure
they reported in Statistical Science:
"1.
A list of rabbis was drawn from Margaliot's encyclopedia
by applying WRR's criteria for their second list, while correcting the
errors they made. Our list differed from WRR's in dropping two rabbis
and including three others. One rabbi who fits the selection criteria
could not be included because he appears incorrectly in WRR's first list.
2. Emanuel was shown
the spelling rules and table of appellations for WRR's first list as they
first appeared in WRR (1986). He then compiled a parallel table
of appellations for our list of 33 rabbis, attempting to follow the rules
and practices of WRR's first list." (Emphasis mine).
Note how in their response they
attempt to implicate Emanuel in their misdemeanor [1]:
"Nevertheless, after the experiment
was finished, Emanuel approved the description of it that appeared in
our paper."
This is baseless: 1. Emanuel explicitly said several months after
the publication of their article, that he never read it. 2. He was extremely
surprised when he learnt that two rabbis from his list of appellations
had been "dropped".
- The criterion for choosing the 33 rabbis was invented ad hoc
and it contradicts the criterion they publicized on earlier occasions.
- Concerning the dates: Contrary to their response [1], they did
not instruct Emanuel to check or emendate all the dates (see later
section IV). They only did so concerning one rabbi (or a few).
- Concerning the appellations: Many peculiar decisions were made
in "list b" (and "list c") which even Emanuel could
not explain. Considering that the instructions given him concerning the
dates were through "hints" ([2] chap. I sec. 1(B)), who knows
how, and how often such "hints" were given concerning the appellations?
- McKay et al indicated two mistakes in Emanuel's "list
b" and asked Emanuel to correct them. But these were not all the
mistakes they detected. In section II we will see from their own words
that there were at least two more mistakes in their opinion (involving
the name "Ayash"). But they chose to correct only what suited
them (See [2], chap. 3).
Paragraphs A-F above describe the direct manipulation of lists b and
c.
But it should be mentioned that this was preceded by preparatory spadework:
- At an earlier stage, McKay et al ordered appellations for
"list a" from Emanuel. They ordered them for only the 32 rabbis
of Havlin's second list.
- Note that from the collection of appellations Emanuel gave them
at this stage ("list a") it was possible to estimate which appellations
would remain in the next stage for the 32 rabbis, after Emanuel
would use the spelling rules and would work according to Havlin's first table of
appellations and names. This is because McKay et al also
understood Havlin's table and the spelling rules (in fact they knew them
better than Emanuel, even suggesting corrections to his work).
- No protocol dictated that the experiments be continued on further
lists, like b or c. Spadework like this would allow one to act unrestrictedly
in the next stage. For example:
- If the significance expected according to #2 is about 0.00002 (many
appellations would be expected to survive in the next stage) - one can
choose to simply not continue to the next stage.
- If the significance expected according to #2 is about 0.002 (about
half of the number of the appellations in Havlin's list could be expected
to survive) - one can use steps A-F above, to "sufficiently"
downgrade the significance.
II.
The "Ayash" Episode:
When I realized that McKay et al had secretly erased the appellations
of R. Aharon of Karlin and R. Yehudah Ayash from Dr Emanuel's appellations
lists ("list b" and "list c", see [2] chap. 1) without
Emanuel's knowledge, I phoned Emanuel [3] and asked him to send me the
original "list b" or "list c". But, acting on McKay
et al's instructions, he refused.
So I tried to find out which were the missing appellations. In the course
of our conversation:
- Concerning R. Aharon of Karlin, Dr Emanuel mentioned only two appellations:
"Rabbi Aharon" and "Rabbi Aharon Ha'Gadol", as I reported
concerning them in [2].
- Concerning R. Yehudah Ayash, Emanuel originally claimed that no
appellations exist. But following my retort that this was almost impossible
and also most unlikely considering Emanuel's method of compiling lists
b and c, Emanuel finally verified that he used the name "Rabbi Yehudah"
, the surname "Ayash" and the full name "Yehudah Ayash"
. I mention these names in English because Emanuel did not say
whether he had written "Ayash" with one or with two yuds.
Interestingly, for "list a" which was bound by absolutely
no grammatical rules Emanuel wrote the name "Ayash" as follows:
"שאייע" and "שאייאע".
But for "list b" (and "list c") he was supposed
to spell the names according to the spelling rules of WRR's first experiment
so write McKay, Kalai et al in their article in Statistical
Science:
"2. Emanuel was
shown the spelling rules and table of appellations for WRR's first list
as they first appeared in WRR (1986). He then compiled a parallel table
of appellations for our list of 33 rabbis, attempting to follow the rules
and practices of WRR's first list." (Pg. 163)
Thus Emanuel was supposed to do the final spelling according to the
given rules and not according to his private opinion. Therefore, I concluded
that Emanuel must have written those names as follows: "
שאיע " and "שאיאע", because that is how the given spelling
rules dictate. Thus I finally reported in [2] that "A conversation
with Emanuel helped us to deduce the missing appellations".
Even more, McKay et al did the same thing themselves. In an article
[4] describing their second list for War and Peace they write:
"The independent expert mentioned above used for the a-priori
experiment the forms שאייע and שאייאע,
but for our "experiment" here we follow the 'grammatical"
dictum that Witztum has set in the case of Oppenheim, i.e. that must
not use a "double yud" in transliterating foreign names, so
we write שאיע and שאיאע." [Note that in
[5] they admit that this spelling rule was already utilized in Havlin's first list.]
In other words, they write here that their expert, Dr Emanuel, used
"שאייע" and "שאייאע"
for his first a priori experiment ("list a"),
and that according to the spelling rules it is written
"שאיע" and "שאיאע".
Note that in their reply McKay and Kalai stress:
"Conclusion: Witztum did not use the spellings provided by Emanuel."
And then they hypocritically add:
"We also wonder why Witztum's article fails to mention that he
changed the spelling."
They say this knowing full well that the only spelling publicized
by Emanuel was in "list a", that is, before the spelling
rules were applied. But after the spelling rules were applied the spelling
had to change as they explicitly write themselves!
It is amazing that they basically claim in their defense that Dr Emanuel
used his own initiative in the spelling and ignored the spelling rules
which he had been requested to use, by
McKay et al!
(Note that, according to them, Emanuel's mistake was his reliance on
spelling in an unvowelized text while the requested spelling was of a
vowelized text. See further details about this in [6]).
III. Concerning the
Dialog with Dr Emanuel.
McKay and Kalai complain about my dialog with Dr. Emanuel, even though
Kalai himself (and others who worked with MBBK) contacted Prof. Havlin
and queried him on the phone concerning his work and his methods. Let
us remind the reader:
- Havlin publicized his explanations to his lists after they raised
questions about them. They, however, never asked Emanuel to prepare a
list of explanations.
- They only publicized Emanuel's lists long after they were prepared.
- When I contacted Dr Emanuel and asked for his reasoning he said
that so much time had passed that he no longer remembered.
- McKay et al concealed the "game" they had played
with Emanuel's lists (see [2], chap. 2), so they were obviously angry
when my direct contact with Emanuel exposed their fraud.
- In "Concerning McKay's Response to our article 'Of Science
and Parody'" [6], we added more information from what Emanuel said
which made it clear that the work of "Anonymous" (and McKay)
relied on nonprofessional books. This is obviously embarrassing for McKay
and it is understandable why he prefers that these things remain unknown.
In conclusion: McKay et al's procedures concerning Emanuel's
lists were both procedurally and ethically wrong. When this was exposed,
McKay and Kalai reacted not like scientists but rather like frustrated
lawyers and resorted to mud throwing. We ignored their baseless
and unsubstantiated insults.
IV. Concerning
their treatment of the dates:
Concerning their treatment of the dates McKay and Kalai claim as
follows:
"The issue of dates is not much different. In their own experiment,
WRR replaced or deleted some of Margaliot's dates on the basis of historical
evidence. We did not originally ask Emanuel to do the same, but of his
own accord he started to make comments about the inaccuracy of some of
the dates in the Margaliot encyclopedia. Emanuel is an expert on such
historical questions, so we then asked him to check all the dates. This
gave us a compilation of dates of the best possible historical accuracy
without the need for any subjective choice of our own."
But:
A. Reminder:
In our article [2] (chap. 1) we already discussed the strange procedure
which led to addition/omission/replacement of a few dates, a process which
makes their date changes "contaminated" and useless for
any experiment.
1. This is the sequence
of events according to Dr Emanuel quoted in [2]:
"
At a certain stage I understood that this matter interested
them. Perhaps they asked it for one particular rabbi and I went and did
the same for them all
"
Compare this to the what McKay and Kalai say:
"
so we then asked him to check all the dates".
2. Note that nowhere
in their Statistical Science paper did McKay and Kalai ever mention
that they asked Emanuel to check the dates. Only after the publishing
of our article [2] was the following claim created:
"
so we then asked him to check all the dates".
B. Chronology of their dates' treatment:
- On 7 November
'96, Dr Bar-Natan et al wrote to Prof. Aumann outlining a proposal
to have all the dates relating to our experiment examined by an independent
expert. We basically accepted this proposal, and in a letter of reply
to Prof. Aumann (14 November '96) we added three minor remarks. But Bar-Natan
et al retracted from their proposal.
- We consider this the
reason they never had the audacity to ask Dr Emanuel to check the dates:
It would be too obviously improper to retract from a mutually agreed examination
and instead do it unilaterally.
- After Dr Emanuel examined
some dates they showed some interest; but they did not instruct him to
check them all. This left them the freedom to use:
- All date changes,
- Or, some of them,
- Or, none of them.
Conclusion: It is hard to imagine a more defective procedure.
V. Concerning the
accuracy of their dates' changes:
Let us reread the end of the above quotation of McKay and Kalai:
"Emanuel is an expert on such historical questions, so we then
asked him to check all the dates. This gave us a compilation of dates
of the best possible historical accuracy without the need for any subjective
choice of our own."
But:
- True, Dr Emanuel is a historian. But Mckay et al
should have made clear that his expertise lies in the Middle Ages, a period
encompassing only three out of the 32 personalities in the second
list.
- McKay and Kalai claim that they achieved:
"a compilation
of dates of the best possible historical accuracy". But this is their
mere personal opinion and since they understand little of these matters
it is worthless.
On the other hand one can demonstrate that there are mistakes in their
"corrections".
An example:
McKay et al claim that the death date of R. Yehudah Hasid (no.
15 in the second list) is in doubt. Therefore they erase it. What creates
this doubt? They briefly explain in their article in Statistical Science
([7], in the References, entry Gedaliah):
The source of his death date (relied on by WRR) is in the book of R.
Gedaliah of Siemiatycze, "Pray for the Peace of Jerusalem" [8].
But the date there contradicts the date in another book called "The
Travels of Rabbi Moshe Yerushalmi" [9].
Their conclusion that this creates a doubt is nonsense, as we will soon
see. Therefore, we cannot believe that this "correction"
was done by any qualified historian.
Because this case is obvious we will quote the sources and let the reader
judge for himself.
1. The book "Pray for the Peace of Jerusalem" of R. Gedaliah
of Siemiatycze ("source A"), a close disciple of R. Yehudah
Hasid, traces R. Yehudah's life in detail from when he arrived in Jerusalem:
- R. Yehudah came to Jerusalem on Wednesday, Rosh-Hodesh Heshvan (=30
of Tishri) 5461.
- On Friday R. Yehudah immersed in a mikveh and became ill.
- A detailed description follows: R. Yehudah's behavior that Friday
night, what he said, what his son in law said to him, and what his disciples
did. The author mentions the name of the doctor whom they wished to bring,
and why they did not call him that night. These details accord remarkably
with what we know of that period and of that doctor.
- There is a detailed description of what happened that Sabbath (Saturday)
morning. How R. Yehudah recovered and prayed, but then "fell back
into bed as before".
- Then "he lost his speech and he could not speak at all until
Monday, until he died without saying anything".
- "And when he died he was immediately buried on that day with
absolutely no delay, and the hasid R. Zalman Byaller was also buried
that same day".
In that year the first of Heshvan was on a Thursday, and therefore Monday
was the fifth of Heshvan.
2. But in the book "The Travels of R. Moshe Yerushalmi"
(published in the year 5529, from now on "source B"),
we find a different description:
- "And it happened that when R. Yehudah Ha'Hasid came [to the
Land of Israel] with four hundred disciples, he decided that as soon as
he came to Jerusalem he would go to the Western Wall and then the Messiah
would come".
- "And behold R. Yehudah went up and came to Jerusalem, and on
Friday he went to immerse in a mikveh, and after immersing he intended
going to the Western Wall. But when he immersed, and his disciples were
helping him immerse, he touched a reptile while he was immersing in the
water, and he shouted to the disciples that they should take him out lest
he die, and the disciples took him out from immersing and he died and
was buried before the Sabbath (Saturday)".
According to this source he died on Friday, 2nd of Heshvan
5461. Therefore McKay et al claim (in Emanuel's name) that there
is a contradiction between source A and source B, and therefore
the date should be considered doubtful.
3. The basic task of an historian is to weigh his sources. Let us compare
them:
Source A:
- The testimony of an eye witness who participated in the events himself.
- Detailed relating of the events.
- Agreement of details given in it to our knowledge from other independent
sources (for instance, the name of the doctor, and the fact that it was
dangerous to go out because of the Arabs).
Source B:
- Hearsay from decades (about sixty years) after the event.
- The story has an obvious legendary-Messianic flavor: R. Yehudah
wishes to bring the Messiah straight away, but is prevented by the powers
of impurity (the reptile).
- No details included are verifiable from another source.
How can a second hand testimony from a later period (decades later)
contradict the detailed and reliable testimony of a witness (who both
saw and heard)?
In conclusion, the weight of "source B" cannot compare
with "source A" and cannot be considered a valid contradiction.
4. Because it is hard
to imagine that this "correction" was made with the sanction
of any authentic historian, doubt is cast not only on beginning of McKay
and Kalai's statement:
"This gave us a compilation of dates of the best possible historical
accuracy
"
but on its end as well:
"
without the need for any subjective choice of our own."
Especially considering what we said above in section IV.
VI. The Standard of their Work:
McKay and Kalai declare:
"In our replication we tried very hard to apply WRR's rules correctly.
One necessary exception was that Rabbi David Ganz had to be excluded because
he was already in WRR's first list. (WRR excluded him for the same reason.)
Otherwise, neither Witztum nor anyone else has found an error in our selections."
See our article [2] chap. 1 where we unequivocally prove that:
- MBBK ordered appellations for 35 personalities from Emanuel who
prepared and delivered them.
- MBBK, a posteriori, chose 33 personalities from this
list.
- Their criterion for this choice was ad hoc, and contradicted
the criterion publicized by MBBK on earlier opportunities.
- Thus MBBK achieved the worst results possible.
Conclusion: It took a lot of nerve to write the above quoted
assertion.
VII.
What Did Our Measurements Prove About Emanuel's
List?
As we demonstrated in [2] and as we repeated and added in the preceding
sections, the changes in the composition of the list and the dates are
"contaminated" and unusable. The only possible use for Emanuel's
appellations is in investigating the claim that WRR's success stemmed
from "freedom" in choosing appellations.
In [2] chap. 2, I described how we repeated WRR's experiment with just
one difference: We used Dr Emanuel's appellations instead of Prof. Havlins's.
The experiment succeeded to the probability of 0.0036. An additional measurement
showed that this was consistent with the original experiment's result
if one takes the size of the sample into account. (Concerning their claim
that we altered the spelling of "Ayash", see above section II
that this is not so: We utilized the necessary spelling rules as they
themselves explicitly wrote. If they claim that Emanuel erred and misapplied
the rules, this should obviously have been rectified).
Thus we proved that the success of the original experiment was not because
of "freedom" in choosing appellations: Emanuel's list was completely
unbiased towards WRR; if there was any bias it was to MBBK's benefit.
VIII. The Criticism
against our Consistency Test:
McKay and Kalai raise several arguments against our consistency test
described in [2] chap. 2:
A. They claim: There
is no mathematical rationale in comparing a randomly created sub-group
to a group assembled by an expert. The latter has "strong internal
structure" which affects its behavior in a way that a randomly assembled
sub-group "cannot match".
But:
We think Emanuel's list has no "internal structure" which
could influence ELS appearances, especially in proximity to specific dates.
Therefore we see no need to answer this claim until McKay and Kalai give
some demonstration of this "internal structure" in Emanuel's
list.
B. They claim: That Emanuel's
appellations are, on the average, the most important and should therefore
succeed more, according to our theory.
But:
- The assumption of McKay and Kalai that Emanuel chose the most important
appellations is baseless. Emanuel omitted many major appellations. He
gave no reason for his decisions and claimed that he could not remember
his rationales. For more about this see [2] chap. 3.
- According to their theory, that there is no code phenomenon, these
appellations are not "more important". So why does Emanuel's
list succeed (in ratio to group size) as would be expected if there is
a phenomenon?
C. They claim: That the
complement of Emanuel's list of appellations (that is Havlin's group minus
Emanuel's group) succeeds 250 times better than Emanuels' list, even though
both lists are similar in size.
But:
- For accuracy's sake, the complement of Emanuel's list succeeds
about 90 times better than Emanuel's list. By the way, the difference
in size is: "Emanuel" has 63 word pairs and the Complement has
73 pairs.
- The question should be: Is the result for the Complement reasonable
as compared with that for a randomly created sub-group of the same size?
We repeated our test exactly as it is described in [2] chap.
2, and concluded that the probability of receiving such a result (or better)
for a randomly created sub-group of Havlin's group, of the same size,
is rather large: p=0.3.
D. They claim: That they
examined the odds of the data in claim C, according to our test, and received very small
probability, less than one in 65,000. (By the way, even without repeating
their measurements it is easy to see that there is a simple mistake in
their calculation [10]).
But:
- What is the source of the result presented by McKay and Kalai and
why is it different from our result in the previous paragraph?
It turns out that they measured something completely different:
They measured the probability of a randomly created sub-group of the Complement,
with 63 pairs, succeeding the same as Emanuel's group (or less). This
is a statistical mistake because it involves conditional probability.
We will illustrate this with an extreme case: We have two mutually
exclusive groups of word pairs. Group A has 1,000,000 pairs and group
B has 1,000,001 pairs. We are given that:
(a) The success of B is twice as that of A.
(b) No single pair of B can raise the success of the rest of the group
by a factor of two.
Is there any sense in the following test: What is the probability that
a sub-group of B, having 1,000,000 pairs, would have equal (or worse)
success as A?
It is clear in advance (according to (a) and (b)) that this
cannot happen.
But according to McKay-Kalai one would "measure" a "probability"
of one in a million!
- Thus all that their
"measurement" proved was what
is obvious with no measuring at all: That the Complement group is stronger
than Emanuel's group. The correct question should be what is the probability
that a sub-group of this size and strength could be randomly chosen from
Havlin's group. The answer, as stated before, is p=0.3.
References:
McKay, B. D. and Kalai, G. (2001).
"Replication of the famous rabbis experiment a reply to
Doron Witztum". Available
at http://cs.anu.edu.au/~bdm/dilugim.
Witztum, D. (2000).
New statistical evidence for a
genuine code in Genesis.
A telephone conversation with Dr S.
Emanuel, 31 Dec. '99, about 13:00.
Bar-Natan, D. and McKay, B. D. (1999).
Equidistant letter sequences in Tolstoy's "War and Peace".
http://cs.anu.edu.au/~bdm/dilugim/WNP.
Anonymous (1999). Equidistant letter
sequences in Tolstoy's " War and Peace": Witztum's
"refutation" refuted. (Assertion 4)
http://cs.anu.edu.au/~bdm/dilugim/WNP.
Witztum, D. (2001).
Concerning McKay's Response
McKay, B. D., Bar-Natan, D., Bar-Hillel, M.
and Kalai, G. (1999). Solving the Bible Code puzzle. Statist. Sci. 14
No. 2 150-173.
Yaari, A. (1946). Travels in the Land of
Israel, Tel-Aviv, Israel, 328-329.
Ibid, 448.
The mistake: According to them they
calculated 65,000 random sub-groups applying the given procedure
in [2] (chap. 2, #3). But it is easy to calculate
that according to this there are only less than 14,000 different
choices!