BS”D     Last update: Shvat 5769 (Feb. 09).

 

A Review of the Attempts to Invalidate
the Torah Codes

 

Introduction

The battle being waged against the phenomenon of Torah Codes is emotionally highly-charged and loaded with preconceptions. Some of the critics involved are scientists who seek to negate the research, not on the basis of objective observations and evidence, but driven by a belligerent worldview. Given these conditions, their pseudo-scientific approach can hardly qualify as a yardstick for establishing truth. We are fiercely attacked and even vilified by these opponents, who feel justified by their philosophy to mislead and distort. Any means seems to justify their goal: to deny the existence of the Torah Codes.
        This is the hallmark of the criticism in the works of professors Brendan McKay, Maya Bar-Hillel, Dror Bar-Natan and Gil Kalai (MBBK). In their joint Statistical Science paper, they criticize the work of Witztum, Rips and Rosenberg (WRR) which proved the existence of Torah Codes and was published in the same journal five years earlier. Since MBBK think that WRR’s thesis proves the divinity of the Torah, they are forced by their antagonistic philosophy to attribute the success to deceit. Their starting point is not the question of whether there was deceit, but only how the assumed deceit could have been perpetrated.
        A similar bias is found even with certain religious scientists. Their a priori antagonism to the idea of a divinely-place code in the Torah provokes an a posteriori criticism designed to justify that antagonism. As a classic example, Professor Avraham HaSofer wrote the reasons for his opposition to the codes even before having a copy of the pre-print describing our research!

The purpose of this survey is to review in short the claims of our critics, along with our refutations of these claims. In this survey, we also refer the reader to relevant articles which refute specific claims more comprehensively.
        We think that the elementary mistakes and the deliberate deceptions exposed in MBBK’s work (as shown in this survey) are a direct result of their strong opposition to the implications of the Torah Codes. This is typical of a critique whose conclusions are already decided even before any examination is done.
[Note: To better understand the background to the controversy, read: A Brief Chronology of Torah Codes Research: Up to the First Scientific Publication.

MBBK's Paper in Statistical Science

MBBK's article in Statistical Science is a paper that encompasses the main criticisms against the Codes. It is considered to be the main critique of the Codes research, and is the only one published in a peer reviewed scientific journal. Let's concentrate on it.
        The central message in MBBK’s paper is the accusation that WRR cheated by "cooking" their data (actually, they wrap this accusation in softer language). The main statistical work presented in MBBK’s paper is the "Study of Variations", aimed at proving the “cooking” accusation.

A.  MBBK claim: The “Study of Variations” proves that WRR "cooked" their data.

In their "Study of Variations" MBBK made many variations of WRR's experiment, changing parameters, functions, and the like. MBBK claim that the variations "almost always" make the result worse than the original. They assert that this suggests that “optimization” (i.e. “cooking”) of parameters and functions took place in WRR’s experiment.
        But MBBK know full well that no such “optimization” was available since WRR were constrained to using the same parameters and functions as in their previous experiment. Therefore MBBK instead claim that there was an “optimization” of the data. For this purpose, they invented the following hypothesis:

“…the apparent tuning of one experimental parameter may in fact be a side-effect of the active tuning of another parameter or parameters.”

That is: that “optimization” of the data (mainly the names and appellations) also causes the result to worsen when variations to the parameters and functions are applied, and the effect is similar to the effect of “optimization” of the parameters and functions.
        Relying upon their hypothesis, MBBK assert that the results obtained for the "Study of Variations" proves that WRR "cooked" their data.

The refutation:

In order to have scientific meaning, MBBK's "Study of Variations" must be based both on:
(A)        A firm (proven) hypothesis;
(B)        An unbiased set of independent variations.
Failure of (A) will render their study worthless.
Failure of (B) will nullify not only the study itself, but also cast serious doubts on the integrity and honesty of the testers.
        Even assuming for argument’s sake that MBBK's work has scientific significance, the absence of an objective closed set of variations gives the study's results two possible interpretations:
(1)        Proof that "tuning" was involved in assembling WRR's data.
(2)        Proof that "tuning" was involved in assembling MBBK's variations.

In our comprehensive research paper MBBK’s Study of Variations we demonstrate that there was indeed a grave failure of (B) [and maybe also of (A)], and that the results of MBBK's “Study of Variations” indicate not (1) but (2).

Our paper does not deal with details of bibliography and Jewish history. It only analyzes MBBK’s work logically and mathematically-statistically, describes several control experiments and presents many experimental results. Its main subjects are as follows:

• We point out many serious logical and statistical flaws in MBBK's work, each of which devalues their entire work.
• We bring many examples of serious mathematical-statistical mistakes and deceptions.
• We show how MBBK revealed only some of their results, and that the way they chose to present those results seriously skews the true picture which would be drawn from their own variations. We also explain the fallacy of MBBK's a posteriori excuses for their partial presentation of their results.
• We submit their thesis to control experiments. For example we check how their thesis performs on an admittedly "cooked" list­­­­ – the list they themselves "cooked" to succeed in "War and Peace". This experiment is based on their own prediction in their paper in Chance: That the results of their list for "War and Peace" should worsen and/or improve to the same extent as WRR’s list.

But their predication fails. The experimental results destroy their thesis: Applying the variations to their list in "War and Peace" worsens the results only in less than half of the variations!

Thus the control experiments indicate that MBBK’s results in the "Study of Variations" are due to their "tuning" of its variations.

• We unfold the "evolution" of their "Study of Variations". This evolution went through at least four stages. The researchers changed the set of variations time after time, and made a posteriori changes in their presentation of the results; each new presentation (even of the same variations) was advantageous to MBBK’s goal. We bring further experimental evidence of MBBK's "tuning", showing that there is no connection between their presented results and any "optimization" by WRR.  

In conclusion: MBBK's "Study of Variations" is proven to be not only flawed and invalidated, but also a product of deception and suppression of vital data. We strongly recommend reading our paper against the "Study of Variations". Even someone unknowledgeable in Hebrew and Torah bibliography and unable to detect MBBK's deceptions in these fields, will be able to see through MBBK's methods.

[Update: Following our criticism, McKay published his response, The Analysis of Variations – a reply to Doron Witztum, in which he reconsidered his hypothesis (titled “Reconsideration of our hypothesis")… Read about it in our paper:

McKay's Reconsideration of his Hypothesis
Following our Criticism
of his "STUDY OF VARIATIONS"

All this alone makes MBBK's study worthless. Removing the elegant wrappers from their other assertions against WRR brings out the same bad smell as the "Study of Variations". For example:

B.  MBBK claim: Using a new list prepared by independent expert, we did an experiment which “mimics” WRR’s, and it failed!  

MBBK engaged and guided Dr Simcha Emanuel, a specialist in rabbinical history at Tel-Aviv University, to compile several lists of names and appellations without WRR's knowledge. One of the lists was intended to “mimic” WRR’s second list.
        MBBK claim that Dr Emanuel's new list failed in the permutation test and that this proves that WRR’s result was achieved through “cooking” of the data. MBBK consider the experiment on this list to be the most telling of "perhaps the most important class of experiments".

The refutation:

In our paper New Statistical Evidence for a Genuine Code in Genesis, we thoroughly examine the lists publicized by MBBK, information received through our conversations with Dr Emanuel, and excerpts where MBBK quote their expert. It becomes clear that their experiment was indeed important and instructive. We draw the following significant conclusions:

• MBBK deceptively fabricated an "independent experiment". They presented a complete failure of the results while hiding an important fact:
• The necessary conclusion derived from Dr Emanuel's list of names and appellations is that WRR's success stemmed not from "cooking" of names and appellations (as MBBK claim) but from the existence of genuine codes in Genesis.

Let us briefly add:
Emanuel's new list contains names and appellations of the personalities included in WRR’s second list, which hecollected without seeing Prof. Havlin's original names and appellations for it. We repeated WRR’s original experiment exactly, with one single change: Instead of Havlin’s names and appellations, we used Emanuel’s. The experiment succeeded with considerable significance.  For the details given in our paper click here.

 [Update: McKay and Kalai responded to this paper. In our opinion, everything they write is only an attempt to distract attention from the serious allegations we made in the above paper. Here is our full response.]

C.  MBBK claim: WRR had sufficient "wiggle room" in choosing data to "cook" a successful list.

MBBK assert that “the rules and constraints laid down by the first list left sufficient room for maneuver in the second list to “cook” a second list no less successful than the first list.”
        This claim is based on another preliminary claim:

C1.  MBBK claim: We did "the same thing" in War and Peace.

MBBK assert that they exploited the flexibility available in the selection of appellations to “cook” a list which succeeded in War and Peace. This supposedly proves that the “wiggle room” in the selection of appellations was “more than enough” to produce an artificially strong result for WRR’s second list.
        Actually, they prepared two lists. Bar-Natan and McKay publicized their first list (BM1) on the Internet in Sept. '97. Following our sharp criticism, they altered it and created a second list (BM2) which is presented in the final version of their article “ELSs in Tolstoy's War and Peace" on the Internet, and in their paper in Stat. Sc.

The refutation:

Their claim that they did “the same thing” as us in "War and Peace" is nonsense. Because few of their readers have enough knowledge in Hebrew or rabbinical bibliography they create a smokescreen to fool them. Let's clear away the smoke:  

(1)   MBBK claim that list BM1 is not much different than Havlin's list ("83 appellations were left unchanged, 20 were removed, and 29 were added") and that their appellations were bibliographically correct and certified by Prof. Cohen.

But:
a.         MBBK's numbers are deceiving. They say that only 29 out of 112 (83+29) appellations in their list are new, that is 25%. But the correct figure is 42%. This is also the percentage of the new "appellation-date" pairs in their sample: They erased 48 of 124 of the original pairs, and added 55 new pairs. For details, click here.
b.         We have explained how their work is based on absurd mistakes in Hebrew and rabbinical bibliography, and on deliberate deception. We refer the reader to our article: “A Refutation Refuted, or: How the List of Famous Rabbis Failed in War and Peace”:
Part 1 [related documents: document1, document2, document3.]
Part 2  [related document: document4.]
c.         Concerning Cohen and their method of comparison (“the same thing”) see later sections (3) and (4).  

(2)        After our pointed criticisms Bar-Natan and McKay moved to list BM2. They claim that this involved only a "small number of changes" and that "hardly any of the small number of changes" was due to our criticism on BM1.

But:
a.         This list is significantly different than BM1 (23 out of 131 pairs are erased, and 31 new pairs are added).
b.         Contrary to their claim, 11 changes to appellations are in direct response to our criticism, to avoid the most obvious mistakes. This was a serious blow to the significance of their results and forced them to fix up their list by making the rest of the changes.
c.         Because changes of appellations did not allow strong enough results for BM2 through manipulation of appellations alone, they allowed themselves to exploit “flexibility” in domains where WRR certainly used none and contrary to their claim that their list came to decide the following question:

“Was the flexibility available in the selection of appellations at the time the lists were prepared sufficient that biased selection could produce a strong result?”

For details, click here.

(3)        Even MBBK realized that the changes they made in Havlin’s list were worthless without the confirmation of an expert in rabbinical bibliography. Therefore they rely on Professor Menachem Cohen.

But:
a.         Prof. Cohen is not an expert in rabbinical bibliography.
Professor Cohen is an expert on the Bible, and not on rabbinical bibliography, the relevant field. Nevertheless, MBBK present him as an expert in this field, and this is how he is presented in their article in Chance: “Menachem Cohen, a colleague of Havlin’s from the Faculty of Jewish Studies at Bar-Ilan University”.
        The following parable may clarify the point. Suppose a statement of opinion of a chemist is used to invalidate a work in the physics of elementary particles. The chemist is presented as “an expert”, “a colleague” of the physicist who did the work, since he belongs to the same Faculty of Natural Science! No doubt this is a misleading way of presentation.
        The same thing is done by MBBK in our case. Professor Cohen is expert in the accuracy of Biblical texts, but not in rabbinical bibliography which is a completely different field. Since the subject here is outside his field, how can his opinion be presented as “an expert opinion”?
        In conclusion MBBK have no “expert opinion” to rely on. In contrast, Professor Havlin has an international reputation in rabbinical bibliography and has authored hundreds of publications in the field.
b.         Besides Cohen not being an expert, he seems to have never thoroughly investigated BM1 at all: The most startling example is that he finds that BM1 “was prepared according to the same criteria as presented by Prof. Havlin”. This finding contradicts MBBK’s own assertion that they deviated from these criteria and ignores their report of some of these deviations.
c.         Because of our criticism, MBBK "cooked" list BM2 which is considerably different than BM1 (see section (2)). BM2 was publicized in the final version of their article “ELSs in Tolstoy's 'War and Peace’" on the Internet and in their Stat. Sc. Paper.
But BM2 has no professional confirmation, even from Cohen. For details, click here.
d.         Note that in Galileo MBB write that Cohen participated in preparing BM1. This is a gross deception. For details, click here.

(4)        MBBK claim that they did "the same thing" as WRR. What does this mean? They write that they worked within the framework of rules (that is the linguistic rules and Havlin's rules) or broke them "to the same extent" as WRR. They rely on Cohen who wrote that their list indeed complied with Havlin's rules.

But:
a.         The requested letter from Cohen suffered a "small error": Even though MBBK admit that they broke Havlin's rules (and give examples) Cohen generously testifies that their list “was prepared according to the same criteria as presented by Prof. Havlin”!
b.         Perhaps this is why they leave out this problematic sentence whenever they quote Cohen (in Chance and Stat. Sc.).
c.         The actual technique used to ensure that BM1 and BM2 were "the same" as Havlin's original list lacks any scientific merit, and enables deceit. For details, click here.
d.         They could not receive a confirmation from Cohen that they "broke the rules to the same extent…" since such a confirmation requires quantitative and detailed examination, which Cohen never did.
e.         Havlin wrote to Cohen in Jan 2000 informing him that MBBK were using his letter in a misleading way. He asked Cohen to either make clear that he never did the required quantitative and detailed examination, or, alternatively publicize his findings. But Cohen has never bothered to reply. For details, click here.

In conclusion: MBBK have no professional confirmation for their central claim that they did “the same thing”.

(5)        We have experts’ opinions which confirm Havlin's rules and list, disqualifies MBBK's list and rejects the assertions of Cohen and MBBK. For details, click here.

We refer the reader to our article Of Science and Parody: A Complete Refutation of MBBK’s Central Claim which factually and logically destroys their claim that they did "the same", without discussing rabbinical bibliography or linguistic issues.

(6)        Early in the controversy we suggested (and repeated it in our response in Galileo and elsewhere) a simple scientific test to check MBBK’s claim that WRR’s success was solely created by exploitation of the flexibility of the rules or by breaking them. The suggestion was to jointly appoint an independent expert to prepare a new list of names and appellations for the rabbis of the second list (L2), using the same rules, i.e., the linguistic and Havlin's rules. If this new list succeeded, it would prove that the original success was neither due to “flexibility” of the rules nor to “deviations” from them.
a.         But MBBK have not picked up the gauntlet.
In Sept. 2000 we offered a million dollar bet that such a list would perform better in Genesis than in War and Peace. This suggestion was the headline of a full article in the weekend (Sukkoth) supplement of the most popular paper in Israel (Yedioth Acharonoth).  The same article quoted Prof. Bar Hillel's response: "Why should anyone want to do that foolish experiment?" (Emphasis ours).
b.         Now it became obvious why they never agreed. We wrote before (in our refutation of MBBK's second claim) that the names and appellations for L2 prepared by Dr Emanuel, the expert engaged and guided unilaterally by MBBK, performed successfully in Genesis. We may add here that his names and appellations fail in War and Peace. Because MBBK probably realized that an independently prepared list would do even better, they chose:

• To not answer our challenge.
• To conceal the true significance of Dr Emanuel's data. For details, click here.
• To forge another list in his name. For details, click here.

 [Update: Our paper Of Science and Parody: A Complete Refutation of MBBK’s Central Claim forced McKay to respond. We responded to this, showing that his response included not even one scientific answer to what we had said. He once again tried to distract people's minds from our paper's conclusion and our first response. In a reply, we also rebuffed this attempt.]

(7)        Concerning MBBK’s claim that WRR directly optimized the results by exploiting “beneficial” choices pertaining to the dates: In our article Concerning the Choices of Dates for WRR’s Rabbis Samples we show that careful examination of all the choices indicates WRR’s perfect integrity. Alternative choices, based on MBBK’s suggestions, would have yielded better results - sometimes by a factor of 2 or 3, sometimes by a factor of 10 or 100, and sometimes by a factor of tens of thousands. For details, click here.
All this starkly contradicts MBBK’s report and the impression created by their article.

[Update: Once again, McKay sets up "straw men" in place of our actual arguments and storms against them… We discuss this in our response.]

D.  MBBK claim: There are serious flaws in the measuring method.

MBBK thereby assert that the statistical results derived are therefore without value.

The refutation:

a.         We are of the opinion that the results do indeed reflect true statistical significance.
b.         Note that this methodology (using the permutation test) was proposed not by WRR, but by Prof. Diaconis. The details of the test were written down and submitted to several well-known statisticians (including Diaconis) for approval before the experiment was conducted. For details, click here.
c.         Nevertheless, in response to MBBK’s criticisms a new test was run on the second list (L2) in a manner that addresses their concerns. The results merely improved. For details, click here.

E.  MBBK claim - "The Textual Argument": The Book of Genesis accepted as kasher nowadays cannot have any codes.

The argument is that the text of Genesis accepted as kasher nowadays and used by WRR for their experiment, is not the same as the original text received at Mount Sinai (at least regarding the spelling of its words). The many differences would destroy any encoding that may have existed in the original text.

The refutation:

MBBK argument is "according to your opinion:" "According to your belief that a code exists in Genesis, how can it be present in the modern text of Genesis?"

They assume that:

1. If there is a code in Genesis it will be found only in the original text given at Sinai.
2. The Torah text considered kasher today,is not identical with that given at Sinai (at least not the spelling). According to them, our text contains many spelling errors which should suffice to destroy any code that existed in the original Genesis text.
   
   Then they claim that assumptions a and b lead to an inevitable conclusion:
3. No code could possibly be found in the Genesis text accepted as kasher by the Jews today.

 But:
The "textual argument" is a classical case of foolishness masquerading as science. Someone, it seems, forgot they we are discussing a system of codes that include information about the future, created by He who knows future events in advance and personally supervises and oversees them.

Is it conceivable that the code-maker knew of every future event - except which text would be accepted as kasher (valid) by the code-receivers when the code was revealed?

For more details, click here.

F.  MBBK claim: WRR broke their agreement with Prof. Diaconis concerning the permutation test.

MBBK claim that WRR used a permutation test that differed from that agreed upon with Prof. Diaconis, and this was done behind his back.

The refutation:

a.     This claim is absolutely false. To read our response to this outrageous claim, click here.
b.     When we published our response, MBBK refused to admit their mistake. Instead McKay and Kalai publicized an article on the Internet where they unsuccessfully attempted to conceal the fact that they had been caught red-handed. Their article is replete with untruths and suppression of facts. To read our response click here.

[Update: The book Tzofen Bereishith precisely describes the process of how WRR paper was publicized. This, and especially its photos of the relevant letters exchanged between Professor Aumann and Professor Diaconis, decisively disprove the ridiculous claim of MBBK.] 

G.  MBBK claim: WRR's further experiments suffer from the same "problems".

This claim is written in the introduction to their paper and in chap. 9 there.

The refutation:

(1)        WRR conducted an experiment known as "The Nations Sample". MBBK misleadingly describe it in section 9 of their paper with the subheading "The 70 Nations Experiment”. Here too they claim that the WRR’s results are due to an exploitation of “wiggle room” which existed in the data. In addition they describe (in sec. 10, pg. 165 example #3) a similar experiment conducted by BMS (Bar-Natan, McKay and Sternberg) on War and peace. Their experiment includes an alternative list to that of WRR for the "Seventy Nations". BMS claim that their list was compiled according to the same guidelines and constraints as WRR's, and succeeded even better.

But:
a.         We refer the reader to our two-part paper The Nations Sample (Part A, Part B). One will find a complete response to all their assertions and see that they are nonsense.
b.         They “demonstrate” the “wiggle room” in the choice of data in a very distorted way, while concealing the results of much simpler choices (see Part A of our paper).
c.         Their list for the "Seventy Nations" is based on deception. (See Part B of our paper).
d.         We used BMS's data to make some experiments of our own. A statistical test which uses the list of data prepared by BMS, shows that the true results for their text are exactly as expected to occur randomly, whereas in Genesis there is a real success with high significance! (See Part B of our paper).
        Until this day they have given no reply for this. 

(2)        About eight months before the printing of MBBK's paper in Stat. Sc, we gave a lecture at the "Center for the Study of Rationality" in the Hebrew University of Jerusalem, in MBBK's presence. This lecture included two new studies (which have also been publicized on the Internet), both conducted to avoid all of MBBK's past objections about selection of data. Both studies yielded highly significant results. Despite this, no reader of MBBK's paper would have any idea that these two studies exist because they are totally ignored. Almost ten years later we have still heard nothing from them. These two studies are described in:

• Personalities of Genesis and Their Dates of Birth.
• A Replication of the Second Sample of Famous Rabbinical Personalities.

[Update: The first study was published. Click here to read it.]
After reading these two papers one will understand why MBBK prefers to ignore them, and also realize the worth of their statement (pg. 152):
       “Nothing we have chosen to omit tells a story contrary to the story here.”

 

Conclusion:

We have destroyed the main claims of MBBK one by one. We have also shown that they have utilized illicit methods involving lies and deception. We have shown that their criticism lacks any basis:
a.      Their fairy tales about the huge “wiggle room” exploited by WRR to produce successes are nonsense and their examples of “wiggle room” were almost always based on distortions and deception.
b.      We have even shown that in many cases their own alternatives yield even better results, thus providing more evidence that the original experiment of WRR was done honestly.
c.      Their “Study of Variations”, MBBK’s statistical “proof” that WRR “cooked” the list of names and appellations, is nothing but deceit and “cooking” of variations.
d.      Measurements that we made on data collected by MBBK, pertaining to the second list of personalities and the “Seventy Nations”, have proven that the Torah Codes is a true phenomenon.