BS"D
17 Elul 5759 (29 August 99)

To read Hebrew words in this text, please use (ISO – Visual) Hebrew

 

The Nations Sample

By Doron Witztum




Part II: The Range of Stories

Introduction
I. The Hierarchy of Stories
II. The Great "Proof" of Deception
III. Is the Significance They Report Accurate?
IV. Did Their Principal Story Really Succeed?
V. Without Any Stories

Appendix to Part II
  1. The list of 136 prefixes
  2. Do BMS know how to contrive a story?
  3. Concerning RPWL as the preferred measuring device
  4. BMS' main story
  5. The best quartet
Acknowledgements
Bibliography and Notes

Introduction:
        BMS [1] assert that they investigated 136 different prefixes in Genesis (including the four prefixes investigated by us in our original experiment. Their list can be found in the Appendix, sec. 1). They report: "Amazingly, three of the four forms used by WRR were the best three in our list of 136!" They write: "This is a rather improbable result; we believe it suggests that Witztum, Rips and Rosenberg cheated and tried many possible prefixes before deciding which four to use." They claim that we actually investigated a large number of possible prefixes and only afterwards concocted a story to justify our use of some of the most successful ones.

1.        According to BMS's model [2], for any given text and for any list of prefixes it would be possible to fabricate a plausible explanation of why the most successful ones were chosen. As proof of this they cite the fact that they were able to contrive an explanation for the four most successful prefixes in WGP (i.e., four of the 136 mentioned above), and to achieve a result of 5/108 in the permutation test! This experiment shows, so they claim, that contriving rationalizations is a simple matter because there are so many potential stories to choose from.
        In this part of the article we will examine their assertions one at a time:
1. Are all explanatory stories really equal?
2. Does the success of our prefixes indeed indicate deception?
3. Is the significance they claim to have achieved accurate?
4. Did their principal story really succeed?

In Chapters I, II, III, and IV we will see that the answer to all four questions is "NO," and we will also see why, in our opinion, their assertions are without any foundation.

2.        It also turns out that in the course of compiling their lists of prefixes BMS made a great contribution towards proving that the success of our prefixes does indeed stem from a unique phenomenon in the book of Genesis. An analysis of their list of prefixes shows that there is a significant difference between the success of the prefixes in Genesis versus that of their control text, WGP. In Chapter V we will examine the success of the "best quartet" in Genesis versus that of WGP. We will see that the success of the "best quartet" in WGP is just what one would expect to receive by chance, while the best quartet in Genesis is a case of genuine success. The probability of receiving a similar result by chance would be 0.00042!

I. The Hierarchy of Stories:
        BMS claim that for any four words one could contrive a likely story, and that all explanatory stories are of equal status. This is sheer nonsense. The definition of nationhood (which our explanation revolved around) has been a serious topic of discussion for thousands of years in many different cultures. In the Introduction to Part I of this article I presented excerpts from encyclopedias, dictionaries and traditional sources dealing with this subject. This was not a "contrived" story; but a fundamental discussion of the defining characteristics of nationhood. All other " stories" about aspects of nationality are about issues that are peripheral and secondary to the essential concept.
        In BMS's list of prefixes one can find terms such as "dance" and "way," etc. Whoever tries to claim that "dance" and "way" define nationhood, or form the basis of a story of the same rank, is assuming that the reader is lacking in sense.

II. The Great "Proof" of Deception:
        BMS claim to have found a tremendous proof that WRR cheated. What is their "proof"? They claim that three out of four of the prefixes we selected were the most successful of the 136 prefixes they examined. Their assertion is erroneous on two accounts:
1.        The data that BMS present to the reader is misleading. The measures that are relevant here are the ranks of P1 and P2 in the permutation test – that is, r1 and r2 respectively. These are the only measures of significance we used in the original experiment on the "Nations Sample." (Concerning the "story" BMS told to justify presenting irrelevant data – see the Appendix, sec. 2). Therefore, if BMS chose to assign rankings according to the statistic P2, they should have done so using r2. Out of the 136 prefixes examined, the rankings of our prefixes using r2 were: 1, 3, 9 and 16 (according to BMS's own calculations [3]). According to this data our prefixes enjoyed quite a moderate success.
2.        Note that BMS's "proof" is based on the assumption that there is no real phenomenon. Therefore all prefixes should be of equal significance (or lack thereof). This assumption is clearly a logical error: our research hypothesis in fact predicts greater success for these four prefixes.
        This is the height of absurdity! They label a success predicted by the research hypothesis as proof of deception!

III. Is the Significance They Report Accurate?
        BMS report a significance of r2 = 5/108 for the four prefixes they selected. However, by their own admission this is inaccurate. In their article BMS criticized the use of the original randomization test for the Nations Sample. They claimed that the results were meaningless for several reasons particular to this experiment [4]. These objections do not apply to the RPWL test (Randomization by Permutations of Words' Letters), see Part I and the Appendix there. For further details see the Appendix here, sec. 3. For this reason, RPWL is the preferred method in this case.
Using the RPWL test the results for BMS's four prefixes (using 100,000,000 permutations) are:
r1(BMS) = 6.67 X 10-5, r2(BMS) = 1.22 X 10-4.
Changing the test produced an enormous difference in the results for r2 in WGP. In Genesis, by contrast, the difference in the results using the two measurements is small – less that one order of magnitude (see Part I).
BMS found their four prefixes through acknowledged optimization using the original permutation test. In Chapter V we will see what happens when optimization is conducted using the RPWL test.

IV. Did Their Principal Story Really Succeed?
        No! As it turns out, the story they concocted for the four most successful prefixes in WGP is actually a failure – despite their claims to the contrary. The illusory success of their story is entirely based on the omission of a number of prefixes, which BMS neglected to mention. However, the neglected prefixes appear in the source text upon which BMS claim to have based themselves (see the Appendix,sec. 4).
Therefore, there is no story; there is only an arbitrary selection of prefixes, nothing more. It turns out to be not as simple as BMS imagined to contrive a story that works, even if one permits himself to consider all stories of equal status and to ignore the fundamental issue of hierarchy.

V. Without Any Stories:
1.        As we mentioned above, BMS published a list of 136 prefixes. One reader of their article, Yosef Beremez, proposed investigating how the "best quartet" out of this list succeeds in Genesis, versus the "best quartet" in WGP, thus obviating the need for any explanation at all.
        In order to compile the "best quartet" for any text T, one must rank all the prefixes in BMS's list twice: once according to the value P1, and again according to the value P2 (prefix with lower Pi-value gets higher ranking). We are interested in the four prefixes that appear at the head of each of the two lists. To determine which is the best quartet of the two, we compare the P1-value for the union of the "P1 four top prefixes", with the P2-value for the union of the "P2 four top prefixes" . The set with the lowest value is the "best quartet."
        As it happens, the P1 quartet was the lower one both in Genesis and in WGP. The best quartet in Genesis, as determined by this procedure, includes the prefixes:
םע , בתכ , בכר , תודש ("nation of," "script of," "chariot of," and "fields of").
        The best quartet in WGP includes the prefixes: יהלא , ימד , רש , and ללמ ("god of," "blood of," "prince of," "speech of").
        The significance of the statistic P1 was determined for each of the two quartets using the RPWL test. The results were as follows:
For the best quartet in Genesis: r1(G) = 4.0 X 10-10
Whereas for the best quartet in WGP: r1(WGP) = 6.16 X 10-6
        The enormous difference between these two results compels us to evaluate the probability of receiving such a small value for r1(G). This can be achieved by means of a simulation in which we compare this result with the result randomly expected from an experiment involving the list of 136 prefixes in Genesis. It turns out that the probability of receiving such a low value of r1(G) is 0.00042.
        Details on the measurements referred to in this section can be found in the Appendix, sec. 5.
2.        BMS made experiments (using the permutation test) [4], in order to compare the distribution of the 136 prefixes in Genesis, with their distribution in 10 similar texts. It is clear from their data that the results for Genesis are significantly exceptional. Details can be found in the Appendix, sec. 3.1-2.




Appendix to Part II

1. The list of 136 prefixes:
        Here is the list of 136 prefixes as published by BMS in Table 2 of their article. The prefixes are arranged alphabetically:

1. הקירה תמודיקה – the empty prefix. 2. בא - father of, progenitor of . 3. יבא – father of, progenitor of. 4. תמדא – land of . 5. לכוא – food of . 6. תמוא - nation of 7. יפוא – character of 8. תזחא - + estate of, property of. 9. יא – isle of, see Genesis 10-5. 10. ייא – isles of, see Genesis 10-5 . 11. ליא – lord of, magnate of. 12. יליא – plural of ליא . 13. שיא – person of, man of 14. לכא – food of. 15. לא – god of. 16. הלא – god of, celestial power of . 17. יהלא – plural of הלא . 18. ףולא – head of, leader of. 19. לילא – idol of, false god of. 20. ףלא – thousand of, family of, part of the tribe of. 21. יפלא – plural of ףלא . 22. תמא - + nation of. 23. ישנא – people of , men of. 24. ץרא – country of. 25. םשא – guilt of. 26. תמשא – guilt of. 27. תשא - + woman of. 28. דגב – cloth of, dress of. 29. ידגב – plural of דגב . 30. תריב – capital of. 31. תיב - + house of, family of dynasty of. 32. ןב - + son of, native of. 33. תונב – + plural of תב . 34. ינב – ןב plural of. 35. תב - daughter of, female native of + 36. יתב – plural of תיב . 37. לובג – border of. 38. רובג – hero of. 39. רבג – man of, hero of. 40. ירבג - plural of רבג . 41. דודג – battalion of. 42. יוג – nation of. 43. עזג – race of. 44. רג – convert of. 45. ירג – רג plural of. 46. רובד – speech of. 47. לגד – flag of. 48. ילגד – plural of לגד . 49. ןיד - justice of. 50. יניד – plural of ןיד . 51. םד – + blood of 52. ימד – plural of םד . 53. ךרד – way of. 54. יכרד – plural of ךרד . 55. תד – religion of 56. יתד – plural of תד . 57. בהז – gold of. 58. תוכז – privilege of. 59. ןמז – time of, period of, fate of. 60. רמז - song of, melody of. 61. תרמז – variation of רמז . 62. ערז – seed of, offspring of. 63. לבח - region of, district of. 64. אטח – sin of. 65. יאטח - plural of אטח . 66. ליח - army of, corps of. 67. יליח – plural of ליח . 68. קח - law of. 69. יקח – plural of קח . 70. ברח – sword of. 71. עבט – nature of, character of. 72. ןהכ - priest of. 73. ינהכ - plural of ןהכ . 74. חוכ – alternative spelling of חכ . 75. חכ - force of, vigour of. 76 . תוחכ - plural of חכ . 77. ףסכ – money of. 78. יפסכ – plural of ףסכ . 79. בתכ – script of. 80. ביתכ .- orthography of 81. םאל - nation of. 82. שובל - dress of, clothing of 83. ןושל – dialect of. 84. ןשל – dialect of 85. לכאמ – food of, diet of. 86. בשומ – place of, seat of. 87. לזמ – luck, destiny, constellation of. 88. עבטמ – coin of. 89. בתכמ – letter of, writing of. 90. ךאלמ – angel of. 91. ךלמ – king of. 92. יכלמ – plural of ךלמ . 93. תכלמ – queen of. 94. ללמ – speech of. 95. לשממ – rule of, government of. 96. גהנמ – manner of, custom of. 97. סמ – tax of, levy of. 98. יסמ – plural of סמ . 99. תועמ – money of 100. םוקמ – place of. 101. הארמ – appearance of, view of. 102. סומנ - + manners of, custom of, laws of. 103. סנ – flag of, banner of, miracle of. 104. יסנ – plural of סנ . 105. תושנ – + plural of תשא . 106. אישנ – president of. 107. ףוס – end of, conclusion of. 108. למס – symbol of, emblem of. 109. ילמס – plural of למס . 110. רפס - book of. 111. ןווע – alternative spelling of ןוע . 112. ןוע – sin of. 113. לש ז"ע – idolatry of . 114. ריע – city of, angel of. 115. םע – nation of. 116. ירע – towns of. 117. עשפ – crime of. 118. יעשפ - plural of עשפ . 119. אצאצ – child of. 120. אבצ – army of, troops of. 121. שאר - head of, leader of. 122. ישאר – plural of שאר . 123. חור – spirit of. 124. בכר – chariot of 125. דוקר – dance of 126. הדש – + field of 127. תודש – + plural of הדש 128. ריש – song of, hymn of, music of. 129. יריש – plural of ריש . 130. םש – + name of, fame of 131. תומש - + plural of םש . 132. תפש – language of. 133. רש – ruler of, head of, minister of, prince of. 134. ירש – plural of רש . 135. שרש – root of 136. בשות – resident of.
Note – I have reproduced BMS's list here verbatim (including the mistakes).         

2. Do BMS know how to contrive a story?
        And how! As it turns out, BMS have a special talent for concocting stories. Two of them (BM) were co-authors of an article [5] describing how (almost) all of the choices we made in our original "Famous Rabbis" experiment were biased in our favor. One of the obvious flaws in their analysis was their choice of statistical tests. They chose to check the bias in our choices by calculating a statistic that was only conceived two years after the original experiment was performed (i.e., the rank in the permutation test) instead of calculating the measure of success that was actually used at the time of the experiment! When faced with criticism over this [6] they "concocted" a story [7] to "explain" post hoc why they did something so peculiar. (The reader may have already guessed that had they used the more relevant statistic, their analysis would have in fact demonstrated the sincerity of our work).
        In their article concerning the Nations Sample BMS demonstrate their impressive acrobatic ability. They managed to do precisely the reverse of what they did in the article mentioned above. The experiment on the Nations Sample really was conducted using the permutation test (which had already been applied by then to the second list of Famous Rabbis). Yet this time they chose to base their investigation of bias using the old measure, the one that was used in the original work on the Famous Rabbis sample! Does this seem strange to you? Don't worry – they have a story to explain this as well....
        They claim [8] that we could not have used the permutation test for our alleged optimization because we did not have sufficient computing facilities. They support this claim by noting that in our original article on the Nations Sample we write that we were unable to perform the billion permutations necessary for calculating the ranking of P1.
        One cannot deny that this story is very cute, but does it reflect reality? Unfortunately it does not. They did indeed quote us accurately, but they left out a minor detail that BMS should be made aware of [9]: The program for calculating the rankings of P1 was 45 times slower that the one for reckoning the P2 rankings. It was because of this slowness that we decided to forgo calculating the ranking of P1 out of a billion permutations. But regarding the much quicker program for establishing the ranking of P2, the statistic that BMS claims we were unable to calculate, there was no technical obstacle to the running of repeated experiments of this type (had it occurred to us to do anything so stupid). Therefore this is the test that BMS should have used were they genuinely looking for evidence of bias.
        As I mentioned earlier, when it comes to concocting stories, Scheherazade did not come up to their ankles.

3. Concerning RPWL as the preferred measuring device:
In part I of this article and in the Appendix there, I brought the necessary background for RPWL method and described how it was applied to the Nations Sample. At this point I'm not going to argue with the speculations that BMS raised in order to undermine the original permutation test. Partly because they raised peculiar ideas (for instance, they wrote that there is a dependency between the ELSs of "רמג םע" and the ELSs of "רמג בתכ" , etc.) and partly because they admit to having no true explanations. The only issue that may be taken seriously (if their data is correct) is their claim of making certain experiments, and getting results that indicate certain anomaly in the distribution of the rankings in the original permutation test.
1. BMS' data concerning the original permutation test:
         BMS report in Section 7.1, that they investigated the distribution of the rankings of the 136 prefixes in Genesis. The 136 rank orders were put in 10 bins 0,1,….,9, where bin i contains permutation rank orders 100000i+1 through 100000(i+1) out of a million. They observed "non-uniformity" and "an excessive number of ranks in the smallest bin", and that the minimum "appear to be smaller than expected". To understand the situation better, they created texts equal in length to Genesis, that were supposedly very similar to Genesis. BMS report:
"To investigate whether non-uniformity and exaggerated extremes are the norm, we ran the same tests on 10 control texts. We generated each control text by randomly permuting the order of the words within each verse of Genesis except for the verses in Chapter 10. Chapter 10, where all nation names are found in a specific fixed order, was left alone in order to make the comparison with Genesis and WGP more meaningful." To ensure independence of the results, ELSs lying entirely inside the section of the text, containing Chapter 10 of Genesis were ignored.
        The results for r2 are tabulated below. Text G is Genesis. The first ten rows show the numbers of prefixes for which the rank order lies in each of bins 0,1,...,9. The last three rows show the smallest, average, and maximum rank orders out of a million.
Bin G Text 0 Text 1 Text 2 Text 3 Text 4 Text 5 Text 6 Text 7 Text 8 Text 9
0 23 15 19 22 23 18 17 14 13 19 19
1 17 13 13 8 15 12 6 13 18 13 18
2 9 17 13 16 15 8 18 13 15 11 13
3 13 13 14 7 9 7 16 18 10 12 11
4 11 9 7 13 7 10 12 9 13 7 7
5 12 16 11 13 12 16 5 12 8 10 8
6 10 10 7 9 22 22 10 8 13 18 10
7 14 9 13 20 10 9 13 11 15 16 10
8 16 13 15 15 14 14 23 15 13 11 13
9 11 21 24 13 9 20 16 22 18 19 27
min 493 5314 3991 1584 608 874 4979 180 6184 6247 2481
mean 463125 504905 517197 496493 458654 534142 519963 522828 505910 512851 514246
max 990921 999807 999923 999141 997583 999864 999849 999229 997025 991129 998778
BMS conclude:
"The most obvious characteristic of these distributions is their inconsistency. A few are near-uniform, but others are skewed markedly in the positive or negative direction. Both the minima and maxima appear exaggerated.
Several things should be clear from these results. Firstly, WRR's assumption of uniformity in the rank orders is unfounded, as many of the texts give profoundly non-uniform distributions. More importantly, the non-uniformity may be more pronounced at the extremes of the distribution where WRR measure their 'significance levels'."

2. Comment on BMS' Data:
        BMS arrived at the conclusion that "the non-uniformity may be more pronounced at the extremes of the distribution", both by checking occupation numbers of bins 0 and 9, and by measuring the minima and maxima.
        Looking at the same table and checking the tendency to low ranks at the extremes, we can't escape the conclusion that the data for Genesis are exceptional. It is pronounced in each of the following parameters:
These exceptional results for Genesis complete the picture we already have from the measurements described in Chapter V above.

3. Data concerning the RPWL test:
         In order to make a similar experiment using RPWL method, we created 10 texts following the description of BMS. To save time, (this method needs much more computations), we first checked the extremes. We checked bin 0 and bin 9; (the occupation numbers were found using the extreme 30 prefixes at each edge). We also measured the minima and maxima for each text. The results for r1 are tabulated below.
Bin Text 0 Text 1 Text 2 Text 3 Text 4 Text 5 Text 6 Text 7 Text 8 Text 9
0 13 14 8 5 13 12 15 14 14 9
9 10 16 20 8 11 13 12 11 16 13
min 2177 16145 3064 34204 14822 2867 5334 2984 7771 13605
max 985212 998915 992263 977197 998763 996302 997474 980781 997363 994129

The results from a similar experiment done for r2 are tabulated below:
Bin Text 0 Text 1 Text 2 Text 3 Text 4 Text 5 Text 6 Text 7 Text 8 Text 9
0 14 17 14 9 12 15 12 10 10 11
9 12 24 15 7 14 13 13 12 13 15
min 5112 17038 7705 21471 11176 6775 1076 1659 10225 10626
max 984823 996099 998372 999790 999204 997275 995230 989584 995961 998581
We can clearly conclude from these tables, that by using RPWL method we have in the control texts.

4. BMS's main story:
      BMS's main story, " Celestial Guardians" [10] is based on Nachmanides commentary to Chapter 18 of Leviticus. There, as BMS put it, "he discusses the celestial beings who represent and supervise the Nations of the world".
  1. BMS claim that in the course of this discussion Nachmanides uses the descriptive terms: רש (prince), יכלמ (kings of), םיהלא (gods), and ןיריע (angels).
  2. BMS contrived a set of "guidelines," which they claim leads to the selection of these four prefixes: 1. יכלמ , 2. רש , 3. ריע , 4. יהלא .
        By examining the relevant passage we can see right away that:
  1. Nachmanides actually used a much broader range of descriptive terms, of which BMS arbitrarily selected four.
  2. Their supposed "guidelines" are a nothing more than a joke.
        Therefore, they have no explanatory story; just an arbitrary choice of prefixes.

1. Nachmanides descriptive terms:
        We will present here two excerpts from Nachmanides commentary (Leviticus 18:25), and we will highlight the relevant terms:
אמטתש ץראה רובעב ,תוירעב בותכה רימחה - "ץראה איקתו הילע הנוע דוקפאו ץראה אמטתו"
םירבד) בותכב רבדה דוס לבא ,ץראב תויולת ןניאו ףוגה תבוח תוירעה הנהו ,תושועה תושפנה איקתו ןהב
םשה יכ ןינעהו ."'וגו ומע 'ה קלח יכ 'וגו םימע תולובג בצי םדא ינב ודירפהב םיוג ןוילע לחנהב" (ט-ח ,בל
רשאכ עודי לזמו בכוכ םהייוגל םתוצראב םעו םע לכ לע ןתנו ,םינוילעב םינותחתה חכ םשו ,לכה ארב דבכנה
םלוכל קלח יכ ,"םימעה לכל םתוא ךיהלא 'ה קלח רשא" (טי ,ד םירבד) רמאנש והזו .תונינגטצאב עדונ
רשו" (גי ,י לאינד) בותכש ןינעכ ,םהילע םירש םתויהל םנתנ ןוילע יכאלמ םהילע םיהובגו ,םימשב תולזמ
(גי קוספ ,םש) ביתכדכ םיכלמ םיארקנו ,"אב ןוי רש הנהו" (כ קוספ ,םש) ביתכו ,ידגנל דמוע סרפ תוכלמ
לבא ,םלועה לכל םינודאה ינודאו םיהלאה יהלא אוה דבכנה םשה הנהו ."סרפ יכלמ לצא םש יתרתונ ינאו"
לשומו רטוש ןיצק םיכאלמה ןמ הילע ןתנ אל ,ומשל תדחוימ 'ה תלחנ איה בושיה תועצמא לארשי ץרא
יל יכ םימעה לכמ הלוגס יל םתייהו" (ה ,טי תומש) רמאש והזו ,ויבהוא ערז ומש דחימה ומעל התוא וליחנהב
םיהלא לא םתא ויהתש אל ,"םיהלאל םכל היהא יכנאו םעל יל םתייהו" (ד ,אי הימרי) ביתכו ,"ץראה לכ
ןלהל) רמא ךכלו ,ומשל םתויהל תוצמה יבורבו תוירעה תשודקב וצראב בשויה םעה שדיק הנהו .ללכ םירחא
קוספ ,םש) ביתכו ,"ץראה םכתא איקת אלו םתוא םתישעו יטפשמ לכ תאו יתוקוח לכ תא םתרמשו" (בכ ,כ
יתלדבה רשא םכיהלא 'ה ינא התוא תשרל םכל הננתא ינאו םתמדא תא ושרית םתא םכל רמאו" (דכ
תא ונל ותתב ,םירחא םיהלאו םירש םהילע ןתנ רשא םימעה לכמ ונתוא לידבה יכ רמאי "םימעה ןמ םכתא
לכ איקת דבכנה םשה תלחנ איהש ץראה הנהו .ומשל םידחוימ היהנו םיהלאל ונל ךרבתי אוה היהיש ץראה
םע הרז הדובע לולכל ךלומה הריכזה תאזה השרפהו .תוירע םילגמו ז"ע ידבוע לובסת אלו התוא אמטמ
ןכו ,"היבשוי תא ץראה איקתו ,םיוגה ואמטנ הלא לכב יכ הלא לכב ואמטת לא" רמא םלוכ לעו .תוירעה ןורכז
ךכלו ,הרז הדובע רוסיא רמוח אוהש ,"יל תויהל םימעה ןמ םכתא לידבאו" (וכ ,כ ןלהל) הינשה השרפב רמא
םתא םכל רמואו" (דכ קוספ ,םש) רמאנש ,ץראה םהל ןתנ ןכ רובעב ומשל םידחוימ םהש ינפמ יכ רמא
הנהו ."םימעה ןמ םכתא יתלדבה רשא םכיהלא 'ה ינא התוא תשרל םכל הננתא ינאו םתמדא תא ושרית
םימעהו הילע םילשומה םיתרשמה רובעב ,המילש הב הרהט ןיא ,דבכנה םשל לכהש פ"עא ,ץראל הצוחב
יהלא אוה יכ ,"ארקי ץראה לכ יהלא" (ה ,דנ היעשי) בותכה רמאי ךכלו .םתוא םג דובעל םהירש ירחא םיעות
סורהלו םינוילעה תלשממ ריסהל "םורמב םורמה אבצ לע" ףוסב דוקפי אוהו לכה לע לשומה םיהלאה
(די ,ד לאינד) רמאש בותכה ןינע והזו ."המדאב המדאה יכלמ" לע דוקפי ןכ ירחאו ,םיתרשמה תכרעמ
ןיריע תרזג איה רצנדכובנ לע רזגנה אוהה רבדה יכ רמאי ."אתלאש ןישידק רמאמו אמגתפ ןיריע תרזגב"
וררועתי ןתוליצאמ יכ ןיריע וארקיו ,ךכ תושעל ןהמ ןילצאנה תוחוכה לע ורזגש אתלאש ןישידק רמאמו אמגתפ
ודוג רמא ןכו ליחב ארק תיחנ אימש ןמ שידקו ריע ולאו" (אי י ,םש ,םש) רמאש ומכ ,תולועפה לכב תוחכה
והזו ,ןכ תושעהל ורזג ןכ ירחאו וילע ןוילעה ןוצרה המ ולאשש רמולכ ,אתלאש ןישידק רמאמו ."'וגו אנליא
יהלא ךרבתי דבכנה םשה הנהו .ךרבתי ותאמ לכה יכ ,"איה האליע תרזגו" (אכ ,םש) לאינד ול רמאש
,אל םירבד) "ץראה רכנ יהלא ירחא הנזו" םעט והזו ,'ה תלחנ איהש לארשי ץרא יהלאו םלועה לכב םיהלאה
ץראה יהלא טפשמ תא ועדי אל" (וכ ,זי ב"מ) רמאנש והזו ,ותלחנבו םשה ץראב םירכנ תוהולאה יכ ,(זט
אל םייתוכה הנהו ."ץראה יהלא טפשמ תא םיעדוי םניא רשאכ םתוא םיתיממ םנהו תויראה תא םב חלשיו
םהישעמכ םש ושעו םשה ץראב םאובבו ,תויראה תא םהב חלשל םהיהלא תא םדבעב םצראב םישנענ ויה
'וגו םכתא ץראה איקת אלו" (די ,אי םישודק) ארפסב ונש ןכו .םתוא םיתיממה תויראה םהב חלש םינושארה
רכנ לא ומע ןיאו" (וטש וניזאה) ירפסבו ."הריבע ירבוע תמייקמ הניא ,תוצרא ראשכ הניא לארשי ץרא --
רש הנהו אצוי ינאו רמאנש ןינעכ ,םכב טולשל אבל תומואה ירשמ דחאל תושר אהת אלש -- (בי בל םירבד)
(חל ,הכ ןלהל) רמאנש ,הולא ול ןיאש ימכ המוד ץראל הצוחב רדה לכ" (ב יק תובותכ) םרמאמ אוהו ."'וגו ןוי
רמאל 'ה תלחנב חפתסהמ םויה ינושרג יכ (טי ,וכ א"ש) רמואו ,םיהלאל םכל תויהל ןענכ ץרא תא םכל תתל
…"םירחא םיהלא דובע ךל
בתכו .ךיניע וב יתחקפש רחא םהב שרופמ ונארתו ,םיבותכב םיבר תומוקמב אוה הזה ןינעהו…"
הנשי אל םשהו ,םילבקמהמ אבי יונישהו דחא םשה יכ ונעדי" (זט ,אל םירבד) ךליו תשרפב א"ר
טפשמ תא בותכ ןכ לע םוקמה יפכ לובקה חכ רומשל םשה תדובעמו ,המכחב םה םלוכ יכ וישעמ
ליכשמהו ראש םהש תוירעב קבדה םוקמה ךפהו ,רכנה יהלא תא וריסה בקעיב רמאו ,ץראה יהלא
שקבל תרשמ רש אוה יכ (אכ ,י לאינד) "םכרש לאכימ" קוספמ ילע בישת לאו .ל"ז וירבד ולא ".ןיבי
(גי ,ה עשוהי) וחיריב עשוהיל הארנה אבצ רש היה ןכו .הלשממו תוכלמ רש אל ,לארשי לע םימחר
ונתויהב הז היהש םגו ,(הל ,טי ב"מ) והיקזחב ןינעכ םהיתומחלמ םוחלל וחלש םשה יכ ול הארה
"…ץראל הצוחב
         In all, we have highlighted 26 relevant terms. Here they are in the order in which they appeared:

1. בכוכ star. 2. לזמ constellation. 3. תולזמ constellations . 4. יכאלמ the angels of. 5. םירש lords. 6. תוכלמ רש the prince of the kingdom of (a prince exercising royalty. 7. רש the prince of. 8. םיכלמ kings . 9. יכלמ the kings of. 10. םיהלא gods. 11. םינודא lords . 12. םיכאלמ angels. 13. ןיצק chief. 14. רטוש observer . 15. לשומ ruler . 16. םיתרשמ servants. 17. םילשומ rulers . 18. םינוילע celestial powers. 19. ןיריע the wakeful ones. 20. ןישידק the holy ones. 21. ריע a wakeful one. 22. שידק a holy one. 23. יהלא gods of the. 24. תוהולא gods. 25. לא the god of. 26. ירש the princes of.
(The translation of the 26 terms was taken from the same source used by BMS.)

2. Contriving guidelines:
        BMS explain that they arrived at their four prefixes using three guidelines:
  1. They only selected terms that are used explicitly by Nachmanides.

    In fact, as we have seen, Nachmanides actually uses explicitly many more terms than the ones they mention.
  2. "When a word appears in the quotation in both singular and plural form, we always prefer the (shorter) singular form". However,
  3. "When it appears only in plural form, we can't know if it is reasonable to use it in singular form and hence we keep it in plural form".
1. בכוכ . 2. לזמ . 3. ךאלמ . 4. רש . 5. ךלמ . 6. ןודא . 7. ןיצק . 8. רטוש . 9. לשומ . 10. תרשמ . 11. ןוילע 12. ריע . 13. שידק . 14. הלא [11]. 15. תוהולא . 16. לא .

5. The best quartet:
        In this section we will describe the measurements mentioned in Chapter V, which were conceived and executed by Yosef Beremez.

1. How the "Best Quartet" is determined for any text T using P1:
a. We will accept BMS's list of prefixes ki, where 1< i < 136.
b. Every prefix ki defines a set Ki of 68 word pairs (X, prefix X), where X is one of the names from the "Table of Nations" (see Appendix to Part I).
c. When we calculate the values of c(w,w') for the pairs in set Ki, we receive a set Ki(T), in which there are n(Ki,T) values of c(w,w'), where n(Ki,T) is a number between 0 and 68.
d. We then calculate the value of P1 for the set Ki(T).
e. We rank the sets Ki(T) in ascending order of P1 values, so that K1(T) has the lowest value of P1 (the set of word pairs for which K1(T) is its set of c(w,w') values, will be denoted K1).
f. The "Best Quartet", Q(T), is defined as the union of the four sets: K1(T)-K4(T).
g. To measure the significance of the statistic P1(T) for the quartet Q(T), we perform the randomization PRWL on the set of pairs Q, which is the union of the four sets K1-K4.

2. Evaluating (through simulation) the probability of receiving a significance of r1(T):
        Because of the enormous difference between the values of r1(G) and r1(WGP) we are compelled to evaluate the probability of receiving such a low value of r1(T). There is no easy way to make such a calculation. One cannot, for example, simply determine how many quartets can be drawn from a list of 136 items, because these quartets are not independent. Therefore we used a simulation as follows:

  1. In place of the set Ki(T) in step c. above, we will build a set K'i(T) in which there are
    n(Ki,T) random numbers chosen by lottery from the segment (0,1] (this is the range of values of c(w,w')), produced by a random function in the computer.
  2. We then carry out steps d-f as above, using K'i(T) in place of Ki(T).
  3. In this way we receive the set Q'(T), for which we calculate the value P'1(T). (Step g. above, which was needed to guard against the effects of any possible dependence between the values of c(w,w'), is unnecessary when random numbers are involved.)
  4. We repeat the series of lotteries and calculations in steps 1-3 numerous times, and examine the ranking of r1(T) among the set of values of P'1(T).
3. Results and Conclusions:
 Hence, the probability that r1(G) is so small is p=0.00042.
 Hence, the probability that r1(WGP) is so small is p=0.354.

Acknowledgements:
        I would like to thank Yoav Rosenberg and Yaakov Rosenberg for preparing the programs used in making these measurements. I'm grateful to Yosef Beremez for contributing to both parts of this article with his creative ideas and valuable work. I wish to express my thanks to Dr. S. Srebrenik for helpful discussions and suggestions.

Bibliography and Notes:
 1. The Introduction to their article.
2. Section 3.4.
3. See Table 4 in the Appendix to their article.
4. Section 7.1-2.
5. M. Bar-Hillel, D. Bar-Natan, B. McKay; One Can Skip in War and Peace Too, Galileo, No. 25 ('97), pp. 52-57.
6. See Document 4 at this site.
7. M. Bar-Hillel, D. Bar-Natan, B. McKay; The Torah Codes: Puzzle and Solution, Chance, vol. 11, No. 2 ('98), pp. 13-19.
8. See their comment about Table 4, and at the end of Chapter 4.
9. Copies of the programs were deposited with editorial board of Statistical Science at the time, and they also received a copy.
10. In section 3.2.
11. See Table 5, number 5.