Can stylometry prove Jesus, or even Paul, didn’t exist? Short answer, no. Yet, that is a main focus in the book, Christ Before Jesus, a work that tries to further the Jesus Myth case.
As a former mythicist, someone who accepted the conclusion that Jesus was a mythical figure, the argument was rather intriguing as it is something rather new. When looking at the various cases that different mythicists make, the topic of stylometry doesn’t really appear. Nor do most even reject Paul; they simply accept that he was a historical figure.
However, stylometry, or simply the study of linguistic style, is not new when it comes to New Testament studies. So what are we to make of Matthew Britt’s and Jaaron Wingo’s, the authors of “Christ before Jesus,” arguments?
To cover their entire argument, I will first look at their methodology and results in regards to their stylometric experiments. By showing major flaws in both, I will show how many of their conclusions are not supported, and thus we can deal with the actual history itself. After going through their other base arguments, I will return to the topic of stylometry, and how it can be applied to New Testament studies properly.
Because of the size of this endeavor, I will be breaking this into a series, so that it is more easily digestible. So let’s jump in, and look at the major flaws in Britt’s and Wingo’s stylometry method.
Marketing
Before we get into their chapters on stylometry (chapters 7 and 8), we already run into a problem. That problem being how they market their book. They make the bold claim that, “Christ Before Jesus presents a first-of-its-kind analysis using proven, peer-reviewed mathematics and software which reveals a second-century origin for the books of the New Testament.”
There are a few problems with this. First, what they are attempting to do really isn’t a first-of-its-kind analysis. Even in their book, they make the claim that computerized stylometric analysis of the New Testament dates back to Anthony Kenny and his work, “A Stylometric Study of the New Testament.” This work was published in 1986. However, even here they get the history incorrect.
Over two decades earlier, Andrew Q. Morton, who was a pioneer in stylometry, had began using a computer to run stylometric studies. The first mention of this is in 1963, where he makes mention of the tests he ran on a computer at the University of Edinburgh. Granted, this was just an article that was published in a British newspaper, detailing what Morton had been working on. But by 1965, he was giving college lectures on this subject, and in 1971, he published “It’s Greek to the Computer,” along with A.D. Winspear.
By the time of Britt’s and Wingo’s writing, at least six decades of research had already gone into this subject, and that’s just counting studies that utilized computers. The study of stylometry in general dates back to at least 1890 with the publication of “Principes de stylometrie” by Polish philosopher Wincenty Lutoslawski. Yet even this work is just the place historians point to when discussing when the basic ideas of stylometry were set out.
What are we to make of this claim that “Christ Before Jesus” is the first-of-its-kind analysis then? Obviously, it’s not the first computational stylometric analysis of Pauline authorship or the New Testament. On page 207 (Kindle Version) of their book, they do mention that the majority of stylometric analysis on the “New Testament texts makes the assumption that looking at them alongside second-century texts would be of no avail.”
This though suggests a misunderstanding of what stylometry is. Why would such analysis want to include second-century texts? Stylometry is not used to date texts. Unless one has a potential match for an author coming out of the second-century, adding second-century texts would factually be of little to no avail. After all, stylometry really is used to determine common authorship, or even conversely, forgery.
Maybe they are referring to the idea that this analysis is meant to reveal that the New Testament texts originated in the second-century. Again though, that’s not what stylometry is used for; it’s not used for the dating of texts. It’s used in regards to authorship. Unless they can reveal who the true author of these texts are, that they are all second-century authors that we have writings from that can be compared, their goal here simply is out of the realm of what stylometry can do.
Finally, maybe what they are referring to here as being a first-of-its-kind is that they break the texts of the New Testament into chapter lengths, and then run multiple analyses on that. Which may be true, but it also shows a misunderstanding of what stylometry can achieve.
In order to achieve a quality analysis, the more information one has, the better. Matthew Brook O’Donnell, in “Designing and Compiling a Register-Balanced Corpus of Hellenistic Greek for the Purpose of Linguistic Description and Investigation,” breaks down the size of corpus that is really needed for various analysis. They break this down into three categories; tense form distribution needing around 2,000 words; word order needing 20,000 words; and vocabulary needing 20 million words within that corpus.
When talking about Paul, looking at the seven largely undisputed letters, those being Romans, First and Second Corinthians, Galatians, Philippians, First Thessalonians, and Philemon, we get around 30,000 words total (at least in English), which would allow us to run word order analysis, but its still a far cry away from determining the author’s possible vocabulary. To make matters worse, a good portion of that word count comes from just two books, Romans and First Corinthians.
Looking at individual chapters, some, when looking at the Greek text here, are less than 200 words. To be fair, Britt and Wingo do recognize that there is a limit, as they explain in chapter 7 how Second and Third John are too short, so they combined them when running their analysis, and while they don’t give us the breakdown of how they divided the chapters in the Pauline corpus, they do imply that some chapters were in fact merged. But this lack of transparency in their methodology will come into play later.
Now, after having some email exchanges with Britt and Wingo, it is this aspect that they claim is a first-of-its-kind analysis. But as mentioned above, there is a reason why such an analysis hasn’t been ran before. It simply isn’t accurate. Because there isn’t enough data, the results simply can’t be trusted, and we will dive more into this in just a little bit.
The second issue with their marketing is that it is misleading. They use the phrase peer-reviewed, which seems to give some weight to their argument. But what they are claiming is peer-reviewed isn’t their study, or anything in regards to that, but the software that they are using, which is Stylometry with R, or simply Stylo.
Stylo is one of a few different stylometric programs that you can freely download at home (Signature by Peter Millican of Oxford University and the Java Graphical Authoriship Attribution Program, JGAAP, by Dr. Patrick Juola of Duquesne University being the primary ones), and it shows great promise.
The issue is that whether Stylo is peer-reviewed or not, it doesn’t reflect on studies done using Stylo. Just because Stylo is a legitimate tool, it doesn’t mean the work done with Stylo is legitimate. This will instead depend on how one uses Stylo, what methodology is practiced.
As we will see, the methodology that Britt and Wingo use is not effective, and thus their results are not accurate, nor should they be taken seriously. For instance, as I’ve already mentioned, the text lengths that they propose to use are too short to provide any real usable information.
Third and final, as mentioned already, stylometry does not provide the ability to date texts. That is not a function of stylometric analysis. That being so, it is not possible for the “mathematics and software” they use to reveal that the New Testament texts originated in the 2nd century. The entire premise here is one that is based on a faulty foundation, yet is put forth with a great amount of confidence.
This idea that the New Testament texts originated in the 2nd century also poses a different issue. Both in private correspondence, as well as in their public Tiktok videos, Britt and Wingo falsely imply that there is a movement to redate the New Testament texts into the 2nd century. While in our correspondence, they mentioned the recent work by Dr. Nina Livesey, this in no way suggests a shift in the general scholarship. Livesey, in this regard, is an outlier. To put it simply, Britt and Wingo are overplaying their hand here.
Stylometry
So that was a good deal of discussion based on a single claim. Yet I think it was important in order to demonstrate a couple of key issues that their book, “Christ Before Jesus,” has. First, it implies that Britt and Wingo have a flawed view as to what can be accomplished with stylometry, as well as its limits. And second, it shows a clear overstatement, and overconfidence, in their position. These are two ideas we will need to keep in mind as we move forward.
Now, what about their methodology? While Britt and Wingo spend the entirety of chapter 7 on their methodology, not much detail is actually given. But we can glean some things that are stated directly, as well as a few that are implied.
To begin, Britt and Wingo do get some of the basics about stylometry correct. But part of the reason they get details correct is because they are also rather vague. They briefly touch on ideas, but never really dive in, which may hint at their lack of comprehension here. One could also suggest that this vagueness may just be a byproduct of Britt and Wingo not having formal training or education in regards to publishing research. While both are possibly true, the latter isn’t necessarily a negative reflection on the actual study.
The issue that arises is that there is a lack of details. For instance, on page 208 (Kindle Version), they discuss their methodology a bit. They mention that they found Eder’s Delta and Eder’s Simple Delta formulas to “provide us with the most reliable results when looking at Greek texts of varied lengths from antiquity.”
They make the claim that most reliable here “means most accurate and likely to attribute texts correctly.” This is based on tests they ran on Greek texts. But they don’t tell us what Greek texts they ran, what methodology they used or built from those tests, the length of the texts, or how they figured out that either formula was more accurate. They also don’t state what formula, either Eder’s Delta or Eder’s Simple Delta, that they ended up settling on.
Granted, both formulas are similar, and are derived from John Burrow’s Delta formula, which is a standard, but if they are switching between the two formulas, that can influence the data, allowing for manipulation. It also removes transparency, which is necessary if, as the authors claim, they welcome people to try to replicate their tests. Without detailing their methodology, or even specifying what formula they are using, replication is virtually impossible.
Britt and Wingo also make the claim that they tailored their approach specifically to Greek texts from around 200 BCE to 300 CE. Yet, they don’t explain how they accomplished this, and what that specifically means.
Even though they don’t go into detail here, we can be certain that part of their methodology here, and the approach they tailored, does not take genre into account. This is certain as on page 211, they state that Stylo does not look at “theme, theology, opinion, topic, or anything like that.” They expand this on page 216 by arguing that Stylo doesn’t detect genre, but looks at just authorship.
That’s patently false though. In James Libby’s 2015 dissertation, “Disentangling Authorship and Genre in the Greek New Testament,” Libby demonstrates how genre is predominant over authorship in stylistic studies. Libby was hardly the first to discuss this issue.
In Adrianus Linmans’ 1998 article, “Correspondence analysis of the Synoptic Gospels,” he states very clearly that:
“As far as the influence on language use is concerned discourse type takes precedence over authorial preferences. Ignoring the discourse type factor in stylometry will almost invariably vitiate the outcomes.”
We can expand on this idea, and see that a variety of other factors can have an impact on stylometric analysis. Andrew Pitt’s 2013 article, “Style and Pseudonymity in Pauline Scholarship,” not only reinforces the problem of genre, but also highlights 8 registers that effect style, and thus have to be accounted for in such studies. Everything from the topic being discussed, to whether the writing is formal or personal will have an effect. Pitts goes on to also highlight that the shared knowledge that Paul and those he’s writing to will have an impact on writing style and word choice. These are all things that have to be factored into these stylometric analyses.
In 2017, Pitts and Joshua Tyra, in “Exploring Linguistic Variation in an Ancient Greek Single-Author Corpus,” demonstrated how the style-shift analysis that Pitts had previously developed worked with a single author. The article tested this method on Josephus, and clearly showed a sizeable stylistic shift in his own writings. Part of the reason for this shift was shown to be because of a change of genre, as well as subject matter.
For instance, they saw a spike in the stylistic shift at Book 12 of Josephus’s Antiquities. This is also what marks the end of the Biblical material in Antiquities, and instead, Josephus moves onto more recent history.
But possibly most telling here is that the creator of Stylo, Maciej Eder, in a 2017 paper titled “Visualization in stylometry,” highlights the fact that the authorial signal can be obscured by different signals, or noise. One of those signals he mentions as being a major challenge is genre.
What should be clear then is that there are a whole host of factors that have to be kept in mind while performing such analyses. To claim that Stylo, or any stylometric process is blind to such factors is a serious misunderstanding which will have serious consequences in regards to any potential results.
This misunderstanding continues to the manner in which they display their results (or test results in this case). In chapter 7 Britt and Wingo display a number of results from the tests they ran. Besides one case, the metric part of stylometric was missing. Their graphs were missing the numbers. As in, the graphs were cropped in such a way as to remove the relevant distance scale, as well as the other relevant material that would have been included, such as what formula was used, how many words were culled, and how many of the most frequent words (MFW) were used.
The removal, or lack of inclusion of this important information at best suggests that the authors have a severe misunderstanding of stylometric analysis, or at worse, they are purposely manipulating the data. One hopes the issue is ignorance, and to give them as much credit as possible, that is the view I will be taking.
That is where we will break it off for this article. In the next article we will look at the potential concerns over their methodology they bring up, and how that further shows their misunderstanding of stylometry.