If you’re making things up in your study, it’s probably not wise to bring that to attention. Yet, that’s exactly what Matthew Britt and Jaaron Wingo do in their book, “Christ Before Jesus.” This is the third part of my critique of their book, and in this article, we examine some additional potential concerns they suggested may be raised, which turn out to be real.
Moving through the final concerns that Britt and Wingo mention, much of the remaining confidence I had in their methodology and approach crumbled away. And it wasn’t always because of the concern they discussed.
For instance, they cover why they don’t use random sampling in their analysis. There’s no real problem there in a general sense. Many stylometric analyses don’t utilize random sampling. But their reasoning was flawed.
Part of their reason for not using random sampling was because the texts they used were already small. But that’s a problem as we already discussed. But here, they go even further. They claim that some of the Pauline letters seem to have line-by-line additions and edits.
How did they come to that conclusion though? It can’t have been through a stylometric analysis, as going line-by-line is far too small of a sample size. We don’t have textual evidence of such a claim, nor do they give any reason to suspect such.
Moving along, their next potential concern is about whether they just made up the results. They attempt to address this concern by saying that their readers can check and confirm their results. Yet, as we saw above, without their methodology, or even just their full results, the actual graphs, there is no way to really replicate their study.
To further try to clear up any concern that the data is fabricated, they make the claim that “there’s no way to make up these graphs without doing something like falsely labeling files or Photoshopping the whole graph.” There is just so much wrong with such a statement.
First, they did “Photoshop” or edit the graphs. They removed pertinent information from them, making the graphs largely useless as they lack necessary data. That’s clear manipulation either through sheer ignorance, or through purposeful intent.
Second, there are many ways to “make up” the graphs. Cherry picking data or methods could have been done, until the results they wanted appeared. Seeing that the authors virtually admit that they played around with Stylo until they found something that seemingly fit, this may not be that far off the mark. And since we can’t replicate anything, as there is no methodology to speak of, we can’t confirm or deny how accurate their results are.
There would be no need to falsely label files here, one could simply manipulate the data or the process that the data is analyzed. Such as by mixing chapters in a haphazard way. Or by experimenting with n-grams until you get certain results. Or by hiding the distance scale and implying the results mean something they don’t really mean.
Third and final, just because you get results, that doesn’t mean those results are accurate. One will get results even if the data used is corrupt, or if the sample is unsuitable. As already discussed, there’s good reason to conclude that their sample size is far from being suitable.
Continuing to their next potential concern, they attempt to defend using stylometry to date these texts. Simply, stylometry isn’t used to date texts. That’s not what it’s used for. Britt and Wingo even seem to partially accept this, by saying that “stylometry doesn’t give you dates per se.”
Yet they try to defend their argument by claiming that we can date texts and assign authors in other ways, and based on that, we can reconstruct general ideas about when Christian texts were written. They go on to claim that if a writing appears alongside a text we know the date of, we can then conclude that it also dates to that same time period.
Yet, as we saw in O’Sullivan’s 2021 article, shared styles can be developed. So a parent and child may share the same basic style, and thus appear close together when analyzed. We could potentially add to this another parent, which would push the time between when the two texts were written back quite a bit.
This isn’t even a stretch from what Britt and Wingo state, as they argue that the closeness could be due to perhaps a student who would have a similar style. Yet, the flip of this would be just as true, that the closeness was due to a teacher. Or potentially a shared teacher. It could also have to do with a shared genre, as demonstrated above. There are many factors that could be at play here.
To suggest that style can determine a date just isn’t supported. Nor do the authors give us any reason, any research, that suggests otherwise.
In a similar manner, the authors also just dismiss the concern that the fact scribes were used to write these letters could have changed the style. Without citing any evidence for their argument, they just claim no, scribes would have no effect on style unless they were forging a letter.
That’s simply not true, and it ignores the impact a scribe could have on a text. Steve Reece has a great book that delves into this topic titled “Paul’s Large Letters: Paul’s Autographic Subscription in the Light of Ancient Epistolary Conventions.” What he demonstrates is that a scribe can have a broad range of involvement.
This could vary from a scribe recording each word, or at times it meant them taking notes, and recrafting them. In the latter case, the scribe’s style would meld with the author’s style. In such a case, both individuals’ styles could potentially be picked out.
In a 2021 study by Eveline Leclercq and Mike Kestemont, titled “Advances in Distant Diplomatics: A Stylometric Approach to Medieval Charters,” they specifically talk about using stylometry to help disentangle different authorial strata, including the issuer of the charter, the dictator, as well as the scribe.
Now, obviously, this study deals with a different time period, but it demonstrates the basic point here, that scribes can have an impact on the style of a writing, and specifically, they can be examined with stylometry.
What should be clear then is that yes, a scribe could have a major impact here. But it gets a bit more complicated. As Timothy Mitchell explains in his article, “Myths About Autographs: What They Were and How Long They Have Survived,” the process of writing was often the product of a community.
In this way, a work could be written or dictated, and then it would often receive some constructive criticism, as well as suggestions for changes, and then brought together possibly with the use of a scribe. And we know Paul had used scribes, and at times, those scribes felt some freedom to add a bit to the text, as Tertius does in the Epistle to the Romans.
To make the claim then that a scribe would have no influence on the style of a text simply is wrong.