The Carrier Problem: Why Bayesian Math Cannot Save Mythicism; Part 1
Of all the figures in the modern mythicist movement, Richard Carrier stands out. He holds a PhD in ancient history from Columbia University. He’s incredibly active in the community, and he has engaged in the scholarship properly. His 2014 work, On the Historicity of Jesus: Why We Might Have Reason for Doubt, successfully passed peer review and was published by Sheffield Phoenix Press.
Carrier has by far made the most serious attempt to argue the mythicist case, and regardless of what one may ultimately conclude about his arguments, his work deserves, at the very least, the courtesy of engagement.
Now, to be fair, Carrier’s book has received some engagement. The flagship, academic response came from Daniel Gullotta, in the Journal for the Study of the Historical Jesus, titled, “On Richard Carrier’s Doubts: A Response to Richard Carrier’s On the Historicity of Jesus”
A number of popular works have also set out to critique Carrier’s work, at least in part, including a variety of articles by James McGrath and Bart Ehrman’s book, Did Jesus Exist, which led to an exchange between the two.
While some critics have pointed out positives in Carrier’s work, overall, they have remained negative because they aren’t swayed by Carrier’s position. Adding to this, the manner in which Carrier tends to respond to said critics, which often is overly abusive, engagement tends to dissolve as most don’t want to deal with the aggressive and insulting responses.
This has led to an awkward situation in which his work deserves to be examined and taken seriously, but for many, it’s not worth it. For the sake of this project, we will dive into Carrier’s work while attempting to navigate the noise of negativity.
In this article, the first of a two-part series, we will focus on Carrier’s methodology, specifically focusing on his use of Bayes’ Theorem as a tool for historical reasoning, as well as the problems that arise from the way that he constructs his “reference class” of comparable figures. The second part will examine the specific evidential claims that Carrier makes about Paul, the Gospels, Josephus, Tacitus, and the rest of the documentary records. In doing so, we will look at why his readings of these texts have failed to persuade scholars in the field.
In time, we will come back to Carrier’s full book, On the Historicity of Jesus, with a full breakdown, but for now, let’s jump into the problem of Bayes Theorem.
What’s Baye’s Theorem, and Why Carrier Wants to Use It
In the 18th century, an English clergyman and statistician, Thomas Bayes, developed a piece of probability theory, which would later be refined by Pierre-Simon Laplace. To put it simply, Bayes’ Theorem provides a formal procedure that can be used to update one’s confidence in a hypothesis as new evidence comes in.
How this works is that you begin with a “prior probability,” which is an estimate of how likely the hypothesis is before considering any new evidence. From here, we ask how likely the observed evidence would be if the hypothesis is true, compared to how likely it would be if it were false. The ratio of these two likelihoods, applied to the prior, yields a “posterior probability,” our updated confidence in the hypothesis after considering the evidence.
In general, there is nothing wrong with Bayes’ Theorem. It’s often used successfully in medical diagnosis, spam filtering, machine learning, and a variety of branches of empirical sciences. It works well within these domains because the inputs are typically derived from large, well-defined datasets. For instance, we know quite reasonably what fraction of patients with a given symptom have a given disease. We have records of millions of patients from which we can derive this information.
As long as we have good inputs that we can feed into Bayes’ Theorem, then the results we produce can be highly accurate. The challenge of applying this sort of Bayesian reasoning to history is that the inputs are not nearly as well-defined or as good as those we see in the medical field.
When it comes to ancient history, we don’t have a database of millions of first-century religious movements with known outcomes, against which we could calibrate our priors about whether any given alleged founder was historical or mythological. Instead, what we have in the historical record are sparse, often fragmentary, and ambiguous sources.
In history, we often rely on informed judgment calls about what counts as a relevant comparison, how much weight to place on a piece of evidence, and how to handle the gaps in the record. To assign some numerical probability to all of this is virtually impossible.
Nevertheless, Carrier argues that Bayes’ Theorem ought to be the foundation tool of historical reasoning. In his 2012 book, Proving History, he goes as far as to make the case that all sound historical reasoning is implicitly Bayesian, whether historians realize it or not. Historians who have engaged in Carrier’s methodology, though, have concluded that this sort of formalization really adds nothing of value while introducing a misleading appearance of precision.
Even if we set this broader objection aside, Carrier’s particular approach of using Bayes’ Theorem to the question of the historicity of Jesus fails on its own terms, and it fails at the very first step in the process: the construction of the prior.
The Reference Class Problem
As we discussed, Bayes’ Theorem requires a prior probability. In Carrier’s approach to the historicity question, he derives this probability from what statisticians call a “reference class,” which is a set of comparable cases against which our own case can be measured
If we want to estimate, before considering any specific evidence, how likely it is that Jesus was a real historical person, we have to ask: of all of the figures who resemble Jesus in some relevant respects, what fraction have turned out to be real?
This would be rather straightforward if we had a significant database that we could consult, as in the case of a medical diagnosis. Even then, though, we still have a major issue; the construction of a reference class is one of the oldest and most notorious problems in the philosophy of probability.
The problem breaks down as follows. Any given object or event belongs to a wide array of different possible reference classes, and the choice of which class we use will often determine the answer one gets.
Let’s apply this to Jesus. If we reckon that Jesus was a “first-century Galilean Jewish apocalyptic preacher,” we get a reference class in which virtually every member will be historical. It’s not really an issue. However, if we change this to Jesus being a “savior figure who dies and rises,” we then have a class where most members would be mythological. The prior we end up with largely depends on which class we choose, and there is no neutral, theory-independent way to make that choice.
The way that Carrier attempts to solve this problem is to construct what he calls the “Rank-Raglan reference class.” Doing so, he draws on the work of the early 20th-century folklorist Lord Raglan and his predecessor, F.R.S. Otto Rank, both of whom proposed checklists of recurring features in the biographies of mythological heroes.
Raglan’s original list contained twenty-two features. He then applied his list to a set of legendary figures and found that most of them scored highly on it. Building on this, Carrier, who scored Jesus on that same list, also noted that he scored highly, with a match of 20 of the 22 features. The conclusion is that Jesus belongs in a reference class with Romulus and Heracles, mythological figures, rather than with more historical figures like Apollonius of Tyana or Simon bar Kokhba.
From this reference class, Carrier derived his prior, and observing that most who would fit in this same class are mythological, concluded that the prior probability of Jesus’s historicity, before any specific evidence is considered, is somewhere between zero and one in three. Right off the bat, this construction begins with problems, and only multiplies quickly from there.
The Rigged Rank-Raglan Reference Class
The first and most basic problem with the Rank-Raglan list is that it was constructed by selecting features that recurred among a set of figures already taken to be mythological, largely with regard to Western cultures. Raglan was not attempting to develop a neutral diagnostic for distinguishing historical figures from non-historical figures. He was trying to identify the patterns common to mythological hero narratives.
This basic problem is exacerbated by Carrier’s modifications to different features, which contribute to Jesus scoring even higher on the list. Carrier does not inform his readers why he made these changes or how he even made them. This can be seen in the change from the original “hero’s mother is a royal virgin” to Carrier’s more ambiguous “hero’s mother is a virgin.”
Carrier does this multiple times, and it appears to be intended to allow Jesus to rank higher on the list than he naturally would. This further rigs the entire setup that Carrier has, and largely invalidates his conclusions before they even begin.
But we also have other problems here. Second, the assignment of Jesus to this reference class depends on accepting a particular reading of the Gospel narratives; specifically, counting features that come from late, theologically developed layers of the tradition and declaring them essential to the historical Jesus question.
To give Jesus a higher score, Carrier had to count events attested only in one Gospel, while ignoring the other sources. In other words, he had to stack the deck, even though many of the elements he used are exactly the ones that mainstream historians have long rejected as later, legendary embellishments.
By stacking the deck in this manner and modifying the Raglan scale, Carrier is able to take a Jesus, who would have scored a 7 or 8 if we just used Mark, up to a 20. Doing so gets Jesus above the score of 11, which was necessary to place Jesus in the same reference class as someone like Romulus or Heracles; in a reference class populated by mythological figures.
A third problem, which has become increasingly damaging to Carrier’s case in recent years, is that the Rank-Raglan reference class, even taken on its own terms, is not properly representative of figures from a historical period relevant to Jesus. By taking Jesus and placing him in with a class of figures from a distant mythological past, from cultures whose historical records are fragmentary or just non-existent, you’re not only stripping him out of his context, but you’re also completely ignoring it.
It’s the very problem of selecting a reference class that just doesn’t fit, which is the underlying problem to all of this. If we were to take Jesus and place him in a different reference class, one built around figures from his own time period, from the Levant, the conclusion is strikingly different.
And this somewhat gets at another problem. If we were to treat Alexander the Great in the same manner, using the same modified Rank-Raglan scale, including all the legendary material, we would have yet another individual who scored highly and would be in the same reference class as Jesus. Even someone like George Washington, if we treated him in the same manner, would get pretty close to that same class.
The Deeper Problem: Mathematics as Theater
Let’s say we grant Carrier his reference class, which we really shouldn’t; there is still a deeper problem with the entire project that this reference-class issue helps illuminate. Bayesian reasoning, in fields where the inputs cannot be empirically calibrated, where they are much more loose, does not add rigor to historical thinking. It merely gives it all a bit more flash here, while hiding the judgment calls behind an air of more intense rationalization.
When a historian writes that “Paul’s reference to James the brother of the Lord constitutes strong evidence for the historical existence of Jesus, because it is difficult to imagine why Paul would have invented a personal acquaintance with the brother of a mythological figure,” the reader can see exactly what the historian is claiming and can engage the argument directly.
When Carrier translates the same kind of judgment into a likelihood ratio, say, by claiming that the probability of Paul’s testimony given the historicity hypothesis is 0.6 while the probability of the same testimony given the mythicist hypothesis is 0.4, the underlying reasoning is no more rigorous than before. It is simply hidden behind a number.
And because the final result is the product of dozens of such likelihood ratios, each of them assigned by Carrier’s own judgment about contested evidence, the apparent precision of the final figure gives a dramatically misleading impression of certainty.
This is what we could call mathematical theater. It’s the use of numerical notation not to discipline the reasoning, but instead to dress it up in borrowed authority. The actual judgments that Carrier makes, such as which reference class to use, how to read Paul, or how to evaluate Josephus, are still doing the work. But it looks much more impressive with a fancy calculation behind it.
The mathematics are not doing anything of substance here. And in fact, may be hurting Carrier’s case more than they ever could help. There are really two reasons for this.
First, as several scholars have pointed out, Carrier is not the first to use Bayesian reasoning regarding the life of Jesus. Richard Swinburne, a professor of Philosophy, used Bayes’ Theorem not only to argue for the probability that Jesus was, in fact, resurrected, but also for the very existence of God.
This shows the underlying problem. For history and philosophy here, we do not have a database that can be used to accurately enter the proper inputs into Bayes’ Theorem. And because of that, our choices will have a massive impact on the outcome. This allows us to easily manipulate the data on purpose, or even to let our biases creep in unintentionally, as, after all, we are still the ones making independent judgment calls.
Second, the potential error when it comes to Bayes’ Theorem only multiplies exponentially over time. For Carrier to reach the conclusion he does, he has to make hundreds of different judgment calls. If one of those is wrong, it can increase the error in the outcome at each subsequent step. If additional inputs are incorrect, this just keeps stacking. As we’ve already seen that the reference class that Carrier is using is rigged in his favor, we simply can’t trust any output from his use of this theorem.
What this largely amounts to is Carrier creating an exceptionally elaborate procedure for stating his prior assumptions and then declaring victory. The theorem he uses doesn’t change the underlying historical judgments that he makes; it merely launders them through equations.
A Note on Carrier’s Intellectual Honesty
Now, it is worth pausing for a moment, before closing this first part, to make a point about Carrier himself. Whatever the failures of his project, Carrier is not a charlatan in the way that some popular mythicists are. He has actual training in a relevant field, engages with primary sources, and, on a number of occasions, actively distances himself from the wilder fringes of Mythicism. When it came to D.M Murdock and the Zeitgeist film, he explicitly criticized it as “parallelomania.” When the book Christ Before Jesus came out and attempted to use stylometry to prove Jesus didn’t exist, he rightfully called them out and showed just how flawed their work was.
He is, in this sense, the most serious figure the movement has produced, and the response to his work should be correspondingly serious.
But the seriousness of the presentation does not entail the soundness of the conclusion. The fact that Carrier’s approach is more sophisticated and informed than that of his popular-mythicist contemporaries only means that the response to him must be more sophisticated and informed.
The use of Bayes’ theorem here, and the mathematical theater that it creates, really is the fatal step in his process. So while Carrier has produced the most rigorous-looking case for Mythicism in the literature, it doesn’t mean he has produced a sound one.
Finishing Off Part 1
Baye’s Theorem is what gives On the Historicity of Jesus its distinctive flavor, and it does much of the work of impressing lay readers who are unfamiliar with the underlying mathematics. But this same theorem is what collapses the foundation of his argument.
Carrier’s reference class is rigged by construction. It’s stacked in his favor, and it will only result in its failure as those base errors build up to a point of no return.
We will see how this turns out once we begin looking at how Carrier handles the specific evidence. Because even with a defective prior, one could in theory imagine that the documentary record might point so strongly in the mythicist direction that the posterior calculation would still favor non-historicity. Carrier himself argues that the evidence does just this.
But as we will see in part 2, by taking various claims one at a time, why Carrier’s reading of this evidence has failed to persuade the rest of academia. His case will fail not only at the level of methodology but also in the evidence.
