Millions of people rely on Rotten Tomatoes, but not many of them know how it actually works. Rotten Tomatoes scores are calculated in the
simplest of ways: the share of positive reviews a movie received by well-established film critics. This score does not distinguish between an extremely good review and a barely positive one.
That is crazy. As a data analyst, if I was trying to assess the quality of a product, I would never take a set of nuanced reviews and turn them into what statisticians call “binaries” (yes or no, 1 or 0, positive or negative). By doing it this way, you lose tons of pertinent information. Instead, I would try to assess the positivity of the review on a continuous scale, like 0-100 or A-F or even 1-5 clamshells, and then take the average or median of those scores.
Luckily, there is a review website that does just that:
Metacritic. Whatever the other merits of these sites, Metacritic’s method of scoring movies is simply better than Rotten Tomatoes’.
Metacritic takes reviews from critics, gives them a 0-100 score, and then averages those scores. Metacritic has a higher threshold for the renown of the critic whose reviews are considered for the site, so while the Rotten Tomatoes’ score for
Aquaman includes the reviews of 312 critics, Metacritic only uses 49. (The movie got a score of 55.) Also, as Allison Wilkinson
explains in Vox, Metacritic gives more weight in its average to the most highly respected critics, like those from the New York Times.
A comparison of the Metacritic and Rotten Tomatoes scores for the hits
Crazy Rich Asians and
A Star is Born reveals the differences between the two sites.
Crazy Rich Asians, a fun romantic comedy, scored a 91 on Rotten Tomatoes, while
A Star is Born, a leading contender to win Best Picture at the Academy Awards, received a 90. A virtual tie. On Metacritic, by contrast,
Crazy Rich Asians got a 74, compared with an 88 for
A Star is Born. Both are pleasant movies, so I can see how going by just yes or no answers, they could be rated similarly. But to me and most critics,
A Star is Born is a much higher quality movie. If I could only see one of them, it would be wise to go with Gaga.
Statistics are a powerful tool for assessing culture, but only when used properly. The next time you are deciding which movie to see, if you want to see something great, and not just good, consider what goes into that review sites’ calculations.