Films should not be penalized for getting 10/10 ratings - A new look at why not.

Dan,

I am sympathetic to your problem.

The IMDb ratings are nothing more and nothing less than the aggregate ratings of people who have accounts on the IMDb. It is probably not representative of the views of the population as a whole nor does it reflect any serious critical assessment of the quality of a film. It is at best a qualitative measure of populist film preferences and at worst entirely misleading. It is highly inappropriate to use the IMDb ratings as a means to assess independent or hobbyist films. The “They Shoot Pictures, Don't They?” website (http://www.theyshootpictures.com/gf10...) would be more appropriate.

Consider these inherent shortcomings:
1. The IMDb poll is a self selected survey. Self selected surveys are inherently flawed and statistically unsound.
2. Not everyone provides their ages and gender when registering for an account. No one can say with certainty that people provide correct demographic information.
3. Assuming that people provide accurate demographic information, the underlying population of the IMDb membership is not representative of the general population or may not be representative of the population of the film going population. People with accounts on the IMDb are predominately males between the ages of 18 and 29 who live in the United States.
4. Comparing the ratings of any two films are at best problematic since the populations rating each film may have very little overlap. Since this is a self selection survey sampling methodologies cannot be used to make that comparison.
5. There are significantly more votes for recent and heavily publicized films.
6. Initial ratings for recently released films tend to be much higher reflecting the perspective of the film’s fan base and declines in time. For instance, Avatar had a rating of 9.03, a weighted rating of 8.95 and a rank of 21 on 21 December 2009. It now has a rating of 7.98, a weighted rating of 7.92 and is no longer in the IMDb Top 250.
7. The methods that IMDb uses to reduce voting fraud generally do not work well for films with few votes.

Thanks for your response. You have pointed out a very good list of shortcomings. I do understand that it is not a true measure of a film, but the reality is that IMDB and other similar sites are a way for films to get mainstream exposure. When the average person sees a low rating, they skip over the film.

I think there is a fairly simple solution to the root issue and one that could be implemented by a tweak to the algorithm that uses pattern detection to root out fraudulent scores.

When any score is received, the system should check and record the IP, country, server IDs, unique system identifiers. If scores within a certain range (either high or low) are repeatedly coming in with the same identifiers, then only THOSE scores should be weighted or eliminated. There are industry standard ways of rooting out click fraud - Google uses the same sort of thing to root out fraudulent Adwords clicks.

This would go a long way to creating a system that is both fair and useful to both film makers and the general public.

Dan,

Your should post your suggestion to use pattern recognition to root out fraudulent scores as a separate suggestion. Click fraud also applies to attempts to inflate STARmeter ratings.

On the other hand, IMDb may already be using IP and other kinds of filtering to reduce the problem. IMDb has indicated that it does not disclose the methods it uses to make it more difficult to game the system.

On a separate note, I find the IMDb Top 250 and ratings interesting, but understand their limitations. Unfortunately, many users of IMDb do not.

Useless...a good movie always will be on top...in 98% of cases.

Why is Dan Dassow always responding to the issue of vote bombing? Does he work for IMDB?