IMDB's rating system is killing the indies

  • 1
  • Idea
  • Updated 7 years ago
Archived and Closed

This conversation is no longer open for comments or replies and is no longer visible to community members. The community moderator provided the following reason for archiving: Old thread

The 1 star trolls are killing the independent movie ratings. I know this has been brought up before, but I could not find the thread. I doubt these trolls have even seen the movies they are rating "1's" since several were only ever shown at small film festivals. I think imdb either needs to raise the minimum number of votes required to post an actual rating, or rethink their formula. Why on earth would "1" votes count more than anything else. Imdb claims it is an attempt to stop vote stuffing, but ones are just the flip side of stuffing. Plus, when was the last time any movie actually warranted a "1"? Please do something about this, imdb! I'm an independent filmmaker trying to get distribution for a doc and my film's rating is abysmal because of this system. I know that mine is not a unique case.
Photo of Cory Poplin

Cory Poplin

  • 2 Posts
  • 0 Reply Likes
  • angry

Posted 7 years ago

  • 1
Photo of Dan Dassow

Dan Dassow, Champion

  • 16662 Posts
  • 18789 Reply Likes
Please see these threads for a discussion on this matter:
https://getsatisfaction.com/imdb/topi...
https://getsatisfaction.com/imdb/topi...

It is highly inappropriate to use the IMDb ratings as a means to assess independent or hobbyist films. The IMDb ratings methodology was designed to widely released films, it works poorly for films in limited release with relatively few votes.
Photo of Emperor

Emperor, Champion

  • 6418 Posts
  • 3021 Reply Likes
That and look at the reviews, which tend to be more useful than ratings (although not for Evil Dead, apparently).
Photo of bluesmanSF

bluesmanSF, Champion

  • 10815 Posts
  • 6434 Reply Likes
Shawshank, top on the top 250, only has 58% of votes being 10. When I see some of these with 90% of votes being 10's, I know it's not yet time to consider the user ratings. I would think someone in the biz, and only looking at the rating, is not very good at his/her job (and, probably not too successful).
Photo of Randy

Randy

  • 2 Posts
  • 1 Reply Like
"it works poorly for films in limited release with relatively few votes."

And yet they display the numbers anyway. Does that make sense to anyone?

There ought to be an additional user-dependent rating, where only the votes of those other users who most closely match your own ratings (if any) are taken into account.

I find I'm in disagreement with the top 1000 voters, and the people who vote on popular films, pretty much all the time.

(sorry if this appears twice... I think it ate my previous comment)
Photo of Emperor

Emperor, Champion

  • 6418 Posts
  • 3021 Reply Likes
There's a "view more comments" link as it only displays the last two.
Photo of Dan Dassow

Dan Dassow, Champion

  • 16662 Posts
  • 18789 Reply Likes
Randy wrote: "There ought to be an additional user-dependent rating, where only the votes of those other users who most closely match your own ratings (if any) are taken into account."

If another user makes their ratings profile page public, IMDb will allow you to see how correlated their ratings are to yours by looking at their profile page. For instance:
http://www.imdb.com/user/ur1000000/ Col Needham's (IMDb CEO) page
http://www.imdb.com/user/ur0127678/ My profile page

If/when you find someone who generally agrees with you (+60 or better), you can look at their ratings page to see how well they rate films you are considering.

The IMDb ratings are nothing more and nothing less than the aggregate ratings of people who have accounts on the IMDb. It is probably not representative of the views of the population as a whole nor does it reflect any serious critical assessment of the quality of a film. It is at best a qualitative measure of populist film preferences and at worst entirely misleading. It is highly inappropriate to use the IMDb ratings as a means to assess independent or hobbyist films.

Consider these inherent shortcomings:
1. The IMDb poll is a self selected survey. Self selected surveys are inherently flawed and statistically unsound.
2. Not everyone provides their ages and gender when registering for an account. No one can say with certainty that people provide correct demographic information.
3. Assuming that people provide accurate demographic information, the underlying population of the IMDb membership is not representative of the general population or may not be representative of the population of the film going population. People with accounts on the IMDb are predominately males between the ages of 18 and 29 who live in the United States.
4. Comparing the ratings of any two films are at best problematic since the populations rating each film may have very little overlap. Since this is a self selection survey sampling methodologies cannot be used to make that comparison.
5. There are significantly more votes for recent and heavily publicized films.
6. Initial ratings for recently released films tend to be much higher reflecting the perspective of the film’s fan base and declines in time. For instance, Avatar had a rating of 9.03, a weighted rating of 8.95 and a rank of 21 on 21 December 2009. It now has a rating of 7.98, a weighted rating of 7.92 and is no longer in the IMDb Top 250.
7. The methods that IMDb uses to reduce voting fraud generally do not work well for films with few votes.
Photo of bluesmanSF

bluesmanSF, Champion

  • 10815 Posts
  • 6434 Reply Likes
And, you're misunderstanding the issue. 1's don't count more than other numbers. If it's a rating from a top 1000 voter, it is weighted differently. There is an unexplained weighting system that is the issue.

This conversation is no longer open for comments or replies.