"Titles you rated higher:" sometimes not

  • 1
  • Problem
  • Updated 1 year ago
  • In Progress
I visited this page:
http://www.imdb.com/user/ur2495297/
and looked at the ratings comparison between us, and saw this:

My ratings, on the right, are correct. Unfortunately, they're only higher than the other user 2 out of 5 times. (The Titles you rated lower were fine.)

Since we can't Refine or Export the other user's ratings, I have no reasonable way of knowing if these numbers are their ratings.

I don't see this with every user I visit, but not just this one. Also bad: 
http://www.imdb.com/user/ur8532993/
http://www.imdb.com/user/ur3975509/

My user id is ur2875687

This happens on Firefox and Chrome on Windows 10; scripts/extensions off.
Photo of bderoes

bderoes, Champion

  • 995 Posts
  • 1338 Reply Likes

Posted 1 year ago

  • 1
Photo of Col Needham

Col Needham, Official Rep

  • 6834 Posts
  • 4809 Reply Likes
Thanks for the problem report. We have opened a ticket with the appropriate team.
Photo of bderoes

bderoes, Champion

  • 995 Posts
  • 1338 Reply Likes
Just for the record, this is still happening on all 3 of these user pages.
Photo of Col Needham

Col Needham, Official Rep

  • 6834 Posts
  • 4809 Reply Likes
It's not actually bug so the ticket is to better explain how things work, which I can do here until the page is updated ... from a recent email from the team responsible for the feature: 

The algorithm does not compare ratings on an absolute basis because one person's "7" might not mean the same as another person's "7".  For example, from a real extreme case which illustrates the situation well: There's an IMDb customer with 1,000+ ratings, all but 29 of which are "10". So for them to rate a title 9 means that they actually thought it was a below-average film, whereas most other people's rating of 9 would represent an exceptional film.
Photo of bderoes

bderoes, Champion

  • 995 Posts
  • 1338 Reply Likes
So my 9 is higher than ur2495297's 10? That's what the screenshot shows above for both Sunset Boulevard and The Artist.

What mathematical theory is used to justify that? Something about percentiles, I suspect.

While I agree that ur2495297's distribution skews much, much higher than mine, and I've rated 10 times as many films, it doesn't seem valid at all to decide that ur2495297's 10 in these cases is a score below perfection, much less a score below my 9.

For ur2495297, a 10 puts a film in the top 20%.
For me, a 9 puts a film in the top 2% (my 10's amount to 0% (12 / 3671 = 0.3%)).
But who's to say those particular films aren't in ur2495297's top 2%?