Upcoming improvements in user rating logic

  • 1
  • Announcement
  • Updated 6 years ago
Archived and Closed

This conversation is no longer open for comments or replies and is no longer visible to community members. The community moderator provided the following reason for archiving: Old thread

Early next week (on or around June 24th) we will be introducing some small improvements to the way user ratings are calculated and displayed on titles.

Although the vast majority of titles will be unaffected, the rating on some films (especially those that have received a relatively low number of votes) might change, even if the total number of votes cast does not.

As always, we look forward to your constructive feedback.
Photo of Giancarlo Cairella

Giancarlo Cairella, Official Rep

  • 1107 Posts
  • 1037 Reply Likes

Posted 6 years ago

  • 1
Photo of bluesmanSF

bluesmanSF, Champion

  • 10815 Posts
  • 6429 Reply Likes
Nice! Can't wait to see it roll out smoothly!

Thanks for good news, Giancarlo!
Photo of Emperor

Emperor, Champion

  • 6418 Posts
  • 3002 Reply Likes
Thanks for looking into this and coming up with a solution - it has increasingly become one of the biggest issues here, and if there is a way to address it then it will be greatly appreciated.
Photo of Victoria Masina

Victoria Masina

  • 36 Posts
  • 6 Reply Likes
Yay :) Thank you :)
Photo of Matt Frame

Matt Frame

  • 52 Posts
  • 3 Reply Likes
Perhaps you could be so kind as to apply these changes ASAP to my film, 'No Joke' which has been unfairly victimized by your current (and obviously) crooked rating system?
Photo of Dan Dassow

Dan Dassow, Champion

  • 13078 Posts
  • 13272 Reply Likes
GC, I'm looking forward this change. I hope the roll out goes smoothly.

Without disclosing proprietary information, could you provide some insight how on this will affect titles with fewer than 100 votes?
Photo of Dan Schmidt

Dan Schmidt

  • 4 Posts
  • 0 Reply Likes
This is good news. Thank you for listening to the concerns of film makers.
Photo of Ken Waters

Ken Waters

  • 7 Posts
  • 1 Reply Like
What is going on with the imdb star meter? Will it go back to normal on Monday?
Photo of bluesmanSF

bluesmanSF, Champion

  • 10815 Posts
  • 6429 Reply Likes
Hello.. If so, it's coincidence. That's a completely different subject/issue.

The problem with Starmeter was addressed, as you saw, here https://getsatisfaction.com/imdb/topi...

It it hasn't appeared fixed yet, probably Monday would be a good guess.
Photo of Believe Again

Believe Again

  • 15 Posts
  • 0 Reply Likes
Great news! Thanks Giancarlo! Hope it solves the issue. look forward to see what's happening.
Photo of Matt Frame

Matt Frame

  • 52 Posts
  • 3 Reply Likes
Wow, fantastic! Thank you, guys!
Photo of Emperor

Emperor, Champion

  • 6418 Posts
  • 3002 Reply Likes
I just took a look at your film - 7.4!! Much better. At least looking at that it seems the weighting has been reduced or even removed.

It'll be interesting to see what the patterns are.
Photo of Matt Frame

Matt Frame

  • 52 Posts
  • 3 Reply Likes
Indeed. Thanks for your advice the past couple of days Emperor. Greatly appreciated!
Photo of Victoria Masina

Victoria Masina

  • 36 Posts
  • 6 Reply Likes
It's the average score now for films with low vote count. :) I wonder what the cut-off is. LOL, should we ever get close to the point that the weighed system kicks in: panic... ;)
Photo of Emperor

Emperor, Champion

  • 6418 Posts
  • 3002 Reply Likes
No problem Matt, glad it got sorted out for everyone - you are "lucky" that you ran into this and the solution popped up, a lot of poor souls have been fighting to get this fixed for months!!

It's the average score now for films with low vote count. :) I wonder what the cut-off is. LOL, should we ever get close to the point that the weighed system kicks in: panic... ;)


I think you all should be fine - hopefully, staff have worked out the best cut-off point for this, so that proper votes drown out naysayers to the point it makes no impact. If you look at top rated films you'll see thousands of 1 and 2 votes, but they've reached a critical mass of votes that it is a mere blip - the trick is hitting the sweet point somewhere between. Let's be honest, if IMDB don't get it right I'm pretty sure we'll hear about it on here (and sooner rather than later), so they could fine-tune it if need be.
Photo of Victoria Masina

Victoria Masina

  • 36 Posts
  • 6 Reply Likes
I just checked and mine was changed now as well! :) I'm happy, thank you, I got satisfaction :D
Photo of Victoria Masina

Victoria Masina

  • 36 Posts
  • 6 Reply Likes
Yeah. I don't know, this kinda sucks - but for me on the main page it shows the right score. I posted a pic in a separate comment, can't insert a pic here in a reply to a comment. Of course it doesn't matter that I see the 7.9 if everybody else sees a 3.4 - I'd rather have it the other way around, lol...
Photo of Emperor

Emperor, Champion

  • 6418 Posts
  • 3002 Reply Likes
I am seeing it as 7.9.

I believe the scores on the main page are updated every day or so, so it is possible some new scores haven't popped up everywhere (although it should do by now) or that some people are seeing cached versions.
It is not only the caching, but also seems to depend on the layout setting in your IMDb profile - if that is switched to the old layout, you still get the old scores it seems.
Also, several other pages have not yet been converted, for example on all the "combined details" pages, the old score was still displayed. For example for "Grooming Giselle": http://www.imdb.com/title/tt1670991/c...
Photo of Emperor

Emperor, Champion

  • 6418 Posts
  • 3002 Reply Likes
Ah yes, I updated my comment on the post below - it seems the combined view hasn't been updated and it might be they are on a different updating cycle or frequency. I don't use it (for reasons like this) and the important pages are the front page and the ratings page (most viewers will not be using combined views), both of which show the right ratings. I'm sure the combined view will update eventually and perhaps staff can give it a kick to update a little sooner given the changes.
Yes, you are right. I also think they will change it soon.
Photo of Bilal

Bilal

  • 62 Posts
  • 34 Reply Likes
What has changed? Anybody please tell us the differences than previous one.
Photo of bluesmanSF

bluesmanSF, Champion

  • 10815 Posts
  • 6429 Reply Likes
What I've seen is that the rating shown, instead of the weighted version, is now closer to the mean average.

No one has explained what exactly changed because they don't reveal much because people will cheat the system. But, they've seemingly taken away, or adjusted, much of the weighting until films reach a certain number of ratings.
Photo of Emperor

Emperor, Champion

  • 6418 Posts
  • 3002 Reply Likes
Matt Frame broke the votes down for his film, in a previous thread:

https://getsatisfaction.com/imdb/topi...

It was 3.7 and is now 7.4 with the weighting removed (and 16 more votes):

http://www.imdb.com/title/tt2278120/r...

You can go through the main thread for this issue (now archived I see) and check out how other people's ratings have changed too:

https://getsatisfaction.com/imdb/topi...
Yeah, but apparently the "new" score is only shown on the dedicated rating page? Take Matt Frame's "No Joke" for example: it is true that it now displays 7.4 on the ratings page as the score, but on the main page for the movie, it still says 4.0. Will this also be changed?
Photo of Emperor

Emperor, Champion

  • 6418 Posts
  • 3002 Reply Likes
Not for me. I am getting the new scores on the main page of ever one I've checked.

edit: From the posts below, it looks like the combined view isn't showing the updated rating (another reason I don't use it). Hopefully, this will also change when that page is updated, but the important thing is that the right rating is showing on the main page and the voting break down.
Photo of Victoria Masina

Victoria Masina

  • 36 Posts
  • 6 Reply Likes
@Tobias - this is what the main page looks like:
Photo of Bilal

Bilal

  • 62 Posts
  • 34 Reply Likes
Grooming Giseell is a very bad movie. Please do not promote your movies by giving a 10. At least it deserves 3.4/10 with the older rating system. Now it's 7.9 with the new system. IMDb failed again again and again!
Photo of Victoria Masina

Victoria Masina

  • 36 Posts
  • 6 Reply Likes
You don't even KNOW my movie. Not to mention you don't know how to spell it... So that's interesting, you must be one of the trolls who voted on it. It hasn't been released yet and only a few of the cast/crew plus family and friends have seen it. Everybody loved it. Unless you are somebody with a grudge against a member of my crew or cast I submit you are just a very bitter little troll who campaigns against indie films for twisted reasons only you understand.
Photo of Kay J

Kay J

  • 2 Posts
  • 0 Reply Likes
Bilal
Did you see this movie? You seem one of the troll...
Photo of Jay Storey

Jay Storey

  • 22 Posts
  • 4 Reply Likes
I hope it all goes well Victoria.......most people working in the industry want their peers to succeed, at least those I work with do, but I think this is more about the changes in a process and not individuals.
Photo of Victoria Masina

Victoria Masina

  • 36 Posts
  • 6 Reply Likes
People like Bilal want to play movie god - entitled bullies who feel on a power trip when they watch their one single vote drop a little movie to oblivion. It's not gonna happen for them with the big movies - a drop of water on a hot stone - but with mine he gave it a huge blow when he voted. It must have bin exhilarating. Did you shudder, Bilal? Did you sigh with pleasure? And how did it feel when you suddenly saw the score going up? I'm sitting here with a smile on my face...
Photo of Bilal

Bilal

  • 62 Posts
  • 34 Reply Likes
IMDb mess the rating system up again. You don't use Weighted Average Ratings. You have started using Arithmetic mean rather that weighted one.

You scared of a lot of people because they have always been complaining about their title's bad scores. So that's why you had to change your rating system entirely. IMDb failed again. Sorry for our loss
Guys, would you please calm down?
Photo of Victoria Masina

Victoria Masina

  • 36 Posts
  • 6 Reply Likes
Either Giancarlo and Bilal are the same person or they have a remarkable similar writing "style".
Photo of Kay J

Kay J

  • 2 Posts
  • 0 Reply Likes
Did you see these comments? This person must be the troll! It's weird he upset too much. Besides it's unnatural he has before and after snapshot of short films title.
Anyway GC did a good job. The following person misunderstood. The troll ruined the site by score bombing.

-------------------------------------
> FUCK You ... replied to Upcoming improvements in user rating logic, an update about IMDb.com.
> Fuck IMDB Staff becouse they ruined the site with this fucking ''improvements in user rating logic'' .
> A picture taken this week for Highest Rated Short Films With User Rating Between 7.0 And 10 :
> And now :
> From 1000 to 24000 thanks to IMDB Staff ...now all short movie are 10 rating... by the way the site is worth now O (Zero)!!!
>
> Now to delete my account !!!
-------------------------------------
> FUCK You ... replied to Upcoming improvements in user rating logic, an update about IMDb.com.
> IMDB Had Ruined The Site !!! SHAME on you !!!
-------------------------------------
> Bilal replied to Upcoming improvements in user rating logic, an update about IMDb.com.
>
> IMDb fuck the rating system again. You don't use Weighted Average Ratings. You have started using Arithmetic mean rather that weighted one.
>
>You scared a lot of people because they have always been complaining about their title's bad scores. So that's why you had to change your rating system entirely. IMDb failed again. Sorry for our loss
-------------------------------------
Photo of Emperor

Emperor, Champion

  • 6418 Posts
  • 3002 Reply Likes
It isn't very helpful quoting a troll's posts that have otherwise been deleted - it gives them a little extra attention, which is what they are after.
Photo of Victoria Masina

Victoria Masina

  • 36 Posts
  • 6 Reply Likes
It's very odd indeed. A disturbed individual. I have a theory, and if I may quote myself from my reply to him:

"People like Bilal want to play movie god - entitled bullies who feel on a power trip when they watch their one single vote drop a little movie to oblivion. It's not gonna happen for them with the big movies - a drop of water on a hot stone - but with mine he gave it a huge blow when he voted. It must have bin exhilarating. Did you shudder, Bilal? Did you sigh with pleasure? And how did it feel when you suddenly saw the score going up? I'm sitting here with a smile on my face..."

What do you think? Am I onto something? LOL

PS: My film is not a short, it's a feature film. He seems to think that this is about short films or something.
Photo of Emperor

Emperor, Champion

  • 6418 Posts
  • 3002 Reply Likes
Probably best not to break out the deerstalker and magnifying glass for a round of amateur sleuthing - it never ends well.
Photo of Victoria Masina

Victoria Masina

  • 36 Posts
  • 6 Reply Likes
Hehe, not entirely true, Emperor ;) Miss Marple, Jessica Fletcher, Cadfael, Nancy Drew, Hildegarde Withers, L.B. Jefferies etc. :D
Photo of Emperor

Emperor, Champion

  • 6418 Posts
  • 3002 Reply Likes
I'm more Columbo.
Photo of David D. Mattia

David D. Mattia

  • 18 Posts
  • 4 Reply Likes
Here's the thing -- a lot of actors depend on that score because believe it or not, it does often make the difference between choosing one candidate over another. I used the STarmeter when I cast a part in a pilot I produced -- but in my case -- I am only a writer and a producer so who cares what my score is? Well the answer to that is I DO. I CARE. My score went from 31,000 to 900,000 in one week. There is no mathematical formula on earth that can create a drop like that. I had a very good score for a writer because I am a famous writer OUTSIDE of the film industry and my fans watch what I am doing -- if they watch on IMDB to see if there are more episodes of a show coming out -- who am I to tell them they can't do that? I NEVER tell them to look for me on IMDB. Their INTEREST in my work is a sign of my marketability -- but IMDB does not see that. I think the agenda is to only count searches from people and sites that Amazon sees as worthy. They are trying to push aside the web series people because Amazon can't make money off of them. This a fascist way to run the Hollywood industry. IMDB is no longer a reliable source. It's a tool for Amazon.
Photo of Emperor

Emperor, Champion

  • 6418 Posts
  • 3002 Reply Likes
This thread is about votes on films, the thread about Starmeter issues is here (see the post from staff in particular):

https://getsatisfaction.com/imdb/topi...
Photo of bluesmanSF

bluesmanSF, Champion

  • 10815 Posts
  • 6429 Reply Likes
Fascist? It's a ranking and if you went down, someone else went up. Can you point out that those ahead of you don't belong there? No? Then how can you complain it's not accurate? You can't. You're just unhappy it went down.

Now, read this thread. It's not even about the subject you're talking about. So, why are you posting here??
Photo of Victoria Masina

Victoria Masina

  • 36 Posts
  • 6 Reply Likes
I think the star meter thing was some problem that has been fixed. I remember someone mentioning something like that here on the board... Amazon has now its own studio and is going to produce shows and movies, I wonder how that will change the dynamic once they've produced something.
Photo of Giancarlo Cairella

Giancarlo Cairella, Official Rep

  • 1107 Posts
  • 1037 Reply Likes
This is unrelated to the main topic being discussed Please reference the new topic here: Starmeter drop
Photo of David D. Mattia

David D. Mattia

  • 18 Posts
  • 4 Reply Likes
Listen, don't be snide -- I am on here because that is where and email from IMDB sent me -- also -- I am not resentful of the success of others -- actors lives have no effect on my life for the most part. My point was that there has to be something wrong with a system that rises a writer (me) from 30,000 to 993,000 in one week when in fact I still have the same fan base -- if not a bigger one -- that was the point. It did not require a snide or rude response. That kind of attitude will not serve you well in life. Anyway, they say the problem has been fixed -- fine -- but it hasn't been fixed, it has been changed and there is no way to know why it has changed other than to assume that they have sold out the way a score is determined to another company. It's no secret.
Photo of bluesmanSF

bluesmanSF, Champion

  • 10815 Posts
  • 6429 Reply Likes
An email didn't send you to a thread about the user ratings system to post about StarMeter. And, if it did, you certainly can see it would be inappropriate and refrain from doing so, yourself, could you not? Just because someone sends you somewhere, you are not obligated to post comments there. Improvements in how user ratings are displayed has nothing to do with changes in the StarMeter feature. Just because you're not happy with your rating doesn't mean the system hasn't been fixed, either. If they're using different criteria because there were hundreds, if not thousands, of instances of people with very little going on, getting high ratings, I'd say it's a fix. But, what's it got to do with how users have rated a film?

Thanks for the lesson in life. I am sure you know all about life and not being able to navigate a message board thread is just a blip on an otherwise spotless screen for you.
Photo of David D. Mattia

David D. Mattia

  • 18 Posts
  • 4 Reply Likes
I clicked on the email and this is where I am again. I don't give a shit about my rating -- I am trying to point out something you do not know -- it is IMPOSSIBLE for a ranking to go from 30,000 on average to nearly 1,000,000. So if mine can drop that much, all the others who are complaining are justified in doing so. When you have 879,000 readers looking at your work each month, give me a call. Maybe I can give you a shout out and see what it does for your IMDB score -- and your attitude too.
Photo of bluesmanSF

bluesmanSF, Champion

  • 10815 Posts
  • 6429 Reply Likes
Why would I have an IMDb score? I don't write...why would I have "readers?" What does any of this have to do with user ratings on films? Re: "if mine can drop...the others are justified..." because the world revolves around you, right? No thanks on the "shout out," dude. I don't want your name attached to anything I ever do.

Is this your work?

Trotting Rachorse Trainer David Mattia Ditches Horses and Moves to Hollywood.

David Mattia – Have You Met Miss Jones?

Writer David Mattia, a ntive of Newark, New Jersey, ...


Rachorse? ntive?
Photo of David D. Mattia

David D. Mattia

  • 18 Posts
  • 4 Reply Likes
Hey -- I don't care what people write about me as long as they spell my name right. Don't be a jack off -- please. Stop. I never write about myself -- so it can't be mine but I like that somebody wrote it. Just stop. Be a grown up and stop.
Photo of David D. Mattia

David D. Mattia

  • 18 Posts
  • 4 Reply Likes
...and thank you -- I did not know that I had an unauthorized website. I am happy to see that -- The picture is nice. Obviously that's a fan who created that page and I think it's a nice attempt -- and also you have to take into consideration that 75% of what is written about me is translated from French into English -- give them a break --- okay?
Photo of Believe Again

Believe Again

  • 15 Posts
  • 0 Reply Likes
Thx, looks better than before, though still spam votes remain.
Maybe everyone knows this is not the real solution.

Dear Col Needham and Official Rep of IMDB,
If you really wants to provide a fair rating system, you could think about following idea.

1) Identification
You can take an identification procedure using a credit card or something before people vote. Then people can vote only one time at least. It will reduce the spam/self multiple votes.

For example, I got multiple "1" votes on the same day, from the same profile. (non-US user etc.)
Obviously it seems from the same person. We can avoid this type of issue.

2) Write the reason of voting
Put people under an obligation to write the reason why they vote so. What point is good or bad. It will help the audience to understand how it's like even from the internal vote.
Amazon reviews are juggled, but still it's better than current IMDB system, because of comments and identification.

1) and 2) will reduce the number of voting, but the voting quality will be better.

3) Show only limited people the score(rating), if it's low number
If it's under 100(or 500) votes, you could use hidden score(rating) board. People can see the score(rating) of a small number of votes using specific url parameter.
(For example, http://www.imdb.com/title//?h-score=on etc. or unique id that the producer can generate)
This score(rating) doesn't show up on any "top XXX" lists.

The audience of the most small independent films is mainly the cast/crew, their family and friends. They tend to give a "good job" score(rating) on the movie. It's natural, but I know it's not helpful to the regular audience.

I think the star number for the small independent film is almost for the self satisfaction, for their confidence and for the promotion.
Film maker can use the link with parameter on their HP or when they send it to the distributors or write on their resume.
But the regular audience doesn't see usually.

4) Show who vote what movies
You could show the information that who vote what, like book reviews. Don't need to show personal information. It will help the audience to judge whether the review is reliable or not, whether it matches to their preference and keep off spam votes.

5) Remove "Top 1000 voters"
At least, it is obviously wrong way to weight the spam votes from who has never seen the movie but just give spam votes randomly, and it's also bad to take the vote lightly from who saw the movie but doesn't vote a lot.

You don't need to weight Top 1000 voters and control the number of 10 score(rating) any more, if you get adequate system.
You may be able to calculate the score(rating) using the standard deviation.

6) Delete "Top XXX" list and show different list
You may disagree about this. But if you quit to show "Top XXX movies", less people will try to juggle the score(rating). Juggled top XXX movies dissatisfy both audience and filmmakers. Instead of "Top XXX movies", you could provide another personalized list based on the preference. People don't need score(rating) fighting anymore.

7) Rate each element (option)
People can rate story, sound, camera work, music, lighting etc. each element of the movie. Producer can see them and choose whether they make it public or not.
It will very helpful for the positive filmmakers. We can know what point should be improved.
Also if the producer make it public, it will lead people better understanding.

These are my ideas.
It's very important to satisfy not only regular audience but also filmmakers.
Remember, one of the important customers and financial resources of IMDB is independent filmmakers.
They use withoutabox, IMDB pro and may pay for advertisement in future.

I believe every film maker made a vigorous effort to make the movies. They should not be ruined by trolls. Every movie should be treated respectfully.
It's time to think about renewal of the rating system.

The current system is already running. If you have a concern about system migration or anything, contact me. (http://lnkd.in/KwqNXj) I can give you more ideas. (I used to conduct a consultation for company's IT systems.)
I hope future renewal of IMDB make all filmmakers and audience happy.
Photo of Bilal

Bilal

  • 62 Posts
  • 34 Reply Likes
Will there be anything else? You ask too many things. Nothing will change, don't worry about that!
Photo of bluesmanSF

bluesmanSF, Champion

  • 10815 Posts
  • 6429 Reply Likes
i disagree with bilal that nothing will change....most of that will make people stop voting. Require a credit card and people will lose their minds!

I also don't see why a film maker should think he should be exempt from trolling when no one in the rest of the world is. People need to relax a little and have some common sense. One or to bad votes? Really??? How many bad ratings does President Obama get on any given day? I saw a baseball player opt not to dive for a ball he had no chance to catch the other day and thousands of people booed him for making a good play. I've seen police and firemen get bottles and rocks thrown at them. Really? A low vote? Oh, the humanity!
Photo of Believe Again

Believe Again

  • 15 Posts
  • 0 Reply Likes
I think so, bluesman. If IMDB was a new site, it's not a good idea to require a credit card. But already the site has a lot of users, got tons of votes including spam votes enough to make down on its credibility.
At least IMDB Pro users were already indentified. No change for them in the point of their view.
It would be tough for independent filmmakers to get votes even from their friend. But better than getting a lot of spam votes.
Obviously this idea would be bad for voting business companies.
Photo of Believe Again

Believe Again

  • 15 Posts
  • 0 Reply Likes
oops, "No change for them in the point of their view. " -> not much change
Photo of bluesmanSF

bluesmanSF, Champion

  • 10815 Posts
  • 6429 Reply Likes
That, I think, was another part of the weighting being unfair is that lower weight was given for accounts with no demographic data. This was probably the friends/family joining just to vote for one film. It makes up third to half the votes on many/most of these. If you can't get them to even show what country they live in, good luck getting a credit card. Removing weighting for films with low votes is a huge step. One thing no one has been considering is that the issue was only "weighted score" and anyone in the industry trying to use data or anyone truly interested could still still see the actual votes...I just thought they should just switch and display the real numbers and let those interested in seeing weighting, click to see it.
Photo of Emperor

Emperor, Champion

  • 6418 Posts
  • 3002 Reply Likes
There are similar (if less controversial) suggestions:

https://getsatisfaction.com/imdb/topi...

https://getsatisfaction.com/imdb/topi...

Feel free to make those suggestions (although you may want to split them up as there are a variety addressing different aspects) and see what gets support.

This conversation is no longer open for comments or replies.