'Weighted' Average System Clearly Corrupt?

  • 2
  • Question
  • Updated 6 years ago
  • Answered
Archived and Closed

This conversation is no longer open for comments or replies and is no longer visible to community members.

It's somewhat amazing to me that a filmmaker such as myself can hold an online screening for my indie documentary (in my case 'No Joke') in conjunction with having our film festival premiere (this past week @ NXNE) as means to promote viewers to provide feedback and reviews of the film (good or bad) and yet after 13 votes with an arithmetic mean of 8.3, median of 9, the current rating sits at 2.9. Clearly this system to root out corruption is horribly corrupt in itself. Why, just because several votes and reviews come in over a short amount of time, that is reason to disregard them as 'suspect?' Just because we have ASKED people to review and vote on the film does not mean that we have asked them to review and vote UNFAIRLY. We have NEVER asked any viewer to give an inflated vote or review. To be treated as such is an absolute insult. It's difficult enough to find the energy and finances to complete a indie production such as 'No Joke' but at the very least one should be able to expect that after the years of work their work will be treated and judged FAIRLY. In regards to IMDB, this obviously isn't a reality. So, many thanks IMDB for FAILING dismally to give the people who took the time to review and vote on my film a fair chance for their collective voices to be heard. Please enjoy the countless dollars I have pumped into your company on continued frivuloucities as it's fairly obvious that you are not using them to design a review and rating system that works. Shame on you!
Photo of Matt Frame

Matt Frame

  • 52 Posts
  • 3 Reply Likes
  • frustrated

Posted 6 years ago

  • 2
Photo of Petar Gagic

Petar Gagic

  • 1 Post
  • 1 Reply Like
Just because something is getting a generally positive rating, it automatically means it must be a scam? WHAT?! Fuck that. I watched No Joke at 4 in the morning, and I wrote the review right after watching it. I loved it so that's what I wrote in my review, that's why I gave it a positive rating. People are giving their honest opinions and they're being denounced, that makes no sense to me.

IMDB, get your shit together.
Photo of Dan Dassow

Dan Dassow, Champion

  • 13731 Posts
  • 14186 Reply Likes
Although you have encouraged people who have seen your film to give a fair rating of your film, your film only has 13 votes. The weighted average vote for your film is effectively meaningless. Much fewer of these votes, those from regular voters, are counted for the weighted average.

The IMDb ratings are nothing more and nothing less than the aggregate ratings of people who have accounts on the IMDb. It is probably not representative of the views of the population as a whole nor does it reflect any serious critical assessment of the quality of a film. It is at best a qualitative measure of populist film preferences and at worst entirely misleading.

Consider these inherent shortcomings:

1. The IMDb poll is a self selected survey. Self selected surveys are inherently flawed and statistically unsound.

2. Not everyone provides their ages and gender when registering for an account. No one can say with certainty that people provide correct demographic information.

3. Assuming that people provide accurate demographic information, the underlying population of the IMDb membership is not representative of the general population or may not be representative of the population of the film going population. People with accounts on the IMDb are predominately males between the ages of 18 and 29 who live in the United States.

4. Comparing the ratings of any two films are at best problematic since the populations rating each film may have very little overlap. Since this is a self selection survey sampling methodologies cannot be used to make that comparison.

5. There are significantly more votes for recent and heavily publicized films.

6. Initial ratings for recently released films tend to be much higher reflecting the perspective of the film’s fan base and declines in time. For instance, Avatar had a rating of 9.03, a weighted rating of 8.95 and a rank of 21 on 21 December 2009. It now has a rating of 7.98, a weighted rating of 7.92 and is no longer in the IMDb Top 250.

7. The methods that IMDb uses to reduce voting fraud generally do not work well for films with few votes.
Photo of Matt Frame

Matt Frame

  • 52 Posts
  • 3 Reply Likes
Well, it's always nice to hear that the current system 'generally' work well in other instances Dan (and I do appreciate you posting and with such detail) but it is fairly obvious that in the case of my film, 'No Joke' the system has failed dismally. I am in the process of contacting all those who may have reviewed the film (obviously I don't know all of them personally) and are asking them to come to this thread and verify that a) their review was genuine and b) the score they gave was not in any way inflated as a result of interference on my part. On top of that, I am posting several messages on the 'No Joke' Facebook page which has over 1,400 followers to fill them in on the details. Hopefully more folks will feel compelled to come forward to verify the authenticity of the reviews. Cheers, Dan.
Photo of Critic

Critic

  • 1 Post
  • 0 Reply Likes
While I understand that we live in a completely corrupt world and cannot trust anything that anyone does or says, I would like to know what the point is in having a 10 star rating system, huh IMDB??!

I watched Mr. Frame's film at the "Hot Docs" theatre in Toronto. It was this awesome little film's world premier. I took notes immediately following the presentation and then came here to write a thoughtful and, hopefully, witty review of a wonderful film. Mr. Frame in no way asked me to write a review. I WANTED TO DO EVERYTHING I COULD TO SEE AN INDEPENDENT FILM SUCCEED, and I thought that the IMDB was a place for that. Apparently not??

How about taking the time to write to the participants of the reviews, rather than just ASSUMING that they are corrupt??! That may be a better idea.
I am completely vexed by this and DEMAND that you restore my 9 star review of "No Joke".

Figure it out, IMDB!!!
Photo of Alanna Kazba

Alanna Kazba

  • 1 Post
  • 0 Reply Likes
Can the film lovers that monitor the ratings of imdb please watch the movie trailers on YouTube and understand all the hard work that was put into this film and the obstacles that have already been put up preventing it to be the success it should be? Only then will the imdb rating monitors, that lowered the rating to a much lower, inaccurate rating, realize that they are becoming part of the problem as well. This is a beautiful film that deserves all the noteriety and praise that it can get. I understand it probably seemed fishy as we all rated around the same day but we had an online password protected screening that was offered to everyone interested (actually I'm pretty sure Matt the director offered it to everyone on Facebook as it was in a Facebook event) and the password for the online screening is only valid a couple days or so.. I know imdb is truly interested in proving an accurate and Just description and rating so I understand the situation, however please be as concerned with fixing the mistake as you were flagging movie lovers trying to promote a film thats success was unjustly blocked so much already. Contact me for further proof or verification that I infact am a human being that participated in the online screening and wrote a review to promote the film. Yes, people can love a film that much and the people on the film and always want to see it's success despite setbacks. Truth be told that movie Kind of saved my life, I can show you the original Facebook conversation where I told Matt that, and later talked to a comedian in the movie. It allowed me to see the humor in life and the beautiful truth which is sometimes very sad. The film was perfect for me on the day I watched it because I was home sick dealing with an illness doctors haven't diagnosed yet, which effects my brain and I needed something life REAL and funny to take my mind off the terrible thoughts I was thinking. Imdb is this a joke? It was my drug addicted mother that taught be you have to find humor in the tragedy or you will go insane. So the movies right up my alley! I hope we were laugh about all this later IMDB.

Please don't inaccurately Rig ratings to your liking!!!
Photo of Emperor

Emperor, Champion

  • 6418 Posts
  • 3004 Reply Likes
See this thread for the main discussion on this:

https://getsatisfaction.com/imdb/topi...
Photo of Shawn Bordoff

Shawn Bordoff

  • 5 Posts
  • 2 Reply Likes
As the producer of No Joke I attended the North By North East screening on Saturday June 15th. I did a Q & A with Amanda Edgar-French, the co-producer. We encouraged people to review the film on IMDB. Even though I felt that the reviews would be positive since it's a great documentary, I did nothing to influence anyone to be more favorable. Those reviews are the opinions of the viewers and have no association with the production or crew of No Joke. For anyone who is suspicious of the IMDB ratings, please contact me at media@nojokemovie.com and I will provide you with a Vimeo screener so you can actually watch the entire movie for yourself.
Photo of Duncan

Duncan

  • 1 Post
  • 1 Reply Like
Hey there. My name is Duncan and I didn't give the film a super-glowing review or high score. I didn't trash it either, though, because I think the film has value and is somewhat worth watching if you're into a documentary about average comedians. I barely know the film makers. They IN NO WAY told me to rate the film highly. They were very clear about rating the film honestly. They went to a lot of trouble to contact people and told them to rate the film honestly because the more ratings, the more traffic, and more exposure. I reiterate, they INSISTED on an honest review. INSISTED. This was not an attempt to 'stuff the ballot box'. Your rules need review, IMDB. I understand the problem you might have with indie filmmakers 'gaming the system' online but I assure you, this is not one of those cases. They tried to co-ordinate everyone and get honest reviews up quickly and now it seems like they're being punished for actually being on top of things. It isn't right.
Photo of Ashley Katherina Hunking

Ashley Katherina Hunking

  • 1 Post
  • 1 Reply Like
I was one of the reviewers that gave a positive review of "No Joke" and I stand behind every single word I wrote. I was never asked to give an inflated review of the movie. IMDB has SERIOUSLY failed this movie. I don't really know what to say, except encourage others to see the movie???!! But...that doesn't negate the fact that "No Joke" is clearly being penalized for a system that is corrupt (or can't live up to it's mandate - Internet Movie Data Base) and through your comment above, YOU ALSO CONCLUDE IS CORRUPT, Dan! You say, "It is probably not representative of the views of the population as a whole nor does it reflect any serious critical assessment of the quality of a film. It is at best a qualitative measure of populist film preferences and at worst entirely misleading."

WOW. This is all very upsetting because this film deserves the reviews it's getting, but you conclude it can't possibly reflect any critical assessment. That is IMDB's fault not the movie or the filmmaker.

IMDB should rethink their business model, if it can't support its users in the way the claim to...
Photo of Emperor

Emperor, Champion

  • 6418 Posts
  • 3004 Reply Likes
As DD points out, this is a problem because of the low number of votes it has, which means higher weighted votes will completely throw it off (especially as the weighting is already, presumably, compensating for what it might think is vote stuffing - so you get the double whammy).

As you get more viewers this should even out. I'm not sure what you distribution plans are, but I'll keep an eye out for it.
Photo of Stephen Spinola

Stephen Spinola

  • 1 Post
  • 1 Reply Like
I am going to add my review now. I hadn't planned to review the film originally but these guys have been through enough! Between law stuff and this garbage I think these guys need a break. I enjoyed the film both times I watched it. I wouldn't give it a 10/10, but it was definitely enjoyable. Anyway, I am going to go review it now and I hope IMDB gives you guys some of those points back.
Photo of Nick Carefoot

Nick Carefoot

  • 1 Post
  • 0 Reply Likes
Sorry but I hate movies hence why I use to work at a movie rental store I use to RESPECT IMDB but now I see what you guys do to indie movie makers this is GROSS. Time to start using rotten tomatoes!

At least they don't censor reviews, censor scores and harass + belittle amateur productions with no budget. You guys are disgusting!
Photo of Chris Allen

Chris Allen

  • 1 Post
  • 0 Reply Likes
I'd like to weigh in only to say that I did review the film and that Matt did ask those who have seen it to review it. He never influenced my rating in any way though.

This is a cut and paste from his message to me:

If you would like a password to view the film for free (in exchange for maybe writing a brief, honest review on IMDB) then drop me a message and I'll fire one off to you.

So he couldn't have been more plain about just asking for an honest review. Hopefully we'll see more votes out there so IMDB will just take the votes as is. The fact that is shows a small amount of votes is enough for any website user to figure out on their own this is a small sample group, you don't need to further weight it down while showing a small amount of reviewers. Gives the impression that a handful of people rated the film very poorly which is just not the case.
Photo of bluesmanSF

bluesmanSF, Champion

  • 10815 Posts
  • 6429 Reply Likes
While it's nice of you to mention, it's pretty much irrelevant to the situation. Especially in light of the news that they are changing the system because of complaints such as this.

See : https://getsatisfaction.com/imdb/topi...
Photo of Matt Frame

Matt Frame

  • 52 Posts
  • 3 Reply Likes
...and hold the phone. Why is the thread considered 'answered?' Because Dan Dassow has deemed it to be so? What is extremely interesting is that CLEARLY the same system that allows the voice of the majority to be muted by a single 'elite' allows for Dan Dassow (a rating bomb apologist) to determine that he has solved the issue. If you search 'rating bomb' on this IMDB community forum you will see Dan Dassow repeating the same pointless trip as he did above. Over-explanation is clearly a clear indicator of guilt and it doesn't take a rocket scientist to see that Dan Dassow is the damage control that IMDB has employed to simply bore the issue into submission. Thanks Dan!
Photo of bluesmanSF

bluesmanSF, Champion

  • 10815 Posts
  • 6429 Reply Likes
Thank you. I would have expected it to appear in the call out thread. But glad you said it. He dedicates a LOT of time and effort to help the site and its users, day in and day out. And, though it's clear the post you made came from frustration, to push it on Dan was not called for, in any imaginable way. I am glad you posted that.

Thanks for linking it.
Photo of Matt Frame

Matt Frame

  • 52 Posts
  • 3 Reply Likes
No worries. I agree with you that it was uncalled for and I am going to try to have the thread deleted altogether but I am needing to have somebody with administrative rights to do that for me. Do I just need to create a new thread asking for that to occur?
Photo of bluesmanSF

bluesmanSF, Champion

  • 10815 Posts
  • 6429 Reply Likes
Hmm, I'd say it might get deleted when seen anyways. Though, they might also, just merge all these threads on the same topic, then mark it all resolved and "archive" the whole thing, taking it off the active board. But, unfortunately, staff actions slow considerable on weekends (especially Sunday), so it might not happen until Monday. I'd say we leave it alone for a while and let it sink down the page as new topics come. They'll probably see your mentions of deleting and just do it based on what's there.

Let's just keep fingers crossed that the fix is a good one and rolls out smoothly! It's likely to hit some glitches, but hopefully it works well!
Photo of Dan Dassow

Dan Dassow, Champion

  • 13731 Posts
  • 14186 Reply Likes
Matt,

I saw you apology in the indicated thread and accepted it.

Like bluesmanSF, I hope that the fix corrects the underlying problem and the roll out goes smoothly.
Photo of Matt Frame

Matt Frame

  • 52 Posts
  • 3 Reply Likes
Whoops, I started a new thread requesting that the offending one be deleted. I'll leave it be and hopefully it'll all sort itself out. Thanks bluesmanSF for steering me in the right direction here. I am relieved that this is working itself out. If you change your mind and would like to view the film then I would be very happy to provide you with the online screener details. My email is media@nojokemovie.com.
Anyways, take care and, again, thanks :)
Matt.
Photo of Shawn Bordoff

Shawn Bordoff

  • 5 Posts
  • 2 Reply Likes
We realize that our film has yet to be released and has only been seen by relatively few audience members. We only ask that anyone who rates or reviews No Joke to do so once they have seen the movie and to be honest in their evaluation.
What has hurt us is not that there are less than 30 ratings but that few that count are blind bombs. Regular, registered users have given the film a 1 or 2 rating without actually viewing the film. That really is unfair.
Photo of bluesmanSF

bluesmanSF, Champion

  • 10815 Posts
  • 6429 Reply Likes
That's good, but not really the issue. It's not those votes or reviews that are the problem but the weighting system itself, which works poorly for films with low vote totals. You're doing the right thing by getting people to see it and vote. That said, the site has announced that they've now changed what we'll see for these films and that the change will take affect early next week. So discussing it is meaningless until we see what this change is/does.

Fingers crossed!

https://getsatisfaction.com/imdb/topi...
Photo of Gerald Gerald Geraldson

Gerald Gerald Geraldson

  • 3 Posts
  • 0 Reply Likes
I wrote my review for no other reason than loving the film. I've seen it three times and I wasn't asked to write any sort of promo for the project. I'm really kind of outraged by this weighted systems flawed logic. So all the reviews are pretty positive so something must be up? If this system represented a real human mind it be paranoid delusional.
Photo of bluesmanSF

bluesmanSF, Champion

  • 10815 Posts
  • 6429 Reply Likes
We get it. You all decided, on Fb, to "stir shit."

But, really...why waste your time when, after many months of discussion about this, a fix has been announced??? You seem insane ranting while you don't know yet what the fix looks like. All this ranting makes the title you're talking about seem highly suspect. The only time I've seen anything like this, accounts got closed and IMDb credits removed, lists removed, users got blocked from the site, etc... Rant on...let's see where it gets you.
Photo of Matt Frame

Matt Frame

  • 52 Posts
  • 3 Reply Likes
BluesmanSF, I think that these folks do have a legitimate argument given that they did take the time to view and then review the film. Clearly I have stoked the fires a tad by informing them of the vote bombing but, again, like their reviews, their responses are completely organic. We have not asked anyone to do anything other than to express their HONEST opinion about the film, and now, the vote bombing. I think it's a tad disrespectful to make an assumption that these folks don't have a mind of their own on this thing. They're upset that their votes and reviews have (seemingly) been trumped by a voter or 2 who clearly have not seen the film and are bombing out of spite. At this point it is not really helpful to continue to attack these folks, in my opinion. Dan has been given the password to our online screener so he can make the determination for himself if the film has been a victim of a vote bomb. If my apology and subsequent agreement to let sleeping dogs lie was good enough for Dan then I would hope it is good enough for you. Cheers, Matt.
Photo of Emperor

Emperor, Champion

  • 6418 Posts
  • 3004 Reply Likes
Other people have been able to get the problematic vote removed. Obviously, it isn't open season for anyone to remove low votes, but staff have clearly looked into previous reports of this kind of activity and removed the vote if it is part of a pattern of 1 star votes.

Post the details and a link to your film in the main thread:
https://getsatisfaction.com/imdb/topi...

and over on the main thread for this on the IMDB forum (calmly):
http://www.imdb.com/board/bd0000041/n...

No guarantees (and if they are working on a fix they might not be following this up now), but it is your best bet for getting this fixed.
Photo of Matt Frame

Matt Frame

  • 52 Posts
  • 3 Reply Likes
Will do. Thanks for his Emperor.
Photo of Peter

Peter, Champion

  • 6440 Posts
  • 7756 Reply Likes
On the other hand, Giancarlo has said that individual votes can't be removed: https://getsatisfaction.com/imdb/topi...
Photo of Matt Frame

Matt Frame

  • 52 Posts
  • 3 Reply Likes
Really? Ugh. That's too bad. I just spent half an hour writing a new thread to ask for precisely that. Oh well, the IMDB vote bomb journey continues, I guess....
Photo of Emperor

Emperor, Champion

  • 6418 Posts
  • 3004 Reply Likes
It is interesting that he says it, but people here and on the IMDB thread have had these votes removed. I don't know what the process is or if they are still doing it, but it that is the eventual outcome.

I just spent half an hour writing a new thread to ask for precisely that.


There is no point starting a new thread if there are already threads on a topic already on here - it has been flagged up here and could be flagged up on the main threads for this. If staff are/were monitoring them they'll see them, whereas a separate thread will just disappear. The system here really encourages people to take a collective approach, as it helps get your voice heard (and, as they are working on a fix, it has also got results).
Photo of Matt Frame

Matt Frame

  • 52 Posts
  • 3 Reply Likes
Good advice and heeded. Thanks Emperor....
Photo of Gerald Gerald Geraldson

Gerald Gerald Geraldson

  • 3 Posts
  • 0 Reply Likes
I wrote my review for no other reason than loving the film. I've seen it three times and I wasn't asked to write any sort of promo for the project. I'm really kind of outraged by this weighted systems flawed logic. So all the reviews are pretty positive so something must be up? If this system represented a real human mind it be paranoid delusional.
Photo of Matt Frame

Matt Frame

  • 52 Posts
  • 3 Reply Likes
UPDATE: In the 24 hours since I began to publicly call attention to the inital vote bomb of my film 'No Joke' (and the problematic IMDB voting system in general) we have had 2 more. I am publicly asking IMDB staff to determine if these 2 new voters (and the original) even were in the same city as our film festival premiere last week. If not, then I respectfully ask that these users immediately have their accounts suspended as they clearly have not seen the film and have done such out of spite.
Photo of bluesmanSF

bluesmanSF, Champion

  • 10815 Posts
  • 6429 Reply Likes
Airplanes...look into it. I am thousands of miles from my home postal code. Should all my ratings be thrown out? Funny how you state your 10's can have their opinion, which might be wrong, but someone giving any less is a ballot-stuffer who hasn't seen your fine film. Incredible.

Further, it's possible to not have a postal code entered...or a wrong one entered.

Didn't some of your voters watch it over the internet? Should those be removed if their postal code doesn't match yours?

Only 14 of your voters even list a country, let alone a postal code. Pinpoint the others, Einstein.

But thanks for the lesson, Chief.

You have 28 votes, and only "5" is not represented. Your votes are about as evenly distributed as could be possible considering the first few are usually 10's (and usually not from IMDb regular users or voters).
Photo of Matt Frame

Matt Frame

  • 52 Posts
  • 3 Reply Likes
(sigh.)

Your argument is that MDB has no way of determining the location of a voter? They do. As mentioned earlier there is something called Geotargeting via IP Spidering. Each and every visitor to IMDB will be tracked and as much info from the visitor will be lifted, down to the size of the display of their monitor. This is used for marketing, analytics, and fraud prevention among others. IMDB is owned by AMAZON and to even suggest that they wouldn't take advantage of your personal info for monetary gain via geo-targeted marketing is laughable. All IMDB would need to do would be to check the IP spidering info of the voters who have given the film a gutter rating. If their location at the time of the review was outside, say a 500 mile radius, pretty safe to say they weren't in Toronto and therefore did not view the film.

It's called Geotargeting. Maybe you should look into that.

As for your assertion that the votes are spread out evening just because most of the numbers have been used? Man, you keep digging bigger and bigger intellectual holes for yourself. Obviously in your world, if there are 10 basketball players, 7 of which are ten feet tall, 3 are nine feet tall, 6 are eight feet tall, 5 are seven feet tall, 2 are 6 feet tall, one is four foot tall, 2 are three, 1 is two feet tall, and 1 is one feet tall this makes the average height of all the basketball players...5 foot?

Wow. Now that's embarrassing.

Honestly man, given the weakness of your arguments, most people would come to the natural conclusion that you are not a very smart human being....at all. Like, really not smart. Me? I am still holding out hope that you'll bring some rationality to this ongoing conversation and that a solution to the vote bombing of my film can be found.
Photo of Shawn Bordoff

Shawn Bordoff

  • 5 Posts
  • 2 Reply Likes
None of the votes should be thrown out if all were counted equally. Most of the reviews had positive ratings. These are real people that watched our film. The single vote from a top-1000 user is a 2. That is the vote that has all the bearing in No Joke's rating due to a broken weighted system. We sincerely doubt that user even watched the film but just blindly bombed it. That is just wrong. If you can't see that bluesmanSF, I feel sorry for your friends and family.
Photo of bluesmanSF

bluesmanSF, Champion

  • 10815 Posts
  • 6429 Reply Likes
What makes you think I can't see that? I've posted hundreds of times that this is a problem. So, i can't imagine where you're getting that. I've never said anything that could, in any way, come close to saying that "top 1000" voters should have additional weight in regards to "weighted average" displayed.

They are all counted equally, by the way. Just not in the "weighted average" which is a useless statistic that should be, in my opinion, available if search, but not advertised ahead of or in place of actual votes. And, if there is to be a rating feature, no ratings should ever be removed without proof of abuse. Guessing someone didn't see a film because they rated low is not proof.

Again, why are you guys knocking yourself out posting about what might be a moot point? If anything is "wrong" it's that flooding the boards about a non-issue should be allowed.
Photo of Matt Frame

Matt Frame

  • 52 Posts
  • 3 Reply Likes
What choice do we have? You keep making sweeping statements and assertions about our legitimate complaint that our film was vote bombed...but then complain bitterly that we are flooding the board by responding to them?

Wow.

What I cannot fathom is why you are so hell bent on attacking us while defending a system that you have repeatedly acknowledged is broken, hence why is it getting fixed. What on earth is the point to your continued arguments, other than needing the last word on the subject now that your feelings are a little bruised?
Photo of David DJ Roy

David DJ Roy

  • 1 Post
  • 0 Reply Likes
As one of many people who has fell victim to Matt's DeFrame's famous cyber bullying campaigns. I can truly say that there is more made up drama about this movie then there is in the actual movie. Matt is the little "producer" that cries wolf time after time. Matt love to get fixated on a small detail and then blow it way out of proportion. So please take him with a gran of salt. As for the movie it is a very well edited reality TV style piece of whatever. It's up there with The Jersey Shore and The Housewives of (Pick a city). This movie is Matt's way of force feeding you his byist opinions then tries to pass them off as facts by calling it a documentary. It is just sad that The "No Joke movie" has turned into a joke...

This conversation is no longer open for comments or replies.