Addressing the score-bombing cyber attack on the IMDB site

  • 1
  • Question
  • Updated 6 years ago
  • Answered
Background.
It is easily verified that IMDB has been experiencing a score bombing activity. Several accounts have been assigning a score of 1/10 to a large majority of newly released U.S. films and some newly released foreign films. Because these accounts have assigned a score of "1" to thousands of films they have gained the status of TOP 1000 Voter accounts, such that their votes are given great weight in the calculation of "weighted average scores."

A number of small film companies have complained about this to IMDB. They are concerned about financial injury arising from such score tampering. IMDB staff have responded by changing the algorithm for "weighted average score" for films with less than 100 votes, on the principle that the effects of such score bombing will be washed out for films that receive large numbers of votes.

New information.
I have been further analyzing the score bombing issue by analyzing the posted statistics on films released in 2013, including some foreign films for which U.S. vote totals are understandably small. I have also been reading the posted complaints about fake votes and score-bombing problems on the community boards.

1. The extent of score bombing appears to be growing with time.
- Six months ago the score bombing activity appears to have been limited to a handful of IMDB accounts.
- Several weeks ago the score bombing activity was discernable as about 20 accounts that were assigning scores of 1/10 to newly released films
- Now, in mid-Oct 2013 there are 3 to 4 dozen TOP 1000 Voter accounts that are voting a score of 1/10 to newly released films.

2. There is evidence in the voter records that indicates that the score bombing is originating on accounts that are declared to be U.S.-origin accounts.

3. In my professional opinion, the pattern of score bombing activity indicates that most or all of the score-bombing is a coordinated activity by one or more persons. The duration, extent and pattern of the activity indicates that this most likely is not a result of manual voting by one or more persons using a dozen accounts. It most likely is arising from the use of an automated voting software by one person, or by automated voting software that has been shared and is being used by several persons.

Analysis and Comments
1. The motivations of the parties that are performing this attack are unknown. Everyone (including IMDB champs and myself) should avoid posting their personal speculations about the motives behind this score-bombing as if they are facts.

2. The previous actions by IMDB staff are no longer adequate to address an attack that is now generating > 36 votes of 1/10 from TOP 1000 Voter accounts. The average scores of films with vote totals of 5,000 and more are being affected. In addition the posted vote for TOP 1000 voters for almost every recent film is now corrupted. It is bad policy to continue to post information that you know is corrupted.

3, If the use of Bots to vote 1/10 on recently released films continues to grow apace and reaches the level of 100s or 1000s of votes for each film, then the score bombing problem could become a very serious issue. It would be prudent to act now before the problem becomes more severe.

4. A short-term stop-gap is available:
IMDB staff could readily access TOP 1000 user accounts and look at their score distributions and determine which accounts are being used for score bombing. Courts have long determined that there is such a thing as "insincere processes" and that a prima facie determination of "insincere processes" is permissible IMDB is fully within their rights to determine that an account that has a record of assigning thousands of movies a score of 1/10 is using "insincere processes." As such, IMDB can take whatever administrative action it wishes to take against such an account.
If IMDB chooses not to do this, they should at least make a highly visible announcement to their user community that explains the situation and what they know about it.

The trust of the user community in the integrity of IMDB information is one of your most valuable business assets. Please, do not imperil that trust.

5. In the long-term, this issue should be treated as a cyber-security attack against the IMDB site. IMDB cybersecurity specialists should be tasked to develop a defense against automated voting and any similar activity.
Photo of Gordon Michaels

Gordon Michaels

  • 17 Posts
  • 3 Reply Likes

Posted 6 years ago

  • 1
Photo of Emperor

Emperor, Champion

  • 6418 Posts
  • 3004 Reply Likes
Because these accounts have assigned a score of "1" to thousands of films they have gained the status of TOP 1000 Voter accounts, such that their votes are given great weight in the calculation of "weighted average scores."


Not strictly true. There is a weighting given to top voters, of which the Top 1000 Voters are a subset, although quite how big the wider group is, and how the weighting is applied is unknown as people will try and game the system.

- Now, in mid-Oct 2013 there are 3 to 4 dozen TOP 1000 Voter accounts that are voting a score of 1/10 to newly released films.


This has long been a problem, unless you can present any evidence.

See, for example, the Shawshank Redemption - one of the top rated films on IMDB and the first to get a million votes. Despite the 620,000 10 star votes, there are 23,000 1 star votes and it has 922 votes from the Top 1000 (possibly the highest percentage of them voting on the same film) and their average is 8.4, significantly lower than that from other categories of voters:

http://www.imdb.com/title/tt0111161/r...

This also has impact on your analysis point 3 - we already have tens of thousands of 1 star votes on some films and have had for a long time.

3. In my professional opinion, the pattern of score bombing activity indicates that most or all of the score-bombing is a coordinated activity by one or more persons.


You are going to have to give evidence for this (and perhaps let us know what qualifies this as a professional opinion) - some of it is coordinated (as is some of the trolling of the forums) but quite a bit is clearly being applied by individuals looking to wind people up. For example, those reporting score bombing on their films tended to find that this would attract further score bombing - some of the individuals were clearly keeping an eye on the uproar about this and throwing in more votes to get a reaction (typical troll behaviour).

For example, see this person's experience and the subsequent discussion:

http://jneilschulman.rationalreview.c...

It is even possible that cleaning up the trolls:

https://getsatisfaction.com/imdb/topi...

would also have some impact on the score bombing, although you wouldn't know until you got your hands dirty and started digging.

The motivations of the parties that are performing this attack are unknown. Everyone (including IMDB champs and myself) should avoid posting their personal speculations about the motives behind this score-bombing as if they are facts.


I've described my explanation as educated guesswork and that the reasons for the score bombing are always going to be difficult to pin down, you are the one making firm statements about motivation and means without the evidence to back it up.

3, If the use of Bots to vote 1/10 on recently released films continues to grow apace and reaches the level of 100s or 1000s of votes for each film, then the score bombing problem could become a very serious issue.


You've presented no evidence to suggest this is largely due to "bots" - I watched a documentary on the "farms" in places like Bangladesh which sell bulk friends and likes on social media, and the sad truth is that it seems to be cheaper and more efficient to have people sit their at computers logging in and out of accounts all day. I'd be surprised if people offering votes for sale on IMDB are any different, especially as I assume there are checks behind the scenes to prevent flood voting and other suspicious activity.

IMDB staff could readily access TOP 1000 user accounts and look at their score distributions and determine which accounts are being used for score bombing. Courts have long determined that there is such a thing as "insincere processes" and that a prima facie determination of "insincere processes" is permissible IMDB is fully within their rights to determine that an account that has a record of assigning thousands of movies a score of 1/10 is using "insincere processes." As such, IMDB can take whatever administrative action it wishes to take against such an account.


This I agree with, although without the recourse to legalese. If IMDB finds such patterns then this is clearly an abuse of the system (you don't need to work out why they are doing this or why) and they'd be well within their rights to scrub the lot of their votes and shut the accounts down - I've advocated them doing this a number of times.

It'd no only address a serious problem, but also open up the Top 1000 Voters o legitimate users of the site who have taken the time to actually watch the things they are voting on, so clearly know their onions - which is, presumably, the reason for giving top voters an extra weighting in the first place. Cleaning this up is the only circumstances I can see that'd justify keeping the weighting. Even if you switched the weighting to Top Reviewers, as I've suggested before:

https://getsatisfaction.com/imdb/topi...

You'd still need to clean-up the problem voting accounts.

So I don't see much evidence to justify some of your claims (and some activity undermines it), but I agree with the solution - IMDB needs a more proactive response to the various problems they have and it should be easy enough to do it without it costing a fortune.

5. In the long-term, this issue should be treated as a cyber-security attack against the IMDB site. IMDB cybersecurity specialists should be tasked to develop a defense against automated voting and any similar activity.


That would depend on what you find when you start digging - if there is evidence of automated activity then steps would need to be taken, although it is possible some of these are all in place, but IMDB won't discuss it.
Photo of Gordon Michaels

Gordon Michaels

  • 17 Posts
  • 3 Reply Likes
Emperor:

First, in reviewing the recent history of complaints on getsatisfaction.com, I have noticed that you have assisted many people with problems and that you have been extraordinarily helpful and competent. So, I just wanted to acknowledge that up front and to say that although I may have a different assessment than you on a few nuances of this score bombing problem, that I very much respect the work and service that you have been providing. You certainly seem to be on the side of the angels.

1. Regarding my professional experience: Would you message me on IMDB (to gordonm888) and I'll respond with info on my professional background. I'd prefer not to post this information on an open chat board.

2. Regarding the weighting algorithms for averaging scores, you apparently know more than I do (although all of us users are in the dark.) Thanks for the info.

3. Regarding 1 star votes: I agree that they have always existed. I have long assumed that some IMDB voters are binary: that is they either vote 1 or 10. Other 1 star votes are presumably made for all sorts of reasons and because of all sorts of objections to the movie. I wish that were different, but at least most of those 1 star votes for The Godfather and other good movies can be presumed to be sincere voters who just have a different approach to scoring movies or a different set of values. That is not and never has been a problem that I am objecting to.

My immediate concern is the IMDB TOP 1000 voter accounts that appear to be voting 1/10 indiscriminately on thousands of movies within days of their release date - this is spam with malicious intent I have only recently become aware of the problem, so the best I have been able to do is to perform research and analysis to estimate the incidence of such voting. I'm open to be advised that I am wrong by people who have been living with this problem for a longer period of time, but I'd like to ask some questions about your examples and assertions.

Regarding Shawshank, it has 58 1 star votes from 922 TOP 1000 Voters, which is an incidence of 6.3%. Given that the incidence rate of 1 star scores in the popular vote for Shawshank is only 2%, I estimate that there are about 38-40 anomalous 1 star votes for Shawshank among Top 1000 voters. This agrees reasonably well with my claim that 3 to 4 dozen accounts have been spamming 1 star votes. However, I cannot tell if those 38-40 anomalous 1 star votes from Top 1000 Voters have always been there or whether it used to be twenty and has recently grown to 38. Perhaps you know, but I don't.

My approach to my analysis was as follows.. Because Shawshank and other films in the TOP 250 are a pretty obvious target for automated voting programs regardless of when they were activated, I adopted the approach of looking at randomly selected obscure films from different time periods in order to form qualitative opinions about whether the number of rogue accounts was increasing with time. I did this because obscure movies are most prominently featured on IMDB during the narrow time period surrounding the date that they were released - I hypothesize that it would be easy for automated voting programs to find and score bomb any movie that is a new release -but only if the voting program was active at the time of the movie's release. However, for those automated voting programs that came on-line significantly later, it would be increasingly rare that it would find and score bomb very obscure (i.e. low total vote) movies that had been released earlier. (I do have specific conjectures and assumptions about why that would be so.) So I analyzed TOP 1000 Voter voting on obscure (i.e., low total vote) movies that were released at different points of time between 2010 and now to determine whether the number of anomalous 1 star scores from TOP 1000 voters was a function of release date. I did indeed observe a trend. It appears to me that the problem of rogue 1000 Top Voter accounts has been growing with time.

I certainly admit that my analysis does not conclusively prove that the score bombing problem is growing - there is uncertainty. But it was the basis for my statement "The extent of score bombing appears to be growing with time."

I will say this: if films have been receiving 30-40 1 star votes for a long time, then I don't understand why IMDB drew the line at only protecting those films that receive 100 votes or less. Clearly, 40 spammed 1 star votes would also injure a film receiving 500 or 1000 votes and films with far more than 100 votes should have been protected as well. My impression from past posts on the Problem boards is that IMDB selected the 100 vote total because the problems reported to them involved 1, 2, or 3 fake votes of 1 star - not 30, 40 or-50 fake votes of 1 star. So none of the history as I understand it is consistent with your claim that there have always been 3 or 4 dozen rogue accounts that have been spamming 1 star votes. So, I'm am asking now: are you sure that 3 or 4 dozen TOP 1000 Voter accounts have been spamming 1 star votes for a long time and that the number is not growing? To the best of your knowledge, how long has this been occurring at this level?

4. Regarding the website that you've helpfully posted- the site which has an account about a troll spamming a particular film and "advice" on how to spam IMDB: Well, this is discouraging and indicates there is a wider problem. But this behavior is actually different than the immediate issue that I am pointing out. Someone who posts 103 1 star votes on a specific film is a major problem, I agree. But, in my opinion, this is probably not the same guy who is posting 1 star votes on thousands of films, much less 40 1 star votes on thousand of films. The behavior I have been discussing is different in duration and scope and would likely arise out of different motivations.

Certainly, in an ideal world all bad problems and all evil things need to be addressed. But I personally cannot address all problems on IMDB. The specific problem I have been addressing is the indiscriminate 1 star scoring of thousands of recently released movies by several dozen TOP 1000 Voter accounts. Other problems, such as revenge-minded individuals attacking specific movies, are different and enough off-topic that I really have no basis for commenting on them.

5. Regarding automated software programs for interacting with websites: Unfortunately, such bot programs are currently a problem at a fairly large number of internet websites. If IMDB is not being attacked by such programs then they are the exception. Get on Google and research the subject. The basic software can be acquired from internet websites and modified for a specific site such as IMDB by people with some programming expertise. There are chat rooms on the internet on the technical aspects of such software, just as IMDB has chat rooms on movies. There is no need to imagine rooms packed with low-cost foreign workers to explain score-bombing: a single person could activate a bot program on their computer and then be off drinking beer with his buddies while the 1 star votes are being placed on IMDB. It would be straightforward to leave the program on his computer and schedule it to look up new releases on IMDB once a week and vote each of the new releases a score of 1 star.
This seems to be to be a far more likely scenario -because it is much easier to organize and execute than rooms packed with people for months at a time. Furthermore, most lone hackers and trolls do not persist in manually spamming websites for months on end, because they grow bored of it and move onto other things. Automated voting software never gets bored, however.
Finally, all of the descriptive information about the 30-50 TOP 1000 Voter accounts that are spamming 1 star votes seems to me to be superficially consistent with the capabilities of automated software for interacting with internet sites. Sorry, but i suggest that the burden of proof should be on those people who hypothesize that this is arising from anything other than automated voting software.
There is no shame in being ignorant of automatic voting software. But it seems odd to me that someone should continue to deny such an obvious explanation once they have been informed of it.

6. I am glad to see that we agree on the advisibility of identifying and eliminating the rogue accounts that are spamming IMDB. And I agree (and certainly hope) that IMDB may have already taken some cybersecurity measures to address these kinds of issues but simply isn't telling us. Amen to that!

Look, I have now spent a lot of time on this issue. I am - or at least I claim to be (from your point of view) - a highly educated person with some exposure to and expertise in this field, and I have been sincerely trying to help IMDB by identifying a problem. I have no axe to grind and I have nothing to gain personally if IMDB does or does not address this problem. The IMDB staff and you should have sufficient maturity and discernment to be able to review what I've submitted and make a judgment. Do I sound like some disgruntled wildman with poor judgment? or do I sound like a reasonable, intelligent person with a reasonably stated concern and logical arguments? I do not claim to be a know-it-all but I ask that you and IMDB staff remain open to the possibility that you don't know it all either. If you require that community members submit a PhD thesis on their problem and a blood sample before you will listen to people that are trying to help you, then you are setting the bar too high.

I love the IMDB and I want to see it remain as a high-quality, trust-worthy site - but right now my trust is being shaken by what appears to me to be a refusal to (1) acknowledge and (2) address a problem that has already diminished the integrity of some of the information that you post on your site.
Photo of Emperor

Emperor, Champion

  • 6418 Posts
  • 3004 Reply Likes
3. Regarding 1 star votes: I agree they have always existed, and I have assumed that some voters are binary: they either vote 1 or 10.


I must admit, I had assumed this was a clear sign of shifty behaviour but I have seen people with public profiles that show mainly 1 and 10 votes. Now I've not spoken to them about it, but I suppose some people just want a like/dislike system, I still think this is rare though, but it'd be worth also assessing such voting patterns in a broader context of their activity. As you say - it'd need analysis to see how indiscriminate it is, you'd also want look at the timing (if this is recorded - it isn't shown on the ratings page, although some people have requested it, but I assume the system must keep a record for sorting).

Given that the popular vote averages only 2% as 1 star votes for Shawshank, I infer that there are about 40 anomalous 1 star votes among Top 1000 voters. This corresponds to the 3 to 4 dozen accounts that I claim are spamming 1 star votes.


I think you are focusing on the Top 1000 Voters because they are shown as a separate category, but the weighting is applied to to a lot more people than that, we just don't know how many - you can use them as a sign that something is afoot, but it'd be difficult to draw any hard conclusions on the number of 1 star spammers.

Someone who posts 103 1 star votes on a specific film is a major problem, I agree. But, in my opinion, this is probably not the same entity who is posting 1 star votes on thousands of films, much less 40 1 star votes on thousand of films.


It is currently not possible to tell without access to the backend.

What we do know is that some of the Top 1000 Voters are trolls who spam 1 star votes in to get a reaction and that some of these trolls can have access to a few hundred accounts (although this may be a small group of, say, a dozen trolls who have a dozen accounts or more between them - we know for certain at least one such group exists, although we don't know if they also vote 1 star on projects they've not seen. I, personally, wouldn't put it passed them).

We can pretty much guarantee that some people offering paid for votes are just using multiple accounts and that it'd be in their interest to offer weighted votes, which I have found being advertised on sites offering reviews for cash.

It is certain that there are different people involved with different motivations, which result in different patterns of abuse, but trying to quite figure out who is doing what is, as far as I can tell, extremely difficult.

There is no need to imagine rooms packed with low-cost foreign workers; a single person could use such a program to do most of this anomalous voting almost overnight and then leave it on their computer to vote on new releases once a week or so.


I don't need to imagine such farms - they exist. This is he documentary I watched:

http://www.channel4.com/programmes/di...

http://www.marketingmagazine.co.uk/ar...

http://www.theguardian.com/technology...

http://www.itwire.com/it-industry-new...

https://www.youtube.com/watch?v=jer2HH...

Not the only similar investigation with footage of these click farms:

http://www.france24.com/en/20131002-i...

It is an extension of click farms, which rely on people:

https://en.wikipedia.org/wiki/Click_farm

I assume they use social media farms in the same way because most sites will have processes in place to detect bots or the botnets would run wild with them.

Furthermore, most trolls do not persist in manually spamming fo sites for months on end - they grow bored of it and move onto other things.


You underestimate the persistence of IMDB's trolls. Some are linked to and discussed in the discussion I've mentioned. There are a few who have been operating on IMDB for years as the systems IMDB have in place are slow and IMHO inadequate for tackling the problem, plus some of them have access to resources that mean they can get around such systems (one apparently works in a mobile phone shop, so has, in theory, access to an infinite number of numbers which they can use to authenticate new accounts).

There will also probably be some people who are self-appointed champions of IMDB, who will see this not as spamming but as countering vote stuffing activity, and I'd imagine a decent percentage of people causing problems for the indie film-makers were operating along these lines, even if we know that there were clearly trolls operating in the same area (although the two groups may not be easy to separate, as some might be getting a kick out of spoiling people's vote-stuffin, which is borderline). I can't guarantee there are that many (if any) but it seems likely as a lot of the small films being hit had potentially "suspicious" voting activity with large numbers of 10 votes.

Finally, all of the descriptive information about the 30-50 TOP 1000 Voter accounts that are spamming 1 star votes is superficially consistent with the capabilities of bots, i.e., automated software for interacting with internet sites.


I'm not denying that there aren't some automated systems involved (I reckon I could write something that'd run in a browser to spam votes, although I can't guarantee it wouldn't trigger alarms, but with enough time, practice and accounts you could come up with a script that runs more naturally) but we do know that the problem is a complex mix of different means and motivations.

Look, I have now spent a lot of time on this issue. I am - or at least I claim to be( from your point of view) a highly educated person with some exposure to and expertise in this field, and I am sincere about trying to help IMDB by identifying a problem. I have no axe to grind and I have nothing to gain personally if IMDB does or does not address this problem. The IMDB staff and you should be able to review what I've submitted and make a judgment as to whether I sound like some disgruntled troublemaker with poor judgment - or whether I am a reasonable person with a reasonable concern that has been reasonably stated.


I might not agree with all the inferences you draw (even though you may be partially correct, I'd need to see the evidence and that'd need access to the backend - it isn't information they'd make public), but I can't fault you on identifying that there problem or on the way to fix it. It is an important contribution to the larger debate.

This is only part of the wider problem, but it might be some of the measures needed to fix this will also help address other areas (like trolling in general) or it might need some specialist approaches. Until you start digging through the data it is difficult to say...

What more do I need to do?


Like me, I assume you've said what needed to be said and the ball is really in IMDB's court. At least until another angle emerges on this problem that highlights new sides of the issue...
Photo of Gordon Michaels

Gordon Michaels

  • 17 Posts
  • 3 Reply Likes
Quickly.
1. I agree that there may be a wider problem with vote fraud on IMDB. But there is a well defined problem with 30 to 50 TOP 1000 Voter accounts. I believe that about 20 of them are coming from the same automated voting software using the same settings. I also beleive that any person who has a background in cybersecurity would immediately arrive at the same preliminary conclusion -that there is automated voting programs that are involved.

2. Don't you think its odd that this voting by the TOP 1,000 Voter accounts started years ago and has supposedly never increased or decreased over the years? What would explain that? have you really thought that through?

3. All of your posted sites have quotes from Facebook officials and officals from other social media saying that they move to eliminate fraudulent votes as soon as they become aware of them. Why isn't IMDB taking the same policy? Why do they continue to ignore the 3 or 4 dozen TOP 1000 Voter accounts that are score bombing.

4. All of your websites on social media fraud mention foreign workers being paid to place Facebook Likes and Dislikes, IMDB is never mentioned. Isn't is a leap of faith to believe that someone is paying foreign workers to vote place fraudulent IMDB votes

5. In my original post I reported to you that the Top 1000 Voter accounts that are score bombing show up as U.S. users in the vote distribution tables That is inconsistent with your hypothesis that this problem is partly due to rooms full of foreign workers.

6. My methods for making numerical estimates is certainly not exact, but I am using standard methods for generating an approximate estimate for the extent of the vote fraud. The fact that my estimates are not exact does not mean that my estimates lack value -in fact, my experience is that this kind of methodology is reasonably reliable. And my methods are much better then no estimates at all. If you can't provide quantitative estimates, then don't look down your nose at people who are using easily understandable logic to generate best-possible estimates.

You have said:

"What we do know is that some of the Top 1000 Voters are trolls who spam 1 star votes in to get a reaction and that some of these trolls can have access to a few hundred accounts (although this may be a small group of, say, a dozen trolls who have a dozen accounts or more between them - we know for certain at least one such group exists, although we don't know if they also vote 1 star on projects they've not seen. I, personally, wouldn't put it passed them).

We can pretty much guarantee that some people offering paid for votes are just using multiple accounts and that it'd be in their interest to offer weighted votes, which I have found being advertised on sites offering reviews for cash. "

Can I ask how you "know" all this? What is your basis for these statements. Does this represent an IMDB analysis that has been communicated to you privately? Or is this based on public statements made by IMDB staff? Or is this a conjecture on your part, based on the television documentaries you have seen?

Also, I am unaware of anyone paying people to make votes, I quickly dropped that conjecture after I started looking in more detail at the pattern of fraudulent voting. Are you reasonably confident that people are being paid to place fraudulent votes on IMDB? Can I quote you on that to public media contacts that I have?

This conversation is no longer open for comments or replies.