Will new rules apply for markets suspending daily?

I'm glad that this question has been raised. Obviously, this has an affect on me as the new Sports Category Editor, but also on the direction that hubdub will take and what users want.

I'm excited about the upcoming football season and am glad to see users getting more creative with their questions recently.

To me, it sounds like the questions that we are trying to answer are these: 1) do we want to limit the number or types of questions that come up in the sports categories to what would be considered more "newsworthy" or what larger numbers of people want to see....or leave this area open, encouraging users to get involved and be creative?
2) if we go for the latter route, what would be the best way for the admins to handle this volumen?

As bayoubear pointed out, there could be multiple questions on each of the 14-16 NFL games each week. I've already seen comments about the possible different fantasy league type questions (have to admit that I'm a big fan of these myself), so I would anticipate a number of questions about different statistics and matchups. Then, throw in the NCAA/College football questions and all of the variations of these. I might have to quit by day job! :)

What are other users thoughts?

Months back I created this NHL play-off question:
http://www.hubdub.com/e/Market/NHL_Pl...

I then spent 10 minutes a day updating and double checking the stats. Every two days (or games as teams got eliminated) I posted the stats to the comments page so that people could see who was winning and check my stats or figures, When it was all over, I tied my settlement request to the stats in the comments (it was my integrity on the line if someone could find an error).

If I hadn't done all that work, there is no way Donal would have allowed my question. And he would have been right.

If you have a complicated question (and by complicated I mean that it involves stats gathering and math operations) that goes beyond the usual work required to settle a Sports question, then the onus should be on the question creator to gather the stats, do the math, and display the relevant info in a comment on the question page, along with a link to the pages or site you used to get the info. Then submit the settlement request and refer to the comment where all the data and math is.

It's a lot of work. If you feel that you don't want to have to do that, then why should an already overworked Category Editor have to do it for you? In other words, if it's too much hassle to do yourself, your question isn't a good Hubdub question, and shouldn't be submitted.

That seems to me to be the only reasonable way to approach this. It's better than "banning" questions. It also makes more sense than a laissez-faire approach where all the extra work is piled on the Editor. Make it easy for the Category Editor to like your "creative" question by removing the donkey work from them. If all they have to do is check some of your stats (which MUST be correct) and recheck your math, they'll be happy and you'll be happy.

I have to admit that infernalmachine's fantasy hockey question was one of my favorite sports questions since I've been on this site. He also did a great job managing and updating the question on a regular basis. If you look back at the question, you can see that he spent a great deal of time on it (which also brings us back to one of the issues that brought this discussion here - what is reasonable to be expected of time and effort put in to the settlement of questions...as well as the timeliness of this discussion prior to the start of the regular season of other sports). So I like the "onus" concept. As those types of questions can be fun, I would hope that we wouldn't have to limit questions to the "who will win" variety only.

letmewinplees, along with some other users have also been very good about providing the results of some of the various stats questions, but as the new category admin, I also feel responsible to check the math and make sure everything is correct.

I'm not sure what the feasibility is, but have to admit that I kind of liked the idea of question approval. Not sure that we're prepared to implement something like that right away, but perhaps an idea to keep in mind. I'm new to the whole admin thing, so I can only share from my new and recent experience, but I've spent a great deal of time responding to flags about starting odds of different events and "what if this happens" scenarios.

Also, I hope that my comments have not sounded like I was not willing to put in some work here. My intentions are directed more at concerns about taking over a very active category, watching the questions and hubdub evolve and anticipating the direction that we are going with the sports questions (as well as others).

Thanks for everyone's feedback

Interesting and useful discussion.

Further on lemewinplees ideas. We have actually thought quite a lot about the first two questions. For the first question one feature we have talked about a lot was having a 'question sweeper'. Any question that didn't achieve a minimum level of interest within 24 hours would be unpublished (note also we are hoping to change the terminology from voided to unpublished as it is with less prejudice). We still this is a good one but we haven't got to it yet.

The second idea (having a question holding area) has a more interesting history. We actually designed something like this prior to launch (it worked a bit like www.thestandard.com) however we never had time to implement it. Since launch I've actually started to think it would become a barrier to new users becoming question creators. I've been reading a lot about Wikipedia recently. I've always been fascinated that they kept the ability for any reader to edit any page even without logging in. The reason they kept that is that they have found that is the way they first get new editors. Putting up barriers would reduce spam and vandalism but at the expense of growing the community. One thing I think we could improve is the ability for question creators to converse before creating questions. To help that we are hoping to put in message boards on the site within the next couple of weeks (which would also replace Get Satisfaction).

Nigel,

The low "interest" markets come in several flavours. The main reason they come up here is subsequent to the previous discussion in comments for a Hit-By_Pitch daily total market. It was suggested that the amount of extra Cat Ed work to research and gather the relevant stats was not worth the effort given the low activity or number of predictors. This then spilled into a defense of "side-bets". So in this case, the low interest was only seen as a problem wrt the extra load on the Category Editor.

Other "flavours" of low interest markets include in-play foreign exchange trading (where a few players used to be able to make huge profits with little competition - like the Dow questions but without the market instantly responding to the current state of the index) and markets of such small interest that there is insufficient oversight by the community to determine if initial odds are being set reasonably, or if in fact some kind of gaming is going on.

But, in defence of low interest markets, and I paraphrase Chris (admin) here, I like a good prediction on Scottish football, which isn't all that popular at Hubdub. So a sweeper has its uses, but there's a down side as well.

Perhaps each of these "flavours" can be handled individually. Putting some onus on question creator to share the burden of figuring out the winning option should take care of the first example. Superusers are pretty diligent at finding the bad odds, as are all players in general.

The second case could be handled by 2 steps. First reduce the maximum bet to $1000 from $10,000. If that's too drastic, then reduce it on all in-play questions (which after all are only allowed if they gain admin approval -- the Dow/Nasdaq questions have that approval implicitly as of now).

Step 2, and this helps both the last minute series of $10K bets on the Dow as well as low activity/participation questions, consider implementing a formula that restricts the amount (by percent) of domination any player can have on any 1 market. Example: Say the limit is 25%. If the total of open (ie not cashed-in) predictions on a market is $40,000 and I want to drop $10K on an option, I can because my 10K will be one-fifth of the $50K (after I bet). But if I want to drop another $10K immediately afterwards, which would mean I would have 20K of the total 60K (ie 1/3 = 33.3% > 25%), then it wouldn't let me bet that much. I could (math skipped) bet up to $3333. If I want to bet more, I must wait until others have bet up the total to at least 4 times (based on 25% limit) what I have bet.

Notice some form of this also works wonders with low activity markets, because it forces the player who wants to play it heavily to wait for enough activity to justify the big wagers. This also encourages that player to publicize this market with friends or general community.

In response to your second point, Nigel, I'd like to throw this idea into the ring.

The correct setting of initial odds is REALLY difficult in some (many?) cases. It is also difficult to determine if another's question has good or bad initial odds. And the problem is compounded because we can't (and shouldn't, to be fair to people who have already predicted) change the odds once a question is open.

Now if we had some form of "question holding", by which I assume you mean what was previously discussed here as user-vetting of questions before they go live, then I have a suggestion for getting more acceptable initial odds. If users (or just susperusers and editors and admins) get to quickly pre-approve questions in some form, then it shouldn't be all that hard to allow the "vettor" to also suggest their own odds. If it requires 3 approvals to go live, then the 4 sets of initial odds (creator plus 3 vettors) can be averaged to produce a more palatable result. This would reduce the strain on creators, give assessors a way to make the odds more to their sense of what's right, and generally reduce the number of voids for bad odds. This system would probably work better in the superuser context than in the larger user-base, because it extends what the superusers are already doing in another form (flagging and suspending questions with bad odds to be voided)>

Welcome to the discussion Nigel! Good to hear your input and more about what's going on behind the scenes at hubdub.

Lots of good input here. After reading the above suggestions, here are a couple of things that I'd like to toss out in response:

There seems to be some agreement as far as the putting the onus on the creator of the question. Other discussions among Super Users and Category Editors seem to lean towards encouraging more of this as well. Even in doing this, I feel that there is still some responsibility of the Category Editors to verify accuracy (unfortunately, this is more true with some users than others).

As far as the approval period, the biggest issue that I see with the "thumbs up" process would be that users are deciding on the same questions that they may be wagering on (and thus have vested interest and influence on how much they may or may not win). The Super users have been pretty good about flagging questions quickly. However, the current process still requires the question to either remain open or suspended until the Category Editor approves. I don't see any problem with the SU's having more authority - especially in questions where the starting odds are more clear (those where professional bookmakers already have odds out).

I don't know how I got this cupcake picture here, but it's making me hungry....

Up until quite recently I was all in favor of having some form of 'hold and vote' system for new question creation so that no trading occurs until it receives a sufficient number of votes. Now I am not so sure as I am concerned it might present a barrier to question creation which is the life-blood of the site. (However you could potentially make a counter argument and say that if I knew I would get feedback on my question before it went live then maybe I would be more likely to make a question). Still not sure on the right answer so I think we should make some small 'no regrets moves':
1. Introduce a forum where question creators can discuss new questions
2. Change the terminology from voiding of new questions to unpublishing of new questions
3. Streamline the question creation process

After that we should re-visit this question.

On the second question of low activity questions we are working out the details of a new automated market maker. The problem with the current market maker is that the prices aren't volatile enough on new markets (if all the early money goes on one outcome then that should significantly move the price) but they are too volatile for established markets (Obama's price shouldn't really move more than 1-2% per day). The new market maker should resolve that plus what it will mean is that new questions could start at roughly the right price without too much being handed to the first predictors. e.g. I could start it at 50% but it would only take H$200 to move it to 70%.

I like infernalmachine's idea for max market holding. Let's revisit that after we change the market maker.

Thanks for the really interesting discussion. I thought I'd summarize and let you know where we're at on the community side of things.

I've identified three main problems you guys have raised:
1. Overwhelming the Category Editor and site with questions
2. Having many questions with few predictions placed
3. Having 'trivial' questions or sidebets and how to judge whether they're acceptable

In resonse to No. 1 - the sports cat ed is obviously swamped and doing a valiant job at getting through questions, but this is an internal issue and shouldn't be grounds for limiting questions.

For 2 and 3 - in Hubdub's community spirit, we don't want to limit questions but we want to create solid and accurate prediction markets. Ideal prediction markets are:
1. Markets that move (ie short term, or volatile long term);
2. Markets that have lots of predictions placed, both number and quantity
3. Markets that are well structured (simple, have clear outcomes, settlement info easy to find).

For now, we are not implementing any bans on question types, so my advice would be to keep trying new questions that hopefully meet the criteria above. What we are doing from the admin side to try and get more markets fitting the above are:
- Publishing good question creation guidelines (will be out shortly)
- Promoting good questions through the community aspect - voting on them, commenting on them, and publicizing them
- Looking into potentially rating questions and having a question creator leaderboard
- Creating an on-site forum to discuss questions and hubdub tips, that will be monitored by me and the supers.

Other points you've raised
1. Question creators should be more accountable for the questions they create - Agreed, we've started adopting this ethos by referring clarifications back to the author.
2. We should have a holding period - I think that there are already too many barriers to question creation, which are offputting to a new user. Again, I would avoid putting a strong mechanism like this in place in favour of being quick to jump on and suspend problem questions (which the supers already do) and providing constructive discussion and feedback.
3. Limiting bet sizes - I quite like this idea, but it entails another huge discussion. Probably best to save for another thread :-)
4. Recategorisation of the markets - this was one I had too, but again involves drawing some sort of line between 'news' and 'side news'. I'm happy to entertain more discussion on it though.

While I did raise the issue about the growing number of new types of questions and the potential volume - and even going by the example that letmewinplees provided above, I am guessing that those are just a few of the many possible variations (e.g. how many yards will Randy Moss have, who will have the most yards - this team or that team; etc etc)...I do agree with Tisha that this is probably an internal issue. Yep, it's a lot of work, but as long as questions meet the criteria above and are what users want, then it will be up to us hubdub folks to figure out how to best accomodate that.

letmewinplees - you stated that you weren't sure your question has been answered. As the initial question was "Will new rules apply for markets suspending daily," keep in mind that hubdub is in Beta, so is constantly evolving and changing. It sounds like we're not ready to place any "bans" on questions at this time as long as they meet the criteria above. The trial markets are one way of testing the criteria.

Tisha identified a few changes that are in the works, as well as summarizing other points for further consideration. Nice summary Tisha!

Just my thoughts for now....

There is a saying in software that you should 'pave the cow paths'. That means you should see where the cows walk and that is where the path should be. I think we should use the same rule here:

Question creators shouldn't feel limited in the questions they create (beyond the existing rules). If you have an exotic question type then either create it or post it to the forum (coming soon!). If it is successful and it works out then it will become established. If it becomes contentious or too difficult to settle then we may have to agree a new rule on it.

We've had questions before which were clearly marked as settlement details would come from the question creator (who agreed not to participate). I can't see any problem with having them again.

Hi letmewinplees - your consideration of the cat eds is commendable! However, ultimately it's our job to settle questions, regardless of whether they are fun to settle or not.

As Nigel has said, it would take a huge influx of tedious questions to effectuate a ban, because ultimately we don't want to limit question creators, and we have quite a solid supporting team in place now so we are generally happy to take on bigger workloads if it means more questions and more users.

It is always appreciated when question creators provide solid settlement details and links for their questions (and also when they keep members informed via the comments pages!). This could well be a requirement in the future, but as we've said, for now it doesn't seem to be a big enough issue to warrant a rule being made so it will be up to negotiation with the individual category editor.