jenniandboys's dashboard

i agree with the voluntary deal. hubdub's "motto" we don't like rules is a good one imho. i run (my very limited questions) by an admin via email before i make-publish them. i choose to do that. making it mandatory would be a step backwards imho. i'm gonna try to be more active in creating questions. kinda feel guilty only wagering. from my very little experience of creating questions, i know it can be a hard thing to do. to spend time researching, typing, making the odds (hardest part) then to have it unpublished is very discouraging. have a nice day. – dieseldoggg, on August 09, 2008 17:30

One challenge for us we only have a three person development team so we can't do everything we would like to. However what things are you looking for? – Nigel Eccles, on August 08, 2008 22:28

As a hubdub user i was wondering why when anyone suggests things, they can never just get a answer its always " its being considered or discussed seems to be the motto for this site.

Glad to hear something like this is being considered (thanks Nigel) and thank you IM for your comment. I really think everyone would benefit from it. As a creator, I would feel more confident knowing my question could be double checked by the admin team. I would be less apprehensive about creating them knowing there was a greater chance of my question not being "unpublished"(voided), flagged, suspended, etc. If I choose, my question could be placed in a review "bucket" and then returned to me with suggestions from the "Hubdub reviewing team". For those questions that I feel more confident in, I could just opt for it to go straight to publication. I think this could also help lighten the load for the staff. – letmewinplees, on August 08, 2008 18:33

We are reviewing question creation in the next couple of weeks and this is something we are considering. – Nigel Eccles, on August 08, 2008 15:34

This is a very good idea. I've felt that way too. You have a good question, but you're not sure about the initial odds, or how clear your question will be to other players. What I really like is that what you're suggesting is a voluntary thing, No-one makes you ask for inspection. You only choose to if you want. If the question settles very soon, you might not do this. The only thing I would add is that it might be even better if it was sent to a superuser (or a Category Editor). They are really good at inspecting questions too. Other great thing about your idea is that it might be a whole lot easier to set up. Thanks for this. I will pass it on. – InfernalMachine, on August 08, 2008 15:31

I was just creating a new question and as I was about to push "publish question" or whatever that button is right before your question "goes live", something dawned on me. I was thinking about how I would really like someone to review it before I publish it. Why not have that as an option? Instead of all questions going live immediately, have an option for cat editor's review. If a question creator would like their question reviewed allow them to send it directly to the editors. Once reviewed, the creator can be alerted "changes have been made to your question" or "no necessary changes noted by editor" and the creator can then publish it. With this option, the creator can decide for themself if they would like their question to go in to "holding"
ANYONE LIKE THIS IDEA?
IS THIS OPTION POSSIBLE?

But the idea is to have less bad questions, not less good ones. Obviously people who have shown they almost always create good questions would not have their questions held up. But "bad" questions use up a huge amount of staff and superuser time, and also often end up creating anger among the people who bet on these questions before they are "fixed". That's the reason for even suggesting holding questions to check them. – InfernalMachine, on August 08, 2008 05:31

imho its works ok now. is there a fool proof way to handle it? i would say no. making the question making process longer will cause more problems than it does good. i make very view, and when i do and hit publish its gratifing to see it get submitted. i would suggest asking serval of the people who create alot of questions if they want them them held up. imho it will cause less questions. – dieseldoggg, on August 08, 2008 02:35

Hi letmewinplees - your consideration of the cat eds is commendable! However, ultimately it's our job to settle questions, regardless of whether they are fun to settle or not.

As Nigel has said, it would take a huge influx of tedious questions to effectuate a ban, because ultimately we don't want to limit question creators, and we have quite a solid supporting team in place now so we are generally happy to take on bigger workloads if it means more questions and more users.

It is always appreciated when question creators provide solid settlement details and links for their questions (and also when they keep members informed via the comments pages!). This could well be a requirement in the future, but as we've said, for now it doesn't seem to be a big enough issue to warrant a rule being made so it will be up to negotiation with the individual category editor.

I agree with what you're saying here. This could turn into a huge problem. We will have to come up with restrictions on questions like you describe. But it will probably also turn out that what Tisha says is true. We will not be banning "types" of questions. But types means things like side-bets in general. What will probably have to happen, if things develop as disastrously as you suggest, is that we will have to ban questions that are too difficult to settle. Normally this means questions where the actual results can't be determined. But in this case it would be more about the number of steps taken, or number of web pages visited, that would attempt to reduce the really time-consuming questions. It's also possible that Mark will just get fed up and start limiting the number of questions per day like this. There is a precedent for this. In Business, a number of of rules appeared that mean, for example, that no questions about any stock market other than Dow Jones and NASDAQ are allowed to be asked. But lots of people really hate these rules. Another rule, in Sports this time, is that all questions about games must suspend before the game. But there are a set of questions which may be run in-play, at the discretion of the Category Editor. So if it gets really bad, Mark, with the support of the entire staff, may simply have to say that all complicated side-bet questions must be approved by him before they can run. You submit them, he decides if he wants to let that one run, and otherwise he voids it. But notice that at no point were one type of question banned. I notice you seem kind of in a rush to get this line drawn. But a decision will take a bit of time. Mark started his position a week ago and is just learning what's required. Other staff will have to be consulted. It's not going to be decided right away. Ultimately, rules that block things irritate people. Part of the answer is also in asking people to be reasonable. If the people who create these questions slow down a bit, and ask less of them, then no rule may be required. In fact, if they don't want a strict rule preventing them from asking any of these, it's in their own best interest to cool it before frustration leads to a rule being imposed. – InfernalMachine, on August 06, 2008 15:26

I understand the idea of question clarity and trying to come up with "common ground" for producing questions. I still don't see an answer to my question about drawing the line.

I came up with a scenario, hoping this will help get my answer(this I think would be a nightmare for admin):

Lets say I love American football. I love statistics, math and the excitement of it all. I decide I am going to create questions for each team (32 teams in NFL). Here are the questions:

How many total passing yards will the (team name) have this week?
How many total rushing yards will the (team) have this week?
How many total interceptions will the(team)....

Then after making each of those 32 times, I go on to make:
What will be the total passing yards for all games...
What will be the total rushing yards...

Ok so it will take me a while to make all those questions, but I do it successfully. Then I challenge all my friends to wager on it (I have 14 friends, no idea what the average is on that). Lets say 10 of them support me, want to see more questions like this...they love them, too! With all the brackets (starting % done very fair) the value of the questions are good. Lets say each of those questions get $4,000 on it.

These questions have met the criteria, the starting odds were perfect. There's sufficient user interest. I even check back periodically to see if there are comments. I post links to where the answers can be found...do all the responsible ("right") things to do as a creator.

What will the category editor do? mrperfkt said himself he still feels he should check the math before settling.
This is why I keep asking if new rules will apply. Tisha states "For now, we are not implementing any bans on question types, so my advice would be to keep trying new questions that hopefully meet the criteria above"

If Hubdub really takes off and the number of users increases in the way we are all hoping it does, I foresee huge problems. I think the start of football season will attract alot of new folks and settling questions will become more difficult of a task.

COMMENT PLEASE!!

Thanks for the really interesting discussion. I thought I'd summarize and let you know where we're at on the community side of things.

I've identified three main problems you guys have raised:
1. Overwhelming the Category Editor and site with questions
2. Having many questions with few predictions placed
3. Having 'trivial' questions or sidebets and how to judge whether they're acceptable

In resonse to No. 1 - the sports cat ed is obviously swamped and doing a valiant job at getting through questions, but this is an internal issue and shouldn't be grounds for limiting questions.

For 2 and 3 - in Hubdub's community spirit, we don't want to limit questions but we want to create solid and accurate prediction markets. Ideal prediction markets are:
1. Markets that move (ie short term, or volatile long term);
2. Markets that have lots of predictions placed, both number and quantity
3. Markets that are well structured (simple, have clear outcomes, settlement info easy to find).

For now, we are not implementing any bans on question types, so my advice would be to keep trying new questions that hopefully meet the criteria above. What we are doing from the admin side to try and get more markets fitting the above are:
- Publishing good question creation guidelines (will be out shortly)
- Promoting good questions through the community aspect - voting on them, commenting on them, and publicizing them
- Looking into potentially rating questions and having a question creator leaderboard
- Creating an on-site forum to discuss questions and hubdub tips, that will be monitored by me and the supers.

Other points you've raised
1. Question creators should be more accountable for the questions they create - Agreed, we've started adopting this ethos by referring clarifications back to the author.
2. We should have a holding period - I think that there are already too many barriers to question creation, which are offputting to a new user. Again, I would avoid putting a strong mechanism like this in place in favour of being quick to jump on and suspend problem questions (which the supers already do) and providing constructive discussion and feedback.
3. Limiting bet sizes - I quite like this idea, but it entails another huge discussion. Probably best to save for another thread :-)
4. Recategorisation of the markets - this was one I had too, but again involves drawing some sort of line between 'news' and 'side news'. I'm happy to entertain more discussion on it though.

funny to see your mood evolve from :( to :| to :) hehe – kruijs, on August 06, 2008 12:20

In the stock market, when you want to buy shares, you need to agree with a seller on the price you pay. If you can't agree, there's no sale. A market-maker is a mathematical formula that allows a Hubdub market to act as if there is always a seller willing to give you the price (in percent) that you see on the Prediction Amount page, and always a buyer (for when you cash-in). Other prediction markets use a system much more similar to the stock market, with asking prices and selling prices. It's much harder to find what you want at the price you were hoping for. Here, the market-maker acts like a supermarket whose prices move according to supply and demand, but always has enough merchandise no matter how much you and others buy. It's not a perfect system though. The convenience of having virtually infinite shares always immediately available for sale (and always willing to be bought back) is balanced by strange behaviour that can appear when there's little capital invested, or when there's a lot. That formula is what they are going to tweak. As to your question, Nigel is saying that the Obama market, which has the weight of a lot of money in it is jumping around too quickly still. The weight of all that money that people have already wagered should mean that your bet can't make it move that much. (Your $100 bet today should be like a drop in the ocean of the $200,000+ already in there.) But in fixing that, we're going to notice that in markets with little money already invested (new questions for instance), a small bet will have a larger effect than it does now. Hope this helps explain this a little. – InfernalMachine, on August 06, 2008 11:40

"3. Streamline the question creation process" Yes please! 1. The question creation page is a bit ugly, and looks really intimidating. Since the best way to learn about questions is betting on them, why not have the question design page look like an actual question page. You type in text where it eventually goes. You type in or maybe even drag the bars to enter initial odds. Add an optional design wizard for newcomers that takes them step by step through the process. Just a suggestion. 2. Consider whether there are the right data structures being gathered and add what's missing. Allowing the user to clearly communicate whether a question must be decided based on what is known on a given fixed date (and prompting them to provide that date, which we might call Decision Date), or if it can have its suspend date regularly extended until the event in question has actually happened, would really help players know precisely what they wagering on, and admins what they are settling or re-opening. If the wizard is well designed, there should be little confusion about Suspend time versus Decision Time, and once question-creators know what they're doing, they can build their question without a wizard. 3. There's a real problem with the actual meaning of questions becoming buried in the background text and the settlement details. (This is doubly a problem because large portions of these are initially hidden.) There are also questions which ask one thing at the top and a contradictory "real" question in the More Background section. I'd like to see the background text split into two text boxes: the whole question and commentary. The whole question contains everything that a user needs to know to understand the question they are actually answering with their prediction. (This is important because the 100 character limit on the question text cannot contain some questions' full extent). If the whole question (you could just call it the question if you call the current question text "headline") is not found in that box, it is not a valid question. Perhaps the whole question also requires a link to a relevant article. The commentary is for comments only. Nothing here can change the interpretation of the question. I think this would greatly improve question style in one easy step. Also, settlement details should appear clearly on the main question page between the whole question and the commentary. – InfernalMachine, on August 06, 2008 11:26

Hold and vote is probably too slow if we open it up to the larger community. But if we use the knowledge that superusers have developed about "good" questions and problem question, I think you can have a fast streamlined process that replaces much of what they do now with a far more effective method of fixing questions before they hit the pavement. If they are also able to (or at least the original question creator is) change odds and add options or take them away (in other words, a submitted question is an editable persistent object that both superuser and creator can tinker with before it goes live), then they can help the creator to fix their own question. If, instead of voiding a question for bad initial odds you simply send it back to them with instructions to fix them, whereupon they send it back for approval, we solve both the needs of the new question creator and the responsibilities of the superuser, with none of the current drawbacks of fixing questions which have already been wagered on. – InfernalMachine, on August 06, 2008 10:51

I just had an idea about stats gathering. Trust is earned. While anyone can create a question, not everyone's word that the stats add up to X can be trusted. One way around this is to develop a small stable of willing and trusted sports stats-checkers within the community..They might even check each others numbers and math. They would be the go to people for you to settle a question. A complicated question would only be allowed to run if at least 2 of these people agreed to help settle it with you. Just an outside-the-box idea. – InfernalMachine, on August 06, 2008 10:33

A simple process of say 3 superusers and the Category Editor (maybe with a few more superusers appointed) quickly checking out new questions with their trained eye for problems would probably save the admins a huge amount of time overall. Once a question is open for wagers, any problems with it become twice as tricky to deal with, and there are sometimes long discussions among admins as to how to handle this problem questions. I think you could probably get down to a max wait time of 8 hours, and depending when the question is published, an average of less than an hour. also, we would probably institute some kind of question-creator ranking, so that someone like sdchargers, based on their track record for "good" questions would not get the same scrutiny as a new player, or someone who's questions keep being voided (that's not to pick on them btw. we all lose when a question has to be voided.) – InfernalMachine, on August 06, 2008 10:22

good point. you could make the first bet unlimited. that is, equal to the normal max bet. only after your first wager does the limit kick in. the other possibilty is to pretend there's already some money been bet by a phantom player. so your bet is no longer 100% of the money bet. if the max market share is 25% and the max bet is $1000 generally, then you put $3000 of phantom money in the "kitty", so that if the player bets the whole $1000, they now have 25% of the total. sorry. should have said that before, but wanted to keep it as simple as possible. – InfernalMachine, on August 06, 2008 10:11