InfernalMachine's dashboard

Yes. Does seem like a reasonable idea.

I have noticed, though, that you often can only get odds-reports on games that day and tomorrow. So questions that are created earlier than 2 days before the game might not have sufficient information to use this method.

This is a very good idea. I've felt that way too. You have a good question, but you're not sure about the initial odds, or how clear your question will be to other players. What I really like is that what you're suggesting is a voluntary thing, No-one makes you ask for inspection. You only choose to if you want. If the question settles very soon, you might not do this. The only thing I would add is that it might be even better if it was sent to a superuser (or a Category Editor). They are really good at inspecting questions too. Other great thing about your idea is that it might be a whole lot easier to set up. Thanks for this. I will pass it on. – InfernalMachine, on August 08, 2008 15:31

But the idea is to have less bad questions, not less good ones. Obviously people who have shown they almost always create good questions would not have their questions held up. But "bad" questions use up a huge amount of staff and superuser time, and also often end up creating anger among the people who bet on these questions before they are "fixed". That's the reason for even suggesting holding questions to check them. – InfernalMachine, on August 08, 2008 05:31

I agree with what you're saying here. This could turn into a huge problem. We will have to come up with restrictions on questions like you describe. But it will probably also turn out that what Tisha says is true. We will not be banning "types" of questions. But types means things like side-bets in general.

What will probably have to happen, if things develop as disastrously as you suggest, is that we will have to ban questions that are too difficult to settle. Normally this means questions where the actual results can't be determined. But in this case it would be more about the number of steps taken, or number of web pages visited, that would attempt to reduce the really time-consuming questions.

It's also possible that Mark will just get fed up and start limiting the number of questions per day like this.

There is a precedent for this. In Business, a number of of rules appeared that mean, for example, that no questions about any stock market other than Dow Jones and NASDAQ are allowed to be asked. But lots of people really hate these rules. Another rule, in Sports this time, is that all questions about games must suspend before the game. But there are a set of questions which may be run in-play, at the discretion of the Category Editor.

So if it gets really bad, Mark, with the support of the entire staff, may simply have to say that all complicated side-bet questions must be approved by him before they can run. You submit them, he decides if he wants to let that one run, and otherwise he voids it.

But notice that at no point were one type of question banned.

I notice you seem kind of in a rush to get this line drawn. But a decision will take a bit of time. Mark started his position a week ago and is just learning what's required. Other staff will have to be consulted. It's not going to be decided right away.

Ultimately, rules that block things irritate people. Part of the answer is also in asking people to be reasonable. If the people who create these questions slow down a bit, and ask less of them, then no rule may be required. In fact, if they don't want a strict rule preventing them from asking any of these, it's in their own best interest to cool it before frustration leads to a rule being imposed. – InfernalMachine, on August 06, 2008 15:26

In the stock market, when you want to buy shares, you need to agree with a seller on the price you pay. If you can't agree, there's no sale. A market-maker is a mathematical formula that allows a Hubdub market to act as if there is always a seller willing to give you the price (in percent) that you see on the Prediction Amount page, and always a buyer (for when you cash-in). Other prediction markets use a system much more similar to the stock market, with asking prices and selling prices. It's much harder to find what you want at the price you were hoping for. Here, the market-maker acts like a supermarket whose prices move according to supply and demand, but always has enough merchandise no matter how much you and others buy. It's not a perfect system though. The convenience of having virtually infinite shares always immediately available for sale (and always willing to be bought back) is balanced by strange behaviour that can appear when there's little capital invested, or when there's a lot. That formula is what they are going to tweak.

As to your question, Nigel is saying that the Obama market, which has the weight of a lot of money in it is jumping around too quickly still. The weight of all that money that people have already wagered should mean that your bet can't make it move that much. (Your $100 bet today should be like a drop in the ocean of the $200,000+ already in there.) But in fixing that, we're going to notice that in markets with little money already invested (new questions for instance), a small bet will have a larger effect than it does now. Hope this helps explain this a little. – InfernalMachine, on August 06, 2008 11:40

"3. Streamline the question creation process"
Yes please!
1. The question creation page is a bit ugly, and looks really intimidating. Since the best way to learn about questions is betting on them, why not have the question design page look like an actual question page. You type in text where it eventually goes. You type in or maybe even drag the bars to enter initial odds. Add an optional design wizard for newcomers that takes them step by step through the process. Just a suggestion.
2. Consider whether there are the right data structures being gathered and add what's missing. Allowing the user to clearly communicate whether a question must be decided based on what is known on a given fixed date (and prompting them to provide that date, which we might call Decision Date), or if it can have its suspend date regularly extended until the event in question has actually happened, would really help players know precisely what they wagering on, and admins what they are settling or re-opening. If the wizard is well designed, there should be little confusion about Suspend time versus Decision Time, and once question-creators know what they're doing, they can build their question without a wizard.
3. There's a real problem with the actual meaning of questions becoming buried in the background text and the settlement details. (This is doubly a problem because large portions of these are initially hidden.) There are also questions which ask one thing at the top and a contradictory "real" question in the More Background section. I'd like to see the background text split into two text boxes: the whole question and commentary. The whole question contains everything that a user needs to know to understand the question they are actually answering with their prediction. (This is important because the 100 character limit on the question text cannot contain some questions' full extent). If the whole question (you could just call it the question if you call the current question text "headline") is not found in that box, it is not a valid question. Perhaps the whole question also requires a link to a relevant article. The commentary is for comments only. Nothing here can change the interpretation of the question. I think this would greatly improve question style in one easy step. Also, settlement details should appear clearly on the main question page between the whole question and the commentary. – InfernalMachine, on August 06, 2008 11:26

Hold and vote is probably too slow if we open it up to the larger community. But if we use the knowledge that superusers have developed about "good" questions and problem question, I think you can have a fast streamlined process that replaces much of what they do now with a far more effective method of fixing questions before they hit the pavement. If they are also able to (or at least the original question creator is) change odds and add options or take them away (in other words, a submitted question is an editable persistent object that both superuser and creator can tinker with before it goes live), then they can help the creator to fix their own question. If, instead of voiding a question for bad initial odds you simply send it back to them with instructions to fix them, whereupon they send it back for approval, we solve both the needs of the new question creator and the responsibilities of the superuser, with none of the current drawbacks of fixing questions which have already been wagered on. – InfernalMachine, on August 06, 2008 10:51

I just had an idea about stats gathering. Trust is earned. While anyone can create a question, not everyone's word that the stats add up to X can be trusted. One way around this is to develop a small stable of willing and trusted sports stats-checkers within the community..They might even check each others numbers and math. They would be the go to people for you to settle a question. A complicated question would only be allowed to run if at least 2 of these people agreed to help settle it with you. Just an outside-the-box idea. – InfernalMachine, on August 06, 2008 10:33

A simple process of say 3 superusers and the Category Editor (maybe with a few more superusers appointed) quickly checking out new questions with their trained eye for problems would probably save the admins a huge amount of time overall. Once a question is open for wagers, any problems with it become twice as tricky to deal with, and there are sometimes long discussions among admins as to how to handle this problem questions. I think you could probably get down to a max wait time of 8 hours, and depending when the question is published, an average of less than an hour. also, we would probably institute some kind of question-creator ranking, so that someone like sdchargers, based on their track record for "good" questions would not get the same scrutiny as a new player, or someone who's questions keep being voided (that's not to pick on them btw. we all lose when a question has to be voided.) – InfernalMachine, on August 06, 2008 10:22

good point. you could make the first bet unlimited. that is, equal to the normal max bet. only after your first wager does the limit kick in. the other possibilty is to pretend there's already some money been bet by a phantom player. so your bet is no longer 100% of the money bet. if the max market share is 25% and the max bet is $1000 generally, then you put $3000 of phantom money in the "kitty", so that if the player bets the whole $1000, they now have 25% of the total. sorry. should have said that before, but wanted to keep it as simple as possible. – InfernalMachine, on August 06, 2008 10:11

There's be no figuring things out for the player. The maximum bet which is mentioned when you're deciding how much you want to wager would just no longer be the static $10,000 like now. It would be the top amount we decide on, until you approach your 25% of market share. Then it would get smaller until it reaches $0. Later on, when more other players have wagered money, it would go up again. The game would do the math for you. – InfernalMachine, on August 05, 2008 11:14

Also consider that there are usually about 5 such markets running at the same time (sometimes 10). So if you can manage the same strategy on each, you can easily pull in $10,000 each business day on almost zero risk. That's 50K a week, and that's not counting taking risks or betting heavily on low odds options that are actually medium shots. – InfernalMachine, on August 05, 2008 11:08

In response to your second point, Nigel, I'd like to throw this idea into the ring.

The correct setting of initial odds is REALLY difficult in some (many?) cases. It is also difficult to determine if another's question has good or bad initial odds. And the problem is compounded because we can't (and shouldn't, to be fair to people who have already predicted) change the odds once a question is open.

Now if we had some form of "question holding", by which I assume you mean what was previously discussed here as user-vetting of questions before they go live, then I have a suggestion for getting more acceptable initial odds. If users (or just susperusers and editors and admins) get to quickly pre-approve questions in some form, then it shouldn't be all that hard to allow the "vettor" to also suggest their own odds. If it requires 3 approvals to go live, then the 4 sets of initial odds (creator plus 3 vettors) can be averaged to produce a more palatable result. This would reduce the strain on creators, give assessors a way to make the odds more to their sense of what's right, and generally reduce the number of voids for bad odds. This system would probably work better in the superuser context than in the larger user-base, because it extends what the superusers are already doing in another form (flagging and suspending questions with bad odds to be voided)>

Nigel,

The low "interest" markets come in several flavours. The main reason they come up here is subsequent to the previous discussion in comments for a Hit-By_Pitch daily total market. It was suggested that the amount of extra Cat Ed work to research and gather the relevant stats was not worth the effort given the low activity or number of predictors. This then spilled into a defense of "side-bets". So in this case, the low interest was only seen as a problem wrt the extra load on the Category Editor.

Other "flavours" of low interest markets include in-play foreign exchange trading (where a few players used to be able to make huge profits with little competition - like the Dow questions but without the market instantly responding to the current state of the index) and markets of such small interest that there is insufficient oversight by the community to determine if initial odds are being set reasonably, or if in fact some kind of gaming is going on.

But, in defence of low interest markets, and I paraphrase Chris (admin) here, I like a good prediction on Scottish football, which isn't all that popular at Hubdub. So a sweeper has its uses, but there's a down side as well.

Perhaps each of these "flavours" can be handled individually. Putting some onus on question creator to share the burden of figuring out the winning option should take care of the first example. Superusers are pretty diligent at finding the bad odds, as are all players in general.

The second case could be handled by 2 steps. First reduce the maximum bet to $1000 from $10,000. If that's too drastic, then reduce it on all in-play questions (which after all are only allowed if they gain admin approval -- the Dow/Nasdaq questions have that approval implicitly as of now).

Step 2, and this helps both the last minute series of $10K bets on the Dow as well as low activity/participation questions, consider implementing a formula that restricts the amount (by percent) of domination any player can have on any 1 market. Example: Say the limit is 25%. If the total of open (ie not cashed-in) predictions on a market is $40,000 and I want to drop $10K on an option, I can because my 10K will be one-fifth of the $50K (after I bet). But if I want to drop another $10K immediately afterwards, which would mean I would have 20K of the total 60K (ie 1/3 = 33.3% > 25%), then it wouldn't let me bet that much. I could (math skipped) bet up to $3333. If I want to bet more, I must wait until others have bet up the total to at least 4 times (based on 25% limit) what I have bet.

Notice some form of this also works wonders with low activity markets, because it forces the player who wants to play it heavily to wait for enough activity to justify the big wagers. This also encourages that player to publicize this market with friends or general community.

Months back I created this NHL play-off question:
http://www.hubdub.com/e/Market/NHL_Pl...

I then spent 10 minutes a day updating and double checking the stats. Every two days (or games as teams got eliminated) I posted the stats to the comments page so that people could see who was winning and check my stats or figures, When it was all over, I tied my settlement request to the stats in the comments (it was my integrity on the line if someone could find an error).

If I hadn't done all that work, there is no way Donal would have allowed my question. And he would have been right.

If you have a complicated question (and by complicated I mean that it involves stats gathering and math operations) that goes beyond the usual work required to settle a Sports question, then the onus should be on the question creator to gather the stats, do the math, and display the relevant info in a comment on the question page, along with a link to the pages or site you used to get the info. Then submit the settlement request and refer to the comment where all the data and math is.

It's a lot of work. If you feel that you don't want to have to do that, then why should an already overworked Category Editor have to do it for you? In other words, if it's too much hassle to do yourself, your question isn't a good Hubdub question, and shouldn't be submitted.

That seems to me to be the only reasonable way to approach this. It's better than "banning" questions. It also makes more sense than a laissez-faire approach where all the extra work is piled on the Editor. Make it easy for the Category Editor to like your "creative" question by removing the donkey work from them. If all they have to do is check some of your stats (which MUST be correct) and recheck your math, they'll be happy and you'll be happy.

That would be great. It would be good to have specific markets with set closing times like for the Dow/Nasdaq. I think you might want to have a restriction on the number of in-play questions per day though, especially with the non-US markets which traditionally have had lower liquidity and participation. – InfernalMachine, on August 02, 2008 12:29

I agree about the problem of slow loading of pages. It's a real problem if you want to cash-in a prediction late in the day. You have to refresh the question page, find your predictions, find the one you want to cash-in and hit the button. More often than not, by this time the prices have shifted, and if you have the "cash-in without reloading" option set, it won't let you cash-in. So you have to go through the whole process again, and sometimes again and again. The code for making predictions is clearly not designed for heavy volume or activity. – InfernalMachine, on August 02, 2008 11:30

I like things as is. Once you have in-play questions, you are going to have people who feel that this is not straight prediction. They are right, to some extent. People who follow the action have much more information to guide their choices than those who don't. But this is potentially true of most questions on hubdub. Many markets shift between creation and suspension, and those who follow the news have a marked advantage over those who bet it and forget it. Even a Sports question about a particular game which suspends before the game starts can be affected by news of injuries to important players.

So, the "purists" do not stand on as solid ground as they may think they do when they criticise in-play as not being pure prediction.

I think that the in-play markets are the most exciting and fun part of Hubdub, and that there is a large community who play the game daily because of these markets. They're here to stay, and the overall game would be diminished if they weren't.

When we come to the issue of when to close these suckers, I personally don't think that pushing back the suspend time really helps all that much, and it does restrain betting activity considerably, or else cause disastrous losses for some players. The reason for tinkering with the suspend time is either given as "making it more about predictions", or "stopping people from making too much money".

If you push the suspend time back and back until you reach Dow open at 9:30am ET, you will progressively reduce activity on these markets, because even a hour before closing it amounts to a guess (or as some call it, a lottery). The fact is, the closing figure of the Dow is in itself not that interesting an issue (except maybe up or down by how many percent). What gives these markets their actual interest is in predicting, in meaningful steps of time, where is the market going from here. If you overly interfere with this, you just make the market too scary to play heavily.

The issue of excess profits being made in the last few minutes before suspension is perhaps a much more pressing issue. But to some degree we may be overestimating the percent profit that can be made at that time. As an option goes up toward 100%, the percent profit becomes really low. Also, the real gains people make in these markets is when they choose a low percent option just before it becomes the most likely (so some high gains are offset by high losses).

The exception to this is the people who put $10,000 bets (repeatedly) on near-sure things at 90%-99%. The max profit per bet here is between $100 and $1000, but a few of them will sum to about $2000 profit on a sure thing. If there's say 5 markets like this, that's $10,000 in profit on sure things in the last few minutes of the market being open. Clearly being already wealthy can easily lead to being wealthier.

But if we want to deal with that, I think there's a much better idea than playing with suspend times. Basically why not have the max bet on in-play markets be reduced to $1000 or even $500? This at least opens up the wagering when a new option becomes the likeliest to more of the people playing.

If, on top of this, we still feel that too much daily profit is being made on financial markets, then I'd suggest taxing the profits on in-play questions. Maybe we siphon off a quarter of the profit at cash-in or settlement time. People get the fun of playing, but we compensate for the imbalance in profit-making that these financial questions have compared to the rest of the site.

Another approach might be to not take any more bets on an option when it reaches 95% unless it subsequently drops below 95%, but that's getting a little too complex.