Eternacon 2018 Followup: Finally figuring out the points issue

  • 2
  • Idea
  • Updated 1 year ago
So as the first follow-up conversation to Eternacon, I believe that the issue of point allocations should be finally put to rest. Mostly because it took a little bit too much of the future of Eterna discussion :P

Anyway, my suggestion is this dynamic scoring function to determine the number of points that a puzzle should be worth:



With a function like this, the possible scores (subject to change) would look like this



The older and fewer solvers a puzzle has will increase its score, with a maximum of 500 points, and newer puzzles with more solvers will be worth the fewest points, at 30. And the maximum number of points that a puzzle with many solvers can have is 50 points.

The rate of change for which puzzles gain points can be altered, as well as the fraction lost based on the number of solvers by altering various values here. However, this is all based on the idea of dynamic point allocation and definitions for puzzles. I"m honestly not too sure if that's a viable option for the site given the number of puzzles and continuing growth of the game, so let's just say this post opens feedback, and for those who post ideas, make them *concrete models* for scoring puzzles.

Also, I think something we need to think about before you post ideas for using a bot to solve a puzzle is to realize that the difficulty of many puzzles originates from the constraints. So unless a bot is trying to solve on sequence constraints, then it's not a useful metric for determining puzzle difficulty.
Photo of Brourd

Brourd

  • 450 Posts
  • 82 Reply Likes

Posted 1 year ago

  • 2
Photo of Brourd

Brourd

  • 450 Posts
  • 82 Reply Likes
Oh, two other things.

1st - players who solve their published puzzles should probably receive the minimum number of points for solving, which is 30 points, or 30% of the current 100 points. However, this will still increase the number of solvers by 1.

2nd - This is dynamic scoring, which requires the number of points that players gain/lose to be dynamic. Whenever the database is updated (probably monthly, unless it can be done in real time), players will gain and lose points based on the puzzle value at that point in time.
(Edited)
Photo of Jieux

Jieux

  • 15 Posts
  • 3 Reply Likes
Dynamic scoring would make the game very interesting and I welcome whatever changes the more established and involved players believe should be done in this regard... the only thing for which I am adamant is that I think players should receive ZERO points for solving their own puzzles and that should be retroactive... such is my 13th post in 6 years.
(Edited)
Photo of Brourd

Brourd

  • 447 Posts
  • 82 Reply Likes
Zero points is fine with me ^_^

Ideally all of the above would result in a serious retroactive allotment of points, and then dynamic point awards from this point onward.
Photo of Jieux

Jieux

  • 15 Posts
  • 3 Reply Likes
It was mentioned by Cynwulf28 that the reason he (and others) solve their own puzzles is to save their solutions... I actually think this is very reasonable... What I suggest is that when a puzzle is submitted the solution is automatically saved for the person submitting the puzzle... I still think zero points should be allocated for self-solving, although I also understand the argument that intensive puzzle makers are then being put at a disadvantage by offering to other players points that they cannot themselves obtain... As y'all know I'm in it for the bubbles, not for the points, but my primary k'vetches regard the multistate puzzles (especially the illegal dark side ones) that take minutes to make and are virtually impossible to solve... I'm also not a fan of the Bobs and the spread energies and the like as I feel these are all gaming the system... Again whatever the more established and involved players believe should be done can carry the day, but needed to correct that the previous post was my 14th not my 13th and this will be my 15th. Enough Already!
Photo of JR

JR

  • 241 Posts
  • 20 Reply Likes
I think it will take too long for dynamic points based on # users. so:

I would have 4 independent rankings plus one grand total:

1) puzzles solving==30 points if you solve your own puzzle, 100 points otherwise.
2) Lab solutions==whatever the calculation is now. 
3) puzzles solved last 30 days== (same calculation as #1 (for newer players).
4) Top ten ranking (new)==  puzzles selected with 10 or less solvers, 100 points for solving and no points for solving your own puzzle.
5) Add 1-5 together for overall ranking.
 
Re-calc all for new rankings redo.

  
Photo of Brourd

Brourd

  • 447 Posts
  • 82 Reply Likes
Providing some kind of weighted ranking for solving puzzles with less than 10 solvers is a rather fascinating idea. However, it runs into similar issues like dynamic scoring, in that there is going to be a constantly updated value based on the number of solvers the puzzle has. For example, when a puzzle is first published, the first 10 players will receive 100 points, and then none after that will receive any. It mostly rewards those who are on all the time to solve puzzles, and doesn't alter the reward for more difficult/older puzzles.

Besides that, pretty much everything is identical to the current method of allocating points, which doesn't seem to fix the issue players have with correlating a tangible solving reward with puzzle difficulty.
Photo of JR

JR

  • 241 Posts
  • 20 Reply Likes
Item 4 above is a slow dynamic, the scoring would be to select player puzzles with 10 or less solvers ( or a lower number)  then sum the number of those puzzles each player has solved ( should be fast because not many players have solved those puzzles) then replace the previous ranking with the new ranking ( you can move up in points as well as down)
 
Calculating the ranking Sunday evening weekly should work.
The selected puzzles will change each week.

 You get nothing for solving first. I have no interest hanging around waiting for a puzzle to show up so I can be first solver

And the 30 points for solving your own puzzles does address what I think is the main issue.

So I disagree with your reply.

Photo of JR

JR

  • 241 Posts
  • 20 Reply Likes
If you are worried about who solves what first then you may want to forget thoughts about dynamic scoring and replace it it with a "how long did It take to solve this puzzle" type of scoring.
You would need to create a timer that logs first started and solved puzzle time. Then rank by time. I would think you get one shot which would be when you first open the puzzle.  
Photo of Zama

Zama

  • 37 Posts
  • 9 Reply Likes
I hope you were joking about timing puzzles. I work them for a few minutes at a time over the course of hours or maybe days. How about we just make it slot machine scoring- you solve a puzzle- a wheel spins and wherever it stops.... ;) You could even add dynamic scoring to the wheel's scoring range.
Photo of JR

JR

  • 241 Posts
  • 20 Reply Likes
If you limit the puzzle selection to say "puzzles with 6 or less players" and update the
rank weekly you will be able to rank those players anyway you want, the calculation time will be minimal. I would want that ranking both separate and added to a grand total. 
Photo of Brourd

Brourd

  • 447 Posts
  • 82 Reply Likes
So your request is just to have a ranking for players who have solved some select population of puzzles? I'm sure that's a reasonable request.
Photo of JR

JR

  • 241 Posts
  • 20 Reply Likes
Not quite, not a selection of puzzles, exactly: puzzles with 6 or less solvers older than 10 days. Those players can then be given points for who solved first, etc and/or  just for solving the puzzle.  Re-calculated weekly. 
I think of this as bonus points for taking the time to solve tough puzzles. 

 
Photo of JR

JR

  • 241 Posts
  • 20 Reply Likes
What exactly is dynamic scoring anyway. All I know is that it sounds sexy.
Scripting will take too long as Brourd explained above. A database system
should solve that problem. 
I am talking here to collect ideas.

Creator is to be excluded from following calcs.
Say you have a file keyed by puzzle # that collects first   puzzle#,creator,  first
solver, second solver, etc up to 6th solver. Point allocation is 100 points for solving 
puzzle. ( reply with your prefered point distribution). First through 6th gets extra points based on ranking. Once a 7th player solves the puzzles the extra points are removed. 

I would like to know (1) if this is a direction interest players want to head and (2) how can this be "gamed". - in other words - better try to break it now rather than later.
 
  
(Edited)
Photo of Brourd

Brourd

  • 447 Posts
  • 82 Reply Likes
Dynamic scoring, with respect to puzzles, would be the idea of changing puzzle scores based on some changing variable. In the case of my proposal above, those variables would be time and number of solvers. It's my opinion that finding an algorithm to determine puzzle score/difficulty based on the ability for an algorithm to solve it is quite unlikely. Most algorithms/bots don't solve to the constraints of a puzzle, and there are some structures that algorithms may not be able to solve based on internal inconsistencies or constraints.
Photo of JR

JR

  • 241 Posts
  • 20 Reply Likes
The issue I am having with time is what I call the "new player conundrum". We want
new players not be penalized for showing up to the party late, years late in some cases. Nor do we want to penalize players for taking their time solving puzzles ( the count issue). So I avoid these problems by creating the database of solvers by puzzle# up to a specific number. As an example: Solvers up to 6 receive 700,600,500,400,300,200 respectively ( or whatever). Once solver 7 kicks in those points are subtracted down to 100 for those players on the list and the puzzle is dropped from the database.  A quick glance gave 1500 single player puzzles and 800 switches below the 6 threshold.  
Not cutting off point competition at some point (at solver 7 in my example) and assigning 100 points for a solution will be a bit insulting for us slowpokes so I wish to avoid.
This proposal should give enough leeway for top player competition without dragging everyone else (who may not want to be competitive) into the fray.
Your thoughts please.
  

    
Photo of Brourd

Brourd

  • 447 Posts
  • 82 Reply Likes
Yeah, I understand the 'new player conundrum', but I feel like the main conversation besides that was finding a way to reward players for solving old, difficult puzzles, which I don't see any developer input or comments here for anything. Kind of discouraging, considering that nothing is ever agreed upon if nobody official provides input. :P

The idea of rewarding players based on solving order is fine, but I think it disproportionately rewards people for solving fast, not to mention that the reward isn't something that anybody can benefit from.

I'm going to post a second model for puzzle difficulty which could be useful, and doesn't use a dynamic scoring model.
Photo of JR

JR

  • 241 Posts
  • 20 Reply Likes
Take my proposal and award an even amount of points for the 6 slots. The age of a "Old puzzles" is going to be to much of a moving target to nail down with players coming and going.
Unless I can think of something - awarding points on age of puzzle is way too easy to game.. 
But I will start thinking specifically about " reward players for solving old, difficult puzzles". 
If a developer does pipe in they might answer why "reward players for solving old, difficult puzzles" is important to them. I hate those puzzles.
Looking forward to your proposal. 
Photo of Brourd

Brourd

  • 447 Posts
  • 82 Reply Likes
Well, the time and number of solvers changes constantly. If the puzzle points earned for solving a puzzle changed on a weekly basis, it would reward players with a ranking based on the difficulty of the puzzle they want to solve. And, you can change the minimum and maximum number of points a puzzle can be worth through the function, or change the rate it increases points based on its age. And if a harder puzzle with fewer solvers was actually easier than what it appeared to be, more people would try and succeed to solve them. This would lower the resulting score of the puzzle, and changing the appropriate point totals of players. I don't disagree that it's complicated, and possibly even impossible to implement, but it's (or even another similar function based on only the number of solvers) is the most fair implementation for scoring a puzzle.

I think the only issue I have with your method is that the point value doesn't define the difficulty. It rewards players for solving puzzles first, and with higher point values for solving first. When really, even if it's for the first 5 solvers, I as the first solver shouldn't be getting more points than the other solvers while the puzzle has less than 5 solvers, simply because maybe I solved the puzzle first 4 years ago. However, if all 5 solvers receive the same number of points, then that's not a problem.

Ultimately, the goal is to provide a difficulty rating and appropriate reward for solving any puzzle in the database. We can provide rewards for individuals who solve puzzles first, but it doesn't make sense from the perspective of people who don't want every puzzle in the database to be worth 100 points.
Photo of JR

JR

  • 241 Posts
  • 20 Reply Likes
I think you have to have a cutoff to make the manageable. You could use my framework above but decrease points for number of players solved. For example:
One player gets 800 points, the second solver gets 600 and drops first player to 600 points, 3rd players drops all 3 to 400, etc, 7th player drops all to 100.
Any closer? 
Photo of Brourd

Brourd

  • 450 Posts
  • 82 Reply Likes
Puzzle Scoring Based on Sequence Constraints

Instead of using a constraint for time and number of solvers, an additional scoring method could be based on the sequence constraints of the puzzle.



So this function takes the fraction of G-C and G-U base pairs, and uses them to determine the points awarded for solving a puzzle.

For a brief explanation, C1 and C2 are the max points awarded for solving a puzzle with a certain number of G-C base pairs and G-U base pairs. F_GC and F_GU are the fraction out of 1 for that particular base-pair in a design (For example, for a design with 10 base pairs, maximum number of G-C base pairs equal to 2, the F_GC would be 0.2). Here is a matrix with example scores based on C1=300 and C2=100. C_F_GU is a constant fraction of G-U base pair which acts to shift the center of the parabolic function. The reasoning behind this is that puzzles may be *more* difficult with a high A-U base pair requirement, so you could shift the center towards a higher G-U fraction to account for this.



This method isn't dynamic, given G-C and G-U constraints are fixed, but unlike the previous dynamic scoring, it doesn't capture the true difficulty of some puzzles, where difficulty may exist due to the structure.
(Edited)
Photo of Brourd

Brourd

  • 450 Posts
  • 82 Reply Likes
Also, by multiplying -F_GC by some constant, you can increase or decrease the rate of the function. There is quite a bit of leeway in how points could be awarded here.
Photo of JR

JR

  • 241 Posts
  • 20 Reply Likes
Could you give a specific example with an easy puzzles with lots of solvers and
an example with a difficult puzzles with few solvers. No math or as little math as possible please. Thanks.
Photo of JR

JR

  • 241 Posts
  • 20 Reply Likes
To summarize my comments:

Assign points as follows: 1 solver   - 1200 points 
                                        2 solvers - 1000
                                        etc.
                                        6 solvers  - 200 points
                                        7 and above 100 points
Points for solving own puzzle - 20.

Create new database with 1-6 solvers and creator, add/subtract points as required, delete record from database after 7th player adjustments. 

The above would award solvers of difficult puzzles, not award first solvers.

Yet to be discussed - 30 days rankings and what to do and where to post lab points.