[Strategy Market] Xmas, cubs and illusions

  • 1
  • Idea
  • Updated 8 years ago
Apparently these are bad strategies - so please optimise them for the worst score possible (if they do actually come out negative as expected)

strategy 1 - all base pairs are GC
strategy 2 - all base pairs are AU
strategy 3 - all base pairs are GU
Photo of Edward Lane

Edward Lane

  • 139 Posts
  • 8 Reply Likes

Posted 8 years ago

  • 1
Photo of Jeehyung Lee

Jeehyung Lee, Alum

  • 708 Posts
  • 94 Reply Likes
Dear Edward,

Your strategy has been added to our implementation queue with task id 64. You can check the schedule of the implementation here.

Thanks for sharing your idea!

EteRNA team
Photo of Jeehyung Lee

Jeehyung Lee, Alum

  • 708 Posts
  • 94 Reply Likes
Dear Edward Lane

We are glad to report that your strategy has been implemented and tested.

While implementing your strategy, we have made small changes to the parameters you specified to optimize the performance.

Note that we'll always run a optimization over the parameters you specify, so you won't have to worry about fine tuning all the numbers you use.

Just the idea and rough numbers are enough to run your algorithm!

Length : Your strategy was implmented with 10 line of code.

Ordering : We ran your strategy on all synthesized designs and ordered them based on predicted scores. The correlation of your strategy's ordering with the ordering based on the actual scores was 0.0299821298564. (1.0 is the best score, -1.0 is the worst score. A completely random prediction would have 0 correlation)

Please note that the numbers specified above will change in future as we'll rerun your algorithm whenever new synthesis data is available.

More detailed result has been posted on the strategy market page. Thank you for sharing your idea, and we look forward to other brilliant strategies from you!
Photo of Edward Lane

Edward Lane

  • 139 Posts
  • 8 Reply Likes
Hi Jee,
thanks for implementing the strategy but I don't quite understand the 'scoring system' and so all appears to be a bit confusing.

I think I'll drop you an email with an excel sheet attached so I can explain how I thought the scoring worked.

clearly there is some other system - so it probably means that all my strategy market suggestions - have a scoring system that works in my head but doesn't compare to the existing scoring system. Which may mean they are all rubbish.

The reason I mention it is that this strategy was expected to get a 'negative overall result' - for example if all bonds are GC you tend to get bad synthesis results (which I can see by looking at the results of this strategy market) - but somehow that has still given this strategy a positive score.

Perhaps the small tweak you made is to change the 'polarity of the strategy' ?

Edward :)
Photo of Jeehyung Lee

Jeehyung Lee, Alum

  • 708 Posts
  • 94 Reply Likes
Edward, when we implement your strategy, we perform additional optimization to make sure your strategy is utilized to the full.

Clearly giving the positive to all GC is a bad strategy, so our optimization flipped the scores.

If we have not ran the optimization, then the resulting correlation would be exactly the negative of the current one : -0.02998
Photo of Edward Lane

Edward Lane

  • 139 Posts
  • 8 Reply Likes
hmm you say 'clearly' all GC is bad - and we know it - but it's not 'clear' to a new player unless there is a strategy market that says 'try to build it all with GC' and it's reported as having a really bad score :)

so if you 'optimize the strategy' like you have here by actually testing the reverse of the strategy you end end up with the strategy market suggesting that building all GC is a positive value (unless you make a comment on this linked page saying what has been done) .

So I'm glad that's cleared up in this instance - though I'm now worried about the other strategies - did any of those get 'reversed' in the same way - how would I know? :)
Photo of Jeehyung Lee

Jeehyung Lee, Alum

  • 708 Posts
  • 94 Reply Likes
That is a good point Edward.

According to my investigation, this is only the case where the parameter actually "reversed." In other cases the optimization changed parameters in relatively small magnitude to maximize the performance.

Let us think about a way to clarify this...thank you for pointing this out!
Photo of Edward Lane

Edward Lane

  • 139 Posts
  • 8 Reply Likes
I thought that was what the negative values were for - nupack's test is the only one with a negative score, currently on -0.157 I was hoping that all GC would get a worse score than the nupack test. I'm surprised that it didn't - but I'll address that in an email with an excel sheet probably sometime over the weekend )
Photo of paramodic

paramodic

  • 77 Posts
  • 7 Reply Likes
I think it's that GC heavy RNAs, as well as AUs (if G is absent, then C must be absent as well, otherwise a fold will not occur), and GUs will still come together, at least in the computer model. That means that it still has SOME relevance, but not much.
Photo of paramodic

paramodic

  • 77 Posts
  • 7 Reply Likes
This can be exemplified in any of my (RNAb) series puzzles.