Wuami charts: a new tool for predicting good OpenTB designs

  • 4
  • Idea
  • Updated 1 year ago
  • (Edited)
Omei introduced me to an awesome lab tool: “I think Michelle has created a new powerful tool for predicting the lab experiment results.”  
What can the tool be used for?

  1. Making better designs as we could get feedback before getting results back

  2. Good TB designs have strong pattern that is easy to learn to spot

As Omei said about the charts: “What's most significant about these is that they are *not* based on the results.  They are strictly based on the NUPACK predictions, and so players can potential get them as they are designing.”

Best round 2 ABC2INC design

(sorted after Global Fold Change ON)

http://www.eternagame.org/game/solution/7113337/7252060/copyandview/

Characteristic pattern in winning ABC2INC designs


Best round 2 ABC2DEC design

(sorted after Global Fold Change OFF)


http://www.eternagame.org/game/solution/7113332/7233512/copyandview/

Characteristic pattern in winning ABC2DEC designs



Other great designs may have a slightly different pattern

Here is one of Nando/Brourds. Notice that the top right and bottom left image. Each holds the C concentration against either A or B. They are very similar.



http://www.eternagame.org/game/solution/7113332/7243586/copyandview/
Omei: At this point, I would think that the most favorable relationship between the upper right and lower left panes would be that they are as similar as possible, but that might not be a requirement.

I think symmetrical around the upper left/lower right diagonal of those two panes would be ideal. Just as symmetry around the upper right/lower left diagonal of the first and last panes would be ideal.


But the lower right is the one that really counts.  The others are more useful for judging how to make the lower right one better.

Additional chart explanation

  • The DEC and INC chart for winning designs, have the exact reverse pattern of each other

  • The scale at the X and Y axis are for concentrations of the inputs.

  • The P in Pbound stands for proportion of the inputs that are bound.

  • The blue color represent fmax - for the most fluorescent light. (fmax >= .9)

As Omei explained: Think of blue and light brown as high and low." Blue represents ON" and light brown represents "OFF"

Afterthought

omei: But then, I think there are going to be innovative designs that don't look good in these charts, just because there are significant aspects of RNA that NUPACK gets wrong.

eli: Yes, like coaxial stacking...

omei: Exactly, and pseudoknots



Thumbs up to Michelle from here.
Photo of Eli Fisker

Eli Fisker

  • 2236 Posts
  • 495 Reply Likes
  • euphoric

Posted 1 year ago

  • 4
Photo of Astromon

Astromon

  • 192 Posts
  • 26 Reply Likes
Do we have access to this tool? Do the colored lines in the pic represent any part of the structure of the puzzle?
Photo of Eli Fisker

Eli Fisker

  • 2236 Posts
  • 495 Reply Likes
Hi Astromon!

Omei has written a news post about the tool plus included a fusion table with data on the two hard labs in Round 2.

The colored charts do not represent the design structure. Just concentrations of the inputs.

You can see the inputs A, B and C at the sides of the charts. The [ ] around the input letters is a short hand for concentration.
Photo of rhiju

rhiju, Researcher

  • 403 Posts
  • 123 Reply Likes
Eli, nice post! can you also include the links to the fusion tables in your post above? i bet some players will find this getsat post first, before seeing omei's news item.
Photo of Eli Fisker

Eli Fisker

  • 2236 Posts
  • 495 Reply Likes
Rhiju, thx!

Here is a link to the fusion table itself.
Photo of spvincent

spvincent

  • 49 Posts
  • 10 Reply Likes
So this is a graphical representation of all those experimentally-derived equilibrium constants?
Photo of Omei Turnbull

Omei Turnbull, Player Developer

  • 980 Posts
  • 308 Reply Likes
No, even though these were produced after we had the lab data, these charts are constructed solely from NUPACK's in-silico prediction.  The fact that they seem to do so well at the prediction of the global fold change measured in the lab is what makes them such a game changer.  (Pun acknowledged.)

The current batch of retroactive predictions can act as a "training set" for Eterna players.  By the end of the week, we should have another fusion table ready containing all the designs currently submitted for the hard designs in the current round.  Players may then, if they want, delete some of their current submissions and replace them with others.  If there is sufficient interest, we might be able to set up a server where you could test new sequences as you design, and then decide which you want to submit.
Photo of whbob

whbob

  • 191 Posts
  • 57 Reply Likes
I'm looking forward to seeing the fusion table for current designs. 
Is there a way to tell where a 6/1 relationship between [A][B] to[C]^2 is on these graphs? Would you use log n of the concentration difference?
I would very much like to use a server for test!
Photo of Eli Fisker

Eli Fisker

  • 2236 Posts
  • 495 Reply Likes
Bigger chart images in the fusion table under Cards


Omei has updated the fusion table and enhanced the size of the charts.



How to find the bigger charts in the fusion table


  • Find the charts under Cards

The new and bigger charts can be found under Cards 1 that is hiding under the main part of the fusion table (Rows 1). Red box highlights how to get to them.






Make your own fusion table copy


The benefit of having your own copy, is that the fusion table will remember the changes you make to it, so you don't have to remake these choices each time you open it again.






Make filters for the cards


Now to make good use of the Cards, it will be helpful to make filters so you don't get all the Cards in one big mess.

Perhaps you only want to see the ABC2INC designs. You can get these by making a filter and sorting by puzzle name.




Choose the lab puzzle you wish to see





Sort by Global Fold Change, Score or KDON



For instance, I want to have the Cards sorted by Global Fold Change.

Omei explained: You can filter these just like you do for rows.  To change the sort order, click on the tab and then "Change card layout"




How to change the sorting




By having the setting Sort Z to A, you will get them sorted in the order of highest to lowest.




You can also change how many Cards you get shown side by side on the screen.



Now save and you are good to go.



Further advice for reading the graphs



For reading the numbers on the Wuami charts:

  • 1e-15 is a lower concentration of the inputs than 1e-3.
Photo of Omei Turnbull

Omei Turnbull, Player Developer

  • 980 Posts
  • 308 Reply Likes
OK, we've now got wuami charts for the current Round 4 submissions to the full A*B/C^2 puzzles. (As of 3/29/18.)  I see a much bigger percentage of promising (looking) designs compared to Round 2.  At least some of those are almost surely going to be improvements over the best Round 2 designs!

The devs are going to be discussing the possibility of setting up a server so that you can get a wuami chart "on demand".  I can't predict how long it would take, but it's more likely to be minutes than hours.  But a lot will depend on the level of interest expressed by players.  So if you have strong feelings about their usefulness (or not), today is the day to express that.
Photo of Eli Fisker

Eli Fisker

  • 2236 Posts
  • 495 Reply Likes
I think the Wuami charts are going to be extremely valuable for our designing for openTB.

Here is the perspective: These charts will grant us all the power to see if the design switch in the right direction. This is what jandersonlee was capable of doing with math for the Logic Gate labs and the Riboswitch labs, that allowed him + Mat and I with him, to hit winners extremely fast. The charts will help us all to hit target way faster. 

Can we have the Wuami charts, please... :)
Photo of Astromon

Astromon

  • 192 Posts
  • 26 Reply Likes
I would like a wuami chart please!
Photo of Omei Turnbull

Omei Turnbull, Player Developer

  • 980 Posts
  • 308 Reply Likes
@spvincent pointed out that the wuami chart I used on the news item as a very promising INC design didn't resemble much the best one from Round 2.

That's because none of the INC submissions were particularly good in Round 2. We thought perhaps it was inherently harder than the DEC version. But for a diagnostic device, we really want both. Of course, I could be wrong, but this design by Andrew Kae sure looks good to me, based on what I've observed so far in wuami charts.
Photo of Eli Fisker

Eli Fisker

  • 2236 Posts
  • 495 Reply Likes
@spvincent and @Omei, you both bring out an important point. That we did not have good inc designs in round 2. 

Omei, I agree with your judgement of the above design, since it is the mirror opposite of the better ABC2DEC designs. I count this a good sign. 

This also makes me think that it is important that we get the Wuami charts from round 1 where there were good ABC2INC designs. 
Photo of Omei Turnbull

Omei Turnbull, Player Developer

  • 980 Posts
  • 308 Reply Likes
In a comment to the recent Eterna news post, @JR asked if I could post some examples of well-scoring vs. poorly scoring designs from OpenTB Round 2, where we used the same oligos as the current Round 4, but now have real lab data to compare against.  These screenshots are from the fusion table for the Round 2 results that Eli posted about above.

Here are the four DEC designs with the highest Global Fold Change.

... and here are four near the bottom of the rankings.


Michelle is generating four charts for each design.  The most important one, which summarizes the algorithm's prediction for the interaction of all four inputs, is in the lower right corner.

Here's a blow-up of the lower right corner of the first design, where I have added some annotations.

If the design is good at all as a DEC switch, the upper left corner will be dark blue and the lower right corner will be light brown.  If you see a design where the color of these two corners are switched, you are probably looking at a INC design.

The next thing to look at (I believe) is the red line segment that I have overlaid on the straight part of the yellow band.  Paying attention only to the straight part is important because being straight indicates the prediction that there is a linear relationship between the log concentrations of A*B and C^2 in that range.  Having this linear relationship reflected in the lab results is a requirement, but by itself it is not enough.

The third thing to look at is whether this linear relationship holds in the full range of concentrations that we expect to see in human blood. I actually don't know what that range is, but based on the original paper, I've drawn a grey rectangle of a reasonable estimate. So the extent of the intersection between the linear boundary and the central box is important.

The fourth thing to look at is whether the red line is parallel to the cyan diagonal I have drawn. If they aren't parallel, it means the prediction of the C exponent is something other than 2.

Lastly, it is relevant how close the red line is to cyan diagonal.  For the perfect switch, it would be pretty close to, but above, the diagonal. 

Keep in mind that this is all new, and no one expects the charts to be perfect at predicting the lab scores. The question is simply whether we can do a better job with them than without them, and I have little doubt the answer is yes. With more more eyes and more time, we'll get better about reading the charts, and perhaps the charts themselves will evolve.
(Edited)
Photo of dl2007

dl2007

  • 4 Posts
  • 0 Reply Likes
This design has not identical upper right and lower right panes, but it has more or less
even car like forms in all four panes and good colors. Is it looking good ? Any opinions ? 
Photo of Omei Turnbull

Omei Turnbull, Player Developer

  • 980 Posts
  • 308 Reply Likes
We haven't really seen good solutions for the INC puzzle yet, so I feel like I'm going out on a limb, but here's my thinking.  The pane in the lower right looks quite good -- a nice long linear boundary that covers ~3/4 of the central region.  On the other hand, the two panes on the left suggests that the B oligo has an excessively large influence on the result, compared to the A and C.  But this is in turn balanced by the upper right, where A has significantly less influence than C.  So maybe it all works out and it is a great design. 

In any case, given that we have so little experience, I would say it is definitely a high value submission.
Photo of Astromon

Astromon

  • 192 Posts
  • 26 Reply Likes
Thanks, Looks like small amount of green for good designs are INC the same? Will a good design for INC have blue on right side and brown on the left? here's an observation >    http://prntscr.com/j0kyci    .
Photo of Omei Turnbull

Omei Turnbull, Player Developer

  • 980 Posts
  • 308 Reply Likes
The four corners of each of the 4 panes in the wuami chart represent combinations of very high and/or very low concentrations of the inputs.  As such, I'm inclined to think they aren't very predictive of how good the design will be. But you, like all players, should be free to make your own hypotheses about what is significant, and design accordingly.
Photo of Omei Turnbull

Omei Turnbull, Player Developer

  • 980 Posts
  • 308 Reply Likes
@Eli just drew my attention to the wuami chart for one of his current submissions

We won't know anything for sure until we get results, but this looks to me like the most favorable OpenTB chart I've seen. He is going to follow up with some elaboration as to how and why he created the design, but I'm so stoked I asked whether I could call attention to it immediately.
Photo of Eli Fisker

Eli Fisker

  • 2236 Posts
  • 495 Reply Likes
I have been asking Omei questions about some Wuami charts.


Monster entangled hairpins

By the way, I found something else I consider interesting.


https://s3.amazonaws.com/eterna/labs/concscan_R108/8524273.png

I made a design that resembled the style of some of Brourds round 1 designs, just it has coaxial stacking.

It is basically one big entangled hairpin. It gives a real long stretch of linearity for the top left and bottom right windows.


Link to the lab design itself.


So after getting the charts I'm now working on making variations of it. With its inputs moved around.

This design above has the leaving inputs before the staying inputs. This was why I made this exactly combo of input order in it. Because I have seen earlier that OFF switches favors having the leaving inputs before the staying inputs. Whereas ON switches tends to favor having staying inputs before leaving inputs.


Background post

Input order - depends on if it is an ON switch or an OFF switch



Long linear stretch - long switching stem


The reason this particular chart caught my attention was that it had a far longer linear stretch than the others.




And the thing I think cause this linear behaviour to go on for longer is exactly what I have been rather concerned about and mostly trying to avoid. Longer switching stems.

NB, the switching stem isn't one long continous stem but rather a couple of switching stems right after one another. Its an arm that has an "elbow" (loop).

I still think a real long switching stem with no breaks is a generally bad idea. Although some leniency likely will be given if the stem has some more switch like sequence, as GA and GU rich sequence in the strands. Or using weaker bases like AU or GU inbetween.



Long switching stem - timed delay?


I think the longer hairpin acts as a brake, and slow down the switch between the C's being on and the A+B being on.

I basically think this long hairpin may act as a time delay. Just as the one mentioned in the video on computing with dominos that I called attention to when we began solving TB switches. (minute 8)






Selffolding elements

eli: My selffolding element design type sparks a whole other type of pattern. Absolutely not linear. They are probably no good. But I wonder if this approach may be useful for something else, since these switches sparks a really sharp shift.



omei: Since all the graphs that include C predict that the C concentration makes no difference, I would conclude that for some reason C isn't (predicted to be) having the effect it should.

eli: Ok, that would actually make sense. I put in C as a C lock, so it were supposed to hold the A and B under lock. But that bit isn't working then. So while this worked well for sublab, the C may not be strong enough to do the job. And even if it did do the job, the switch would be more like quick shift and not a gliding version.


Wuami charts for round 1 available


Omei made round 1 wuami charts available. You can get to them like this:

https://s3.amazonaws.com/eterna/labs/concscan_R104/7001278.png

Just interchange the DesignID with the last number to see the design.

But also a heads up. Here is Omei's judgement:

"I've looked over round the wuami charts vs the fusion tables for OpenTB Rounds 1 and 3.  They don't seem so consistent as for Round 2, but then I don't see any particularly good Global Fold Changes, either."


Round 3 should be up and available in the same link access style, but I haven't been able to locate them - I did change Round number too. Spill if you get lucky. :)





(Edited)
Photo of Omei Turnbull

Omei Turnbull, Player Developer

  • 980 Posts
  • 308 Reply Likes
I think I've resolved the issue with the OpenTB Round 3 wuami charts.  The URL https://s3.amazonaws.com/eterna/labs/concscan_R106/6982575.png now works for me.  You should be able to just change the design ID.  Let me know if there are still problems.
Photo of Eli Fisker

Eli Fisker

  • 2236 Posts
  • 495 Reply Likes
How to get a look of just your own charts


If you want to get an idea what your own charts look like, a way to isolate them is to look up your player ID number. 

You can find it on your profile page in the link. Let's say you are Astromon, this is how your player ID number will look like: 





Then you just insert that number in the filter under Player ID




Link to the spreadsheet
Photo of Omei Turnbull

Omei Turnbull, Player Developer

  • 980 Posts
  • 308 Reply Likes
Update: wuami charts for the hard designs in Round 1 and 3 are now available on Amazon's S3. The URLs follow the same pattern as those for round 2, i.e. https://s3.amazonaws.com/eterna/labs/concscan_R<synthesis round>/<design ID>.png.  Examples are https://s3.amazonaws.com/eterna/labs/concscan_R104/6978045.png and https://s3.amazonaws.com/eterna/labs/concscan_R106/7003106.png.

Since the specific results are not directly pertinent to Round 4, I'm not planning on collecting them into a fusion table unless someone says that would appreciate that.
Photo of Eli Fisker

Eli Fisker

  • 2236 Posts
  • 495 Reply Likes
I'm still having trouble getting to the round 3 designs. 

Here is a round 3 design id: 7306248

https://s3.amazonaws.com/eterna/labs/concscan_R106/7306248.png

Link to the design. 
http://www.eternagame.org/game/solution/7254825/7306248/copyandview/
Photo of Omei Turnbull

Omei Turnbull, Player Developer

  • 980 Posts
  • 308 Reply Likes
You're right, they still weren't public. :-(  I think I've fixed it now.
Photo of Eli Fisker

Eli Fisker

  • 2236 Posts
  • 495 Reply Likes
Working now :) Thx!
Photo of Eli Fisker

Eli Fisker

  • 2236 Posts
  • 495 Reply Likes

Making Monster entangled hairpin designs from scratch


I have gotten some questions in relation to my Monster entangled hairpin design/series and a related design. I have done a mashup of my replies and added some extra.


Here comes some details on how to make this puzzle type from scratch or fairly easy make variants by swapping inputs around in already solved designs.


I have been making all my Monster entangled hairpin designs over the same basic template as the first design that got the great wuami chart that Omei called attention to. I will work from that design as an example.


I wish to see my little Monsters replicate and mutate, since Wuami’s chart made me believe in them. So I have decided to cite Omei:


"Well, you can certainly quote me as thinking this is a very positive approach worthy of more designs, regardless of whether they are small mods or reusing the pattern behind it."



The puzzle and its pattern


First the design with state 3 and 4 (which are the only ones that move - state 1-3 are identical)


Link to the design itself


I have drawn up its main character



Recall the pattern of the entangled hairpins in the forum that I shared earlier?


Here it is:


The Monster entangled hairpins pattern is basically the same pattern - just not a double pattern as the above. Plus spread out over a whole lab design.


Omei drew up the simplified pattern.


But instead putting in just one input (R or A) at the time, I play them in pairs and triplets. (eg AB and CCR)



Reverse engineering the making of the design


I took my original design and reversed the process of making it, to show how to make designs like these in action.



1) First I put in the inputs, at both ends of the sequence


Put in the inputs in in the order you want them at each end of the sequence. Then you will have a yellow A stretch in the middle.





  • C-inputs:


You need something like 10-12 bases from each of the C inputs. Typically 11 will work.



  • A and B inputs:


From the A and B inputs I typically take a little longer stretch. 11-13 bases.



  • Reporter: I don't use the full sequence of the reporter



2) Then you "split" that middle section in two portions. Each of these you make complementary to the far opposite end.



Or shown simpler




After having filled in the two middle regions so they are complementary to each its far end of the puzzle, I will typically have an unstable puzzle.


To get it stable I change a bit here and there in the stretch complementary part to the input. (Highlighted by black rings)



How to get the puzzle stable:


  • Mutate in the middle of this middle stretch

  • Make stronger complementarity to one or the other far end sets of inputs

  • Make an AU or a GC into a GU

  • Make a mismatch



3) Never mutate in the input complements. They need to stay true to the original inputs. Only mutate in the sequence that targets them, if necessary.


The middle complementary stretch (the stretch of A’s) of the puzzle is basically the only part of the puzzle that needs change.



4) If the portion of the inputs that you have made complements to, are not optimal for whatever reason, you simply slide the complement to get grip of a better stretch of the input. Similar if you want to make variations of your design, this is a way to make them.



Vary your C’s


One more thing. The stretch you pick for pairing with the two C inputs, it’s a good idea not to make them totally identical. Else your complementary sequence may favor taking the nearest C for turnoff and not the one furthest away.


Omei: Re "The stretch you pick for the two C's, it’s a good idea not to make them totally identical".  did you violate this in the specific example of http://www.eternagame.org/game/browse/8489874/?filter1=Id&filter1_arg2=8524273&filter1_arg1=...?



Eli: Well noticed :) You are correct. It was my first one and I got lucky. Lucky in that it was working. In later designs I got in trouble for it. Also it may not work in lab.


I learned the hard way that it is not probably not a good idea picking the exact same region from both C inputs. You may have the turnoff sequence target the nearest one instead, which is not what you want. So vary them a bit.



List of variations in the Monster entangled hairpin style



How to make more monsters? - Freeze it!


When you have a working design, you can move around the elements in it and make other input combos. You don't have to work from scratch all over again.

When you have first arrived at a stable solve and want to swap some inputs around, the freeze function is worth gold. Just pull a freeze and the structure sticks. You can then undisturbed slide around inputs.


Then go check the state where this input complement is to bind and update the bases to match it. Then pull an unfreeze.


After this you will often need to do a bit of work around to make the puzzle fit to the new conditions.


Be patient, it will take a while getting a stabilized solution when you work from scratch. But you will get the hang of it and it will get easier and easier.


I have used the freeze option to move around ccr to rcc etc. Even to swap around eg cc and ARB parts. In the latter case I just delete all the bases in between the input binding sites.


If you search for monster in the title field for the hard INC or DEC lab, you will find my puzzles that are made in this style. You will get an easier time making a new input variation, if you move around inputs in some of mine.



Expectations and predictions


  • I expect the Monster entangled hairpin series designs in ABC2DEC to do overall better than the same in ABC2INC.

  • For ABC2DEC I expect the #leavingbeforestaying designs to do better in general than the #stayingbeforeleaving

  • For ABC2INC I expect the #stayingbeforeleaving designs to do better in general than the #leavingbeforestaying


#stayingbeforeleaving - the staying inputs are placed before the leaving inputs

#leavingbeforestaying - the leaving inputs are placed before the staying

(Edited)
Photo of Eli Fisker

Eli Fisker

  • 2236 Posts
  • 495 Reply Likes
Another thing that could also be interesting in relation to the Monster entangled hairpin puzzles, is to vary the length of the C's input in relation to the A and B inputs. 

Make the C's shorter in one version and the A and B's longer. Plus vice versa. This I think will change the balance between the A/B and C's part.

This kind of puzzle is kind of a teeter tooter, a balancing act. Better results may come from getting the balance just right. And to get to that, variations over this theme could be a way to achieve that.  
Photo of Eli Fisker

Eli Fisker

  • 2236 Posts
  • 495 Reply Likes
Potential chart window fusion?


By the way, I have considered something. I still haven't had luck with it. I share it in the case it may be of use

omei: Yes?



eli: I have been thinking about fusing parts of designs, with other designs in cases where only one of the conditions looked really good.

So fuse parts from a design where the top left window is good, with a design where the bottom right window is good. Still haven't had luck. But considered it to be a potential short cut for learning what may help balance A versus B

omei: I read the chart as saying the A vs B relationship is very good, but the C's aren't doing their part.

eli: Yup
The A vs B part is separate and could get cut out from that design
I basically made them form a long hairpin with themselves
Plus added a few extra bases as lock on them.
As far as I recall. I have different designs giving a similar chart to the above
I have tried integrate it in a Monster entangled hairpin design. But it is a bit different. I hope I may succede

omei: From what you say, my first thought would be to loosen that lock.

eli: Ok, it has a lock in the hairpin itself, plus a C lock on both sides of it
Actually a double C lock

omei: ... or perhaps, if you have the C complements adjacent to to the lock, put some space between them, so that you don't have a situation where coaxial stacking is allowing the C's to bind without opening the lock.

eli: Ah.
Ok, There isn't much space to give from


Design link

But hereby the idea is passed on. There are several designs that have good A/B windows for the ABC2INC designs in this round. Perhaps we may be able to use some of them for a fusion. 
(Edited)
Photo of JR

JR

  • 241 Posts
  • 20 Reply Likes
My take for a good design is: First rotate upper LHS clockwise 90 degrees.  Numbered clockwise: charts 1,2, and 4 are similar. Chart 3 is symmetrical around a 45 degree downward sloping line
splitting the chart in two.  
Photo of cynwulf28

cynwulf28

  • 82 Posts
  • 23 Reply Likes
I noticed the same thing, but also, each quadrant (sub-chart) rotates by 180° to get from the wuami chart for ideal ABC-INC to the wuami chart for ABC-DEC.
Photo of Eli Fisker

Eli Fisker

  • 2236 Posts
  • 495 Reply Likes

12 april


Omei: 

Since the wuami charts are voodoo to me, I don't know whether this is even possible -- but the "ideal" wuami chart on the "ideal" OpenTB design would have the border stripe along the full main diagonal, with all blue on one side and all buff on the other.


This would be a prediction that there just 2 possible foldings, full ON and full OFF, with the desired relationship between the inputs determining which state it folded into.



More uses for wuami charts


I have been wondering about if we can use the wuami charts to tell us about patterns for other labs than the hard. Here comes what I see so far. The following is from a discussion with Omei the other day.


Slightly abbreviated and with a little added also.



ABRI and ABRO wuami chart patterns


There really is a pattern for the best scoring A (AND) B - RI and A (AND) B - RO labs. I can now say what they want. Or what they ideally want.


I bet we can use the B/C and A/C windows too for the A/C and DEC sub labs. To tell if the design switch in the right direction. :)


Here is the wuami chart for the winning ABRO design, Round 106/ round 3. The reporter needs to be off when A and B binds. Hence there is absence of light. (No bright blue)



Design


Top left window shows the relation between concentrations of A and B.


Here is the wuami chart for the winning ABRI design (round 106, round 3)



Design


The reporter needs to be present when A and B binds. Hence strong presence of fluorescent light. Meaning presence of bright blue.


I also think it will be good if the window is symmetric.


Judging from past winners, this current round ABRI design is probably good:



Design sequence from ABRI: ACUCUAACAUGGUAAACAAACCAUCUAUCACCAUGUUAGAGCUUAGUAGUCUGUUCUUGUAUCUGUUAAAUUCAGACUACUAAGA


I haven't had the server chew through all the ABRO designs yet, but judging from past winners, this current round design is probably among the better ones:


eli [5:41 PM]

Design sequence from ABRO: CAGCCUGUUCUAGCAUCUGUCCUCUGGAACAGGCUGGUCAGAACUUAGUUCAUGGUGAUUAGAGGCAAACCAUUUAUCACCAUGG


eli [5:51 PM]

The reporter needs to be present when A and B binds. Hence strong presence of fluorescent light. Meaning presence of bright blue.



ABRO and ABRI pattern sum up


eli [6:06 PM]

ABRO - A/B window as much light brown as possible in a top right square. With a half glorie of blue around at bottom left.

ABRI - A/B window as much bright blue as possible in a top right square. With a half glorie of light brown around at bottom left.


This way there is an indication of a switch between A and B being ON and OFF, shifting with high or low concentration of A and B.


omei [9:41 PM]

@eli your AB charts are very interesting.  They show me I need to be more cautious about my interpretations of the chart, as they are not what I would have expected.


Do you have the URL for a batch or batches of AB puzzles from round 2?


eli [9:45 PM]

For round 2 of the AB puzzles, I have no url's, I think I ran single sequences of some of the top designs to get an idea of the wuami chart patterns.



ACDEC, BCDEC, ACINC and BCINC patterns


eli [9:49 PM]

I'm finding patterns for the AC and BC labs too


eli [9:51 PM]

The AC and BC labs are not as clean as the hard labs and ABRI and ABRO, but there still is a pattern. It is slightly different between rounds.


The AC and BC labs could also be solved in more different ways than the harder labs, so it makes sense that their wuami chart patterns vary more too.



eli [10:01 PM]



omei [11:09 PM]

Well, to start with, Michelle chose the parameters specifically to evaluate the hard puzzles.

If she wanted to evaluate them as A * B puzzles, she would have chosen different parameters and we would have gotten a different set of charts.  How different, I don't know.


eli [11:11 PM]

Good point

Still there are patterns to find which is what I find really interesting



omei [11:14 PM]

So for evaluating designs as other puzzles, we're really trying to extrapolate beyond the intended purpose.  That doesn't mean that it is useless. But it does mean that they may not conform to my expectations.


eli [11:14 PM]

The AC and BC off switches have the blue to the left side in the window, whereas the AC and BC ON switches has the blue to the right side of the window.


So I think we can already use this to say if the puzzle is switching in the right direction. Not exactly what shape the blue should have, but still a hint.


I took the top ten designs from each round and ran the charts for them. Here are the links I have:


Top r2 acinc

http://wuamicharts.eternagame.org/result/29ad57389a24245e40181f52c7662993/


Top r1 acinc

http://wuamicharts.eternagame.org/result/f1e872cc5fa986e2d434a31af7260ecb/


Top r3 bcinc

http://wuamicharts.eternagame.org/result/d9fc377a5ce1f046a8fab98ec7e6613d/


Top r2 bcinc

http://wuamicharts.eternagame.org/result/261203c7351a5564374b83797c8663c9/


Top r1 bcinc

http://wuamicharts.eternagame.org/result/adafd1920313135ea01a2ed05cafb7d0/


omei [11:24 PM]

The ones I've looked at seem reasonable me.  Like the hard puzzles, the most important thing to look at is probably the center part of the diagonal (in the appropriate pane).

Photo of Eli Fisker

Eli Fisker

  • 2236 Posts
  • 495 Reply Likes

Controlling the concentrations of the switch


Here is a discussion between Omei and I from yesterday. Slightly abbreviated.


Design


I got stuck on my last two INC monsters, so I decided to look at the ACDEC lab instead. I have managed to make a design that looks really good, from what I think you would like it to look.


(top right window)


eli [10:02 PM]

Even better, I can control if the light brown fills more than half the diagonal or less.



Opposite example, where the blue dominates.


Sequence: ACCUGUCUCUCUAUGUGUCAGAACUUAGUAACAGAUGCAGGAACAGCGCACACAUAGAGAGACAGGUGCCUGUUCCUGCAUCUGU



eli [11:17 PM]

There are two ways to do it.

1) Reduce the length of the input at the one end of the sequence or the other. That way you can move the A or the C input to dominate the puzzle.


2) Mutate the full middle stretch (the base stretch that is not paired with the inputs.) The first half of the mutants tends to move the light brown to the one side of the diagonal, the last half tends to move the light brown to the other side of the diagonal.


I get a longer diagonal stretch if I don't use the full input lengths.


I bet we can use the same approach for the monster puzzles. Since the ACDEC puzzle is of the engangled hairpin type.


omei [11:22 PM]

This is great!


omei [11:22 PM]

Yes, panes 1-3, at least, are what I would like to think represent a very good switch.


eli [11:23 PM]

Why pane 1 + 3?

I have focused solely on the top right window as it is the A/C concentration


omei [11:23 PM]

Good, in this context, refers mostly to the range of A and B concentrations that the right relationship is maintained.


On the other hand, I'm not sure whether the wuami charts tells us everything.  In particular, I’m not sure how sensitive they are to the energy difference between ON and OFF states, which puts a maximum on the fold change.


They suggest that oligo B is not playing an unwanted role.


That’s probably not a big deal if all we care about are the sequence as an A/C switch.


But if we want to use it as a component of a full switch, we would ideally like the A/C part to be independent of B and the B/C part to be independent of A.


eli [11:30 PM]

That’s interesting. What you suggest is that we can also use the wuami charts for finding good candidates for fusion for the hard labs.


omei [11:31 PM]

Back to the maximum possible fold change between two states, potentially the width of the transition region (from blue to buff) should be relevant.  But it may not be sensitive enough to be useful.


Once I saw the chart you just posted, yes.


eli [11:32 PM]

So far I have picked the A component to fill more than the diagonal, since it is the one that I want to be on in the final state. So I choose it to dominate.


omei [11:33 PM]

I've been aware of the fact that many of the A/C and B/C designs haven't been "pure", but hadn't followed up on that line of reasoning.  Until just now, when I saw your chart.


What I expect to happen as shift the divide, assuming you keep it parallel to the main diagonal, is that you will be lowering or raising the concentrations at which it switches, but won't be modifying the fold change itself.

(By "divide" I mean the rainbow stripe that separates the blue from buff.)


So being able to dial the switching concentration in as we go is a big plus, since we don't know enough about the actual device to know what concentrations will work best there.  They might be different than the range that Johan's experiment uses.


eli [11:42 PM]

I'm really happy about how useful the charts turns out to be


I can use them to control my switch much tighter.


A lot of my sub lab switches looks good as far as I have checked, but I can use the charts to make some even better versions


Entangled hairpins looks really good for the sub labs too

(Edited)
Photo of Eli Fisker

Eli Fisker

  • 2236 Posts
  • 495 Reply Likes

Rumors of hidden gems


The way NuPACK is set up in lab, it underestimates designs that are potential winners. Here is how to spot them.


As long as state 4 folds the same way as states 1-3, and state 4 is easily disrupted, I consider it a promising submission. (Omei)



Puzzle collaboration


First Cynwulf started a lab puzzle collaboration with me by sending me a puzzle starter. Then Omei took up the thread. Among us we have brewed up two batches of monsters.


Omei had a stable monster. I however wanted mutants that had the complements pairing strongest possible to the complements of the complements. (red line)


CUGUUCCUGCACCAUCCAUCACUCAGAACUUAGAGAGAGACAGGUAAGAUGGUGCAGGAACAGCUCUCUAUGUACCUGUCUCUCU


The mutant we had stable had a GU pair. I made that a GC which made the design unstable.



Here is the wuami chart of the above design before I started mutating it.



Here are a few of the mutant wuami charts:




Eli: Here is the mutant run I wanted to run on your monster: http://wuamicharts.eternagame.org/result/1cbc67d755d8c8ead4f2c4b5a596d7c7/

nr 3 looks quite good. It isn't stable on its own

I plan on running further mutations on that one.


[It has a much longer diagonal line in the top left and bottom right window. Which are the pattern we are striving towards.


Both designs are unstable. Since they are unstable, they are not designs we would normally consider submitting.]


Omei: I suspect that for #3, the game would show state 4 in the same folding as 1-3, because the mod moved the linear divide past the main diagonal.  Is that what happened?


Eli: Yup


Demonstration that all 4 states folds the same:


And then usually what is needed to get a stable solve is to make a GU out of one of the middle bases or a mismatch


Omei: I consider that a "win" because in the lab, it should just show it switching at high reporter concentrations.


eli: So I should probably pick a couple that go to either side of the diagonal, as to get a better chance of hitting right?


omei: That's what I think, yes.



How to effectively mutate monsters?


Eli: I picked that specific mutation spot, because it was just in the middle of the "middle complementary to complement" bases, where I expected to get the most out of a mutation run (base 47-49)


Omei: Since we won't even be using this reporter in the diagnostic, that doesn't concern me at all.


Eli: I'm using this spot as kind of a tip the balance between the states


Omei: Exactly.


eli: It’s where I expect the most gain for the least effort

Other general mutations in the complement to the complements area will likely work too, but this is the fastest way to get the potential best ones. (red line in the first image)



What way should the design lean in the Wuami chart?


Eli: By the way, I'm a bit in doubt with this. I was assuming that for the hard INC puzzles it was good to have the blue part dominate the 4'th window.

I mean expand a bit beyond the main diagonal. Since they have to get turned on


Omei: It's hard to know without actually getting lab results.  But more is not the same as better in this case.


As long as state 4 folds the same way as states 1-3, and state 4 is easily disrupted, I consider it a promising submission.


Eli: So I should probably pick a couple that go to either side of the diagonal, as to get a better chance of hitting right.


Omei: That's what I think, yes.

(Edited)
Photo of Eli Fisker

Eli Fisker

  • 2236 Posts
  • 495 Reply Likes

Diagonally - Strawberry stairway to switch heaven



          

           Image credit


Max Goff asked me about the Wuami charts:


Hi Eli, so what are you looking for when using the wuami charts to evaluate your current lab designs? Do you put much stock in the percentage of adenines in a design being lower or not? Anything else? I've hit the limit on submissions and would now like to delete the poorer ones to create better ones -- but I'm a bit stymied as to what constitutes a 'better' design. What criteria do you use?



Hi Max!


I don't look much at base percentage for the time being, as long as they are not to any extremes. I do try to avoid longer stretches of G and U's. But I don't always manage and I sometimes forget.


Here is what I look for, for the hard labs:


1) Diagonal line in top left window

2) Diagonal line in bottom right window

3) Top left and bottom right window being alike.

4) The longer the diagonal line - the better



Image examples on promising designs



These are examples of designs that have good wuami charts. Most are picked from this forum post. But I think having them beside each other will help make the pattern stand out.



I have drawn in a read line to highlight the diagonal line present in the designs.



ABC2INC



ABC2DEC




The two hard labs have the opposite color pattern.




Wuami charts and the sequences


I have tried pick different types of designs to show that there are different ways of getting good wuami charts.


Here are the wuami charts themselves and the sequence to the designs. You can copy out their sequences to find them in the labs. A few of these designs are spillovers from round 2.


Promising ABC2INC designs


http://wuamicharts.eternagame.org/result/66b55ce38840671775f1ccd948f10997/

http://wuamicharts.eternagame.org/result/6a23091bcb10296bb2095d26008c7398/

http://wuamicharts.eternagame.org/result/cd491649c2859d6680b1c97fdf288a6d/

http://wuamicharts.eternagame.org/result/0526c005812ce4ef4ca63a3bf25c019a/



Promising ABC2DEC designs


http://wuamicharts.eternagame.org/result/c66b606d769679d09ecdedd35b3f5e9e/

http://wuamicharts.eternagame.org/result/aba48e665cf4467d0ba2e555f0d81e73/

http://wuamicharts.eternagame.org/result/916b5de57a19122ca032db0b7c5bf3a4/

http://wuamicharts.eternagame.org/result/5d45696b3fdf04508a1845b5622aa01a/

(Edited)
Photo of Eli Fisker

Eli Fisker

  • 2236 Posts
  • 495 Reply Likes

Omei and I have been discussing wuami charts again. Here is a discussion from yesterday.



Suspect A * B (RI) and A * B (RO) puzzles

eli: I have noticed something. If I have a hard design that is only stable in the first 3 states, and then dump it in the opposite lab, then the 4th state tends to go stable. I first had it happen with a unstable double entangled hairpin puzzle I was working with.


Here is a unstable monster example. 3 first states stable in hard INC, 4 state stable in hard dec:





CUGUUCCUGCACCAUCCAUCACUCAGAACUUAGAGAGAGACAGGUAGGAUGGUGCAGGAACAGCUCUCUAUGUACCUGUCUCUCU

That kind of confirms to me what you have been saying that there is nothing inherent ON and OFF about a puzzle.


I wonder if I can make an A * B (RO) and and A * B (RI) design in engangled hairpin style?


I wonder how important it is to make state 2 and 3 stable in these puzzles. Because I think I can make the puzzle solve state 1 and 4 with the ABR at one end of the puzzle.


Here is an ABRI example:


CAUCCAUCACCAUGUUCCUGCAUCUGUCAGAACUUAGUCAUCCAUCACCAUGUUCCUGACAGAUGCAGGAACAUGGUGAUGGAUG (#BAR--)



It looks entirely different to the past ABRI and ABRO winners, when it comes to the wuami plot. But it starts having a linear relationship for A and B



Now I'm trying to determine if this is a good or bad thing.


It is similar to our hard lab solves in that the A or B condition on their own not necessarily are satisfied.


As far as I recall you protested the more usual A and B (RI) wuami plots, exactly because the linear relationship was missing.


Here is an winner from a past A and B (RI) lab



eli: I made an A * B (RI) design in a double entangled hairpin style and got an even more linear chart:


CAUCCAUCACCAGAACUUAGUCGCAGGAACAGGGACUAAGUGACUAAGUUCUGGUGAUGGAUGAGAACUUAGUCCCUGUUCCUGC






These linear stretches are not turning the same way.


I didn't mentioned, but I was trying with an A and a B at either side of the sequence + each input having a reporter.


And it isn't stable at all in any states


omei: Certainly, not every sequence that satisfies a specific puzzle is going to match my "ideal" wuami chart.

... for multiple reasons.  In the case of the A*B puzzle, the puzzle itself is "suspect".


eli: I am considering if not an double but unentangled solve style would be more appropriate for the A * B (RI) lab as that would allow for a solve to the 4th state where both A and B are present.


I like the solve I got in the single entanglement, as it looks like I would expect from the hard labs plus the AC BC ones.


Why is the A*B puzzle suspect?


We haven't used them to fuse for the hard labs, only the smaller sub labs (ACDED, BCDEC, ACINC, BCINC) seems suited for that


omei: I argued for re-defining the puzzle after the first round, but pretty much everyone else thought that making round-to-round comparisons was more important.


eli: Ok, so my linear single entangled version is likely the one that makes most sense, had the puzzle been set up normal. It fits the chart pattern I have come to expect


omei: So I don't want to discourage you from exploring whatever seems intriguing.  But for A*B, expect a discrepancy between what I think an ideal chart would look like, and how the design scored in the lab.


eli: Yes, so I won't get a good score if I put an entangled solve up. But an entangled version would give the nicest wuami chart


omei: I wouldn't say that a linear wuami chart won't give a good score for A* B; I wouldn't be surprised at all if it did.  It just isn't what the A*B puzzles are really designed to evaluate.



The smallest of sub labs


eli: Here is a crude drawing of the wuami patterns I expect to work for the RIRI/RIRO labs

All my RIRO/RIRI entangled designs looks similar to this.




eli

I have only cut out one window from the wuami chart

for sensor A only the A/C window is relevant

for sensor B only the B/C window is relevant

For sensor C only the A/B window is relevant

It looks like these designs mainly have one readout color


omei: This doesn't seem right.  Do you have actual chart examples we can look at?


eli:  Moment

And yes, there doesn't seem to be a switch. Plot from the best sensor A RIRI.



http://www.eternagame.org/game/solution/7113220/7140078/copyandview/


There is a shift in the A/C window. So I'm probably wrong. So my entangled solutions are perhaps too long stemmed to work in those sub labs. I will leave them in, as I want that confirmed.


omei: OK, so the top two panes here suggests that for the most, this is acting as A sensor.  That is, the output color is (almost) determined by only the A concentration.


eli: But if the whole of the two top windows are blue, isn't it still working as an A sensor?

When there is only an A and an R in the puzzle 


omei: The fact that most of the diagram is blue rather than brown says that this sensor switches at a low concentration of A, i.e. has a low KD value.

If the dividing line moved right, it would indicate switching at high concentrations.


eli: Ok, so what would we want an A sensor RIRI to look like?

Its an ON switch, so it should go on at low concentrations, right?


omei: No, all blue would suggest that the reporter bound even the lowest concentration of A tested in the lab.


eli: Ah, so it is no good then

I didn't realize

And all brown would mean that the input didn’t bind at all


omei: But remember, if Michelle wanted a wuami graph whose primary purpose was to predict one of the simple puzzles, she would have given NUPACK different parameters than she did.  Without looking at lots of examples, we really don't know how closely these other panes reflect how a design will score as a different puzzle. Basically, we are extrapolating way beyond the design range.


eli: True, they are still nice.


omei: (The design range of the chart, that is.)


eli: Yes, the concentrations of the puzzles are not the same as the charts


omei: ... And I think what they "suggest" is probably at least somewhat indicative of what you want them to suggest.


eli: Yes, so they are pointing in the right direction.


omei: Mostly, I don't want you to be disappointed if they turn out to be much less useful than for the full puzzles.


eli: I just didn't read them cautious enough. I appreciate your input.


And basically I'm using them to try out different stuff that I otherwise wouldn't have. There would only have been a single monster without the charts.


Because I dislike real long switching stems and my entangled puzzles in the RIRI and RIRO lab are going to the extreme of that



Perspective on length of input complements


omei: The one you posted, yes.  It also shows that the design is somewhat sensitive to high levels of B and C, which is not really what we want, but that will be very hard to completely get rid of, i.e. to make an oligo complement sequence that has zero affinity for the other oligos.


eli: true


omei: The disadvantage of long switch stems is that the slow down the kinetics -- the time it takes to switch to respond to changed conditions.  Since Johan's experiments, for the most part, haven't measured that independently, designs that switch more slowly than his experiment is design for can be seen as "poor" instead of "slow".


But long compliments (that make long stems) improve specificity, which is also an important goal.  We don't want "irrelevant" mRNAs messing up the reading of the three we care about.


So regardless of how well scoring designs are in the current round, there are undoubtedly going to be numerous experiments in the future that involve more criteria.


I actually have a feeling that in the long run, the best compromise between speed and specificity will involve even longer complement stretches, but with "defects" (mismatches or bugles) along the way.



Simplified entangled hairpin


eli: By the way, I noticed something interesting.


One of my sub lab entangled designs had a double of the one inputs


http://www.eternagame.org/game/browse/8489861/?filter1=Id&filter1_arg2=8740533&filter1_arg1=8740533


I had to make a complement to the input A complement and to turn off complement to the complement I needed an identical copy of the input


omei: You did this on purpose, right?


eli: Yes

but I don't recall seeing it this clearly before. It seems to be specific to the RIRI puzzles.

The reporter and input needs to be together. There really isn't the usual 4 elements to an entangled hairpin - two inputs and two turnoff areas.


Here there are just 3, as the reporter and the input gets played together as just one input.  


So there is (A+R)' first. At the far end of the puzzle is the (A+R)'' and inbetween are the turnoff sequence to the (A+R)'’


(The same pattern happens in my A * B (RI) entangled designs too. It seems to happen whenever one has a bunch of inputs/reporters that needs to be on together at once in one state but not be present in another.)

(Edited)
Photo of Eli Fisker

Eli Fisker

  • 2236 Posts
  • 495 Reply Likes

Just got a rather fine wuami chart for a double entangled hairpin design, that I made by fusing two miniature inc puzzles.




Full sequence

CCUGUCUCUCUUCUGAAGGACAGGCCUGUUCCUUCAGAACUUAGUUCCAUCACCAGAGAGACAAAUGGAACUAAGCUGUCUCUCU


As an hard INC puzzle it is stable in the first 3 states.



As a hard DEC puzzle it is stable in the 4th state.



Here are the smaller partials:


BCINC:

AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAGAACUUAGUUCCAUCACCAGAGAGACAAAUGGAACUAAGCUGUCUCUCU



ACINC:

CCUGUCUCUCUUCUGAAGGACAGGCCUGUUCCUUCAGAACUUAGUCGCUAUUUAUUUAUUAAAAUAAAUAAUAUAAAUAAAUAGC




I fused the partials. I had the reporter positioned at exact same spot in both puzzles, so I could make the puzzles overlap.


I only moved a little around. I moved a lot around for the hard dec version.


Basically I think I may have luck making these double engangled hairpin designs work, if I give up the idea that I want a stable puzzle. As long as I have set of stable states between the two labs, I think they shall be fine.


Also I have come up with another pattern for solving the hard puzzles. I managed to make a puzzle yesterday with a half decent wuami chart. (except for one window). I do not have the sequence length to easily make a solve. I think however it will show promise when we get just a little longer sequence. Basically I can't see why it won't work. It involves two reporters. I think it may be particularly helpful for ABC2INC puzzles.



The above pattern would in theory also allow that one can move the one of the entangled hairpins to the other end of the sequence, since the two sets of entangled hairpins becomes independent.



Which would suit me really well, since I am of the opinion that order matters. That way I could make both the ACINC and the BCINC have the optimal position of their inputs. Similar with ACDEC and BCDEC.


Not sure how they will affect each other.




Omei: Hi Eli! Just sitting down to read.

OK, caught up with the reading, if not complete comprehension.

What seems most interesting to you?


eli: the fusion of the micro puzzles and that I think I can make it work


One thing that I have a hard time grasping is the high and low concentration thing. What is the hard INC lab compared to the hard DEC lab?


I mean I want to understand if I have a puzzle that has potential, by viewing which lab it gets state 1-3 stable in, versus the lab that it gets state 4 stable in. I can see that it happens quite a lot


omei: I'm not sure I would expect any structural pattern.


eli: I'm thinking about the pattern of which state are being stable in relation to the puzzle type

Perhaps I can formulate differently


What characterize a hard inc lab that is stable in state 1-3?

What characterize a hard dec lab that is stable in state 4?


omei: I think about it not in terms of the puzzle states (which are pretty arbitrary) but the wuami charts.


eli: Ok, let me hear your thoughts. I usually end up learning something. :)


omei: Both puzzles should have the linear dividing line in pretty much the same place.


The real difference is which corner the blue is in.


The transition between blue and buff happens over a very small range.


And the difference in states 3 and 4 that were chosen are also very close together.


So it just takes a small shift in the diagonal line to move it to one side or the other of the conditions that define state 3 and state 4.


This small change doesn't have a major effect on the folding, it just flips which side state 4 falls in.


eli: Inc puzzles charts seems to have the blue dominate and dec puzzles seems to have the light brown dominate. I can easier disturb the balance if I allow the puzzle to go unstable.