Help Design the Round 2 Experiments for OpenTB

  • 2
  • Idea
  • Updated 3 years ago
Following the closing of Round 1 of the OpenTB Challenge, there will be a considerable length of time before the experimental process is complete and we know the results.  Rather than waiting for those results before opening a new lab, we'll be starting a new round more or less immediately, but with a new set of puzzles.  Rhiju has decided to open up the design of the new set of puzzles to players, especially those who actively participated in Round 1.

The goal is still creating a switch that calculates A*B/C^2 on the concentration of the three 50-base sequences recently shown to be an excellent predictor for active TB.  But there are a huge number of choices to be made as to how to frame that in a set of Eterna puzzles and lab procedures.

Here are the main opportunities for changes that can be considered:
  1. Choice of oligos: The A, B and C sequences still need to be selected from the from the following 50-base sequences

    [A] GCAGGAACAACAGAUGCAGGAACAGGCUGCACAGCUCAGCACAACAUUCC
    [B] CCAUGGUGAUGGAUGGUUUGGAAAGGGAAUGUUGGUGCCUUUUGUGCCAC 
    [C] AUUACUGUACAUAGAGAGACAGGUGGGCAUUUUUGGGCUACCUGGUUCGU

    but the changing the subsequence, including the length, is feasible.
  2. Choice of reporter:  Changing either the reporter sequence and/or its length is feasible.
  3. Experimental concentrations: Changing the concentrations of the TB oligos and/or the reporter is feasible. 
See https://getsatisfaction.com/eternagame/topics/the-tb-challenge-how-should-we-select-a-b-and-c-segmen... for a discussion that we had before Round 1.  That discussion was held before we had any experience solving the puzzles, and the only thing being considered was the selection of the subsequences.  Now that we have a better sense of how difficult the puzzles are, we can take that into consideration.
Photo of Omei Turnbull

Omei Turnbull, Player Developer

  • 966 Posts
  • 304 Reply Likes

Posted 3 years ago

  • 2
Photo of Brourd

Brourd

  • 437 Posts
  • 79 Reply Likes
A question about number 3: The only constant concentration in the actual experiment is that of the reporter oligonucleotide, correct? The actual experiments will involve concentrations of oligonucleotides TB-A, TB-B, and TB-C of varying levels, in order to test the A*B/C^2 condition, and won't just consist of 4 separate experimental conditions. Although, I didn't closely follow the development of these puzzles, so it's possible I'm confusing some details.

As for the concentration for the reporter oligonucleotide, is that based on the experimentally (or possibly computationally) determined dissociation constant? Increasing or decreasing the concentration for any reporter sequence would require some recalibration to compensate for the change in signal.

Finally, while it may not be feasible, wouldn't it be in the best interest to use the full length of the TB oligonucleotides?
Photo of Omei Turnbull

Omei Turnbull, Player Developer

  • 966 Posts
  • 304 Reply Likes
Although there was discussion of doing the experiments by holding the reporter constant and continuously varying the the inputs, that model was never adopted. The arithmetic puzzles are being tested in basically the same way as all of Johan's labs. That is, for each experimental "run", all the input concentrations are constant, and the concentration of the reporter RNA is varied over many orders of magnitude. 

As an example, for Round 103, there are seven experimental conditions:

MS2
MS2 + A (100 nM)
MS2 + TB_B(new/C) (5 nM)
MS2 + TB_B(new/C) (100 nM)
MS2 + A (5 nM)+ TB_B(new/C) (5 nM)
MS2 + A (5 nM)+TB_B(new/C) (100 nM)
MS2 + A (100 nM)+TB_B(new/C) (100 nM)

(This puzzle used the fluorescently labelled MS2 protein instead of a fluorescently labelled RNA molecule (i.e. the"reporter"), but the experimental process is the same.)

These conditions include the ones that correspond to the puzzle states in the game, but add some additional ones to get a better sample of the possible input ranges.

Changing the reporter sequence between rounds isn't a problem because the most important experimental result is the ratio of KD values between the various states of a puzzle. The absolute KD values are affected by all the oligos present, inputs as well as reporter, so there is no meaningful way to compare them across experimental rounds.

For now at least, I'm going to take a pass on your last question.
Photo of worseize

worseize

  • 29 Posts
  • 12 Reply Likes
What is {KD} ? 
Could you give a picture ?
I would like to make designs in many ways but still have no idea what we need most . I try to fold as many nucleobases as possible .
I hear that good design have a plateau at the beggining in melting plot .

1) In my opinion {in "melting plot" graph} if second state have less % of unpairing bases at particular degree  than first state  then it folds as we suspect  - is it true ?

2) What is better fold reporter or oligo fully or make +2 pairs but -1 oligo/reporter pairng base ?
Photo of Omei Turnbull

Omei Turnbull, Player Developer

  • 966 Posts
  • 304 Reply Likes
Hi, worseize!

Your first question, what is KD, has a well defined answer.  It is determined as the concentration of the reporter RNA that is needed to make the fluorescence of the design be half of its maximum value.

Here's an illustrative graph.

(In this case, the MS2 protein played the same role as the reporter RNA does in the current lab.)

Your other questions don't have clear-cut answers.  The melting plot is something of a legacy from when we were doing different experiments.  It's not clear it really tells us anything relevant to our current switches.  But then again, it may and its significance has just been overlooked.

And as what design characteristics give the best results, that's what we are all trying to figure out.  It's a moving target, with each lab (potentially) posing a new problem, or a new set of conditions.  So if you want to get seriously involved, you need to study past results and try to figure out what worked best there, and why, and how that should be generalized to apply to the current lab.  I think johana will be releasing the latest results, Round 103, this weekend at Eternacon.  Study those results, follow the discussion of them here on the forum, and see what players who have done well in recent rounds are submitting for this round.  And then post your own thoughts, as well as submit new designs.

Good luck, and have fun!
Photo of whbob

whbob

  • 190 Posts
  • 57 Reply Likes
Photo of worseize

worseize

  • 29 Posts
  • 12 Reply Likes
Where could I get { Fmax & Fmin } in eterna game graphs ?
Photo of whbob

whbob

  • 190 Posts
  • 57 Reply Likes
There is usually a spreadsheet posted after a lab is completed and analyzed. See the Lab Archive list in the resources tab on the home page.  
I have not seen Fmax and Fmin, just Fmax posted.  
As explained in the above link that I posted, Fold score= 0 for an Fmax= 0.3, Fold score = 30 for a Fmax at or above 0.7. so you could interpret the Fmax if the Fold score is between 0 and 30.  
 
Photo of JR

JR

  • 240 Posts
  • 20 Reply Likes
Question about #2: Any guidelines as to what makes a good reporter? Was the current one chosen due to it being a "better" one?
Photo of Omei Turnbull

Omei Turnbull, Player Developer

  • 966 Posts
  • 304 Reply Likes
As I understand it, the choice of a short reporter at weak concentration in this round was motivated by the desire to make the reaction as quickly reversible as possible. One advantage of this is that it opens up the possibility of doing more measurements within whatever time slice Johan gets allocated on expensive machines that are in high demand.  The disadvantage is that because the reporter can only form a fairly weak bond to the switch, the switch is harder to design.  So it's a trade-off.

I certainly hope that Rhiju and/or Johan join into this discussion.  They'll be able to be more specific, as well as correct anything I say that is outright wrong.
Photo of whbob

whbob

  • 190 Posts
  • 57 Reply Likes
If the reporter can be modified, I would suggest using UCCUUAACGG.  This would allow UCC to compete with the first part of the C oligo and CGG to compete with the last part of the C oligo.  Lower concentrations of C should allow the reporter to bind instead of C.
Photo of jandersonlee

jandersonlee

  • 549 Posts
  • 122 Reply Likes
Regarding 1: So it seems that the targets are actually 50 bases rather than 20. Great news! We can effectively shorten the working ogliotides by binding to part of them rather than the whole sequence and some designs already do this. Is there a particular target region we should try to include to avoid false positives? Could we have longer target samples next time - and if so, how long might be feasible? 20 bases? 30? 40? all 50?

Regarding 2: The current reporter is short enough and "weak" enough in binding that we effectively need the whole binding site free in order for it to bind. These properties seemed troubling at first, but actually may be keys to success in the long run. Are there other reporter molecules that are "lab ready" or "lab tested"? If so, is there a list we could look at? If not, are there at least known constraints on what the reporter must look like to be able to function and/or to bind to the fluorescent portion in order to work?

Regarding 3: You mentioned once before about the "fragility" of the samples and how there is a limited time that they can function. Are there some constraints on how many concentrations we can use, either per sublab or per overall lab? Is there one sample/slide used for all sublabs, or are there more than one sample/slide used with different concentrations for different sets of sublabs? (We skipped a lab round - could that give us time/funds to run two different sets of sublabs next time or is the constraint the analysis time/process/equipment?)

If all of the sublabs were measure with the same concentrations, or subsets of the same concentrations, would that help? Right now we have some at 0nM, 5nM, and 100nM of some reactants/oglios for example. What if instead we targeted 5nM, 25nM and 100nM or 1nM, 10nM and 100nM to get three points on a log scale for some sublabs instead of a binary choice. We may not know up front what shape we want the mid structure to be, but its measured response may help us to tweak the designs one way or another.

When designing, I typically look at 64 relative concentrations in-silico to help analyze the designs and get some sense of which way to tweak them. While 64 in-vitro concentrations is likely infeasible, could we possibly get away with more measurements per sublab (e.g. 6? 9? 16? per sublab) assuming the same target set for all sublabs? Granted, points for a given sublab might still only be based on a few (e.g. 2 for a "binary" sublab and 4 for an AND/OR logic-style lab) but the extra data might help us to better understand the way the designs are switching, just as the many titrations of MS2 in earlier labs gave us curves to help better understand the performance of those switches.
Photo of Omei Turnbull

Omei Turnbull, Player Developer

  • 963 Posts
  • 303 Reply Likes
These are all good questions; I have fewer answers, let alone good ones.

Re 3), I think you are referring to a comment that the MS2 protein quickly degraded the chips Johan uses, and so we probably wouldn't be using it much in future labs.

Right now, the experimental bottleneck seems to be access to the equipment where the final experiment is done.  I know that is what has held up results for Round 103.  I think that all the experimental prep work was completed long ago, but the image workstation, where the experiment is performed and the raw data is collected, was available only two days ago.  When you said we skipped a lab round, had you just given up on getting data for that round, or did I miss something else?

All the designs for a round are present on the chip (slide), and Johan reports in his spreadsheet the results of all experimental conditions for all designs. At least one past synthesis round has included both RNA and MS2 output puzzles, and Johan reported both sets of measurements for all the designs.  But for scoring purposes, he only used the measurements that were appropriate to each puzzle.

There have been multiple chips in recent rounds; I think that was primarily because of the problem with the MS2 protein degradation of the chip.  But there is undoubtedly RNA degradation as well.  The chips are a significant expense, and If all the experiments can be completed using just one before it "wears out", that would be the preference.
Photo of jandersonlee

jandersonlee

  • 549 Posts
  • 122 Reply Likes
By skipped a round I mean we delayed the end. If that was due to lack of access to the equipment it doesn't leave any slack in the schedule. But as you say the chips are somewhat expensive -- if we skipped a month does that mean there might be funds to do a two-chip month if we can speed up the processing?
Photo of Omei Turnbull

Omei Turnbull, Player Developer

  • 963 Posts
  • 303 Reply Likes
Ah.  There may have been some scheduling considerations in that decision, but I think it was mostly to get more submissions.  That's why the individual player quotas were bumped up so much.

Lack of submissions is also a strong driver for asking players for input on the puzzle design.  That is, what changes would make the puzzles easier to solve?
(Edited)
Photo of Omei Turnbull

Omei Turnbull, Player Developer

  • 963 Posts
  • 303 Reply Likes
Re 2), I strongly agree that the short reporter, in combination with the low concentration specified in the puzzle definition, as limited our flexibility in binding to it in this round. 

The reporter sequence used in this round was the same as that used in Round 98 -- the first of the RNA In/out puzzles.  After working on that round, I (and probably other players) asked whether we could use a longer reporter to give us more options.  So for round 99, Johan used a 20-base reporter, and in fact that did lead to higher fold changes (i.e. better switches.)  So one factor in deciding on the length of the reporter is the tradeoff between better switches and potentially longer experiments.  In the current round, the choice was to see how weak a binding we could work with.  But given that many players have found the resulting puzzles too difficult to solve, we want to try for a different balance this time around.

Something else just occurred to me.  Under the simplifying assumption that the designs are reaching thermal equilibrium in the experiments (and in the eventual diagnostic), there is really no chemical distinction between inputs and outputs other than which one fluoresces. A reporter RNA that binds to some other RNA in the blood sample anywhere as strongly as it binds to our switch RNA would be a real problem. That seems like it would be another factor in favor of a reporter that was about the same length as the input oligos.

(Having said that, it is my personal feeling that we don't really know what it will take to convert a switch design we come up with in the lab into part of a functioning diagnostic device.  There may well be a whole new set of constraints that have to be taken into consideration.  We certainly don't want to shoot ourselves in the foot by designing switches that we know, or should have known, won't work in a final diagnostic.  But our first emphasis should be on choosing experiments that teach us about the principles of designing switches, and not so much on trying to optimise for the conditions of an eventual diagnostic when we don't know what those conditions will be.)  
Photo of Omei Turnbull

Omei Turnbull, Player Developer

  • 963 Posts
  • 303 Reply Likes
BTW, I have passed on to Rhiju and Johan the specific question of whether there are any constraints on the reporter RNA sequence due to the fact that it needs to be bound to the fluorescent protein.
Photo of Brourd

Brourd

  • 435 Posts
  • 79 Reply Likes
" In the current round, the choice was to see how weak a binding we could work with. But given that many players have found the resulting puzzles too difficult to solve, we want to try for a different balance this time around."

Why does a weak binding Reporter oligo cause difficulty in the puzzle?
Photo of whbob

whbob

  • 187 Posts
  • 57 Reply Likes
High scoring miRNA sublabs had from 15 to 20 oligo bases attached to the main sequence as I recall.  When one oligo replaced another oligo, there had to be at least 6 overlapping bases.  There seem to be very few opportunities in the current sublabs to meet those objectives.  I don't know if extending the length of the RNA sequence past 85 bases would help, but I have a feeling that it might.  
Photo of JR

JR

  • 238 Posts
  • 19 Reply Likes
I don't think a weak Reporter is the problem, I can push the Reporter around just fine. Where I have the problem, I think I need more overlap between oligos so I can
get them to push each other around. I can get one, but not both at the same time.
Photo of Omei Turnbull

Omei Turnbull, Player Developer

  • 963 Posts
  • 303 Reply Likes
Perhaps the easiest way to think about why reporter binding energy is significant is to consider the limit -- that there is no energetic advantage in the reporter binding to the design.  In that case, we're clearly not going to satisfy the puzzle constraints, nor measure any fold change in the lab.
Photo of Brourd

Brourd

  • 435 Posts
  • 79 Reply Likes
That doesn't determine the difficulty of the actual "puzzle." Rather, that's an issue with the thermodynamics of the experiment, and the necessity for a reaction that is reversible on a relatively short timescale. Granted, the stability of the dimer complex between the reporter and the RNA sequence can be increased with a change in the base pairs that constitute the dimer.

However, there is a question as to whether or not you want the reporter to have an energetic "advantage" to binding to the design, given the reporter should associate and dissociate with the target sequence with ease in order to act as the signal for the experiment.
Photo of Omei Turnbull

Omei Turnbull, Player Developer

  • 963 Posts
  • 303 Reply Likes
I'm not sure what you mean by "That doesn't determine the difficulty of the actual puzzle".   Lowering the concentration of the reporter in the game reduces the number of sequences that satisfy the puzzle constraints.  As an example, here is a sequence that would bind to the same 10-base reporter in the Round 98 RNA In/Out puzzles, but won't bind in the current round because of the lowered reporter concentration. 

Photo of Omei Turnbull

Omei Turnbull, Player Developer

  • 963 Posts
  • 303 Reply Likes
@JR: re " I think I need more overlap between oligos ...", this was first impression when I looked at the set of oligos by eye.  But when I did a more systematic search, they seemed quite adequate to me.  Perhaps your criteria for "overlap" is different than mine, but take a look at Kernel Attractor Calculation for the TB Challenge and see whether that helps any.
Photo of Omei Turnbull

Omei Turnbull, Player Developer

  • 963 Posts
  • 303 Reply Likes
@whbob: My rule of thumb for how many design bases should pair with the oligo to get the best results is "as many as possible, while achieving the required energy balance between states."  But that number is going to depend entirely on the details of each puzzle definition, and won't even be the same for all the oligos in the same puzzle. If you are starting with a particular number in mind as a goal, it might actually be counterproductive.
Photo of Brourd

Brourd

  • 435 Posts
  • 79 Reply Likes
From the perspective that being limited in the total number of sequences affects the difficulty of the puzzle, sure, it sounds like a rather tough pickle that you're now limited to a total of 76^4 number of possible sequences to design, rather than 80^4 or 82^4.

Granted, in my opinion, the difficulty of the puzzles stems from the same reason the XOR puzzles were difficult, in that players are given conditions that appear to be contradictory in nature, but really just require some manipulation of the sequence to be solvable.
Photo of Brourd

Brourd

  • 435 Posts
  • 79 Reply Likes
To better define what I mean by contradictory conditions, we'll use the XOR gate. In that puzzle, the player is given a condition that with either input A or input B on their own, there should be a signal, but when both are present, there should be no signal. In the context of Eterna sequence design, this seems backwards, but there is an easy algorithm for solution design that gives computationally valid solutions.

Likewise, the contradictory condition for the A*B/C^2 designs is that there are two constraints that require the presence of all five oligonucleotide sequences, but in different concentrations. Again, there is a fairly simple algorithm for making computationally valid designs, but the addition of a medium based time constraint causes the most difficulty, as well as a change in the agency a player has over the final step of the designing process.
Photo of jandersonlee

jandersonlee

  • 549 Posts
  • 122 Reply Likes
@Omei: "...there is no energetic advantage in the reporter binding to the design.  In that case, we're clearly not going to satisfy the puzzle constraints, nor measure any fold change in the lab."

I believe that the reporter should be relatively energetically neutral in order to be an effective reporter. Opportunistic. If it's binding site is available, it will bind, but it shouldn't "force" it open. That's so that what we measure is the relative effect of A, B, and C due to their presence/absence/concentrations.

The argument that making it one base longer adds a 10x factor for disassociation time as the concentrations of other oligos change is also good reason to keep it short.
Photo of jandersonlee

jandersonlee

  • 549 Posts
  • 122 Reply Likes
@Omei: "...there is no energetic advantage in the reporter binding to the design.  In that case, we're clearly not going to satisfy the puzzle constraints, nor measure any fold change in the lab."

I believe that the reporter should be relatively energetically neutral in order to be an effective reporter. Opportunistic. If it's binding site is available, it will bind, but it shouldn't "force" it open. That's so that what we measure is the relative effect of A, B, and C due to their presence/absence/concentrations.

The argument that making it one base longer adds a 10x factor for disassociation time as the concentrations of other oligos change is also good reason to keep it short.
Photo of jandersonlee

jandersonlee

  • 549 Posts
  • 122 Reply Likes
Ah, I went back and looked at the other thread. It seems that at least 20 NTs seem to be required to get reasonably unique BLAST results and more than 20 gives us slower in-silico designing. So the designs that currently target subsets of the 20 NTs might give us more false-positives in the final results. At the same time, does the eventual TB tester need to work in conjunction with the full O(4000) RNA or is there some intermediate step that amplifies/filters the sample down to shorter segments?

Would it be helpful if we run the in-vitro labs against the full 50NT oglios even if the in-silico designing only scores the performance against a subset?

Could the player possibly select which K-NT portions their design gets scored against (where K is in a range of 20-30) rather than always using the same segments?
Photo of Omei Turnbull

Omei Turnbull, Player Developer

  • 966 Posts
  • 304 Reply Likes
Re letting the individual players select which bases are used for scoring: that doesn't sound feasible.  The measurement process doesn't produce any information about what specific bases are doing.  But maybe I didn't understand the question.
Photo of jandersonlee

jandersonlee

  • 549 Posts
  • 122 Reply Likes
Not which bases are used for scoring - nature takes care of that. Rather which bases are used in the in-silico simulation as the "target" for binding. I'm presuming that this would allow the full 50-base sequences to be used in the labs, but that players could design their sequences to bond to different regions of those sequences. The "scoring" that matters isn't in the in-silico simulation, it's what the reporter does in the wet lab.
(Edited)
Photo of Omei Turnbull

Omei Turnbull, Player Developer

  • 966 Posts
  • 304 Reply Likes
Now that I understand it, I think it's a great idea if Nando and Johan can handle it on their ends.
Photo of Jennifer Pearl

Jennifer Pearl

  • 197 Posts
  • 31 Reply Likes
The number of complexes that may form is a consideration with Sara and NUPACK. I have NUPACK working with DPAT for getting dot plots and one 2 state design with 3 complexes takes about 20-30min to run on my laptop (not sure how long it will take on the Amazon server). If there are more complexes the time will increase significantly. I'm not sure how that affects length of oligos but I would think the longer they are the more they might bind to each other and do weird stuff. 
(Edited)
Photo of jandersonlee

jandersonlee

  • 549 Posts
  • 122 Reply Likes
I have found that I can reuse some of the NuPACK calculations for different concentrations. What takes the long time to compute is the original sequence pairings based on lengths and numbers of complexes; after that I can compute 64-different concentrations reusing that setup in under a second. The 3-element RIRI/RIRO sub-labs (e.g Seq+{A,B,C}+Reporter) are the fastest to process. The 4-element sub-labs (e.g. S+A+B+R, S+A+C+R, and S+B+C+R) are a bit slower and take about a minute. I have not yet got code to evaluate the 6-element sub-lab designs (S+A+B+C+C+R).

Making the complexes longer would probably slow things down.
Photo of Omei Turnbull

Omei Turnbull, Player Developer

  • 966 Posts
  • 304 Reply Likes
I have a proposal for a change that is outside the scope of the the three items I listed in the opening post: Change the two "A AND B" puzzles to be defined and scored as "A * B" instead, since multiplication is the operation that is needed to decompose the overall A*B/C^2 into sub-expressions.

Solutions for A * B would not be all that different from A AND B.  But they would be scored differently, and I think we would want to change the concentrations of A and B in states 2 and 3 from the binary absent/present to low/high.
Photo of whbob

whbob

  • 190 Posts
  • 57 Reply Likes
I'm confused.  I have been reading the logic as : A*B = A AND B; A + B = A OR B.  I have expected that A*B/C^2 would mean both A and B oligos, are required to be present in state 4 and that they would displace the two C oligo's.    
Photo of Omei Turnbull

Omei Turnbull, Player Developer

  • 966 Posts
  • 304 Reply Likes
For the logic gate puzzles, we're treating the inputs and outputs as binary values, so inputs are either present or not present.   We're not too interested in what happens at intermediate concentrations.

It's the reverse for the arithmetic puzzles.  We know that no riboswitch can calculate these R values over the entire range of non-negative values.  What we're interested is how well we can calculate it for the limited range of concentrations that we expect to detect in human blood.  So those low and high concentrations for A and B should be a better choice of test conditions that 0 and very high.
Photo of jandersonlee

jandersonlee

  • 549 Posts
  • 122 Reply Likes
I like this idea. And measuring more than 4 states would help us see it as a multiplication table. E.g. 5nM, 25nM, and 100nM of each (9 states). But I think it's even more critical in the A/C and B/C sub-labs where 5nM/25nM should in theory give a similar result to 25nM/100nM.
Photo of johana

johana, Researcher

  • 96 Posts
  • 45 Reply Likes
Hi all,


REPORTER:

I will try to fill you in on the current choice of reporter.

The 10 nt RNA reporter was computationally predicted to have a low-nM dissociation constant, similar to the MS2 protein, and it is expected to be reversible on the time scale of our experiments.

According to my estimates, extending the reporter increases the affinity by nearly an order of magnitude for each extra base. Most of this should be driven by a lower off-rate. In Rounds 98 and 99, we actually measured this off-rate, which largely confirmed that the short oligo in most cases comes off in minutes, or at least less than an hour. The longer reporter, however, was in many cases at least partially irreversible (there could be multiple binding states). Of course, the binding of either reporter is determined by the design sequences and can be reduced significantly by the players.

The problem with having reporters that bind with a dissociation constant less than 1/h is that we have to wait several hours for each condition to reach equilibrium. Since each experiment is typically on the order of 12-20 h, that sets a hard limit on the number of conditions that we can test in any given round. Our chips can only be reused a couple of times at present. If we don't measure equilibrium, it is very unclear what we are actually measuring, i.e., a combination of affinity and kinetics. Now, if players really desire longer reporters we could perhaps explicitly measure the unbinding at the end of the experiment so give us a sense for how meaningful the results from that design is.

CONCENTRATION:

For the OpenTB puzzles, the concentration of the reporter was set to 25 nM. The thinking is that a 20X change in input oligo concentrations (the conditions chosen for two-oligo puzzles) can lead to a 20X change in the reporter Kd. An experimental concentration of 25 nM is therefore roughly halfway (in log-space) between 5 nM and 100 nM (5 x 20). It is chosen to maximize the signal difference for a good design that is ON at 5 nM and OFF at 100 nM. If we change this concentration, we may find that design are always binding the reporter (or never) even if they technically switch states between the two experimental input oligo concentrations. It's by no means set in stone, however, so we are curious to hear about your other options and suggestions.

OLIGOS:

The length of the oligos was chosen to make them long enough to be unique sequences, even in the context of the human genome, and provide enough specificity. We are also limited in the total length of the design (~84 nt) since that's the upper limit for oligo pool synthesis. Three oligos (or more), each of length 50 nt, could not all bind fully to the same design.

There are constraints in terms of affinities also for these oligos. In the absence of competing structures or binders, oligos of length ~20 nt are practically irreversible if they are fully complementary to the target. We consequently found that many of the logic gates do not switch fully back and forth between the four states even if they correctly report the first state they encounter. If we are interested in the initial state (i.e., TB present or not) this may still be fine. but are curious to hear your thoughts.

Since the oligos do not have to match the design sequence perfectly we have plenty of freedom to tweak the design sequences to give us reversible binding if necessary. For example, we can use longer oligos but alert players that a full match may not be ideal and with the design length limiting the total binding to the puzzle solution.
(Edited)
Photo of Omei Turnbull

Omei Turnbull, Player Developer

  • 966 Posts
  • 304 Reply Likes
Can you say more about why a good design should be ON at 5 nM and OFF at 100 nM?  That is, how does that translate into the R = 1/4?  As part of that question, are we aiming for an R that is less than 1/4?  More than 1/4?  Or within some epsilon of 1/4?
Photo of johana

johana, Researcher

  • 96 Posts
  • 45 Reply Likes
The [A]/[C] puzzle, for example, has [A]=100 nM in the ON state and [A]=5 nM in the OFF state. [C]=100 nM.

I think that R = 1/4 refers to the cutoff ratio, which means that if you're above the signal should be ON, and below it should be OFF. 1/4 =25/100 ~20/100. In other words, if [A] is intermediate between 5 and 100 nM you land on roughly on the cutoff (1/4).

You are, in other words, aiming for the largest signal possible from concentration ratios around the 1/4.
(Edited)
Photo of jandersonlee

jandersonlee

  • 549 Posts
  • 122 Reply Likes
Another reason to target more measurement states for the X/C puzzles.
Photo of jandersonlee

jandersonlee

  • 549 Posts
  • 122 Reply Likes
@johana: "Three oligos (or more), each of length 50 nt, could not all bind fully to the same design."

True enough, but many designs don't target the full complementary sequence in any case. Having a longer oglio (in-vitro and in-silico) would allow designers to have flexibility in what portion of the oglio they choose to target. The main downside is that it makes the in-silico modeling slower, which is why I asked if players could *choose* what sub-range to target in-silico on a per-design or per simulation run basis. Instead of a full-freeze mode, could we dial-up or dial-down the computation by choosing shorter or longer regions of the oglios to model against?

Is it not the case that in final assay testing we will be presented with long RNA strands, or is there some process that will detect and multiply only particular sub-segments?
Photo of Omei Turnbull

Omei Turnbull, Player Developer

  • 966 Posts
  • 304 Reply Likes
@johana: From the various comments posted, it seems like lengthening the reporter (to give more flexibility for what subsequence the design grabs hold of) while adding additional criteria for what makes a desirable switch (switching range, and dissociation constant), is a good win-win combination.   (But not a total win all-round, because of the probably additional computational time.)

In round 99, you used a 20 base reporter, and it was nice in that it had both purine-rich and pyrimidine-rich stretches.  Would you consider it a candidate for the next round?
Photo of Omei Turnbull

Omei Turnbull, Player Developer

  • 966 Posts
  • 304 Reply Likes
Thank you, Johan, for the explanation. It clears up a number of things I have been wondering about.

Is it worth considering tweaking the experimental process so that each reporter concentration is measured twice -- once as you increase the reporter concentration and a second time as you decrease it?

The thought is that a significant part of the score for the design would be the extent to which the two measurements at the same concentration are the same.  That would allow us, as players, to learn to design for both good fold change and reversibility.  Presumably, this shouldn't need to double the total experimental time because you could shorten the resting time at each concentration to be in the range that you really want, instead of waiting for slow switches to come to equilibrium.
Photo of Atanas Atanasov

Atanas Atanasov

  • 42 Posts
  • 15 Reply Likes
Hi all,

There is a lot more that I need to read to understand most of the discussion so excuse me if I am asking dumb questions.
From my observations of the game and reading through the scoring description for the MS2 signal, there is this window in Free energy that is supposed to score the maximum in the switch and baseline scores. This windows currently seems to be 1.36kcal/mol wide for the AB/CC puzzles. This is kind of narrow I guess which makes the AB/CC puzzles hard. Can we make that windows wider?
My understanding is that this window for the switches that differ in concentration (and not present oligos like in the MS2 puzzles) is determined by the difference in the concentration between the two states. In our case this is 2x100nM vs 2x300nM. As far as I understand we are constrained in the proportion of the concentration but not the actual concentration values. 

What if we keep the proportion but increase/decrease the difference in the concentration - would that increase the window of free energy?
So does anyone know how the "red" energy value in the game is calculated? My observation is that it is a function of the concentration but haven't tried to reverse-engineer it yet.
(Edited)
Photo of Atanas Atanasov

Atanas Atanasov

  • 42 Posts
  • 15 Reply Likes
I was looking at that function and it seems that to expand the window, the concentrations have to go down, but that would make the designs more unstable :(
Photo of Omei Turnbull

Omei Turnbull, Player Developer

  • 963 Posts
  • 303 Reply Likes
Your last observation seems counterintuitive to me.  Can you show how you got to it?
Photo of Atanas Atanasov

Atanas Atanasov

  • 42 Posts
  • 15 Reply Likes
I might be wrong, my math is a bit rusty when it comes to logarithms. Here is my logic:

I am not sure on what constraints we have for the concentrations.
For example, for a switch 5nM vs 100nM do we have to keep the proportion 1:20? Or do we have to keep the 95nM difference? Or are we free to change those as we see fit?

I think the function for the energy "penalty" is similar to -ln(x) where x is proportional to the concentration.
If x and y are the current switch concentrations, the window is -ln(x/y).

If we have to keep the difference, lets call it "a" - then the windows is ln(1+const / (a+const) ) which I think is a decreasing function (hence the above), but I might be mistaken as those constants might be negative.


If we have to keep the proportion, that is, change the concentration to ax and ay, then the windows is the same: -ln(ax/ay) = -ln(x/y)  If the function is not exactly -ln(x) then this might not be true, lets hope.

If we have more freedom in changing the proportions then we might be able to increase the window.

Having the exact function for the energy "penalty" for a bound oligo at a given concentration would help to clarify this.
(Edited)
Photo of Atanas Atanasov

Atanas Atanasov

  • 42 Posts
  • 15 Reply Likes
I think I found the function for the window width:
W(x, y) = k * T * ln(y / x),
where the switch concentrations are x and y (y > x), k is the Boltzmann constant and T is the temperature in Kelvin.

For two oligos the function is:
W(x1, x2, y1, y2) = k * T * ln(  y1 / x1 * y2 / x2 )

k = 0.0019872041 kcal/(mol * K)
37 C = 310.2 K

For T = 37 C the function looks like:
W(x, y) = 0.6164 * ln(y / x)

For a switch 5nM vs 100nM, the window is:
W(5, 100) = 1.8466 kcal/mol  (which matches my earlier calculation based on the game)

For a switch 2x100nM vs 2x300nM the window is:
W(100, 100, 300, 300) = 1.3544 kcal/mol  (which almost matches my earlier calculation based on the game)

Some examples to get an idea how this thing behaves:

2x100 vs 2x400: 1.71 kcal/mol
2x100 vs 2x500: 1.92 kcal/mol
2x100 vs 2x900: 2.71 kcal/mol
At T = 47 C, 2x100 vs 2x300: 1.40 kcal/mol
Photo of Astromon

Astromon

  • 182 Posts
  • 23 Reply Likes
Do you mean for players to go to puzzle maker and make T/B lab puzzles
(Edited)
Photo of Omei Turnbull

Omei Turnbull, Player Developer

  • 966 Posts
  • 304 Reply Likes
No, the puzzle maker isn't capable of creating Eterna puzzles with RNA inputs and outputs; Nando has to do that for us.  The intent here is to help decide the parameters of the puzzle, such as what specific RNA sequences to use and what their concentrations should be.
Photo of Astromon

Astromon

  • 182 Posts
  • 23 Reply Likes
Okay, I really have no idea about that, so I'll let the more experienced players do this part and I will watch and hopefully learn!  Thanks!  
(ps,  Any way we can keep the last three current open TB labs up maybe a week or two longer? I am close to designs.   you could close the other ones then close the last three in a week er two while your getting the others ready for testing  :D
Photo of Omei Turnbull

Omei Turnbull, Player Developer

  • 966 Posts
  • 304 Reply Likes
 It seems quite reasonable to keep the old ones open until the new ones are ready.  I'll pass the request on, along with any other seconds that come in.

But don't put off submitting any designs you have, there are no guarantees.
Photo of Astromon

Astromon

  • 182 Posts
  • 23 Reply Likes
I wont thanks!!   I was talking with jieux last night and he wanted to do more labs at 15 each sub-lab but pressed for time.  I hope they do get extended! Thanks!
Photo of jandersonlee

jandersonlee

  • 549 Posts
  • 122 Reply Likes
The A|B|C-RIRI|RIRO sub-labs use a 0nm concentration for one of the states, which is an "infinite difference" in concentrations:

(1) A     0nM, B     0nM, C     0nM, R 25nM (0:0:0)
(2) A 100nM, B     0nM, C     0nM, R 25nM (INF:0:0)
(3) A     0nM, B 100nM, C     0nM, R 25nM (0:INF:0)
(4) A     0nM, B     0nM, C 100nM, R 25nM (0:0:INF)

The A|B/C-INC|DEC sub-labs switch from a 5nM to 100nM of the "active ingredient", a 20:1 swing:

(5) A     5nM, B     0nM, C 100nM, R 25nM (1:0:20)
(6) A 100nM, B     0nM, C 100nM, R 25nM (1:0:1)
(7) A     0nM, B     5nM, C 100nM, R 25nM (0:1:20)
(8) A     0nM, B 100nM, C 100nM, R 25nM (0:1:1)

The "hard sub-labs" as I sometimes call them (or AB2CINC and AB2CDEC) are currently the "odd ducks" when it comes to concentrations, but really only introduce two new experimental setups.

(6)   A 100nM, B     0nM, C 100nM, R 25nM (State 1, repeat of 6 above, 1:0:1)
(8)   A     0nM, B 100nM, C 100nM, R 25nM (State 2, repeat of 8 above, 0:1:1)
(9)   A   50nM, B   50nM, C 300nM, R 25nM (State3, new 1:1:6)
(10) A   50nM, B   50nM, C 100nM, R 25nM (State4, new 1:1:2)

But some there must be some transitional states as well when flushing concentrations as they are not all additive-only changes. How many are there? Could some of them be usefully measured?

To help test the "divisor" nature of A/C and B/C it might be useful to have a few more "transitional" data points such as:

(11) A   50nM, B   50nM, C   50nM, R 25nM (State4, new 1:1:1)
(12) A   50nM, B   50nM, C 200nM, R 25nM (State4, new 1:1:4)

or possibly:

(13) A 100nM, B 100nM, C 100nM, R 25nM (1:1:1)
(14) A 100nM, B 100nM, C 300nM, R 25nM (1:1:3)
Photo of jandersonlee

jandersonlee

  • 549 Posts
  • 122 Reply Likes
Ah. I forgot the A(AND)B - RI|RO sub-labs which add:

A 100nM, B 100nM, C     0nM, R 25nM (1:1:0)

I did like the 3-state sub-labs in the trial run which had a stair-step shape on either side of the switch ON/OFF concentration ratio. Could something like that be brought back for an upcoming round? 
Photo of jandersonlee

jandersonlee

  • 549 Posts
  • 122 Reply Likes
I keep trying to figure what might be good "stair-step" concentrations for A/C. Assuming that you want it to switch at around 1:4, then perhaps 1:2 (or 1:1) and 1:10 could be brackets on either side of the switch condition. Assuming the upper C concentration was 100nM, that would make the A concentrations 10nM and 50nM (or 100nM). To get the stair-step you would also check C at 20nM and A at 10nM (or C at 10nM and A at 10nM).

However, this assumes that the "switch" states are reversible, and in highly bonded cases, that may not be true -- which would be fine for a one-shot disposable TB-test stick, but not so great for an expensive experimental laboratory slide with 10000 designs under test.

If we could afford to use multiple slides rather than reuse one, then we could do tests that only changed the ratios in one direction so for example.

A@0nM,C@100nM
A@5nM,C@100nM
A@10nM,C@100nM
>switch at around A@25nM>
A@50nM,C@100nM
A@100nM,C@100nM

plus a second run with C@200nM:

A@0nm,C@200nM
A@10nm,C@200nm
A@20nM,C@200nM
>switch at around A@50nM>
A@100nM,C@200nM
A@200nM,C@200nM

The extra data-points at 10nM and 50nM in the first run might help us to better design switches with the sensitivity threshold bracketed near 1:4.

The second run with 2X the concentration of C would help to ensure that we are getting a relatively consistent divisor, since in the target end-use application, the concentrations of both A *and* C would vary.

Likewise, the A(AND)B design tests seemingly require reducing the concentration of one of A or B at some point, or else using two slides.
(Edited)
Photo of Atanas Atanasov

Atanas Atanasov

  • 42 Posts
  • 15 Reply Likes
Do I correctly understand the goal of the puzzles - we need to measure if a - e < x < a + e and the INC puzzle measures if a - e < x and the DEC puzzle measures if x < a + e.
In the case of AB/CC, x = [A]*[B]/[C]*[C] and a = 1/4?
If that is so, then why do you need a stair-step? Or you'd like the stair-step to be used only for A/C and B/C puzzles?
Are the A/C, B/C and AB puzzles a backup option in case we can't find a good solution to AB/CC?
(Edited)
Photo of Omei Turnbull

Omei Turnbull, Player Developer

  • 966 Posts
  • 304 Reply Likes
@jandersonlee: The various combination of concentrations are relevant in two different contexts: 1) the states defining the in silico puzzles that players design to, and 2) the concentrations at which Johan does the measurements.  It seems that mostly you are making suggestions for context 2.  Could you highlight those cases where you're suggesting a change in the puzzle states?  Thanks.
Photo of rhiju

rhiju, Researcher

  • 403 Posts
  • 122 Reply Likes
@atanas, yes A/C, B/C, etc. are backup -- if they are easy to achieve, we might feed their outputs into a second multiplier (that you all would design in later rounds) Ideal of course would be AB/CC.. 
Photo of jandersonlee

jandersonlee

  • 549 Posts
  • 122 Reply Likes
@Omei: My understanding is that we want a "divisor" response curve, not a binary conc(A)/conc(C) > 0.25. At least from the in-silico modelling that I've done, that seems to be more practical or realistic in the given concentration ranges. While players might design in the game to a "binary" state condition, having the in-vitro lab data for varying concentration ratios can likely help us to better tune the designs to the target ranges as time goes on. I've been doing most of my designing looking at 64 varying in-silico concentrations for most designs and it gives more insight than just looking at 2 or 4 states.

So for the A/C sub-labs for example, players might continue to design for:

State1: A@5nM,C@100nM
State2: A@100nM,C@100nM

perhaps with optional 

State3: A@10nM,C@100nM
State4: A@50nM,C@100nM

But if we measure the in-vitro data at *all* of those states we will get better response curves for those lab-rats who delve into such stuff in tweaking designs for subsequent rounds.

Ideally I'd love to see the same thing done with the AB2C labs, with 16 states measured (in-vitro) to get a better response curve. E.g. pythonically, something like:

for concA in [12,24,48,96]:
    for concB in [12,24,48,96]:
        measureLab(A=concA,B=concB,C=concC,R=25)

Where concC would be whatever would be expected anticipate about 6 or more of the states to be leaning RO and 6 or more of the states to be leaning RI.

Understandably we may not use all of these states for in game modeling unless we are running in parallel on a Cray, but having the option to *pick* some of these states for running the model (as opposed to 0 vs 4 in freeze mode) might help to develop better designs. E.g. What if when you only have one state showing, that's the only state whose energy values get calculated?

And if we cannot do 16 states, could we do the 6 of nine such that 3 lean RO and 3 lean RI where the missing 3 states should be "flip a coin". (I.e. 3 concentrations each for A and B, with C chosen to be something times (2X? 4X?) the mid-range concentration.)

for concA in [5,25,125]:
    for concB in [5,25,125]:
        if concA*concB != 625:
            measureLab(A=concA,B=concB,C=???,R=25)
(Edited)
Photo of Atanas Atanasov

Atanas Atanasov

  • 42 Posts
  • 15 Reply Likes
@jandersonlee Correct me if I'm wrong, is your goal to more precisely measure the concentration of A where the A/C switch triggers using this curve? So later rounds we can improve the precision of the switch by moving that trigger concentration closer to the desired proportion?
For the game I think you don't have to run all those cases. I think you can easily calculate it based on the energies displayed in game - it is proportional to the exp(diff in energies of the on and off state) which is reported in the game.
For the real experiment, I guess we can't measure energies. Could it be that we can approximate the trigger concentration based on the on and off signal measurements without running all those additional experiments?
Photo of jandersonlee

jandersonlee

  • 549 Posts
  • 122 Reply Likes
@Atanas: I wish to try and see how the "switch" progresses at various concentrations, yes. There isn't a binary now it folds this way, now it folds that way change point. Instead it may be 5% one way, 75% another way, and 20% other undesired folds, at one set of concentrations and 55% one way, 10% another way, and 35% undesired folds at a different set of concentrations.

It's kind of like we are working with a "black box" into which we can poke a stick in a hole and measure how far in it goes. (I.e. How well does the reporter bind?) The more holes we can poke through and the more measurements we can take, the better chance we have of knowing the shape of what is in the box.

The energy model used in the game is just a model which "approximates" how we expect the RNA to fold in nature. It gets us in the right ballpark, but doesn't guarantee success. If you look at RNAsubopt (or the NuPACK equivalent) you will see what I mean about so many different simultaneous foldings of given design under given conditions.
(Edited)
Photo of Omei Turnbull

Omei Turnbull, Player Developer

  • 966 Posts
  • 304 Reply Likes
@jandersonlee, I would love to take a closer look at the stats you are generating and posting for many of your designs.  Have you written up how to interpret them anywhere?
Photo of jandersonlee

jandersonlee

  • 549 Posts
  • 122 Reply Likes
@Omei: No public writeups at this point.
Photo of jandersonlee

jandersonlee

  • 549 Posts
  • 122 Reply Likes
1) Would it be possible to increase the number of submissions per player from 3 to (say) 30 for the Sensor A/B/C - RIRI/RIRO labs? It may be useful to test the response of some sequences to varying concentrations of A/B/C as a building block to using snippets for other sublabs such as A*B-RI/RO and the big kahunas A*B/C^2-INC/DEC, especially if:

2) Will these labs also be measured with TB-A/B/C at 5nM and R at 25nM or just at 0nM and 100nM (even if the scoring doesn't use those measurements)?

I find I have more than 3 approaches I'd like to check out for building an A-RI component for example.
(Edited)
Photo of Omei Turnbull

Omei Turnbull, Player Developer

  • 966 Posts
  • 304 Reply Likes
In past labs, and therefore I presume the current lab, all the designs for the round have been measured on the same set of conditions.  (This led to some odd data, such as when an miRNA puzzle was measured with the MS2 protein.) So there shouldn't be any reason you can't submit experiment designs as part of any puzzle that you haven't used up the slots for.
Photo of Astromon

Astromon

  • 182 Posts
  • 23 Reply Likes

Omei Turnbull

  • 623 Posts
  •  
  •  174 Reply Likes






Thanks, Astromon.  It appears to be true for all the puzzles in the main challenge.  I've alerted those who can fix it.

Congratulations for being the first player to hit 4!







Thanks, Astromon.  It appears to be true for all the puzzles in the main challenge.  I've alerted those who can fix it.

Congratulations for being the first player to hit 4!
Photo of Astromon

Astromon

  • 182 Posts
  • 23 Reply Likes
they want 11000 designs so yeah 30 plus probably.  :)
Photo of calebgeniesse

calebgeniesse, Researcher

  • 8 Posts
  • 2 Reply Likes
We have increased the per player submission limits for all of the puzzles in the main challenge. Apologies for the inconvenience!
Photo of whbob

whbob

  • 190 Posts
  • 57 Reply Likes
The getting started round 2 sub labs are still only taking 3 submissions. Is that going to remain only 3 submissions? 
Photo of rhiju

rhiju, Researcher

  • 403 Posts
  • 122 Reply Likes
yea, we'd like players to focus their attention not on the getting started labs but on the labs that might achieve more complex arithmetic operations for the openTB signature. This worked to get lots of submissions for the hard puzzles in the last round. Let us know if you think we should set it up differently!
Photo of whbob

whbob

  • 190 Posts
  • 57 Reply Likes
No, not a problem :)  Just checking.  The labs will have my undivided attention :) 
Photo of worseize

worseize

  • 29 Posts
  • 12 Reply Likes
AB/C^2 takes 10 - 60  times more time to make solution . Up to 6 hours good design for me. Soo is it better anyway make 5-10 that solutions or better to make 50-600 sub missions  ?
Photo of rhiju

rhiju, Researcher

  • 403 Posts
  • 122 Reply Likes
let's spend more time on the AB/C^2 puzzles. Those are the central targets, and are nearly impossible for computers. So those solves are by far the most valuable for eterna's forays into TB medicine -- imagine how amazing it will be if your hard work leads to a calculator that we can test in a diagnostic! 
Photo of Astromon

Astromon

  • 182 Posts
  • 23 Reply Likes
I like to work up to 4 labs at once in different tabs. Will doing that slow down the A/B/C-dec puzzle even more?
Photo of Astromon

Astromon

  • 182 Posts
  • 23 Reply Likes
I like to work up to 4 labs at once in different tabs. Will doing that slow down the A/B/C-dec puzzle even more?