Player Driven Development & Tools for EteRNA (beyond scripting).

  • 8
  • Article
  • Updated 6 years ago
From: Rhiju Das
To: Chris Couteau
Cc: Jeehyung Lee
Sent: Tuesday, April 23, 2013 12:42 PM
Subject: Re: EteRNA_thoughts

Hi Chris:

Thanks for contacting me, and sorry for the late response -- I've been out of town.

I've been reading your posts with much interest -- I'm amazed that you're finding the same articles that our 'expert' group also keeps track of. For example, regarding the SHAPE 'soft constraints' paper of Washietl et al, my group has checked out that code -- its not accurate in all cases of known structure, but neither is the current approach in eterna.

The validation and inclusion of known tertiary motifs is also fascinating, but we will probably wait until building a 3D version of eterna to incorporate those ideas. They are obviously critical, however, if we want eterna to be an all-purpose RNA design platform.

So what can we do? Our dev team is swamped.

What API could we create in eterna for players so that *you* could implement the ideas you've suggested and any new ones?

We've got a test scripting language, and are working on a 'library' to promote sharing of papers and writing of new ones.

What if you had read-access to the eterna database -- could you (with other players) create novel applications to highlight tertiary motifs, or implement alternative SHAPE-guided modeling algorithms.

I also wonder if the same kind of API would permit 'experts' (like Washietl) to make their own tools available as on-line apps that directly interface with eterna.

Let's stay in touch about this. Also, if you have a chance, it would be worth starting a forum thread that links many of your ideas, and also asks players to brainstorm about what the eterna team could do to make the ideas happen. Again, the constraint here is that we have limited resources -- the best thing would be to create mechanism for players themselves to develop and share new tools.

As part of that thread, please feel free to post the above reply from me.

Best,
Rhiju
-----------------------------------------------------------------------------------------------------

So, the question is, what can we do with this potential opportunity? I think it important enough, and have agreed to canvas the community for interest and skills and projects. What is missing that you might be able to implement? What tools do you need that we just don't have? Can you turn it into an application that works directly in EteRNA? Are you writing code for scoring functions, motif selection or something else?

The dreams will require organizational planning and implementation---you have any relevant skills that you could participate with?

The floor is open for discussion.
Photo of eternacac

eternacac

  • 274 Posts
  • 19 Reply Likes

Posted 6 years ago

  • 8
Photo of eternacac

eternacac

  • 274 Posts
  • 19 Reply Likes
signed_ETeRNA_Thoughts_forumpost
Hide Details

From: Chris Couteau
To: rhiju
CC: jeehyung

Rhiju,

A preliminary survey indicates some interest and skill sets, so I have made the following post to the forum.

https://getsatisfaction.com/eternagam...

Whomever finally leads such an undertaking will likely need a full understanding of the software development environment for EteRNA, and the organizational structure and points of contact for EteRNA, the dev team, and any business-organizational plan that would intersect such an undertaking. You are probably governed by some set of non-profit and other rules related to Stanford and CMU that we must not violate.

I have postulated in chat the possibility of using a software "hackathon" in the mode of Facebook or Google, to rapidly prototype solutions that we may be able to use, realizing that this may be nothing more than speculation on my part. It also occurs to me that we might be able to crowdsource funding for implementing software development outside of normal channels, the above constraints being observed.

Thinking about how and what might assist EteRNA and the players.

Regards,

Chris [eternacac]
Photo of eternacac

eternacac

  • 274 Posts
  • 19 Reply Likes
And his response, which seems to confirm the possibility...

RE: signed_ETeRNA_Thoughts_forumpost

From: Rhiju Das
To: Chris Couteau
CC: jeehyung

I think this is awesome. I will keep monitoring the thread.

Rhiju
Photo of eternacac

eternacac

  • 274 Posts
  • 19 Reply Likes
So, Players--the ball is in our court.
Photo of eternacac

eternacac

  • 274 Posts
  • 19 Reply Likes
Ok, our first success!

Jnicol let me know that he had already made a javascript port of the Vienna 1.8.5 Fold algorithm, so I asked Jeehyung if that was of use/interest to him. His response was:

Re: EteRNA_player_js_port_of_Vienna1.8.5Fold

From: Jeehyung Lee
To: Chris Couteau
CC: rhiju

Hi Chris,

the js port could indeed help us a lot - especially for the development of
scripting system, and possibly many more web site functionalities. If you
could share the link with us - that would be great. I'll also ask jnicol
about this.

Jee
Photo of eternacac

eternacac

  • 274 Posts
  • 19 Reply Likes
This makes me think that since virtually all of Vienna is coded in C/C++, if we could parse out the modules and use the emscripten link posted on the forum, we could port the entire suite in a very short time, which might be even more useful.

Also, a GitHub account might be a way to solicit additional coding help from expert coders, and manage projects.
Photo of eternacac

eternacac

  • 274 Posts
  • 19 Reply Likes
I asked Jeehyung if a javascript port of the entire Vienna package would be of use/interest, since he was excited by jnicol's work on the js-port of Vienna fold.

And he says that yes it would be both, and would like to talk with us about it more.

So, if you have the interest and skills, go look at the Vienna suite and start thinking if you would be willing to help out.

I'm looking at emscripten with clang as a possible toolset for the job, and Mozilla has been doing some awesome work porting highend games with physics packages in just days! Could we be so lucky that that might work?

Soembody with better skills than I might want to take a look too :)
Photo of eternacac

eternacac

  • 274 Posts
  • 19 Reply Likes
Software found on bit bucket that may be of interest:

rna-js allows you to deploy Web Services
https://bitbucket.org/fjossinet/rna-js

rna-js-clients provides a documentation and several source code samples for basic clients using the Web Services exposed by the embedded server of rna-js.
https://bitbucket.org/fjossinet/rna-j...

More advanced clients can also be used, like Assemble2 and PyRNA.

PyRNA is a python library delegating all the computations to the rna-js Web Services. It transforms most of the data computed as pandas data structures. Consequently, this library allows you to generate and mine RNA data without prior installation of algorithms on your computer.
https://bitbucket.org/fjossinet/pyrna
Photo of eternacac

eternacac

  • 274 Posts
  • 19 Reply Likes
This may have some useful code--you'll have to look at source code. Says JAVA not Javascript, but still might be convertible.

http://varna.lri.fr/
http://varna.lri.fr/index.php?lang=en...
Photo of eternacac

eternacac

  • 274 Posts
  • 19 Reply Likes
JBrowse is a fast, modern genome browser written primarily in JavaScript.

Very light server resource requirements. Serve huge datasets from a single low-cost cloud instance.

Easy to install, embed and customize. (So why not for EteRNA?)

http://jbrowse.org/
Photo of eternacac

eternacac

  • 274 Posts
  • 19 Reply Likes
Rtips: fast and accurate tools for RNA 2D structure prediction using integer programming

http://rna.naist.jp/

Does pseudo Knots! Using Integer Programming.
http://www.ncbi.nlm.nih.gov/pmc/artic...
Photo of eternacac

eternacac

  • 274 Posts
  • 19 Reply Likes
and in the FWIW context, I think Haskel is worth using. Now.

http://www.haskell.org/haskellwiki/Ha...

"Enter Haskell. Let’s start that haskell is more than a bit weird at first take, even for an old software guy like me. Haskell has been around for a while, but has only recently started to find a flourishing community. At JanRain we’ve known about haskell for quite sometime. Internally we use darcs which is written haskell. Haskell allows us to write highly parallel-able, asynchronous code (lazy evaluation), that works, is easy to maintain/test, and runs at near C speed. I think that sums it up :).

The Results. Benchmarks are benchmarks, non-the-less, it’s what we have. What we measured was our haskell code doing over 100X throughput vs our legacy code. I’ll say it again, because it sounds good. Over 100x increase in throughput."

" Haskell allows us to write highly parallel-able, asynchronous code (lazy evaluation), that works, is easy to maintain/test, and runs at near C speed. I think that sums it up :)."
...
"Will we do more haskell? Absolutely. For a certain class of backend problem it will now be the language of choice here at JanRain."
Photo of eternacac

eternacac

  • 274 Posts
  • 19 Reply Likes
But short of Haskel, you could try functional javascript--

http://code.google.com/p/functional-j...

http://osteele.com/sources/javascript...
Photo of eternacac

eternacac

  • 274 Posts
  • 19 Reply Likes
Turns out there already is a GitHub-public account at https://github.com/EteRNAgame/EteRNA-...
Photo of eternacac

eternacac

  • 274 Posts
  • 19 Reply Likes
from Rhiju this:

The main issue we are facing is finding good folks who can help us dream up and code up good functionalities in eterna -- we are recruiting from stanford, berkeley, and CMU now, but its going to take time to train everybody.

So we want there to be a maximum multiplier effect for the code that our current development team can spend time on...

Perhaps the best thing for you to brainstorm is what kind of APIs we need to put into eterna so that savvy players can start adding their own functionalities. We probably will not be able to go to a model where players are themselves writing the core server code in javascript or flash, due to potential security issues.

However, we could create new APIs for user-generated content.

For example we did this for 'player puzzles' and kind of had this for the 'strategy market' -- and we are trying to do this over the next months for in-game scripting and for player and expert projects.

What would you and others need to contribute the new content and functionality that you and others really want?

Feel free to post...

Best,
Rhiju
Photo of eternacac

eternacac

  • 274 Posts
  • 19 Reply Likes
from Jeehyung this:

Hi Chris (eternaac) and jnicol,

Thanks for agreeing to help us out on the scripting interface development. As we previously discussed, I want to keep an active thread on Eterna scripting thread to focus on 2 big problems.

1. What do Eterna expert players need from the scripting interface to implement really interesting/practical stuff?

2. How can we make the ViennaRNA & (other RNA libraries) port easier to bring in the needed functionalities as rapidly as possible?
[ jnicol has already made a progress on porting ViennaRNA to javascript (www.amelior.com/js/vienna185.js) and Chris has been running experiments with emscripten as well]

Please feel free to throw in any other discussion topics, idea as well. We would also love to hear how you ported vienna, and your impression on emscripten, etc!

Jee
Photo of Eli Fisker

Eli Fisker

  • 2223 Posts
  • 485 Reply Likes
PROGRAMMERS PLAYGROUND

(Mat got this great idea and I am adding this forum post for him)

I was thinking about an idea after reading Eternacac’s forum posts Player driven development tools for Eterna beyond scripting and Qt GUI Toolkit Ported to JavaScript via Emscripten. Here are two of Rhiju's comments:

"The main issue we are facing is finding good folks who can help us dream up and code up good functionalities in eterna".

"We probably will not be able to go to a model where players are themselves writing the core server code in javascript or flash, due to potential security issues."

But what if dev didn`t give access to the main server, but on a separate server (which could be smaller) with limited access to a selected number of players? Just like the earlier beta testing area.

Having a limit would mean reduced server requirement and load. That means devs could use a small computer to run it.

On a separate server players could show dev what they are able of, with no or low potential security issues for the game. Also a smaller server would also limit the risk for players that test and code. If they should make a bad change, it will not affect the big server. This again would likely make more players dare to test their programming skills. There could be made individual accounts for that extra server, so one player could not close down the testing ground for the others, if unlucky.

It could be a sandbox for players ideas. If players develop any useful code that could be used in the game, dev could integrate that into the game later. Also we could all go and see which changes that we love and absolutely want implemented.

EINando888 has made a real fine offer. As he said:

“I could help with hardware and (a little) bandwidth, if they don't have this "small" server and need one. For what it's worth, I rent a box online, not amazon but another provider called Rackspace. I use it mostly only for the purpose of developing Vinnie. Well, I'd be glad to offer this host to EteRNA, if it can help."

Thx to Starryjess for the edit
Photo of Eli Fisker

Eli Fisker

  • 2223 Posts
  • 485 Reply Likes
Home for volunteer developers - like that thought, JL. :)

Spacefolder, thx for sharing this advice for novice programmers. Great teacher you had.
Photo of Eli Fisker

Eli Fisker

  • 2223 Posts
  • 485 Reply Likes
SCRIPTING SANDBOX

If you want a place to play and not having your temporary and perhaps not working scripts among the published scripts, Brourd suggested you can post them on the test server. Our former beta test site.

LINK TO TEST SERVER

Hoglahoo tested it out. It seems that every script that gets posted in the our regular Eterna page, turns up on the beta test site. But everything posted on the dev beta test side, doesn't get mirrored on the main site. From Hoglahoo's testing out:

hoglahoo: oh ok, so whatever is published on .com shows up on .org but not other way around?
hoglahoo: Eli, I created on .org but I don't see it on .com. So I guess it's a one way copy machine or something?
Eli Fisker: Great
hoglahoo: yep, so this should work
Eli Fisker: So we have our script playground
Eli Fisker: :)
Photo of Eli Fisker

Eli Fisker

  • 2223 Posts
  • 485 Reply Likes
Ok, JL, so now the two scripting pages appears completely seperated. Thx for telling.
Photo of kws4679

kws4679

  • 20 Posts
  • 0 Reply Likes
Totally agree with your opinion Eli. But Jee cares few bad users who try to manipulate code to solve puzzles without algorithmic calculation (e.g find posted solution from other users and hard coded them into script). Currently, dev server is completely seperated from production server and Jee,Diana and me have each branch can't be interfered with others. Actually I implement developing environment that can make experts users download, analyze our script related codes and test "locally" on their own computers. But your opinion is great since it isn't too difficult to make a new instance on aws with same environment as production server. I'll ask it to jee. Thx Eli
Photo of jandersonlee

jandersonlee

  • 549 Posts
  • 122 Reply Likes
> 1. What do Eterna expert players need from the scripting interface to implement really interesting/practical stuff?

A) I think it would be great to have a testing harness that let users run a function against selected kinds of puzzles/labs. For example, if I am designing a StrategyMarket function, I might want to run it against some number of random lab designs that had been synthesized to see if it seems to be predictive of scoring. It would also be useful to select switch(FMN) versus non-switch puzzles

Or if I was designing a puzzle solver, I might want to test it against a number of random puzzles with between 50 and 100 solvers (to see how it works on moderately hard puzzles) or 200 to 500 solvers to test it on easier puzzles.

B) I tend to design by thinking of how to avoid making potential miss-folds. Having a version of RNAsubopt available might help a puzzle-solver script to distinguish a solution that folds per the model but has a lot of alternative foldings at similar energies from a solution that has a strong bias for the target form. Knowing the Free energy of a particular shape for a given sequence might also useful, particularly for switch puzzles where multiple shapes are a given.
Photo of Eli Fisker

Eli Fisker

  • 2223 Posts
  • 485 Reply Likes
I wholeheartely agree with what JL is saying here:

"A) I think it would be great to have a testing harness that let users run a function against selected kinds of puzzles/labs. For example, if I am designing a StrategyMarket function, I might want to run it against some number of random lab designs that had been synthesized to see if it seems to be predictive of scoring. It would also be useful to select switch(FMN) versus non-switch puzzles

Or if I was designing a puzzle solver, I might want to test it against a number of random puzzles with between 50 and 100 solvers (to see how it works on moderately hard puzzles) or 200 to 500 solvers to test it on easier puzzles."

All very useful.

Things I would like do be able to drag out from the scripting interface:

1) Energy from spots in a specific lab. So I can get the energy for a particular quad. And preferably more.

2) Total energy of a specific element in a specific lab

3) Total energy of a whole design

4) Total energy of a whole element paired with how well it actually folds when viewed with the CSE tool.

5) I will that we will be able to search for specific elements and size of elements, so we can pick up different data on them. I will like to be able to see the energy of a multiloop and specify what other informations I want, like quad energy around it. I have made spreadsheets where I check into this manually. For now I can say that the total quads around a multiloop (The nearest ones should reach a certain treshold of energy and in particular not have too little, but not necessarily have full power either.) They should be in a certain range. I feel we will have much stronger predictive power in the future for different sizes and types of multiloops, with being aband being able to look into the numbers combined.

6) I will also like to be able to search for a certain type of meltplop. That could also help in the voting area to sort out a little of the designs I will rather not see. A bad meltplot is a bad meltplot. :)

I have already mentioned a lot of ideas for what I wish us to be able to do, in this forum post EteRNA dreams

I'm sure we can come up with many more ideas on what we need from a scripting interface. We have a ton of fine suggestions in the forum already.
Photo of eternacac

eternacac

  • 274 Posts
  • 19 Reply Likes
So should we, can we use something like this to track the ideas/solutions as they progress?

Trello Board:
https://trello.com/tour?board=%2Fboar...
Photo of jandersonlee

jandersonlee

  • 549 Posts
  • 122 Reply Likes
I've been playing with some alternative interfaces, like a "Switch Ring" view of the sequence and bonds:



It would be nice to have a home where I could put these so that others could try them and help develop them further.
Photo of eternacac

eternacac

  • 274 Posts
  • 19 Reply Likes
Fantastic tool. How do I get to use it?

Did you see this post https://getsatisfaction.com/eternagam... ?

Or did you develop this great tool unaware the technique had been mentioned?
Photo of jandersonlee

jandersonlee

  • 549 Posts
  • 122 Reply Likes
I may have been influenced by that forum thread. I don't remember. I was working on it around the same time.
Photo of SpaceFolder

SpaceFolder

  • 10 Posts
  • 0 Reply Likes
Do you want demonstrably professional coders or do you also want people like me who like to code for fun and come up with some nifty things every now and then? Would breaking up significant coding assignments into small pieces and distributing to the "crowd" be useful? I have a feeling I would be the low guy on the totem poll given the kinds of high-quality coders you seem to have access to.
Photo of jandersonlee

jandersonlee

  • 549 Posts
  • 122 Reply Likes
I think we need a (1) build/test/development server, and (2) (git?) repository server, and (3) suitable "sandboxes" for people to play in. If it is reasonably sandboxed, then all skill-levels are welcome. (AKA crowd-sourced development.)
Photo of SpaceFolder

SpaceFolder

  • 10 Posts
  • 0 Reply Likes
I'm tickled that you guys are even considering this...just delighted!
Photo of jandersonlee

jandersonlee

  • 549 Posts
  • 122 Reply Likes
Uhh. Who is "you guys"? I'm just a player. :)
Photo of SpaceFolder

SpaceFolder

  • 10 Posts
  • 0 Reply Likes
Oh sorry. My tickledness still stands though. You said "all skill levels welcome". I interpreted that to mean you were one of the promoters.
Photo of jandersonlee

jandersonlee

  • 549 Posts
  • 122 Reply Likes
I think the basics of crowd-sourcing are much like that of chat: by default, you are welcome to join, unless you make a nuisance of yourself, in which case you are banned.

Sandboxing development makes it harder to interfere with one another. You could still become a nuisance by suggesting all kinds of weird and unnecessary changes, but that doesn't often happen.

Typically one or two people will become the final arbiters of what changes are included in the "master" version line, but beyond that, people do their own thing, knowing that that have to play by certain rules to get their changes folded back into "master".
Photo of jandersonlee

jandersonlee

  • 549 Posts
  • 122 Reply Likes
And here is a alternative Dot Plot Viewer that lets you step through sub-optimal foldings and see the dot-plot in a ring:

Photo of rhiju

rhiju, Researcher

  • 403 Posts
  • 123 Reply Likes
@jandersonlee -- did you write that tool yourself?
Photo of jandersonlee

jandersonlee

  • 549 Posts
  • 122 Reply Likes
So far. I leveraged the Vienna tools of course, but most of it is Javascript and runs in the browser. I do need a server to serve up the JavaScript and run RNAsubopt on demand though (as I don't have a JS version of that.) Gotta love HTML5 w/ Canvas!
Photo of eternacac

eternacac

  • 274 Posts
  • 19 Reply Likes
Great tool, love the color use. did you know there was a similar concept posted in the forum--but you have made it glorius!

I think jnicol has laready made the RNAsubopt port to javascript, or is working on it. If not, ElNando is trying Emscripten. Let's hope it goes well--he could possibly port all of either or both versions of Vienna. Or show that it really is possible.
Photo of xenosapien

xenosapien

  • 7 Posts
  • 0 Reply Likes
Wow. There's some serious thinking going on in this community. Kudos to all involved! I'm still getting my "sea legs" around here but seeing topics like this is very motivating for digging into the deeper science of it all (papers, research, etc.).

I only hope I can catch up before it (EteRNA) comes out in 3D at a theater near me :).
Photo of eternacac

eternacac

  • 274 Posts
  • 19 Reply Likes
Check my profile for a list of some papers that might be of interest, as well as all the top players--most have something somewhere on the site or the wiki.
Photo of eternacac

eternacac

  • 274 Posts
  • 19 Reply Likes
An RNA visualization tool in Java woth looking at that can be run from a webpage, or on your own computer:

use the sample files on right side of download page.

http://jviz.cs.sfu.ca/download/downlo...
Photo of rhiju

rhiju, Researcher

  • 403 Posts
  • 123 Reply Likes
@eternacac -- have you been able to find anything similar for easy plotting of RNA secondary structures in Javascript? For other resources, such as the RMDB we have been using Java-based VARNA

http://varna.lri.fr/

But its been a pain to maintain because of Java issues in some browsers, and would like something analogous in javascript...