Seadragon Ajax on the iPhone

  • 4
  • Question
  • Updated 9 years ago
  • Answered
Hi guys, I had a couple questions/ideas regarding Seadragon Ajax running on the iPhone.

1) Will you guys publish the recent modifications you've added to seadragon.com for the iPhone to the js library available on the live labs site (which IIRC wasn't up to date to the recent modifications either), and also to the open-source project on codeplex (like the touch controller, etc) ?

2) What about using the HTML5 client db storage as a tile cache ? (eg drawing a tile into a canvas, and storing what canvas.toDataURL() returns)

3) Last time I checked you were using active rendering, with a timer running at 60fps. I was wondering whether you think it'd be a good idea to also have passive rendering in order to have more resources available for the rest of the page/app embedding seadragon ?

Thanks
Photo of johnlanchois

johnlanchois

  • 1 Post
  • 0 Reply Likes
  • thankful

Posted 10 years ago

  • 4
Photo of Aseem Kishore

Aseem Kishore, Former Employee

  • 702 Posts
  • 124 Reply Likes
Hey John,

I don't have an answer to #1 at the moment; we'll get back to you on that.

To #2, can you tell us what benefits that would have over the regular browser cache? I'll tell you that canvas.toDataURL() is unfortunately too slow for real-time use. It's also a synchronous API that blocks the main UI thread while the image is encoded, which is unacceptable to us.

To #3, we're actually a mix of the two. We actively render (trying to achieve as close to 60 fps as we can get) during animation and while new tiles are being blended in, but once an animation is finished and we're done resolving, we don't do any major work until the next animation begins. So while the user isn't interacting with Seadragon, we use up minimal resources. Is there a specific reason you ask, though?

Thanks for your interest!

Aseem
Photo of Bill Crow

Bill Crow, Seadragon Group Manager

  • 100 Posts
  • 38 Reply Likes
Official Response
Hi John,

To #1:

We're a labs organization; lots of stuff we do is just an experiment; other things get integrated into products or we share them (via www.seadragon.com or other channels) to gather feedback, or enable and inspire new ideas.

We gave the Seadragon Ajax code to the team at Microsoft delivering the ASP.net Ajax Control Toolkit open source project. They've integrated Seadragon into the toolkit and it's framework. They're managing this open source project, and anyone can contribute enhancements.

Recently, we've been experimenting with some improved mobile support on the www.seadragon.com website. There's no immediate plans to incorporate these experiments into our hosted Seadragon Ajax library or contribute to the open source version in the Ajax Control Toolkit. For our small team this is really just an issue of time, resources and priorities. Our current priority is continued experiments for forward looking features.

The iPhone touchscreen enhancements are using the documented Safari Javascript interfaces. Anyone is free to contribute enhancements to the open source project based on these platform specific browser extensions.

Bill Crow
Seadragon Group Manager
Microsoft Live Labs
Photo of Daniel Gasienica

Daniel Gasienica

  • 73 Posts
  • 19 Reply Likes
Thanks Bill for addressing this question.
–Daniel
Photo of Sean Rice

Sean Rice

  • 4 Posts
  • 0 Reply Likes
I've got it working on an iPhone. It's a bit slow.... I'm going to look into optimizing, but here's what I have, and this should get you started. Any ideas to make this less clunky?


var viewer;

function init() {
viewer = new Seadragon.Viewer("container");

viewer.addEventListener("open", afterCreation);

viewer.openDzi("GeneratedImages/dzc_output.xml");
viewer.setFullPage(true)

}
function afterCreation(viewer) {
viewer.clearControls() ;

//Add touch events now

document.getElementsByTagName('canvas').item(0).addEventListener("touchstart",startMoving, false);
document.getElementsByTagName('canvas').item(0).addEventListener("gesturestart",startGesturing, false);
}
function startMoving(e){

e.preventDefault();

this.startX = e.targetTouches[0].clientX;
this.startY = e.targetTouches[0].clientY;
this.centerCoords = new Seadragon.Point(this.startX, this.startY);
if (e.targetTouches.length == 2){
this.centerCoords = Seadragon.Point(((this.startX+e.targetTouches[1].clientX) / 2), ( (this.startY+e.targetTouches[1].clientY) / 2));

} else {
this.currX = this.startX;
this.currY = this.startY;
this.addEventListener('touchmove', keepMoving, false);
this.addEventListener('touchend', stopMoving, false);
}
}
function keepMoving(e){
e.preventDefault();
if (e.targetTouches.length != 1)
return false;

var delta = new Seadragon.Point((this.currX - e.targetTouches[0].clientX),(this.currY - e.targetTouches[0].clientY ));
this.currX = e.targetTouches[0].clientX;
this.currY = e.targetTouches[0].clientY;
delta = viewer.viewport.deltaPointsFromPixels(delta);

viewer.viewport.panBy(delta);

}
function stopMoving(e)
{
e.preventDefault();
viewer.viewport.applyConstraints();
this.removeEventListener('touchmove', this.touchMoveHandler, true);
this.removeEventListener('touchend', this.touchEndHandler, true);
return false;

}
function startGesturing(e){

e.preventDefault();
this.startZoom = viewer.viewport.getZoom();
this.startPoint = new Seadragon.Point(e.position);

this.addEventListener('gesturechange', keepGesturing, false);
this.addEventListener('gestureend', stopGesturing, false);
}

function keepGesturing(e)
{
e.preventDefault();
// Only interpret gestures when tracking one object. Otherwise, interpret raw touch events
// to move the tracked objects.

var zoomdelta = e.scale;
viewer.viewport.zoomTo((zoomdelta * this.startZoom), this.centerCoords);

return false;

}

function stopGesturing(e){

e.preventDefault();
//set zooms back to within threshholds

viewer.viewport.applyConstraints();

this.element.removeEventListener('gesturechange', this.gestureChangeHandler, true);
this.element.removeEventListener('gestureend', this.gestureEndHandler, true);
return false;

}

Seadragon.Utils.addEvent(window, "load", init);
Photo of Aseem Kishore

Aseem Kishore, Former Employee

  • 702 Posts
  • 124 Reply Likes
This actually looks pretty good! Do you have a demo of this running somewhere? =)
Photo of Sean Rice

Sean Rice

  • 4 Posts
  • 0 Reply Likes
Yes, my demo for this is available here:
http://www.seanrice.net/testing/seadr...

Let me know if you see any way to make this more responsive. I've been testing on an iPhone 3g, and I'm not sure if the bottleneck is downloading the resources, or something to do with the javascript execution.

By the way, it appears this works also on android devices that have multitouch enabled (others just have to use the + and - buttons to zoom)
Photo of Sean Rice

Sean Rice

  • 4 Posts
  • 0 Reply Likes
After looking at others' Seadragon implementations for iPhone, it looks like mine performs on par with theirs, though my puny little shared goDaddy server space probably makes it a little more chunky. Those with better servers/bandwidth may get a better experience.

Still, any advice on how to optimize for mobile devices would be most welcome!

Mostly, I just want the experience to be smooth. It would be nice if you could zoom and pan as quickly (or close) as when looking at an image in iphoto. Even if it takes a little while to load the tiles. Right now, it jumps around a lot when panning or zooming. And it almost breaks the experience for me.
Photo of Aseem Kishore

Aseem Kishore, Former Employee

  • 702 Posts
  • 124 Reply Likes
Sean, multi-touch is a super tricky problem, so I sympathize. (I'm actually working on multi-touch interactions for an iPad magazine viewer right now!)

For smoothness, I think this just boils down to rendering speed, out of your control. There are ways to hardware accelerate graphics on mobile WebKit devices (iPhone, iPad, Android), so I've added investigating this onto my wishlist: https://github.com/aseemk/seadragon-a... =)
Photo of Aseem Kishore

Aseem Kishore, Former Employee

  • 702 Posts
  • 124 Reply Likes
Oops, put the wrong link. Updated my post.
Photo of odyssey

odyssey

  • 5 Posts
  • 0 Reply Likes
Hi Sean,

I have joined your code with my Gigapan-Seadragon project, allowing the viewing of gigapixel images on Iphone. I added beta support for the gyroscope and code to use Gigapan tile sources.

I changed the animation time to 0 seconds and have panning set to immediate, so the response seems a bit better.

As far as using the gyrsoscope to look around, it has some problems in that it gets far more sensitive when zooming in to deeper levels (hand shake becomes evident) and Seadragon does not like to load and render tiles when things are shaking around even a little, so I added code that would require minimum movement to have a pan event take place, and this helped the loading of the tiles, but it makes the gyroscope response jerky. I wish I knew how to tell Seadragon to load the tiles even when things are panning a bit. I have the gyroscope set to stop updating position when a finger touch event is happening, so you can freeze the view by holding your finger on the screen. You can pan and zoom with touch events of course.

I would like something where maybe the gyroscope becomes less responsive when zooming in deep, where maybe you would have to turn multiple circles to pan all the way across an image.

I would also like to figure out how to have the minimum zoom set to the images height, so that the image takes up the full height of the display and panning left and right with the gyroscope would have no input from the vertical movements when zoomed all the way out, just looking around left and right.

Gigapan tile sources have black bit taking up the remainder of the last tile on the right, so setting the experimental horizontal wrap in the config produced undesirable results, so I turned wrapping off. If there was a way to have it render on the bottom instead of the top it would work though.

Here are the examples, best on an IOs 4.2 device (updated Iphone 4 or Ipod touch 4th Generation)

www.3d-360.com/i/ Aspen Highlands
www.3d-360.com/i/?id=66626 Currently the world's largest image, at 276 Gigapixels!
www.3d-360.com/i/?id=8196
www.3d-360.com/i/?id=41230

You can replace the id with your favorite Gigapan image id.

www.3d-360.com is my site for exploring 3D-Gigapixel images (using Anaglyph, nvidia 3D vision, 3D Tv's, and 'wiggle vision' viewing technologies) I have a patent pending method for aligning the left and right images in real time for zero eye strain when zoomed in deeply into an image and anaglyphing on the fly. Currently there are over 100 high resolution 3D panoramas to explore (click on the images to enter the 3D viewer, there is a viewing method selection in the top left, the anaglyph method is set up for standard red-cyan glasses)

Cheers

Jason
Photo of odyssey

odyssey

  • 5 Posts
  • 0 Reply Likes
While it is smoother now, it is still not as smooth or responsive as Google Earth or Google Maps on the Iphone, so it is obvious that the hardware is capable of smooth, fast, responsive maps, but maybe that javascript is not yet up to the task. Maybe if we could integrate some hardware css acceleration it would be better? Regardless, the above is still fun.
Photo of odyssey

odyssey

  • 5 Posts
  • 0 Reply Likes
While absolutely not Seadragon, I tried out using the Google Map javascript API to view these Gigapan tile sources, results are similair in speed, but the GoogleMap viewer does not show the lower rez while loading the higher rez tiles. It does seem to do a bit of better mouse touch, but still no where near as good as the native iphone Google Map application.

Here are the results:
http://www.3d-360.com/g/
http://www.3d-360.com/g/66626
http://www.3d-360.com/g/8196
http://www.3d-360.com/g/41230

again, you can replace the number with your favorite Gigapan ID
Photo of Sean Rice

Sean Rice

  • 4 Posts
  • 0 Reply Likes
Hey! I'm glad that you found my code useful! Good call on the tweaks. It looks like any performance issues are due to phone processing speeds. I've been looking at this stuff on a 3g, and an 2nd gen iPod touch.