What features should we add to Photosynth?

  • 1
  • translation missing: en.shared.topic.styles.singular.talk
  • Updated 7 years ago
We've got our backlog list of things that we know will make Photosynth better, but I'm sure we're only seeing some of the opportunities here. What would make Photosynth drmatically more useful and fun for YOU?
Photo of David Gedye

David Gedye, Principal Group Manager

  • 176 Posts
  • 78 Reply Likes
  • unsure

Posted 12 years ago

  • 1
Photo of dflipb

dflipb

  • 1 Post
  • 0 Reply Likes
I am also would like to download the resulting pointcloud files of my synths. And if it would be able to stitch together a 3D model with that information, that would be veryhelpful have it export to maya or Studio max
Photo of Michael

Michael

  • 1 Post
  • 0 Reply Likes
A few ideas:

1) Superior navigation. Currently, I get confused trying to look at a particular angle or zoom in. Because Photosynth projects photos onto the plane based on where it expects the photo was taken from, it over-emphasizes the photo at the expense of the scene. I want to see, move around the scene where the photos were taken, not just have a fancy new way to look at photos individually.

This is a particularly big problem when you have large scenes that have more than one object in them. Navigating around a room is difficult; navigating between two rooms is nigh on impossible. You have to jump back to the flat picture view and much around.

2) 3D scene generation. What the point map represents is that you can infer a rough 3D scene and map photos onto it. What would be cool is if you then tried to generate the real scene; creating polygons, not just points, using the pictures as data. I imagine you would have to take picture metadata into account, where it exists (exposure, time of day, etc) when generating textures to control for lighting differences, and so on.

I would particularly like to see something like this in Virtual Earth. The ability for, say, a college to take thousands of photos and have Virtual Earth generate 3D buildings, classrooms, etc is hugely valuable.
Photo of mike

mike

  • 4 Posts
  • 1 Reply Like
I agree, the navigation feels a bit obtuse. It is also hard to determine where you can and cannot go in the synth which makes it easy to get lost and unable to find your way back.

Regardless... this is a really cool piece of software!
Photo of JackH

JackH

  • 43 Posts
  • 0 Reply Likes
I also agree regarding the navigation:
Photo of rebuilder

rebuilder

  • 1 Post
  • 0 Reply Likes
Agreed, some tweening would be nice. Maybe in cases where there's sufficient data to get a decent 3d pointcloud the point information could be used to warp the photos, sort of like using the point cloud data as a depthmap for each photo. This way the photos wouldn't just float in space, but you could really go around them.
Actually it seems some MS researchers have been working on something that might be helpful:
https://www.youtube.com/watch?v=mNLx9p...
Photo of Seika

Seika

  • 5 Posts
  • 0 Reply Likes
In the previous version, there is an option to view an overall point cloud of the collection, and also marker of viewpoints in the picture. But that feature is removed in this release.
I think it helps a lot for large collection, such as outdoor scenes.
Photo of Graeme Whelan

Graeme Whelan

  • 1 Post
  • 0 Reply Likes
I'm not sure if I'm allowed to mention the 'G' word on a Microsoft dev team blog, but I read something about Firefox, so here goes...

Has anyone tried to use photosynth on the images from Google Street View to provide additional viewing angles etc? I'm not into real-estate, but a friend of mine who works in the industry said that was the only thing missing from Street View, and that the commercial value of using the technology this way would be enormous. Apparently a lot of houses take a long time to sell because of poor quality or no photos. Either the resolution is too low, or they take far too long to load on the page, and photosynth could solve both problems. Try contacting a local real-estate industry group and you might find that your dev shop is suddenly very well funded!
Photo of bitplane

bitplane

  • 56 Posts
  • 11 Reply Likes
Street View is a competing product, I seriously doubt MS would consider using Google's data (or Google would give it to them at a reasonable price), this is the one thing that will give Microsoft the ability to compete with Google Earth's 3D models
Photo of mike

mike

  • 4 Posts
  • 1 Reply Like
I agree with bitplane. I can easily see this crushing Google street view (which I think is pretty awesome) if photosynth catches on in a big way. Everyone will be photographing everything and uploading it for photosynth to process.
Photo of speakermagnet

speakermagnet

  • 1 Post
  • 0 Reply Likes
How about integration with MS Map Live Search's "Bird's Eye View" instead of Google's Street View? It typically provides 4 orthogonal vantage points for any particular place which should serve as a great starting framework for synths
Photo of mix

mix

  • 1 Post
  • 0 Reply Likes
A Mac version & GPS. (I've been reading other peoples comments.)
Photo of bernardo

bernardo

  • 3 Posts
  • 0 Reply Likes
instead of walking around an object taking photos, will it work if i just set up my camera on a tripod, set the lighting and backdrop and then rotate the object on a lazy-susan? in other words, does the back ground need to change for the program to do its thing? thanks
Photo of JackH

JackH

  • 43 Posts
  • 0 Reply Likes
There is an instructional video showing about that, but the background in that video is very white.
Photo of NicolasBoo

NicolasBoo

  • 2 Posts
  • 0 Reply Likes
== End-user:
- Keyboard mapping for people that have some handicap and accessibility
- Different size of controls (accessibility too and to embed)
- See a real composed view from the different images (I mean, you set the position and orientation of the camera and then, it manages to display every image to compose an image in a way...)
- localization of the product (support many other languages)
- RSS on Synth and on User collections
- More Search! :)

== Content creator:
- The image processing is client-side, so why can't we see the result before the whole element uploading. I mean I want to check if all is fine before uploading 200 MB of pictures.
- Privacy for the content
- Commercial use and bigger space to upload
- Folder for the "your Synths", in that way we can apply the share options to the folder (like the SkyDrive)
- Composing older Synth (update and/or add new images to an existing Synth) (and RSS on this). Should be great for privacy though.
- Better statistic on Synth (like views by image, quota for every image in Synth..)
- GeoTagged element
- 3d Social Tagging
- Public composition (for example, we open a Synth about The Eiffel Tower and we would like to open this for everyone and we only have to mark the picture that we want to add to the main Synth) (and RSS on this)

== Web developper:
- API or at least something to work on this content and interact with it (like the earlier preview with some commecial products)
- A better integration in Virtual Earth 3d like bird eye view :)... I know that the PhotoSynth Team has just integrated the VE one. I'm sure you're preparing something even greater!
- Different size of controls (accessibility too and to embed)... like I said previously... It's good if an API is available as we can create our own controls.
- A real SLA for the future :) (by now it's normal... remember TerraServer? in 1998 ^^)... it will have to be announced on your website so we can rely on your product.

I have to say that you have done a real great job, we all have many ideas about how to use this technology, keep going and good luck!

Nicolas.

EDIT:
I would love to see this:
Glennon said:
--- time browsing support. Same scene through time.
Photo of Mark Wilson

Mark Wilson

  • 3 Posts
  • 1 Reply Like
A cross-platform version. GPS. RSS support. A version that doesn't need a client app? More guidance on the various keyboard shortcuts. Integration with photo-sharing sites (like Flickr)... basically all the stuff that people asked for on this afternoon's Backchat call.
Photo of stealthtech

stealthtech

  • 25 Posts
  • 3 Reply Likes
GPS, The ability to transition from one Synth to another. Synth 1 is the room Synth 2 is the lamp in the room. I would like to transition from Synh 1 into the Lamp.
Tagging
Mac/Linux or Just make the Client SilverLight plug-in. I know there is some DirectX 3D stuff for performance preventing cross platform.
Is the building of the Synth on the client or in the cloud? what is the purpose of the cloud? I would think it is done in the cloud but I read thing that contradict this. If not then how about a pure Client Side for hosting elsewhere? Or for an Intranet/private projects? If it is on the client then the preview before uploading would be nice otherwise you can't preview until the synth is created in the cloud. But it can still be done, after uploading have a preview with a commit. Or the work around would be to delete.
Photo of david coxon

david coxon

  • 10 Posts
  • 0 Reply Likes
A mac / linux version would make it much more usable as not all users are going to have pc also the ability to embed synths to websites and/or to mash up synths with mapping technologies would make it very commercially usable as a platform.

Another wish list item, but something that would probably be a huge product development would be the ability to navigate time and date as well as space, ie summer verses winter shots and various times of day, i work for an art gallery in the centre of town and there are literally thousands of images, many taken at different times of day and dfferent times of year it would be amazing to use the same photo recognition and stitching to enamble navigate time as well as space.
Photo of Daniel G

Daniel G

  • 4 Posts
  • 0 Reply Likes
I saw this in the "Virtual Tourism" preview, but nonetheless would make a great addition here-- some kind of Windows Photo Gallery plugin, perhaps synth transitions, or synth'd albums that can be published to Photosynth (instead of just spaces?)

I mean, that probably is a little out of realm here, but nonetheless, that would be incredibly useful.
Photo of Bjoern

Bjoern

  • 1 Post
  • 0 Reply Likes
I am pretty sure you guys know the Flickr Camera Finder, maybe you should integrate a similar feature to Photosynth, it would be quite interesting to know what kind of equipment has been used for the Synth, and it would also be an excellent reference for people that want to buy a new camera - add a Synth Webshop to the Site and offer some equipment and the Top 10 Cameras used by Synthers and you'll have an additional revenue stream and will make it a lot easier to use the technology properly for the non-techie people.

As mentioned a "Privacy" option would be great as well and the possibility to chat/discuss Synths on the right side of the screen, there is lots of space there.

Just my two cents, great piece of work so far, thanks guys.
Photo of Kiwi

Kiwi

  • 3 Posts
  • 0 Reply Likes
It would be great if you could actually Synth the images locally and SEE them locally. (And support for Mac would be great, but being a Microsoft project... I guess that it'll be difficult :-)
Photo of Darkphibre

Darkphibre

  • 5 Posts
  • 0 Reply Likes
URL overrides to position camera and choose starting image (may already be there).
Ability to remove photos from the synth, so that we can have lots of lower quality images for creating the matrix, and then leave in only the good images.
The stuff in this video!! :) http://www.istartedsomething.com/2008...
Photo of gaph2000

gaph2000

  • 3 Posts
  • 0 Reply Likes
An option to view only point clouds!

Also, these may or not be developed alrady, still waiting for my synth to finish before looking at others;

Users can contribute to other users synths
Viewing those orange triangles of where the photo was taken
A click and drag to find the picture that best fits the selected area
More 3D controls (Rotate)

Overall it looks great already!
Photo of JackH

JackH

  • 43 Posts
  • 0 Reply Likes
P toggles the points/pics/both.
There supposedly is a rotation control, but I can't find it.

I agree with the other ideas.
Photo of Joshua Podolak

Joshua Podolak, Employee

  • 63 Posts
  • 13 Reply Likes
Or try holding down the "ctrl" button. This works especially well when you're dragging on a halo.
Photo of JackH

JackH

  • 43 Posts
  • 0 Reply Likes
How does one get this halo? I just can't manage to see/produce one.
Photo of marciot

marciot

  • 2 Posts
  • 0 Reply Likes
While the fading effect works well for very closely spaced, mostly planar photos (such as those meant to form a panorama), the "warping" of the faded photos when switching from two distant or orthogonal viewpoints is very disorienting. I think photos should fade out completely before the camera begins moving, then appear only when the camera has settled into place in front of another photo. Hence, rather than having wickedly warping photos, the photos would instead "materialize" out of the point cloud after the move.

So, I suggest Photosynth either provide a user configurable option to the effect of "no photos while in motion" or, better yet, PhotoSynth should incorporate some heuristics to determine when a move would result in out-of-plane moves, and switch into point mode for those moves only, while retaining the fading effect for in-plane moves where it *does* makes sense (such as in panorama shots). I think this feature would make navigation much more seamless, while showcasing those lovely clouds even more!
Photo of marciot

marciot

  • 2 Posts
  • 0 Reply Likes
A "slideshow" mode, which would go through all of the photos sequentially, but would do so in a unique 3D animated fly-though. What I have in mind is that PhotoSynth would compute a path in space that would intersect perpendicularly all the photos, then animate the camera moving along that path. From the user's perspective, it would be just like walking through the scene, and taking pictures, just like the original photographer would have done.

Of course, computing such optimal path that would travel through all the photos in the least amount of time (and with the least amount of nauseating gyrations) would be rather challenging, but I'll leave that problem up to you mathy types!
Photo of Blaise Aguera y Arcas

Blaise Aguera y Arcas, Employee

  • 81 Posts
  • 38 Reply Likes
marciot, have you tried the 'play' button? It might be exactly what you're describing.
Photo of Sierra_Rambler

Sierra_Rambler

  • 11 Posts
  • 0 Reply Likes
Surely you've noticed how in a large synth sometimes individual images end up rendering with their horizon pretty far off-level, even when the rest of the composition is relatively level with the true horizon. It would be cool to introduce a post-synth adjustment tool that allows the creator of a synth to adjust individual images to correct their angle of rotation in the final view. After correction, a partial resynthing could be done -- something like creating a warp or mapping from the original point matrix to a new one, such that the integrity of the synth is the same as ever, but the viewing angles are all adjusted to respect the user's input regarding what the true view was.

Another way to see this is, as good as the program is thus far, it still has little to no objective data regarding the actual camera orientation at the time of shooting (notwithstanding any basic EXIF data that might show a simple 'horizontal vs. vertical orientation')... plus the program has to extrapolate from one image to another in a way, to get them to all meet at the edges, given its various best-guesses on focal distance, camera position, etc. With all that being done, allow the user to correct the final result, essentially introducing a new series of terms or correcting some of the missing constants in the equation... yet with the very simple interface of merely allowing the user to adjust individual image rotation (only).

I hope this request makes sense.
Photo of BlindWanderer

BlindWanderer

  • 1 Post
  • 0 Reply Likes
I'm a long time SL (SecondLife) user and I really miss not being able to just camera about with ease. The SL camera works by locking onto a target and then allowing you to swing the camera about it and move in & out as well. In Photosynth you could do something similar, you could lock the camera onto a pixel in a picture which PS would translate into the 3D position, then you could swing the camera about that point and any picture that had a good probability of having that point could be displayed as the camera angle and position changed. More traditional camera piloting would be cool too. Not being able to freely camera about feels like brain damage, you want to be able to do it but you just can't and it nibbles away at the back of your brain.
Photo of Amir Akbarzadeh

Amir Akbarzadeh, Employee

  • 5 Posts
  • 4 Reply Likes
Try using keys 'a' 's' 'd' 'w' 'e' 'c' for translation.
and keys ';' 'l' ''' '[' for rotation .... notice the third rotation key is " ' " :)

This is even more fun if you first turn off the images by hitting 'p'.

-Amir
Photo of Skinkie

Skinkie

  • 11 Posts
  • 0 Reply Likes
Don't use keys but scancodes for it...
Photo of r-gordon-7

r-gordon-7

  • 9 Posts
  • 0 Reply Likes
I'd like the ability to edit my synths once they've been created - especially because it seems as the program is presently implemented, I can't see my synths before they're posted. In partcular, I'd like to be able to add new images to or delete existing images from the synth after I see how the first iteration has turned out...

I'd also like the ability to delete any of my synths after they're posted - especially if they don't seem to turn out right...

Note, if these features are already available, it should be easier to find this out and to access these features, because I, for one, can't find them...
Photo of Bob_Cat

Bob_Cat

  • 1 Post
  • 0 Reply Likes
I've just seen this on TheRegister and run it against some holiday photos I took last month in Greece. I didn't know if the existence of this tool when I took the photos but at the time I took them it was exactly what I had in mind for the future. I took the photos to preserve something that is rarely seen (a church high in the Greek mountains).
Where it fell down was the fact that it thinks everything is a panorama and when I actually took the photos I took many "reverse angle" shots. In fact I took the entire site we visiting in quite some detail.
I would love to see the system try and 3D model every obviously related picture and then try and see if other pictures map on as well. Perhaps even allowing the user to assist the mapping once it is partially complete. If a 3D model was created and I could push images to the approximate position then the system could work the rest out.
I offer my gallery if the Lab want to experiment.
Photo of Genisis X

Genisis X

  • 1 Post
  • 0 Reply Likes
The ability to use the program without an internet connection would be nice.

-X
Photo of Nashev

Nashev

  • 9 Posts
  • 0 Reply Likes
+1
Need to make and view synth without internet
Photo of r-gordon-7

r-gordon-7

  • 9 Posts
  • 0 Reply Likes
A lot of my panoramas which I've gotten to "stitch together" quite effectively using other panorama programs (e.g. using PhotoVista, for one) - though often only after manipulating them using various of the other programs' parameters and adjustments - don't seem to "syth" together correctly (or completely) here...

When photos don't stitch togeter correctly, don't stitch together in the right sequence, or don't stitch together at all, there should be a way to at least manually drag them into the proper sequence (as for instance there is with PhotoVista) to give the program "a shove in the right direction" as to how they should (and can successfully be) sequenced/"synth'd"...

Again, the ability to preview, edit (beore and after posting) or at the very least delete would seem to be essential but seems to be missing...
Photo of gregers

gregers

  • 2 Posts
  • 0 Reply Likes
Option to move freely around in the 3D world without snapping to images, and without the "wild-skew" ;)
Photo of apritch

apritch

  • 1 Post
  • 0 Reply Likes
At the moment it's not very Web 2.0; what I was expecting with this software was the ability to collaborate with other synthers so I could combine my photos with those taken by other people.

Is this feature planned in for the future?
Photo of BadKarma

BadKarma

  • 1 Post
  • 0 Reply Likes
I agree - that's the real power of photosynth.
Photo of iap

iap

  • 2 Posts
  • 0 Reply Likes
Would be nice "ultimate goal" but the processing may need to be done by the server, and then there are the spamming issues.
Photo of rakerman

rakerman

  • 7 Posts
  • 0 Reply Likes
Mac version. Read EXIF-GPS (embedded GPS location information) and find a way to handle the fact that the Synth may cover a range of locations. Provide GeoRSS. Support distributed computing for local creation of large Synths.