Manual hinting

  • 32
  • Idea
  • Updated 12 years ago
The automatic point correspondence and camera pose estimation is excellent, but where it fails to properly link a pair of images I would love to be able to input manual hints. Even if this were limited to specifying a set of coplanar points in a pair of images, it would allow a link between otherwise separated 3D groups.
Photo of panovoyant

panovoyant

  • 4 Posts
  • 0 Reply Likes
  • undecided

Posted 12 years ago

  • 32
Photo of Warren

Warren

  • 1 Post
  • 0 Reply Likes
I have done panoramic pics b4 and when put in proper seqence they turn out great. We should b allowed 2 set the pics in proper order. Afterall, we know the true seqence.
Photo of raydancer

raydancer

  • 1 Post
  • 0 Reply Likes
I have done a lot of panoramas in the past, but have always used AutoStitch to put them together. I, too, wouldn't mind hinting at which pictures should be stitched together...at least until there is no longer a need for it.
Photo of haydenhibbard

haydenhibbard

  • 8 Posts
  • 2 Reply Likes
I like this idea, too. The way I imagined it is like this:

In the photo uploader, the user could mark individual photos using a toolset included in the photosynther. These marks would be visible before the upload in the program, but not afterwards once the synth shows up on the site.

For elements that might confuse the photosynther, you could 'highlight' areas that it should NOT consider (i.e. mirrors, water, pets, etc.) for figuring out where a photo was taken and what it is of.

For elements that might help the photosynther figure things out, there are several different options. For example, you could draw a colored/labeled line tracing a the same corner in multiple independent photographs, which would help the program know that the line labeled X was the same corner across multiple photos.

Hopefully this makes as much sense to some one else as it does to me.
Photo of VictorEngel

VictorEngel

  • 4 Posts
  • 0 Reply Likes
I agree completely. I just uploaded my first photosynth designed for the purpose, expecting a high synth number. It turned out to be only 12% because most of the pictures weren't matched with each other. I'm interested to know what aspect of the photos causes this.
http://photosynth.net/view.aspx?cid=2...
Photo of alan

alan

  • 40 Posts
  • 2 Reply Likes
I rather like the way syth is automatic and doesn't require manual alignment. I see this as a plus point. The algorithm will be perfected over time.
Photo of FJVarro

FJVarro

  • 1 Post
  • 0 Reply Likes
I would basically like it to autostitch by default, but in the case of my first stitch, let me then go in and say "Hey, dumbass software, that row of shrubs there is the same on in this image, seen from around this direction" so then it can take my 20 sets of 5 images and make 1 set of images. It would also allow for more gaps in photographic coverage.
Photo of cmars_artist

cmars_artist

  • 3 Posts
  • 0 Reply Likes
The "gaps in photographic coverage.. pictures not matched with each other"
These issues could be fixed by letting the user manually connect 2 or more syth halves together...Lets say there is 360 view of a scene but it didn't quite connect all the photos sets, the user could help fill in the gaps by manually numbering or visually stacking the images according to right, left, up, down... so at least when viewers navigate in the UI they can jump to the next closest photo set in some kind of logical order.
Photo of Johan Benesch

Johan Benesch

  • 15 Posts
  • 6 Reply Likes
sounds like a great idea!

the manual part could be make easier than finding specific points in an orhpanhed image:
- drag one photo from one "orphan synth" to one specific image in another "orphan synth".
- then the "synther" would take it from there.

see here for example
http://photosynth.net/view.aspx?cid=6...

217 images
16 "orphan synths" and 17 "orphan images"
Photo of Johan Benesch

Johan Benesch

  • 15 Posts
  • 6 Reply Likes
i deleted the above synth and replaced with this one
http://photosynth.net/view.aspx?cid=f...
285 pics 22% synth, 31 "orhapan synths" 25 orphan images
i added ca 60 more pictures of the 4 corners of this building,
it just got worse ...
Photo of alanmd

alanmd

  • 2 Posts
  • 0 Reply Likes
Some sort of manual override/hint is essential for Photosynth to be more than just a toy.
Photo of wormer

wormer

  • 3 Posts
  • 0 Reply Likes
I really think there needs to be some form of manual matching, I have been trying to get my first 'real' synth set up, 1st time it came up 40%, so I removed a few orphan photos, tried again, got 49%, and a much better synth than the first time, but then I removed a couple more orphan photos, uploaded again, and its down to 30% synthy now, and even less 'connected' than the first two attempts! I tried a 4th time with exactly the same photos, and it came out at 29%, then a 5th time (same photos again) and its at 42%! I don't expect it to be the same each time, its just frustrating to see previously matched photos becoming orphaned at the next attempt, when it should have been a better success rate. Very cool tool though, I have to admit that!
Photo of Nathanael Lawrence

Nathanael Lawrence

  • 795 Posts
  • 55 Reply Likes
Wormer, as to your wondering at differing results from the same photos, the order that you select the photos in when you place them in the synther can affect your results. If you'd like more info, try "Same synth input with different results?".
Photo of wormer

wormer

  • 3 Posts
  • 0 Reply Likes
Interesting, as per that post I simply selected the entire directory all three times, so the order should be the same. Ill have a look and see if I can add them in a more logical order, and see if it makes a difference. Thanks for the link tho!
Photo of markp

markp

  • 47 Posts
  • 3 Reply Likes
maybe there's some kind of random generator / fuzzy logic thing going on behind the scenes? or more cleverly it's even noticed that you've retried the thing and assumed something was wrong first time, so tried an alternate matching algorithm.

or to borrow an idea from evolutionary circuit design experiments, perhaps those "orphan" photos interact with the rest of them in some non-obvious way behind the scenes while it's reconstructing everything, and although they don't appear in the final synth they're essential to having everything else stitch in properly...

(the evolutionary circuit thing comes from reading an article on early experiments with such... guy took an all-purpose 10x10 transistor re-programmable processor and had a connected PC start from a random point and change the chip's design program in an "evolutionary"/"natural selection" fashion with the goal of being able to determine between a 100hz and 1000hz input signal with no reference clock... the final design was very very strange, including some transistors that weren't even connected to the main circuit path - but if disconnected or rearranged, the whole thing fell apart; it was theorised that they had some kind of inductive/resonant influence on the rest of the chip when arranged just-so!)
Photo of markp

markp

  • 47 Posts
  • 3 Reply Likes
Good idea... though it should be something you activate by choice... IE operate the program most of the time in full automatic mode, but have the option of stepping down through progressive stages of manual intervention and correction for difficult stuff.
After all the human eye and brain themselves have a certain susceptibility to optical illusions even with the benefit of stereoscopic, full-motion vision and years of experience. It's not unreasonable to expect that a comparitively simple computer algorithm operating on a pile of static 2D images with no prior experience or knowledge of the layout or context can have occasional hiccups. I'm actually damn impressed it works anywhere near as well as it does!

But I have had one recently where I couldn't see any reason that the shots wouldn't stitch together 100%, but it's had some strange brainfart that means a load of images from near the bottom of a slight hill have ended up conceptually floating in space 20ft above those at the top of the hill & in the wrong direction. Still all synthed with each other, just not connecting to the top area properly and the point cloud makes it pretty obvious. I think a few minutes telling it just for a few crucial pics "that small shape on the horizon is actually the same building that's a gradually larger and more recognisable structure in all the other images, and that certain bit of road also appears a bit further down the frame in them" could potentially have solved it all. It already took long enough to take the pics, reduce them down to a more suitable size with Office Picture Manager once I saw how massive a data load it was (& rotate the vertical ones), then do the synth itself - another quarter hour hinting up those that are likely to be more difficult (i.e. usually not needed for a typical panorama, unless you notice it's dropped one picture for no good reason) isn't really here or there.
Photo of markp

markp

  • 47 Posts
  • 3 Reply Likes
Then again, having just pored over the log of a failed one, the main synth (with 400-ish pics .... and more than 300 semi-orphans in 90 groups) has over 60,000 points in it.

So adding manual hints would maybe only have a very subtle effect and only be useful to force one or two recalcitrant photos to finally stitch in, rather than something you'd do for the whole scene, as it could take hours if not days!
Photo of mrb

mrb

  • 1 Post
  • 1 Reply Like
"So adding manual hints would maybe only have a very subtle effect"

Although there are a lot of points in the log, I guess there's a much smaller number of matched-point pairs? And you could imagine a manually added pair would be given a higher priority than the matches found by photosynth's algorithm.

So hopefully a small number of manual hints would be enough to set the synth back on track if the automatic matching gets a bit confused.
Photo of Johan Benesch

Johan Benesch

  • 15 Posts
  • 6 Reply Likes
and if you do manual hinting of one pic each in two orphan synths the rest should by default be synthed automatically, since already are coupled via their own synth matches.