EXIF information for Photosynth

  • 1
  • Question
  • Updated 10 years ago
Does the Photosynth synther use EXIF information to reconstruct a scene? When processing images for conversion to a synth, is the EXIF information a part of the upload or processing steps?
Photo of jim cseke

jim cseke

  • 5 Posts
  • 0 Reply Likes
  • frustrated

Posted 10 years ago

  • 1
Photo of Nathanael Lawrence

Nathanael Lawrence

  • 795 Posts
  • 55 Reply Likes
Hi, Jim. I know that you're wanting an official response, but I thought I'd compile what I can tell you about it for you and if someone official comes along, they can elaborate on what I've said and clarify or correct where needed.

The synther will use some of the EXIF data in the JPGs, such as focal length (if it is present) in the reconstruction of the scene. In cases where no metadata is present (such as scans of positive or negative film or prints made from the same), the synther will fall back to pure computer vision.
Other metadata that the synther reads from the EXIF set of tags includes the photographer's name which is used to display in the lower left corner of the viewer next to the license for each individual photo; and the latitude and longitude tags are also used to auto-geotag a synth, but these aren't used during scene reconstruction.

Although these pieces of information are stored inside the original images in their metadata fields, in a synth, they are stored in an external file - the .json file inside of collection.synth.bin, if I am not mistaken. I believe that it is this file that SynthExport is reading to output the camera parameters.

As far as I can remember, the photos, themselves, are stripped of all metadata when they are converted to the Deep Zoom Image format. You can confirm this by going to the %temp%\Photosynther\images folder on your hard drive when you are creating a synth with images that originally contained metadata, copying one of the .dziz files out into another folder, changing the extension to .zip, and looking at the image tiles for the different resolutions of the image in a program that reads EXIF tags. You could alternately use Fiddler or another similar program to monitor your HTTP traffic while you are browsing any online synth and you will be able to get the URL to any tile that loads, so that you can then save the tile(s) and examine it|them for EXIF data in the same fashion.
Back to whether any EXIF data is used during scene reconstruction or not, though, I thought that I'd share some links with you where Photosynth team members or those involved in the Photo Tourism and other Community Photo Collection work at the University of Washington (or other people commenting on this same work) have shared information on the subject. I know I've shared one or two of them with you before, but I wanted to put all I that could quickly find on the subject in one place for you. If you visit the original pages, you can use [Ctrl]+[F] or [F3] to bring up your browser's 'Search this page' field. Type EXIF in and you'll quickly find the appropriate bit. In cases of PDFs, you'll probably need to use your PDF reader's built-in search feature, located in the menu bar directly above the document.
On the Photosynth side of things:

David Gedye:
"...the other piece of information that Photosynth uses is the focal length as reported in the photo's EXIF tag. If you crop then you are simulating a lens with a different focal length, and this too will confuse Photosynth. However, Photosynth is fairly resilient to focal length errors, so it usually works."

Amir Akbarzadeh:
"...Then we are trying to instead compute the sensor width from certain exif tags, which seem to not be provided by your camera.

The reason we do all this is that we use the focal length (provided by the exif) together with the sensor width to calculate an initial estimate of the field of view of the image..."

Resizing should be completely fine. If you crop the imge you are effectively increasing the focal length. (I.e. digital zoom). If the crop is off-center, then you're doing what a pan-tilt lens does...
On the Bundler side of things: (Photosynth does do many little things a bit differently than Bundler, so it isn't exactly confirmation that the synther does the same thing, but Noah Snavely did work on them both.)

Noah Snavely, Steve Seitz, Rick Szeliski:
"...In particular, our SfM approach is similar to that of Brown and Lowe [2005], with several modifications to improve robustness over a variety of data sets. These include ... using focal length information from image EXIF tags...."

The Bundler User's Manual contains switches to tell Bundler to use the focal length information from the photos' EXIF tags:
[directs bundler to use the estimated focal lengths obtained from
the Exif tags for each image]

[constrain the focal length of each camera to be close to the
initial focal length estimate (from Exif tags). This option
adds penalty terms to the bundle adjustment objective function]

Sameer Agarwal, Noah Snavely, Ian Simon, Steve Seitz, Rick Szeliski:
"...On each node, we begin by verifying that the image files are readable, valid images. We then extract the EXIF tags, if present, and record the focal length...

...The matches returned by the approximate nearest neighbor search are then pruned and verified using a RANSAC-based estimation of the fundamental or essential matrix, depending on the availability of focal length information from the EXIF tags..."

From a course at Ohio State on Photo Tourism:
"...The main tasks involved in this system are:

ยท Extract the focal length (f) from EXIF tag file (if existed) for each image. ..."

On a side note, this and your previous question don't really have anything to do with the Seadragon products or Deep Zoom Composer. When you tag your questions with the other products then all the other questions that relate to Seadragon.com, etc. will be pushing your question further down, which is probably not what you want. If you'd like to check on the most recent activity for only topics relating to Photosynth, please use this link.
Photo of jim cseke

jim cseke

  • 5 Posts
  • 0 Reply Likes
Nathanael, thank you for taking the time to gather this information.
Sincerely Jim.
Photo of Nathanael Lawrence

Nathanael Lawrence

  • 795 Posts
  • 55 Reply Likes
It's been my pleasure, sir.

Did anyone ever contact you offline about your first question? I've been wanting to make up a similar reply for it but it's going to take me a lot more work because there are video clips and images to cut together in addition to the text and links to gather.

I'm deep in the middle of a project that is eating up all my time, getting a friend of the family's computer back up and running and I absolutely must finish that before I make a reply for you that is worthy of the question, but I intend to come back to it. Hang in there if you don't hear anything from anyone for the next couple of weeks. I hope you're enjoying the summer while it lasts.