Pointcloud exporter!

  • Idea
  • Updated 5 years ago
Peter (on the Photosynth team) ran across the following wonderful piece of work: http://binarymillenium.blogspot.com/2...
A number of you have asked about getting pointclouds out of Photosynth, and the clever binarymillenium has reverse-engineered how to parse it out! We should make the following caveats: a) binarymillenium isn't connected with Microsoft in any way (as far as we know :), and b) the format isn't yet documented or supported (we're working on it), so we can't yet guarantee that it'll be stable. But I'm sure there will be readers eager to try this out. (Binarymillenium, I hope you don't mind our reposting your screenshot!)
Photo of Blaise Aguera y Arcas

Blaise Aguera y Arcas, Employee

  • 81 Posts
  • 38 Reply Likes
  • delighted

Posted 7 years ago

  • 45
Photo of Brad Hamilton

Brad Hamilton

  • 1 Post
  • 1 Reply Like
I'm in the mechanical engineering, shipbuilding industry. We are constantly called on to repair damaged existing systems which have no known historical design data. Often our first task is to grab lots of snaps and start measuring. We have experimented with various point cloud laser devices and have some code to decipher point clouds in AutoCAD and (somewhat) intelligently map to 3D part catalogs for model construction. The possibility\ability to take this to a new level of AI with photosynth is exciting.
Photo of

  • 6 Posts
  • 0 Reply Likes
Also could a 3D model export be created, including texture mapping

& a filter to remove people from textures
Photo of

  • 18 Posts
  • 0 Reply Likes
Filter out people sounds great.
Photo of Enzo Lima

Enzo Lima

  • 1 Post
  • 0 Reply Likes
while you're at it, have it make hot coco and bring dead people to life. One thing at a time, dude. They spent ages building this. Give them time to develop.
Photo of Nathanael Lawrence

Nathanael Lawrence

  • 791 Posts
  • 54 Reply Likes
Enzo, almost all the replies in this discussion besides yours and mine here are nearly 3 years old, so there's been time since the above request for the Photosynth team to have built this. ツ

In fact, some of the more tech savvy members of the Photosynth community have gone ahead and done the work of getting workflows up for generating a mesh for their Photosynth point clouds and then projecting the photos back onto the mesh.

Mark Willis (a.k.a. sir_ivar on Photosynth), among others, showed the rest of us mesh generation and posted a tutorial for creating Digital Elevation Models from Photosynth point clouds.

Greg Downing followed up some time later, showing how he projected the photos back onto the mesh to generate a high resolution image which would have been otherwise impossible to take without moving a priceless artifact:

Josh Harle, inspired by Greg's work, created a workflow for 3DStudioMax:

Henri Astre then added Josh's work into his PhotoSynthToolkit:

There are also other photogrammetry apps which have pursued these goals, as well, such as Agisoft's PhotoScan and web services such as Hypr3D and My 3D Scanner, among others.

Meanwhile, Photosynth hasn't had much more serious development put into it besides moving the viewer from Direct3D (fast, but only worked on Windows in Firefox 3 and Internet Explorer) to Silverlight (slower because it couldn't use your computer's graphics card, but it worked in all web browsers on Windows and Intel Macs), the addition of a new imagery type (panoramas from Microsoft ICE, Photoshop, and the mobile Photosynth apps), and the addition of some menu revisions (examples: Overhead View, Highlights, and Geo-Alignment).

I understand your desire to defend the Photosynth team (and, believe me, I'm in their corner too and want to see them succeed!) but I'm more than ready to see some growth in the visual presentation of synths. Members of the Photosynth team and their colleagues in Microsoft Research have shown sneak peeks at new features, ranging from much thicker point clouds to projecting the photos onto the reconstructed geometry (honestly, that goes back to the original Photo Tourism work at University of Washington and Microsoft Research - note from the 5:00 minute mark onward), and moving from photography in one photosynth or panorama to another (this feature is long overdue!), but none of it has been released yet which can be very frustrating.

Thankfully, as I mentioned above, talented members of the Photosynth community have stepped up and provided ways to get some of these features without waiting for the Photosynth team to deliver them.
Photo of BirdDog

BirdDog

  • 16 Posts
  • 0 Reply Likes
Can someone give a quick Wireshark "for idiots" guide to getting hold of the bin file?
Wouldn't mind a python "for idiots" either.!
Photo of BirdDog

BirdDog

  • 16 Posts
  • 0 Reply Likes
Ok, figgured out Wireshark, Python, and Processing. Not so tricky. Thou, I haven't found many points yet. Any ideas for getting all the points from a whole synth?
Photo of GODLIKE

GODLIKE

  • 21 Posts
  • 1 Reply Like
This is a great opportunity for modelers who must rebuild real objects.
Half of the work is done, practically.
Photo of BirdDog

BirdDog

  • 16 Posts
  • 0 Reply Likes
Is the point cloud in the collection.synth.bin file?
If so, can anyone get it out?
Photo of

  • 3 Posts
  • 0 Reply Likes
preferably in a format that is Maya, Max, Autocad compatible.. such as OBJ
Photo of bitplane

bitplane

  • 56 Posts
  • 11 Reply Likes
This is awesome, great to see you guys at Live Labs actually support people hacking the data and are encouraging people to use PhotoSynth in new and creative ways.

Now all we need is someone to write a tool to smooth the cloud by throwing away noisy points, find planes that are members of the cloud, tessellate them, paint the mesh with the photos and upload into Virtual Earth!

Or is that asking too much? ;-)
Photo of Warren

Warren

  • 9 Posts
  • 0 Reply Likes
Asking too much? Free software that can turn flat photos into a pointcloud is like unbelievable magic already! So I think exporting that as a rendered 3D image ought to be easy. Right?
Photo of

  • 1 Post
  • 2 Reply Likes
The ability to export 3D point clouds and camera position/orientation would move Photosynth from a "Interesting Activity" to a "Research Powerhouse" overnight!
Photo of

  • 1 Post
  • 0 Reply Likes
Quoted for agreement.
Photo of CraigD

CraigD

  • 9 Posts
  • 3 Reply Likes
I've just tried to get that Python script running in c#

http://conceptdev.blogspot.com/2009/0...

I'm not sure it's 100% correct, but am happy for feedback or for someone else to finish it off. Or else the team will release the bin file format tomorrow ;)
Photo of manskj

manskj

  • 1 Post
  • 0 Reply Likes
This is really cool! I just used Photosynth on a Sculpture to make a 3D scanner...and wrote a little bit about my experience including Zhang and Lowe ;-)

Great work guys!
Best Regards,
Mans

Posted about it here.
http://isontech.blogspot.com/2009/04/...

Photo of TDN169

TDN169

  • 14 Posts
  • 0 Reply Likes
I'd like to see an easier way to do it- perhaps a dae file?
Photo of wangbadan007

wangbadan007

  • 1 Post
  • 0 Reply Likes
Photo of SEThateredS

SEThatered

  • 1 Post
  • 0 Reply Likes
Wonderfull. Magnificent. I always dreamed about something like this.
Photo of Nathanael Lawrence

Nathanael Lawrence

  • 791 Posts
  • 54 Reply Likes
There are actually easier ways to export Photosynth point clouds now.

Almost a year ago, Photosynth user, Christoph Hausner, gave us his SynthExport app.

Kean Walmsley from Autodesk then wrote an import plugin for AutoCAD 2011 called BrowsePhotosynth.

Henri Astre has given us his PhotosynthToolkit, which not only downloads the Photosynth point cloud, but will also download small versions of the photos and camera positions in a synth to use in conjunction with Yasutaka Furukawa's PMVS2 for dense reconstruction of any synth.

Josh Harle has provided a tutorial on how to use PhotosynthToolkit.

If you'd like to use the full resolution photos from a synth (fair warning: you're going to need a heap of RAM to use them in PMVS2 if you have very many), use Henri's PhotosynthTileDownloader to save them.

Recently, Josh Harle has modified Christoph's SynthExport to export the camera positions to use your photos with 3D Studio Max. His fork of SynthExport is named CameraExport. His results are very similar to the work that Greg Downing from xRez studios demonstrated near the end of November.

Most recently, the upcoming version of Meshlab (version 1.3.0) will support direct import of any public photosynth as well as its full resolution photos.