Posts Tagged ‘bundler’

Newer Entries »

Dense point cloud created with PhotoSynth and PMVS2

August 22nd, 2010

In my previous post I have introduced my PhotoSynth ToolKit. The source code is available on my google code under MIT license, you can download it right now : PhotoSynthToolKit2.zip. I have created a video to show you what I’ve managed to do with it:

As you can see in this video I have managed to use PMVS2 with PhotoSynth output.
All the synths used in this video are available on my PhotoSynth account or directly:

Workflow

My PhotoSynth ToolKit is composed of 3 programs:

  • PhotoSynthDownloader: download 0.json + bin files + thumbs
  • PhotoSynth2PMVS: undistort bunch of pictures and write CONTOUR files needed for PMVS2
  • PhotoSynthTileDownloader [optional]: download all pictures of a synth in HD (not relased yet for legal reason, but you can watch a preview video)

Limitations

It seems that my first version doesn’t handle the JSON parsing of all kind of synth very well, I’ll try to post a new version asap. fixed in PhotoSynthToolKit2.zip

PMVS2 for windows is a 32bit applications, so it has a 2Gb memory limits (3Gb if you start windows with the /3Gb options + compile the app with custom flag ?). I haven’t tried yet the 64bit linux version but I have managed to compile a 64bit version of PMVS2. My 64bit version manage to use more than 4Gb of memory for picture loading, but it crashes right after the end of all picture loading. I didn’t investigate that much, it should be my fault too, compiling the dependencies (gsl, pthread, jpeg) wasn’t an easy task.

Anyway, PMVS2 should be used with CMVS but I’m not sure that I can extract enough information from PhotoSynth. Indeed Bundler output is more verbose, you have 2d/3d correspondence + number of matches per images. I think that I can create a vis.dat file using some information stored in the JSON file but it should only speed-up the process, so it doesn’t help that much with the 2Gb limits.

Credits

My PhotoSynth ToolKit is coded in C++ and the source code is available on my google code (MIT license). It is using:

  • Boost.Asio: network request for Soap + file download
  • TinyXml: parsing of soap request
  • JSON Spirit: parsing of PhotoSynth file: “0.json”
  • jpeg: read/write jpeg for radial undistort

Furthermore, part of the code are based on:

Please go to the PhotoSynthToolkit page to get the latest version

Share

26 comments »

Posted in photogrammetry

Tags:

My PhotoSynth ToolKit

August 19th, 2010

I have released a ToolKit for PhotoSynth that permit to create a dense point cloud using PMVS2.
You can download PhotoSynthToolKit1.zip and take a look at the code on my google code.

PhotoSynth sparse point-cloud
11k vertices		 PMVS2 dense point-cloud
230k vertices

I also have created a web app : PhotoSynthTileDownloader that permit to download all pictures of a synth in HD. I didn’t have release it yet because I’m concerned about the legal issue, but you can see that it’s already working by yourself:

I’ll give more information about it in a few day, stay tuned !

Edit: I have removed the worflow graph and moved it on my next post.

Please go to the PhotoSynthToolkit page to get the latest version

Share

36 comments »

Posted in ogre3d, photogrammetry

Tags:

Pose Estimation using SfM point cloud

July 12th, 2010

The idea of this pose estimator is based on PTAM (Parallel Tracking and Mapping). PTAM is capable of tracking in an unknown environment thanks to the mapping done in parallel. But in fact if you want to augment reality, it’s generally because you already know what you are looking at. So, being able to have a tracking working in an unknown environment is not always needed. My idea was simple: instead of doing a mapping in parallel, why not using SFM in a pre-processing step ?

input: point cloud + camera shot output: position and orientation of the camera

So my outdoor tracking algorithm will eventually work like this:

  • pre-processing step
    • generate a point cloud of the outdoor scene you want to track using Bundler
    • create a binary file with a descriptor (Sift/Surf) per vertex of the point cloud
  • in real-time, for each frame N:
    • extract feature using FAST
    • match feature from frame N-1 using 2D patch
    • compute “relative pose” between frame N and N-1
  • in almost real-time, for each “key frame”:
    • extract feature and descriptor
    • match descriptor with those of the point cloud
    • generate 2D/3D correspondence from matches
    • compute “absolute pose” using PnP solver (EPnP)

The tricky part is that absolute pose computation could last several “relative pose” estimation. So once you’ve got the absolute pose you’ll have to compensate the delay by cumulating the previous relative pose…

This is what I’ve got so far:

  • pre-processing step: binary file generated using SiftGPU (planning to move on my GPUSurf implementation) and Bundler (planning to move on Insight3D or implement it myself using sba)
  • relative pose: I don’t have an implementation of the relative pose estimator
  • absolute pose: it’s basically working but needs some improvements:
    • switch feature extraction/matching from Sift to Surf
    • remove unused descriptors to speed-up maching step (by scoring descriptors used as inlier with training data)
    • use another PnP solver (or add ransac to support outliers and have more accurate results)
Share

4 comments »

Posted in augmented reality, ogre3d

Tags:

Structure From Motion Experiment

July 8th, 2010

I have taken a new set of picture of the “Porte Cailhau” in Bordeaux. And I have used one of my tools (BundlerMatcher) to compute image matching using SiftGPU. BundlerMatcher generates a file compatible with Bundler match file. So using BundlerMatcher you can skip the long pre-processing step of feature extraction and image matching and enjoy GPU acceleration!

I have used the “bundle.out” file produced by Bundler to get cameras informations:

  • intrinsic parameters: focal, distorsion
  • extrinsic parameters: position, orientation

With these informations you can see the point cloud through the viewpoint of one of the camera registered by Bundler. I’ve added this feature to my current Ogre3D PlyReader. I also have added a background plane to be able to see the picture taken from this viewpoint. This demo is not available for download right now, but you can still watch the video :

The Ogre3D PlyReader and BundlerMatcher will eventually be added to my SVN. I’m currently busy working on another demo, so stay tuned !

Share

6 comments »

Posted in ogre3d, photogrammetry

Tags: