Posts Tagged ‘sfmtoolkit’

New bundler version

May 26th, 2012

I’ve compiled a new version of Bundler with 2 interesting new options:

--parallel_epipolar
--ba sba

BTW those options should be passed as command arguments to bundler.exe directly (and not been added to the options.txt file).

The parallel_epipolar option allows to estimate the fundamental matrices (ransac + lm) in parallel. This problem is embarrassingly parallel but due to some global variables used in the callback passed to lmfit it wasn’t that easy to implement: I’ve used a functor to hide the global variables used by the callback but sadly lmfit doesn’t have a void* userData parameter (only a pointer to the callback). Thus I’ve modified lmfit and add this missing parameter that allow me to pass a pointer to my functor. Furthermore I had to compile almost everything in C++ instead of C to use my functor inside lmfit. Thus I had to fix a lot of malloc calls that weren’t compiling in C++ due to missing cast. To keep a “backward behavior compatibility” this option is disabled by default.

The ba option allows to change the bundle adjustment “engine” used. Here is the list of available “engine”:

  • sba (default)
  • none (for debug only)
  • pba_cpu_double
  • pba_cpu_float
  • pba_gpu_float

Pba stand for Parallel Bundle Adjustment: I’ve integrated mcba from Changchang Wu.

So if you have an Nvidia GPU card and installed the Cuda runtime you can add those options:

bundler.exe list_focal_absolute.txt --ba pba_gpu_float --parallel_epipolar
--options_file options.txt //on the same line

Timing on a 245 pictures dataset:

Bundler BA Time Nb pictures registered
SBA 2h18min 233
PBA CPU double 23min 230
PBA CPU float 9min 230
PBA GPU float 6min 230
none (for debug) 2min 189 (bad reconstruction)

You can download this new version of bundler: bundler-multiBA-parallelEpipolar-x64.zip
Update: the source code is available on the MCBA branch of my Bundler fork.

Share

Moving on

September 26th, 2011

I’ve spent almost 4 years at axyz.fr but it was time for me to move on. I’ve quit my job and I’ve started to work at Acute3D in September: they are working in the structure from motion and dense 3d reconstruction field. You can try their dense mesh creation technology by using Autodesk Photofly. Photofly is calibrating the cameras using Autodesk implementation and then the mesh is generated using Acute3D technology (Autodesk has bought a license of their meshing technology).

I’m working on the development of a calibration system (like bundler) at Acute3D. I’ve decided to work with them as I think that they do have the best technology available and I wanted to work with very skilled people working in computer vision.

I don’t know if I’ll be able to continue writing on this blog as this new job is really more interesting and challenging for me.

Share

News about OpenSynther

May 9th, 2011

I’ve worked a lot on OpenSynther lately: OpenSynther is the name of my structure-from-motion solution. This new version is a major rewrite of the previous version which was using Surf with both GPU and multi-core CPU matching. The new version is using SiftGPU and Flann to achieve linear matching complexity of unstructured input as described in Samantha paper. You can find more information about OpenSynther features on it dedicated page (including source code).

OpenSynther has been designed as a library (OpenSyntherLib) which has already proven to be useful for several programs written by myself:

  • OpenSynther: work in progress… used by my augmented reality demo
  • PhotoSynth2CMVS: this allow to use CMVS with PhotoSynthToolkit
  • BundlerMatcher: this is the matching solution used by SFMToolkit

Outdoor augmented reality demo using OpenSynther

I’ve improved my first attempt of outdoor augmented reality: I’m now relying on PhotoSynth capability of creating a point cloud of the scene instead of Bundler. Then I’m doing some processing with OpenSynther and here is what what you get:



You can also take a look at the 3 others youtube videos showing this tracking in action around this church: MVI_6380.avi, MVI_6381.avi, MVI_6382.avi.

PhotoSynth2CMVS

This is not ready yet, I still have some stuff to fix before releasing it. But I’m already producing a valid “bundle.out” file compatible with CMVS processing from PhotoSynth. I’ve processed the V3D dataset with PhotoSynth2CMVS and sent the bundle.out file to Olafur Haraldsson who has managed to create the corresponding 36 million vertices point cloud using CMVS and PMVS2:

The V3D dataset was created by Christopher Zach.

BundlerMatcher

The new unstructured linear matching is really fast as you can see on the above chart compared to PhotoSynth. But the quality of the generated point cloud is not as good as PhotoSynth.




This benchmark was computed on a Core i7 with an Nvidia 470 GTX. I’ve also compared the quality of the matching methods implemented in OpenSynther (linear VS quadratic). I’ve used Bundler as a comparator with a dataset of 245 pictures:

Linear Quadratic
Nb pictures registered 193 243
Time spent to register 193 pictures 33min 1h43min

On the one hand, both the matching and the bundle adjustment are faster with linear matching but on the other hand, having only 193 out of 245 pictures registered is not acceptable. I have some idea on how to improve the linear matching pictures registering ratio but this is not implemented yet (this is why PhotoSynth2CMVS is not released for now).

Future

I’ve been playing with LDAHash last week and I’d like to support this in OpenSynther to improve matching speed and accuracy. It would also help to reduce the memory used by OpenSynther (by a factor 16: 128 floats -> 256bits per feature). I’m also wondering if the Cuda knn implementation could speed-up the matching (if applicable)? I ‘d also like to restore the previous Surf version of OpenSynther which was really fun to implement. Adding a sequential bundle adjustment (as in bundler) would be really interesting too…

Off-topic

I’ve made some modifications to my blog: switched to WordPress 3.x, activated page caching, added social sharing buttons and added my LinkedIn account next to the donate button…

Share

3D-Arch Workshop

March 31st, 2011


The 3D-Arch workshop was really inspiring: it was really great meeting people working on the same subject, exchanging idea… and the place was really nice too. I couldn’t resist to create some PhotoSynth of the place for future reconstruction: dragon, eagle, statue, door, …

I should have published this post sooner but I wanted to make a double post with my new OpenSynther results… CMVS support in PhotoSynthToolkit is coming! You should expect another post next week with nice results ;-)

Share

New toolkits released

February 22nd, 2011

V3DSfMToolkit

ETH-V3D Structure-and-Motion software was created by Christopher Zach. The original source code with dataset is available at Christopher Zach Open-Source page (GPL license). I have created a windows port of V3DSfMToolkit with scripting wich is available as both binary (V3dSfMToolkit1.zip) and source (github).


I’ve tested this toolkit with the dataset given by Christopher Zach (see above screenshot) the reconstruction looks good but I only managed to get partial reconstruction from my own dataset.

MissStereo

Quasi-Euclidean Epipolar Rectification: MissStereo created by Pascal Monasse, Neus Sabater, Zhongwei Tang. The original source code is available at the IPOL related page under GPL license. You can download my windows port as both binary (MissStereo1.zip) and source (github).

I’m interested by this method to estimate the fundamental matrix without prior focal length knowledge.

PhotoSynthToolkit with XSI support

With the help of Julien Carmagnac (3D Graphist and XSI advanced user), I’ve duplicated the 3DS Max texture projection rendering solution for Softimage XSI:


As usual, the new version of PhotoSynthToolkit including this new feature is available on his dedicated page.

SFMToolkit with sequence matching

SFMToolkit was packaged with BundlerMatcher, my own matching solution using SiftGPU: this is a trivial quadratic exhaustive matching implementation. This implementation is well-suited for unstructured (unordered) pictures but if you are using a sequence of images as input (movie) you can reduce the complexity of the matching to linear. You can do that by only comparing frame N with frame N+1, N+2 [...], N+p. I’ve also fixed a bug that may occur on some system (Windows 7 x64 with UAC activated): thanks to Josh Harle for the bug reporting!

The new version of SFMToolkit is available on his dedicated page: this new feature is hidden, you need to edit “1 – Bundler.wsf” and set SEQUENCE_MATCHING_ENABLED to true (replace false by true).

3D-Arch’2011

I’m going to the 3D-Arch’2011 Workshop at Trento: 3D Virtual Reconstruction and Visualization of Complex Architectures. I hope to see amazing things about 3D reconstruction ;-)


Jean-Philippe Pons (CSTB, Sophia-Antipolis, France): High-resolution large-scale multi-view stereo

Share