Posts Tagged ‘opensynther’

New bundler version

May 26th, 2012

I’ve compiled a new version of Bundler with 2 interesting new options:

--parallel_epipolar
--ba sba

BTW those options should be passed as command arguments to bundler.exe directly (and not been added to the options.txt file).

The parallel_epipolar option allows to estimate the fundamental matrices (ransac + lm) in parallel. This problem is embarrassingly parallel but due to some global variables used in the callback passed to lmfit it wasn’t that easy to implement: I’ve used a functor to hide the global variables used by the callback but sadly lmfit doesn’t have a void* userData parameter (only a pointer to the callback). Thus I’ve modified lmfit and add this missing parameter that allow me to pass a pointer to my functor. Furthermore I had to compile almost everything in C++ instead of C to use my functor inside lmfit. Thus I had to fix a lot of malloc calls that weren’t compiling in C++ due to missing cast. To keep a “backward behavior compatibility” this option is disabled by default.

The ba option allows to change the bundle adjustment “engine” used. Here is the list of available “engine”:

  • sba (default)
  • none (for debug only)
  • pba_cpu_double
  • pba_cpu_float
  • pba_gpu_float

Pba stand for Parallel Bundle Adjustment: I’ve integrated mcba from Changchang Wu.

So if you have an Nvidia GPU card and installed the Cuda runtime you can add those options:

bundler.exe list_focal_absolute.txt --ba pba_gpu_float --parallel_epipolar
--options_file options.txt //on the same line

Timing on a 245 pictures dataset:

Bundler BA Time Nb pictures registered
SBA 2h18min 233
PBA CPU double 23min 230
PBA CPU float 9min 230
PBA GPU float 6min 230
none (for debug) 2min 189 (bad reconstruction)

You can download this new version of bundler: bundler-multiBA-parallelEpipolar-x64.zip
Update: the source code is available on the MCBA branch of my Bundler fork.

Share

News about OpenSynther

May 9th, 2011

I’ve worked a lot on OpenSynther lately: OpenSynther is the name of my structure-from-motion solution. This new version is a major rewrite of the previous version which was using Surf with both GPU and multi-core CPU matching. The new version is using SiftGPU and Flann to achieve linear matching complexity of unstructured input as described in Samantha paper. You can find more information about OpenSynther features on it dedicated page (including source code).

OpenSynther has been designed as a library (OpenSyntherLib) which has already proven to be useful for several programs written by myself:

  • OpenSynther: work in progress… used by my augmented reality demo
  • PhotoSynth2CMVS: this allow to use CMVS with PhotoSynthToolkit
  • BundlerMatcher: this is the matching solution used by SFMToolkit

Outdoor augmented reality demo using OpenSynther

I’ve improved my first attempt of outdoor augmented reality: I’m now relying on PhotoSynth capability of creating a point cloud of the scene instead of Bundler. Then I’m doing some processing with OpenSynther and here is what what you get:



You can also take a look at the 3 others youtube videos showing this tracking in action around this church: MVI_6380.avi, MVI_6381.avi, MVI_6382.avi.

PhotoSynth2CMVS

This is not ready yet, I still have some stuff to fix before releasing it. But I’m already producing a valid “bundle.out” file compatible with CMVS processing from PhotoSynth. I’ve processed the V3D dataset with PhotoSynth2CMVS and sent the bundle.out file to Olafur Haraldsson who has managed to create the corresponding 36 million vertices point cloud using CMVS and PMVS2:

The V3D dataset was created by Christopher Zach.

BundlerMatcher

The new unstructured linear matching is really fast as you can see on the above chart compared to PhotoSynth. But the quality of the generated point cloud is not as good as PhotoSynth.




This benchmark was computed on a Core i7 with an Nvidia 470 GTX. I’ve also compared the quality of the matching methods implemented in OpenSynther (linear VS quadratic). I’ve used Bundler as a comparator with a dataset of 245 pictures:

Linear Quadratic
Nb pictures registered 193 243
Time spent to register 193 pictures 33min 1h43min

On the one hand, both the matching and the bundle adjustment are faster with linear matching but on the other hand, having only 193 out of 245 pictures registered is not acceptable. I have some idea on how to improve the linear matching pictures registering ratio but this is not implemented yet (this is why PhotoSynth2CMVS is not released for now).

Future

I’ve been playing with LDAHash last week and I’d like to support this in OpenSynther to improve matching speed and accuracy. It would also help to reduce the memory used by OpenSynther (by a factor 16: 128 floats -> 256bits per feature). I’m also wondering if the Cuda knn implementation could speed-up the matching (if applicable)? I ‘d also like to restore the previous Surf version of OpenSynther which was really fun to implement. Adding a sequential bundle adjustment (as in bundler) would be really interesting too…

Off-topic

I’ve made some modifications to my blog: switched to WordPress 3.x, activated page caching, added social sharing buttons and added my LinkedIn account next to the donate button…

Share

3D-Arch Workshop

March 31st, 2011


The 3D-Arch workshop was really inspiring: it was really great meeting people working on the same subject, exchanging idea… and the place was really nice too. I couldn’t resist to create some PhotoSynth of the place for future reconstruction: dragon, eagle, statue, door, …

I should have published this post sooner but I wanted to make a double post with my new OpenSynther results… CMVS support in PhotoSynthToolkit is coming! You should expect another post next week with nice results ;-)

Share

Introducing OpenSynther

September 8th, 2010

PhotoSynthTileDownloader

In my previous post I have released my PhotoSynth Tookit but the PhotoSynth tile downloader wasn’t available yet. You can now download the picture of your Synth in HD using PhotoSynthTileDownloader1.zip. But please respect author’s copyright ! I have included a new confirmation dialog box that warn you about the Synth status (unlisted, public) but this is not shown in the video presentation of PhotoSynthTileDownloader.

BundlerMatcher

I have finally released BundlerMatcher, a feature extraction and picture matching tool built with SiftGPU. The main goal of this tool was to replace the slow matching step packaged with Bundler by a faster one using GPU without needing to modify Bundler’s code. You can download BundlerMatcher1.zip or checkout the code on my google code. Warning: this tool needs a 64bit windows OS (tested on 7 and vista) and a decent GPU. All my demos are available under MIT license but SiftGPU isn’t released under MIT so you should take a look at SiftGPU license.

OpenSynther

I’m proud to introduce OpenSynther which is under heavy development. The first goal of this tool is the same as BundlerMatcher: provide a faster matching engine for Structure from Motion tools. To achieve high performance it is coded in C++ and using both multi-core and GPU. Furthermore OpenSynther is using Surf instead of Sift and may in the future also compute the 3d reconstruction (as PhotoSynth does).

OpenSynther current feature list:

  • Jpeg loading + Exif reading
  • multi-threaded Surf feature extraction (based on OpenSurf & ParallelSurf)
  • multi-threaded Surf feature matching (based on OpenSurf matching + Cuda GPU matching)
    • Quadratic complexity O(n²) with n number of pictures

OpenSynther TODO:

  • multi-threaded Surf feature approximate matching (based on FLANN)
    • Linear complexity (hypothetical)
  • integrate 3d reconstruction ?

Benchmarks

These benchmarks were done on an Intel Corei7 920 (8 cores @ 2.66Ghz) + Nvidia GTX 285. The pictures size were 1600×1200 (~2M pixels), I heard that it was the limit size at which PhotoSynth extract features (it can of course use bigger pictures but feature extraction are done at a max size of 2M pixels).

Note:

  • Bundler: I had to reduce picture to 640×480, otherwise I got too many features (would be unfair).
  • PhotoSynth: I didn’t take into account the time spent in tile creation.
  • OpenSynther: in the future I may consider using GPUSurf for feature extraction.
  • BundlerMatcher: to be fair with PhotoSynth I didn’t write the ascii key file (so in fact it’s slower).


Note:

  • PhotoSynth: they did a very good job and managed to have a linear matching !
  • OpenSynther: the GPU is in fact doing the most part of the job as you can see in the table below:
App Name PhotoSynth OpenSynther BundlerMatcher Bundler
Extraction in sec 10 22 39 21 40 78 7 12 24 105 220
Matching in sec 15 31 58 31 104 322 40 170 542 75 440
% matching
done by CPU
100 100 100 22 12 13 0 0 0 100 100
% matching
done by GPU
0 0 0 78 88 87 100 100 100 0 0
Nb pictures 49 98 196 49 98 196 49 98 196 49 98 196



As you can see OpenSynther matching is in fact done by the GPU at 85%.

Share