In my previous post I have introduced my PhotoSynth ToolKit. The source code is available on my google code under MIT license, you can download it right now : PhotoSynthToolKit2.zip. I have created a video to show you what I’ve managed to do with it:
As you can see in this video I have managed to use PMVS2 with PhotoSynth output.
All the synths used in this video are available on my PhotoSynth account or directly:
- Porte Cailhau: http://photosynth.net/view.aspx?cid=1e509490-5657-453d-a2f6-2e55d14ae512
- Goutz: http://photosynth.net/view.aspx?cid=1471c7c7-da12-4859-9289-a2e6d2129319
- Place de la Bourse: http://photosynth.net/view.aspx?cid=e82eca65-60fe-498b-8916-80d1e3245640
Workflow
My PhotoSynth ToolKit is composed of 3 programs:
- PhotoSynthDownloader: download 0.json + bin files + thumbs
- PhotoSynth2PMVS: undistort bunch of pictures and write CONTOUR files needed for PMVS2
- PhotoSynthTileDownloader [optional]: download all pictures of a synth in HD (not relased yet for legal reason, but you can watch a preview video)
Limitations
It seems that my first version doesn’t handle the JSON parsing of all kind of synth very well, I’ll try to post a new version asap. fixed in PhotoSynthToolKit2.zip
PMVS2 for windows is a 32bit applications, so it has a 2Gb memory limits (3Gb if you start windows with the /3Gb options + compile the app with custom flag ?). I haven’t tried yet the 64bit linux version but I have managed to compile a 64bit version of PMVS2. My 64bit version manage to use more than 4Gb of memory for picture loading, but it crashes right after the end of all picture loading. I didn’t investigate that much, it should be my fault too, compiling the dependencies (gsl, pthread, jpeg) wasn’t an easy task.
Anyway, PMVS2 should be used with CMVS but I’m not sure that I can extract enough information from PhotoSynth. Indeed Bundler output is more verbose, you have 2d/3d correspondence + number of matches per images. I think that I can create a vis.dat file using some information stored in the JSON file but it should only speed-up the process, so it doesn’t help that much with the 2Gb limits.
Credits
My PhotoSynth ToolKit is coded in C++ and the source code is available on my google code (MIT license). It is using:
- Boost.Asio: network request for Soap + file download
- TinyXml: parsing of soap request
- JSON Spirit: parsing of PhotoSynth file: “0.json”
- jpeg: read/write jpeg for radial undistort
Furthermore, part of the code are based on:
- Bundler: RadialUndistort + Bundler2PMVS
- SynthExport: C# binary loader
Please go to the PhotoSynthToolkit page to get the latest version
Great work I have no problem working with your examples, however my own synths come up with an error of Value type is 6 not 1… any thought on this, am I making my synths wrong??
http://photosynth.net/view.aspx?cid=d8bbab31-72a5-46de-90b8-314c0bc23e96
is the synth I am working with.
Hi Astre,
I was successful with the ToolKit yesterday and had a decent result of a house. However, I really had to reduce the images down to a low resolution in order for this to work.
I was told to try CMVS so that I could then use the high res images without much problem. So, now I have a Synth with 65 images, but I can’t figure out how to get CMVS to run. It keeps looking for a buld.rd.out file.
Can you help me out?
@John Ellis: I have fixed this bug but not yet uploaded the new version, it should be available tomorrow.
@Eugene: CMVS can’t be used used with PhotoSynth because some information given by Bundler output are not available in the binary produced by PhotoSynth. I’m working on 2 others options:
Astre,
Since you are doing dense matching, does this mean we can in fact reduce our photo sets and still get a decent result with less photos? For example, if I have a photo set of 12 high res images that is only 60% synthy, but the images have good overlap, can I still get a good point cloud from it?
I would be interested in any tips you might have that would help the matching (e.g. stereo pairs of photos? High overlap? More photos?)
@Eugene: I’m not an expert of PMVS2, I’m originally interested in PhotoSynth because I’m hacking Bundler code to create a new outdoor tracking algorithm for augmented reality. Anyway, if your synth is not 100% synthy this mean that you have multiple coord systems (pack of pictures in the same coord system) and I’m only using the first coord system with PMVS2. The number of picture of each coord system is printed at the end of step 1 of my ToolKit. PMVS2 needs to know the position and orientation of the camera in the coord system. So if your pictures are not all registered in the same coord system, the reconstruction won’t use all input images (only those of coord system 0).
@all: I’ve uploaded the new version of the ToolKit : PhotoSynthToolKit2.zip (including the fix for error : Value type is 6 not 1)
Bundler Matcher does not support multi-core computing, and slower than photosynth match, but can get the Bundler Out file, for CMVS.
Look forward to your success:)
@spy: I already have replaced the slow matching step packaged with Bundler by my “BundlerMatcher” using SiftGPU. I’m currently trying to get rid of the .key file access in bundler to keep everything in memory as PhotoSynth does. But I know that I’ll never get a faster or more reliable result… I just need 2d/3d correspondence for my tracking algorithm (and for other people who want CMVS)
Astre,
How soon do you think you will have the 64bit version of PMVS2 for Win available?
Your work looks really interesting, I have a question about the SiftGPU and how it might be used in different applications. Is there a way I can reach you or contact me at eliscio (at) hotmail.com?
@Eugene: I have manage to compile a 64 bit version of PMVS2 but I must have done something wrong because it’s always crashing right after dog computation (I think that one of the dependencies didn’t compile that well in 64bit). My email is visible in my blog menu: About->Contact.
Hi Astre,
You, my good sir, are a genius. I have been looking for a simple way to compile a 3d model from photosynth, but every attempt ended in utter failure. I thank you.
Two questions; first, is it possible to get an e-mail notification for when you finish the 64-bit PMVS2? My adress is
removed by admin. Secondly, will you be working on a program that could do all of these processes automatically?@Kjellis85: Thanks! I don’t have an automatic e-mail notification system, sorry… (BTW I have removed your email from your comment to protect you from spam). I’m currently busy with Bundler code, so once I got something working I’ll take a deeper look inside PMVS2.
Concerning your request for a unique program, the code is open-source (MIT license) so you can do almost whatever you want with it
Or ask Christoph Hausner who wrote SynthExport to add contour export using my code… I’m not really comfortable with C# so I can’t wrote a nice GUI for now.
Astre,
Yes, I realized afterwards your contact address was on the page here. I have sent you an email.
Keep up the good work!
Dose any one know a good tutorial for using builder PMVS2 CMVS tutorials , I am knew to programing and cant manage to find exactly the right way to go through the process
I used to play around with the “Wireshark” method a few months ago, seems there is progress thank to your toolkit.
Tried it yesterday but apparently it wouldn’t
want to connect to the Photosynth server…
Tried to change timeout settings with no success (yet)
Is it still working by now (09/22) or did they
change anything on their side ?
Greetz and thmbs up for your good job
@Bernd: it still works fine for me. FYI I’m not the one you succeeded to replace the “Wireshark” method. It was first implemented in SynthExport (in C#), then I’ve implemented a C++ version and managed to convert PhotoSynth camera’s parameters to Bundler format in order to use PMVS2.
Thx for your reply,so apparently this seems more a problem of synthexport reaching the Photosynth server.
First there were the timeouts and when I increased the timeout settings it just can’t reach the server.
Hmmm – managed to get one synth file immediately after closing Firefox
Worked once ( cloud + cameras )
Couldn’t reproduce this unfortunately…
Any tips or tricks do see how this affects requesting from the server ?
@Bernd: Could you be more precise please ? Are you talking of SynthExport or PhotoSynthDownloader ? You could also provide a link to a synth that doesn’t work for you… Otherwise It’ll be very hard for me to help you. If you are referring to SynthExport, I’ll suggest you to use the SynthExport discussions forum.
Thx for your quick reply.
I was talking about SynthExport – sorry – didn’t
mention
Going to try on another box.Made myself a little “notepad” file with links to the “archeology”
synths ( best pointclouds apparently )
and shoot up synth on a fresh box,fresh win install.
I recently tried your program on a Windows 64-bit and it quit after the dog computation as you noted in the comments. Would trying something with CMVS or just keep using my old 32-bit mac vmware?
I was able to do a low-resolution (thumbnail) version of a photosynth I put together. It was pretty amazing and the colors were much more photorealistic than when I used SynthExport or ARC3D. Thanks for putting this together!
I’m trying anything I can to get a little higher-resolution model together (hence the 64-bit attempt) so please let me know if you have any advice.
@Dave: you need to download the 64 bit version of PMVS2 (pmvs2_x64.zip) and replace it in your Photosynth Toolkit. It’ll work with a window 64-bit but not in a 32-bit vmware. But depending on your pmvs2 options you’ll need a lot of ram (8gig or more). The other option is to use my BundlerMatcher + Bundler and use CMVS to cluster the job in reasonable size (take a look at the second video created by josh harle).
Henri, I’m still experiencing the JSON parsing error with PhotoSynthToolkit2.
Here are two synths that trigger that error message every time for me:
Kelvingrove Art Gallery, Glasgow
Empire State Building Aerial Synth
I’m interested in seeing how your workflow compares against the dense reconstructions viewable in the video presentations of Blaise’s. PhotoSynthTileDownloader downloads the photos just fine, but the toolkit has problems when fetching the binary files.
@Nate Lawrence: I’ve updated both source code and binary (PhotoSynthToolKit3.zip), thanks for finding bugs !
Using the thumbs version I’ve managed to produce a 289k point cloud instead of 168k for Kelvingrove Art Gallery, Glasgow.
The big issue is the memory used, as we can’t use cmvs to create cluster…
Henri, just wanted to say, “Thank You” for all of your hard work. ?
It’s very encouraging to write to someone with problems and see them addressed and fixed so quickly.