I have released a ToolKit for PhotoSynth that permit to create a dense point cloud using PMVS2.
You can download PhotoSynthToolKit1.zip and take a look at the code on my google code.
 |
|
| PhotoSynth sparse point-cloud 11k vertices |
PMVS2 dense point-cloud 230k vertices |
I also have created a web app : PhotoSynthTileDownloader that permit to download all pictures of a synth in HD. I didn’t have release it yet because I’m concerned about the legal issue, but you can see that it’s already working by yourself:
I’ll give more information about it in a few day, stay tuned !
Edit: I have removed the worflow graph and moved it on my next post.
Please go to the PhotoSynthToolkit page to get the latest version
This looks amazing, Astre!
I was just pointed over here by Josh Harle from a thread I’d started on Christoph Hausner’s SynthExport Codeplex page.
I’ve started to try the toolkit with two of my synths. The first encountered an error.
Here’s the synth: http://photosynth.net/view.aspx?cid=afaf7f93-898b-4a54-8594-4d0dd588e3e2
And the error message is:
“value type is 6 not 1″
Also, on the second synth ( http://photosynth.net/view.aspx?cid=df4722d5-e640-42c3-b8bb-4338ac62acdb which I was eventually able to successfully work through your process), I experienced a failure at Step 4 when using my original photos with their longer filenames. I then tried again, using the thumbs (this was what led to the success).
After having completed the process once successfully using the thumbs, I was curious why the original photos had not worked – whether this was due to their larger size (only 10 megapixels apiece) or their filenames. The originals were named like so: 2009 04 22 Obligatory Hydrant Synth 000.JPG.
I have just now renamed the original images to only the last three digits and repeated Step 4 successfully, so the filename does seem to be what was causing the problem the first time I tried this collection. Whether it is the spaces in the filename or simply the length, I do not yet know.
What do you recommend for viewing the final PLY? I opened it in Meshlab, but am only seeing a nontextured mesh. Is this correct?
Wow! Fantastic job Henri! I have been running PVMS and getting frustrated w/ the limitation of the sift public license used with Bundler, it never let me use more than 13-14 images. Your toolkit is a Godsend.
I got the photosynth toolkit to work without any problems at all. Of course now the big barrier is PMVS, it can’t process many images due to the memory issues. I have a synth w/ 130 images I was hoping to use (and more I would like to use w/ 1500 images). In order to prevent it from running out of memory I had to request it use images at level 4 for 1k images, so it was only using 128 pixel high images. The point cloud density actually dropped from 242k to 95k from using PMVS. I re-ran 8 of the images even with this smaller number of images I could not get PMVS to process them at full resolution (20mp) I had to use levels to drop them down to ~900 pixel high, this did much better. I went from 5k points to 120k points!!!
It might be handy to have a way to break the photosynth data up into chunks that PVMS can handle better. Merging the clouds should be easy if they are in the same world space.
The only problem I could foresee is if there weren’t enough cameras in a chunk that see a given area it may not reconstruct well or at all, but I think the possibility of the increased resolution would be worth the trade off.
Oh forgot to mention, I was working from a reduced image set from this synth, http://photosynth.net/view.aspx?cid=2212c5ee-9036-4c5b-87cb-c5f5671ece86
Correction: When I am saying “Step 4″ above, what I meant was “Step 3″.
Thank for both of you, you are my first beta-tester
I’m glad this toolkit is finally useful to other people !
@Nate Lawrence: thanks for filling the bug, I’ll give a try to your synth and try to update my JSON parser that seems to be the source of your issue. I’m also using meshlab as ply viewer + an hand-made viewer based on Ogre3D. And yes the final mesh reconstructed by PoissonRecon.64.exe is not textured.
@Greg Downing: I got the same frustration with PMVS2 and I have compiled a 64bit version of PMVS2 for windows that manage to load more than 4Gb of pictures but it crashes right after the loading. I’ll give more information about my attempt of dense point cloud creation in my next post.
Thanks Henri,
I am sure you will have many very eager beta testers soon, the app rocks. I am looking into CMVS now to see if that will help w/ the dense set problem. Have you worked with this yet?
@Greg Downing: I already have used CMVS with Bundler output but not with PhotoSynth. I still have a lot of work to do: I’m working on a new Matching System for Bundler based on Surf and I also want to try PBA with PMVS2 output… So if you manage to get CMVS working with PhotoSynth that would be nice too !
Hi Henri,
I am trying to download a dense point cloud of this synth (http://photosynth.net/view.aspx?cid=5468035a-d176-4824-9342-57aea5237721) to compare it with a point cloud of the same object made with a laser scanner.
I ran into a problem at step 5_launchPMVS2.vbs. Right after I enter the synth URL and click Ok, I get the following error in an window:
Script: C:\Program Files\PhotoSynthTollKit2\ToolKit\5_launchPMVS2.vbs
Line: 11
Char: 3
Error: The system cannot find the file specified
Code: 80070002
Source: (null)
Any ideas?
Great work,
Kam…
@Kam Saidi: You’ve got this error because you didn’t have downloaded PMVS2 (read the Setup sections in the README.txt file)
Henri,
You are right (of course)! Sorry I didn’t read the setup instructions carefully. Downloaded PMVS2 and PoissonRecon and everything worked.
Merci pour ce bon travaille!
Kam…
Henri,
I’m using the photosynth toolkit 3 from the google code repository on a computer at school and it keeps telling me this error:
Script: …3_prepareForPMVS.vbs
Line:9
Char:1
This application has failed to start because the application configuration is incorrect. Reinstalling teh application may fix this problem.
Code:800736B1
Source: WshShell.exec
It produces this error for all the scripts. It’s not a system access restriction issue, since i have admin access. I’ve tested the same files and scripts on my machines, they work fine.
The machines i’m using at school are running Win XP.
Thanks.
Yang
@Yang Su: It may be because they didn’t install visual studio redistribuable 2008 (vceredist). What happens if you click on PhotoSynth2PMVS (not the vbs) ?
HI,
PVMS is very cool. But in the PhotoSynthToolKit3 step 5 is do not work (5_launchPMVS2.vbs). I have Vista Business 32 bit and 4 giga and 3 giga Turbo memory RAM.I try my synth (42 photo) http://photosynth.net/view.aspx?cid=a86848b1-89b9-44d4-ad47-fcbac8c39da8 Is it much big or what? I try other notebook with 3 Giga ram and it happend same error.
Error is:
Script: D:\PhotoSynthTollKit3\ToolKit\5_launchPMVS2.vbs
Line: 11
Char: 3
Error: 0x800700D8
Code: 800700D8
Source: (null)
and do not open PMVC2.exe. I have correct versions (32 bit from PhotoSynthToolKit3). What say about it?
Please help me somebody.
@PLzi
I found the same problem. To solve just download pmvs2 from here (direct link) http://francemapping.free.fr/Portfolio/Prog3D/CMVS/Win-binaries.zip and put extracted files into Toolkit folder. Or simple wath the first video here: http://www.visual-experiments.com/2010/09/23/pmvs2-x64-and-videos-tutorials/
Hi Astre,
After some installation problems i managed to get to run Toolkit 4 on a 64-bit Windows Professional, 2 Core System. On my Laptop, 32-bit Windows 7 professional i didn´t manage to get PMVS running, is it possible that it doesn´t run on Win32?
But what is more exciting to me is the question, if anybody knows, how Toolkit4 or Photosynth does handle the Radial Distorsion of the pictures. In BundlerV03, there is this possibility to run the RadialUndistort.exe /.sh. After this the radial undistorted Pictures are taken for PMVS-Reconstruction. With the ET-Pictures,the shifting between original and radial undistorted pictures ist about 4 Pixels at the edges of the pictures. Does anybody know, how to get undistort pictures that could be loaded up to creat a synth, or if it´s possible to get undistorted pictures from photosynth. As a bundle adjustment is done there, cameracalibration also should take place. Thanks a lot for your extremly good running and user friendly program. Cheers, Christoph
@Christoph: the PMVS2 version package with my toolkit is a x64 one, so you need indeed a 64bit OS to run it. For the radial undistort this is done while executing the “prepare for pmvs” script: I have imported the function from the RadialUndistort executable package with Bundler.
@Christoph as Henri mentioned probably the easiest way to remove the distortion is w/ PMVS, I believe if you look in the visualize directory you can find undistorted images.
Photosynth also removes distortion as part of it’s process. It uses K1 & K2 values just as bundeler does, the values are in the .json file Photosynth creates. You might notice the radial undistort program introduces slight color shift, if this is a problem can hack Autopano (also uses k1 &k2) to undistort.
Hi Astre, Hi Greg!
Thank you for your quick answears. Now I can see differences between the original Pictures I put in the distort folder and the once in the visualize folder.
As I have the parameters of the camera (K1, K2 (or even more (additional radial distorsion K3; decentering distorsion P1, P2; linear distorsion B1, B2, focal length and principal point))), because I did a camera calibration with Australis, would it be possible to use at least my own values for K1(5.8812e-004),K2 (8.9566e-006) and the focal length which is fixed? Or just to compare those values with the ones of Photosynth/Bundler. Where in this .json-file can I find them?
@ Greg: Do you know also a freeware, that can recalculate pictures with given k1,k2, because Autopano costs 100 $ and I am dealing with this matters as a student writting my master thesis on.
And another question: can Autopano recalculate pictures giving k1,k2 as input? And a last one: is it possible to calculate a cameracalibration with Autopano, based on certain kind of targets for example?
The thing is, that it would be interesting, if the results of the whole process would be better (which means more patches, less noise) if radial undistorted pictures are already uploaded zu photosynth. Because as I think PMVS only takes the given relaltiv orientation and doesn´t recalculate it on its own on the base of radial undistorted pictures.
Kind regards, Christoph
@Christoph, what is Australis? all I get for my google search is that it is a tasty white fish
Yes autopano can do a very good job of calibrating distortion if you shoot a pano w/ the lens. You can also make it remove the distortion by opening the xml file and putting in your distortion values. But again I think there are easier ways to do this, I only use this because I want 16-bit and more accurate color. K1 & K2 is the most common distortion model in computer vision, almost everyone uses them, K3 is considered by some to be a little more exotic. I am sure there are freeware tools to do this (have you tried the radial undistort Astre mentioned?) but maybe someone else has a suggestion, might be a good topic for the pgrammetry forum http://www.pgrammetry.com/
@ Greg, Australis/iWitness is a programm from photomatrix for photogrammetry use. But as you suggest Autopano as a good software to get undistorted pictures, I will try this after downloading a demo version. Thanks a lot, also for the pgrammetry-link, I think that´s now no Toolkit theme anymore. Therefor I want borro any longer. Bye Christoph
Hi Astre and everybody,
I wonder if there already exists a kind of visdata-writting-macro, that can help me to reduce the time of running, when I am changing the parameters to run Toolkit4 launchPMVS for a second time to get a denser pointcloud. As the visibilities can be read out of the *.patch-file, this should be possible. But as I am a totally looser in writing scripts, in Notepad for example, as the files are too big for excel, I am not able to do so. If someone has already written something to save processing-time, I would be interested in getting to known how. Thank you, Bye Christoph
@Christoph: Thanks for the tips! I’ll add this to the next version. I was thinking in another way using the neighbors pictures written in the JSON file but this is easier this way!
I see Greg mentioning radial undistort program introduces slight color shift, I am wondering if that is true, what I think is happening is the program strips the color profile assigned to the image, given that if the photo is in sRGB you would need to inject the correct sRGB profile into the photograph again if you want the color to be as it was before. I could run a test and look at the images in 3D color space to verify this.
This is very common for many online image processors the usually strip all color profile data, so if say an image is in AdobeRGB and processed on a server it will make the browser think it is in sRGB and you will have a large color shift and become de-saturated if the image is very saturated. Even in a browser supporting color management like Safari, and Firefox 3 and 4b.
@Christoph: I’ve implemented your suggestion (vis.data creation from .patch file), this would be part of the v5 release of my PhotoSynthToolkit.
@ Henri
Sounds good, when will you release V5? I´m really interested how this will speed up the process.
The other thing I couldn´t find out myself is, in what units are the camera parameters of the coord_system_0_camera.txt file given?
I think the order is the same like discribed in the bundler manual:
[the focal length, followed by two radial distortion coeffs]
[a 3x3 matrix representing the camera rotation]
[a 3-vector describing the camera translation].
But can you give me a clue how I can translate my value (see above,
January 8, 2011 at 5:45 pm) to the value in the file. Of course I only want to translate f, k1, k2.
I found an equotation f[px]=imagewide [px]*f[mm]/sensorwide [mm], but with that I dont get the results of the file. Is f in the file given in Pixels [px], and what units does k have?
Thanks for your work, and this cool Toolkit.
Bye Christoph
@Henri
Upps, I apologize for naming you by your last name, in the older posts. I don´t know why I did this, I´m sorry. Bye Christoph
Dear all,
I keep encountering problems when trying to extract the pointcloud from Photosynth. The error message states
when trying the SynthExtract,
An error occurred during export: Could not find endpoint element with name ‘ PhotosynthServiceSoap’ and contract ‘ PhotosynthService.PhotosynthServiceSoap’ in the ServiceModel client configuration section. This might be because no configuration file was found for your application, or because no endpoint element matching this name could be found in the client element”
When trying the Synth Toolkit:
The system cannot find the file specified
I’ve followed the tutorials, so I don’t understand whats wrong.. hopefully somebody has any thoughts about it…? Thanks a lot!!
Greets
@Simone: I guess that you mean SynthExport not SynthExtract? Are you using a proxy? Can you give me the link of the Synth you are trying to download?
1. What program do you think is better at image distortion 123d catch, or photosynth?
2. Is it possible to import distorted photos into 123d catch after a synth?
3. Is there a way to implement kinect point clouds into dense point clouds?
I’d like to mix my solutions, so far, I am trying to get the best work workflow, for the most detailed scans possible. I’d like to integrate my kinect sensor with this, since it’s instantaneous in its delivery of a point cloud. I use 123d catch, but it seems they cap the texture, and model quality from what I have read, to keep the cloud processing low for everyone to use. Let me know what you think if you have time
Thanks,
Eric McCann 3D entrepreneur
I meant image distortion removal
Great tool! I found this today while researching about point cloud. It’s good for studying Photosynth result of other users. I also manage to extract HD photos that I Photosynth some times ago.
I also have the “photosynthservicesoap” problem in all my photosynths!!1 For example the Icaromenippus 3D synths. Some help please!!!
@George Kontellis: are you using a proxy? The downloader doesn’t support downloading through proxy. Please try with another internet connection to confirm that your bug is related to your network configuration.
I tried from different internet connections gut with no result.
I also tried different synths.