This viewer is now integrated with the new version of my PhotoSynthToolkit (v5). This toolkit allow you to download synth point cloud and thumbnails pictures. You can also densify the sparse point cloud generated by PhotoSynth using PMVS2 and then create great accurate mesh using MeshLab.
New feature of PhotoSynthToolkit v5:
- Thumbnails downloading should be faster (8x)
- New C++ HD picture downloader (download tiles and re-compose them)
- Tools to generate “vis.dat” from previous PMVS2 call (analysing .patch file)
- Working Ogre3D PhotoSynth viewer:
- Can read dense point cloud created with my PhotoSynthToolkit using PMVS2
- Click on a picture to change camera viewpoint
- No-roll camera system
Warning: the PhotoSynth viewer may need a very powerful GPU (depending on the synth complexity: point cloud size and number of thumbnails). I’ve currently tested a scene with 820 pictures and 900k vertices on a Nvidia 8800 GTX with 768mo and it was working at 25fps (75fps with a 470 GTX and 1280mo). I wish I could have used Microsoft Seadragon 
.
Download:
The PhotoSynthToolkit v5 is available on his dedicated page, please do not make direct link to the zip file but to this page instead. So people willing to download the toolkit will always get the latest version.
Video demo:
Future version
Josh Harle has created CameraExport: a solution for 3DS Max that enable to render the picture of the Synth using camera projection. I don’t have tested it yet but I’ll try to generate a file compatible with his 3DS Max script directly from my toolkit, thus avoiding to download the Synth again using a modified version of SynthExport. Josh has also created a very interesting tutorial on how to use mask with PMVS2:
Masks with the PhotoSynth Toolkit 4 – tutorial from Josh Harle on Vimeo.
Nice !
A simple idea to make the system less restrictive could be to use image proxy (load full res image and downsize it in the soft before send it to the Graphic card texture memory).
The viewer could be generalized as a general SFM scenes viewer … (with a small amount of work..) View Bundler output… V3D ..
@Pierre: Thanks! I’ve already thought to make a general SFM viewer with a plugin system for specialization too. But I need to clean the code first and I’m willing to integrate the new version of Awesomium to help me building nice GUI. The only trouble I see with using this with Bundler output is that my camera system is a no-roll one, so the UP vector need to be computed (which is not the case by Bundler), otherwise the navigation would not be that great… I’d like also to add the possibility to create manual cluster to bypass the lack of CMVS in my PhotoSynthToolkit. Concerning your suggestion for downsizing picture, I’m already using thumbnails but creating a texture Atlas could help too!
Looks great, Henri! I’ll download this to my more powerful PC later tonight.
Regarding Seadragon, I assume you must have a fairly good idea of how the image tiling is to be parsed, at least on the single image level, for you to have written PhotosynthTileDownloader.
Here are a few links to information on the format that you may already be aware of which include two open source Deep Zoom image viewers (although they are admittedly for single images, rather than entire collections). http://bit.ly/f0NT6i
There are also a fair number of videos in this list that talk about how Seadragon works from Blaise, Gary Flake, Bill Crow, etc.: http://docs.com/XFO
And Gary Flake (former head of Live Labs) recently detailed the key strategies deployed in Seadragon on Quora: http://qr.ae/FHEg
The Deep Zoom Collection files are essentially a texture atlas for all images in a collection at a coarse resolution. Daniel Gasienica’s web log posts (listed in the first link) helped me realise this.
I suppose there is an argument for having DZI and DZC generators built into your toolkit in case of Bundler output (and even to save bandwidth when dealing with Photosynth output, although downloading the tiles for the highest resolution and then redundantly creating the DZIs and DZC for a synth when you could just download it may prove less than appealing).
I continue to wish that PhotosynthTileDownloader wrote the original EXIF metadata from the thumbs into the full resolution images at the time of download to preserve original photographer credit as much as possible and as uniformly as possible. I would even go so far as to have code in place that would scan for all photos which did not have ‘Author’ metadata and write the Photosynth username of the synth author to this field where that field was null, but perhaps I ask too much.
I just wish that all people who were using PhotosynthTileDownloader were using photos which offer attribution back to the original photographers in an identical manner, and while I may be careful to do this by hand, it is unlikely that all others will unless the downloader does it for them.
Best regards,
Nate
@Nate: Thanks! Concerning the Seadragon dzi files I’m indeed well aware of how to parse them and I’ve realized few days ago that my previous Javascript TileDownloader was wrong… The tiles were not composed correctly (1px error).
Using a texture mipmap to store a dzi seems to be the trivial solution to implement a GPU-accelerated viewer but it doesn’t solve the principal issue fixed by Seadragon: how do you fit in GPU memory several Go of pictures when you are limited to 512mo (for example) ?
I’ll see what I can do anyway with the Deep Zoom Collection files, thanks for reminding me of that! FYI, this is because I wanted to implement your suggestion (transferring thumbs Exif to HD picture) that I’ve decided to re-write my TileDownloader from scratch in c++. So yes, the existing version does copy Exif data from thumbs to HD picture using JHead (if thumbs are downloaded). At first sight your idea to copy the synth username to the Author metadata seems right but if a user is using someone else picture (downloaded from Flickr for example) I don’t want to be the one (in regards to the law) that is giving him full credits (and Microsoft doesn’t do that either). From my point of view if you publish a Synth and make it public you’ll have to cope with people who don’t respect copyright anyway…
@Henri, thank you for your reply and your extra work in rewriting the tile downloader.
Regarding not wanting to write the Photosynth username to the author tag of photos who have no author information, I had certainly considered the case of people who had created synths using others’ photos scraped from the web.
My intuition was simply that the number of cases where novice photographers had shot their own photos for photosynth and uploaded without first tagging the photos with the desired author information far outnumber the cases in which others’ photos are used without regard to the original photo authors’ attribution. Also, if the original synth author has used the photos without giving attribution to the original photographer(s), my qualms about inaccurately ascribing credit to them are lessened if it means the hundreds of synths whose photos were actually shot by the photosynth member. If the synth author has uploaded others’ photos without giving credit to the source photo photographers, that is their error, rather than the tile downloader’s.
In any case, I don’t expect you to change your mind on this issue, but thought I would make the case, nonetheless.
In defense of your argument, in the case of an unattributed photo from a public source used in multiple synths, this writing of the synth username to the author tag of the photos would actually cause any subsequent re-uploading to Photosynth to treat not only the downloaded version as different to the original (which is unavoidable), but also multiple copies of the uploaded originally identical photo from multiple synths as unique from each other, rather than allowing Photosynth to recognize them as the same file and not upload twice because their author metadata now differs.
My entire request to have the original metadata maintained revolves around three main issues: first: the desire to preserve credit where credit has been originally provided, second: the desire to have this credit uniformly applied for all users of the tile downloader so that subsequent reuploads of the downloaded copies of photos would all be recognized as identical to each other while providing credit to the original synth author (whether the user of the tile downloader had taken the time to provide credit or not), and thirdly: to aid in re-use in various bundle adjusters or MVS programs.
From what you tell me, your current solution fully addresses my third concern as all camera attributes should be transferred (at least in cases where the thumbs have been downloaded – I still wish this was universal). The first and second are partially satisfied (again: pending the user’s choice to download the thumbs), but my experience is that the vast majority of photosynth users do not know how to tag their own photos with photographer metadata before uploading and I still desire some relatively painless way to uniformly give them all credit.
I suppose my earlier argument (in the opposite direction) of original synth author’s responsibility to provide correct attribution for others’ photos which they had used can also be turned around and argued that if they fail to provide author data for their own photos, then the failure is their own. I am only trying to find the default behavior which will best serve the majority of people. I still believe that synths constructed of flickr scrapes or other web scrapes are the edge case, rather than the prevalent source of synth photography.
In any case, that’s probably enough pros and cons for one entry.
On a last unrelated note, I have wondered how complicated it would be to download the highest resolution of the six DZIs which comprise Photosynth panoramas and project them back into a flattened undistorted single image for re-use in creating group synths from multiple users’ photos.
Another day, another challenge.