Archive for the ‘photosynth’ category

« Older Entries

Photosynth website on steroids :)

February 3rd, 2014

Bad idea: hacking during the new year holiday and during the super bowl go Seahawks! Result? A very hacky google chrome extension which is adding several alpha/beta quality features to the photosynth.net website. This is my gift for 2014, enjoy! :)

You can get the extension here. Make sure you are logged in on photosynth.net (and have joined the beta) while using the extension as some photosynth2 feature are still only available to beta users (hopefully this limitation will be removed soon).

Feature added by the extension:

The extension is changing the top menu to:

  • Search: previous Explore page + new links to fullscreen grid based ps2 explore page.
  • Explore: new fullscreen map based explore page.
  • My Synths: previous My Photosynths page + new tab (my map, my ps2, my info).
  • Preview: link to the new preview website + animated preview on the create page.

Search:

Animated preview when you hover a thumbnail of a PS2 + links to new explore page.

Clicking on “Explore most recent PS2″ or “Explore most viewed PS2″ will display a page like this:

The bottom row contains the latest synths of the current user (the one of the current synth being played). You can hover to see a preview and click to load a synth. Experimental feature: it’s using a new version of the viewer capable of quickly unload/load synths (don’t click too fast :-) ).

Explore:

New fullscreen page showing synths (ps1, panorama and ps2) on a map. Sadly there is no proper ranking here so you need to zoom in a lot before being able to see synths. Also there is a known bug in google chrome which is preventing from seeing silverlight content opened in new tab.

My Synths:

The extension add 3 new tabs: My map, My PS2, My info

My map

This page is really complicated, adding a lot of feature… I will propably have another post explaining how to use it.

Basically you can map (=geotag) your own synths (ps1, panorama, ps2). You need to be logged in and click on ‘show untagged synths’ then you can search for a place on the map, move the orange pushpin or right click to reassign it location. Then once the orange pushpin is at the place where you’ve captured your synth, you can click on ‘assign pushpin location’ to assign the location to the corresponding synth. To select synths, you can press and hold ‘ctrl’ and then draw a rectangle, from that selection you can either remove their map position or assign them a text tag. Clicking on ‘movable pushpins’ will allow you to directly move your synth pushpins on the map. You can click on on ‘Find neighbors’ to find synths arround your synth (orange synth = community, green = yours). You can also play with the timeline to only display synths captured in the corresponding time interval. You can also change the url and switch w=0 to w=1 to enable the wikipedia option. This option will search for the closest wikipedia of your synth. Please consider donating to WikiLocation if you are using the w=1 option.

My PS2

This is a new page allowing you to quickly preview all your ps2 synths. They are grouped by topology (spin, panorama, wall, walk) and then sorted by captured date.

My info

Sadly the captured date information is not properly filled by the system (it’s using the upload date). You need to click on ‘fix capturedDate’ to set the captured date of all your ps2 (it might take a while: wait for the ‘done’ alert box).

Preview – Create

The extension is adding an animated preview which will help you to choose the proper topology:

Comments

This extension was created by me only and thus it doesn’t mean that this is representative of upcoming photosynth feature.

FYI this is not my first extension for the photosynth website, I’ve already created one which is adding a webgl fallback viewer if you don’t have silverlight for photosynth1 content.

Have fun exploring all feature introduced in this chrome extension!

Share

2 comments »

Posted in photosynth

Tags:

Photosynth 2 launch!

January 25th, 2014

Happy new year 2014!

This is a short post in case you’ve missed the launch of Photosynth2. I’m glad to be able to break the silence and to show some nice stuff again :) . I’ve developed the WebGL viewer used to display ps2 synths and I’ve been busy capturing tons of ps2′s (I still have a lot of datasets waiting to be uploaded :) ).

If you are using a modern browser you should be able to see this ps2 with the WebGL viewer:

Otherwise we are also generating an mp4 fallback:

In case you are wondering what’s new compared to photosynth1, I’ve created this video showing the benefit of a ‘good’ geometry (ps2) for image-based rendering instead of using only a dominant plane (ps1):

Photosynth 1 is allowing unstructured capture and thus the navigation is very difficult. In Photosynth 2 we are constraining the user to capture images along a single path (1D manifold) and thus the navigation is very simple (and touch friendly :-) ). Photosynth 2 introduces 4 different topology (camera motions):

Spin:

Panorama:

Walk:

Wall:

BTW if you type: “The answer to life, the universe and everything”, on the view page of a synth a magic menu will appear with lot of options so that you can tweak your viewer. You will need to refresh the view page to apply most options and the changes are permanent (store in localStorage).


‘m’ and ‘c’ are also two other shorcuts that you should try…

I should have another post very soon. Stay tuned!

Share

3 comments »

Posted in photosynth

PhotoSynth WebGL panorama viewer

November 1st, 2011

I’ve updated my PhotoSynth Downloader/Parser to be able to process Synth and Panorama as well. The network part of my PhotoSynth downloader was really ugly: I’ve rewrote it from scratch using libcurl. This version is really much faster and doesn’t crash anymore. But I’m a bit concerned about releasing the binary/source code. Allowing the download of a panorama created with an iPhone doesn’t bother me but it’s really different with gigapixel panorama (and my downloader can download both).

Live Demo (require WebGL enabled browser)

My implementation is very strait-forward: my PhotoSynthHDDownloader creates 6 jpg from a panorama url. The 6 faces of the cube are downloaded at full resolution. This solution is suitable for panorama created with an iPhone (low res) but not acceptable for gigapixel panorama (you need progressive download/display). I’m using Three.js as WebGL engine and I’d like to implement a megatexture solution to display very large panorama (I’ve found a very impressive working implementation and played with it: it is working with Google Chrome / Firefox 10).

To be able to handle panorama as well, my Google Chrome extension needs to use megatexture and it would be really more efficient if Microsoft could add the missing cors header to allow WebGL cross-origin (even only for photosynth.net). And as they’ve just moved their data to Windows Azure, I guess that it may be easier to add this header ;-) . Otherwise I’ll have to use my special trick to bypass webgl same-origin-policy restriction.

Share

12 comments »

Posted in html, photosynth

Tags:

2010 visual experiments

January 7th, 2011

Happy new year everyone!

2010 was a year full of visual experiments for me, I hope that you like what you see on this blog. In this post I’m making a little overview of all visual experiments created by me during this year. This is an opportunity to catch-up something you’ve missed! I’d like also to thanks some person that have been helping me too:

Visual experiments created in 2010:

During this year I have added some features to Ogre3D:

  • ArToolKitPlus: augmented reality marker-based system
  • Cuda: for beginner only (at least advanced user could grab some useful code)
  • OpenCL: for beginner only (at least advanced user could grab some useful code)
  • Html5 Canvas: implementation based on skia for graphics and V8 for javascript scripting
  • Kinect: this is a very hacky solution, I’ll improve it later

I also have learned GPGPU programming by myself while coding a partial GPUSurf implementation based on Nico Cornelis paper. But this implementation is not complete and I’m willing to rewrite it with a GPGPU framework based on OpenGL and CG only (not Ogre3D). With such a framework writing Sift/Surf detector should be easier and more efficient.

I have created some visual experiments related to Augmented Reality:

My outdoor 3D tracking algorithm for augmented reality needs an accurate point cloud: this is why I’m interested in structure from motion and I’ve created two SfM toolkit:

Posts published in 2010:

Share

6 comments »

Posted in augmented reality, ogre3d, photogrammetry, photosynth

Tags:

Augmented Reality outdoor tracking becoming reality

December 13th, 2010

My interest in structure from motion was primary motivated by the capability of creating a point cloud that can be used as a reference for tracking reference. The video below is more a proof-of-concept than a prototype but this is an overview of my outdoor tracking algorithm for Augmented Reality:

Analysis

In a pre-processing step I’ve built a sparse point cloud of the place using my SFMToolkit. Each vertex of the point cloud has several 2D Sift features correspondences. I’ve only kept one Sift descriptor per vertex (mean of the descriptors) and put all descriptors in an index using Flann.

For each frame of the video to be augmented, I’ve extracted Sift feature with SiftGPU and then matched them using Flann 2-nearest neighbor search and a distance ratio threshold. The Flann matching is done in parallel with boost::threadpool. The matches computed contains a lot of outliers. So I have implemented a Ransac pose estimator using EPnP that permits to filter bad 2d/3d correspondences.

Performance

My implementation is slow (due to my implementation of Ransac EPnP that could be improved).

Sift first octave: -1
Sift extraction: 49ms 2917 features
Sift matching: 57ms (parallel matching using Flann)
Ransac EPnP: 110ms 121 inliers of 208 matches
Global: 4.6fps (9.4fps without pose estimation)

Sift first octave: 0
Sift extraction: 32ms 707 features
Sift matching: 15ms (parallel matching using Flann)
Ransac EPnP: 144ms 62 inliers of 93 matches
Global: 5.2fps (21.2fps without pose estimation)

The slowness is not a so big issue because it doesn’t need to run at 30fps. Indeed the goal of my prototype is to have absolute pose with this tracking system each second and relative pose using inertial system available on mobile device (or using KLT tracking).

Issue

  • Performance (faster is better ;-) )
  • Point cloud reference is not always accurate (Bundler fault)

In another post I’ll introduce alternative to Bundler: faster and more accurate.

Share

12 comments »

Posted in ogre3d, photogrammetry, photosynth

Tags: