Posts Tagged ‘webgl’

« Older Entries

Improving PhotoSynth extension performance

March 19th, 2012

I’ve recently discover the new transferable object feature in Google chrome. BTW transferable objects are really great! Furthermore you really should use them as I’ve found that structure cloning is leaking (you can try this leaking demo with chrome 17). Thus I’ve started to update my extension and done a benchmark of various solution with a big synth of 20mo (1200k vertices and 1000 pictures).

I’ve also improved a lot my binary parsing by using only native binary parsing (DataView). One thing about DataView: did they realize that if you forgot the endianness parameter it will use BigEndian? (littleEndian = x86, bigEndian = Motora 68k).

Thanks to both optimization the new version is 4x faster! (loading time: 24s > 6s)

solution 1



  • cross-orign was forbidden in contentscript (fixed now since chrome 13)
  • lack of TypedArray copy/transfert -> JS array replacement
  • Loading time: 13600ms (24400ms with the previous version without optimized binary parsing)

solution 2



  • Issue: the UI is frozen during parsing (lack of threading)
  • Loading time: 6500ms

solution 3



  • I’m using transferable objects to both send the xhr.response (ArrayBuffer) and receive the parsed Float32Array (vertex positions) and Uint8Array (vertex colors).
  • Issue: I didn’t manage to spawn a worker from my extension directly (SECURITY_ERR: DOM Exception 18): I’ve been forced to inline the worker as a workaround.
  • Loading time: 6000ms

solution 4



  • Issue: because of this bug workers can not do cross-orign xhr.
  • Loading time: N/A

I’ve also optimized the PLY file generation by generating binary file on the client side instead of ascii.

  • Ascii: 74mo in 9000ms
  • Binary: 17mo in 530ms (~17x faster!)

Sadly I didn’t manage to find a way to accelerate “bundle.out” ascii file generation yet (involving lot of string concatenation and numbers formatting).

I didn’t have updated the extension on the chrome web store yet as I’m working on using the new BufferGeometry of Three.js that should be more efficient. Building BufferGeometry directly from Float32Array (vertex positions array) and Uint8Array (vertex colors array) seems a really better solution IMO.

Mesh compression

I’ve also discover the webgl-loader mesh compression solution recently. I’ve compiled my own version of objcompress.cc and fixed the JSON export. Then I’ve created my own format allowing to pack in one file all the utf8 files (thus reducing the number of http request needed). I will post about it soon: stay tuned!

Firefox extension?


On a side note I’ve also been playing with the new mozilla add-on builder SDK… well IMO Firefox isn’t the best browser for extension anymore. I’ve been very disappointed by the restriction applied to code running in the contentscript (for example prototype.js is not working: I had to use a clone that I’ve built long time ago). And I’m also very concerned about their security strategy: they don’t have a cross-orign xhr permissions field in the manifest. Thus extensions can do cross-orign request to all domains by default! Nevertheless I’ve managed to port the binary parsing of the point cloud but Three.js is not working either :-( .

Share

3 comments »

Posted in html

Tags:

PhotoSynth bundle.out export!

February 1st, 2012

I’ve updated my Google Chrome extension to export a valid bundle.out from a PhotoSynth. The bundle.out generated need to be used with HD version of the images (not the thumbnails). Moreover I’ve removed tracks of length 2 from the bundle.out as I guess that PhotoSynth is keeping them only to get a “denser” point cloud (aesthetic purpose only)? One more thing: you can not use the bundle.out file to run a bundle adjustment as the 2d measurements are unknown (I have computed them by projecting the tracks in corresponding cameras).

I’m sure that some of you have an idea of what to do with a bundle.out file from a PhotoSynth ;-) .
BTW, this export wouldn’t have been possible without Peter Sibley help, thanks Peter! He has help me by giving me the missing information needed to do the bundle.out PhotoSynth export.

Share

25 comments »

Posted in html

Tags:

PhotoSynth viewer with HTML5 FileSaver

January 15th, 2012

Happy New Year Everyone!
This is my small gift for 2012: I’ve updated my PhotoSynth Google chrome extension!

New features:

Other features are coming:

  • basic panorama support (using atlas.jpg only: no progressive download yet)
  • download zip file of whole synth?
  • bug fixes (the current version is still very buggy, especially the menu)
  • other cool stuff are coming! :-)

If you don’t have Chrome you can take a look at my previous video showing most of the new feature (except point cloud downloading). The previous version wasn’t available as it was using my previous trick to bypass sop restriction (too slow to be widely used).

The new version is available on the Google Web Store.

Share

No comments »

Posted in html

Tags:

How to bypass WebGL Sop restriction v2

December 5th, 2011

I’ve previously posted a trick to bypass the WebGL same origin policy restriction in a particular context. But my solution had a major drawback: it was slow due to jpeg decoding in javascript. Hopefully I’ve implemented a new solution which is way much faster thanks to John Bauman suggestion and it works like a charm! 

var xhr = new XMLHttpRequest();
xhr.open("GET", JPEG_URL, true);
xhr.responseType = "arraybuffer";
xhr.onload = function() {
	var img = new Image();
	img.onload = function() {
		var tex = new THREE.Texture(this);
		//do something with this texture...
	};
	img.src = arrayBufferDataUri(xhr.response);
};
xhr.send(null);

The function “arrayBufferDataUri” at the line 10 allow to speed-up the conversion from ArrayBuffer to base64 ascii string by avoiding an extra copy needed to use window.btoa. I’ve found this function here: http://jsperf.com/encoding-xhr-image-data/5.

//From: http://jsperf.com/encoding-xhr-image-data/5
function arrayBufferDataUri(raw) {
	var base64 = '';
	var encodings = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/';
	var bytes = new Uint8Array(raw);
	var byteLength = bytes.byteLength;
	var byteRemainder = byteLength % 3;
	var mainLength = byteLength - byteRemainder;
	var a, b, c, d;
	var chunk;
	// Main loop deals with bytes in chunks of 3
	for (var i = 0; i < mainLength; i = i + 3) {
		// Combine the three bytes into a single integer
		chunk = (bytes[i] << 16) | (bytes[i + 1] << 8 ) | bytes[i + 2];
		// Use bitmasks to extract 6-bit segments from the triplet
		a = (chunk & 16515072) >> 18; // 16515072 = (2^6 - 1) << 18
		b = (chunk & 258048) >> 12;   // 258048   = (2^6 - 1) << 12
		c = (chunk & 4032) >> 6;      // 4032     = (2^6 - 1) << 6
		d = chunk & 63;               // 63       = 2^6 - 1
		// Convert the raw binary segments to the appropriate ASCII encoding
		base64 += encodings[a] + encodings[b] + encodings[c] + encodings[d]
	}
	// Deal with the remaining bytes and padding
	if (byteRemainder == 1) {
		chunk = bytes[mainLength];
		a = (chunk & 252) >> 2; // 252 = (2^6 - 1) << 2
		// Set the 4 least significant bits to zero
		b = (chunk & 3) << 4;   // 3   = 2^2 - 1
		base64 += encodings[a] + encodings[b] + '==';
	}
	else if (byteRemainder == 2) {
		chunk = (bytes[mainLength] << 8 ) | bytes[mainLength + 1];
		a = (chunk & 16128) >> 8; // 16128 = (2^6 - 1) << 8
		b = (chunk & 1008) >> 4;  // 1008  = (2^6 - 1) << 4
		// Set the 2 least significant bits to zero
		c = (chunk & 15) << 2;    // 15    = 2^4 - 1
		base64 += encodings[a] + encodings[b] + encodings[c] + '=';
	}
	return "data:image/jpeg;base64," + base64;
}

I’ve only used this code for my Google Chrome PhotoSynth WebGL extension. This is why I don’t have added extra check for other browser (line 10: xhr.mozResponseArrayBuffer || xhr.response).

Share

4 comments »

Posted in html

Tags:

PhotoSynth WebGL panorama viewer

November 1st, 2011

I’ve updated my PhotoSynth Downloader/Parser to be able to process Synth and Panorama as well. The network part of my PhotoSynth downloader was really ugly: I’ve rewrote it from scratch using libcurl. This version is really much faster and doesn’t crash anymore. But I’m a bit concerned about releasing the binary/source code. Allowing the download of a panorama created with an iPhone doesn’t bother me but it’s really different with gigapixel panorama (and my downloader can download both).

Live Demo (require WebGL enabled browser)

My implementation is very strait-forward: my PhotoSynthHDDownloader creates 6 jpg from a panorama url. The 6 faces of the cube are downloaded at full resolution. This solution is suitable for panorama created with an iPhone (low res) but not acceptable for gigapixel panorama (you need progressive download/display). I’m using Three.js as WebGL engine and I’d like to implement a megatexture solution to display very large panorama (I’ve found a very impressive working implementation and played with it: it is working with Google Chrome / Firefox 10).

To be able to handle panorama as well, my Google Chrome extension needs to use megatexture and it would be really more efficient if Microsoft could add the missing cors header to allow WebGL cross-origin (even only for photosynth.net). And as they’ve just moved their data to Windows Azure, I guess that it may be easier to add this header ;-) . Otherwise I’ll have to use my special trick to bypass webgl same-origin-policy restriction.

Share

12 comments »

Posted in html, photosynth

Tags: