Combining Two Kinects To Make Better 3D Video

suraj.sun sends this quote from Engadget about improving the Kinect 3D video recordings we discussed recently: "[Oliver Kreylos is] blowing minds and demonstrating that two Kinects can be paired and their output meshed — one basically filling in the gaps of the other. He found that the two do create some interference, the dotted IR pattern of one causing some holes and blotches in the other, but when the two are combined they basically help each other out and the results are quite impressive."

Anybody in optics?

[fuzzyfuzzyfungus]

How cost and/or physics prohibitive would it be to exploit the fact that "IR" actually covers a number of frequencies of invisible-to-the-naked-eye light with similar properties? Could one modify a Kinect with appropriate narrow-band filters, so that a second Kinect, with filters for a different narrow band wouldn't even see the dot pattern of the first? If possible, how many Kinects would it be possible for(or, at what point does the required narrowness and wavelength tolerance requirements become absurdly costly?)

Is that A)Wholly impractical, because of some sort of effect the reflecting materials would have on the IR wavelengths, B)Sure, it's possible; but have you checked the supplier's price list for narrowband IR filters recently, or C)Just a bit of ebay and some steady hands?

Perhaps more practically, I wonder if the Kinects could(with some mixture of hardware shutters and firmware or driver mods) be made to trade off sample rate for coverage(ie. if the kinects are ordinarily taking 60 frames/second, could two kinects be made to take 30 frames/second each, turning off their IR source when it isn't their turn, and turning it on when it is) or does their mechanism of operation require too much time to calibrate itself on startup?

Re:Two eyes are better than one

[fuzzyfuzzyfungus]

There is a class of visual inputs that makes the human brain just tie itself in knots, even once you know that the trick is, "optical illusions", Escher stuff, and the like.

I wonder what the class of "optical illusions" for the Kinect's vision system and algorithms is... Off the top of my head, I'd imagine that retroreflective materials might kind of freak it out; but I'd be curious to know if there are any stimuli that cause it to wig out in weird ways, the way that optical illusions do the human visual system.

Re:This is immense...

[L4t3r4lu5]

It's a mixture of the two. He used two cameras to film the live action scenes, but the output was reduced to stereoscopic 3D on the screen.

This is actual 3D on the screen, like a 3D game. You can't zoom in, or even focus, on the background in Avatar. In fact, attempting it gave me a massive headache. With this true 3D rendering of an object, you can zoom, focus, and more importantly pan around objects in the scene, in real time. That is the breakthrough this hack has brought about.

Re:Two eyes are better than one

[Anonymous Coward]

Ear's can't do that, that's the brain. And for the kinect that would be the program in the device it connects to.

Re:Two eyes are better than one

[EdZ]

As the video demonstrates, the Kinect is fooled by spurious pattern projections from other Kinects in the vicinity. This could be solved by replacing the IR source in the 'projector' (actually a point source and a pinhole grid) with one of a different wavelength, and adding appropriate filters to the IR cameras in each Kinect. Each Kinect would then only see IR light of the 'colour' it emits. This would probably require the use of slightly brighter IR emitters.