Head Tracking w/ the Wiimote 169
mrneutron2003 writes "This guy just doesn't know when to stop. Johnny Chung Lee graces us with yet another one of his inventive Wiimote projects. This time it involves using the Wiimote and a pair of inexpensive LED safety goggles (with the standard LED's replaced with InfraRed ones) to allow positional head tracking , achieving an effect similar to what is experienced with three dimensional displays and CAVE systems. The video dramatically illustrates the effect. Game developers take note. This simple little variation on infrared tracking could allow for some seriously immersive gameplay in the future." This guy deserves a medal.
Nintendo! Hire Johnny Lee! (Score:5, Insightful)
Re:Nintendo! Hire Johnny Lee! (Score:5, Funny)
Re: (Score:1, Funny)
Re: (Score:2)
This guy has seriously just handed out some expensive R&D to Nintendo for free.
Seth
Re: (Score:3, Informative)
Re: (Score:2)
Re: (Score:2, Insightful)
Re: (Score:3, Interesting)
Keyboards on consoles ain't news. That's about as stale as a Sega Mega Drive. Not to mention that people would most likely rather use a "normal" computer for browsing the web.
This is a new feature. Brand new. And doesn't need any additional gadgets or any new developments. At the very least, Nintendo should have had something like that demo in its standard applications to show it off and leave the customer dreaming of games that make use of it.
Re: (Score:1)
Re: (Score:1, Funny)
Only problem now is that i can't see the TV
Re:Nintendo! Hire Johnny Lee! (Score:4, Insightful)
Only problem now is that i can't see the TV
And besides, it's not what direction you're looking, it's what direction you're looking from. Move your whole body to the right while continuing to look at the TV and the display on the TV changes perspective. Not to mention depth of field, and distance from the TV. Did you even watch the video?
Why am I even responding to an AC comment?
Re: (Score:2)
Re:Nintendo! Hire Johnny Lee! (Score:5, Insightful)
The PC Gaming landscape is littered with failed head-tracking systems. The reviews inevitably say something like "this thing is awesome, but fatiguing."
There are eye-tracking systems that are not nearly as fatiguing, but if you've seen one, you'll understand why they haven't taken off in popularity.
Re: (Score:3, Insightful)
If you talked about putting a accelerometer in to a controller before the Wii, you'd be laughed at.
Re: (Score:2, Insightful)
Re: (Score:3, Insightful)
Re:Nintendo! Hire Johnny Lee! (Score:5, Insightful)
If you're perfectly perpendicular to your monitor, there is limited arc of motion that your head can make before the monitor is out of your direct line of sight and into your peripheral vision. This artificially limits what you can do in a game and is why head tracking systems have not replaced traditional controls for looking along the X & Y axis.
I'm not saying there is no role for this in gaming, I think it would be great if Nintendo could make it cheaply for the Wii and developers created games that could use it effectively... but that has been tried before in PCs... without much success.
Re: (Score:2)
This artificially limits what you can do in a game and is why head tracking systems have not replaced traditional controls for looking along the X & Y axis.
TrackIR has solved the problem by having a small range of motion translate into a larger range in-game. Now, for FSP-like purposes, like in the video I don't think it'd matter. The headtracking is useful for controlling leaning, such as around corners, and most importantly it will give a sense of immersion, regardless of how much range of control you have. Regular movement is still best controlled by the nunchuk + Wii, with headtracking simply adding on to that.
Re: (Score:2)
A major advantage the Wii has over the other consoles is the discrete aiming of the wii-mote that matches with the discrete aiming of the mouse. Analog sticks work on acceleration whi
Re: (Score:2)
it's all research, man (Score:2)
I guess he'll have to settle for a PhD from Carnegie Mellon instead.
Re:it's all research, man (Score:4, Insightful)
Re: (Score:2)
Re: (Score:2)
There are plenty of examples of high selling peripherals so I don't know what the GGP was talking about.
Re: (Score:3, Insightful)
How about people just send him some money so he'll keep doing what he's doing and make it free?
WiiHelm (Score:2, Funny)
http://www.thinkgeek.com/stuff/41/wiihelm.shtml [thinkgeek.com]
video down (Score:5, Informative)
DUDE (Score:2)
I can only imagine what something that that, coupled with a graphics engine like Assassin's Creed has would do for immersive gaming.
Re: (Score:2)
Polarized filters and LCD shutters aren't anything new, and I'm sure they've been combined with head tracking before... But I don't think it's ever been quite as accessible as it is now. Stereoscopy + head tracking + a pointing device like the Wii remote could make for one hell of game.
Re: (Score:2)
Polarized light not compatible with LCD (Score:5, Informative)
They only work with DLP projectors (uses little mirrors), CRTs, plasma, and upcoming display technologies like Field Effect Displays and LED displays. Obviously there are a lot of display technologies that do work there, but LCD is a very popular technology for widescreen TV and of course, for PC monitors.
Either way you do it, you also have to double the grunt of your rendering system (or half your graphical complexity), and you need specific software support to get it right (you can go a long way with a driver that knows it's rendering for stereoscopy and just produces the correct eye POVs, but the glitches you get in the foreground and HUD are only tolerated by enthusiasts.). With shuttering you need glasses. With cross polarization you need to double the number of display elements (by having two displays or a special display with double the horizontal resolutions). Used in POV applications, all of these technologies are a one-user gig.
Stereo "Wii-D" will probably never happen ; half the audience have an incompatible display device, the system does not have an enormous excess of GPU grunt. Stereo3D would only be common with one of the following display devices...
* Personal head-mounted 3D display (probably VRD goggles)
* Large area wide aspect flatpanel displays with inherent stereo 3D support built in at the factory (which means basically doubling the vertical rez and making a special polarized filter for the screen).
The parallax effect that Johnny Lee demonstrates conveniently exploits the tendency of the human brain to "fill in the gaps" ; I'd be intrigued to see how convincing it really is.
As another poster points out, head tracking really isn't very well received for the PC, because the PC is an inherently static device. You can move your head, but your hands have to remain fixated on the keyboard / mouse. The Wii has an advantage here because the input device moves around with you. Several times during Zelda I got up from my chair and started moving almost involuntarily, my whole body was immersed in the game. I would never have tried that on the PC ; when I feel the urge there it probably just contributes to my neck tension.
If the static, 3rd person POV of Zelda can make this gamer rise up and move, a game armed with a head tracking linked POV would be compulsively immersive, even without stereoscopic 3D.
Muppets? (Score:2, Insightful)
Also, the head tracking is awesome.
Re: (Score:1, Funny)
Re: (Score:1, Funny)
Re: (Score:1, Funny)
Re: (Score:2, Offtopic)
As for the projects he's doing, you're right, it's awesome. Even watching the youtube video of the headtracking gives you the feeling that you were watching a 3D effect, I wonder how i
Aiming a gun by looking at your target (Score:2)
The Wiimote is truly the ultimate hackable peripheral...
Re:Aiming a gun by looking at your target (Score:5, Interesting)
Re: (Score:2)
Wiimote with ability to track more points? (Score:2)
Re: (Score:2, Insightful)
Re: (Score:2, Insightful)
Remember the core wiimote can handle an additional nunchuck or the classic controller, each of which requires more data than the small amount of data per frame required.
If Nintendo wanted to do this anyway I believe they would use a custom device with its own interface (and would almost certainly retain the power connection the current sensor bar uses).
There has been rumour that the next wii will be controller less, people simply acting out the actions to get results.
Thi
Worth mentioning.. (Score:5, Informative)
The TrackIR solution linked above costs around as much as a Wii itself.
Re: (Score:2)
This little software feeds the TrackIR interface from a cheap webcam. Since it doesn't use a Wii it won't make you more attractive to the gender that you prefer being attractive to, but then you get full six degrees of freedom, which a Wii sensor bar can't do (it has only two points to track). Only trouble is the weak framerate most webcams have, and horrible webcam drivers that can suck away a considerable amount of CPU time.
Re: (Score:2)
Note that its "capable of capturing standard video with frame rates of 60 hertz at a 640x480 pixel resolution, and 120 hertz at 320x240 pixels" (see http://en.wikipedia.org/wiki/PlayStation_Eye [wikipedia.org]).
Why isn't this guy working for nintendo? (Score:2)
Of course, having a 42" plasma or larger is going to be the optimal thing here - and since the wii is priced to sell, then a lot of people are going to be using them on their old 1980's 27inch CRT's.
Re: (Score:2)
Insensitive clod, I bought mine in the 90's!
Cool ... (Score:1)
Fantastic (Score:5, Interesting)
Re: (Score:2, Informative)
Re: (Score:2)
The multitouch is fantastic, and at the same time still limited. If you have seen the original multitouch video then you know that there is incredible potential with that technology, way beyond what the iPhone does.
Re: (Score:2)
Re: (Score:2)
Actually, Microsoft have been developing a multitouch screen, not the coffee table thing with multiple cameras looking at it, but one for laptops by using infra-red LEDs & sensors embedded in the back of the LCD screen. It was shown in the recent episode of The Gadget Show.
More info at Engadget [engadget.com]
Better than a medal (Score:4, Insightful)
Nintendo, are you listening?
Editors need to edit. (Score:5, Informative)
You spelled 'too' wrong.
Re: (Score:2)
Just be thankful he didn't spell it "two"
Just to cool (Score:2)
Maybe they meant that it was not to be used for heating?
processing power (Score:1)
Re: (Score:2)
Re: (Score:1)
Re:processing power (Score:5, Interesting)
Re:processing power (Score:5, Interesting)
Not to trivialise what Johnny is doing there is basically measuring the position of the wiimote in relation to the sensor bar - something it already does. The code to do this shouldn't be that difficult. The true genius was in him realising that you could do this easiest by reversing the moving component and the stationary component.
Apart from some smoothing algorithms, this is no more complex than reading the wiimote's pointer position and mapping that to a camera viewpoint.
Re:processing power (Score:5, Insightful)
It's so simple that you can do something with it, without having to wait for IBM, or Nintendo or any other big-$$$ company to bring out the relevant hardware in maybe 5 years.
Re: (Score:3, Interesting)
Re: (Score:2)
http://en.wikipedia.org/wiki/Wii_Remote#Wii_Zapper [wikipedia.org]
More importantly, that technology base is now commodotized and available with every single Wii sold. It would be very different if this were being done with more specialized hardware which only a small percentage of customers bought optionally. The exciting thing about these demons
Re: (Score:2)
Plus we can yell out "Yo mamma wears a wiimote with a chinstrap!!"
Re: (Score:2)
Re:processing power (Score:4, Insightful)
Remind me again... (Score:3, Insightful)
Combine this with the weight-shifting capability of the Fit, and you've got an immersive gaming experience that's second only to the holodeck.
So. Freaking. Cool.
Re: (Score:2)
Just because somebody does cool stuff with a Wiimote on a *PC* doesn't mean you will *ever* see something like this on the Wii. Remember this is homebrew stuff, not 'cool new games' that Nintendo will be releasing next year. The issue isn't that you couldn't have fun with the Wii, but that Nintendo really doesn't release much interesting stuff.
Re: (Score:2)
Really? I think the Wii is dominating this generation because they're the only company that did do something interesting.
Re: (Score:2)
No, what you need is Nintendo willing to do something beside dumbed down casual gaming. All the theoretical cool things that you could do with the Wii means nothing as long as Nintendo can't even get trivial core issues fixed (crippeled online multiplayer, lack of support for play-from-SD, lack for USB storage, lack of proper Wiimote calibration for lightgun shooters, etc.). At this point in time I have
Re: (Score:2)
Yeah, thats what we call crippled.
### Why would you need play-from-SD or USB storage?
Because a ton of people have exhausted the build in 512MB storage (VC games are huge) and also because there simply isn't a good reason why not. When you already support SD, why make it a second class citizen and not allow the same things you allow for build-in storage?
### As for proper calibration...are you trying to play while lay
turning your head. (Score:1)
Re: (Score:2)
Remember the PowerGlove? (Score:2)
It's been done. Remember the Nintendo PowerGlove [wikipedia.org]?
Incidentally, if you've never tried gloves-and-goggles VR, it's cool for about ten minutes. Trying to do things by making gestures in the air is a huge pain. Without tactile feedback, it's tiring and inaccurate. I tried most of the VR systems in the first round, including Jaron Lainer's original system. No good.
It might not suck if the system had an end to end lag of under 10ms. "Turn head, wait for view to catch up" systems drive the user nuts. T
Re: (Score:2)
Apple confronted this with the iPhone touch screen, IIRC, and solved it by having the phone vibrate ever so briefly when a touch was registered. This gave the sensory impression of a button clicking without actually doing so.
I don't see why a developer couldn't do something like this with VR gloves, then. Using JLC's approach, you'd have "Minority Report"-style gloves that had infrared ref
Re: (Score:2)
TrackIR (Score:1)
http://www.naturalpoint.com/trackir/ [naturalpoint.com]
Johnny Lee Rocks! (Score:5, Interesting)
This will likely spur an avalanche of Wii hacks, and could easily cause wiimote sales to go thru the roof..
I'm totally enjoying the adventure Johnny!
Combinations? (Score:4, Interesting)
You don't need a Wii or the remote. (Score:3, Insightful)
Potentially you could just use a webcam with an IR filter in front of it instead of a Wii remote.
Note: 1) there is usually a filter to filter out IR inside most webcams, so that would have to be removed. 2) IR emitter tracking would have to be done on the PC instead of inside the Wii remote.
Re:You don't need a Wii or the remote. (Score:4, Informative)
Re: (Score:2)
* It would have been nice to provide the raw camera data on request, unless it would cost more bandwidth than bluetooth has available. It's something of a happy accident that the Wiimote applies a blob tracker pretransmission (presumably to save bandwidth / power).
* 8 bits is a bit small for a accelerometer. At 12 bits, you can quadruple the precision and the range. The chip they went with was probably chosen because it's cheap, an
HOLY SHIT (Score:4, Insightful)
Why just limit this to games? (Score:3, Informative)
If you had several monitors, this could be used to make them feel as if they were an actual "pilots seat" of a vehicle giving perfect perspective to the "pilot" because they know where the head is oriented and each monitor could produce the proper peripheral and image views for the "pilot"
It would take a little tricky camera work for the robotic vehicle, but I am sure gratuitous funding could solve those problems.
Re: (Score:2)
These things already exist for military application, but its still neat to do it at home.
Why a wiimote? (Score:2)
Re: (Score:2, Insightful)
Split Screen? (Score:3, Interesting)
Re: (Score:2)
This guy is my hero (Score:2)
Johnny Lee (Score:2)
not new, but new implementation. (Score:2)
It's neat, but, big deal.
similar things with python and an ir-webcam (Score:2)
it allows for very similar things.
you can find the code here http://snippets.dzone.com/posts/show/4920 [dzone.com]
be warned, it has no documentation and its really messy.
the included demo is able to track and plot the movement of multiple IR emitters. I have other demos as well like "swing" detections (ie. swinging motions). A more elaborate hack involved using a wireless mouse+LED to simulate a wiimote experience (moving around the mousepointer and c
demo video (Score:2)
http://www.youtube.com/watch?v=UGImvHEtUpE [youtube.com]
One issue left to tackle... (Score:2)
So, I've been thinking... how do you address the click/scroll stuff and the position tracking, all using o
Re: (Score:2)
One possibility... a 360 spherical projection screen where the user remains in the center, lined with
Re: (Score:2)
Re: (Score:3, Informative)