Tracking Gaming Stats With Video Capture Devices

galtish writes "M. Schrag has put together an amazing PC stat tracking system called 'Soul Calimeter' for use with the GameCube version of fighting game Soul Calibur II. The software is as yet unreleased, but uses a cheap video capture card to analyze the video feed from the console and create a stat database. It's not just stats, there's also a web browser-based front-end for analyzing the stats and starting circuit matches, and voice synthesis using AT&T Natural Voices for in-game commentaries on the action. His website includes pics of the stats screens, and samples of the synthesized audio commentary. Pretty sweet - I'd love something like this for the weekly Halo LAN matches..."

Mirror

http://www.schaefer.nu/mirrors/homepage.mac.com/ms chrag/SoulCalimeter/ [schaefer.nu]

Lot of work

It's really too bad they had to jump through so many hoops to get this to work.

Nothing like making a developer's life hell by making them interface it like this.

Re:Lot of work

In the (sadly few) posts in this story I see an amazingly underwelming response to this achievement which, personally, I find rather amazing. Maybe it's because I've worked with image processing (CISP at Lockheed Martin during college) and appreciate how damn difficult reliable image recognition is, even when recognizing sub-images from a fixed (but still decently large) selection of image inputs.

IMHO this is one of the most clever and thorough hackings I've read about on /. in months. This is excellent work to solve a technically difficult problem in an unorthodox (and risky, in terms of complexity) way, and they do it with style (the commentary stuff is mint) and extensibility (the API is very open FWICT.) The only thing I lament is a lack of more detail (maybe that's the problem?), but I think the site has switched to simple static pages to minimize the slashdot effect (though probably not needed in this case) and I expect to be able to find more details on a later visit.

And I'm afraid I must disagree with your notion that somehow it is a shame that it was so difficult to do. Necessity is the mother of invention, and here it shows (and pays off) in spades. What, are we supposed to rally against Nintendo (and all console makers, for that matter) for failing to provide a port with some open API to scan internal game variables? Frankly, I'd be way less impressed were they to in fact do that by either hacking onto the motherboard electrically, or worse running the game on an emulator (MAME or such) and peeking at RAM.

Nothing like making a developer's life hell by making them interface it like this.

Eh? These are not "developers" and no console maker expects antyone to want (much less implement) anything like this! They are hackers!

Again, I just want to say that I think this is one of the most underrated and under-commented yet excellent stories I've read here in a long time, and I plan to revisit the site later to learn more. This may be applicable to many other things from security cameras and home automation to MythTV-type device commercial skipping.

Then again, maybe I just don't realize how easy it is to do real-time image processing with object recognition in a reliable way using cheap under $50 capture cards on a 1GHz Athlon these days. Were that the case, though, I'd expect to read more about such efforts. This is a first for me.

Thanks for reading. And no, I do not know the authors(s) and I've never heard of this until now.

Yay! A /. story that's interesting

While his project is altogether useless and probably a complete waste of time, the site is well-documented, well written, and the project is very high in hack-value. More stories like this, please!

Obviously this is just the first step of this project. It looks like they can identify characters and health meters, as well as winners and losers, however the next obvious step is to identify moves on-the-fly so that a running commentary can be played. Also, since they are running video capture HW/SW, they could even have their epic fights saved and played back with computer commentary. Now that would be geeky!

Dudes, I want this for my PC!

Running commentary on everything that happens in the office? Priceless!

"Hey, man, another coffee?"

"Hey, Aztec, you scored good in that meeting. Give ya a kickdrop and a sideswing. Need more stamina though..."

"Hey, Sandy, nice skirt you're wearing there! You already got ten eyeballs."

"Hey... WTF!? You can't unplug m-/$!

Camera

Don't point the camera at games.slashdot.org . . . it might blow the PC up!

Simple Awesome

I've never imagined that a 1ghz computer could capture in real time stats from a video feed. This is amazing.

Pretty cool

Reading the descriptions, it watches the health bars and the squares at the character select screen. Which would be simple enough image detection, since they're always in the same x,y coordinates.

It would be really cool if it could grok the movements of the fighters and say things like "Kilik delivers a fierce scissor-kick to an uppercut combo!".. But you cant have everything.

Shoot!

I must turn off my PS2 for playing Strip Poker!

Time?

I know some people who have WAY too much time on their hands..

Should do this for online poker

What would be convenient would be something like this, but for playing poker for $$$ online. So the computer can advise (or even play for you).

"I advise you fold this marginal hand."
"There is a 45% chance that this opponent is bluffing."
etc.

Halo

This is a great idea. We were just thinking the other day about a video capture based stat tracking system for our Halo LAN matches. Sometimes large portions of our potential Halo-playing time are taken up by arguments about team balancing (AKA "team wanking"), but if a computer could analyze the stats and make balanced teams for us we wouldn't have to worry about that anymore. Stats on who's best/worst at killing who, and with what weapons, would be really interesting. Plus the automated trash talking would be great fun! Having a robotic voice affirm your greatness after you get a triple killtacular would just be that much more awesome :-)

I am over-fucking-welmed!

This is a truely breathtaking impressive hack. By people who've been playing Soul Calibur for more than 3 years every day. The last part would actually ask for some stupid jokes, but as somebody who writes software for a living I must say I'm really impressed by this cool piece of work.
And I really don't wanna compete with those guys in Soul Calibur. :-)

Coooool.

Wow. As an avid Soul Calibur fan whose roommate is always talking trash, it'd be nice to have some hard stats to back up the asskickings.

If these guys don't make any movements towards sharing this with the general public, I very well may have to work on hacking up something like this myself.

Geez . . .

If only they had bigger breasts. I'd buy it then.

Oooh really?

His website includes pics of ...

I'm giving it 5 minutes.

try the natural voices

If you have never heard what AT&T Natural Voices sound like you should listen to the demos [att.com]. I first discovered these over two years ago and although they haven't changed that much quality wise, they are the best synthesized voices I have ever heard.

-- paper

Why cheap out?

A quality capture card that doesn't suck is the Winfast TV which runs about $60. It has no problem capturing 640x480 30fps DVD or higher compression video on a decent system.

No point "saving" $35 to buy a piece of junk that has virtually no uses except for one application.

Ben

Gaming sluts?

I first read the title of this as "Tracking Gaming Sluts with Video Capture Devices"

But then again, I guess that title works too, huh?

Just What I Needed

Documented proof of my ass getting kicked.

I'm sorely disappointed

You're writing software to measure Soul Caliber. WHY isn't it called Soul Caliper?

This is incredible

It's really, really cool. We need something like this for our games!

It's too bad it doesn't look like it can detect Guard Impacts, which can turn a fight around. (A Guard Impact is when you press toward or away from the opponent and press the block button the instant a hit would strike, causing you to parry or sidestep the attack and basically giving you a free hit on your opponent -- unless *he* Guard Impacts, that is.)

Go and play around with the voice synth

GOD I WILL KEEP THIS ONE IN MY BOOKMARKS FOR A LONG TIME!!! The link to ATT'S site for the voice synth mentioned, its great to make it say funny things like "smoke crack mate, crack is good!" in uk english. The address for that page is http://www.naturalvoices.att.com/demos/ HAVE FUN!!!!!

So close....yet so far

Soul Calimeter? That's awful.

Soul [b]Caliper[/b] would have been a much better name.

Re:UT2K4

So does Slashdot, UT2k3, Unreal Tournament, certain servers for Counter-Strike, Warcraft III, all MMORPGs and most other MMOGs, and Starcraft. Guess what, they're all PC games.

If a console game manages to do this, its newsworthy. You know why? Because it hasn't been done before.

Re:UT2K4

Except that UT2K4 stat tracking went down about a week after release. It came back for a few days but AFAIK it is still down, last updated matches were dated March 23rd.

The exact same thing happened with UT2K3, buit in that instance the stat tracking went down for 6 months!!!!

They also had it in the first unreal Tournament, and that worked very well. It was very useful for analysing clan games after completion.

I only hope that they can restore the stats to UT2K4 in a timely manner