Handwriting Recognition on DS

JamesO writes "Zi Corporation has announced a licensing agreement with Nintendo that will allow developers to make use of handwriting recognition. PDAs have been offering handwriting recognition for some time and with the DS's touch screen it seemed inevitable that the console would eventually gain handwriting recognition technology. An agreement between Zi Corporation and Nintendo means that DS developers will be able to utilise Zi Decuma handwriting recognition technology when creating software for the handheld."

SDK

[kevin_conaway (585204)]

What kind of SDK is available for the DS? What language(s) can you use?

On a side note, are there any phones / pdas that have a Python sdk available?

Re:SDK

[RevAaron (125240)]

You can code in Python for Windows CE 3.0+ (2.11+ maybe?), and specifically Pocket PC 2000, 2002, Windows Mobile 2003, 2003 SE and presumably 2005. There is a subset of Python, called Pippy, available for Palm OS. The WinCE port is pretty much a full port of Python; Pippy is very much stripped down. There are also the Zaurus Linux PDAs, which can run Python, though it's not as useful for writing full-on apps as it is on Pocket PC, at least within the Qtopia GUI last time I checked.

Perl too on PocketPC/WinCE; there's even Perl/Tk support. Works pretty well.

Re:SDK

[RevAaron (125240)]

And this answers the parent's question how exactly?

Seems pretty obvious to me. At the end of the post he asked:

On a side note, are there any phones / pdas that have a Python sdk available?

And I then told himabout Python support for two important PDA and Phone OSes, Palm and CE. Incidentally, there's also a Python SDK for Series 60 [nokia.com] Nokia phones. So make that 3 important PDA/Phone OSes.

Re:SDK

[Paladine97 (467512)]

A pretty basic SDK is provided. C and C++ are your options. Feel free to compile Python for it!

See www.dsdev.org [dsdev.org] for all the DS development info you need.

AIN'T nuthin nobody can ever do . . . . .

[craker (919952)]

No matter how well you develop this thing, AIN'T nuthin nobody can ever do to figure out my chicken scratch. I can't even read it right after I write it. They made keyboards for a reason - geeks can't write.

Eat Up Martha

[by Anonymous Coward]

cue the Newton fanboys...

Eat Up Martha

[thatguywhoiam (524290)]

This certainly makes the DS more interesting to me (not that I'd use it as a PDA or anything). But if you can jot notes into the thing, and have it OCR'd for you, it would make it a lot handier than it is right now. Can anyone comment intelligently on how the DS CPU would handle such a thing?

Re:Eat Up Martha

[kevin_conaway (585204)]

Great Simpsons reference. That always makes my day.

Just a nitpick...

[brunes69 (86786)]

Handwriting recognition != OCR, since there are no optics involved.

Re:Just a nitpick...

[CastrTroy (595695)]

essentially they represent the same program. Most OCR programs can analyze any image for text, whether you scanned them, got them from a camera, or drew them with you mouse in mspaint. It's essentially the same problem. When you write on the screen, it creates an image, and analyzes the image to see if it can recognize anything as text. Most OCR programs only do typed text well. Recognizing hand writing is much too hard to do reliably.

Re:Just a nitpick...

[poot_rootbeer (188613)]

Not necessarily. More advanced handwriting recognition algorithms can derive not only from a completed glyph, but also from the series of strokes you use in the process of drawing it.

Try writing out the numeral 5, and then the letter S. Notice that even though the end results may look largely similar, the velocity and direction of your pen as you drew them were considerably different.

Not the same problem at all.

[brunes69 (86786)]

While both involve analyzing glyphs, one involves extracting a glyph from an interpolated image (a scanned document or simmilar), while the other has the benefit of having direct digital input.

The prior is a different problem to solve. The hardest problem with OCR is reliably differentiating between a letter and a non-letter pixel on the page. Once you have the pixels that are just the letter, it is usually simple to figure out what letter it is. This is the idea behind Captchas, to make it as hard as possi

Re:Eat Up Martha

[RevAaron (125240)]

The NDS main CPU is pretty slow- 66 MHz. I've used Decuma on Palm OS on a 200 MHz ARM (Sony Clie NX70v), and used Decuma fine, albeit a bit slowly, with the Clie underclocked at 100 MHz to save power. Decuma is decent HWR; it is in between the character-based stuff like Graffiti and the proper and good word-sentence based real HWR of the Newton or PocketPC's Transcriber or Calligrapher. It makes you write in a box- but you write a full word, it recognizes it letter by letter, and then you have to press a button to actually accept/write the text. Or make corrections, overwriting maybe 'e' for the misrecognized 'o'; then you press that button. On Palm OS, Decuma is about the best you're going to get if you want real HWR, but it isn't too horrible.

Re:Eat Up Martha

[poot_rootbeer (188613)]

The NDS main CPU is pretty slow- 66 MHz.

Still - that's three times the clock speed and several ARM revisions better than the Apple Newton. Plus the DS has a secondary processor that might be usable.

Assuming there's been a modest improvement in HWR algorithmic efficiency over the past 10 years, I would imagine that the DS could do an admirable job.

Re:Eat Up Martha

[thatguywhoiam (524290)]

Still - that's three times the clock speed and several ARM revisions better than the Apple Newton. Plus the DS has a secondary processor that might be usable.

Several ARM revisions, yes; 3x clock speed, don't think so. IIRC the Newton ran somewhere around 160Mhz. Which I remember being very impressive at the time. :)

Re:This is very doable

[Psykechan (255694)]

The (Newton) Message Pad 2x00 processor was a 162MHz StrongARM 110 but the older ones were only 20MHz ARM6. Incidentally, the MP120(2.0) and the MP130 ran the newer OS (same as the 2x00) that was very capable of decent handwriting recognition unlike the original lineup. Having extensively used both a MP120(2.0) and a MP2100 I can attest to this. The 2100's only advantage was speed.

The DS has a 66MHz ARM9 and a 33MHz ARM7. Logically, unless the ARM7 is needed for some specific DS tasks you could have it

Re:Eat Up Martha

[prockcore (543967)]

The NDS main CPU is pretty slow- 66 MHz.

Not that slow. The Palm III had a 16Mhz cpu. At the very least the NDS should be able to handle Grafiti :)

Extra lives?

[Krazyweasl (925959)]

This is going to make 'up up down down left right left right b a b a select start' a pain to enter

Finally

[Edunikki (677354)]

Now I can trash talk other people in multiplayer mode!

Re:Finally

[JonXP (850946)]

The DS has a microphone built in. Theoretically you could already do that with your voice, unlike another popular handheld system.

Vorks evem b3ttar than...

[by Anonymous Coward]

...my Prlm.

Animal Crossing DS

[vodevil (856500)]

I sure hope they put this to good use in Animal Crossing DS, it was a pain in the butt to use the controller to write letters to the villagers.

Re:Animal Crossing DS

[Allison Geode (598914)]

based on screenshots i've seen, animal crossing DS will use a software keyboard. however, this will still be much less annoying than the original, since, of course, you'll use the stylus to hunt and peck, rather than d-pad and A-button (as seen in the original gamecube version.)

Re:Animal Crossing DS

[Burpmaster (598437)]

I just hope they'll stop using that idiotic alphabetic order and give the software keyboard a qwerty layout.

The GameCube version does have a QWERTY layout, and it's the default! (press Z to toggle between QWERTY and alphabetic layouts)

Expanding their demographic

[evil agent (918566)]

Nintendo will now be marketing this to very young kids, once games come out that can teach kids how to write. Of course, this will depend on how good the handwriting recognition will be. This could be really good news for the future of penmanship, or really bad.

The loss of Penmanship.

[nlinecomputers (602059)]

This could be really good news for the future of penmanship, or really bad.

Would that really be so awful. My penmanship was never good to begin with but I find that I so rarely sit down and write with a pen that the skill has badly deteriorated. More so with cursive then printing. I'm not sure that it is a skill we badly need any more in modern society.

Re:The loss of Penmanship.

[mrgreen4242 (759594)]

I was going to moderate this topic, but wanted to post about the difference between the ability to write and the skill of penmanship. They are not at all the same. Being able to write simply means that you know what a letter looks like and are physically capible of holding a writing utensil with enough dextarity to form that symbol.

Good penmanship, on the otherhand, is almost an art. It's a step below calligraphy, and a step above your average scrawl. My HANDWRITING is horrible, but I can still write; I hav

Re:The loss of Penmanship.

[nlinecomputers (602059)]

Yes I'm serious. I personally don't interact with anyone in a third world nation. I am hard pressed to think of but a small handful, perhaps 3 people, that I personally know that DO deal with people in third world nations. And they maintain their contact via email. Perhaps YOU have the jaundiced viewpoint of everyone in Africa is living in thatch roof huts and trying to plow fields of wheat with an ox.

Yes I'm sure that they do exist in the world but from my small world view, the view I have to deal with

Dictionary

[dancingmad (128588)]

There's already a Japanese-English dictionary for the DS, but it's so so at best. A good handwriting system for the machine would be an incredible boon - often times I'm presented with a kanji I simply don't know the reading for (and I can't input it into my electronic dictionary's QWERTY interface). I do know enough kanji to be able to copy many down by writing them, so being able to write say a compund and having the DS spit out a list of possible readings and defintions would be amazing and would help me learn Japanese in the real world (here in Japan anyway) more easily as I could begin decoding stuff in the world witout need of my onerous New Nelson Kanji dictionary.

Re:Dictionary

[Elias Ross (1260)]

I bought a Sharp Zarus in 1997 for about $200 (in yen), which had fairly reliable handwriting recognition for kanji. Kanji I think are even easier to recognize than kana. Although they are more complicated, many can easily distinguished by stroke count and stroke direction alone. Some kana are fairly difficult to tell apart. I had an easier time with kanji than English letters, which have no standard stroke pattern.

I recently got the same dictionary you did for the DS and I had assumed it would have kan

Re:Dictionary

[earthbound kid (859282)]

A Canon Word Tank costs as much as or more than a DS plus cartridge, and it can't play Mario Kart.

Besides, I'm a Sharp-man, anyhow. ;p

Re:Dictionary

[dancingmad (128588)]

I have a wordtank i bought at ks (the baby c30) but its easier to write in kanji then to try and look them up by radical.

I For One

[Paul Slocum (598127)]

Welcane aur new hondwriting recogmition ouerlonds!

Pen is Mightier

[Doc Ruby (173196)]

I've really been disappointed by the tiny progress in using a stylus for input. A decade and a half ago, "pen computing" was "the next big thing". Microsoft even marketed a "Word for Windows for Workgroups for Pen Computing" edition (presumably with a "Spellcheck for Word for..."). The Palm Pilot actually delivered on the Apple Newton's promise to use our pen skills to deal with little mobile devices. But now even Palm devices, like Treos, usually disregard the pen. I think it's not so much the lack of writing recognition (which is more than adequate, though it does have problems), as the lack of any unique pen advantages to compensate for having to use a separate pen rather than an integrated keyboard.

Pens offered an opportunity to use an expressive, intuitive gestural interface. Even mouse gestures have run circles around pen gestures. I'd like to use a pen to indicate multiple selections, associations, layouts, flows, scales, shapes. I think an interface that used chinese symbols as commands on selected objects would have tremendous popularity, and maybe even work with a huge new global zeitgeist that could jump all kinds of boundaries represented by keyboards, especially QWERTY.

We still have the opportinity to use pointers for a really expressive, simple interface "for the masses". I built a "light pen" for my Atari PC over 20 years ago. Even Treos still come with styluses, and now the DS will recognize handwriting. Most people use pens, probably even more than keyboards, especially worldwide. That input mode isn't going away, even if it's not being pushed. Even though OSes and apps still haven't delivered on their potential, there's still lots of pent-up (pun intended) demand to use them. I don't think the breakthru lies in dropping the pen in favor of a fingertip, though I'd like to see some working software that tested that avenue. I think that once we get a pen-centric UI paradigm that does things keyboards and mice cannot, we'll get pens that people won't put down.

Re:Pen is Mightier

[Space Cow (93479)]

I'd like to use a pen to indicate multiple selections, associations, layouts, flows, scales, shapes. I think an interface that used chinese symbols as commands on selected objects would have tremendous popularity, and maybe even work with a huge new global zeitgeist that could jump all kinds of boundaries represented by keyboards, especially QWERTY.

Really? Care to elaborate? Are you suggesting that if I want to increase the size of an image in my word document by 50% that I should enter the 20 or so stro

Re:Pen is Mightier

[Doc Ruby (173196)]

I'm suggesting that tapping the image and scrawling probably a 3-stroke character (like the "big" [hmarty.free.fr] character), without dropping the stylus for the mouse or keyboard, is a good UI. If the gesture strokes are going to be a character, why not use characters that are already understood by literally billions of people?

Re:Pen is Mightier

[WillAdams (45638)]

Well, it depends.

While the overall UI leaves a bit to be desired (PenPoint was much nicer than Windows for Pen Computing 'cause it was built from the ground up as a ``pen-centric'' UI, while Windows for Pen Computing grafter pen interaction on as an afterthought) and Windows XP Professional Tablet PC Edition continues in this vein, there are some really brilliant UI designs / concepts:

- Alias Sketchbook (pie menus done right --- any tool / option is a pen flick away)
- Ambient Design's ArtRag

Re:Pen is Mightier

[Doc Ruby (173196)]

I spent a little time with GRiD a decade ago, and hoped they'd leap to a new paradigm. Some of those GUIs you mentioned suggest one.
<DESIGN-RANT>
I want to see a GUI that is entirely devoted to representing data without boundaries, without onscreen "applications". I want each MIME type identifiable and associated with consistent GUI elements for their specific production (creation/editing), consumption (reading/playing), searching/browsing, sending and quoting operations. With multiple background proce

Re:Pen is Mightier

[WillAdams (45638)]

What I really want to see is an app that allows one to identify discrete bits of data in text stream and then query it in an ad hoc fashion much as you obliquely describe.

William
(who remembers his GRiD laptop with great fondness.)

Re:Pen is Mightier

[Doc Ruby (173196)]

I think most "pen" operations could be one-handed, as I've generally experienced with my Palm Pilot, if the software were better able to indicate the "hotspot" with fingertip motions. I'd start with an onscreen cursor crosshair moved just above the sensed fingertip, but I'm sure years of R&D, with mass market feedback (like keyboards, mouse and even styluses have gotten) would evolve an even better interface.

Keyboards are faster for inputing alphanumerics, but most operations, especially of mobile devic

DS Training For Adults

[pnice (753704)]

DS Training For Adults can already recognize when you write numbers with the stylus. I'm sure it's much easier to do but it still does it on the math part of the "game".

Decuma

[Rydia (556444)]

It would help if the people disparaging handwriting recognition had actually used Decuma. Right now my biggest problem with my PDA (Dell Axim WinMobile2003) is that Decuma doesn't support Mobile 2003 devices, so I can no longer use it. For both Japanese AND English, the software is spot-on. Right now I'm using a hacked version of the Japanese IME that a very clever hacker got working for Japanese and the standard transcriber, and my note-taking (in english) has been slowed down immensely with transcriber ov

Decuma

[arodland (127775)]

I've used Decuma's handwriting recognition software on my Palm, and it's a pretty competent system, didn't take very long for me to get used to. The main reason I ditched it was screen real-estate; I've got a "square" Palm rather than one with Virtual Graffiti, so I couldn't justify giving up that much space when the Graffiti pad was already available. That shouldn't be too much of an issue on the DS, of course, because anything that uses handwriting recognition can be expected to dedicate the touchscreen f

Re:Nintendo's next R rated game

[DwarfGoanna (447841)]

A friend of mine (who is obviously slightly obsessive) became scary-type addicted to Typing of the Dead (or whatever the DC zombie typing thing was called). This could have been the thing to put him over on DS. =)

Re:It better be good

[DigiShaman (671371)]

My handwriting ranks the worlds worst. If Nintendo needs beta testing, they should ask for samples of my writing. Then, I would give them the discription of what I wrote in typed format.

Ya, it's bad. I'm afraid I would freak out the CPU and cause it to shut down in a fit of fear.

Re:Nintendo is Desparate

[bugbeak (711163)]

The Nintendo DS is falling behind in sales against the superior Sony PSP.

From a Gamespot article: [gamespot.com]

"Recent sales figures provided by Dengeki Online revealed that cumulative shipments of the DS have nearly doubled those of the PSP in Japan since both launched in the country last December. As of the end of September 2005, Nintendo sold 3.2 million units of its DS handheld in Japan, while Sony Computer Entertainment shipped only 1.7 million units."

. Dependency on Cartridges: The age of the game cartridge has come and gone. The overwhelming success of the Sony Playstation and its "intellegent" use of CD-ROM based media effectively killed the cartridges as a viable media format for software.

Maybe for home consoles. But do you really want something portable that has moving parts? No matter how damn hard you try, you're going to treat anything handheld pretty badly.

3. Lack of Wi-Fi Internet Connectivity: The DS only allows for multiplayer games via its wi-fi connection with oher DS units but users cannot access public wi-fi drops like with the PSP and different PDA models. Because of this it would be impossible to sync data on the DS and a PC or another PDA because of this design.

And just how much more convenient is it to browse the web with a PSP? Ever heard of something called "input"? Besides, what's McDonald's for? Or the 50-dollar Mario Kart/Wi-Fi dongle package.

4, No Support for Storage Media: The DS does not natively support any form of flash media cards for storing data.

Just how "universal" is the so-called Universal Media Disc? Seen any blanks on sale recently?

5. Dominant Childen Marketing Angle:(not even going to bother with this one...)

Quantity doesn't really matter if Nintendo is the only one who can stay in black. So what if games are aimed for chilren? I don't think adults have gone "Eww" at Mario Party.

The company is too slow to change and adapt with the current market and current technology.

Who came up with Rumble? Who came up with the analog stick? Who came up with the shoulder buttons? Rather, just look at the fscking Revolution controller.

It's flaimbait, it's troll, but I bit.

Re:Nintendo is Desparate

[juched (926032)]

I actually created an account on slash dot just to reply to this initial posting. It seems to me that everyone of his/her comments was inaccurate. The DS is outselling the PSP. Cartridges load much faster and hence the DS does not have the crippling load times of the PSP. The wireless is standard 802.11b and hence it can connect to the PC if the software was written for it. And yes, the DS can play only, as we will all be doing when Mario Kart comes out! Don't even get me started on keeping up with current technology. Nintendo invents the new technology... The examples of analong stick, shoulder buttons, rumble and more are all good examples of that. Just because the didn't use DVD or CD? They might be easy to create, but they are also just as easy to copy! No wonder they are trying to stay away.

Also not cartridges...

[Solr_Flare (844465)]

They are not traditional RAM cartridges like the GBA and n64 and past console systems. The DS uses a proprietary Flash memory media that is currently capable of 256mb of space but expandable to 1gb of space. That's actually quite a lot of space and my understanding is it is relatively cheap on Nintendo's end to produce.

If the current market trends and price drops are any indication as well, Flash media or an equivilant will likely be replacing optical media like CDs, DVDs, UMDs, etc in another generati

Re:Nintendo is Desparate

[amdotaku (909214)]

It's pretty clear that Nintendo has somehow acquired a kiddie image. However, when I actually look at game lists for the PSP and DS and look at titles that at least have a hope of being "good", I don't really see much of a distinction between games that would be appealing to mature audiences between PSP and DS. If anything, the DS matches count and adds greater variety to the mix with games like Advance Wars that the PSP, sticking to more stereotypical genres like plain shooters and rpgs, just won't offer.

Re:Nintendo is Desparate

[TrekCycling (468080)]

The funny thing is that the GBA is often second. This should tell the PSP fanboys something. Real gamers like to play games. The PSP has a great install base with those who like to pirate games, watch movies on a 4 inch screen or listen to mp3s on a device that weighs about 12 times as much as an iPod Shuffle. But for those of us who just want to play good games, Nintendo is where it's at in the portable market.

I use Linux as my desktop at home. I have an XBox and a Pocket PC. So I'm no idealogue. I buy wha