Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!


Forgot your password?
Graphics Math Games

Upscaling Retro 8-Bit Pixel Art To Vector Graphics 325

An anonymous reader writes "Two researchers — Johannes Kopf from Microsoft, and Dani Lischinski from The Hebrew University — have successfully created an algorithm that depixelizes and upscales low-resolution 8-bit 'pixel art' into lush vector graphics. The algorithm identifies pixel-level details (original paper — PDF) to accurately shade the new image — but more importantly, the algorithm can create smooth, curved contour lines from only-connected-on-the-diagonal single pixels. At long last, we might be able to play Super Mario Bros. on a big screen without stretching our beloved plumber's pixels to breaking point. You really must look at the sample images." Scroll down in the paper to see how their technique stacks up against some others, including Adobe's Live Trace.
This discussion has been archived. No new comments can be posted.

Upscaling Retro 8-Bit Pixel Art To Vector Graphics

Comments Filter:
  • Wonderful! (Score:4, Interesting)

    by ModernGeek ( 601932 ) on Tuesday May 24, 2011 @11:21PM (#36235720)
    I've always thought that this would be possible. Now we just need to create a translator that will let us play our retro games with vector graphics.
    • by AmiMoJo ( 196126 )

      An interesting concept, remastering old games. Sega started doing that a few years back with graphical updates of their classic games for the PS2, but I think they were only on sale in Japan. Monkey Island had this treatment recently too, and you can switch between original and updated graphics at any time in the game.

      I think some games lend themselves to being updated because their graphics were hand painted and digitised. A lot of point-and-click adventure games were made that way. Other games' graphics w

    • by EdZ ( 755139 )
      There are already emulators for the few consoles and arcade games that used actual vector monitors (e.g. Vectrex).
      This filter looks quite similar to the existing HQnX series of filters, but with even more aggressive smoothing (from the paper, they appear to use a similar quantisation matrix approach then vector-smooth the result). As the paper shows, the results are fine for a few cases of a single isolated sprite, but for a whole display it looks really nasty, even worse than HQ4X [imageshack.us].

      My personal preference
  • PDF got slashdotted immediately, but the dolphin image shown in the first article is quite stunning.
  • by hal2814 ( 725639 ) on Tuesday May 24, 2011 @11:25PM (#36235738)
    Because we haven't strip-mined the past enough. I'm impressed by the results but weary of its implications.
    • I was originally going to correct you for misspelling "wary" (characterized by caution; guarded; careful), but then I realized "weary" (expressive of fatigue) arguably made more sense in context (as in, you are tired of the things this implies will happen, ie. more remakes). So now I'm just asking which one you meant.
    • One of the things holding me back from forking out the 100$ for the XBOX360 XNA license was my inability to make nice graphic images larger than your traditional ancient sprite types. (I am pretty good at making sprites, or at least I used to be. Been awhile since I have dabbled with it.)

      I feel I could do some fantastic things with an algorithm like this.

    • Re:Great. (Score:5, Interesting)

      by iamhassi ( 659463 ) on Wednesday May 25, 2011 @12:07AM (#36236018) Journal
      You're kidding, right? This is allowing us to play all those games from our youth without them looking like blocks at 1080 HD. If anything this is helping the past, no longer will they need to release a "remake", they can re-release the original with this algorithm attached and they're done.

      Besides, do you think 1080 HD will be the resolution of choice 20-40 years from now? I imagine we'll be looking at wall-sized TVs at some point and 1080 pixels will look awfully blocky on a 10 foot wall. Sharp has already released a 7x680 x 4,320 pixel 85-inch screen, [mashable.com] how long before 4320 is the new 1080?
      • how long before 4320 is the new 1080?

        Not too damned long. At a recent show, multiple panel manufacturers were showing off both 7" and 10" panels at 2560x1600. Nine of those 10" panels glued together gets you a very sweet 30" display at 7680x4800. Next year. I don't want to think about the price, though.

        • Re:Great. (Score:4, Informative)

          by Rei ( 128717 ) on Wednesday May 25, 2011 @03:13AM (#36236614) Homepage

          The maximum theoretical human eye resolution (limited by aperture and focal length) is about 0.35mm per line pair at 1m. For most people with "normal" eyesight, the limit is closer to 0.7 mm at 1m. So for a TV that's 6 feet away and 2 feet tall, there's no point to a vertical resolution higher than ~1800 pixels, and 1080 is more than what most people can perceive. 4320 pixels at 6 feet makes no sense unless your TV is a minimum of 5 feet tall, and for most people, 15 feet ;) So yes, it really does take wall-sized TVs for that sort of resolution to even be useful. A 7" panel (~4" tall) with 1600 pixels vertical resolution? That's 0.06mm per pixel. The maximum theoretical distance in which that could be useful is under 7 inches, and more practically, closer to 3 inches away from the viewer. If you want that kind of resolution, you might as well make it into a planetarium ;)

          • by AmiMoJo ( 196126 )

            You are assuming that the person stays perfectly still. When watching TV your head moves slightly all the time (breathing etc.) and this motion allows you to see a difference between resolutions above and below the thresholds you set out. It only works with high contrast edges though.

            You can think of it a bit like a camera that takes two photos. For the second photo the camera is moved 0.5 pixels to the left, so it is now sampling between the pixels in the first image and doubling its horizontal resolution

      • by Rei ( 128717 )

        I can easily picture taking this one step further. With a couple additional steps, your sprite drawer should be able to recognize matching parts in other frames of the same sprite. You could then allow a customizeable time period -- say, 1/4 to 1/2 second -- where there's a linear interpolation between the two SVG graphics. Aka, when Mario jumps, his legs don't just instantly appear apart; they move there. A smart drawing algorithm could even intelligently determine the appropriate time period to have f

        • by Mprx ( 82435 )
          Not going to work, because of latency. Some animations eg. Mario's legs are only decorative, but others, eg. an enemy preparing to attack, are important game information and shouldn't be delayed at all. There's no way to algorithmically distinguish between them so even the important information will be delayed 1/4 to 1/2 second.
      • by arth1 ( 260657 )

        You're kidding, right? This is allowing us to play all those games from our youth without them looking like blocks at 1080 HD.

        No, and no.

        The first statement is false because most of "all those games" have sprites and other graphics with hard angles, which this algorithm can't handle; everything becomes a spline. I'm sure there are people who would appreciate it if games got transformed so it looks like the fighters hit each other with soft nerf bats instead of swords, or teeth stopped looking sharp and scary. I'm not one of them.

        The latter statement is false because whether a bitmap image looks like blocks doesn't depend on the

        • Re: (Score:3, Insightful)

          I know "TFA" is rarely "R"ed around here but they use template-matching to preserve intentional hard angles. They use a guy with a sword as one of the EXAMPLE images!
  • Being able to convert low resolution pixel art into nice vectors like that is kick ass. I wish they had released a utility to try it out on other materials. (like my own..)

    I do some of my best work in a work envelope around 100x100 pixels.

    • IIRC, Inkscape had a nice raster to vector function that I had good luck with. I seem to remember it allowed for a bit of tweaking too, to achieve best results.
  • Paper mirror (Score:5, Informative)

    by SheeEttin ( 899897 ) <sheeettin@@@gmail...com> on Tuesday May 24, 2011 @11:30PM (#36235788) Homepage
    EvilHom3r over on Reddit seems to have mirrored the paper (as images) here [imgur.com].

    My verdict: Yeah, it looks all nice and smooth, but with all upscaling, it's basically interpolating data. The original just didn't have that much detail, so you can only get so much out of it. Sometimes it works, sometimes it doesn't.
    (Oh, and it makes all text look pretty bad. Kinda Comic Sans-y, if I can say that without invoking instant hate.)
    • by syousef ( 465911 ) on Tuesday May 24, 2011 @11:51PM (#36235928) Journal

      EvilHom3r over on Reddit seems to have mirrored the paper (as images) here [imgur.com].

      My verdict: Yeah, it looks all nice and smooth, but with all upscaling, it's basically interpolating data. The original just didn't have that much detail, so you can only get so much out of it. Sometimes it works, sometimes it doesn't.

      (Oh, and it makes all text look pretty bad. Kinda Comic Sans-y, if I can say that without invoking instant hate.)

      I know what you mean but it's pretty impressive nonetheless. The flaw I found interesting was in the keyboard image. I'm sure the intention here is square keys, but their algorithm made all the keys round. Some things in a pixelated image should not be smoothed but without human context that's a very hard call to make.

      • by arth1 ( 260657 )

        Also, the pins of the chip was a failure, becoming sawtooths instead of parallel pins.

        The problem with this algorithm, like many others, is that it's subject to an extension of the Pareto principle. In 80% of cases, it may be an improvement, but that doesn't help if it makes the result worse in 20%. Cause it's the failures we notice, and judge quality by.

        Algorithms like this also don't work well for animated sprites. Our imagination can fill in the gap between Mario holding the hammer above his head and

    • by Twinbee ( 767046 )

      With the doom face example, their algorithm falls down slightly. Basically, it's almost as if they have to decide between a sharp edge/curve, or a gradiated colour section, like as if the decision is a dichotomy. I'm sure the math is complex, but they should always strive towards the most general all-encompassing technique, rather than try to build more arbitrary 'stages'. A sharp edge could be said to be a gradient, but a very sharp changing one, and vice versa...

      • Re:Paper mirror (Score:5, Informative)

        by qubezz ( 520511 ) on Wednesday May 25, 2011 @01:54AM (#36236372)
        If you read the article instead of looking at the pictures, the authors point out that this is one situation where the algorithm fails - the method is optimized for edge-finding in hand-drawn 8-bit pixel graphics. In more photo-realistic images like the doom face, especially later games where the graphics were rendered in 3d/hi-res and imported into the game (Donkey Kong Country [blogspot.com] would be a good example of this), vectorization doesn't work well due to the smoothed anti-aliased graphics.
        • by Twinbee ( 767046 )

          Yes, but I can imagine many would try to solve the issue by adding a kludge creating more special cases, rather than approach the problem with the better goal of unifying the alleged difference between a sharp, or blurred edge. A matter of degree rather than type/quality.

          • by Rei ( 128717 )

            Looking at the images, I don't think it'd be that hard to detect the level of antialiasing in the image. In fact, they're already kind of doing it when they decide whether to blend areas or to leave them distinct. To me, that leaves the obvious solution: rather than a binary between "sharp" and "blurred" being determined by the level of color difference, you use the level of color difference (with a particular focus on matching hue) to decide the maximum transition distance. So, for example, a stark colo

      • They point that out in the article indeed, the face is added as an example of where it goes wrong. Their algorithm simply doesn't work well on anti-aliased images (scaled-down photos), it's primarily designed to handle typical hand-made sprites with much sharper edges and colour differences.

    • by Dracos ( 107777 )

      You denigrated comic sans by the transitive property of bad. No hate from me.

  • by Announcer ( 816755 ) on Wednesday May 25, 2011 @12:40AM (#36236138) Homepage

    It looked like it was on the verge of a Slashdotting, so I CORALized it.

    http://johanneskopf.de.nyud.net/publications/pixelart/paper/pixel.pdf [nyud.net]

    Very interesting, and quite effective.

  • This looks a lot like what you might end up with if you use the image tracing function in Flash to convert bitmaps into...vectors.

  • I redraw bitmaps as vector art as a little side business, and I have to say this is the best "livetrace" automated program I've come across. Shameless plug: Vectify.com [vectify.com]. Still no substitute for hand drawn (rounded corners that should be sharp; variable, wobbly line widths; but very good for an automated process). I've scanned through the PDF paper, and it looks like it's too CPU intensive to be used in retro games in realtime: from page 6, "There are many avenues for future work. Obviously, it would be n
    • I wonder if it would be feasible to cache these results.... by definition these sprites don't change appearance very often. If you could detect that sprite A is being drawn at point X,Y, and just draw the cached and pre-generated high-res vector art in the right location, would that be any more practical for real-time rendering?

  • The real test for the Kopf-Lischinski algorithm will be how it can handle Nethack [bbspot.com].

  • This looks good because it is specialized for 8-bit manually created graphics. 8-bit graphics with a broad palette has little shading, so it's usually unambiguous which pixel attaches to which adjacent pixel. Vectorizing 24-bit color images, or 8-bit images from photographs, requires more guesses about how wide things are and where the edges are supposed to be.

    Nice if you really want to play 80's games in higher resolution.

  • by anti-pop-frustration ( 814358 ) on Wednesday May 25, 2011 @03:35AM (#36236692) Journal
    That we like the way pixel art looks in the first place!

    This project is technically interesting but as art it completely misses the point. Not every piece of human creation needs to be updated, upscaled, "improved" and redone.

    I know the originals [imgur.com] make more sense to me.
  • by Keyboarder ( 965386 ) on Wednesday May 25, 2011 @04:42AM (#36236916)
    Zoom in on Mario a bit... Enhance...
  • by bertok ( 226922 ) on Wednesday May 25, 2011 @05:24AM (#36237052)

    NVIDIA and AMD should implement this and related algorithms for their LCD upscaling!

    The algorithm looks like it could be parallelized onto the GPU easily enough, and doing it transparently at the video card driver level would enable PC gamers to play classic games on large LCDs without the thumbnail sized pixels.

    Just imagine what this scaling algorithm could do for classic low-res games like Diablo.

    This is why closed-source sucks... I'd have done it myself by now if I had access to the AMD GPU Windows driver source code!

  • Can someone link to the MAME build that supports this? ....
    What do you mean they're not coding this directly into MAME source?!?!?!

  • by drolli ( 522659 ) on Wednesday May 25, 2011 @06:06AM (#36237164) Journal

    Commodore 64 Porn arrives in high-res

  • The image of the dolphin comes from Super Mario World, which was for the Super Nintendo. That was a 16bit game on a 16bit system. Anyone who played the 8bit Super Mario Bros games can tell you there were no dolphins in those games.

They are called computers simply because computation is the only significant job that has so far been given to them.