Please create an account to participate in the Slashdot moderation system

 



Forgot your password?
typodupeerror
×
Emulation (Games) Nintendo Programming

Beyond Emulation: the Massive Effort To Reverse-Engineer N64 Source Code (arstechnica.com) 61

Slashdot reader thereitis shares a report from Ars Technica that "delves into the reversing community's efforts to produce usable C source code from N64 game binaries." Here's an excerpt: Early this week, with little warning, the Internet was graced with a Windows executable containing a fully playable PC port of Super Mario 64. Far from being just a usual emulated ROM, this self-contained program enables features like automatic scaling to any screen resolution, and players are already experimenting with adding simple graphics-card-level reshaders, including ray-tracing, as well. The PC port -- which was released with little buildup and almost no promotion -- wasn't built from scratch in a modern game engine, in the manner of some other now-defunct Super Mario 64 porting projects. And its release has nothing to do with a recent leak of internal Nintendo files dating back to the Gamecube days. Instead, the port seems to be a direct result of a years-long effort to decompile the Super Mario 64 ROM into parsable C code. This kind of reverse-engineering from raw binary to easy-to-read code isn't a simple process, but it's an effort that a growing community of hobbyist decompilers is undertaking to unlock the secrets behind some of their favorite games.
This discussion has been archived. No new comments can be posted.

Beyond Emulation: the Massive Effort To Reverse-Engineer N64 Source Code

Comments Filter:
  • Wow, that's a lot of work just for Mario. Now if they did Legend of Zelda, that would be worthwhile. :)
    • Re:All that work... (Score:5, Informative)

      by AmiMoJo ( 196126 ) on Thursday May 07, 2020 @05:42AM (#60031392) Homepage Journal

      A lot of it is automated. Look at what machine code the compiler produces and figure out what C code that relates to. Ghidra can do a decent automated job of it if you want to try it.

      Super Mario 64 is helped by the fact that it used a known version of GCC with optimization disabled because at the time it was buggy. A later Japanese release of the game enabled optimisation with a later version of GCC and is proving harder to turn back into C code.

      Incredible effort, well done.

      • gcc was used in the N64 SDK? I guess it makes sense since it has support for lots of architectures and it's used in the embedded world
        • Re: (Score:3, Informative)

          by AmiMoJo ( 196126 )

          Yes, when any company makes a new architecture they usually add support to gcc as a kind of baseline. It's almost the minimum standard now, you have to have gcc support. Makes sense, your new architecture can make use of all the existing tools and workflows people have, and it puts you well on your way to having a decent debugger too in the form of gdb.

          • > usually add support to gcc as a kind of baseline. It's almost the minimum standard now

            Sure, but his point is about what happened more than a quarter century ago.

            Can confirm buggy optimizer - my CS level was the first class doing C++ on gcc at the same time that Nintendo was developing N64. Very frustrating to learn CS on a buggy compiler and too many kids dropped out from frustration.

            • by AmiMoJo ( 196126 )

              The bugs were not GCC per se. The whole game is C and assembler anyway. The optimisation bugs were for the specific CPU in the N64.

        • Re:All that work... (Score:4, Informative)

          by UnknownSoldier ( 67820 ) on Thursday May 07, 2020 @11:15AM (#60032372)

          gcc has been used on:

          * DS games
          * PS1 games
          * PS2 games (along with SN Systems C++ Compiler)
          * PS3 games (along with SN Systems C++ Compiler)

          I know because I've shipped games on those platforms. (On the Wii there was Metrowerks CodeWarrior.)

          • Thanks. I never knew what kind of tools console SDKs used.
          • (On the Wii there was Metrowerks CodeWarrior.

            Which Konami, interestingly, used for their Playstation2 Dance Dance Revolution games. (Thank goodness for incredible amounts of symbol data being left in every PS2 version's executable file, which is how that was learned. Now I just hope someday it can be determined if it uses DWARF 1.0, or 2.0, and if there is a way to extract the other, non-symbol debugging data in a readable form, as it would go a long way in making my effort to reverse engineer PS2 DDR mixes easier. XD)

            • Ooooh, I completely forgot that CodeWarrior was available on PS2.

              Speaking of old games ...

              You should fire up Master of Orion 2 in Borland C. ALL the function names and global variables are exposed! Looks like someone forgot to strip symbols when shipping. Whoopsie-daisy! :-)

      • by DarkOx ( 621550 )

        Ghidra can do a decent automated job of it if you want to try it.

        Yes its amazing what that can do for a free tool, IDA is even more impressive.

        That is long way from producing .c files for a big project like a video game, you can feed back into your favorite compiler though. I do security work so I patch binaries quite often to do things like add instrumentation, bypass checks to see if a debugger is running etc; but that is a long way from just being able to do my own build from sources generated off your binary. Heck even in the world of IL and smali where reflection a

      • by Toad-san ( 64810 )

        Yeah, I hated it whenever I ran into stuff that had been "optimized" ... got real ugly, real unrecognizable after a while.

        But I just happened to run across this: https://github.com/n64decomp/s... [github.com] An N64 decomp (assembly (ugly, don't recognize a line) and C). Are we talking the same stuff?

  • by DrXym ( 126579 ) on Thursday May 07, 2020 @06:00AM (#60031430)
    Nintendo won't allow for this because if the game is source then the game can be ported to anything. Oh and it's their IP. It's far easier for them to disrupt this project than it would be to disrupt clean room emulators.
    • by bobbied ( 2522392 ) on Thursday May 07, 2020 @06:11AM (#60031444)
      Which is likely why it was released without much notice or fanfare. Once the source code is leaked out, it's hard to get the genie back in the bottle so you need to get it done before the IP owner has a chance to shut you down.
      • by DrXym ( 126579 )
        Yeah you get a code drop. Not much good if it contains bugs, or people are interested in working on it and can't because Nintendo shuts down any public repos. And it's not much good to people who want to play the game either and want a stable and trusted location where they can download binaries.
      • Important to note that _only_ the reverse-engineered source code is published. You still need an original copy of the game to produce the graphics and sound resources.

      • by AmiMoJo ( 196126 )

        It's legal on the same basis as emulators that re-implement firmware are. For example until a few years ago the Gameboy boot ROM hadn't been dumped so emulators had their own versions created from scratch. Similar to how the PC BIOS was cloned from the specification.

        The port is legal. You need the original ROM to extract the game assets, it doesn't come with them. Same as emulators that need ROMs to work.

    • by Anonymous Coward

      Nintendo can not allow whatever they want.

      Meanwhile the internet will continue to ignore them and carry on as usual.

      The internet's curiosity and nostalgia just doesn't give the slightest shit about what big corporations think.

    • by AmiMoJo ( 196126 )

      They can't do anything about it. There is no copyright infringement here.

      The source code was reverse engineered. Some people took the Mario 64 ROM binary and used analysis tools and an identical copy of GCC to the one it was compiled with to reverse engineer the source. There are tools that can do a lot of the work automatically because GCC with no optimisation enabled produces very predictable machine code.

      The source doesn't include any of the binary assets like graphics or sound. There is a supplied tool

      • I'm pretty sure that's not good enough.

        I once worked on a project that had actually legal reverse engineering of a binary. That involved a strict setup where one group of people were allowed to disassemble the ROM in question. Then they wrote a specification document.

        A completely different group of people then took that spec and wrote fresh source code implementing the same behavior. These people were not allowed to ever disassemble or debug the original ROM at all. (If they ran into problems, they could wr

        • by AmiMoJo ( 196126 )

          That's what they did though. The binary is a set of instructions describing the operation of the game. They took that description and turned it into brand new C code that performs the same function.

          It's been out for a while now, Nintendo have not hit GitHub with a DMCA.

          • They did it through a mechanical transformation, not through a wholly new creative effort by humans. That doesn't cut it legally, regardless of whether Nintendo bothers to pursue it or not.

      • by bws111 ( 1216812 )

        You've got to be kidding. Of course it is infringement. This is no different than scanning a book, using analysis tools and an identical copy of Word, and 'reverse engineering' the source of the book, then publishing it themselves.

      • by tlhIngan ( 30335 )

        The source code was reverse engineered. Some people took the Mario 64 ROM binary and used analysis tools and an identical copy of GCC to the one it was compiled with to reverse engineer the source. There are tools that can do a lot of the work automatically because GCC with no optimisation enabled produces very predictable machine code.

        The source doesn't include any of the binary assets like graphics or sound. There is a supplied tool that extracts those from the ROM imagine.

        This is all legal, no copyright

      • You can't use the source code to reverse engineer the source code. You can reverse engineer a black box to develop an alternate implementation, but that's not what was done here. This would be more akin to translating a copyrighted work, which is not ok without permission.

  • by AxisOfPleasure ( 5902864 ) on Thursday May 07, 2020 @06:25AM (#60031452)

    While the authors have said they're doing their best to avoid stepping into any illegal areas, we all know full well that any day soon these poor guys will most certainly be getting a nicely worded email from Nintendo's favourite law firm asking them to "Can the project or face the consequences!".

    • I don't think you can find a loophole that gets around this being a pretty monumental piece of infringement. I'm going to wager not just copyright infringement, but also trademark infringement as well.

    • What I don't understand is why people would spend so much time and effort, when they know from the start that what they're working on belongs to someone else. Just...why? Do they not know that Mario is property of Nintendo? He's their most famous mascot and is tied to them like Mickey Mouse is tied to Disney. Do people similarly assume that Mickey can be appropriated to whatever purpose they want?
      • I suppose one reason might be the experience. Sometimes a project like this can be like spending your time doing a jigsaw puzzle; it gets the brain working and tunes up your logical processes. It rewards you with non-quantifiable internal benefits. Or at least a feeling of accomplishment.
      • What I don't understand is why people would spend so much time and effort, when they know from the start that what they're working on belongs to someone else. Just...why? Do they not know that Mario is property of Nintendo?

        > I speak only from my own experiences in attempting to reverse engineer Konami's Dance Dance Revolution series, but for me doing something like this can have a few reasons behind it. For one, it is a nice challenge. It is also fun seeing what the game engine is capable of, and enabling cool mods, all of which is made a lot easier when the game is put in a form that is much more readable.

      • Comment removed based on user account deletion
    • by DrXym ( 126579 )
      If they're more or less machine translating assembly language into C and prettying it up it would be hard to see how they can avoid a nasty gram.
  • But that was a crazy process for a small program. I might have known the C program that I was trying to reconstruct too, I don't remember. But I couldn't imagine the work that would have to go into a program this big.

  • Well done , I'm sure all the devs will be chased into obscurity by N lawyers soon cause someone couldn't resist slashdotting it just like last years Super Mario C64 port. *clap clap* I'm sure the lack of promotion was intentional and not just an oversight waiting for an eager slashdotter to fix it.
  • by jellomizer ( 103300 ) on Thursday May 07, 2020 @08:14AM (#60031678)

    I have seen a problem with many old games running on new architecture. While they run, the will often lack a degree of the charm from the original version.

    For example many of these games were meant to play on a crappy CRT TV. where Pixels were not Square, and there was color bleeding, then putting them on a High def screen say a 3840x2160 resolution things look very blocky, and I havn't found a filter yet, that really emulates the old mode well.

    But on the IBM PC CGA-VGA the standard resolution for graphics at 320x200 the Standard Text block took a 8x8 pixels. That means we could in theory run old DOS Games in pure Text mode and still have resolution left over.

    • by HiThere ( 15173 )

      This isn't the game, it's the game engine. And it's an old game engine. I can't see this hurting Nintendo at all, even if they just ignore it. There might even be a way that they could use it to help themselves. Id got some benefits from the available source of their game engine.

      • There's a rumor that Nintendo wants to do a Mario compilation for the switch including M64, Sunshine, and Galaxy 1 and 2. If that rumor is even close to true then they'll probably have problem with it.
        • Kind of like what happened with the remake of Metroid 2....they got close to a release, and somehow discovered the AM2R project, and so they pushed a C&D them (instead of figuring out means to maybe pull AM2R into the Nintendo fold).....
      • I can't see this hurting Nintendo at all, even if they just ignore it.

        Even if Nintendo isn't directly hurt by this now and wanted to just ignore it they can't.

        If they don't publicly try to do something to protect their IP they might set a possible president that could cause trouble for them further down the line. What if the next game engine someone tries to reverse engineer is one that Nintendo is still profiting from? As a defense the dev/group doing the RE could point at this N64 project and say "Nintendo didn't have a problem with that so why are they complaining now?".

        • If they don't publicly try to do something to protect their IP they might set a possible president that could cause trouble for them further down the line.

          Yeah, that's ⦠unsubstantiated. Completely.
          Lucasfilm has no problem going after people who use their IP in ways they don't like, same with Capcom, SEGA, VALVe, etc, and they all have varying amounts of permissiveness with regards to what not-for-profit things fans can do - SEGA even has been really amicable towards people who have been reversing the Sonic the Hedgehog series, a community that has existed for over 25 years now!

  • Hey assholes, this is what you should be doing with those giant A.I. clusters instead of making motherfucking chat bots.

  • You must not have seen C code....

The most difficult thing in the world is to know how to do a thing and to watch someone else doing it wrong, without commenting. -- T.H. White

Working...