Porting Games From Binary 178
CowboyRobot writes "My favorite Slashdot links are those that inspire me to embark on meaningless and time-consuming quests. This is one of them. Two Canadian game developers at Digital Eclipse have a thorough explanation of how to port a game using nothing but the binary stream coming out of the cartridge. They use the TRS-80 and Phantasy Star III as examples."
secrets galore.. (Score:3, Funny)
Re:secrets galore.. (Score:2)
Re:secrets galore.. (Score:2, Informative)
This binary converter [nickciske.com] may be useful for those not "in" on the joke.
All my games are so old... (Score:5, Funny)
Re:All my games are so old... (Score:2)
http://www.threedognight.com/lyrics/One.html
When Bad means Good (Score:5, Insightful)
It's this kind of pointless endeavour that gives geeks a bad name.
It's this kind of pointless endeavour that makes me happy to be a geek.
Some people climb mountains, other disassemble 8085 binary code.
Re:When Bad means Good (Score:2)
In this case, we had access to relatively accurate source code
Sounds like the only disassembly they had to deal with was in the few bits that were different in the shipped version.
Re:When Bad means Good (Score:5, Informative)
He motivated me to rewrite another one of my old favorite games, Pango... though I haven't got very far
-dk
Re:When Bad means Good (Score:2)
Re:When Bad means Good (Score:2)
-dk
Death to the evil Sno Beez (Score:2)
Brilliant game, BTW.
Re:When Bad means Good (Score:2)
Re:When Bad means Good (Score:1)
Re:When Bad means Good (Score:1)
Re:When Bad means Good (Score:1)
link to the printer-friendly version (Score:5, Informative)
Badly researched? (Score:5, Insightful)
The reason: hardware.
Even your average 80's arcade machine relies upon custom hardware for virtually everything. The main program spends most of its time simply adjusting registers to control sprites etc, and reading from hardware to detect collisions and so on. This new code you've generated for a new CPU will still expect the same supporting hardware...
Re:Badly researched? (Score:4, Insightful)
If you rebuild an executable for system X from a binary dump for system Y, you don't just disassemble it, but you put in macro's for all of the opcodes for system Y. These macro's are the glue that emulate parts of Y's hardware on system X.
It's comparable to the difference between an interpreted language and a compiled language. An emulator is a virtual machine that interprets opcodes for machine Y and translates them to instructions for machine X on the fly; this solution compiles the tanslations into an executable for machine X once.
Re:Badly researched? (Score:2)
These are op-codes that, when rearranged, make pretty pictures?
Re:Badly researched? (Score:5, Insightful)
That's a dubious claim at best. A rom image is likely to contain device driver calls that set register bits that have no equivalent on other hardware. In order to produce a single device operation, a series of opcodes is likely to be required that is guaranteed not to appear in the same order on all software for the platform, and may not even have an analog on the new platform. That means at the very least you have tailor your macros for every image, if not for sections of an image. In the worst (and most likely) case, procedures and algorithims comprised of hundreds of opcodes and used to manipulate hardware on the original system would be completely invalid on the target, and would have to be implemented completely differently. That's the same thing as porting the software, but with some extra steps added.
Their example in the article is perfect to illustrate this. After they converted the FM synth codes to sampled PCM data, do you really think the sound code from the original ROM came even remotely close to working? I call bullshit.
Furthermore, there is no evidence in the article that this project was actually attempted, much less completed. Even if it was, I would bet significant sums of money that their tools wouldn't work to translate other games from Genesis to Gameboy without as much work as went into the tools in the first place. If their techniques ever were to work, it's likely that it would only be while translating between two extremely similar systems. Did anybody with a clue at the ACM read this article before posting it?
Re:Badly researched? (Score:1)
Re:Badly researched? (Score:2)
I didn't know that owning a video game cartridge implied any knowledge of how it was created. You have no idea if the methods described in this article were used in the creation of the GBA version of the game. I'm willing to bet that if these methods were used, they were only used on a limited number of code sections, labor intesive, and not directly reusable.
it's not very hard to find something with equivalent
Re:Badly researched? (Score:2, Informative)
Remember, all custom chips have reliable inputs and outputs
Are you kidding? We're not talking about LEGO toys like the IBM PC or Tandy. Machines like the C64 and Amiga have autonomous, cycle-exact digital/analog hybrid custom chips which all have control ov
MOD PARENT UP (Score:2)
Re:Badly researched? (Score:2)
Re:Badly researched? (Score:2)
This is where you're wrong. It's entirely possible for an operation to be necissary on the target platform that is unnecissary on the source platform. No amount of translation will get you around that. I also continue to stand by my PCM/FM sound argument. If the code does something completely different, that's hardly translation.
Re:Badly researched? (Score:2)
Re:Badly researched? (Score:2)
More Bad Research [blat.info]
Re:Badly researched? (Score:2)
Decompiling to C must have changed quite a bit since I last tried it, around 10 years ago. If not, it seems like you'd run into nothing *but* problems with platform-specificity.
That is, the only C decompilers I've tried did in fact take a binary and output C- however, the C was just a bunch of inline assembly statements and
Re:Badly researched? (Score:2)
As a side note, I love one-line loops.
There lies the hard part (Score:1)
Either that or you create macros or a virtual machine to handle the hardware access.
Now a remake can be hard to do, one example is Telengard for Windows [buildingworlds.com] which is based on the old C64 classic game by Avalon Hill.
Re:Badly researched? (Score:2)
At the lowest level is the microcode program that is hardcoded into the cpu itself. So, the 'CPU' instructions, as we think of them in an assembly program binary, are really virtualize
Re:Badly researched? (Score:1)
Am I missing something? The software will still give it the same appearance, right?
Suboptimal resource use (Score:5, Interesting)
Re:Suboptimal resource use (Score:2)
Re:Suboptimal resource use (Score:1)
I doubt that. Precalculating a table of cosine values isn't going to take an appreciable amount of time on modern hardware, and you can be sure that lookups are going to be much faster than calling any actual math routine, even on modern hardware.
Also, think about this: if the tables are built into the application, and if you replaced the lookups with calls to actual cosine routines,
Re:Suboptimal resource use (Score:2)
Re:Suboptimal resource use (Score:2)
1)Old machines had MUCH less memeory than we do now. So they had lookup tables, but those lookup tables were not huge. Rarely more than 1-300 entries.
2)Old machines had much slower memory buses. Even emulating a pentium 1, they had 66 MHz buses. Nowdays, 300 MHZ buses are common. This extra speed makes up for any loss due to cache coherency problems.
3)Cache coherency problems only occur on loaded systems. Frequently, that won't be the case.
Re:Suboptimal resource use (Score:2)
It can be useful sometimes (Score:5, Interesting)
What we've done is imported "Yars' Revenge" from the Atari 2600 did some elevation emulation, ported the code, and re-compiled to make it work on the Intellivision. We may be selling that cartridge commercially as there is a great need for 8-bit cartridges.
Which is nice.
Re:It can be useful sometimes (Score:1)
Or not.
Re:It can be useful sometimes (Score:2)
games 1 [thinkgeek.com]
games 2 [qvc.com]
Am I missing something? (Score:2)
Pardon me, but I think one of two things have happened here.
1) The esteemed moderation pool (of which I have occasionally found my self a member) missed a clearly +1, Funny post (great need for 8-bit Intellivision carts?) and modded it as Insightful/In
Hm.. (Score:1, Funny)
Damnable geeks.
Sweeet... (Score:3, Interesting)
Re:Sweeet... (Score:2)
You should have seen the grin on my face the instant I rolled over the link and saw the filename in the status bar.
Let Baldur's Gate, Neverwinter Nights, Morrowind, and all the rest KNEEL and COWER before the ONE TRUE RPG!
Re:Sweeet... (Score:1)
Seriously, though, I first started programming on one of those, with BASIC, and a tape drive when I was 6 years old. I would be awesome if I could play the games from it again on my Athlon.
Re:Sweeet... (Score:2)
Check out:
Ira Klang's TRS-80 Central:
http://www.trs-80.com/
TRS-80 Emulators:
http://www.vavasour.ca/jeff/trs80.htm
http://discover-net.net/~dmkeil/
TRS-80 Software:
http://home.planet.nl/~srahman/trs80_s
(I was a TRS-80 user from way back as well... published games under the Pacific Software name back in the early 80s.)
Article text (Score:4, Informative)
What if you have to port a program, but all you have is a binary?
Bridging the Gap
Typical software development involves one of two processes: the creation of new software to fit particular requirements or the modification (maintenance) of old software to fix problems or fit new requirements. These transformations happen at the source-code level. But what if the problem is not the maintenance of old software but the need to create a functional duplicate of the original? And what if the source code is no longer available?
This exact problem arises when trying to reproduce the original play of old arcade games on modern devices. The game play is so well known that anything short of the original is unacceptable. Often the source code is available, but it may be incomplete and may not cover all of the patches that were added to later production models. In addition, it is too expensive to provide copies of the original hardware.
Providing faithful emulations of video games (and old home computers) is our primary experience with this process. But the same techniques can be applied to other areas. Aging hardware and software can be replaced by new hardware with a completely compatible program.
BRIDGING THE GAP
The general problem can be expressed as "bridging the semantic gap." You must create a program that precisely maps the meaning of the original program onto a host system. Primarily this means an interpreter of some kind for the target processor's instruction set, but one must also deal with I/O (input/output) devices. Such an interpreter is known as an emulator. (See "The TRS-80: A Simple Emulator" on page 54 of this article.) If the program is automatically converted to a different language, it is called a translation.
Why are we tackling the problem at such a low level? Mainly to achieve the highest fidelity possible to the original. Emulation is about mapping semantics. The semantics of hardware are usually well documented either by circuit diagrams or chip-specification documents. Internal layers inside software are usually designed to much looser standards, so it is unlikely that specification documents--if they exist--completely describe the behavior. In fact, the software itself is the most authoritative description available. It is true that chip specifications are not always complete or accurate, but chips are reused and over time the deviations become widely known.
The semantic gap between the target and host systems is not purely an abstraction. It can be quantified as follows: G = number of host instructions to emulate one target instruction. Given G and some idea of the relative speeds of the host and target system, you can quickly decide if emulation is feasible. The problem here is that the value of G is a function of an actual emulator. A rule of thumb is that G is at least 10, but practically speaking, 10 is lower-bound for systems that are quite similar. Although time is usually the overriding concern, there is an analogue for a semantic gap in terms of storage space.
Figure 1
Emulation and translation start with the same inputs (the ROM data and the hardware documentation) and produce the same result: a copy of the original running on a PC. The difference comes in how the ROM data is handled. For the emulator, it is simply a parameter to the program. For the translated version, the ROM is converted and compiled into the executable.
The semantic space gap is the ratio of the size of the host program to the size of the target program. For an emulator, the host program size is broken into two pieces: the host's representation of the target program and the emulator code and associated tables. Unless the host and target are radically different, there is little to be gained by significant changes to the representation of storage. Thus, if the emulator code is ignored, the semantic space gap is typically exactly one.
Intuitively, the value of G really depends on how differe
Inspiring Slashdot links (Score:1, Funny)
This is as close to having a girlfriend as the average Slashdotter can get.
Gain v Pain.. (Score:5, Insightful)
Fatal error: Call to undefined function... (Score:2, Funny)
Wow, those guys ARE good. They even ported their company web server to a TRS-80.
But some areas of the code still need work:
int fork() { return 1; }
Let's see... (Score:5, Insightful)
*Each assembly instruction interpreted by an interpreter
*Compile once to run N applications
"Translation":
*Each assembly instruction translated to a C macro
*Compile N times to run N applications
Looks like emulation wins. If an emulator has JIT then "translation" losses its only speed advantage too.
RAM advantage (Score:1)
JIT translation is RAM-hungry. "Translation" has a RAM advantage as well, which is useful on handheld devices with little RAM.
Idea depreciated: DUMB (Score:5, Interesting)
For 1, the NES uses mappers to display games. On emulators, many mappers are not supported. NES game producers also put hard-coded timings in games. So if your recompiled game isnt the exact same multiplier of clock frequencies, many will exhibit starnge behavior or just lock up.
Next, the SNES had pretty much basic AppleII GS hardware with the exception of the 32 channel sound card. It had a sound cpu which could hold 64k code along with samples. A problem that the makers of ZSnes had was determining the random noise generators formula. On the older games like Chrono Trigger, the wind would sound like square waves going up and down. That sounds fun, compiling a game when you find out that the hardware emulation controls wernt right.
Jump to PS2. Who would have thought that a comuter like that would be available to the public? One that has little GFX ram but a huge bus. Not to mention a full FS to "compile". What pitfalls occur in the 3D hardware?
Emulation is still better as it offers a replacable shim to modify and add features. You can also use other emulators.
Re:Idea depreciated: DUMB (Score:1)
Re:Idea depreciated: DUMB (Score:2)
Sorry for the confusion.
Re:Idea depreciated: DUMB (Score:1)
Re:Idea depreciated: DUMB (Score:2, Interesting)
The Apple IIGS and Super NES had the same CPU (65c816), but that's it; their I/O architectures were NOTHING like each other. Apple IIGS and Super NES video weren't even close, and neither were their sound chips. The IIGS had a memory-mapped dumb frame buffer; the Super NES's video was tile-based, somewhere between that of the Sega Genesis and that of the Game Boy Advance, and VRAM was accessed through a couple I/O ports.
Re: not as DUMB as you might think (Score:3, Interesting)
The GBA does not have the power to emulate the Mega Drive (Genesis) even with a JIT compiler. Hence, a different method would be needed. Either simulation, recompilation, or translation. Translation was the best option given what they had to work with.
Emulation is wonderful and all but it certanly doesn't work in all situations.
FYI, I believe that PS1 was emulated and PS2,3 were translated in Phantasy Star Collection. They're not
Re: not as DUMB as you might think (Score:2)
Ambigous. Do you mean that the PS1 code was emulated in the PS2? If so, you're wrong. There's 2 main CPU's in the PS2. One is for PS1 and a bios-like chip to interface to the main CPU to make the PS2 rom calls. They also use this shim to add anti-aliasing.
Re: not as DUMB as you might think (Score:1)
Re:Idea depreciated: DUMB (Score:1, Informative)
Re Creepy Crawler: DUMB (Score:1)
For posting a reply, why in hell wouldn't you read the F*&^ing article?
The article talks about the EXACT reason they chose tranlsation vs emulation. They are translating to the Gameboy and in their words:
The CPUs are so close in performance that direct emulation can immediately be ruled out. The larger cartridge size means we can accommodate some code expansion caused by translation.
Previously in the article they mentio
Digger (Score:4, Interesting)
Amazing feat. It's completely rewritten in C to gain exactly the same functionality as the original code, with only the binary / dissassembled machine code to work with.
Re:Digger (Score:1)
yay! glad to see I'm not the only geek doing this (Score:2, Interesting)
I've finished the Epyx game JUMPMAN
http://www.classicgaming.com/jmanproject
and I'm currently working on Beyond Castle Wolfenstein
This is a very time consuming hobby, but for me it's a matter of preservation than anything. I don't want to have to dig out the PCjr from my closet everytime I want to play these oldies.
-jeff!
Re:yay! glad to see I'm not the only geek doing th (Score:2)
Maybe I'm just suffering from old-timer's disease?
Too much self modifying code, the guy is a fool (Score:2, Interesting)
He does not talk about dynamic recompilers (that use page management dirty-bits to track self mofifying code)
He does not talk about self modifyingh code
he does not talk about timing loops and differences in different opcodes
He does not talk about lot of things.
I suspect he is not the one to talk about this stuff.
Tom Dowdy (old email dowdy@apple.com) knows all about DR he wrote a paper on it.
So did the guys who wrote a DR for recoding Mac 68K on an Inte
These are a few of my favorite links... (Score:5, Funny)
My favorite Slashdot links are those that go down within about 12.5 seconds of showing up on the front page. I know I must be missing something really great!
Project Odin (Score:5, Interesting)
The guys at OS/2 Netlabs [netlabs.org] have been doing this for years now. It'ts called Project Odin [netlabs.org] They run Win32 apps on top of OS/2 with no emulation: they "translate" binaries on-the-fly. They even run Win32 drivers on OS/2!!
Re:Project Odin (Score:1)
I don't think they're actually translating calls to Windows functions to OS/2 functions - that would be neat (and very very hard, due to side effects)
Re:Project Odin (Score:1)
These guys are hardly experts (Score:5, Interesting)
I've had to suffer through these guys' port of Joust on shockwave.com (in fact I have the high score for the month currently) and it has several inexcusable flaws. The most annoying is that it doesn't always respond to keypress events properly - rather critical functionality for a game. It doesn't actually seem to drop the events, which would be bad enough, but it ignores them for as much as half a second at a time and then spits a whole bunch out together. If your game wasn't screwed up when it seemed to ignore the event, it's damn sure going to be screwed up when they all get played back in a burst along with the other keys you hit to recover from the original failure. I know Windows is not a real-time system, and I've worked on real real-time systems (microsecond response times) so I do know the difference, but there's no excuse for a delay this long on an unloaded modern system. Other games don't seem to be afflicted by the same problems. The problem is not in Windows; it's in Digital Eclipse's emulation of the hardware on which the game originally ran (even if the code is translated the non-CPU hardware still has to be emulated).
There was also a time when the game's speed calibration was totally broken. I'd play on my 600MHz laptop and it would be just about right, but when I went to my 1.5GHz desktop guess what happened. Yeah, everything in the game was moving about 2.5x as fast and the game was unplayable. This only persisted for a week or two, but it's still not something that should happen in a version that's released to the public. These technical failures, combined with their apparent acquiescence to Shockwave's desire to add deliberate player-killing features to their translation (the very laws of physics in the game change after you get a good score, and I've looked at the original ROM code so I know exactly how they're doing it) have left me with the impression that Digital Eclipse is both incompetent and unethical.
Re:These guys are hardly experts (Score:1)
Re:These guys are hardly experts (Score:1, Informative)
... some mook sitting in a chair propping up a can of mountain dew with his belly thinks that dissing the free (as in beer), well intended, open-hearted work of someone else (did you really *suffer* through the joust port? poor baby) earns him some cool. please.
yeah yeah, you've got a grasp, but so what?
pounding on s
Re:These guys are hardly experts (Score:3, Insightful)
Programs that simply don't work right, that don't meet quality standards we'd apply to every other product category, is a perfect illustration of why the public at large views geek culture - that includes you, we're on Slashdot - as a bunch of overpaid lazy clods.
Mindless flames that make unwarranted assumptions (such as "propping up a can of mountain dew") are a perfect illustration of why the public at large and even the geek community views Slashdot - and particularly the anti-community of aptly named a
TRS-80 games (Score:4, Interesting)
Odd TRS story: the bicycle manufacturer GT made a mountain bike called an RTS-1 (Rear Tuned Suspension). It shouls have been the more logical TRS-1 (Tuned Rear Suspension), but the Tandy trademark apparently stopped them using the initials TRS. This was about 10 years after the TRS had gone out of production!
Re:TRS-80 games (Score:2)
I ask, because I was under the impression trademarks only apply in a certain industry (ie/ I've seen a furniture store called McDonalds). Correct me if I am wrong...
an ML/OS compiler? (Score:3, Interesting)
Re:an ML/OS compiler? (Score:1)
You would need some sort of emulation going through the whole thing. There could be lots of tricky situations where you are not sure of where branches are going to go.
You could have a software emulator and someone play through the whole game, I suppose, but there is no guarantee you would get all of the br
Re:an ML/OS compiler? (Score:2)
Geesh - this is hard? (Score:2)
And it tells you "Paul is Dead" at the same time.
These kids now adays...
This was for the Game Boy Advance... (Score:2)
IMHO, It was smarter to do it this way than try to emulate the genesis on a GBA. The latter might be possible one day, with enough hacking breakthroughs, but this is no less generalizable. They have a perl-based translator which they can improve, much like an emulator often need to be improved the more games it is to support.
This is why you keep original source (Score:1)
Other links (Score:2)
http://www.gtoal.com/sbt/ [gtoal.com]
http://groups.yahoo.com/group/staticrecompilers/ [yahoo.com]
It is unclear to me how they deal with more complex situations such as self-modifying code or code that jumps into the middle of an instruction.
Re:Emulation (Score:5, Informative)
Re:Is that porting? (Score:3, Informative)
It is emulating, as they themselves say:
Providing faithful emulations of video games (and old home computers) is our primary experience with this process. But the same techniques can be applied to other areas. Aging hardware and software can be replaced by new hardware with a completely compatible program.
Re:Is that porting? (Score:2)
The article clearly shows that it does not run on a virtual machine.
The binary is converted to asm. Then the asm is converted to C with C assembly calls, then compiled into an EXE to be run at any time without a virtual machine.
Re:Is that porting? (Score:1)
My usual understanding of emulation is an executional one, i.e. the old app runs within a simulated environment. However, the devleopers here are using macros to emulate the older hardware. This is an overload of the term "emulate". They are emulating at a source level, whereas the final compiled program is not an emulation by my original understanding.
RTA: They're -translating- not -emulating- (Score:5, Informative)
Not running the existing code through a software emulator on new hardware.
They are aiming to (for example) map the display instructions from Pacman on the Atari 2600, to x86 windows API display instructions.
They will also have to translate all game logic more times than not (to valid x86 logic instructions), and may have to alter the stored data in the event of differences in endian-notation.
The resulting translation will not suffer from the overhead emulators create.
Interestingly, I see this very feature as becoming one of their largest problems. pre-pentium-speed game programmers tended to rely on the clock speed of their original target hardware to set the update rate for the game. Trying to play frogger on even a 486 would be an impossible blur.
Timing control will be their biggest hurdle.
Re:RTA: They're -translating- not -emulating- (Score:1, Informative)
The only thing they're doing differently is they save the translated code instead of just caching it in memory.
Re:RTA: They're -translating- not -emulating- (Score:1)
They do know factors such as ROM speed (wait states), CPU speed, clock cycles per instruction, etc.
Furthermore, most games time based on interrupts (video or a timer int) rather than just counting from 1 to 1 million or what have you.
Re:Fatal (Score:1)
looks like the server never died before
Re:Fatal (Score:1)
The message I got was:
Slashdot geek error: Call to overworked function: website_die() in db/db.php on line 88
I'm not sure what this is in binary.
Re:Why Phantasy Star III? (Score:2, Informative)
Re:Why Phantasy Star III? (Score:4, Informative)
Re:Why Phantasy Star III? (Score:1, Interesting)
Those games run fabulously and flawlessly on the GBA. They really do look and act like the originals.
Re:Secret Contra Code!!!! (Score:1)
that is actually the Konami code, it works in pretty much every konami game out there, just enter it at their logo.
Re:Secret Contra Code!!!! (Score:1)
I'll stop my sarcasm there. It worked on 2 games: Life force, and Contra. Gradius 3 had a modified version of the code.
A far cry from "pretty much every konami game out there"