Catch up on stories from the past week (and beyond) at the Slashdot story archive

 



Forgot your password?
typodupeerror
×
XBox (Games) Games

Xbox 360 Game Patching Costs $40,000 256

hypnosec writes "It costs developers a total of $40,000 to release a single patch on Xbox Live, making it a difficult platform for smaller developers to grow on. This revelation was made by Tim Schafer of Double Fine Studios — which recently drew a lot of charitable donations as part of a campaign to create a contemporary point and click game. He went on to say that this is just too high a fee for smaller developers to pay, making it hard for them to do well on the platform. This makes sense, since requiring just one patch could massively cut into the profits for a company."
This discussion has been archived. No new comments can be posted.

Xbox 360 Game Patching Costs $40,000

Comments Filter:
  • by LostCluster ( 625375 ) * on Tuesday February 14, 2012 @05:28PM (#39037751)

    Patches are not cheap to deploy, you've got to bother your customers and pay for bandwidth. It makes a whole lot more sense to put the effort into getting the right code onto the disc before it ships.

    • by Anonymous Coward on Tuesday February 14, 2012 @05:31PM (#39037801)
      That's odd, since they're pretty cheap to deploy on the PC.
      • by wolrahnaes ( 632574 ) <seanNO@SPAMseanharlow.info> on Tuesday February 14, 2012 @05:42PM (#39037927) Homepage Journal

        In this case the patches also have to go through the console's usual certification process which obviously involves Microsoft or Sony employees spending time on it. Also remember that until the current generation of consoles, games were expected to work right out of the box and not need patching. Obviously that didn't always happen, as anyone who's used cheat devices like Gameshark can attest to some big sellers had many revisions over the years and games like Morrowind on Xbox had game-breaking bugs which required re-buying the "Game of the Year" edition to fix, but the idea is that console games should not be treated like PC titles where launch-day patches are almost expected.

        I'm not defending the exact numbers, $40,000 does seem rather high, but between actually charging for the certification work, CDN space, and bandwidth used plus adding a "try to get it right the first time" charge it might not be unreasonable.

        • by Moryath ( 553296 ) on Tuesday February 14, 2012 @05:46PM (#39037979)

          Hell, in this generation the consoles THEMSELVES are "ship now, patch later" bullshit... Xbox360, PS3, Wii, all of them constantly need "updates." And rarely do they ever improve functionality.

          • by Moryath ( 553296 ) on Tuesday February 14, 2012 @05:58PM (#39038119)

            In case you think I'm joking... out of the last 10 disc games I bought for the Xbox360, 9 had a 0-day patch already sitting on Xbox Live. Out of the last 10 games I bought on Xbox Live, 7 of 10 had a patch the day I bought it. NONE of the games I currently own for the 360 has gone unpatched. Not one. And to top it off, their "interface updates" made half the dashboard themes pointless and unusable, since most of the visuals wind up hidden behind that stupid bottom-half "grey shadow" area.

            The Wii games aren't patchable (which got them into a bit of trouble when Metroid: Other M turned out to have a game-breaking bug) but how often have they pushed out console updates? And what have they done with them really? Except for the one that allowed for larger SD storage and the swap-trick to "play games" off of storage (really, just leaving internal storage blank and swapping the chosen item from SD into it on the fly), what have they actually patched? It doesn't seem they've done much of anything, certainly the interface never improved.

            And let's not even get started with the garbage updates on the PS3, that actually REMOVED features...

            • by PRMan ( 959735 )
              Nintendo has tried in vain to remove hacks like the Homebrew Channel. Many of the patches have been trying to stomp it out and closing the bugs that allowed it to be possible in the first place. I would guess the remaining ones have to do with their new "channels".
            • by wolrahnaes ( 632574 ) <seanNO@SPAMseanharlow.info> on Tuesday February 14, 2012 @06:20PM (#39038387) Homepage Journal

              Don't even get me started on Microsoft's boneheaded implementation of patching for games purchased on Live. Why the hell it downloads the original version then only bothers to patch when I want to actually play the game is mind boggling. If a title was released years ago and hasn't had a patch in quite some time, how hard can it be to make the version I'd download if I bought it today be patched right off the bat?

            • by deek ( 22697 )

              Wii games can be patched. Look up Zelda Skyward Sword. You can download a patch for it. It may be an exception to the rule, though. I'm not experienced enough with the Wii.

              Yep, the PS3 has removed features with their patches, which I think is a horrible precedent for device updates. I definitely got burnt when they removed Linux support. I can't believe our legal systems allowed them to do it. The amusing thing is, it accelerated the attempts by hackers to crack the console. Hence, it achieved exact

              • Technically the Skyward Sword patch patches the ~save game data~ of affected games, not the game software itself. It's basically just a normal Wii channel that scans for affected files and fixes them. A good thing to be sure, but not really comparable to the patching of the game software itself.

        • Re: (Score:2, Insightful)

          by Anonymous Coward

          $40,000 is approximately what it would cost to store and deliver 150 MB to 14 million people with Amazon CloudFront.

          That's Call of Duty MW3 numbers using a 3rd party CDN at regular pricing.

          I think it's safe to say MSFT is gouging on patch delivery.

          • by Luckyo ( 1726890 )

            May I suggest that microsoft does more then serve as a pipe? Things like QC cost money.

            • by jedidiah ( 1196 )

              Things like QC cost money assuming it's actually being done rather than the vendor taking your money.

              It's kind of like that whole "support" thing.

            • Re: (Score:2, Insightful)

              by Guppy06 ( 410832 )

              If QC actually happened, would there be so many patches to begin with?

          • by Dahamma ( 304068 )

            I'm not saying the number isn't still high for most titles, but the console manufacturers do significant testing on all games and patches as well.

            Add in a few hundred man-hours of work for the various mastering, verification, certification, and functional testing steps and that could easily be in the low 5 figures of labor cost...

          • by Anaerin ( 905998 )

            $40,000 is approximately what it would cost to store and deliver 150 MB to 14 million people with Amazon CloudFront.

            That's Call of Duty MW3 numbers using a 3rd party CDN at regular pricing.

            I think it's safe to say MSFT is gouging on patch delivery.

            As of January 9, 2012, 66 million Xbox 360 consoles have been sold worldwide

            If your target market is more than a quarter of the sold 360's (16.5 million), then MSFT are undercharging, by your figures. And your cost analysis doesn't include any compatibility, QA, or security testing. It's also meant as a disincentive for releasing broken games first then patching later. I think it's safe to say MSFT is doing the right thing on patch delivery, as much as that will unfortunately hurt small developers. Perhaps they could introduce some kind of "rate-based" patch charge instead - Patche

    • by jeffmeden ( 135043 ) on Tuesday February 14, 2012 @05:34PM (#39037831) Homepage Journal

      Patches are not cheap to deploy, you've got to bother your customers and pay for bandwidth. It makes a whole lot more sense to put the effort into getting the right code onto the disc before it ships.

      Epic first post. I was going to suggest that he not think of it as a "Patching Fee", he should instead consider it a "Don't fuck up" fee... It does sound exorbitant, but that's life in the big city.

      • by Lumpy ( 12016 )

        Bother customers - Free
        Distribute Patch - Free if you TORRENT IT. Many game companies do this. Hell ID was king of posting patches on MegaUpload and FileFront. Again FREE.
        Patching is a fact of life for games, but it is not expensive at all to get the patch out to players. MSFT is simply gouging developers hard.

        I'm guessing that cost is why Fallout III and New Vegas were bug riddled And Skyrim seems to be a bugfest as well.

        • No, those games are riddled with bugs because they were made by PC developers that have the mentality you do about patching. Console developers were punished for years with no option to patch a game once it hit shelves so they put more time and effort getting it write the first time.
        • by Moryath ( 553296 ) on Tuesday February 14, 2012 @06:20PM (#39038377)

          I'm guessing that cost is why Fallout III and New Vegas were bug riddled And Skyrim seems to be a bugfest as well.

          You probably haven't played many Bethesda games then. Bethesda in general release incredibly buggy titles. 90% of the bugs fixed in the PC versions of Fallout:NV, Fallout 3, Oblivion, Morrowind, and earlier Bethesda titles have only been patched because people in the mod community got so fucking fed up with Bethesda's incompetent patch division that they did the patching themselves and released it to the community at large.

          I'm all for pointing out that Microsoft gouges developers on the cost to issue patches over Xbox Live, but blaming Microsoft for Bethesda's shitty coding is just being blatantly ignorant of history.

        • Bethesda has some sort of company policy against releasing games that aren't hugely ambitious and deeply buggy with fairly high resource requirements. I'm not sure why; but Morrowind, its two expansion packs, Oblivion, its two expansion packs, Skyrim, Fallout III, and New Vegas, all followed this pattern.

          Unfortunately for the consoles(and pretty shamelessly on Bethesda's part) it is rumored that some of the nastier issues with Skyrim on the consoles are more or less unfixable; because 512MB of RAM just d
    • Patches are not cheap to deploy, you've got to bother your customers and pay for bandwidth. It makes a whole lot more sense to put the effort into getting the right code onto the disc before it ships.

      Having worked on porting the Unreal and UT Series to consoles, I know just how much testing takes place, and it really is done to a very high level. But having said that, once a product is in the wild, well, anything is possible especially with the way that folks generally try to do the "silly" things. In most games, achievement systems even reward many of those things that players years ago would have never gone through and done.

      I agree with "Get it right" for console games, but I also agree that if a prob

    • Comment removed based on user account deletion
    • by History's Coming To ( 1059484 ) on Tuesday February 14, 2012 @07:42PM (#39039241) Journal
      You know what's more expensive? Losing customers. I'm on a rural connection where it takes around 24 hours to download 1Gb, so after requiring a ~3Gb download on Battlefield 3 I'm not going to risk buying anything from EA again. The game was so broken on release it required a "patch" of 60% of the game disc? No thanks, I'm not going to bother in the future. If it's not right, don't release it. To top it all you had to download extended content even if you weren't going to pay to activate it. Nonsense all round.
  • Does this guy pay all his parking tickets twice?

  • Is this inclusive of a fee for bandwidth costs? What is the reason for the fee to be so large?

    Microsoft is pushing cloud computing now right.. why not charge based on how many users download patches?

    • Re:Why? (Score:5, Informative)

      by Zaphod The 42nd ( 1205578 ) on Tuesday February 14, 2012 @05:39PM (#39037891)
      Its because Microsoft has to take time to certify every patch put out on Live, just like they certify every game software put out for the 360. That said, Microsoft is a business, and is trying to make money from their licencing agreements. Consoles cost tons and tons of money to design, and then you actually sell them ON A LOSS. You have to make up all those millions purely through licences with developers, one way or another.

      That said, they're shooting themselves in the foot making it hard for people to develop for their platform. Indie developers need access. And the whole expensive and drawn out certification process means that PC gamers get patches for games weeks or even MONTHS before console gamers see them, even if they're for the same game. Its not that they do PC first, they do both simultaneously (or console first) but PC goes out as soon as PC is ready.

      If they embraced the 360 as more of a general purpose computer that can do gaming well for cheap, then they could skip the certification process and be more like PC. But right now they're shooting for a perfect, controlled console environment.
      • no it is more like Microsoft treating the Xbox like IBM does with Mainframes. Or sun used to do with Sparcs.

        For every step you have 12 managers who need to sign off of it.

        Or you can do it like apple, or google and just let it go through. If it breaks it is the developers fault.

      • Re:Why? (Score:5, Informative)

        by Missing.Matter ( 1845576 ) on Tuesday February 14, 2012 @06:00PM (#39038137)

        That said, they're shooting themselves in the foot making it hard for people to develop for their platform. Indie developers need access.

        That's exactly what Xbox Live Indie Games [wikipedia.org] is for. Now this has its own associated problems, but it's not as expensive to develop for due to a less stringent review process.

        • Yeah, I was thinking about that. If you develop with Microsoft XNA the fees are all much less. But you also get special restrictions. *shrug*
          • If you develop with Microsoft XNA the fees are all much less. But you also get special restrictions. *shrug*

            One of them being that you can't have characters speaking a constructed language. Another being that you can't port a game from another platform; you have to write it from scratch in C#. I've been told that the best-practice workaround for the latter is to develop single-player games solely for other platforms and multiplayer games solely for Xbox 360.

      • games also needs mods / user maps and the PC rocks at that!

      • Just so I get it, how exactly is it the game maker's fault if MS is too dumb to calculate and sell their consoles above production cost?

    • It has nothing to do with bandwidth, $40,000 buys you a lot of bandwidth. And your solution doesn't work at all. Charge based on downloads? So, if lots of people download it... it ends up costing just as much. So this is only good for people who patch games that nobody plays anymore, which is... nobody. I guess the idea is that smaller games will be downloaded less than larger games? That kinda works, but its roundabout.

      They just need to have more licencing options available for indie developers. Have it
      • I disagree with your blanket statement that $40k buys a lot of bandwidth. Having worked int he telecom industry I can tell you that yes... that would be the fee per month for 1 very large circuit. But patching is an entirely different thing. I used to work for ATT and every time SOE had a patch for one of their games, their circuits would hit their "burstable" cap that day... sure they'd get the bandwidth they needed but each patch cost them well in excess of $40k and this was back when they just had EQ1. I
        • Do you have any idea how many users EQ1 had? Definitely factors of 10 or 100 larger than indie game audiences at least.

          That said, fair enough.
      • by laffer1 ( 701823 )

        Why do you think Microsoft should eat the cost of distributing patches?

    • by Lumpy ( 12016 )

      Funds from Every patch goes to buy more chairs for Steve Ballmer.

  • That's the point (Score:5, Interesting)

    by i kan reed ( 749298 ) on Tuesday February 14, 2012 @05:35PM (#39037849) Homepage Journal

    Microsoft has said that they don't want their reputation as a retailer ruined by games requiring numerous patches that not all users can get. They say they consider it a fine on premature releases. It's also to "encourage" dlc to be charged for through their store system, so they can get a cut. If you release the content through a patch then use some sort of exterior store to unlock it, MS doesn't get a piece of the action. Part of the idea is good: companies pay for the "deliver first, make it work later" attitude that has been a little to prevalent. Part of it is money grubbing. I'm pretty neutral on the concept.

    • They say they consider it a fine on premature releases.

      Careful, lest you give hookers any ideas...

    • by spopepro ( 1302967 ) on Tuesday February 14, 2012 @05:49PM (#39038011)
      I was just talking with my boss about microsoft support and found out that for our exchange problems, a tech support call is $250. If they tell you where to find a setting, $250. If they spend 2 weeks and have multiple techs on a call, $250. I came to a realization that it wasn't so much about the fact that MS wanted to nickle and dime for tech support as much as they want to impose a penalty for not RTFM. This sounds like it's sort of in the same spirit.
    • by Nemyst ( 1383049 )

      Indie devs don't have the luxury of being able to afford thorough QA. More often than not the team consists of an art guy and a programmer who set some money aside to work on their project full-time for a few months. They'll do their best to find and fix bugs, but holding them to the same standards as AAA multi-million projects is entirely stupid.

      There's a reason the indie scene is thriving on PC.

      • That's why there's a channel for indie devs to submit their projects that go through a peer review process instead of the standard certification: http://en.wikipedia.org/wiki/Xbox_Live_Indie_Games [wikipedia.org]
        • So how does one translate an existing game's code into C# so that one can release through XBLIG? Sure, I can accept that programmers would have to write the view layer (graphics, sound, input) from scratch for each platform. But I thought one of the benefits of a model-view separation [wikipedia.org] was that different platforms with different view needs could share the model layer (physics, collision, AI) code. The requirement that all games be written in C# breaks this. (Yes, C#; there's no Emit for dynamic languages or
        • Lets be honest with ourselves. XBLIG is definitely a step in the right direction but Microsoft treats it little more than a novelty. You are required to use C#/XNA with the .NET Compact Framework, something never designed for the kind of programming you will inevitably be doing and as such you will be fighting it every step of the way. Microsoft put absolutely no thought in how people will find your games, and there's absolutely no quality control. I've dealt with XBLIG and outside of a few miracle games, i

    • I wonder when it became an issue to patch games? I remember seeing arcade games change as updates were released for them in the arcade. They couldn't push out patches to N64 and PSX games, but they still did different versions of them. They'd produce a batch. Update the game to fix issues. Then the next batch would have a slightly different version.

      It's how games have tended to work for a while. I guess it's probably online play (which I would guess tends to require matching versions to work well) a
    • Quality control is fine but then why take weeks and cost so much to get a rejection due to bugs or low quality? When a game crashes I don't believe many blame Microsoft but instead point fingers at the ISV (example: Bethesda). Why do this? To strictly control and squeeze all the money they can from the supply chain where quality seems to not be secondary.

      Instead this seems like "The Law of Unintended Consequences". In an effort to control the system Microsoft has put in place a barrier to entry, they've

    • I'm all for it.
      Multi-million dollar marketing, forcing release into a window is not always conducive to a completed product getting shipped.
      I presume $40k is the only possible reason that somebody might decide to delay a much-fanfared release until it's ready.
      Looking at it the other way, if patching was free, I suspect a shit-load more shoddy stuff would appear.

      I'm not having a go at iterative development (minecraft), DLC etc - and I appreciate it when a publisher/developer pushes out a patch to correc
  • If only... (Score:5, Funny)

    by Megaweapon ( 25185 ) on Tuesday February 14, 2012 @05:38PM (#39037883) Homepage

    Microsoft would pay small rebates for every patch for Windows they released...

  • Quality Control (Score:5, Informative)

    by adisakp ( 705706 ) on Tuesday February 14, 2012 @05:41PM (#39037911) Journal
    I work in the video game industry but this opinion is only my own. I personally don't think the costs are unreasonable.

    Microsoft has a pretty stringent testing requirement for patches. It's not as simple as slapping up a new binary to download. It costs them money to test patches against technical requirements. There is bandwidth involved for downloading patches as well. The developers have to pay for the bandwidth and testing costs. Charging for patches also discourages sloppy software with lots of patching after the fact. Not all XBOX 360's even have hard drives so patches have to be relatively minor and fit on memory cards if necessary.
    • by trongey ( 21550 )

      Microsoft has a pretty stringent testing requirement for patches. It's not as simple as slapping up a new binary to download.

      Clearly, you're only referring to patches provided by companies outside of Microsoft.

      • I can think of one way to work around Microsoft's perceived technical advantage on its own console. It involves building your own brand of small-form-factor PC and marketing it as if it were the fourth console. I wonder why Valve hasn't already done this to promote its Steam store.
        • I completely fail to see how they'd manage to undercut every single existing PC/component manufacturer out there.
          Always thought they were missing a trick by not embracing benchmarking though. Your PC gets a steam score/analysis and it tells you how a particular game should run on your system before you buy it.
          They could then make a few dollars off certifying platforms - i.e. you buy all these bits, it should be fine - but if you buy this complete Alienware system, then we can tell you right now, you'll ge
          • I completely fail to see how they'd manage to undercut every single existing PC/component manufacturer out there.

            If nothing else, they'd be able to buy and build in bulk, without adding a ton of parts that aren't needed for their particular design. They also could aim to be revenue neutral, making up the difference in game sales.

        • I have had that same thought about a Valve console. All I can figure is that they just think that there isn't any money in hardware...and that it would be too hard to break into the market.

          They are doing pretty well...hard to argue with success. I just worry that PC gaming is ultimately on the way out (hope I am wrong).
    • So you are doing what has already been done by the company releasing the patch then charging them for it? I can understand bandwidth costs but 40,000 is outrageous.
  • by Anonymous Coward on Tuesday February 14, 2012 @05:46PM (#39037975)

    We didn't need API's for physics. We were kids then. There was a coder then that was a genius in his day. It's been a while, It was the early 90s. Back when
    writing code for a nintendo meant just compiling code and downloading to a cartridge that plugged into a conventional nintendo. We coded for amiga too.
    ahh... and the original port of Mad Dog McCree. I tell you if had to pay $40k to code for a target platform, we just wouldn't do it. We would write a killer app for another platform that will help propel our games platform popularity. I think what happened is game coders are tied to the API, and no longer write hardcore code. Because if Microsoft was screwing me over I'd write code for an alternate platform. I'd write code that was so damn good it would actually encourage gamers to buy the platform my game is written for. Now Microsoft is locking the xbox so you can't sell your old game to gamestop and buy something else. well I hope this kills the the xbox.

    I think it's time to develop a killer open gaming platform. The technology is here to do it. Imagine a world of no more DRM.
    I

    • by elrous0 ( 869638 ) *

      I think it's time to develop a killer open gaming platform.

      Attempts to develop an open console or handheld have been tried many times. But no one has ever succeeded at it. Good luck.

    • by Zaphod The 42nd ( 1205578 ) on Tuesday February 14, 2012 @06:08PM (#39038221)
      Oh, it costs a great deal more than 40K to licence a mainstream xbox 360 game. That was the cost of EACH PATCH.

      The thing is, yes, gaming used to be cheaper. Uh... so? It used to be you could make a video game with 2 programmers and 4 artists, Doom and Mortal Kombat both had barely more than that. But these days? 50+ developers for some projects? And you're targeting a gaming console that is sold at a loss? You have to understand the business model involved. Consoles are consoles, not PCs. (as much as I might wish otherwise).
      • Consoles are consoles, not PCs. (as much as I might wish otherwise).

        Then why do only consoles tend to have large (over 24") monitors connected to them? I'd drop consoles in an instant if developers started making PC games that could use several game controllers, but for various reasons, it appears there aren't enough people willing to connect a PC to a TV big enough to fit two to four people around it. Nor are the kind of people who used to crowd around a 19" bedroom TV willing to crowd around a 21" PC monitor for some even odder reason.

        • Split-screen games are pretty hard to come by even on consoles these days. Most gaming is done over the internet. I'm trying to think of how many major titles this year included split-screen or some other type of local multi-player. The wii supports it, but the wii is all kinds of gimmicky so it has a different feel going on from mainstream gaming. How many other modern games do? I guess Modern Warfare lets you do 2 player split screen still? I don't think Battlefield does.... I guess Gears of War does... A
    • This is really just a bunch of rambling, you've got no cohesive point. Because of modern physics APIs... games cost more... so you're stuck on one platform... because of DRM?

      Yes, DRM is bad... and... some other things you said are points too... but none of it makes sense together.
    • Isn't the PC an open gaming platform?

      ~S

      • by tepples ( 727027 )
        Yes, the PC is far more open than a console. But though HDTVs have PC inputs, next to nobody uses them outside the Slashdot-reading geek crowd. So PC games end up optimized for the desk instead of the sofa.
    • by Anaerin ( 905998 )
      So, how exactly did you fix bugs in games after they were shipped? You couldn't? Then you know nothing about what this entire conversation is about. Oh, and it's not MS that's locking the XBox, it's the games publishers, because they want to kill the second hand market.
    • Please look at a recent game - and then tell me how you'd just knock this up in a bit of machine code, more efficiently.
      Most people don't write hard-core code, well mainly as somebody else has done it for them, allowing them to get on with something more productive.
  • by LordZardoz ( 155141 ) on Tuesday February 14, 2012 @06:02PM (#39038175)

    In the PS2 / Gamecube era, patching a console game just did not happen much. It was the XBox that introduced the notion by having a built in ethernet port, the Xbox Live service, and the built in hard drive. On the plus side this has led to certain egregious problems being fixed.

    On the downside, it has become a crutch. Getting through Lotcheck used to be more difficult. It is still an unholy pain in the ass, but the big publishers can afford to drop a patch, and the revenue gained by being able to hit a launch date mandated by a marketing campaign will make up for it. If the company is big enough (EA, Ubisoft), and the title has the potential to move the needle for hardware sales, a great deal of completely terrible bugs can be forgiven if a launch day patch is forthcoming.

    Smaller developers need to anticipate that they wont be able to patch the game at launch simply due to the financial constraints though.

    END COMMUNICATION

    • On the downside, it has become a crutch.

      Amen! The quality of console game software has gone way way down since patching became standard and I'm tired of it.

  • This makes sense, since requiring just one patch could massively cut into the profits for a company.

    This makes sense for smaller houses but a title that rakes in more than a million in sales on the first day of launch this wouldn't be that much of an issue. If your studio is making niche games then yeah.

  • The sole evidence presented is the statement "I mean, it costs $40,000 to put up a patch – we can’t afford that!". The second article simply refers to the first.

    What is this $40K? Are developers literally getting an invoice for $40K from Microsoft, or is that one of those "that's X number of hours @ so much per developer hour" kind of multiplications? If it's an invoice, is it really a flat fee in that nice round number or is he just pulling this number out of the air?

    For a bunch of people wh

  • Mind you, this $40k is the SUBMISSION fee. It is entirely possible that whatever patch you submitted just does not pass. If that happens you don't get your money back - fix what they cited and resubmit, cash and all.

    I worked for a videogame publishing company for a period of time and we had one submission that actually failed due to botched paperwork by someone higher up. This, of course, led to a week of office wide griping about how a fuck up that cost the company more than a year of our wages was done by

Two can Live as Cheaply as One for Half as Long. -- Howard Kandel

Working...