Breaking Down the Demigod Launch 70
In addition to the piracy troubles that plagued Demigod's launch (and partly exacerbated by them), Stardock and Gas Powered Games ran into severe networking issues that hampered their ability to accommodate players with a legitimately purchased copy of the game. Brad Wardell has now posted a frank, detailed explanation of what happened and how they dealt with it. Quoting:
"Demigod's connectivity problems have basically boiled down to 1 bad design decision and 1 architectural limitation. The bad design decision was made in December of 2008 when it was decided to have the network library hand off sockets to Demigod proper. In most games, the connection between players is handled purely by one source. ... So in Demigod, on launch day, Alice would host a game. Tom would be connected to Alice by the network library and then that socket would be handed to Demigod. Then, Alice and Tom would open a new socket to listen for more players to join in. As a result, a user might end up using a half dozen ports and sockets which some routers didn't like and it just made things incredibly complex to connect people and put a lot of strain on the servers to manage all those connections.
First post? (Score:1, Offtopic)
I mean Grandpa's BT HomeHub, not a high-end enthusiast router with custom Linux build.
Re: (Score:1)
Re: (Score:2)
Play it online using GameRanger and it's nearly flawless. From the Demigod Downloads page [demigodthegame.com]:
If you are having issues playing Demigod online, try downloading GameRanger. GameRanger is a free download that lets you play games and demos online with friends and opponents worldwide.
Re: (Score:3, Informative)
Re: (Score:3, Informative)
BitTorrent has similar issues. You'd have thought they'd realise that multiple sockets = fail for most home network equipment.
What are you talking about? AFAIK my torrent client uses one port only. And as with any P2P protocol, the only issue is to allow/enable/kicktherouterinto incoming connections. And if Grandpa can't set up NAT, he's not going to be a full member of the herd, and there's nothing we can do about it.
Re: (Score:3, Informative)
I eventually manually limited BT to 10 peers total, which essentially made it pointless to use. All because my router couldn't handle the packet load.
Note that downloading ISOs using a multi-part download resuming app to saturate my downtream from one source does not cause this issue. It's P2P-specific.
Re: (Score:2)
Note that downloading ISOs using a multi-part download resuming app to saturate my downtream from one source does not cause this issue. It's P2P-specific.
Well, that's kinda the whole purpose of P2P, isn't it? Don't blame the protocol for your dodgy hardware.
Re: (Score:2)
So what IS good DSL hardware, then? I've had the problem with both privately acquired hardware and trhe standard router issued by my provider.
Re: (Score:3, Informative)
Never had any performance related issues, even on the 24Mbps I had back in Sweden. The homebrew debian firewall can be replaced with m0n0wall, pf sense or similar if you prefer web based administration.
Re: (Score:2)
Cisco Aironet 1200 (first Cisco AP I found): £60 = not home user (Router + AP + cordless phone = free on BT broadband + phone tarrifs)
Netgear Gigabit Switch = not home user
Tell me again how this helps me get BitTorrent to not crash my consumer networking gear. Please read the question.
Re: (Score:2)
See my reply here:
http://games.slashdot.org/comments.pl?sid=1238997&cid=28028381 [slashdot.org]
Sure you have to install an OS, but it's a pretty simple installer. Once you've set with network card is which, and an IP address, it's all web based configurations. I'd just use whatever wireless gear you happen to have, or buy something for cheap. Gigabit switches are overkill, unless you transfer large files over your local network. I bought a consumer 5 port gigabit switch from Fry's for $20, so it's not exactly the
Re: (Score:1)
Been using BitTorrent, BitComet, Azureus, OneSwarm for at least 10 yrs now and never had this problem. I have a standard cable modem running thru a linksys modem the entire time....never had any problems and I connect to more than 500 at a time...what OS are u running and what are your system specs?
Re: (Score:2)
Not tried with Win7, though.
Re: (Score:2)
I think that's a bit overboard, I've used both a Netgear and Linksys wireless router for my ADSL2+ and I've not had any issues using BT. Hell, I even use my NAS as the BT client now and it works great.
I'd say that the issues with the GGPs BT is more that he/she has a craptacular ADSL router or that its QoS is turned on but isn't configured correctly. Either way, there are solutions to the issues and they don't require building an expensive network.
Re: (Score:2)
First, get a DSL modem that isn't a NAT gateway. As a general rule, embedded consumer hardware isn't very good and will lock up randomly under load. Find a DSL modem that acts as a transparent network bridge so that all it has to do is pass unmodified packets back and forth. Sorting through all those connection states with its limited CPU is going to kill it.
Next, find a small/old PC for cheap and install one of the many router distribution on it. I use pfSense because it comes with QoS so that I can dr
Re: (Score:2)
It's not the "packet load" causing your problem. I've yet to see a router that can't handle enough packets to saturate any home internet connection you are likely to get. What causes the problem is the number of TCP connections. When bittorrent connects to a peer, it establishes a TCP connection. When your router does NAT (well technically PAT) it dynamically assigns outbound connections a port number. When the computer at the other end sends information as part of that established connection it sends
Re: (Score:2)
Yes, ip_conntrack_max is what I have to tweak on my router to eliminate that problem. It probably applies to many BusyBox-based routers.
I have to telnet in as root, and:
echo 5120 > /proc/sys/net/ipv4/netfilter/ip_conntrack_max
It dies pretty quickly with the default value of 512.
Re: (Score:2)
telnet in as root?
echo 5120
Does that all mean that I can't use BitTorrent without purchasing more hardware? You can guess why home users don't seed, can't you.
Re: (Score:2)
Ok, we get it. You're a home user, and you want a free fix that doesn't cost anything.
Might I suggest you look up the acronym TANSTAAFL? You can get "non-consumer" equipment on ebay for cheap... just not for free. In addition, what are your measured upstream/downstream rates for your internet connection?
Re: (Score:2)
It's usually the number of connections that kill a router when using p2p. A lot of low grade home routers can only handle 64 connections especially if you're using wifi.
Look at the number of connections allowed per torrent (uploading and downloading). Also take into account if you've altered your browser for more connections.
A multi-part dowload app has a lot less connections compared to a p2p app.
Re: (Score:2)
I think he's mistakenly referring to the router lockup that can be common if you allow your client to open too many connections. Some home routing equipment wont release old connections quickly enough, so a bittorrent client opening a few hundred an hour can cause them to become unresponsive.
Re: (Score:2)
It needs that one port so it can hear requests, then it coordinates with the other peer to setup a new connection, and since the port requests comes from an internal source you don't need to forward ports.
IRC.
Re: (Score:2)
Shit hardware exists. You can't necessarily patch it or fix it but you can replace it.
That said, I've never paid more than about £20 for a router, even back in the dark days of early broadband. My last one was an "eTec" (rebranded Conexant-something-or-other) and the one before that was a Trust (rebranded Conexant-something-or-other). They were the cheapest routers I could find and my purchasing decision for both went:
1) Does it do 8Mb?
2) Does it have >1 Ethernet connection?
Bearing in mind that
Failure to build in an idiot margin (Score:4, Insightful)
Yes, this probably could have been handled better, but many home NAT boxes are so broken that they ought to be recalled. I'm actually a bit surprised that there's not a WRT54GL equivalent with well-behaved 2.6 wireless drivers that ships with an open-source configuration interface; it seems that it'd be cheaper than constantly reinventing the wheel.
Re:Failure to build in an idiot margin (Score:5, Funny)
maybe they didn't want to be associated with dick smoking fags
Dick just likes to smoke a fag now and then, what's so wrong about that? You should see what Tom and Alice are up to sometimes...
Re: (Score:1)
Why don't I ever have mod points when I need them. Got a good laugh out of that :)
Re: (Score:2)
That's true, and UPnP support is a joke, with users often having more luck with it disabled.
However, UPnP is a hideously over-engineered and over-complicated standard, so it's not surprising that few routers implement it correctly.
NAT-PMP is sheer simplicity and elegance in comparison, as you would expect from Stuart Cheshire.
Re: (Score:2)
I don't seem to be able to get uPnP firewalling to work with anything. I've even tried two different versions of WRT54G (1.1 and 5) running DD-WRT (23sp1 and 24, IIRC.) Right now I have... yes, the v1.1 running DD-WRT V24 VOIP. Everything except uPnP is working fine so far...
wow (Score:2)
You know, my real feeling is why is this on Slashdot, when these devs apparently didn't consider these obvious questions. Do they even have a QA department?
Re: (Score:1)
They probably did (I sure hope so). However I would bet:
1) Their QA people where co-located with the servers
2) (and/or) Connecting via decent networking hardware.
Naturally, this worked fine.
Somebody should have thought: "Humm, people are going to be running this at home, using random ancient routers/hubs, connecting through a crappy ISP, maybe we should make sure it will work then."
That is where they failed, not testing the software like it was going to be used.
Re: (Score:2)
Do they even have a QA department?
Probably but the department was using the same router as the Development team. Also the problem happens over time with QA vs. Development they both start to think similarity about solving problems and testing them.
Re: (Score:3, Informative)
Who would have thought that routers might not allow for large numbers of listening sockets on its client machines?
I wouldn't have. I've dealt with pretty good equipment my whole life because I love computers and am willing to buy nice equipment. The people who don't love computers and don't buy that equipment don't tend to use it like I do anyway, so I've never encountered this in my informal tech support either.
these devs apparently didn't consider these obvious questions. Do they even have a QA department?
That's quite a leap there. What you consider an obvious question after reading the summary doesn't seem so obvious in a vacuum when you have the entire rest of the game to consider as well. It's nearly impossib
No Stress Testing?? (Score:4, Insightful)
Re: (Score:1)
Re: (Score:1, Insightful)
How did they not perform any simulations or trial runs with players connecting out over the internet in order to stress test this? If they did, how could their testing not notice that routers in general don't like large numbers of listening sockets on it's clients? This is a serious amateur mistake.
One has to put it in perspective. One "amateur" mistake can easily be one "damn it, this shit isn't working but my balding, out of touch manager is insisting on creating unrealistic timetables for our application" mistake.
Re:No Stress Testing?? (Score:5, Informative)
They did, it was the beta test. The trouble is that not everybody was impacted, and the beta group was not a great cross-section of the entire playerbase. Until they got this fixed, I for example couldn't play, at all. A friend of mine in the US with a similar setup wasn't having problems, except when trying to play me. Now I can, and it's pretty fun.
I followed this throughout the process (because I couldn't play), and a number of times they thought they fixed it but the solution didn't work once they scaled it up to lots of players. If you read the full TFA, the issue was actually something buried pretty deep in the networking library they licensed, it wasn't handling large amounts of traffic. (They had to fly in the lead developer of said library to figure that out, though.)
Stardock are not that experienced when it comes to multiplayer games, so this was something of a mess. They probably should have known better. They didn't. That's just how it goes. They're pretty honest about it, at least.
Re: (Score:1, Flamebait)
"That's just how it goes" is the excuse made by every slipshod hand-waver who believes that there's never enough time to do it right, but always enough time to do it twice.
First: Demigod is peer to peer? Whoa, 1992 just called, they want their design paradigm back. Peer to peer is inherently unstable, unreliable and uncontrollable. Nobody (else) uses it for an "AAA" PC title, for those reasons. It's not an inspired choice, it's an insane one, by a company that thought they could palm off responsibilit
Re: (Score:2)
First: Demigod is peer to peer? Whoa, 1992 just called, they want their design paradigm back. Peer to peer is inherently unstable, unreliable and uncontrollable. Nobody (else) uses it for an "AAA" PC title, for those reasons. It's not an inspired choice, it's an insane one, by a company that thought they could palm off responsibility (and cost) on to their players rather than doing their own hosting.
Actually, nearly every RTS game uses peer-to-peer networking, and it's a perfectly reasonable choice when done correctly. Topology was not the problem here, as is demonstrated by the fact that the game plays online just fine through GameRanger.
Second: this whinging about the "network library", and needing to fly the developer in to fix it is simply pathetic. Raknet is an absolute joy to use, is well documented and comes with loads of examples and the source code. The issue is that they assumed that a network library designed in the last decade would deal with their retarded prehistoric requirements, and they didn't bother to check that or test it until way too late in the day. I'd love to hear the story from the Raknet side...
From what's been said, RakNet was only used for their Impulse matchmaking infrastructure, not the in-game networking. RakNet claims to offer peer-to-peer support, but has very limited support for dealing with NAT routers, making it useless for real-world situations. S
Re: (Score:2)
Well, Demigod was made by Gas Powered Games. Some of the other games (Supreme Commander) use the same model. Last I checked, that wasn't made in 1992.
As was mentioned too, Gameranger solved a lot of the problems. The game actually worked if you could get into a game, but the lobby/connection stuff was a real mess. They've been pretty open that they screwed it up.
Some of your comments are silly though. "by a company that thought they could palm off responsibility (and cost) on to their players rather than do
Re: (Score:2)
And Demigod has a full, rich solo player campaign, right?
Typical fanboi response: decry an apple for being sour, and they'll praise it because it's like an orange.
Re: (Score:1)
Peer to peer is practically mandatory for an RTS because there's a large number of entities in play at any time and requiring the server to relay all the computations for them would mean huge amounts of data have to be sent. This isn't an FPS where you have one character per player and maybe a small number of projectiles that behave in easily predictable ways.
Re: (Score:2)
Wrong. I was in the Beta test and the matchmaking/network system never really worked then either. I almost always had a full game of Bots. I think there might have been an actual player in maybe one or two games. Since the actual release won't force-populate a game with bots, now there's a lot of very empty games. I have not seen a game go beyond 2v2 yet. The Beta effectively did NOTHING to stress test player load since it was stuffing all the games with AI.
Now perhaps i
This is why I wish we could start using IPv6 (Score:3, Insightful)
I know, beating a dead horse here. . .
The idea of P2P connections between members of a multiplayer game seems like pretty much a no-brainer, but because of NAT, it's a total P.I.T.A. Everyone likes to say, "We don't need IPv6, just use NAT". But, NAT makes lots of things more complicated (like needing to setup port-forwarding to accept in-bound connections, or using a third-party server that isn't behind a NAT that everyone makes an outbound connection to). Plus, port forwarding sucks because only one compu
Re: (Score:2)
Re: (Score:2)
"You'll encounter NATv6."
"ISPs will nickel and dime you the same way in IPv6 as they nickel and dime you today in IPv4."
Yes, I know NATv6 exists. Yes, I suppose some ISP's will try to keep charging for IP addresses, but the thing is, the market will change. Right now, every ISP charges for additional addresses because they MUST - they have to pay money for IP addresses, and they have a very limited supply. There is artificial scarcity.
In a market where there is no artificial scarcity, I guarantee that (at l
Re: (Score:2)
Why multiple ports? (Score:1)
Why listening on multiple ports?
Listening on a single port works just fine. All Bittorrent applications I ever used only required one port to be forwarded.
Ha (Score:2)
1) It's not worth keeping logs, certainly not for diagnosis, if they aren't synch'ed between servers when you're doing anything which relies on >1 server. If you have a client which will only ever choose ONE server and do all it's actions through that server, you can use the logs unsynch'd. Otherwise, they MUST be synch'ed. It's not hard to work this out.
2) Timesyncing costs *nothing* in practical terms. It doesn't even have to be to the Internet/UTC, so long as they are all synch'ed to each other.
3)
Re: (Score:2)
"Using this as an excuse to put off demos, etc. is baloney."
Not really. What kind of moron publisher would put out a demo when connectivity isn't working properly? Do you really want people trying the demo and going "hey, it doesn't work!"?
Re: (Score:2)
Replace "demo" with "game". Now see how stupid the problem is regardless of that.
The demo could EASILY have been their test-bed for the system while they were still building it. If the demo fails, not a big problem. If the game fails... ARGH.
This is what BETA's are for (Score:2)
"The public beta could EASILY have been their test-bed for the system while they were still building it."
There, I fixed it for you.
Re: (Score:2)
Re: (Score:2)
Semi-public beta's don't give your servers the THRASHING that a true fully public beta will. If you want to really test out and prove your infrastructure and networking code, you need to put it through the crucible of a real public beta. Though, it sounds like, from your post, the GPG/Stardock people should have probably been aware that something wasn't right with the networking even during their semi-public beta.
I'd probably do a limited beta first, then later on, when I had resolved most of the issues dis
Re: (Score:2)
No. If a demo goes wrong, it's a big problem. People take that to mean the game is broken too, and move on.
Age of Conan's disaster of a demo (though they called it an open beta, it was really a demo) turned a lot of sales into non sales VERY quickly with its sheer amount of suck. Marketing is going to look at that and very quickly realize that putting out a demo before your day 1 "gotta have it right away" have made their impulse purchases isn't a very good idea.
Of course, then this whole networking shit hi
Re: (Score:1)
Yeah I agree to a certain extent. Speaking as an owner and avid player of Vegas 2 it is worth the money and highly enjoyable if you have friends and want to play Terrorist hunt or Versus.
I have the same issues with these companies not putting out a descent demo anymore. I usually wait til someone puts a torrent up and then I try it...if I like it I go buy it, if not then it gets deleted...what I had to do with Vegas 2 and with NFS Undercover.
Most games have this problem to a certian extent. (Score:1)
Any valve game with a server browser crashed my stock AT&T Speedstream modem. When I complained to AT&T they stated that it was actually a known issue, and they were willing to sell me a 'high performance' modem for 90 bucks. I went to the local Best Buy and bought a $30 Motorola modem and never had the problem again.
(The AT&T rep seemed convinced that I was really just a heavy torrenter.)
Game is not worth this attention (Score:1)
GameRanger saved the day (Score:2)
Two things: There was little stress testing because GPG has a byzantine Internet policy, and forbids its workers from using anything but the Web at work. No holes were poked for them to test online.
Second, Scott Kevill's GameRanger quickly pulled the slack in for Demigod, and supported the game online just two days after launch. As I write this, there are around 100 people playing Demigod on GameRanger right now.
http://www.gameranger.com/ [gameranger.com]