Server Structure in EVE Online 141
Massively takes an interesting look at the server model used by EVE Online. It's unusual for a MMOG because it doesn't divide the player load among different servers or "shards." Instead, the same cluster handles the entire EVE universe and all 300,000 subscribers (total; record concurrent load is around 40,000). The EVE Dev Blog recently announced some upgrades to keep things running smoothly and allow for battles involving over 1,000 ships. They call the technology StacklessIO.
Shards (Score:4, Interesting)
Re:Great! (Score:5, Interesting)
While I personally don't find Eve the game to be very entertaining, I love to here any new information about it. That's partially because stories like this [rockpapershotgun.com] can only come out of a game like Eve, but also because it's pretty much the only MMO right now that's really doing things their own way and not just following in the footsteps of WoW and Everquest before it.
Performance upgrades are a must. (Score:5, Interesting)
This was one of the largest reasons why I had left EVE. I was in a fairly large alliance. (SMASH) and I had partaken in my fair share of large battles, but we had one with around 400 people total in one system. The game was just in agony trying to run all that.
I actually saved an image of the fight.
http://s219.photobucket.com/albums/cc252/Drakin030/?action=view¤t=MassiveFight.jpg [photobucket.com]
You will notice to the right, the list of players was cut at the top, the client started to bug out. Also during the battle you really had no control over what was going on. The speed was about a frame every 15-30 seconds. After each from you could be dead....Or another part of your HUD/Overview was missing.
It was battles like that in which I look forward to, but it was to the point where whoever hit the fire button first won, because if you got caught in the stream of lag before you enabled your guns, you would not fire a shot.
I'm glad to see that they are working hard on the performance, I just hope it's good enough to sustain at least a 400 man fight.
Re:Great! (Score:3, Interesting)
That review was the worst, seeing as he only played 20% of the game by avoiding the corporation and PvP aspect.
Though its mitigated by the fact that his reviews really aren't reviews at all, they're "I don't really like videogames, so here's my low-budget flash animation with voice over that tells you, again, why I don't like videogames."
Re:Shards (Score:3, Interesting)
They might consider bringing up just a small number of shards to see what happens. Right now strategies, techniques and culture may be so entrenched that it stifles innovation, whereas having 3 or 4 shards would allow new techniques to develop.
Re:Shards (Score:4, Interesting)
Actaully, EVE is the first MMO that I've ever gotten into, and I think a lot of it is the skill system for me. I maybe get in 2 - 3 hours a day of good playtime, 4 days a week. So, about 8 - 12 hours a week. It's nice to know that, as long as I keep track of when my skills finish training, I can log in from work, start the new skill, log back out, and not be at all behind in skills as somebody else who started the same day as me.
IMO, EVE really gives a lot to the gamer who has to balance his gaming life with his work and social lives.
MMOs (Score:2, Interesting)
quote from article: "However, this model affects a lot more than its running costs and complexity and may be practically required for any successful next-generation PvP based MMO."
"Could the single server approach become commonplace in the next generation of PvP-based MMO? I, for one, hope it does."
I played Eve for awhile, it is very cool how so many players in one place just change the approach of tasks which are common place in every MMO it seems. I would love for hardware to get a jump on software right now so we can have more games like this, but I think the only reason why it works well with Eve is the near unending world it has created. In a game like WoW, the ocean is the barrier, in Eve there is so much empty dead space that you can just chill in for hours and see nothing. I think if this was adapted for a land based game, the land would have to be even larger causing busy quests (travel from point a, to b, kill 'tree', travel from point b to c, turn in) would take forever to complete. I could see ways of this working but it is unlikely we will get to see it anytime soon.
To above comments:
There was a group that transferred from one MMO to EVE, and took down a high level faction as complete newbs. They did this by rushing one of there big ships, with more starting ships then they could target. They would then literally just keep doing 1 damage till they won.
Also one thing that has attracted so many to Eve, is that casual players have just as much chance as hardcore players to keep up. Since skills are RT, they are both gaining skills at the same rate in a way, allowing casual players to log on, have fun and then just train while they are at work (which a lot of WoW players do, using 3rd party software those cheaters!!! I don't hate you, I just hate the system we chose to live in...). At the same time it allows hardcore players to do what they want. A lot of people in Eve have taken roles such as; trying to take over sectors, assassin guilds, mercs, tradesmen, pirates, police, politics and even spies. Eve has created a wonderful system and we should build off there success in the near future.
~~Noname
"doesn't divide the player load" (Score:4, Interesting)
Actually, players are divided among servers. Specifically, all players in a single solar system (just "system" in EVE lingo) are isolated to a single thread within a single stackless python process.
This design does not permit real multicore concurrency for a single system. Every player action in space is handled by a single thread. Stackless python provides "microthreads" which gives the illusion of concurrency, but it's really just cooperative multitasking. Lacking real concurrency, EVE can not scale beyond what can be processed in a single thread in a reasonable amount of time. Presently that amounts to about 1000 ships/players.
Microthreads provide none of the concurrency mechanisms (locks, CAS, etc.) that permit threads to safely share data. Thus, the stackless design of EVE can not be scaled using SMP. In my opinion this is a major design flaw of EVE.
The stackless microthread design was chosen for ease of development. Today, that choice plagues the game; it can't be fixed (to permit real concurrency) without a major refactoring of the server side game engine. If EVE will ever scale to what the players actually expect (by virtue of the fact that they don't hesitate to try 1000+ ship battles) then CCP needs to begin thinking about this refactoring; systems should be capable of leveraging multiple cores on demand. That means abandoning microthreads, which is the right decision, because the design imperative behind the choice of stackless (ease of development) is obsolete; half a decade later the new imperative is scalability.
The StacklessIO thing is an improvement to asynchronous IO that CCP deployed earlier this week. It's nice in that it enabled about 1000 players/ships to fight in a system with less "lag" then had been the case with only 500 or so ships. The 32 bit nodes will still crash when they run out of RAM, buy hey, it's an improvement.
Re:Performance upgrades are a must. (Score:2, Interesting)
For one, the client handles things a lot better now. You're able to turn off irrelevant brackets, which leads to some massive performance improvements.
As for the server side, in the last week or two since the StacklessIO update went live on their servers, the servers have been able to handle large fleet battles incredibly well. Module lag is all-but nonexistent, even with several hundred per side battles. The problem is now one of memory; the removal of the IO-caused lag bottlenecks has shifted the bottleneck to server memory. Thankfully, that's a bit easier to fix, since the Stackless IO update lets them compile a 64 bit version of the client that can actually use more than 2gb of RAM.
Re:"doesn't divide the player load" (Score:3, Interesting)
Right right right right right... and wrong.
Great post, but I have one qualm. Microthreading/tasklet models are not explicitly incompatible with SMP systems. Theres nothing inherent to microthreads that would prevent, say, eight separate hardware threads all working in tandem to run a single solar system & its batch of microthreads.
Classical concurrency coordination mechanisms like mutexes and locks are not ideal for microthreading environments, so perhaps scaling out with these concurrency primitives might make abandoning stackless a logical step. Something like an actor model or a message passing scheme may however work extremely well in a stackless environment. Particularly if all you are doing is processing messages, you still want that ability to context switch extremely quickly.
My understanding is the main hold-back for concurrent Eve servers is things like guns firing at a ship thats already blown up, but the local thread doesnt know the ships blown up (not possible in their present non-concurrent environment). You can deal with this either by holding mutexes on the target, or you can use message passing and simply have the blown up ship send a message back saying "sorry, you cant shoot me, i already asploded," and then deal with that failure message in an async fashion.
If I were CCP I know what path I'd be taking. But thus far, CCP has focused largely on platform wins like asynch IO (although given the CPU usage their IO USED to be taking up perhaps that was in order) and proxy servers. They've systematically been ignoring the question of real concurrency. Its a problem I'd love to take a crack at, although it sounds like another one of those things where I'd be locked in a tower for years unshaven half blind & three quarters crazy. Ultimately though, there is no choice for CCP: this is a path they MUST pursue if they want their game to ever work. Presently 400 person combat is miserable, and recent enguagements have crested far past that to over 1000 people in system. Single threaded execution will never work for a world without shards.
Re:Great! (Score:5, Interesting)
That review is dead on accurate for solo type players. Still, it's intended to be funny which it succeeds at, even the most hardcore eve fanboy can admit it's mostly true. It just completely misses the good parts of eve.
Everyone should try eve once. There is about a 75% chance you'll loathe it with every fiber of your being and go out of your way to talk about how awful it is whenever it gets mentioned. However, for the people looking for something different in MMOs, eve can be fun on levels untouched by any other.
Can't distribute 1 sol over N blades = !scalable (Score:4, Interesting)
The reason why EVE is lagging is because the architecture of a sol is not distributed at all. Each sol is a monolithic process.
Any given IBM blade can run any number of sols, but the fewest it can run is 1, not a fraction of 1. Second Life has this same problem, a non-scalable architecture in which several sims can be run on one CPU, but the fewest you can ever run is 1, and it's not possible for N different CPUs to carry the load of 1 sim. That places a limit on the activity within any 1 given sim, or sol here.
When any given sol can be run across N blades in EVE, then we'll have a decent scalable architecture, and not before.
Improving the I/O speed with Infiniband will help, sure, but it's not the answer to non-scalability of sol activity. That can only be done by total redesign of the sol process to make it distributable over an arbitrary number of blades.
Re:"doesn't divide the player load" (Score:3, Interesting)
i did not know that: good point!
however, real horizontal scaling has to be able to go beyond SMP scaling; with this in mind i dont see a interpreter lock as necessarily a dealbreaker. running an interpretter per CPU and using a combination of networking and shared memory for message passing is still viable, and those remoting techniques would ultimately be required down the road even with SMP scaling.
actually, the inability to scale on SMP systems gives me faith that CCP wont do the wrong thing yet again: they cannot take that easy single VM route out, mercy be. this is a huge challenge, but its one ccp needs to step up and face.
Re:Shards (Score:3, Interesting)
The player with the most skill points doesn't always win, because the player with the most skill points can still only throw around part of their skill points.
No single ship, can effectively engage any other ship.
A sniper battleship is a rather large investment of money(for some people) and a large investment of skill points. Yet a single determined ewar cruiser pilot with who has been playing for less than a month can prevent him from firing. A week old player in a frigate can be responsible for the death of a 3 year old veteran. The frig pilot may not be able to kill him on his own, but he can certainly prevent him from leaving the battlefield.
Training times increase geometrically, while the benefit from the skills remains linear.
I'm talking in hours (Score:3, Interesting)
While technically that's true:
1. It was there at a lower level too, hence the claim that only top-levels get catered for, is false.
2. So basically a variant of the common fanboy defense, "well, you could go and do <insert pointless thing> or do it again and again, and that ought to count as more hours the game has." It's been floating around about various games since at least the 90's.
I'm talking in hours. Take the time needed to do _all_ the quests and dungeons that don't need level 70 and/or epic gear. All the quests starting with hunting boars in the Orc newbie area _and_ wolves in Northshire, all the gathering cactus apples _and_ Milly's grapes, and all the instances starting with the Deadmines _and_ RFC. That total is your first number of hours. Then take that dozen instances and measure how much time you spend inside them from start to finish. Once each, because on the second run you don't see that much new content. There are a couple of exceptions, but not that much. I haven't done the exact maths, but as gut feelings go, I'll stick with that 1%.
Oh, spare me the bullshit armchair-shrink gig. You don't know anything about my age, personality, or how I play the game. Yes, I _am_ a middle aged guy _and_ employed _and_ have other accomplishments. And that is exactly _why_ I couldn't care less about a grinding to willy-wave about another piece of gear at the end. I have better things to willy-wave about than "yay, I finally got some virtual boots." Yes, I have RL accomplishments instead. And given that my whole point is that it's the road not the destination, and that I _don't_ want to grind for my top tier gear, you couldn't be more wrong anyway. But that won't stop you from doing... an ego-masturbation gig in which you pretend to know what's wrong with anyone whose tastes differ in the slightest from yours. Geeze, dude, do you even listen to yourself?
You presume to submit how _I_ feel... Heh... How about: I actually feel that I've already done Uldaman at the right level, and feel no need to solo it again at 70? It's still the same instance. I've already _seen_ those halls. But nah, that wouldn't be as good as a strawman that supports your preconceived conclusion, would it? Sorry about that.
Get over yourself. Some people can just have different styles without having some "personality problem" or being impatient kids. You're not the platinum stan
Re:I was wondering about that... (Score:2, Interesting)
But that aside, no, I'm finding the interaction in EVE quite good actually - voice comms (built in, or ventrillo) and chat channels. The reason I like it, is I actually find it more immersive - as a capsuleer, you're in the pod in the middle of your ship, and you're immortal - it therefore makes sense that you communicate almost exclusively by 'remote'.
It's also the only game... (Score:3, Interesting)
..that has a real underground economy, and drug-dealing cartel.
http://www.massively.com/2008/09/24/outlaws-of-eve-online-masudi/ [massively.com]
Re:Great! (Score:4, Interesting)
I was a beta tester in Eve, and played the game when it first came out. The game had an amazing community before it was ever released, and there was a lot of anticipation around it. When it opened up, it was this huge world that was mostly empty, and a lot of players went solo, or in small corporations. It had much more of a wild west feeling, especially out in 0 sec. The economy was a grand experiment, and there were many creative ways to turn a profit.
I ended up quitting Eve because I felt that a lot of the things I enjoyed about it were being lost as the game matured. As large corporations and alliances moved into space, we lost the freedom and uncertainty of a lawless land. CCP considered many of the more "creative" ways of making money exploits, and banned players for using them, instead of fixing loopholes in the mechanics which were the root cause. In short, politics and heavy-handed policy enforcement killed the game for me.
Now I'm not saying they went the wrong direction with it. I think it was a natural maturing of the game which led to where it is now. But I do think they lost their most loyal player base by doing that, and made it more of a mainstream endeavor. While I can't fault them for wanting to expand their audience, and pull in some big revenue, personally, I wish they'd taken a different direction. In any case, I'm always interested to hear how "the experiment" is progressing, and I do think they've made a fresh contribution to the gaming industry. It will be interesting to see where their ideas evolve from here.