Technical Details Behind the LAN-Party Optimized House 123
New submitter Temporal writes "Yesterday, Slashdot reported on my LAN-party optimized house. But, lacking from the internet at that time were key technical details: How do I boot 12 machines off a single shared disk? What software do I use? What does my network infrastructure look like? Why do I have such terrible furniture? Is that Gabe Newell on the couch? The answer is a combination of Linux, PXE boot, gPXE, NBD/iSCSI, and LVM snapshots running on generic hardware over generic gigabit ethernet. I have even had several successful LAN parties with a pure-Linux setup, using WINE."
It stands to reason that... (Score:5, Funny)
...plenty of wine would make a Linux-centric LAN party tolerable, and perhaps even enjoyable.
Re: (Score:2)
...but... isn't beer enough already?
Re: (Score:1)
I prefer to keep my LAN parties at a classy level that beer cannot provide.
Re: (Score:3)
You're drinking the wrong beer. You probably aren't even using the right kind of glass..or any glass.
Re: (Score:1)
Re: (Score:2)
In that case, Lindesfarne Mead would surely be the tipple of choice.
Re: (Score:1)
Not if Wolowitz crashes the party... ain't enough WINE nor WoW that can counter that. Damn, but Penny needs to stop spilling to Bernadette....
Re: (Score:1)
if RMS joins in, hand-rolled intoxicants all round!
Re: (Score:2)
Desk height (Score:2)
No, the burning question was what kind of chuckle-head would spend all that money and stick his friends on fixed height desks all (it looks from the pictures) at writing height, not typing height.
He's going to give all his friends who are not very tall shoulder and wrist problems.
Look at the pictures. They're all (except for one very tall guy) very badly positioned for work.
He's going to need a to dole out ibuprofen by the pound to his guests.
Re: (Score:2)
Adjustable chairs are pretty easy to come by.
Re:Desk height (Score:5, Interesting)
It's a fair criticism. I wanted to make the desks lower but I also wanted the monitors to sit higher when folded up, and the desks were getting stupidly deep, so I had to compromise. In practice, though, people aren't typing at these desks, they're gaming, which in my experience (as someone with some RSI problems) is not as sensitive to desk height.
Re: (Score:2)
I can think of a couple of ways around it, but each with their own disadvantages, and what you did seems simplest and clean. That said, I'd definitely prefer height adjustable rolling chairs (floor scratches and storage space be damned).
Incidentally, where does the sound come from on the stations? And how loud is it in that room, as I don't see many sound absorbing surfaces.
Re: (Score:2)
You could use USB headsets, but that is a good question as it does not appear to be answered anywhere in the article.
Thank-You (Score:2)
Kudos to you.
Crashman?! (Score:2, Funny)
Why would you ever name a device Crashman? This just seems ominous!
"Which machine do you want to use for the LAN party?"
"Oh, Crashman looks really stable."
Your house is great! (Score:1)
Re:Your house is great! (Score:4, Interesting)
Yes, that's exactly what I'm doing -- PXE-booting the whole OS over iSCSI. I'll edit to make that clearer.
Re: (Score:1)
Re: (Score:2)
I'm bootstrapping gPXE using regular PXE.
Re: (Score:2)
Yes, that's exactly what I'm doing -- PXE-booting the whole OS over iSCSI. I'll edit to make that clearer.
I've looked at some of the pricey solutions that do various versions of this, and what you've done for free is very impressive. I do want to confirm that I'm reading it right though, and that you're basically booting the same image, so the machine names and such will all be identical within Windows, is that correct? Have you seen that cause any issues for any games? Have you tried any LAN games like this (preferably more recent than Quake)?
Clearly having the same machine name and SSID makes it unusable f
Re: (Score:2)
Yeah they all end up with the same machine name. I disabled automatic network discovery so they won't broadcast themselves, since it would just lead to confusion. File sharing can be done by copying files to and from a share on the server machine. I don't see why games would care since they operate at the TCP/IP level, not using computer names. I have only tested a couple things so far, though, like UT2k4... will be trying more today (LAN party!).
Re: (Score:2)
All you can 'PXE' boot is WinPE, but most anyone wouldn't care about the distinction between that and PXE-bootstrapped iSCSI. PXE boot iPXE, then iPXE interprets root-path to start Windows with their native software iSCSI support. ipxe.org is a place you can do it yourself. xCAT automates everything from stgt iSCSI creation, to unattended windows install from DVD into iSCSI root, but may be a bit much for a home setup when you can just learn how to do it yourself manually at not much more effort.
How well does that perform? (Score:1)
I can't imagine a single machine serving out over iSCSI to have performance acceptable to play any modern, intensive game. How's it all work?
Re:How well does that perform? (Score:5, Informative)
I can't imagine a single machine serving out over iSCSI to have performance acceptable to play any modern, intensive game. How's it all work?
I couldn't imagine it either, but it turns out it works fine. Obviously the load times aren't blazingly fast but no one has ever complained about them being slow either.
Note that most games load all data upfront. Once they've done that, the game runs without doing much I/O.
Also note that an iSCSI image can be fully cached client-side, so if you load the same game twice, it's probably going to load directly from RAM the second time. (Most games are 32-bit so there's a good 4GB of RAM in the machines doing not much other than disk cache.)
Re: (Score:1)
Very cool. Thanks for posting that write-up.
Re: (Score:2)
PXE, et al, use TFTP, if I remember rightly. In principle, there's nothing to stop the files being delivered by multicast FTP (yes there are at least three, they use Scalable Reliable Multicast, FLUTE or NACK-Oriented Reliable Multicast respectively). Since OS images and the games themselves don't differ between machines, if you have N machines you get file delivery about N times as fast. (About because lost packets are resent, so it's not truly linear improvement.)
Re: (Score:2)
PXE, et al, use TFTP, if I remember rightly. In principle, there's nothing to stop the files being delivered by multicast FTP (yes there are at least three, they use Scalable Reliable Multicast, FLUTE or NACK-Oriented Reliable Multicast respectively). Since OS images and the games themselves don't differ between machines, if you have N machines you get file delivery about N times as fast. (About because lost packets are resent, so it's not truly linear improvement.)
Indeed, as mentioned in the blog, I was at one point trying to develop a UDP-based blog device protocol that would broadcast blocks back, on the assumption that all the machines would be loading the same data at the same time. However, when I got the system up and running without that, and didn't see any performance problems, I decided to abandon that idea and focus efforts elsewhere.
Re: (Score:2)
That makes sense. The bottleneck defines the upper limit of performance. If you do go back to working with UDP, I do suggest grabbing either the NRL's NORM (snapshot version since they don't produce official releases any more for some reason) or FLUTE, since those give you the libraries you want for no effort. Both are listed on Freshme....sorry, Freecode. (That new site name STILL sounds too much like Freescale, the Motorola chip spinoff.)
Re: (Score:2)
Well with gPXE (pretty much dead now as the meat of the project forked to iPXE), most setups use tftp to transfer ~64 kilobytes and then other protocols take over (like http for a tftp workalike loading into ram or iSCSI for block storage loading content on-demand). Now with games, level load times may vary, but generally everything is in memory (or at *least* cache) when it comes down to actual gameplay.
Stateless ram-based OSes delivered can work too, but the multicast generally isn't worth it. A system
Re:How well does that perform? (Score:4, Informative)
If all the image needs to do is hold the basic OS and a single game deployment, why not pxe the whole image to a dedicated hardware ram drive, that can make full use of the sata controller?
Acard has a number of such devices, and while pricey, would absolutely floor disk io performance in a game rig. [acard.com.tw]
There are quite a few other devices of this type on the market as well.
Using these in the systems, you could still netload the system images to the game rigs with pxe, but when the image has finished being pushed, just reboot them and you have a bitchin fast row of locally booting systems. Power them off when they need a new configuration pushed.
Re: (Score:2)
If the machine serving as an iSCSI host has sufficient RAM, it already almost works that way:
1. Client A boots up. Loads OS from solid-state disk. Subsequent clients are likely to load directly from the server's RAM cache, which just got populated automatically.
2. Client A runs a game. Loads game from disk. Subsequent clients are likely to load directly from the server's RAM cache...
So the first load is at disk (SSD) speed, and the subsequent loads are at RAM speed (although both instances are limited
Re: (Score:2)
Or just stick 16gb+ ram in your machine, setup a soft-ramdisk...
Best would be if things were ECC compatible, but well-tested regular ram would probably do fine.
Re: (Score:2)
Agreed, but most "massive amounts of ram" capable desktop boards are multiprocessor cuda designs.
This means a cpu hit and cpu bus io bottleneck for disk io.
This bottleneck will also be having to service exchanges between the videocards and system ram, and program execution requirements. (Most games are uniprocessor, even today.) The ram is present, but to get to it you have to talk to another processor/core.
The issue with disk io vs cpu involvement is important. It is the single most important feature of a
Re: (Score:2)
I'm still thinking that with memory bandwidth in the many-GB/sec, the bottlenecks won't be apparent - if your entire game image is a GB or two, it'd still be loaded in a fraction of a second.
And I know for a fact we can have single-processor(multicore, though), four-slot(or 6-slot Intel) memory motherboards, and with 4GB DDR3 cards...
Re: (Score:2)
We found that SATA SSDs weren't just faster than 15k SCSI drives, but dramatically so in database servers. We saw at least 10x performance boost despite being "only" SATA drives.
I wonder if you've considered this route?
Re: (Score:2)
I think you might be confusing iSCSI (a network protocol) with SCSI (a disk hardware interface). My physical disks are all SATA, and are a mix of SSD and HDD. The bottleneck is network bandwidth, so using SSD everywhere isn't going to improve performance much.
Re: (Score:2)
Games do not spend that much time doing disk IO after they've started. And most modern drives deliver nowhere near a gigabit of streaming performance anyway.
Re: (Score:2)
Well, almost all SSDs do [anandtech.com]. (1Gb/s is about 125 MB/s).
Re: (Score:2)
An SSD would not be my first choice when trying to improve the performance of a gaming machine. But yes, you're right, almost all SSDs do.
Re: (Score:2)
Why not?
Modern games aren't bottlenecked at the disk. Even if they were, the bottleneck then becomes the network, since it's been read from disk into RAM at this point. But try it -- next time you fire up a modern game, when it's attempting to load a level, watch the hard drive light. Usually it flickers from time to time. Then fire up Defrag or something, which you *know* will generate a lot of disk activity, and compare.
And that's just loading. Once it's loaded, it's in RAM. The disk is just a nice big pl
Haha ... (Score:5, Interesting)
"I purchased 12 copies of Windows 7 Ultimate OEM System Builder edition, in 3-packs. However, it turns out that because the hardware is identical, Windows does not even realize that it is moving between machines."
Re:Haha ... (Score:5, Informative)
"I purchased 12 copies of Windows 7 Ultimate OEM System Builder edition, in 3-packs. However, it turns out that because the hardware is identical, Windows does not even realize that it is moving between machines."
Yeah. I actually learned this after having purchased only one 3-pack, but went ahead and bought three more 3-packs just to be legal. With this much attention paid to my setup, I don't want to be caught pirating.
Re:Haha ... (Score:5, Funny)
Re:Haha ... (Score:5, Insightful)
In other words, you think doing the legal thing is wasting money.
Re: (Score:2)
Unfortunately, you may still run into legal headaches. OEM system builder is for people who build a system, then sell it to another party.
But MS may see 12 systems with the same key live (Score:3)
But MS may see 12 systems with the same key live at the same time.
Re: (Score:2)
i think that a google employee can afford to have legitimate software licenses. :-/
also: "I highly recommend that anyone emulating my setup actually purchase the proper Windows licenses even if your machines are identical."
Re: (Score:2)
I'm sure MAC addresses aren't the same. It's surprising if Windows doesn't use them for identification.
God dammit (Score:5, Funny)
Every computer using geek has an Asian girlfriend except me.
How they fuck do they do it?
You have to be white, nerdy, and socially awkward. I've got all three! Come hither, ladies!
Re: (Score:2)
Meet them, talk to them and convince them that you are a good person. Fixing their computer and paying for their internet access helps too.
Re: (Score:2)
Actually the lure of the exotic cuts both ways. To Miss Whateverstate you may be just some loser nerd from a similar high school that grew up watching B grade TV comedy. To a girl that grew up on a farm in China near the Russian and North Korean borders you may be an interesting technocrat with an impressive command of English and full of witty comments she's never heard before.
Re: (Score:2)
To a girl that grew up on a farm in China near the Russian and North Korean borders you may be an interesting technocrat with an impressive command of English and full of witty comments she's never heard before.
I'm not sure if you meant to say that you think this describes my girlfriend. But if so, that's extremely racist. Christina was born and raised in Boston. And I'm dating her because she's a gamer geek, not because she's Asian.
Think that this song is about you? (Score:2)
Also why assume it's racist? It could just as easily describe a Caucasian that has grown up in a different environment to the person she is attracted to.
Re: (Score:1)
Re: (Score:1)
Re: (Score:2)
The girl from a farm I used as the example speaks and writes very well in four different languages. I was writing about differences in how people are seen from different viewpoints.
Re: (Score:1)
Actually the lure of the exotic cuts both ways. To Miss Whateverstate you may be just some loser nerd from a similar high school that grew up watching B grade TV comedy. To a girl that grew up on a farm in China near the Russian and North Korean borders you may be an interesting technocrat with an impressive command of English and full of witty comments she's never heard before.
For my part, I started dating Kenton not because he was exotic to me, but because he understood me better than almost all of the guys I've dated before. I was a linguistics major and I have an abnormally large vocabulary and tend to use pretty complicated syntax - Kenton never seemed to be put off by that, in contrast to many guys I've known. We also have a lot of common interests, video games being one of them (I work in game development). I personally think I'm wittier, but that may be because I tend t
Re: (Score:2)
Now it appears you both think I'm writing about you - I'm sorry I gave that impression.
Re: (Score:2)
Re: (Score:1)
might be a trend there, three out of the other six IT techs in my employer's company have asian wives as I do. there is a traditional cultural thing where asian women are not treated well by the men. in the USA they see that things can be different, other men have proper respect for women. if you are kind, dependable, genuinely interested in them, have a job you can be their Mr. Right.
I have also seen that there are many Asian women dating or married to software engineers, but I think it would be a mistake to believe that they all a) were born and raised in Asia, b) have a past history of experiencing patriarchal abuse or discrimination, or c) are involved with their partners only or primarily because of the kindness, dependability, and genuine interest that their partners show in them as contrasted with their previous partners (or other role models they've seen via the media, relatives
RemoteFX (Score:1)
32Mbit = somewhere under 6 in the 3-6 mbps range (Score:2)
So he does not seem to get the full 6 mbps that http://www.sonic.net/ [sonic.net] offers on DSL. Must be to far from the CO or RT to get the full speed.
he bases Comcast but they have faster speeds and better upload. But the Sonic.net directv bundle is better for TV.
Re: (Score:2)
I'm not quite sure what you mean. How is 32 less than 6?
A typical Comcast package appears to be 15mbps, according to their site. Comcast Business Class does offer pretty good bandwidth, though.
Re: (Score:2)
sonic.net also has 20Mbps or up to 40Mbps broadband on Adsl+2 but 40 is dual line.
Re:32Mbit = somewhere under 6 in the 3-6 mbps rang (Score:4, Interesting)
sonic.net also has 20Mbps or up to 40Mbps broadband on Adsl+2 but 40 is dual line.
Yes, that's what I have. The 40Mbps dual-line "Fusion". Due to distance from CO, it gets 32Mbps, but it is a consistent 32Mbps.
Re:32Mbit = somewhere under 6 in the 3-6 mbps rang (Score:4, Informative)
I think he's having a mental fault and mixing units.
32mbps = ~4MByte/s
which is better than the ~24mbps I get off Comcast (a solid 3MB/s off Steam content servers, at least until my cablemodem starts having seizures.)
Fake!!!! (Score:4, Funny)
This is a made up story by the liberal media.
Blizzard clearly stated that no one is interested in LAN parties and the whole concept is dead. Please move on. There is nothing here to see.
No.. (Score:2)
The question isn't "How", its "Why".. money doesn't seem to be the big issue here, so why not spring for Server 2008R2 and manage all the boxes from there? it does all this updating/registering/etc your hacking together, and for around $800, versus your hourly rate x hours hacking, seems less expensive and the result is a heck of a lot more manageable. I'm all for the do-it-yourself type, but managing disk images? Yeah I can spend my time better elsewhere
Re: (Score:3)
The question isn't "How", its "Why".. money doesn't seem to be the big issue here, so why not spring for Server 2008R2 and manage all the boxes from there? it does all this updating/registering/etc your hacking together, and for around $800, versus your hourly rate x hours hacking, seems less expensive and the result is a heck of a lot more manageable. I'm all for the do-it-yourself type, but managing disk images? Yeah I can spend my time better elsewhere
No, it really does not do anything like what I'm doing... and anyway, setting this all up was lots of fun.
Disk images are trivial to manage... (Score:3)
Why bother spending $800 bucks for an OS you may not particularly like the style of anyway? For a task like this, I really don't see a particular advantage that Windows would hold unless all you know is Windows. The tools to do this sort of stuff are trivial to work in Linux (I personally think easier than Microsoft tools, but that may be a preference).
I would give MS the benefit of the doubt on a setup suggesting AD account management, though I haven't tried 389 which may have a nice integrated feel. St
Remote Stations (Score:2)
Re: (Score:3)
Monoprice sells 32' USB extensions that have a repeater on the end. I'm not sure how they get away with just one repeater instead of two as seems mandated by the USB spec, but they do and it works.
Re: (Score:2)
Also, do you use hubs on the end for the keyboard/mouse or do you run a separate cable for each one? If you use hubs, have you run into issues with using them? Do you have gaming quality peripherals on the end as well (high dpi and polling mice and high poll rate keyboards)?
I ask as gaming peripherals tend to put more strain on USB connections then no
Re: (Score:2)
Does this result in noticeable input latency that you can feel through gaming, similar to the sort of input drifting feeling you get with wireless peripherals?
No. According to the USB spec, maximum allowed round-trip latency to any peripheral (regardless of the number of hubs) is 1.5 microseconds, many orders of magnitude below what a human being could sense. Electricity will propagate over a 32' cable in around 50ns, another couple orders of magnitude smaller still.
Also, do you use hubs on the end for the keyboard/mouse or do you run a separate cable for each one? If you use hubs, have you run into issues with using them? Do you have gaming quality peripherals on the end as well (high dpi and polling mice and high poll rate keyboards)?
I have a hub attached to each one, with a cheap keyboard and a Logitech MX518 mouse. Guests are welcome to bring their own peripherals if they so choose.
I ask as gaming peripherals tend to put more strain on USB connections then normal peripherals and can crap them out when they're operating on the edge (like the 32' repeater you mentioned). They draw more power and operate close to limitations of the spec in terms of latency.
The hubs are powered. They are USB 2.0, so
Re: (Score:1)
Buy a G500 and do a side by side comparison with your MX518. It's not snake oil.
I was overall just curious if
Nice, but... (Score:2)
If you need more than a damned good switch, internet connection, and wireless AP to optimize your LAN parties (real LANners have their own boxen, TYVM) you're doing it wrong.
Re:Dude, that's lame (Score:5, Informative)
Honestly I'm kicking myself for not having written everything up before going public. The vast majority of people who saw the original post will not see the technical details. :/
Re:Dude, that's lame (Score:5, Funny)
It's like you went public beta before working out the kinks~
Re: (Score:2, Funny)
That's google for ya.
Re:Dude, that's lame (Score:5, Funny)
Release early, release often. I thought that was the way we're supposed to work.
Re: (Score:2)
I'm blown away by how much you put into the project. The cabinet rigs in particular came out really slick. Congrats, and go get your employer to use it as a demo for SketchUp.
Re: (Score:2)
get your employer to use it as a demo for SketchUp. ;)
Or perhaps to follow Microsoft's lead and create some games, as it leads into experimenting with lots of technologies. Perhaps Linux gaming will get a boost too.
Re:Dude, that's lame (Score:4, Insightful)
Honestly I'm kicking myself for not having written everything up before going public. The vast majority of people who saw the original post will not see the technical details. :/
A lot of people wouldn't care about the technical details either. You can be a gamer without being a geek.
Kickass house, dude. Pay the AC no attention, envy is not a pretty thing. Thanks for the technical article, there's definitely stuff in there everyone can use, even if they don't have a house purely optimized for LAN parties. In fact, I wouldn't mind seeing additional details, tutorials on setting up the images and the like.
Re:Dude, that's lame (Score:5, Interesting)
Re: (Score:2)
As the author of Syslinux/PXELINUX I found the article rather interesting and enjoyable. :)
Thank you for your useful software! Sadly I'm no longer using pxelinux since I switched to Windows, but I believe I'm still using your TFTP server. :)
Re: (Score:2)
I love you.
Re: (Score:2)
Re: (Score:2)
Re: (Score:1)
Re:Dude, that's lame (Score:4, Informative)
You do realize that it is extraordinarily easy to install cat5e wall plates and drops in most modern homes, right?
You can use powertools, have an rj45 modular crimping tool, and know where to buy bulk cat5e right? (If not, can I please verify your geek card...)
All you need is hidden utility closet to house the punchblock and local switch hardware, and you are golden. With how small some of this hardware has gotten, you might even be able to get away with a breakerbox enclosure from lowes, assuming you put some ventilation in it.
My current home was once refit as a beauty salon by the previous owners, and has so may utility hookups in the living room that I could have a christmas tree made entirely of christmas lights and not blow any fuses (hookups for dryer chairs have beefy amp ratings). If I wasn't such an antisocial recluse, and actually had lan party friends to come over I could really do crazy shit with my place too.
But I don't, so I haven't and won't. But if you want to run cat5e in your house, the only thing stopping you is inertia, since as far as I know you don't need an electrician to run the stuff, being so low voltage.
Re: (Score:3)
The only thing stopping me from running cat5e in my home is not inertia, it's the large structure made of wood that runs up, down, and horizontally through the walls. Unless I wanted to cut holes in the walls so I could drill pass through holes between floors, there's no way to do it completely in the walls. Of course, you could put up molding in the corners and along the ceiling to hide everything. Some might even go under the carpet (which I did in one room).
Re: (Score:2)
Generally that problem can be solved with 36 inch flexible drill bits (they exist and are awesome and not too expensive), maybe some flexible drill bit extenders (you may need the equivalent of a 100 inch drill bit), and some fishtape. Usually, you just drill down until you hit the unfinished part of your basement, or you drill over until you hit an air duct and run plenum rated through that with the fishtape.
Re:Dude, that's lame (Score:4, Interesting)
The right way to do it requires no more than a 36' drill bit. If you are ordering online or with a building supply guy who is savvy, what you are looking for is called a bell hanger bit. They are not only quite long, but very flexible. (being largely spring steel, they snap quite easily if you drill too far off axis though) They commonly come in the smaller sizes needed to run simple low voltage lines like thermostats and door bells (hence the name) but can be found in larger sizes needed to run Ethernet. They are just the ticket for cutting a hole big enough for the outlet box and drilling from outside that, down through the base plate and into the cavity below. You then go downstairs and open another small hole and drill down to the basement. Having an assistant is invaluable when fishing the line, s/he can listen for the fish tape knocking in the first floor cavity if there is any doubt at all where your drill came out.
Even with Plenum rated cable, I'd avoid running it in an air duct if there was any way at all to avoid it. It's better to go around, using an extra handful of meters of cable for the longer route, then it is to have the duct cleaning guy get his brushes tangled in it next year. If you do use the duct anyway, use a grommet where it passes through the sheet steel, not only to protect the cable from being cut by the sharp edge, but also to reduce leakage from the ducts into the wall cavities.
Re: (Score:2)
Being as how I have been thinking of this much recently, how would you recommend I handle my situation? I have a townhouse built on a slab, two floors and an attic. How do you run a cable from the attic to a lower level through the second floor? I have thought about using the furnace exhaust, but would be afraid of melted cables there...
Re: (Score:2)
Re: (Score:2)
The main issue I have is running through two levels. I suppose the bendy drill bit would work, just use a old telephone cable hole that is in my bedroom on the second floor for the runs to the lower level.
My next house will have a basement, now I see the need for it :)