John Carmack on Coding a Linux IP Stack & Winmodem 424
An anonymous reader pointed us to an interesting article over at Shugashack talking about John Carmack wanting to
rewrite the Linux IP stack and write a software Winmodem driver for Linux. He wants to squeeze the latency out of both of these components to try to kick MS into gear to write better stuff on their side. If it turns out true, it could be pretty cool for gamers on both platforms. Don't put any stock in it, its just a rumor, but its interesting.
More info? (Score:1)
rewrite what stack? the one in 2.2 or in 2.4? (Score:1)
Linmodems.org (Score:5)
^-- This group has been working on the winmodem drivers for linux for quite a while. There are even plenty of goodies to download.
Pretty cool for just about everyone (Score:2)
Re:Carmack Is A Selfish Bastard (Score:1)
Hmmm... let's look at what he's done, release quality games for Linux, released source code to some excellent games, done some great stuff for openGl. John Carmack is the man!!
I Wonder... (Score:1)
Go Carmack Go... (Score:2)
I applaud Carmack for his work, if he is in fact going to rewrite the IP stack. However, does he, or anyone else for that matter, think this will "pressure Microsoft" to modify their implementation? Hasn't Microsoft operated with this air of "We'l do what we want, when we want?" Why should this change that now?
You are a Selfish Bastard (Score:1)
linmodems (Score:1)
-russ
Re:I Wonder... (Score:1)
/jarek
Do we want to allow other OS's to use winmodems? (Score:2)
On another note: has anyone figured out how to make an external winmodem yet?
Interesting... (Score:1)
Phyrkrakr
"God doesn't play dice"-Einstein
Hoooooold on - (Score:1)
Bzzzzzzt.
Latency is caused by a ton of factors - and the IP stack, unless you're talking about running thousands of clients on a single, multi-processor machine, aint really one of them.
Chalk it up to the rumor mill.
--
Blue
Re:I Wonder... (Score:1)
Carmack (Score:1)
http://slashdot.org/users.pl?op=userinfo&nick=J
maybe he'd like to comment?
Re:I Wonder... (Score:1)
-----------
"You can't shake the Devil's hand and say you're only kidding."
MS/BS (Score:1)
Paolo Wrote:
both vendors would be challenged to improve TCP/IP Performance in order to keep pace with Linux
Ok, you might be right about Apple, but I doubt Microsoft would try to improve their tcp/ip stack. Microsoft would instead realease anther FUD saying that it (Microsoft) has a better overall performance or even better stack for some obscure reason other than standard benchmarks or tests.
Those who don't study history will repeat its errors.
Those who don't will find new ways to error.
-author unknown
Re:Carmack Is A Selfish Bastard (Score:1)
If you've never played any of his games of never benefitted from any of his work, then why would you even care what he does?
If you have benefitted from his many years of work, then get over yourself and let the guy do whatever he wants to do. He's earned it!
John's just a man, but it amazes me how many people demand that he keep slaving away at making more games for them, like he's some sort of indentured servant or something.
Of course I know I've just wasted my time talking to someone who probably hasn't done a thing for anyone but himself.
Won´t help, "PowerPlay" looks more promising (Score:1)
Tweaking a winmodem driver however might result to some reasonable gain (connect with sub-max but reliable speed, switch off compression, tweak modem protocol). But this doesnt change anything on the ISPs side. And doesnt help anyone with an external modem. Or ISDN/ADSL/Cable.
Valves PowerPlay looks much more promising: router-priorization of game packets (less ping, less packet loss), UDP header compression, bandwidth guarantees, later a special modem real-time protocol. And some more stuff like multicast for voice etc.
Linmodem is a waste of time (Score:1)
I really dislike software modems. If you want to decrease latency attributable to modems, the way to do it is optimize the modem firmware ala USR.
The reason (Score:5)
From a Gamespot write-up [gamecenter.com] on PowerPlay:
PowerPlay, a set of standards and protocols that will apply to both game developers and Internet service providers (ISPs). By working on both sides of the problem, PowerPlay promises to bring LAN-like gameplay to a modem near you. If you think that sounds like something that is well beyond the range of a single company, you are right--and that's why Newell got Cisco Systems involved.
Newell is Gabe Newell of Valve, makers of Half-Life. The reason Carmack's doing this is to contibute to the PowerPlay project.
John...wants to take a stab at rewriting the Linux IP stack and a soft modem driver to see how much latency he can remove, and use that to pressure Microsoft into cleaning up the Windows stack. That would be really useful data to have.
And here's the official PowerPlay website [powerplayinfo.com].
Pablo Nevares, "the freshmaker".
Latency Reduction (Score:3)
I believe part of Carmack's goal in rewriting the stack and working with softmodems is to specifically allow game optimizations. (duh)
One limiting factor in Quake's internet performance is the lag time created by poor modem implementations. Many modems are excessively stringent about enforcing buffering, with the effect that my ultimate last ditch blaster shot is delayed because the 'modem' wanted to fill its buffer.
In theory, rewriting the stack and modem support while stress testing it with a busy quake connection will allow inefficiencies to be eliminated.
I doubt that there is anything 'wrong' with current linux implementations - they are just naturally focused on http/ftp designs.
Zephyr
Alfredo
Re:Do we want to allow other OS's to use winmodems (Score:2)
If the Linux community ever wants to see Linux catch on for the general public, Linux will have to address the needs of the general public. One of those needs is that we'd like our winmodems to work under it. If the Linux community banded together and decided that they didn't want winmodems supported, it could only hurt the OS's chances of winning over "average" users.
If you're to have Linux as the OS of the elite, that's one thing. But if you're looking to see Linux catch on, this is a step that I think needs to be taken.
Re:Go Carmack Go... (Score:3)
Interesting Microsoft patent in this area... (Score:1)
I haven't spent the hours required to understand all the details, but it sounds like every network stack I've ever worked with. It has a bunch of extraneous implementation details thrown into the claims to make exact matches of prior art unlikely, but that's probably just to drive up the cost of litigation.
I say "go on you silly patenters". It will take a large mass of ludicrous examples to motivate enough voters to overcome the industry lobbyists.
Re:Go Carmack Go... (Score:1)
At least if nothing else, MS will spin some fud on why latency is not such a bad thing.
Re:Do we want to allow other OS's to use winmodems (Score:2)
An external winmodem? The closest thing I have seen is from the Sound-HOWTO [l0pht.com] about using the sound card to impliment 9600 bps FSK methods. This method is more useful for hams, but I'd imagine you could use a modemless laptop coupled to a payphone handset in a jiffy.
Re:Do we want to allow other OS's to use winmodems (Score:2)
And I never have a problem finding a real modem now. What problems are you having finding a real modem? Almost any computer parts dealer, online or not, carries regular modems in plenty of stock.
John Carmack (Score:1)
Re:Do we want to allow other OS's to use winmodems (Score:4)
Of course *real* modems have their advantages, especially for people who want the least CPU utilization and lowest ping times, especially gamers or for dial-in servers. But there isn't anything inherently evil about software-based modems that require us to stamp them from the earth. Remember the big trend a couple years ago towards multipurpose DSP's that could drive your modem and your sound card (for instance, MWave)? They're great ideas... IF the companies that designed and built them allowed others to tap into the power and support other platforms.
--bdj
You can't remove most of the latency in modems (Score:1)
Re:I Wonder... (Score:3)
If you think it's not flawed, you have a serious misconception about Linux.
If you think Alan Cox would be perturbed by someone wanting to fix it, you have a serious misconception (or, more probably, a series of them) about Alan.
Now, is it fatally flawed? No, of course not.
BTW, although Alan did the initial work, there's an awful lot of other people's code in there. I don't hang out on the kernel lists, but I don't believe Alan would have a claim on it as "his code", even if he were inclined to make such a claim.
It's moot, however, because the GPL makes it our code.
A bit more than heresay (Score:4)
Blue's News went into a bit more detail about why this quote means what it means. Yesterday, Cisco, Valve, and others mentioned a new concept called PowerPlay which is something that will involve ISPs, equipment, and software code. Epic is already coding PowerPlay information into the Unreal Tournament engine so that they and others will be able to take advantage of it. Basically, PowerPlay is a ISP-side enhancement to improve the quality of dial-up gameplay.
The reason Carmack is quoted here is because id wasn't on the list of companies officially supporting PowerPlay. As you can well guess, this raised many eyebrows in the gaming world, and the reason is simply that Carmack didn't want to be involved with the implementation of it. No doubt, id will put code in their upcoming products, but Carmack himself doesn't want to do work. Why? Other projects.
Also, Gabe Newell is pretty well respected in the gaming community, so these words probably aren't meant to give rise to any rumor mills. They're probably the truth.
So as a 'rumor', it's a very well-founded one. Carmack is familiar with Linux program, so I'm sure the concerns about Carmack possibly interfering with the work of others will turn out to be baseless. It'll all work itself, and if we get any code out of Carmack, it's bound to be very good code.
Re:Do we want to allow other OS's to use winmodems (Score:1)
I have to admit I bought a WinModem for my in-laws over Christmas (a replacement for a "real" modem) as an experiment since performance is foremost in her mind. It actually works surprisingly well. I haven't noticed any real performance problems, although browsing tends to be modem limited rather than CPU limited (well, if your using IE it is. If your using Netscape, the latest Cray isn't enough.)
---
Good Job, Taco! [OT] (Score:1)
Glad to see you are taking some editorial responsibility. When rumors are posted, they should be marked as such. I can remember a several times when
Maybe stories should have a rumor flag on them, so that anyone who wants only serious news can filter? Just a thought.
Re:Do we want to allow other OS's to use winmodems (Score:1)
oops... I should have said "not foremost in her mind".
---
Re:Do we want to allow other OS's to use winmodems (Score:1)
Re:I Wonder... (Score:1)
The IP stack? (Score:2)
I find it curious why he would want to tweak the IP stack. I'm sure you might be able to squeeze a bit more efficiency out of it, but I can't imagine shaving off more than a 0.5ms (much less?) versus the 100-150ms the modem introduces.
---
Re:Carmack Is A Selfish Bastard (Score:2)
1. Nobody is forcing anyone to purchase Quake or any other game.
2. His changes would be _free_.
3. I assume his changes would help others, not just Quake customers, right.
4. So what if a person enjoys nice cars? I know I do. More power to him.
5. Don't be an asshole. He's pry brought more enjoyment to people than you ever will.
- Jeff A. Campbell
- VelociNews (http://www.velocinews.com [velocinews.com])
Re:I Wonder... (Score:1)
Re:Go Carmack Go... (Score:1)
You're not that great of a source, either, bub. And this is coming from a person who doesn't even like first person shooters all that much, so don't give me your "John Carmack weenie" crap.
- Jeff A. Campbell
- VelociNews (http://www.velocinews.com [velocinews.com])
TCP is too slow... (Score:2)
Exactly... try the serial, TTY and PPP drivers (Score:2)
However, you really don't want to have the PPP code in the serial interrupt handler, which is where it would have to live if one were watching each byte as it comes from the UART.
Re:MS/BS (Score:2)
Redmond, WA - Today, in reponse to Linux' high performance TCP/IP stack, a Microsoft spokesperson announced that they would not be changing their software. "Suckers^H^H^H^H^H^H^HConsumers are not interested in a fast TCP/IP stack. As our tradition is, we listened to the consumer and are going to provide the ability to display 92 billion shades of green. Suckers^H^H^H^H^H^H^HConsumers like colors, and we feel that this continues to provide the value in our products that suckers^H^H^H^H^H^H^Hconsumers have come to know and expect. Bill Gates had the follow to say at the press release. "I choose 92 billion because that's my personal net worth. Oh, and green because that's the color of money, and I like money."
-BrentRe:Carmack Is A Selfish Bastard (Score:2)
Any other coder would be very happy to have any form of recognition (this is not a 'bad thing'). Just because someone has attained it means nothing, other than a combination of luck and skill. What's wrong with that? In the end, he ends up with a little publicity, and Linux ends up with some useful code - for free.
I'm amazed you know Carmack so well. How are you so damned certain what he wants and doesn't want? It seems to me that this is a Good Thing for all involved.
- Jeff A. Campbell
- VelociNews (http://www.velocinews.com [velocinews.com])
Re:Go Carmack Go... (Score:2)
You know, troll, you need to learn to read before you can write.
As I said, I don't even really care much for most of the software Carmack has written (ie. I don't play first person shooters very much). What I am saying, however, is that there is NOTHING negative coming from this. So what if he receives public recognition? Just about every coder out there would love to be well regarded for what they do. Don't blame Carmack because he has been successful at it.
You have never said what it is exactly that you have against the man? There are a bunch of other famous coders out there - aren't they the same? Is it their fault they are well known?
- Jeff A. Campbell
- VelociNews (http://www.velocinews.com [velocinews.com])
Competition (Score:2)
Besides, the whole idea of Linux pressuring MS to improve ought to bring a smile to the face of every consumer on earth -- this is that whole "competition" thing we've been missing throughout most of the 1990's.
----
Re:Carmack Is A Selfish Bastard (Score:2)
In the great scheme of things, that ranks right above winning a farting contest.
---
Oh, I see. You're in competition. That makes things more clear.
Anyhow, coding a 3d game like Quake isn't exactly trivial. If it is, please supply a link to your own code and we'll talk.
- Jeff A. Campbell
- VelociNews (http://www.velocinews.com [velocinews.com])
Re:Interesting... (Score:2)
Well, I kind of assume that if this is more then a rumor, it'd be something that was developed during 2.5, not something whipped out for a frozen 2.3 and then not integrated.
-BrentRe:Go Carmack Go... (Score:5)
It seems they're very afraid of him for two reasons. One has been touched on several times here already; if he chooses to support some sort of gaming api that they didn't develop (graphics, sound, what have you), it makes it nearly impossible for them to gain a stranglehold on that aspect of the system.
Their real fear, though, is apparently that his gaming engines are damn near operating systems unto themselves, and should he decide to turn them into a 3d operating environment, he'd eat their lunch with it.
The report that the guy returned to them basically said it remained to be seen if Carmack was interested in such a thing, but that should he decide to do so he would pretty much own that arena. So I'd say that should he do something with the specific intention of making them pay attention, pay attention they would.
Skills (Score:2)
Does all of that skill necessarily imply that Carmack can necessarily write a stack for Linux, that is truly better for games? I mean, it'd have to be able to handle all sorts of hardware, all sorts of circumstances, SMP, etc. etc. And, we must remember that kernel development is pretty different from app development in many other ways, too..
The article, after all, phrases it as "take a stab at" the project. Maybe he'll just flick a few bills off his megawad of cash, and fund some development, should he not eke out enough milliseconds to bug MS. heh heh
Because ping is king... (Score:2)
modem latency (Score:3)
So a software modem that knew ip could help a lot, and be much nicer than an old style hardware modem
Re:Go Carmack Go... (Score:2)
--
Re:Carmack Is A Selfish Bastard (Score:2)
--
Re:modem latency (Score:2)
Those 300 interrupts you talk about are just to move memory into the buffer and hardly CPU intensive. Software modems are a different beast, where ALL off the free CPU time is spent trying to mathematically create a decent signal. Now, if we had this "software modem" flashed to the modem's DSP, or offloaded to an alternate processor off the main bus, this would be a huge win.
Re:Pretty cool for just about everyone (Score:3)
I don't quite get this argument that if Linux improves its TCP/IP stack, the Microsofts and Apples of the world will begin shaking, suddenly stop what they are doing to completely rewrite a critical piece of their respective OSs that have been through years of debugging. I mean, Oracle makes a hell of a better database with many more features, but did that stop them from releasing Sql 7.0; this software is basically a buggy piece of shit and after one service pack there is still this fucking lame trusted connection bug that affects every single one of our users every single day AND THEY STILL don't have a fix for it yet!!! So I guess you will be waiting a LONG time for your newly rewritten TCP/IP stack. And it's probably just as well, anyway. That's what makes Linux so alluring, so attractive. Bugs get stomped out right away, and the ones that don't at least you can fix yourself!
Re:Go Carmack Go... (Score:2)
If he had said something like that 2, or even 1, year ago I would have laughed too. We are living on Internet^h^h^h^h^h^h^h^hLinux time, though.
Why do you think MS got W98SE out the door so soon after the release of W98? They felt the "pressure" of people jumping on the Linux bandwagon to get ipmasquerade. They were in such a hurry to do this that they went out and bought NAT2000 instead of developing it in house.
3D/online gaming is going to be a big hit on Linux during the next year. Within 6 months, we'll have a fast and stable OpenGL environment and good drivers for most or all consumer 3D accellerators. If Linux can run those games better than Windows (i.e. so much better that it is mentioned in game reviews in game magazines), you can bet your ass that MS will feel pressure.
MS is going to copy anything that is better in Linux to try to minimize the number of people going to Linux because of missing features in Windows.
also... (Score:2)
Re:Carmack Is A Selfish Bastard (Score:2)
Then again, the same could be said for natural selection, but look how THAT turned out. In theory, these trolls shouldn't even exist.
- Jeff A. Campbell
- VelociNews (http://www.velocinews.com [velocinews.com])
Re:Carmack Is A Selfish Bastard (Score:2)
What of it?
If he contributes something where it is needed, it's still a good thing. If one of these phantom 'better coders' comes in and improves on his code, even better.
Doesn't mean you should rail against the guy. You're not being forced to play his games, and if his changes are less than desirable, I assume Linus wouldn't include them. It's a win-win situation. Why the anomosity then?
- Jeff A. Campbell
- VelociNews (http://www.velocinews.com [velocinews.com])
Re:Latency Reduction (Score:3)
Yes, of course. I didn't realize right away why Carmack would want to rewrite the softmodem code - essentially he'd want to write his own modem and optimize it for latency. Knowing his style, I'd say chances are he'd optimize for throughput, reliability and low cpu overhead as well. That done, it would be a trivial step to take the linmodem code and package it as a real modem, thus standing the the modem industry on its ear.
There's another part of the chain that needs to be rewritten and that's the Linux scheduler. In order to produce an excellent linmodem implementation with minimum latency you'd have to have garaunteed realtime response in the kernel. It's not so hard to do (thought doing it extremely well requires the usual talents) and it's my understanding it's being looked at by a number of people - Andrea Arkangelli's name comes to mind.
How to get latency down (Score:2)
As I understand, Linux supports the ToS bits. Using the "minimize delay" bit together with an ioctl to REDUCE the size of the modem driver's outgoing buffer would probably have more impact than just about anything else one could hope to do, at the kernel level. Is there really some other aspect of the kernel that causes latency that I am not aware of??
At the modem level (winmodem driver), I could imagine again reducing the buffering, so that multiple packets without the ToS minimize delay bit don't get buffered in front of time critical packets. Turning off V42, V42bis, etc would help latency considerably, but when the errors occur, you're stuck with retransmitting, and you're stuck behind buffers before your retransmitted packet can get sent, so latency with errors is even worse without error modem-level mgt. Maybe a new modem-level protocol using Reed-Solomon (or similar) error correcting codes, with feedback (as low urgency packets) to adjust the number of error correcting bits added, so that there would be a very low likelyhood of ever retransmitting a packet...
It's hard to imagine they'll go after the actual modulation itself or clock recovery. There is latency involved in a Vertibi decoder, but my (limited) understanding is that it's usually only a handful of bits.
So if you've read all my babble, what do you think could really be done in the kernel and/or modem itself to lower the latency. What do you think the powerplay folks will really manage to do before March?
...AC, I know, should really get a slash account someday
paul@pjrc.com
Re:modem latency (Score:2)
Don't know where you get your figures from, but the FIFO saves you interrupts - it doesn't increase them. It means you get 1 interrupt per 16 bytes of data - not 1 interrupt per byte of data. You can always turn the FIFO off.
Second, the modem hardware is good for at least 60ms of your latency, acording to the guy behind an old popular macintosh network game Bolo, which was writen to studdy networking issues.
Quite possibly - but given the baud rate of the average modem, that 60ms is bog all.
So a software modem that knew ip could help a lot, and be much nicer than an old style hardware modem
Gimme a cable-modem any day.
The biggest problem on PC's is the old UART chips just don't cut it. They're badly designed - instead of having RTS/CTS flow control with highest priority, and letting the chip handle it all automagically (like the Phillips IM26C91 family of UART/DUARTS and quad-UARTS which are exceedingly good in my experience), you have to bring the CPU into the process so that you can manually flip the RTS/CTS lines. Which frankly, sucks and causes latency problems.
*sighs* Legacy hardware - don't you just love it?
Si
Carmack is just freaky-cool! (Score:2)
Personally I think the person who came up with the idea of softmodems should be stripped of his degrees and forced to go through a college CS course. I don't know about these days but one of the first things we learned in Hardware 101 is that you buy cheap UARTS so that your expensive processor doesn't have to dick around with serial. Of course, modems are passe' and no one will be using them in a couple of years anyway, but it's the principle of the thing...
A few things were taken out of context... (Score:5)
No way do I intend to rewrite the linux TCP/IP stack.
I had mentioned to Yahn that it would be an interesting experiment to yank all the networking code (TCP/IP + PPP + serial driver) into user space so some cross-boundary optimization experiments can be made. I doubt there is any apreciable latency in the TCP/IP stack, but there might be some scheduling losses somewhere through PPP and the serial driver. Even if a send packet call goes syncronously all the way to the serial ring buffer, giving no real potential for latency reduction, there are still lots of possible experiments with making decisions based on normally hidden data.
Just like the GLX driver work is Good For Me as a graphics programmer, going through all the guts of the networking stack would be Good For Me as a netowrking programmer. I may pursue this, and I may collect some interesting data, but I seriously doubt it would be any contribution to the standard network stacks.
I had a long talk with a couple people from Valve about the PowerPlay initiative, but they couldn't give me enough specific technical details for me to endorse it. I'm all for improvements in networking infrastructure, but at this point, there isn't anything actually there, just an intention to improve gaming. They need to tell me SPECIFICALLY what I am supposed to be endorsing. At some point, bits have to go into packets and routers need to make decisions on them. Changes at that level is what I want to hear about, not strategic company relationships.
Some networking things that could be real improvements:
UDP header compression. I am still surprised this isn't a standard.
An ioctl to set modems to a realtime mode that makes different tradeoffs in modulation, error correction and compression. I'm not sure if the modem standards allow things to be renegotiated dynamically or not.
Making routers actually pay attention to QoS bits, and defining specific queuing behavior for them. A "realtime" packet should never be added to a queue if there is another one from the same connection that is already in the queue. Multiple queuings just add latency that can't be controlled at the application level. Jump-to-head-of-queue would sure be nice, but that's probably too much to ask for in the name of "games".
I have commented in the past that software modems have the (unrealized) potential to reduce latency over conventional modems. There is some savings by just avoiding a serial FIFO, but more savings could be had be integrating more knowledge from higher levels of the communication stack. Specifically breaking modem comrpession/error correction packets at PPP boundaries (or avoiding them altogether when apropriate), for instance.
John Carmack
Do not confuse the modem with the UART (Score:2)
What you are describing are deficiencies in the standard PC UART (based on National Semiconductor P/N 16550A). It has been known for over a decade that the 16550A is unsuitable for anything like modern, multitasking systems. Unfortunately, like so much of the PC, we are tied to a legacy standard choosen twenty years ago for completely obsolete reasons.
If you want to improve modem efficiency, use a better serial interface and larger buffers. USB would be a good start, as there is already a standard. Make the interface tunable, so you can use larger buffers with fewer interrupts and higher latency (good for web browsing, downloading, etc.) or smaller buffers and more interrupts to reduce latency for gaming.
However, none of this has anything to do with the modem itself. Moving the modem DSP functions off the modem card and onto the system bus and CPU increase overhead and latency, without a significant savings in hardware costs. As near as I can tell, they are popular only because the current commodity market makes saving three bucks a unit significant when you are building a million of them.
The rewrite is already in progress (Score:2)
While the 2.2.x IP stack certainly requires a rewrite, the Linux kernel networking gurus have already thought of that, and since 2.3.15 (so in 2.4 final) the networking is fully multithreaded.
The really big problem is that much of the power/scalability gained by the multithreading (which in essence consists of eliminating global kernel locks) is not used because of the Linux core design (the bottom-half interrupt handlers are globally serialized).
Of course, there's a solution to this too: rewrite the interrupt handling core. This is already being done under the misterious "softnet" name (go here [sunet.se] for details). This won't go probably into 2.4 because it involves too many fundamental changes.
Re:So much certainty, so few facts. (Score:2)
Re:Linmodems.org (Score:2)
And they don't seem to have much in the way of source code, other than a piece that will take a lucent-based winmodem on or off hook.
They do, however, have the closed-source binary that Lucent wrote to support their pci winmodem under linux.
Re:Anti-trust (Score:2)
Face it -- the average PC buyer wants spend $800 and get a complete package including monitor, printer, modem, *and* a respectable Megahertz number. Because they can't include last year's CPU, the OEMs pretty much have to cut corners everywhere else. So they save $5 and package a winmodem -- no big conspiricy.
Since many Winmodems don't even work in Windows NT (or 2000?), I'm sure Microsoft isn't that happy about the situation either. It deprives them of some upgrade revenue. On the other hand, if you have the secret Bill Gates memo ordering Dell to ship winmodems to stop the evil 1% market share of Linux and OS/2 users, you'd better put up or shut up.
--
Re:Do we want to allow other OS's to use winmodems (Score:2)
I do believe that it is quite within the world of Microsoft to have dedicated modems, DSL or otherwise, that require Windows software to operate, with integration similar to the WinModem. Such a thought distresses me, on several levels, and my wish is for choice.
Re:... John-Carmack-Turing-Test (Score:3)
LetterRip
We don't have to have just one stack (Score:2)
A full stack with all the stuff needed for all the different parts of TCP/IP and the kinds of buffering that is needed for all this different ways it will be used.
AND a striped down games stack that only talks to other stacks like itself. It would not need to respond to anything but gaming packets. You could also move part if not most of the stack to the ISP side of the wire. this could remove a lot of overhead. I mean what could be done in terms of NETBIOS type stack for local LAN (not routed) gaming. Who needs IP addresses if you are all on the same segment? heck.
The same kinds of things could be down with win modems (if we can figure out how to control them) no buffering low latency exect...
The point is that we can have choices we can do thing differently we don't have to be locked into a single stac
Re:Software Modem (Score:2)
That will never happen if you count the additional CPU overhead of doing things like DSP and error detection/correction in software against the software modem.
I'd prefer to buy the software modem, because they're about half the price.
You get what you pay for. Software modems will always be a drag on CPU and RAM resources in your machine. A friend of mine says that he has measured between 7-10% extra CPU usage when running a Winmodem in his P-III 450 under NT. Is 7-10% CPU on a processor that fast worth saving $20 on a modem? Buying a CPU that is 7-10% faster will often cost you more than $20 difference.
Re:It's not the stack (Score:2)
I can't understand why a serious gamer would buy a Winmodem. They pay huge premiums to get the fastest 3-D accellerated video cards and sound cards and the fastest CPU's, then they scrimp on a modem that is going to suck 7 to 10% of their speed back away. Crazy. Modem manufacturers should specifically label hardware modems as better for gaming to justify the slightly higher price.
Making QoS work for gaming (Score:5)
Making routers actually pay attention to QoS bits, and defining specific queuing behavior for them.
This is happening, largely as a result of an IETF standards effort called DiffServ, which is defining building blocks from which the network behaviour required by a particular application can be provided, even in the presence of network congestion. Most routers today can interpret 8 priority levels (IP Precedence) of which 6 are user accessible - unfortunately they are not set up to do this because it is a configuration nightmare. The good news is that my company and others are going to take out the configuration hassle and make it easy for ISPs to configure their routers for DiffServ (we make network management software to configure this).
Most twitch games will want priority queuing, or at the least some guaranteed access to bandwidth for critical packets. Unfortunately this is exactly what Voice over IP (VoIP) requires, which is probably much more lucrative for the ISPs since it can be charged by the minute... Dedicated gaming networks may well be able to set priorities that make sense for gamers, though.
A "realtime" packet should never be added to a queue if there is another one from the same connection that is already in the queue. Multiple queuings just add latency that can't be controlled at the application level. Jump-to-head-of-queue would sure be nice, but that's probably too much to ask for in the name of "games".
This is like many multimedia applications, e.g. VoIP and videoconferencing, where if a packet is delayed it is no use so it might as well be dropped. One approach to this is for the application to become adaptive (or maybe the IP stack or linmodem code, given some hints from the application), i.e. to not send packets so frequently in times of congestion. At least then the offered load to the network is reduced, which is important in the non-DiffServ case.
The other approach is to use DiffServ to implement a low latency service, in which every suitably marked packet goes right to the top priority queue. Through careful limiting of what traffic gets into the network with that packet marking, the idea is that the top priority queue is almost always empty so low latency packets go right through every router with minimal queuing delay.
It's worth noting that if the queuing is working properly, the 2nd packet should never run into the first packet anyway, so it's probably easier just to throw DiffServ at the problem. This also avoids the need to keep track of things on a per-flow basis - when there are 1000s of real-time sessions going through a larger router, you really don't want to have to keep track of sessions (aka flows in QoS speak).
The Linux host sending the packets could implement the 'dump earlier packet' behaviour in its queuing mechanisms, but it would have to be per-flow, and the same arguments apply - it's better to reduce host, modem and router latency so that the packets go right through and have no chance to bump into earlier packets.
If you're interested, the Linux-DiffServ project [lrcwww.epfl.ch] is based on the 2.2 kernel - they've done a pretty flexible and complete implementation. More general links on QoS including DiffServ can be found here [orchestream.com], under the Links button.
DiffServ technology is coming, and in fact exists in basic form in every Cisco router. The big issue for gaming is getting ISPs to implement DiffServ, and to charge for it in a sensible way so it's accessible to gamers.
Re:A few things were taken out of context... (Score:4)
The comment on the modems is interesting, I guess with more GeForce type cards we'll see a bit of unloading of the CPU which would make softmodems less of a performance hit.
Having them software controllable definately does give some flexability, and I'm sure that some tweaking could be done, perhaps out of band signalling, where the application could flush the buffer, forcing the modem to send a packet now, instead of waiting for more bytes. Being able to control the packet sizes the modems use between themselves would offer some benefit, because you wouldn't have packets being held too long, or being split and thus taking longer.
Is the hardware (what little there is) similar enough that it's feasible to write drivers, or does every winmodem speak a different language? It would be good to have Linux winmodem support, if only to make it easier to build sub-$500 systems to be able to get more people online.
What's with this assumption that Linux has a bette (Score:2)
NT's TCP/IP stack seems to outperform linux farely substantially, because it's threaded, and generally BSD based.
NT keeping up with Linux's TCP/IP performace? Maybe on a 486, but not on a scaled SMP or even normal high end machines.
Linux lacks many of NT's optimization features (like IO completion ports), or evn sufficient use of any threading.
Also, lets not forget Windows 2000 currently holds the networking speed record.
Re:Can Linux be any more pathetic? (Score:3)
Jesus Christ, it seems like about every technology is in a "this is being fixed" or "it's planned for release soon" stage on Linux.
Uh, and this is different from other OSes how? Microsoft is always saying the same sorts of things. Otherwise everyone who was using NT 3.5 would have just stayed there and not upgraded to 4.0, and the people on 4.0 wouldn't be at all interested in 2000. Even the commercial UNIXes have areas they are working on improving. This is something that is industry wide.
Now do you see why people who've been computing for years (read: grown up since the HaX0r days, actually work with modern OSes)
How about someone who is in their mid 30's, and works every day with Solaris, works occasionally with AIX and IRIX, and has the occasional misfortune of having to deal with Windows NT? I've tried the *BSDs as well.
just might think Linux is a piece of shit toy OS?
Actually, I think only people who have some sort of ulterior motive or are incredibly stupid would think that. For most people and most purposes, Linux is pretty good already. Is Linux perfect? No. Is any OS perfect? No. There are areas where Linux is better than the *BSD's and areas where the *BSDs may be better than Linux. There are areas where the commercial *nixes are better than Linux, and areas where Linux is better than the commercial *nixes. In my opinion, Linux is better than NT in just about every way that matters, other people may not agree. Sure, you can come up with wacky configurations and benchmarks to prove whatever you want, but those things matter little in the real world. Let me set up the parameters for the test, and I can prove that a Geo Metro can outperform a Lamborghini Diablo (one answer below).
Have fun waiting around for it.
As slow as Microsoft is, I think I will have less time to wait to get important things with Linux than I would have to if I used NT.
Now for the answer to the Metro vs. Diablo: Give each of them only 5 gallons of gas and a 200 mile course. The Metro wins as the Diablo would have to be pushed more than half way.
So THAT is what Bolo was for? (Score:2)
I remember that game. Whoo boy do I remember it! Too bad it didn't survive the PowerPC transition.
Cheers,
Ben
Reducing Latency (Score:3)
Buffering is the main problem. You have to look at the end-to-end system and find all of the places that data is being buffered. Examine hardware and software, add up all the bits and bytes of buffering.
Packet/frame size should be kept small. If it is too large, break it up into a sequence of smaller packets. Transmit four 128 byte packets instead of one 512 byte packet.
If possible, build the packet as it is being transmitted, a word at a time, rather than building a complete packet and transmitting it.
On the receiving side, you can process the data in a packet as it is being received, you don't have to wait for the last byte in the packet.
Error detection/correction hardware and software can add large amounts of latency.
Watch out for FIFOs in the transmitter and receiver. They can add substantial amounts of latency.
The ideal situation is to take a sample of a parameter and immediately put it on the wire. The receiver then processes the sample as soon as it receives the last bit of the sample.
A slower modem speed may actually reduce latency if it enables you to avoid the use of error correction. A dumb modem is the ideal.
Re:A few things were taken out of context... (Score:2)
And a thousand coders stand and cheer!
This is why so many of us like this guy.
How to do IP over modems right (Score:5)
A better way to do this is to have the modem accept blocks from the networking software and send packet-sized blocks as blocks. Get rid of the serial port emulation entirely; it's an anachronism and a bottleneck. We'd then have an IP modem. No more PPP byte-stuffing and control character translation.
The hardware interface for a Winmodem potentially makes this possible with existing hardware. So it's feasible to do this in software alone. Both ends of the wire have to talk the new protocol, so the ISP end has to be updated too. But many of those systems have a T1 in one side, an Ethernet out the other, and a DSP in the middle, so they're reprogrammable.
Good move. Go for it.
Re:Moderate this up (Score:2)
It is the nature of internet news to be a little flaky now and again. It's the nature of the internet, there is little that you can positively confirm. Also, it's not like
But once again, you don't.
I think you missed the point... (Score:2)
So you are saying, that we are getting often files slower than we could on the same connection for some rason that is in stack? Then we gotto remove that stupid limit NOW. I want my modem work at it's fullest.
Then the Internet collapses as all the major backbones get saturated, and the world as we know it ends. Tragedy of the Commons, my friend.
Thank you very much - extremely relevant! (Score:2)
Cheers,
Ben
Some classic and related rants (Score:2)
(Lots of other good rants as well...)
Cheers,
Ben
Re:Moderate this up (Score:2)
I used to love Slashdot. Now it just doesn't seem the same. Read some of the posts (if you can wade through the ones about Natalie Portman), they're filled with hostility, pride, and more and more ignorance. There's a bitter quality about the site lately, and it's a big turn off to intelligent discussion.
And isn't that what Slashdot is about? Intelligent discussion. For the past few days my posts have been followed by some AC who keeps spouting obsceneties at me. What is that all about? One AC alone isn't enough to get to me, but it's just like a giant neon sign pointing out the general trend in this community.
It used to be (mostly) just a bunch of Linux-users talking about stuff they liked. Now it's... I don't know, but it's changed. It's high-profile, just like Linux itself. More and more people are coming here, and more and more garbage is being posted -- by users and the staff. And I know I'm not the only one who feels this way.
(sigh) oh well...
-----------
"You can't shake the Devil's hand and say you're only kidding."
Re:Moderate this up (Score:2)
Why should I treat Internet news with a lower regard than the news I get off the TV or a newspaper? There's no reason it should be "flakier". If the Slashdot crew cared enough to post actual news, and not just rumors, they would confirm their information before they made a headline out of it. Sure, they admitted that it was possibly incorrect, but that didn't really stop anyone from treating it like gospel. And how hard would it have been to confirm? Couldn't someone have picked up the phone and called Carmack? Ok, so he was in Hawaii, but that's a pretty lousy excuse for posting a story with essentially no legitimate content.
And as for your first comment about my posts. Admittedly, I am a bit of a critical person but don't mistake what I said: I wasn't criticizing John Carmack in any way. This about the crew at /. alone. And I feel free to criticize them, namely because I think that they often do a piss-poor job of sticking to their job. Their motto is "News for Nerds. Stuff that matters." Maybe they should look at it sometime.
Finally, I fully expect that someone will respond to this by saying "If you don't like it, then leave" -- they always do. Somehow, there are a number of people who always seem to think that using this forum is a black/white issue: We either have to love it or leave. Well my only comment to those people is "Blow it out your ass." That was my eloquent statement for the night.
-----------
"You can't shake the Devil's hand and say you're only kidding."
Re:A few things were taken out of context... (Score:2)
I have a webpage [visi.com] with some of my thoughts on this.
At one time, Multitech made modems that understood the UUCP G protocol, which is an XMODEM like response for every packet protocol. Multitechs would recognize when a UUCP conversation was happening, and start sending response packets back to the server, knowing that the modem's error correction would ensure the packet would be delivered reliably. This improved things dramatically because you only had the latency induced by your serial link, and not the communication over the phone lines, and the other computer's serial link.
Something similar could be done with PPP. In fact, I bet the PPP headers could even be stripped off by one modem and added back in by the other. Also, once USB modems start happening, the latency induced by serial communications will be cut dramatically.
My only worry is that a lot of modem software expects to talk to serial hardware, and so USB modems won't be as popular as they should be for this reason.
Anyway, read my webpage for more details, as well as some ideas for improving IP routing of time sensitive data without using QoS.
Re:A few things were taken out of context... (Score:2)
Could you be a little more precise here? The modem doesn't really have an idea about packets -- all that is at the PPP, IP and TCP/UDP levels. This problem barely affects games at all -- for stream sockets, however, you have such an option. At least you have one the _other_ way (TCP_CORK on a stream socket in Linux 2.2.x), where a non-full packet is _never_ sent.
I've heard most Winmodems speak different languages. Perhaps we need some IEEEEEEEEEEE standard for software modems?
/* Steinar */
Re:A few things were taken out of context... (Score:2)
I mean, a way for a program to tell the modem to send *now*. It's possible to do this at the protocol level, but you never really know what the modem hardware will do. It'll just sit there until it feels good and ready to send. And it's usually right, it can cram more data in. But if you care about latency more than throughput...
Re:Software Modem (Score:2)
Re:TCP is too slow... Hello? McFly? (Score:2)
This is wrong. You get them as fast as your connection can handle it. At least after the current max speed is found, a new TCP connection starts slow and then speeds up (sending one Megabyte in a second to a client that can handle only one Kilobyte per second will mean 1023kB of bandwidth wasted only to find out that it is in fact wasted). This is one of the reasons HTTP/1.1 adds persistant connections as opposed to one connection per file as in HTTP/1.0.
More things to consider (Score:2)
It's fantasic that strides are being taken by the gaming development community to pound on the networking vendors to give gamers (specifically modem users) a "better" Internet. A lot of work has been done in this area in the past couple of years in light of VoIP, but I always envisioned the Diff-Serv/Int-Serv as a perfect app for online gaming =)
However, its not the vendors that are the problem, in this case. Most every router manufacturer has a current "QoS game" that is some adoption of the Diff-Serv approach. The problem lies with the network providers themselves. Rather, its the inter-peer handoff of packets from one provider to another that is the actual core of the problem at hand, as diff-serv "mappings" from one domain to another (ie. passing packets from SprintLink to UUNET) can vary greatly, not to mention the cost negotiations and political games at hand. It can be either really simple or really complex, and if it was simple, we'd probably already see something working =)
To compound this problem, the largely assymetrical nature of the Internet would require these mappings to be identical on both the forward AND reverse path. Its easy to map a packet from inside a network all the way to a peering point. Its a bit more difficult to appropriately map the return path FROM the peer point back to the internal recipient.
I'm not saying that its impossible. The packet marking approach would work great for a completely on-net connection (i.e., where a packet stays entirely within a single backbone provider). Its just right now the inter-domain marking support simply doesn't exist in production form on today's networks.
I think the current focus in high performance networking still relies on the characteristics of the peer-peer or client-server protocol that is used to exchange data. Much can be done at the application layer to realize current bandwidth availability and network characteristics (i.e. loss, delay variation, etc.).
Using a simplistic approach of having the user specify their local link rate is right on the mark in a LAN environment. It makes the protocol simple with few corner cases.
However, with on-line play, relying on the local link rate/capacity of the player's Internet connection does not take into account the many link variations that lie between the player and the server they connect to. It is this transitory network where the majority of bottlenecks occur. It is also where a smarter protocol that utilizes congestion avoidance, adaptive rate control and proper error recovery (at a network level, not a user-level data recovery standpoint) can GREATLY enhance overall user "goodput" (goodput = the users' perceived network throughput). Of course, this creates a more more complex protocol for more cases to consider, but this is actually what we want.
Take for instance TCP. By utilizing the SACK option on TCP connections, a user can achieve 20-50% increased goodput over a convential TCP connection on the same "noisy" Internet link. Of course gaming doesnt use TCP, but it *is* an example of a case where people said "stick a fork in it, its done" about a network protocol (TCP).
Again, I think it is extremely beneficial in examining everying to improve the quality of on-line play (especially for high latency, low bandwidth users). But I still believe that the most gains can be made in putting together a network savvy protocol that is aware and proactive on the connection with its peer. It's not necessarily the delay, but the delay variation thats the most aggrevating for online gameplay.
-Jason Keimig
Information and Telecommunication Technology Center
University of Kansas
Re:A few things were taken out of context... (Score:2)
If the application had a way of telling the modem 'ok, screw bandwidth, just get those last 150 bytes there ASAP' there's probably be a significant latency decrease.
The problem is that the modem can't tell a download (where bandwidth is critical) from a game, where latency is critical.
Nics.. (Score:2)
I'm not the original poster, but here is a test on my local lan:
PING omni (192.168.20.17): 56 data bytes
64 bytes from 192.168.20.17: icmp_seq=0 ttl=255 time=2.4 ms
64 bytes from 192.168.20.17: icmp_seq=1 ttl=255 time=0.9 ms
64 bytes from 192.168.20.17: icmp_seq=2 ttl=255 time=0.9 ms
64 bytes from 192.168.20.17: icmp_seq=3 ttl=255 time=0.9 ms
64 bytes from 192.168.20.17: icmp_seq=4 ttl=255 time=0.8 ms
This is a 10-BT network.
My local machine has an old WD NIC (not quite sure which one - it's ISA though, and utilizs the 8390 chipset) and the other machine has an SMC Ultra 16 (ISA and 8390 again.)
I tried it from another machine, this one with a PCI Intel EtherXpress 10/100 (operating at 10Mbps)
PING 192.168.20.17 (192.168.20.17): 56 data bytes
64 bytes from 192.168.20.17: icmp_seq=0 ttl=255 time=1.3 ms
64 bytes from 192.168.20.17: icmp_seq=1 ttl=255 time=0.6 ms
64 bytes from 192.168.20.17: icmp_seq=2 ttl=255 time=0.6 ms
64 bytes from 192.168.20.17: icmp_seq=3 ttl=255 time=0.6 ms
64 bytes from 192.168.20.17: icmp_seq=4 ttl=255 time=0.6 ms
So I'd guess that
Hope this helps you when you buy your next NIC.
Re:It's not the stack (Score:2)