Points for jlevie
jlevie: As previously explained, I'm giving up on this crappy site which is quite often down or experiencing errors, and when it is up it is full of folks posting questions who are too clueless to read your comments or follow the directions contained therein (even though in their profiles they all claim to be analysts, system admins, or network engineers).
everyone else - don't bother replying. I will only accept an answer, _any_ answer originating from jlevie, and no one else. Why jlevie - he seems to be one of the very few folks here with a clue, and he has the patience of 20 saints (which obviously I do not).
So, jlevie - I await your response (I frankly don't care if it only consists of a single character).
-Jon
everyone else - don't bother replying. I will only accept an answer, _any_ answer originating from jlevie, and no one else. Why jlevie - he seems to be one of the very few folks here with a clue, and he has the patience of 20 saints (which obviously I do not).
So, jlevie - I await your response (I frankly don't care if it only consists of a single character).
-Jon
I'd really like to see if I can get you to re-consider your decision to stop participating in EE. I'd agree that you've caught the "short end of the stick" on a few of the questions, but I can tell from the comments that you've offered that you are very knowledgeable w/respect to Linux. And no, I don't know why some of the questions went to me rather than to you. That same thing has happened to me on occasion and it can be rather irritating, but that's just the way it happens sometimes. And yes the current instability of the site is very frustrating. It has been a lot more better in the past and I'm sure it will be better later.
I'll agree that it takes a lot of patience sometimes, especially when the questioner has little knowledge at that point in their experience with computers. But if you think back on your early experiences with computers and operating systems I think you'll realize that all of us were like that to at least some degree once. Personally, I got into computers before USENET really even existed and I remember vividly how difficult it was to find solutions to even the simplest of problems. There simply wasn't anyone around anywhere to ask questions of. So I don't mind sharing the knowedge that I've gained and am willing to patiently work with someone else in an attempt to help solve their problem and educate them at the same time.
So please reconsider, I feel you'd be doing yourself a disservice by packing it in at this point and the users of EE would be losing access to a very knowledgeable expert. If you'd like to continue this discussion, I'd be glad to exchange email with you. Send me an email (jlevie@bellsouth.net) and we can talk about it.
I'll agree that it takes a lot of patience sometimes, especially when the questioner has little knowledge at that point in their experience with computers. But if you think back on your early experiences with computers and operating systems I think you'll realize that all of us were like that to at least some degree once. Personally, I got into computers before USENET really even existed and I remember vividly how difficult it was to find solutions to even the simplest of problems. There simply wasn't anyone around anywhere to ask questions of. So I don't mind sharing the knowedge that I've gained and am willing to patiently work with someone else in an attempt to help solve their problem and educate them at the same time.
So please reconsider, I feel you'd be doing yourself a disservice by packing it in at this point and the users of EE would be losing access to a very knowledgeable expert. If you'd like to continue this discussion, I'd be glad to exchange email with you. Send me an email (jlevie@bellsouth.net) and we can talk about it.
Jon, I agree with jlevie. The answers I've seen you give out to questions posed have been illuminating and detailed, and I'm pretty sure that there are plenty of people reading them and benefitting, not just whoever posted the question. I'm sure that sometimes it's frustrating to see your good advice ignored, but it nonetheless remains good advice, and stays accessible to everyone else who uses this site.
Vijay
Vijay
ASKER
OK - I'm willing to continue on a trial basis
Let's start by turning this into a real question, rather than a points giveaway - here it goes:
I need a way to unify the behaviour of ipchains-based masquerading and the portion of the 'ip' command (part of iproute2) that handles policy routing ('ip rule')
History:
I have a firewall that port-forwards ftp and web connections to internal servers (RFC1918 space), and uses external IPs from an ISDN provider for public access to these services. I need all other internal connections (actually, if we figure out how to do it from all internal machines, I am more than clueful enough to adjust the masks, etc to exclude the internal servers) to use the cable modem as their default outbound path. I have a general rule in ipchains that masq's any internal host.
I've tried using fwmark to mark the packets from internal machines (using ipchains), and then attempted adding a seperate routing table that has the cable modem as the default route (rather than the ISDN), and setting up an
'ip rule' that matches the fwmark index and speficies the alternate routing table (the one that uses the cable modem as the default). Packets still go out the ISDN exclusively. I've also tried adding specific 'ip rule' entries that match the src IP of internal machines - that just winds up breaking their connectivity (apparently, 'ip rule' bypasses ipchains, or at least that the symptom I seem to see).
Any clues?
Suggestions I will not accept:
Changing the DNS for the external IPs of the servers to be the cable provider IP - not only do we need multiple IPs (for SSL sites), but I would never trust a cable network in a production system (well, not yet anyway).
Anything else changing the basic config - if you don't know how to get some masq'd machines to use one default route, and other masq'd machines to use a different default route (and please don't say something stupid like 'set the default route on the different internal machines to different IPs' - that would indicate you don't understand my question), then don't even bother replying.
Solutions in which I am interested (but probably won't give points unless step-by-step commands are given, since I am currently exploring these options anyway):
Using the NAT functionality of iproute to do the same thing as ipchains -j MASQ
A cisco solution (our ISDN router is cisco - 2600 series - overkill, I know, but the T1 is coming...).
Adding '-y' to some ipchains rulesets - I generally don't use this option much, but it seems to show some promise.
I don't feel to bad about leaving the title of this as "points for jlevie" - I imagine it will take someone with at least his amount of points to solve this one (feel free to prove me wrong).
-Jon
P.S. If you need a network diagram, I'll send you one by email. I figure not posting one here off-the-bat cuts down on folks who really shouldn't try to answer this question (since the layout is rather quite simple), and my ascii art sucks anyway.
Let's start by turning this into a real question, rather than a points giveaway - here it goes:
I need a way to unify the behaviour of ipchains-based masquerading and the portion of the 'ip' command (part of iproute2) that handles policy routing ('ip rule')
History:
I have a firewall that port-forwards ftp and web connections to internal servers (RFC1918 space), and uses external IPs from an ISDN provider for public access to these services. I need all other internal connections (actually, if we figure out how to do it from all internal machines, I am more than clueful enough to adjust the masks, etc to exclude the internal servers) to use the cable modem as their default outbound path. I have a general rule in ipchains that masq's any internal host.
I've tried using fwmark to mark the packets from internal machines (using ipchains), and then attempted adding a seperate routing table that has the cable modem as the default route (rather than the ISDN), and setting up an
'ip rule' that matches the fwmark index and speficies the alternate routing table (the one that uses the cable modem as the default). Packets still go out the ISDN exclusively. I've also tried adding specific 'ip rule' entries that match the src IP of internal machines - that just winds up breaking their connectivity (apparently, 'ip rule' bypasses ipchains, or at least that the symptom I seem to see).
Any clues?
Suggestions I will not accept:
Changing the DNS for the external IPs of the servers to be the cable provider IP - not only do we need multiple IPs (for SSL sites), but I would never trust a cable network in a production system (well, not yet anyway).
Anything else changing the basic config - if you don't know how to get some masq'd machines to use one default route, and other masq'd machines to use a different default route (and please don't say something stupid like 'set the default route on the different internal machines to different IPs' - that would indicate you don't understand my question), then don't even bother replying.
Solutions in which I am interested (but probably won't give points unless step-by-step commands are given, since I am currently exploring these options anyway):
Using the NAT functionality of iproute to do the same thing as ipchains -j MASQ
A cisco solution (our ISDN router is cisco - 2600 series - overkill, I know, but the T1 is coming...).
Adding '-y' to some ipchains rulesets - I generally don't use this option much, but it seems to show some promise.
I don't feel to bad about leaving the title of this as "points for jlevie" - I imagine it will take someone with at least his amount of points to solve this one (feel free to prove me wrong).
-Jon
P.S. If you need a network diagram, I'll send you one by email. I figure not posting one here off-the-bat cuts down on folks who really shouldn't try to answer this question (since the layout is rather quite simple), and my ascii art sucks anyway.
ASKER
BTW, all of you are competing with redhat support (with which I have been wholly unimpressed) to answer this one - if they answer correctly before I get an answer here, I will delete this question (not that I think it's too likely - the most usefeul thing I ever got out of redhat support was how to adjust the kernel version string to allow multiple module versions to exist after compiling new kernels that have the same src tree as existing kernels). Yes, we (or at least my client does) have a support contrsct with Redhat.
-Jon
-Jon
ASKER
BTW, all of you are competing with redhat support (with which I have been wholly unimpressed) to answer this one - if they answer correctly before I get an answer here, I will delete this question (not that I think it's too likely - the most usefeul thing I ever got out of redhat support was how to adjust the kernel version string to allow multiple module versions to exist after compiling new kernels that have the same src tree as existing kernels). Yes, we (or at least my client does) have a support contrsct with Redhat.
-Jon
-Jon
ASKER
Sorry about the duplicate post - EE decided to get wacky as soon as I attempted my initial post.
-Jon
-Jon
I think I know how to do this and I'll try setting it up later today. Further bulletins as conditions warrant...
ASKER
OK - Redhat enterprise support is stumped (frankly, I'm not surprised). I'm still not really sure they understand what I am doing. Things I thought about WRT ipchains (but didn't bother to try) were using the -y option, adding another internal network (and card) and using the -i option, or using the --source-port option to catch packets originating from server daemons. I rejected these ideas because they should all do the same thing as -s (which is match packets from either the internal servers, or everyone else), and that is clearly not where the error lies (verified by the -l option).
jlevie: Do you know if the NAT fucntionality in iproute is pure NAT, or NAT/PAT (I need the latter, at least for the internal workstations). Actually, since it appears that iproute rules are processed before ipchains (since adding a policy rule for internal IPs breaks their connectivity, presumably because they didn't end up getting masq'd), pure NAT would probably work fine since I can use source matches to be sure only internal servers use the rule.
Here is my ugly cisco solution:
Do the NAT/PAT on the cisco for either the servers or the workstations, and let the firewall handle the masq for the other set.
I'm not a big fan of this for several reasons, amongst them: Makes me have to re-work our whole VPN routing/filtering scheme (requiring a whole new security eval as well), I really don't like using ciscos as firewalls, and dammit - I know linux can do this (just not quite how).
If someone can point me to a nice diagram of how the various networking subsystems flow into (or avoid) one another in the linux kernel, I might just give that person all the points, since this is really all the info I need to solve this. This is really one of the biggest pain-in-the-asses of linux - failure to unify the various ip config tools we all know and love (don't even get me started talking about the pains of ipmasqadm port-forwards for hosts on the same network as the target), and failure to at least document the various entry and exit points for packet handling routines (used by ipmasqadm, ipchains, iproute, etc). Actually, I suspect this imaginary diagram would probably resemble a Chicago highway map (given the problems getting packets to flow in a logical fashion), so maybe it wouldn't be so helpful after all.
iproute NAT seems to be emerging as the best potential solution, in my own research (obviously). Still need to verify some capabilities/limitations.
-Jon
jlevie: Do you know if the NAT fucntionality in iproute is pure NAT, or NAT/PAT (I need the latter, at least for the internal workstations). Actually, since it appears that iproute rules are processed before ipchains (since adding a policy rule for internal IPs breaks their connectivity, presumably because they didn't end up getting masq'd), pure NAT would probably work fine since I can use source matches to be sure only internal servers use the rule.
Here is my ugly cisco solution:
Do the NAT/PAT on the cisco for either the servers or the workstations, and let the firewall handle the masq for the other set.
I'm not a big fan of this for several reasons, amongst them: Makes me have to re-work our whole VPN routing/filtering scheme (requiring a whole new security eval as well), I really don't like using ciscos as firewalls, and dammit - I know linux can do this (just not quite how).
If someone can point me to a nice diagram of how the various networking subsystems flow into (or avoid) one another in the linux kernel, I might just give that person all the points, since this is really all the info I need to solve this. This is really one of the biggest pain-in-the-asses of linux - failure to unify the various ip config tools we all know and love (don't even get me started talking about the pains of ipmasqadm port-forwards for hosts on the same network as the target), and failure to at least document the various entry and exit points for packet handling routines (used by ipmasqadm, ipchains, iproute, etc). Actually, I suspect this imaginary diagram would probably resemble a Chicago highway map (given the problems getting packets to flow in a logical fashion), so maybe it wouldn't be so helpful after all.
iproute NAT seems to be emerging as the best potential solution, in my own research (obviously). Still need to verify some capabilities/limitations.
-Jon
ASKER
Almost forgot the "fantasy" solution - if someone wants to write (or point me to some existing) kernel patches for 2.2.15 and patchs for ifconfig, ipchains, etc that allows multiple, seperate IP subsystems to run simultaneously on the same box (all stacks must have access to the same localhost [lo], I imagine), and allows you to specificy to which stack a hardware interface belongs, that would work great (although I'm not even sure if it is possible - vmware seems to pull this off, although not within the same OS - hence, they avoid patching all the ip config tools). If you can present this solution (yeah right), I will give all the points as well as all the other points I've gotten since posting this question.
-Jon
-Jon
Please allow me to make sure that I understand the problem.
You have two Internet access paths, one via ISDN and one via Cable Modem. The ISDN path is used for inbound services and has a netblock associated with it. The Cable modem is used solely for Internet access by all other internal machines. The Linux box needs to act as the firewall for both outside gateways and its inside interface needs to be the default gateway for all inside machines, those which provide external services and ordinary machines.
Is that a fair summary of the problem?
You have two Internet access paths, one via ISDN and one via Cable Modem. The ISDN path is used for inbound services and has a netblock associated with it. The Cable modem is used solely for Internet access by all other internal machines. The Linux box needs to act as the firewall for both outside gateways and its inside interface needs to be the default gateway for all inside machines, those which provide external services and ordinary machines.
Is that a fair summary of the problem?
ASKER
A good summary, but I would add that I need to be able to select which internal machines use which external gateway.
Upgrading to 2.4 and using netfilter/iptables has been in the back of my mind as an eventual possible solution, but I'd like to exhaust the possibilities of 2.2 first...
-Jon
Upgrading to 2.4 and using netfilter/iptables has been in the back of my mind as an eventual possible solution, but I'd like to exhaust the possibilities of 2.2 first...
-Jon
Being able to select which internal machines use which gateway is of course a requirement. I didn't state it as such, as I assumed it to be implicitly required by the fact that some machines were to use one gateway and the balance the other.
I don't see an obvious viable solution using ipchains and iproute. What's needed, to my way of thinking, is the ability to use both NAT (with static translations) and NPAT at the same time. Also iproute operates on either or both sides of ipchains, so to speak, rather than on the path the data streams take within ipchains. My reading of the iproute documentation suggests that one might be able to use it to do NAT, but I've yet to see an operating example of it doing so. Also it's not clear to me what unexpected side affects might occur if both iproute and ipchains are doing NAT/NPAT translations at the sime time to a single inside interface.
On the other hand, I've had no problems setting up an iptables/Netfilter environment that uses static NAT translations for some inside hosts and also provides NPAT translations for all other hosts (much like one might do with a Cisco PIX or IPFilter on *BSD). While I've not yet set up a combination like this that has two outside gateways, it looks to me like that shouldn't be a problem. The only question in my mind is whether it would be necessary/possible to use source policy routing to ensure that packets from the outside IP of the statically translated nodes would be sent out the correct interface. And I think I'd have to configure a gateway with two outside interfaces to find out. I've got one place where there are two Internet gateways, so it is mostly a matter of finding some time to set up a test case to find out if this will work.
It looks to me like an iptables/netfilter configuration should be the easiest & cleanest solution, providing that routing to the correct outside interface is possible and can be done.
I don't see an obvious viable solution using ipchains and iproute. What's needed, to my way of thinking, is the ability to use both NAT (with static translations) and NPAT at the same time. Also iproute operates on either or both sides of ipchains, so to speak, rather than on the path the data streams take within ipchains. My reading of the iproute documentation suggests that one might be able to use it to do NAT, but I've yet to see an operating example of it doing so. Also it's not clear to me what unexpected side affects might occur if both iproute and ipchains are doing NAT/NPAT translations at the sime time to a single inside interface.
On the other hand, I've had no problems setting up an iptables/Netfilter environment that uses static NAT translations for some inside hosts and also provides NPAT translations for all other hosts (much like one might do with a Cisco PIX or IPFilter on *BSD). While I've not yet set up a combination like this that has two outside gateways, it looks to me like that shouldn't be a problem. The only question in my mind is whether it would be necessary/possible to use source policy routing to ensure that packets from the outside IP of the statically translated nodes would be sent out the correct interface. And I think I'd have to configure a gateway with two outside interfaces to find out. I've got one place where there are two Internet gateways, so it is mostly a matter of finding some time to set up a test case to find out if this will work.
It looks to me like an iptables/netfilter configuration should be the easiest & cleanest solution, providing that routing to the correct outside interface is possible and can be done.
ASKER
While I'm at it (and this would go a great deal toward solving this one), does anyone know if it is possible to log the matches of 'ip rule' and 'ip route' entries (ala ipchains '-l' option)?
One of my biggest problems with this thing is lack of logs - tcpdump can tell me what's going in and what's going out, but I'd like to get a better idea of how the packets flow _inside_ the box
I've got this working a bit by using a horrid series of commands to get the connections to the internal servers to acutally translate on the outbound to the cable modem IP (hopefully this doesn't piss off _too_ many clients), but this only works for internal servers - any service on the firewall (i.e. VPN) then wants to jump out the cable modem on the outbound _without_ translation, and of course that doesn't work, since the cable provider filters non-cable network IPs. Also, I'd really like the servers and the workstations to refrain from competing for the same outbound bandwidth. What fun, eh?
Also, it appears most texts on linux NAT are just wrong, or they never bothered to try it themselves. Many texts say to use lines like:
ifconfig eth0:0 1.2.3.4 netmask 255.255.255.0
ip route add nat 1.2.3.4 via 192.168.1.2 table local
but since 1.2.3.4 is already an IP on the firewall with a prior route entry for the interface itself, the command returns with 'RTNETLINK answers: File exists' which is really not surprising....
ARRRGGGH - where are the good docs on this stuff?
-Jon
P.S. I warned you (at least jlevie) I didn't ask easy questions... ;-)
One of my biggest problems with this thing is lack of logs - tcpdump can tell me what's going in and what's going out, but I'd like to get a better idea of how the packets flow _inside_ the box
I've got this working a bit by using a horrid series of commands to get the connections to the internal servers to acutally translate on the outbound to the cable modem IP (hopefully this doesn't piss off _too_ many clients), but this only works for internal servers - any service on the firewall (i.e. VPN) then wants to jump out the cable modem on the outbound _without_ translation, and of course that doesn't work, since the cable provider filters non-cable network IPs. Also, I'd really like the servers and the workstations to refrain from competing for the same outbound bandwidth. What fun, eh?
Also, it appears most texts on linux NAT are just wrong, or they never bothered to try it themselves. Many texts say to use lines like:
ifconfig eth0:0 1.2.3.4 netmask 255.255.255.0
ip route add nat 1.2.3.4 via 192.168.1.2 table local
but since 1.2.3.4 is already an IP on the firewall with a prior route entry for the interface itself, the command returns with 'RTNETLINK answers: File exists' which is really not surprising....
ARRRGGGH - where are the good docs on this stuff?
-Jon
P.S. I warned you (at least jlevie) I didn't ask easy questions... ;-)
ASKER
Sorry, jlevie - my EE page didn't refresh to see your post until after I just posted the above...
I know what you mean about everyone saying NAT should work in 2.2 but failing to see actual examples...
Well, if another day or so passes and I still can't get it with 2.2, I probably will just go to 2.4 - and of course, the points would be yours (as if there was ever any doubt, hehe)
Any caveats of which I should be aware taking a 6.2 (kernel 2.2.15) machine up to kernel 2.4? Any major binary/library upgrades to go along with it?
-Jon
P.S. Am I the only one who is annoyed that the RedHat 7.0 rescue CD never seems to create device nodes for storage hardware (actually, I know for a fact that others have seen this too)? You can do it manually, but what a pain in the ass.
I know what you mean about everyone saying NAT should work in 2.2 but failing to see actual examples...
Well, if another day or so passes and I still can't get it with 2.2, I probably will just go to 2.4 - and of course, the points would be yours (as if there was ever any doubt, hehe)
Any caveats of which I should be aware taking a 6.2 (kernel 2.2.15) machine up to kernel 2.4? Any major binary/library upgrades to go along with it?
-Jon
P.S. Am I the only one who is annoyed that the RedHat 7.0 rescue CD never seems to create device nodes for storage hardware (actually, I know for a fact that others have seen this too)? You can do it manually, but what a pain in the ass.
Combining the last two comments... No, I don't know of a way to get routing decisions logged. Once I started looking at the sources to see if it would be a possibility, but something else more urgent came up and I've never gone back to that.
Yeah, a bunch of the docs fall into the class that I call ORFL (Out Right F*** Lies). And the one thing that the writers could have done and didn't is to present a few working examples (perhaps three or so ranging from simple to complex). To a great degree the Netfilter docs also fall into this category. There are some examples on the net, but I haven't yet seen one that does NAT (all use IPMasquerade, aka NPAT). 'Nother gripe is that nowhere in the docs for IPRroute or Netfilter is any mention of the need to use an ip alias or published static arp for IP's other than the one assigned to the outside interface if you are doing NAT. Of course I knew that would be necessary, but the average reader probably won't. Well, not that arp would be of much help, at least on RH 7,0 & 7.1, as it is completely broken as far as being able to publish an arp entry.
I'll confess that I never upgrade a Linux or Solaris system any more. I always do a complete install because an upgrade typically isn't repeatable and I've also been bitten by things that didn't get properly upgraded and odd thing that didn't get removed. At least with a clean install I know exactly what I'm dealing with and any problems that arise are due to bugs in the new version or changes in applications/daemons. I know that people do upgrade RedHat and apparently successfully. Hopefully, the lack of "known gottcha's" about upgrades implies that they work.
And no you aren't the only one annoyed by that failing of the rescue disk. I can't right now 'cause he's gone to New Orleans for the Jazz Festival, but I have a friend that works for RedHat who might be able to shed some light on the issue. Have you ever filed a bug report on that?
Yeah, a bunch of the docs fall into the class that I call ORFL (Out Right F*** Lies). And the one thing that the writers could have done and didn't is to present a few working examples (perhaps three or so ranging from simple to complex). To a great degree the Netfilter docs also fall into this category. There are some examples on the net, but I haven't yet seen one that does NAT (all use IPMasquerade, aka NPAT). 'Nother gripe is that nowhere in the docs for IPRroute or Netfilter is any mention of the need to use an ip alias or published static arp for IP's other than the one assigned to the outside interface if you are doing NAT. Of course I knew that would be necessary, but the average reader probably won't. Well, not that arp would be of much help, at least on RH 7,0 & 7.1, as it is completely broken as far as being able to publish an arp entry.
I'll confess that I never upgrade a Linux or Solaris system any more. I always do a complete install because an upgrade typically isn't repeatable and I've also been bitten by things that didn't get properly upgraded and odd thing that didn't get removed. At least with a clean install I know exactly what I'm dealing with and any problems that arise are due to bugs in the new version or changes in applications/daemons. I know that people do upgrade RedHat and apparently successfully. Hopefully, the lack of "known gottcha's" about upgrades implies that they work.
And no you aren't the only one annoyed by that failing of the rescue disk. I can't right now 'cause he's gone to New Orleans for the Jazz Festival, but I have a friend that works for RedHat who might be able to shed some light on the issue. Have you ever filed a bug report on that?
ASKER
OK - first the good news - it works.
I've got a set of 'ip route', 'ip rule', ipchains, and ipmasqadm commands that seem to do the trick. Traffic from servers goes out the ISDN, and traffic from everyone else goes out the cable (If someone wants to see these, I'll try to throw them all together in a comprehensible fashion).
Now, here's the weird bit - Traffic from the internal workstations is slow. Identical downloads (same file, same remote server) of a 1M test file yield 250K/s when performed _on_ the firewall, but workstations using the same cable connection (verified by physically unplugging the ISDN line) only seem to get 10-12K (no news yet as to whether or not they can get multiple sumiltaneous downloads at this speed, or if they cut into one another) - ouch.
Any clues as to this new development?
I'll still give all points if anyone has an idea that fixes this speed problem.
I'm already thinking about running user-space proxies to see if that speeds things up (avoiding ipchains and all my carefully-planned policy routing), but I'd like it to work as-is.
-Jon
I've got a set of 'ip route', 'ip rule', ipchains, and ipmasqadm commands that seem to do the trick. Traffic from servers goes out the ISDN, and traffic from everyone else goes out the cable (If someone wants to see these, I'll try to throw them all together in a comprehensible fashion).
Now, here's the weird bit - Traffic from the internal workstations is slow. Identical downloads (same file, same remote server) of a 1M test file yield 250K/s when performed _on_ the firewall, but workstations using the same cable connection (verified by physically unplugging the ISDN line) only seem to get 10-12K (no news yet as to whether or not they can get multiple sumiltaneous downloads at this speed, or if they cut into one another) - ouch.
Any clues as to this new development?
I'll still give all points if anyone has an idea that fixes this speed problem.
I'm already thinking about running user-space proxies to see if that speeds things up (avoiding ipchains and all my carefully-planned policy routing), but I'd like it to work as-is.
-Jon
ASKER
Am I the only one who got an email saying there was a new post here half an hour ago? I guess I'm asking if the failure of the new post to appear is on my end (caching, etc - which I've tried flushing), or on EE (my bet is on the latter - not the first time I've seen this happen).
-Jon
-Jon
ASKER
OK - first, the latest from the testing front - simultaneous downloads max out at about 25k aggregate before they start cutting into each other. Running a socks5 proxy turns up the individual session rate to about 17k, but still seems to max out at 25k aggregate (mind you, all these tests are run by my client - they seem to be reliable in this capacity, however). Now for the whining...
Looks like I messed up the time conversion (who could blame me - I got an email saying someone had posted something one and a half hours _after_ I posted my original "where's the post?" message?). Why is EE sending me email about posts that might occur in the future (but apparently didn't ;-)?
Makes me wonder if the _real_ replies to my question are even getting posted...
To anyone who is having problems posting here - you can discuss this with me via email at jon@unixgroup.com - I will be fair and honest and award you the points here even if you send me comments/answers that eventually solve this via email.
-Jon
P.S. No, I'm not too concerned with spam [by posting my email address], and in fact like to be sure my spam filters are doing their job. Note: this is NOT an invitation to spam me - if you do manage to get spam through my filters, I will still ensure you lose your account with your ISP (and I _do_ regularly submit hosts to the MAPS RSS/RBL [which my filters employ], in case you run your on SMTP server and feel yourself immune).
Looks like I messed up the time conversion (who could blame me - I got an email saying someone had posted something one and a half hours _after_ I posted my original "where's the post?" message?). Why is EE sending me email about posts that might occur in the future (but apparently didn't ;-)?
Makes me wonder if the _real_ replies to my question are even getting posted...
To anyone who is having problems posting here - you can discuss this with me via email at jon@unixgroup.com - I will be fair and honest and award you the points here even if you send me comments/answers that eventually solve this via email.
-Jon
P.S. No, I'm not too concerned with spam [by posting my email address], and in fact like to be sure my spam filters are doing their job. Note: this is NOT an invitation to spam me - if you do manage to get spam through my filters, I will still ensure you lose your account with your ISP (and I _do_ regularly submit hosts to the MAPS RSS/RBL [which my filters employ], in case you run your on SMTP server and feel yourself immune).
ASKER
In case anyone is wondering, the firewall box in this example is a PIII 850, with 256M of RAM (and a large swap space on an AIC72xx, as well)... A simple ipchains masq connection on my home firewall (a P100 w/ 24M or RAM) sees around 200K/s max throughput on the identaical download - from a workstation perspective ('workstation' being a host not within the server subset mentioned in previous posts), a basic ipchains masquerading config (sans policy routing, ipmasqadm, 'ip route', 'ip rule' etc) through the example firewall gets around 200K/s... I will manually isolate the command causing the slowness and post the results here...
-Jon
-Jon
ASKER
Succcess!
Sorry to burst any bubbles, but the speed problem is resolved... I found out the client was running all nets (internal and external) on the same switched segment. I had them seperate the internal and external nets/segments using their collection of switches/hubs, and the speed problem disappeared...
I will post the commands necesary for this config soon (since I imagine that this scenario is not completely unique...)
-Jon
P.S. jlevie - your assistance has not gone unappreciated - After posting this, I will accept a previous answer and award the points to you (I will include necessary commands for this config in my final comment)...
Sorry to burst any bubbles, but the speed problem is resolved... I found out the client was running all nets (internal and external) on the same switched segment. I had them seperate the internal and external nets/segments using their collection of switches/hubs, and the speed problem disappeared...
I will post the commands necesary for this config soon (since I imagine that this scenario is not completely unique...)
-Jon
P.S. jlevie - your assistance has not gone unappreciated - After posting this, I will accept a previous answer and award the points to you (I will include necessary commands for this config in my final comment)...
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
I must apologize for the delays myself - I've been ironing out the kinks (i.e. traceroute doesn't seem to work through the ISDN when done directly off the firewall, but works on internal servers just fine - traceroute must only ever check the 'main' routing table, I guess, or is somehow circumventing 'ip rule').
Anyhow, expect a bit more delay - I came down with a sinus infection yesterday - not feeling up to doing much at present...
Expect a dearth of information when I'm feeling better.
-Jon
Anyhow, expect a bit more delay - I came down with a sinus infection yesterday - not feeling up to doing much at present...
Expect a dearth of information when I'm feeling better.
-Jon
ASKER
OK, here is the script that is run on the firewall at boot-time to make this all work. Prior to running these commands, the firewall already has a default route throught the cable connection, and ipmasqadm port forwards for the internal servers, as well as a general ipchains rule to masquerade _all_ internal machines (workstations and servers). I munged my external addresses to be in 20.30... BTW, did I mention the internal servers actually use two ISDN lines (load-balanced) for external connectivity? No, I did not, because I didn't want to hear silly comments about how that was a source of problems (which it was not). Just thought I'd mention it, since the following might get confusing if you didn't know that...
ip route add table server nat 20.30.55.249 via 10.1.1.2 scope host
ip route add table server nat 20.30.55.250 via 10.1.1.4 scope host
ip route add table server 20.30.55.251 dev eth0 scope link
ip route add table server nat 20.30.23.2 via 10.1.1.6 scope host
ip route add table server nat 20.30.55.253 via 10.1.1.3 scope host
ip route add table server 10.1.1.1 dev eth2 scope link
ip route add table server 20.30.55.248/29 dev eth0 proto kernel scope link src 20.30.55.251
ip route add table server 20.30.23.0/29 dev eth1 scope link
ip route add table server 10.1.1.0/24 dev eth2 proto kernel scope link src 10.1.1.1
ip route add default table server eql nexthop via 20.30.55.254 dev eth0 nexthop via 20.30.23.6 dev eth1
ip route add table server 127.0.0.0/8 dev lo scope link
ip rule add from 10.1.1.2 table server nat 20.30.55.249
ip rule add from 10.1.1.3 table server nat 20.30.55.253
ip rule add from 10.1.1.4 table server nat 20.30.55.250
ip rule add from 20.30.55.249 table server
ip rule add from 20.30.55.253 table server
ip rule add from 20.30.55.250 table server
ip rule add from 20.30.55.251 table server
ip rule add from 20.30.23.2 table server
ip rule add from 20.30.23.1 table server
It is noteworthy that many of the above commands will seemingly not take efect until you clear the route cache manually (I was somewhat surprised by this, since the kernel has been patched to flush the cache quite often for purposes of load-balancing). The command is:
echo 1 > /proc/sys/net/ipv4/route/f
I know the port-forwards are probably unnecessary - they are a temporary measure until I adjust the config to do NAT on the inbound (instead of just outbound, as it is doing now). Also, I suspect some of the route entries in the 'server' table are unnecessary, but I'm simply posting my working config at this point for the sake of expediency - I may post a slimmer config down the road...
Feel free to ask for clarification on any of the above.
-Jon
ip route add table server nat 20.30.55.249 via 10.1.1.2 scope host
ip route add table server nat 20.30.55.250 via 10.1.1.4 scope host
ip route add table server 20.30.55.251 dev eth0 scope link
ip route add table server nat 20.30.23.2 via 10.1.1.6 scope host
ip route add table server nat 20.30.55.253 via 10.1.1.3 scope host
ip route add table server 10.1.1.1 dev eth2 scope link
ip route add table server 20.30.55.248/29 dev eth0 proto kernel scope link src 20.30.55.251
ip route add table server 20.30.23.0/29 dev eth1 scope link
ip route add table server 10.1.1.0/24 dev eth2 proto kernel scope link src 10.1.1.1
ip route add default table server eql nexthop via 20.30.55.254 dev eth0 nexthop via 20.30.23.6 dev eth1
ip route add table server 127.0.0.0/8 dev lo scope link
ip rule add from 10.1.1.2 table server nat 20.30.55.249
ip rule add from 10.1.1.3 table server nat 20.30.55.253
ip rule add from 10.1.1.4 table server nat 20.30.55.250
ip rule add from 20.30.55.249 table server
ip rule add from 20.30.55.253 table server
ip rule add from 20.30.55.250 table server
ip rule add from 20.30.55.251 table server
ip rule add from 20.30.23.2 table server
ip rule add from 20.30.23.1 table server
It is noteworthy that many of the above commands will seemingly not take efect until you clear the route cache manually (I was somewhat surprised by this, since the kernel has been patched to flush the cache quite often for purposes of load-balancing). The command is:
echo 1 > /proc/sys/net/ipv4/route/f
I know the port-forwards are probably unnecessary - they are a temporary measure until I adjust the config to do NAT on the inbound (instead of just outbound, as it is doing now). Also, I suspect some of the route entries in the 'server' table are unnecessary, but I'm simply posting my working config at this point for the sake of expediency - I may post a slimmer config down the road...
Feel free to ask for clarification on any of the above.
-Jon
ASKER
Almost forgot to mention - the above config does not seem to contain any of the 'kinks' I mentioned in previous posts - traceroute works fine from all machines, speed is great, etc., etc...
-Jon
-Jon
ASKER
jlevie - I am about to close this and give you the points (if only for convincing me to stick with the site, and for putting up with me :-)
Would you rather I wait and award you the points next month, or are you needing more to add to your total for this month (assuming you are getting free access from your intense dedication to EE).
Just though I'd ask...
-Jon
Would you rather I wait and award you the points next month, or are you needing more to add to your total for this month (assuming you are getting free access from your intense dedication to EE).
Just though I'd ask...
-Jon
ASKER
Thanks for convincing me to stick with it thus far...
Any chance you know of a good place to bitch about the recent addition of (IMHO retarded) Flash 5.0 content to every single page at EE? I've almost gathered anough resolve to wash my hands of EE [again] if I can't get rid of this extreme annoyance [Frankly, it's not worth my time]... I though the EE site designers knew better. The worse thing is, my browser tends to crash after answering 'no' to about 20-30 of these completely obnoxious dialogues asking if I want to d/l Flash, or run ActiveX crap...
I am completely disgusted by this new turn of events.
Just as I thought things were looking up,
-Jon
Any chance you know of a good place to bitch about the recent addition of (IMHO retarded) Flash 5.0 content to every single page at EE? I've almost gathered anough resolve to wash my hands of EE [again] if I can't get rid of this extreme annoyance [Frankly, it's not worth my time]... I though the EE site designers knew better. The worse thing is, my browser tends to crash after answering 'no' to about 20-30 of these completely obnoxious dialogues asking if I want to d/l Flash, or run ActiveX crap...
I am completely disgusted by this new turn of events.
Just as I thought things were looking up,
-Jon
ASKER
-Jon