langdj
asked on
IP Rule not properly sending Traffic over a certain Ethernet Port
I have server that has a 10g card in it. I need to configure that server so that for a particular IP address (172.16.2.62) traffic goes over two 10g Ports. Here are the rules I have created for that purpose. If I do not setup a rule all traffic will go to Eth0 by default.
Rules for 172.16.2.62 [17:13:01] shock:~ # ip rule show |grep 172.16.2.62
60: from all to 172.16.2.62 lookup eth4
61: from all to 172.16.2.62 lookup eth3
However it seems that only the rule for Eth4 is being implemented [17:13:45] shock:~ # ip route get 172.16.2.62
172.16.2.62 via 172.16.1.1 dev eth4 src 172.16.2.177
cache mtu 1500 advmss 1460 hoplimit 64
As you can see ping will not go out Eth3 to 172.16.2.62
[17:18:02] shock:~ # ping -I eth3 172.16.2.62
PING 172.16.2.62 (172.16.2.62) from 172.16.2.175 eth3: 56(84) bytes of data. ^C --- 172.16.2.62 ping statistics --- 8 packets transmitted, 0 received, 100% packet loss, time 7707ms
[17:18:54] shock:~ # ping -I eth4 172.16.2.62
PING 172.16.2.62 (172.16.2.62) from 172.16.2.177 eth4: 56(84) bytes of data. 64 bytes from 172.16.2.62: icmp_seq=1 ttl=64 time=3.96 ms
Yet Eth3 works for other IPs not setup in the Rules (Notice I am pinging 172.16.1.1 and not 172.16.2.62)
[17:21:02] shock:~ # ping -I eth3 172.16.1.1
PING 172.16.1.1 (172.16.1.1) from 172.16.2.175 eth3: 56(84) bytes of data. 64 bytes from 172.16.1.1: icmp_seq=1 ttl=255 time=0.438 ms
Lastly here is the routing Table.
[17:24:27] shock:~ # netstat -rn Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface 172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth1
172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth2
172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth3
172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth4
172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth5
172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth6
0.0.0.0 172.16.1.1 0.0.0.0 UG 0 0 0 eth0
[17:25:49] shock:~ # ip route show table main
172.16.0.0/16 dev eth0 proto kernel scope link src 172.16.2.170
172.16.0.0/16 dev eth1 proto kernel scope link src 172.16.2.174
172.16.0.0/16 dev eth2 proto kernel scope link src 172.16.2.184
172.16.0.0/16 dev eth3 proto kernel scope link src 172.16.2.175
172.16.0.0/16 dev eth4 proto kernel scope link src 172.16.2.177
172.16.0.0/16 dev eth5 proto kernel scope link src 172.16.2.179
172.16.0.0/16 dev eth6 proto kernel scope link src 172.16.2.178
default via 172.16.1.1 dev eth0
Thanks in advance for your help
Rules for 172.16.2.62 [17:13:01] shock:~ # ip rule show |grep 172.16.2.62
60: from all to 172.16.2.62 lookup eth4
61: from all to 172.16.2.62 lookup eth3
However it seems that only the rule for Eth4 is being implemented [17:13:45] shock:~ # ip route get 172.16.2.62
172.16.2.62 via 172.16.1.1 dev eth4 src 172.16.2.177
cache mtu 1500 advmss 1460 hoplimit 64
As you can see ping will not go out Eth3 to 172.16.2.62
[17:18:02] shock:~ # ping -I eth3 172.16.2.62
PING 172.16.2.62 (172.16.2.62) from 172.16.2.175 eth3: 56(84) bytes of data. ^C --- 172.16.2.62 ping statistics --- 8 packets transmitted, 0 received, 100% packet loss, time 7707ms
[17:18:54] shock:~ # ping -I eth4 172.16.2.62
PING 172.16.2.62 (172.16.2.62) from 172.16.2.177 eth4: 56(84) bytes of data. 64 bytes from 172.16.2.62: icmp_seq=1 ttl=64 time=3.96 ms
Yet Eth3 works for other IPs not setup in the Rules (Notice I am pinging 172.16.1.1 and not 172.16.2.62)
[17:21:02] shock:~ # ping -I eth3 172.16.1.1
PING 172.16.1.1 (172.16.1.1) from 172.16.2.175 eth3: 56(84) bytes of data. 64 bytes from 172.16.1.1: icmp_seq=1 ttl=255 time=0.438 ms
Lastly here is the routing Table.
[17:24:27] shock:~ # netstat -rn Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface 172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth1
172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth2
172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth3
172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth4
172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth5
172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth6
0.0.0.0 172.16.1.1 0.0.0.0 UG 0 0 0 eth0
[17:25:49] shock:~ # ip route show table main
172.16.0.0/16 dev eth0 proto kernel scope link src 172.16.2.170
172.16.0.0/16 dev eth1 proto kernel scope link src 172.16.2.174
172.16.0.0/16 dev eth2 proto kernel scope link src 172.16.2.184
172.16.0.0/16 dev eth3 proto kernel scope link src 172.16.2.175
172.16.0.0/16 dev eth4 proto kernel scope link src 172.16.2.177
172.16.0.0/16 dev eth5 proto kernel scope link src 172.16.2.179
172.16.0.0/16 dev eth6 proto kernel scope link src 172.16.2.178
default via 172.16.1.1 dev eth0
Thanks in advance for your help
ASKER
In an effort to make my answer short I did not mention that this is an IBM SVC with limited networking. A locked down CentOS with no bonding support. It is an issue with my rules actually which has been figured out. (I needed a rule coming back not just going out) Thanks for your help
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
I've requested that this question be closed as follows:
Accepted answer: 0 points for langdj's comment #a40580350
for the following reason:
Question not Solved. Not sure why Expert Exchange would make me close this?
Accepted answer: 0 points for langdj's comment #a40580350
for the following reason:
Question not Solved. Not sure why Expert Exchange would make me close this?
We cannot help you hack the embedded firmware even it is based on CentOS.
ASKER
I work closely with SVC. And the the advise I was giving back was incorrect . I appreciate that the person was trying to help me. My thought is that it seems silly that I would be asked to "close" a question that was never answered. There should be an option for "never answered" if you want true metrics
You get no support modifying IBM machine code. Even hearing "it will not work" is not pleasant to your ear it is the only true answer.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Supported interfaces are: adding routes per interface and configuring VLAN trunking.
ASKER
I've requested that this question be deleted for the following reason:
Comments in question. Back and forth becoming a distraction
Comments in question. Back and forth becoming a distraction
Sorry, http:#a40580508 is final answer. (while the very first is for real CentOS/RHEL)
ASKER
Mr Wolfe,
The solution was that a custom network script needed to be be written on the target SVC with routes back to the source. This was necessary because the version of SVC I am using does not support bonding, VLANS or trunking (as stated earlier) and thus I was "required to make pigs to fly" Here is the example code back to the host that correct the issue
ip route add 172.26.0.0/24 via 172.26.1.1 dev eth3 table main
ip route add 172.26.0.0/24 via 172.26.1.1 dev eth3 table eth3
ip rule add from all to 172.26.0.0/24 lookup eth3 priority 90
ip rule add from 172.26.0.0/24 lookup eth3 priority 90
The solution was that a custom network script needed to be be written on the target SVC with routes back to the source. This was necessary because the version of SVC I am using does not support bonding, VLANS or trunking (as stated earlier) and thus I was "required to make pigs to fly" Here is the example code back to the host that correct the issue
ip route add 172.26.0.0/24 via 172.26.1.1 dev eth3 table main
ip route add 172.26.0.0/24 via 172.26.1.1 dev eth3 table eth3
ip rule add from all to 172.26.0.0/24 lookup eth3 priority 90
ip rule add from 172.26.0.0/24 lookup eth3 priority 90
Reading through your superlong post i just have one advice:
$ locate bonding.txt