Solved

Linux and Dual Nic cards

Posted on 2009-05-05
22
1,143 Views
Last Modified: 2012-06-27
Hello,  I decided to go ahead and try out some called DRBD. I had everything with this program working fine with a single nic interface until I started to lose performance because of the network traffic being generated on a shared network.

So I garbed a couple spare nic cards and put it into both of the Linux machines (running ubuntu). Then reconfigured DRBD to look at the new network attached to the second nic cards. Now here is where my problem is coming in.

Once i added in the new nic card it seems like the machines are having a hard time figuring out which network to shuttle traffic through. This is the first time I have every used dual nic's in a linux box.

Now the primary network where the internet router is, is on the 192.168.0.0/24 network with a subnet of 255.255.255.0. Then DRBD's network is 10.10.10.0/24 with the same subnet. Now the DRBD network has no internet connection.

Now when i do a simple ping to google.com I get:
ping: unknown host google.com

Now i did not have such problems before i added the second nic card in.

Here is the route table:
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.0.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0
10.10.10.0      0.0.0.0         255.255.255.0   U     0      0        0 eth1
0.0.0.0         10.10.10.254    0.0.0.0         UG    100    0        0 eth1
0.0.0.0         192.168.0.254   0.0.0.0         UG    100    0        0 eth0


Pretty much i want to make sure all traffic that isn't ment for network on eth0 or eth1 to go out through eth0 gateway. If that makes any sense.

If you need more info let me know.
0
Comment
Question by:Pyromanci
  • 9
  • 7
  • 2
  • +3
22 Comments
 
LVL 29

Expert Comment

by:fosiul01
Comment Utility
can you ping from the Server where you have 2 nic card installed ??

the ping fail, does it occur from client pc ?? if yes
then

is your cat /proc/sys/net/ipv4/ip_forward

does it show 1 ??




0
 
LVL 5

Author Comment

by:Pyromanci
Comment Utility
Ok, from my PC which is also connected to the 192.168.0.0/24 network I can ping the machine. Also from the machine i can ping everything else on the 192.168.0.0/24 network, just can't get anything to reslove properly via host name. Now the resolv.conf file is configured properly too.
nameserver 192.168.0.254
nameserver 4.2.2.2
nameserver 4.3.3.3

when i run cat /proc/sys/net/ipv4/ip_forward it shows 0. though even if i change it to 1 using:
echo 1 > /proc/sys/net/ipv4/ip_forward
Just for the testing purposes it still doesn't help any.
0
 
LVL 29

Expert Comment

by:fosiul01
Comment Utility
Ok , i am not familier with DRBD method ( i will find out on google )

but let me tell you what i guess

are you trying to sharing internet betwen your Server and internal network ??

what about masquerade option from iptables ? if you dont enabled masquerade, internet would not share between Nic card 2
0
 
LVL 5

Author Comment

by:Pyromanci
Comment Utility
Well DRBD is a network based RAID 1 software. Its recommend but not required to have a dedicated network for it. So on the 10.10.10.0/24 network there are 2 machines and only 2 machines on it. Where the 192.168.0.0/24 network has roughly about 60 mahcines on it (including the 2 machines that are on the 10.10.10.0/24 network).
But as I said the amount of traffic it generates and the other traffic from all the other computers, I can see why its recommend. My Gigibit network went to a crawl when i started mirroring multiple treads of 5GB+ plus on top of the standard network traffic of web surfing and email. The AVG transfer rate on the machines droped to 1MBps

Now the internet is not shared between the 2 connections (it shouldn't be), because all DRBD does it talks to the other computer I tell it to for the mirroring and that is all. So pretty much that's the only program that should talk to that network.
0
 
LVL 5

Author Comment

by:Pyromanci
Comment Utility
Here. Let me give you a network layout. I find it easier my self when i have visual representation of a network makes it easier to understand the explanation.
network-layout.png
0
 
LVL 29

Expert Comment

by:fosiul01
Comment Utility
thats good, its easy

feq qestion, check the edited picture

So, Box A , has 2 nic card and Box B aswell

and i understand, you can ping from Box A ,
but anything from 10.10.10.X network unable to ping ??

is that right ???


Or main router has 2 nic card :nic1 : 192.168.0.1 and nic2 : 10.10.10.1

and linux Box A and linux Box B is connected to that main router via 10.10.10.254 gateway ??

and From linux Box A and linux Box B you can ping to any externer address


if i am wrong please assing the Ip, it would be more easy ...

0
 
LVL 29

Expert Comment

by:fosiul01
Comment Utility
sorry the picture is attached
new-pic.GIF
0
 
LVL 29

Expert Comment

by:fosiul01
Comment Utility
sorry check this one

the previous one is old
new-pic.GIF
0
 
LVL 29

Expert Comment

by:fosiul01
Comment Utility
will be online after 2 hours
0
 
LVL 5

Author Comment

by:Pyromanci
Comment Utility
Your Very close. Your picture is right though.

The differance is this:

Box A you can ping Box B via the 192.168.0.0/24 network and the 10.10.10.X network.
Box B is the same
All the PC's can ping Box A or Box B via the 192.168.0.0/24 network.

Box A and Box B can not ping anything outside of the network (google for example)
The PC's can.
0
 
LVL 5

Author Comment

by:Pyromanci
Comment Utility
OK, i have been playing with things seeing what i need to do to get this all to work.

I found if i delete
0.0.0.0         10.10.10.254    0.0.0.0         UG    100    0        0 eth1

from my route table. Everything starts to work (not 100% avg packet loss on a ping 53% even after a cache flush).  

Now of course
0.0.0.0         10.10.10.254    0.0.0.0         UG    100    0        0 eth1
is a automatic route configuration.

Now all in all i am not sure if this brings up more questions, or helps narrow down a solution.
0
Windows Server 2016: All you need to know

Learn about Hyper-V features that increase functionality and usability of Microsoft Windows Server 2016. Also, throughout this eBook, you’ll find some basic PowerShell examples that will help you leverage the scripts in your environments!

 
LVL 12

Expert Comment

by:rionroc
Comment Utility
Hello

Issue an "echo" command at terminal, if it works, make a script to call it automatically during load.

## ISSUE THIS COMMAND (or type it in the command terminal)
echo 1 > /proc/sys/net/ipv4/ip_forward
echo 1 > /proc/sys/net/ipv4/conf/eth0/forwarding

>Box A and Box B can not ping anything outside of the network (google for example)
Then try to ping again.

## IF WILL NOT WORK, TRY TO ISSUE SOME IPTABLES COMMAND AND REPEAT THE ECHO COMMAND
# Flush all chains
iptables --flush
# Allow UNLIMITED traffic on the loopback interface
iptables -A INPUT -i lo -j ACCEPT
iptables -A OUTPUT -o lo -j ACCEPT

Cheers!
0
 
LVL 5

Expert Comment

by:sleepless6
Comment Utility
are you trying to use 192.168.0.0/24 as the primary internet connection or the 10.10.10.0/24

0
 
LVL 29

Expert Comment

by:fosiul01
Comment Utility
Hi yah

good morning
I am realy sorry for not attending, i was busy due to some family work

have a look to my one

[root@localhost ~]# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.2.0     0.0.0.0         255.255.255.0   U     0      0        0 eth1
192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0
169.254.0.0     0.0.0.0         255.255.0.0     U     0      0        0 eth1
0.0.0.0         192.168.1.1     0.0.0.0         UG    0      0        0 eth0    -------------- this is the default gateway  for Internet browsing


whats your ???

in your case : eth1  and eth0
which one is connected to 192.168.0.x and 10.10.0.x network ??
you need to make 192.168.0.x to main gateway
0
 
LVL 5

Author Comment

by:Pyromanci
Comment Utility
If you look up to the main topic the route table is there, but here it is again

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.0.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0
10.10.10.0      0.0.0.0         255.255.255.0   U     0      0        0 eth1
0.0.0.0         10.10.10.254    0.0.0.0         UG    100    0        0 eth1
0.0.0.0         192.168.0.254   0.0.0.0         UG    100    0        0 eth0

the 192.168.0.0/24 network is the only network out of the 2 that has internet.
0
 
LVL 5

Expert Comment

by:louislietaer
Comment Utility
remove this route

0.0.0.0         10.10.10.254    0.0.0.0         UG    100    0        0 eth1
0
 
LVL 12

Expert Comment

by:rionroc
Comment Utility
(my 2nd comment)
echo 1 > /proc/sys/net/ipv4/ip_forward

Have you tried my first comment?  I hope you have.

Cheers!
0
 
LVL 7

Accepted Solution

by:
diepes earned 250 total points
Comment Utility
I dont think you want to use the Linux as router, 2nd Nic is only for access to DRBD storage.

If the Linux (2nd Card) and the Storage server is on the same subnet, you only need a IP on the interface and they will be able to talk.
You then need to switch OFF routing, it is only needed if you planned to use the Linux box as a router to connect the subnets for other pc's (echo 0 > /proc/sys/net/ipv4/ip_forward)

Look at your routing table, you should have only one route for the 2nd Subnet, and that should be for the local lan, and only default GW 0.0.0.0 route pointing to the original GW (192.168.0.254) on 1st Network card.

To make it permanent, remove any Gateway settings you have for the 2nd(10.10.10.x) network card, it should only have a ip, this will inform linux to use this card for only that subnet to connect to the BRDB in same subnet.


0
 
LVL 29

Expert Comment

by:fosiul01
Comment Utility
hi good morning

have you tryed by delete  @louislietaer said ??

0.0.0.0         10.10.10.254    0.0.0.0         UG    100    0        0 eth1     ??  


it should solve the problem ..
0
 
LVL 5

Expert Comment

by:louislietaer
Comment Utility
thanks fosiul01

Also I would not recommend ip forwarding in that case


0
 
LVL 29

Assisted Solution

by:fosiul01
fosiul01 earned 250 total points
Comment Utility
@louislietaer, yes i agree  here Ip forwarding is not the issue,

ipforwarding would be issue if the clients from Box A and Box B are unable to get intenret,

but here Box A itself is not getting internet but  pc1, pc2 they are fine,
so i guess routing table is the problem

also the author said
"
I found if i delete
0.0.0.0         10.10.10.254    0.0.0.0         UG    100    0        0 eth1

"

this is because, there should be only 1 default gateway, but from his routing table there is 2 default gateway
so he will have to delete 1

which i guess, he found out already
0
 
LVL 5

Author Comment

by:Pyromanci
Comment Utility
Ok,  sorry for the late reply everyone. I got slammed with something else and had to stop working on this problem for a bit.

I have the problem resolved thanks to several of you.

What i ended up doing is altering the /etc/syscntrl.conf file to always keep
 /proc/sys/net/ipv4/ip_forward as 1.

Then i went through and made a cross over cable and removed the router from the 10.0.0.0 network and linked the computer directly.

And finally strip the gateway def from the /etc/network/interfaces file for the second nic card.
0

Featured Post

Better Security Awareness With Threat Intelligence

See how one of the leading financial services organizations uses Recorded Future as part of a holistic threat intelligence program to promote security awareness and proactively and efficiently identify threats.

Join & Write a Comment

Suggested Solutions

Title # Comments Views Activity
2 LAN/WAN on One Server 2 43
Windows 7 Share with XP 22 55
Sendmail STARTTLS error 37 36
Cisco VSS or VCP on GNS3 or IOU 3 22
Some time ago I was asked to set up a web portal PC to put at our entrance. When customers arrive, they could see a webpage 'promoting' our company. So I tried to set up a windows 7 PC as a kiosk PC.......... I will spare you all the annoyances I…
Linux users are sometimes dumbfounded by the severe lack of documentation on a topic. Sometimes, the documentation is copious, but other times, you end up with some obscure "it varies depending on your distribution" over and over when searching for …
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…

728 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

14 Experts available now in Live!

Get 1:1 Help Now