Want to win a PS4? Go Premium and enter to win our High-Tech Treats giveaway. Enter to Win

x
?
Solved

Network routing doesn't come up: RTNETLINK: Network is unreachable

Posted on 2010-09-08
12
Medium Priority
?
6,839 Views
Last Modified: 2012-08-13
Dear experts,

I have a CentOS 4.6 with two NICs:
eth0      Link encap:Ethernet  HWaddr 00:50:DA:B6:EF:E6
          inet addr:192.168.140.5  Bcast:192.168.140.255  Mask:255.255.255.0
          inet6 addr: fe80::250:daff:feb6:efe6/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1659 errors:0 dropped:0 overruns:0 frame:0
          TX packets:161 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:124890 (121.9 KiB)  TX bytes:16949 (16.5 KiB)
          Interrupt:169 Base address:0x4400

eth1      Link encap:Ethernet  HWaddr 00:14:5E:1F:38:D1
          inet addr:192.168.100.110  Bcast:192.168.100.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:85647 errors:0 dropped:0 overruns:0 frame:0
          TX packets:120177 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:15686518 (14.9 MiB)  TX bytes:113475875 (108.2 MiB)
          Interrupt:169

Open in new window


But the routing table doesn't show eth0 network:
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.100.0   *               255.255.255.0   U     0      0        0 eth1
172.16.146.0    *               255.255.255.0   U     0      0        0 vmnet8
172.16.153.0    *               255.255.255.0   U     0      0        0 vmnet1

Open in new window


Executing route add -net 192.168.140.0 netmask 255.255.255.0 eth0 doesn't return any errors, route stays on for half a second and then dissapears (again without any error).

The only error I get is, when doing ifup eth0:
RTNETLINK answers: Network is unreachable

I also received a report (it is a remote production server), that if take down eth1, eth0 network becomes available.

What is the answer, to get the second network available?
0
Comment
Question by:zeldi
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
12 Comments
 

Author Comment

by:zeldi
ID: 33625727
OK, I thing which I have found out: in /etc/sysconfig/network-scripts I had two files route-eth0 (referencing eth1 network) and route-eth1 (referencing eth0 network), which should be inversed. If i reverse the filenames I get the error
RTNETLINK answers: File exists
0
 

Author Comment

by:zeldi
ID: 33625751
Another one: if I remove files route-eth0 and route-eth1 the ifup eth0 doesn't return any error, I get a route to network:
169.254.0.0 * 255.255.0.0 U 0 0 0 eth0 (it's 169!!!)
for a couple of seconds, and then it is removed
0
 
LVL 6

Expert Comment

by:Jelcin
ID: 33625824
Hi, not sure if this helps but on my Ubuntu ifup and ifdown does not work. Try "ifconfig eth0 up".
0
Understanding Web Applications

Without even knowing it, most of us are using web applications on a daily basis. Gmail and Yahoo email, Twitter, Facebook, and eBay are used by most of us daily—and they are web applications. We often confuse these web applications tools for websites.  So, what is the difference?

 
LVL 57

Expert Comment

by:giltjr
ID: 33625836
Do you see any thing in /var/messages?

Do you see anything in the output of dmesg related to eth0.

It sounds as if eth0 is having physical problems and although it shows up, it really is not.
0
 

Author Comment

by:zeldi
ID: 33625889
Hi,

as mentioned before, I get no messages either in dmesg nor in /var/log/messages which would show any errors.

But the eth0 stays up (as can be seen from ifconfig), just the route gets dropped, while the route is up, the ping to remote network works, and then the route is dropped and ping stops working.
0
 
LVL 57

Expert Comment

by:giltjr
ID: 33627673
Without any overriding parameters if you do a ifup eth0 (or any other interface) it will attempt to do dhcp.  If there are no dhcp servers then it will use the "auto config" IP address of 169.254.0.0/16.

Try issuing:

     ip route list table all

or

     ip route list table cache

0
 

Author Comment

by:zeldi
ID: 33627854
As you have noticed in my first post, IPs are configured on the NICs (hence ifconfig shows the configured IPs).
Please find below attached the outputs from the ip route command

--------------------------------------------------------
ip route list table all
--------------------------------------------------------
192.168.100.0/24 dev eth1 proto kernel scope link src 192.168.100.110
172.16.146.0/24 dev vmnet8 proto kernel scope link src 172.16.146.1
172.16.153.0/24 dev vmnet1 proto kernel scope link src 172.16.153.1
default via 192.168.100.1 dev eth1
broadcast 192.168.100.0 dev eth1 table local proto kernel scope link src 192.168.100.110
broadcast 172.16.146.0 dev vmnet8 table local proto kernel scope link src 172.16.146.1
local 172.16.146.1 dev vmnet8 table local proto kernel scope host src 172.16.146.1
broadcast 127.255.255.255 dev lo table local proto kernel scope link src 127.0.0.1
broadcast 172.16.153.255 dev vmnet1 table local proto kernel scope link src 172.16.153.1
broadcast 192.168.100.255 dev eth1 table local proto kernel scope link src 192.168.100.110
broadcast 172.16.146.255 dev vmnet8 table local proto kernel scope link src 172.16.146.1
broadcast 172.16.153.0 dev vmnet1 table local proto kernel scope link src 172.16.153.1
local 172.16.153.1 dev vmnet1 table local proto kernel scope host src 172.16.153.1
local 192.168.100.110 dev eth1 table local proto kernel scope host src 192.168.100.110
broadcast 127.0.0.0 dev lo table local proto kernel scope link src 127.0.0.1
local 127.0.0.1 dev lo table local proto kernel scope host src 127.0.0.1
local 127.0.0.0/8 dev lo table local proto kernel scope host src 127.0.0.1
local ::1 via :: dev lo proto none metric 0 mtu 16436 rtt 1ms rttvar 1ms cwnd 2 advmss 16376 hoplimit 64
local fe80::250:56ff:fec0:1 via :: dev lo proto none metric 0 mtu 16436 advmss 16376 hoplimit 64
local fe80::250:56ff:fec0:8 via :: dev lo proto none metric 0 mtu 16436 advmss 16376 hoplimit 64
fe80::/64 dev vmnet8 metric 256 expires 2132910sec mtu 1500 advmss 1440 hoplimit 64
fe80::/64 dev vmnet1 metric 256 expires 2132910sec mtu 1500 advmss 1440 hoplimit 64
ff00::/8 dev vmnet8 metric 256 expires 2132910sec mtu 1500 advmss 1440 hoplimit 1
ff00::/8 dev vmnet1 metric 256 expires 2132910sec mtu 1500 advmss 1440 hoplimit 1
unreachable default dev lo proto none metric -1 error -101 hoplimit 25

--------------------------------------------------------
ip route list table cache
--------------------------------------------------------
broadcast 192.168.100.255 from 192.168.100.37 dev lo  src 192.168.100.110
    cache <local,brd,src-direct>  iif eth1
local 192.168.100.110 from 192.168.100.112 dev lo  src 192.168.100.110
    cache <local,src-direct>  iif eth1
local 192.168.100.110 from 192.168.100.118 dev lo  src 192.168.100.110
    cache <local,src-direct>  iif eth1
192.168.100.118 from 192.168.100.110 dev eth1
    cache  mtu 1500 advmss 1460 hoplimit 64
broadcast 192.168.100.255 from 192.168.100.44 dev lo  src 192.168.100.110
    cache <local,brd,src-direct>  iif eth1
broadcast 192.168.100.255 from 192.168.100.34 dev lo  src 192.168.100.110
    cache <local,brd,src-direct>  iif eth1
local 192.168.100.110 from 192.168.100.114 dev lo  src 192.168.100.110
    cache <local,src-direct>  iif eth1
broadcast 192.168.100.255 from 192.168.100.19 dev lo  src 192.168.100.110
    cache <local,brd,src-direct>  iif eth1
broadcast 192.168.100.255 from 192.168.100.10 dev lo  src 192.168.100.110
    cache <local,brd,src-direct>  iif eth1
local 127.0.0.1 from 127.0.0.1 dev lo
    cache <local>  mtu 16436 rtt 5ms rttvar 8ms cwnd 3 advmss 16396 hoplimit 64
broadcast 192.168.100.255 dev eth1  src 192.168.100.110
    cache <local,brd>  mtu 1500 advmss 1460 hoplimit 64
broadcast 192.168.100.255 from 192.168.100.114 dev lo  src 192.168.100.110
    cache <local,brd,src-direct>  iif eth1
broadcast 192.168.100.255 from 192.168.100.61 dev lo  src 192.168.100.110
    cache <local,brd,src-direct>  iif eth1
local 127.0.0.1 dev lo  src 127.0.0.1
    cache <local>  mtu 16436 advmss 16396 hoplimit 64
broadcast 192.168.100.255 from 192.168.100.118 dev lo  src 192.168.100.110
    cache <local,brd,src-direct>  iif eth1
192.168.100.114 from 192.168.100.110 dev eth1
    cache  mtu 1500 advmss 1460 hoplimit 64
broadcast 192.168.100.255 from 192.168.100.112 dev lo  src 192.168.100.110
    cache <local,brd,src-direct>  iif eth1
192.168.100.10 dev eth1  src 192.168.100.110
    cache  mtu 1500 advmss 1460 hoplimit 64
broadcast 192.168.100.255 from 192.168.100.60 dev lo  src 192.168.100.110
    cache <local,brd,src-direct>  iif eth1
broadcast 192.168.100.255 from 192.168.100.36 dev lo  src 192.168.100.110
    cache <local,brd,src-direct>  iif eth1
local 127.0.0.1 dev lo  src 127.0.0.1
    cache <local>  mtu 16436 advmss 16396 hoplimit 64
local 192.168.100.110 from 192.168.100.10 dev lo  src 192.168.100.110
    cache <local,src-direct>  iif eth1
local 192.168.100.110 from 192.168.8.55 dev lo  src 192.168.100.110
    cache <local>  iif eth1
broadcast 192.168.100.255 from 192.168.100.16 dev lo  src 192.168.100.110
    cache <local,brd,src-direct>  iif eth1
192.168.8.55 from 192.168.100.110 tos 0x10 via 192.168.100.1 dev eth1
    cache  mtu 1500 rtt 105ms rttvar 105ms cwnd 3 advmss 1460 hoplimit 64
broadcast 192.168.100.255 from 192.168.100.91 dev lo  src 192.168.100.110
    cache <local,brd,src-direct>  iif eth1
192.168.8.55 from 192.168.100.110 via 192.168.100.1 dev eth1
    cache  mtu 1500 advmss 1460 hoplimit 64
broadcast 192.168.100.255 from 192.168.100.140 dev lo  src 192.168.100.110
    cache <local,brd,src-direct>  iif eth1
broadcast 192.168.100.255 from 192.168.100.38 dev lo  src 192.168.100.110
    cache <local,brd,src-direct>  iif eth1
192.168.100.10 from 192.168.100.110 dev eth1
    cache  mtu 1500 rtt 26ms rttvar 42ms cwnd 2 advmss 1460 hoplimit 64
local 192.168.100.110 from 192.168.110.6 dev lo  src 192.168.100.110
    cache <local>  iif eth1
192.168.100.112 from 192.168.100.110 dev eth1
    cache  mtu 1500 rtt 15ms rttvar 28ms cwnd 3 advmss 1460 hoplimit 64

Open in new window

0
 
LVL 57

Accepted Solution

by:
giltjr earned 2000 total points
ID: 33628025
It appears that there is a script that is monitoring the interface and is issuing something like:

     ip route flush dev eth0

This will get rid of any/all route table entries that use the dev eth0.
0
 

Author Comment

by:zeldi
ID: 33628185
Just checking it right now..., I've just found process NetworkManager, which might be doing the dirty job...
0
 

Expert Comment

by:memoo_
ID: 33629069
zeldi ,  i need to check this file /etc/sysconfig/networks-scripts/ifcfg-eth0 and ifcfg-eth1 too, i think that i can fix the error



 
0
 
LVL 12

Expert Comment

by:mccracky
ID: 33631500
If it's a server, get rid of network manager and use the network-scripts.  fix the two route-ethX files.  The "file exists" error, if I remember correctly, doesn't really matter and things should be fine.  If not you can manually set the routing in rc.local.
0
 

Author Comment

by:zeldi
ID: 33657816
yup, it was a NetworkManager, now it is removed from init.

@giltjr: I wasn't really thinking, that any daemon would be doing such a stupid thing, but after your hint did a thorough check of all the processes and there it was.
0

Featured Post

[Webinar] Lessons on Recovering from Petya

Skyport is working hard to help customers recover from recent attacks, like the Petya worm. This work has brought to light some important lessons. New malware attacks like this can take down your entire environment. Learn from others mistakes on how to prevent Petya like worms.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Fine Tune your automatic Updates for Ubuntu / Debian
Make the most of your online learning experience.
There's a multitude of different network monitoring solutions out there, and you're probably wondering what makes NetCrunch so special. It's completely agentless, but does let you create an agent, if you desire. It offers powerful scalability …
In this brief tutorial Pawel from AdRem Software explains how you can quickly find out which services are running on your network, or what are the IP addresses of servers responsible for each service. Software used is freeware NetCrunch Tools (https…
Suggested Courses

618 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question