Solved

Networking Issue / Web server Timeout / Cisco Routing Issue?

Posted on 2007-11-29
13
576 Views
Last Modified: 2010-04-21
The problem is that routing fails. This might not be in the router, but maybe related to another component on the internal network.

Here is the problem:
Connections to the web server timeout. There seems to be no pattern to when these timeouts will occur. I need to get rid of these timeouts.

I have been able to setup a continuous ping to the web servers and the timeouts occur at the same point when the ping fails.

Additional Network Information:
Continuous ping from the internet to the external port on the router does not fail.
Ping from the router to the web server does not fail.
Ping from the same internet location to the web server eventually fails.

The topology is Internet < -- > Router <--> Switch <--> Web Server

I am not sure what is causing this, it might be the router, the web server or a third piece of networking equipment. I have done some testing, but will do additional testing as requested.
0
Comment
Question by:asmo
  • 8
  • 4
13 Comments
 

Author Comment

by:asmo
Comment Utility
Additionally once the ping fails restarting it will not work. The ping only seems to start working again after I have established a TCP connection. I use lynx to connect to the web server. Once this works (usually takes 2-3 attempts to get a page served) the ping will start working.
0
 
LVL 16

Expert Comment

by:Blaz
Comment Utility
Is there a chance that the default route on the web server is changed when the connections do not work? This would explain why you can ping from router but not from the internet location.
0
 

Author Comment

by:asmo
Comment Utility
The web server has four network interfaces, two are not used. The other two, one is a public IP the other is a private IP. The public IP's have been changed (in a consistent fashion).

As far as I know the default routes are set correctly...the box is running solaris 10 (ultrasparc).

The assigned IP range is 29.23.15.0 / 28 (netmask of 255.255.255.240)

Here is the output from netstat and ipconfig in case it helps:

# netstat -rn
29.23.15.176      29.23.15.181      U         1     95 bge0
29.23.15.176      29.23.15.182      U         1      0 bge0:1
192.168.0.0          192.168.0.22         U         1    103 bge1
224.0.0.0            29.23.15.181      U         1      0 bge0
default              29.23.15.17      UG        1  66058
default              192.168.0.1          UG        1  22285
127.0.0.1            127.0.0.1            UH        2   3871 lo0

# ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000
bge0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
        inet 29.23.15.181 netmask fffffff0 broadcast 29.23.15.191
bge0:1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
        inet 29.23.15.182 netmask fffffff0 broadcast 29.23.15.191
bge1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3
        inet 192.168.0.22 netmask ffffff00 broadcast 192.168.0.255
0
 

Author Comment

by:asmo
Comment Utility
Some additional information:
It appears the web server can not ping out. It can ping hosts on the LAN but nothing on the internet (IE google.com).

I can telnet out of the webserver, however, and DNS is configured and properly resolving:

# nslookup cnn.com
Server:         127.0.0.1
Address:        127.0.0.1#53

Non-authoritative answer:
Name:   cnn.com
Address: 64.236.29.120
Name:   cnn.com
Address: 64.236.16.20
Name:   cnn.com
Address: 64.236.16.52
Name:   cnn.com
Address: 64.236.24.12

The fact that I can not ping out seems significant, however...
0
 
LVL 16

Expert Comment

by:Blaz
Comment Utility
Hm. In the "netstat -rn" command output it is written that you have two default gateways. Why? I dont know what the last number in the output is - could you explain?
0
 
LVL 40

Accepted Solution

by:
omarfarid earned 500 total points
Comment Utility
The problem could be (mostly) due to the 2nd default routing since it is going to private network (192.168.1.0).

Remove this routing entry and put static entries for your internal networks (use route add command) and put it part of your startup scripts (in /etc/rc2.d)

0
How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

 

Author Comment

by:asmo
Comment Utility
The last entry in the netstat -rn command is the loopback address, every machine should have this entry.

 I will try removing the default route for the internal network and then add static routing for that network.
0
 

Author Comment

by:asmo
Comment Utility
A couple of other things that I have done/been able to get to work:

I now have a constant ping from the server to the internet working, I'm not sure this was ever broken, the problem could have been an incorrect command line.

I have increased the size of the queues on the router. I do not believe this to have any effect since the queues were not filling, but perhaps under load something was happening.

I have removed the default route for the private network.

I will continue to monitor the pings to see if there is a failure.
0
 
LVL 16

Expert Comment

by:Blaz
Comment Utility
I didn't mean the last line - I mean the last number column 66058 vs. 22285 for the default gateways

Let us know how removing one default gateway works.
0
 

Author Comment

by:asmo
Comment Utility
Oh... my fault... that's the problem with English :-D That column is a usage statistic, how much traffic has gone through the given route.
0
 

Author Closing Comment

by:asmo
Comment Utility
So it turns out the operating system does in fact allow you to do morbidly silly things. After removing the second default route all seems to be working. I am still seeing some timeouts, but these appear to be a bug in the 64 bit APR on Solaris.

Thanks all!
0
 
LVL 16

Expert Comment

by:Blaz
Comment Utility
Hm. If removing one of the default routes was the solution I would wish to receive some points on this question.

I was the one who pointed to a probable trouble with the default route in my first comment (default route changing). And the first one who pointed to the probable trouble with two default routes in my second comment. Both comments were made before the "accepted answer" repeated my findings...
0
 

Author Comment

by:asmo
Comment Utility
I'll see what we can do to get you some points, you did steer the discussion towards the default routes though did not explicitly suggest removing one of them.

I agree with you that you deserve points for this. I'm not sure of the procedure, it used to be I'd post a question titled "Points for Blaz" is that still what is done?
0

Featured Post

What Should I Do With This Threat Intelligence?

Are you wondering if you actually need threat intelligence? The answer is yes. We explain the basics for creating useful threat intelligence.

Join & Write a Comment

Suggested Solutions

In the world of WAN, QoS is a pretty important topic for most, if not all, networks. Some WAN technologies have QoS mechanisms built in, but others, such as some L2 WAN's, don't have QoS control in the provider cloud.
Getting hacked is no longer a matter or "if you get hacked" — the 2016 cyber threat landscape is now titled "when you get hacked." When it happens — will you be proactive, or reactive?
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…

771 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now