Solved

Shoretel Voice Issues

Posted on 2014-09-05
5
901 Views
Last Modified: 2014-09-12
Hey everyone.  I am having some really weird issues with my Metro Ethernet sites and Shoretel.  Please note I am not the greatest at this as I have never had to deal with VoIP issues at my last job.  But here is the breakdown and what I have done.
 
Our company strictly runs HP ProCurve Routers/Switches and a few older 4500/4510g 3com switches.  We have 4 locations using Metro Ethernet 100mbps links and the phones (regardless of type) will randomly go into "Failover" mode and reboot.  The Shoretel tech said this was because they were losing connectivity to their ShoreGear T1k switches at our main location and our network was the cause.  I have ran Wireshark and captured VoIP traffic from our most troublesome site and could find no such occurrence of dropped packets.  Everyone conversation I analyzed had minimal jitter and seemed fine.  But then again I am not a wireshark professional.  I enabled SNMP and have been running Solarwinds Engineer's toolset using NPM, Interface Monitor and monitoring all the routers and switches in between the locations up to the Shoretel switch.  None of them show any dropped packets, errors or discards what so ever.  Their logs are all clear of any out of the norm errors.  The CPU stays low on all the units as well as the memory and the switches have more than enough buffer for the network traffic.  I checked to make sure the ports connecting the Shoretel switches were 100mpbs full as I had seen that could be an issue.  I also switched the HP switches that the Shoretel switches were plugged into to make sure that wasn't the issue.  The weird thing is, the one location that connects via IPSec VPN and has IP Phones (hollister on the map) does not have this issue.  The only common denominator is the Main Internal Router and our Metro Ethernet itself.  I have attached a generic Netmap to kind of show you the layout.  Any help or ideas would be much appreciated.  If you need more information please let me know.  Thanks.
Generic-NetMap.vsd
0
Comment
Question by:cmhtech
  • 3
  • 2
5 Comments
 
LVL 4

Expert Comment

by:Dimarc67
ID: 40307788
You mentioned you've adjusted the switch port duplex settings.  For those links that you've adjusted, make sure that both ends of the link are hard-set, or that auto-negotiation is set on both ends.  Any link with auto-negotiation set on only one end can easily result in a duplex mismatch issue (which can be a major bottleneck).

Also, your network diagram (very nice, by the way) doesn't seem to show the location of the Shoretel switch (apologies if I missed it).  Can you clarify where in the network topology the switch is connected?

Dimarc67
Frederick, MD
0
 

Author Comment

by:cmhtech
ID: 40308185
Thanks for the reply Dimarc.  I have hard set to 100/full on both ends.  Also thanks for the compliment on the network diagram.  The Net Admin before me had the main network in shambles.  As in all servers, switches, PC's, printers, and phones were on the 172.16.0.0/22 subnet, No STP enabled, No port security and static routes for everything, and had no documentation what so ever.  Surprisingly he was a CCNA... Anywho, the Shoretel Switches are behind the L3 switch at our main location.  On the map it is connected to the 172.16.6.0/23 subnet or phones as it is labeled I believe.  Basically a phone call from one of the problem locations would be routed like this:

192.168.10.50 (Remote site phone) --> HP Switch-->192.168.10.254 (remote HP Router inside) to 10.10.10.20(remote HP Router Outside) --> Metro Ethernet --> 10.10.10.15(Main HP router Outside) --> 172.16.0.126 (Main HP Router inside) --> 172.16.0.1 (Main Site L3 Switch) --> 172.16.7.254 (Phone Vlan on L3 switch) --> HP Phone distribution layer switch -->172.16.7.237||.238||.239 (All 3 Shoretel Switches), and then out a T-1 circuit to the phone company.  Make sense?  

I have ran wireshark a dozen times at various points between that communication line and I have been monitoring every device in my power in between that communication.  The Shoretel rep said the packets are probably just dropping along the line and not being discarded or dropped by my equipment.  He also said that network equipment rarely shows when it drops stuff...I think he is trying to cover his own butt and play the whole it's not my job card.  It wouldn't be the 1st, or 2nd or 3rd time they have blamed my network when it was their equipment or configuration.  The only proof I have of dropped packets are on the Shoretel switch logs themselves.  That makes me think they are dropping the packets for some reason but I can't get any assistance from Shoretel on that one.  I don't have jumbo frames or anything like that enabled and the buffer size on all switches are more than sufficient to handle the traffic.  None of the phones at our main location are having this issue.  It is only phones connected to the Metro Ethernet link.  If you need more info please let me know.  Thanks again.

Matt
0
 
LVL 4

Accepted Solution

by:
Dimarc67 earned 500 total points
ID: 40308596
Hmmm...  Not sure I have much to add.  Sounds like you're covering all the bases.

One possible test you can try is to open a port from the internet to the phone switch (if you don't already have one), and test phone calls across the internet from a remote site--bypassing the metro links and/or VPN links.  Results from this may be inconclusive, or they could also help narrow where the issue lies.

For that matter, you can even go so far as to temporarily create more-direct VPN tunnels, temporarily remove links or devices from the network, etc.  Anything you can do to manipulate the routing across different paths could help narrow the scope.  For testing purposes, the less complexity the better.

Wish I had something more specific for you.  Personally, I make myself a royal PITA with service providers when they're not helpful.  It's hard work (and takes a certain myopic arrogance), but it makes it easier for them to solve the problem than to have to put up with me.  :-)

Let me know if you find out anything new.  Interested to hear how you make out with your network, and your vendor.  

Dimarc67
Frederick, MD
0
 

Author Comment

by:cmhtech
ID: 40316061
Thanks for your help.  On Friday morning I am going to remove the Core Internal Router and just use the L3 switch for all my internal routing.  I have QoS configured on the switch so anything going to Vlan 6 (phones) has a high priority of 7 and everything else is 4 or less.  I will also plug in the 3 Shoretel switches at our main location directly into Access ports on that L3 switch onto Vlan 6.  That way it removes one hop from the Router and one switch.  It will at least narrow it down a little bit.  Plus that L3 switch will be able to handle the packets much faster than the cheap branch router ever could.  I'll let you know.
0
 

Author Comment

by:cmhtech
ID: 40320010
Well removing the old core router didn't help.  Although the network is performing better, the phones are still going into "failover" mode.  Looks like I am just going to have to be a PITA to some poor Shoretel rep at this point.  I find it hard to believe that I have network congestion that no SNMP software, network devices themselves or wireshark is picking up on.  And I have QoS configured to give phone traffic the highest priority.  Plus the fact that the branch office with the most issues literally has 20 IP phones and the slowest link is it's 100mbps Metro Ethernet connection back to the main office.  Most of the people over at the location use RDP all day with no drops or disconnects.  I would think if I were having network issues they would drop as well.  Thanks for the help Dimarc.
0

Featured Post

Threat Intelligence Starter Resources

Integrating threat intelligence can be challenging, and not all companies are ready. These resources can help you build awareness and prepare for defense.

Join & Write a Comment

Suggested Solutions

Title # Comments Views Activity
IPv6 Implementation - Cisco ASA5512 and Windows DHCPv6 1 47
Security Alert 2 45
Homegroup issues 6 39
ethernet cat5e lenght 80m 9 37
Even if you have implemented a Mobile Device Management solution company wide, it is a good idea to make sure you are taking into account all of the major risks to your electronic protected health information (ePHI).
If you're not part of the solution, you're part of the problem.   Tips on how to secure IoT devices, even the dumbest ones, so they can't be used as part of a DDoS botnet.  Use PRTG Network Monitor as one of the building blocks, to detect unusual…
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…
Get a first impression of how PRTG looks and learn how it works.   This video is a short introduction to PRTG, as an initial overview or as a quick start for new PRTG users.

743 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now