?
Solved

VOIP over MPLS Latency Issue

Posted on 2010-08-23
9
Medium Priority
?
857 Views
Last Modified: 2012-06-21
I am trying to determine if the output of my traceroutes signal a problem with my service providers MPLS delivery. We have been having call quality issues and I am wondering if this might be the culprit.

 
Here are 3 seperate outputs from the same 3825 over the MPLS to another 3825, commands run with about 60 seconds between them:

 
ORD3825#traceroute 10.30.30.254

Type escape sequence to abort.
Tracing the route to 10.30.30.254

  1 xxx.xxx.xxx.xxx 8 msec 4 msec 8 msec
  2  *  *  *
  3 169.130.82.14 [MPLS: Labels 3360/756 Exp 0] 168 msec 204 msec 204 msec
  4 169.130.82.33 [MPLS: Labels 2599/756 Exp 0] 52 msec 56 msec 52 msec
  5 169.130.80.17 [MPLS: Labels 3268/756 Exp 0] 52 msec 56 msec 52 msec
  6 169.130.82.29 [MPLS: Labels 3531/756 Exp 0] 52 msec 56 msec 52 msec
  7 169.130.80.26 [MPLS: Labels 288/756 Exp 0] 52 msec 56 msec 52 msec
  8 66.251.69.1 [MPLS: Labels 2247/756 Exp 0] 52 msec 52 msec 52 msec
  9 64.80.253.241 [MPLS: Labels 2447/756 Exp 0] 52 msec 64 msec 52 msec
10 64.80.253.234 [MPLS: Labels 977/756 Exp 0] 56 msec 52 msec 52 msec
11  *  *  *
12 64.196.222.153 [MPLS: Label 756 Exp 0] 48 msec 48 msec 48 msec
13 xxx.xxx.xxx.xxx 52 msec *  48 msec
ORD3825#
ORD3825#traceroute 10.30.30.254

Type escape sequence to abort.
Tracing the route to 10.30.30.254

  1 xxx.xxx.xxx.xxx 4 msec 4 msec 8 msec
  2  *  *  *
  3 169.130.82.14 [MPLS: Labels 3360/756 Exp 0] 52 msec 52 msec 56 msec
  4 169.130.82.33 [MPLS: Labels 2599/756 Exp 0] 52 msec 52 msec 52 msec
  5 169.130.80.17 [MPLS: Labels 3268/756 Exp 0] 52 msec 52 msec 52 msec
  6 169.130.82.29 [MPLS: Labels 3531/756 Exp 0] 52 msec 52 msec 52 msec
  7 169.130.80.26 [MPLS: Labels 288/756 Exp 0] 180 msec 204 msec 332 msec
  8 66.251.69.1 [MPLS: Labels 2247/756 Exp 0] 52 msec 52 msec 60 msec
  9 64.80.253.241 [MPLS: Labels 2447/756 Exp 0] 52 msec 52 msec 52 msec
10 64.80.253.234 [MPLS: Labels 977/756 Exp 0] 52 msec 56 msec 52 msec
11  *  *  *
12 64.196.222.153 [MPLS: Label 756 Exp 0] 48 msec 48 msec 48 msec
13 xxx.xxx.xxx.xxx 52 msec *  48 msec
ORD3825#
ORD3825#traceroute 10.30.30.254

Type escape sequence to abort.
Tracing the route to 10.30.30.254

  1 xxx.xxx.xxx.xxx 8 msec 4 msec 4 msec
  2  *  *  *
  3 169.130.82.14 [MPLS: Labels 3360/756 Exp 0] 172 msec 104 msec 180 msec
  4 169.130.82.33 [MPLS: Labels 2599/756 Exp 0] 60 msec 52 msec 52 msec
  5 169.130.80.17 [MPLS: Labels 3268/756 Exp 0] 52 msec 52 msec 56 msec
  6 169.130.82.29 [MPLS: Labels 3531/756 Exp 0] 52 msec 52 msec 52 msec
  7 169.130.80.26 [MPLS: Labels 288/756 Exp 0] 52 msec 56 msec 52 msec
  8 66.251.69.1 [MPLS: Labels 2247/756 Exp 0] 52 msec 56 msec 52 msec
  9 64.80.253.241 [MPLS: Labels 2447/756 Exp 0] 52 msec 56 msec 52 msec
10 64.80.253.234 [MPLS: Labels 977/756 Exp 0] 68 msec 52 msec 52 msec
11  *  *  *
12 64.196.222.153 [MPLS: Label 756 Exp 0] 48 msec 48 msec 48 msec
13 xxx.xxx.xxx.xxx 48 msec *  48 msec
ORD3825#
0
Comment
Question by:wantingtolearn
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 4
  • 4
9 Comments
 
LVL 6

Expert Comment

by:dinkon4u
ID: 33507994
Please post output of command ping to final destination..
0
 
LVL 9

Expert Comment

by:Alex Bahar
ID: 33509752
Your tracert looks good. 48 msec is a fair delay.
You can use the IOS built in IP SLA feature to measure RTP delay/jitter/packet loss with real RTP packets.
http://www.networkworld.com/community/node/30924
http://docwiki.cisco.com/wiki/IOS_IP_SLAs_UDP_Jitter_Operation_Technical_Analysis
Call quality issues are quite difficult to troubleshoot. You may want to check how much bandwidth is allocated for realtime voice (DSCP EF) by your service provider. Also check your CAC (call admission control) settings to make sure you're not oversubscribing your priority queue. On your WAN links, check the output of "show policy-map interface ..." to make sure you are classifying and queueing your voice traffic properly and you're not dropping packets at the interface.
 
0
 

Author Comment

by:wantingtolearn
ID: 33513982
I checked the output of the sh policy-map:

ORD3825#sh policy-map interface s0/0/0:0
 Serial0/0/0:0

  Service-policy output: AutoQoS-Policy-Trust

    Class-map: AutoQoS-VoIP-RTP-Trust (match-any)
      946717 packets, 60802272 bytes
      5 minute offered rate 25000 bps, drop rate 0 bps
      Match: ip dscp ef (46)
        946717 packets, 60802272 bytes
        5 minute rate 25000 bps
      Queueing
        Strict Priority
        Output Queue: Conversation 264
        Bandwidth 70 (%)
        Bandwidth 1075 (kbps) Burst 26875 (Bytes)
        (pkts matched/bytes matched) 8886/575780
        (total drops/bytes drops) 0/0

    Class-map: AutoQoS-VoIP-Control-Trust (match-any)
      0 packets, 0 bytes
      5 minute offered rate 0 bps, drop rate 0 bps
      Match: ip dscp cs3 (24)
        0 packets, 0 bytes
        5 minute rate 0 bps
      Match: ip dscp af31 (26)
        0 packets, 0 bytes
        5 minute rate 0 bps
      Queueing
        Output Queue: Conversation 265
        Bandwidth 5 (%)
        Bandwidth 76 (kbps)Max Threshold 64 (packets)
        (pkts matched/bytes matched) 0/0
        (depth/total drops/no-buffer drops) 0/0/0

    Class-map: class-default (match-any)
      26597 packets, 2577503 bytes
      5 minute offered rate 0 bps, drop rate 0 bps
      Match: any
      Queueing
        Flow Based Fair Queueing
        Maximum Number of Hashed Queues 256
        (total queued/total drops/no-buffer drops) 0/0/0

I dont see anything here.

I tried getting the IP SLA to work and am unable to get it working. I searched cisco's site and found a step by step. It still wont work.

I would like to get it working as I believe it will provide me with great insight as to what and where things may being going amuck.
0
Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 9

Expert Comment

by:Alex Bahar
ID: 33517684
It seems you are correctly marking and classifying voice packets, and you are not dropping any voice packets on your WAN interface. If you can get IP SLA working, it will give you a good idea about how your service provider MPLS network is performing.
    Class-map: AutoQoS-VoIP-RTP-Trust (match-any)
      946717 packets, 60802272 bytes
      5 minute offered rate 25000 bps, drop rate 0 bps
      Match: ip dscp ef (46)
        946717 packets, 60802272 bytes
        5 minute rate 25000 bps
      Queueing
        Strict Priority
        Output Queue: Conversation 264
        Bandwidth 70 (%)
        Bandwidth 1075 (kbps) Burst 26875 (Bytes)
        (pkts matched/bytes matched) 8886/575780
        (total drops/bytes drops) 0/0
0
 

Author Comment

by:wantingtolearn
ID: 33517694
Any insights- step by step to get it working on 12.4? I'm quickly running out of hair
0
 
LVL 9

Expert Comment

by:Alex Bahar
ID: 33518574
Have you checked the 12.4 IP SLA configuration guide? I suggest you to configure proactive monitoring and log onto a SYSLOG server.
See section for RTP configuration: http://www.cisco.com/en/US/docs/ios/12_4/ip_sla/configuration/guide/hsudpe.html
Section for proactive monitoring of jitter packet loss etc http://www.cisco.com/en/US/docs/ios/12_4/ip_sla/configuration/guide/hsthresh.html#wp1082288
0
 

Author Comment

by:wantingtolearn
ID: 33524900
Now today, out of nowhere I am getting dropped packets in my Output Queue. I have made no changes to QOS

Here is the output:

ORD3825#sh queue s0/0/0:0
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 354
  Queueing strategy: Class-based queueing
  Output queue: 0/1000/64/354 (size/max total/threshold/drops)
     Conversations  0/7/256 (active/max active/max total)
     Reserved Conversations 1/1 (allocated/max allocated)
     Available Bandwidth 1 kilobits/sec

Not sure why this would be happening. This is a full T1 and there are at the most 5 people making calls at the same time.

0
 

Author Comment

by:wantingtolearn
ID: 33525324
This output changes as well:

ORD3825#sh policy-map interface s0/0/0:0
 Serial0/0/0:0

  Service-policy output: AutoQoS-Policy-Trust

    Class-map: AutoQoS-VoIP-RTP-Trust (match-any)
      1714149 packets, 110033192 bytes
      5 minute offered rate 0 bps, drop rate 0 bps
      Match: ip dscp ef (46)
        1714149 packets, 110033192 bytes
        5 minute rate 0 bps
      Queueing
        Strict Priority
        Output Queue: Conversation 264
        Bandwidth 70 (%)
        Bandwidth 1075 (kbps) Burst 26875 (Bytes)
        (pkts matched/bytes matched) 20851/1334536
        (total drops/bytes drops) 0/0

    Class-map: AutoQoS-VoIP-Control-Trust (match-any)
      0 packets, 0 bytes
      5 minute offered rate 0 bps, drop rate 0 bps
      Match: ip dscp cs3 (24)
        0 packets, 0 bytes
        5 minute rate 0 bps
      Match: ip dscp af31 (26)
        0 packets, 0 bytes
        5 minute rate 0 bps
      Queueing
        Output Queue: Conversation 265
        Bandwidth 5 (%)
        Bandwidth 76 (kbps)Max Threshold 64 (packets)
        (pkts matched/bytes matched) 0/0
        (depth/total drops/no-buffer drops) 0/0/0

    Class-map: class-default (match-any)
      29565 packets, 3054678 bytes
      5 minute offered rate 0 bps, drop rate 0 bps
      Match: any
      Queueing
        Flow Based Fair Queueing
        Maximum Number of Hashed Queues 256
        (total queued/total drops/no-buffer drops) 0/370/0
0
 
LVL 9

Accepted Solution

by:
Alex Bahar earned 2000 total points
ID: 33527099
You are droping packets in the class-default. These are data packets.
0

Featured Post

Free Tool: IP Lookup

Get more info about an IP address or domain name, such as organization, abuse contacts and geolocation.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Trying to figure out group policy inheritance and which settings apply where can be a chore.  Here's a very simple summary I've written which might help.  Keep in mind, this is just a high-level conceptual overview where I try to avoid getting bogge…
An article on effective troubleshooting
Michael from AdRem Software outlines event notifications and Automatic Corrective Actions in network monitoring. Automatic Corrective Actions are scripts, which can automatically run upon discovery of a certain undesirable condition in your network.…
Michael from AdRem Software explains how to view the most utilized and worst performing nodes in your network, by accessing the Top Charts view in NetCrunch network monitor (https://www.adremsoft.com/). Top Charts is a view in which you can set seve…

801 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question