Solved

VOIP over MPLS Latency Issue

Posted on 2010-08-23
9
810 Views
Last Modified: 2012-06-21
I am trying to determine if the output of my traceroutes signal a problem with my service providers MPLS delivery. We have been having call quality issues and I am wondering if this might be the culprit.

 
Here are 3 seperate outputs from the same 3825 over the MPLS to another 3825, commands run with about 60 seconds between them:

 
ORD3825#traceroute 10.30.30.254

Type escape sequence to abort.
Tracing the route to 10.30.30.254

  1 xxx.xxx.xxx.xxx 8 msec 4 msec 8 msec
  2  *  *  *
  3 169.130.82.14 [MPLS: Labels 3360/756 Exp 0] 168 msec 204 msec 204 msec
  4 169.130.82.33 [MPLS: Labels 2599/756 Exp 0] 52 msec 56 msec 52 msec
  5 169.130.80.17 [MPLS: Labels 3268/756 Exp 0] 52 msec 56 msec 52 msec
  6 169.130.82.29 [MPLS: Labels 3531/756 Exp 0] 52 msec 56 msec 52 msec
  7 169.130.80.26 [MPLS: Labels 288/756 Exp 0] 52 msec 56 msec 52 msec
  8 66.251.69.1 [MPLS: Labels 2247/756 Exp 0] 52 msec 52 msec 52 msec
  9 64.80.253.241 [MPLS: Labels 2447/756 Exp 0] 52 msec 64 msec 52 msec
10 64.80.253.234 [MPLS: Labels 977/756 Exp 0] 56 msec 52 msec 52 msec
11  *  *  *
12 64.196.222.153 [MPLS: Label 756 Exp 0] 48 msec 48 msec 48 msec
13 xxx.xxx.xxx.xxx 52 msec *  48 msec
ORD3825#
ORD3825#traceroute 10.30.30.254

Type escape sequence to abort.
Tracing the route to 10.30.30.254

  1 xxx.xxx.xxx.xxx 4 msec 4 msec 8 msec
  2  *  *  *
  3 169.130.82.14 [MPLS: Labels 3360/756 Exp 0] 52 msec 52 msec 56 msec
  4 169.130.82.33 [MPLS: Labels 2599/756 Exp 0] 52 msec 52 msec 52 msec
  5 169.130.80.17 [MPLS: Labels 3268/756 Exp 0] 52 msec 52 msec 52 msec
  6 169.130.82.29 [MPLS: Labels 3531/756 Exp 0] 52 msec 52 msec 52 msec
  7 169.130.80.26 [MPLS: Labels 288/756 Exp 0] 180 msec 204 msec 332 msec
  8 66.251.69.1 [MPLS: Labels 2247/756 Exp 0] 52 msec 52 msec 60 msec
  9 64.80.253.241 [MPLS: Labels 2447/756 Exp 0] 52 msec 52 msec 52 msec
10 64.80.253.234 [MPLS: Labels 977/756 Exp 0] 52 msec 56 msec 52 msec
11  *  *  *
12 64.196.222.153 [MPLS: Label 756 Exp 0] 48 msec 48 msec 48 msec
13 xxx.xxx.xxx.xxx 52 msec *  48 msec
ORD3825#
ORD3825#traceroute 10.30.30.254

Type escape sequence to abort.
Tracing the route to 10.30.30.254

  1 xxx.xxx.xxx.xxx 8 msec 4 msec 4 msec
  2  *  *  *
  3 169.130.82.14 [MPLS: Labels 3360/756 Exp 0] 172 msec 104 msec 180 msec
  4 169.130.82.33 [MPLS: Labels 2599/756 Exp 0] 60 msec 52 msec 52 msec
  5 169.130.80.17 [MPLS: Labels 3268/756 Exp 0] 52 msec 52 msec 56 msec
  6 169.130.82.29 [MPLS: Labels 3531/756 Exp 0] 52 msec 52 msec 52 msec
  7 169.130.80.26 [MPLS: Labels 288/756 Exp 0] 52 msec 56 msec 52 msec
  8 66.251.69.1 [MPLS: Labels 2247/756 Exp 0] 52 msec 56 msec 52 msec
  9 64.80.253.241 [MPLS: Labels 2447/756 Exp 0] 52 msec 56 msec 52 msec
10 64.80.253.234 [MPLS: Labels 977/756 Exp 0] 68 msec 52 msec 52 msec
11  *  *  *
12 64.196.222.153 [MPLS: Label 756 Exp 0] 48 msec 48 msec 48 msec
13 xxx.xxx.xxx.xxx 48 msec *  48 msec
ORD3825#
0
Comment
Question by:wantingtolearn
  • 4
  • 4
9 Comments
 
LVL 6

Expert Comment

by:dinkon4u
Comment Utility
Please post output of command ping to final destination..
0
 
LVL 9

Expert Comment

by:Alex Bahar
Comment Utility
Your tracert looks good. 48 msec is a fair delay.
You can use the IOS built in IP SLA feature to measure RTP delay/jitter/packet loss with real RTP packets.
http://www.networkworld.com/community/node/30924
http://docwiki.cisco.com/wiki/IOS_IP_SLAs_UDP_Jitter_Operation_Technical_Analysis
Call quality issues are quite difficult to troubleshoot. You may want to check how much bandwidth is allocated for realtime voice (DSCP EF) by your service provider. Also check your CAC (call admission control) settings to make sure you're not oversubscribing your priority queue. On your WAN links, check the output of "show policy-map interface ..." to make sure you are classifying and queueing your voice traffic properly and you're not dropping packets at the interface.
 
0
 

Author Comment

by:wantingtolearn
Comment Utility
I checked the output of the sh policy-map:

ORD3825#sh policy-map interface s0/0/0:0
 Serial0/0/0:0

  Service-policy output: AutoQoS-Policy-Trust

    Class-map: AutoQoS-VoIP-RTP-Trust (match-any)
      946717 packets, 60802272 bytes
      5 minute offered rate 25000 bps, drop rate 0 bps
      Match: ip dscp ef (46)
        946717 packets, 60802272 bytes
        5 minute rate 25000 bps
      Queueing
        Strict Priority
        Output Queue: Conversation 264
        Bandwidth 70 (%)
        Bandwidth 1075 (kbps) Burst 26875 (Bytes)
        (pkts matched/bytes matched) 8886/575780
        (total drops/bytes drops) 0/0

    Class-map: AutoQoS-VoIP-Control-Trust (match-any)
      0 packets, 0 bytes
      5 minute offered rate 0 bps, drop rate 0 bps
      Match: ip dscp cs3 (24)
        0 packets, 0 bytes
        5 minute rate 0 bps
      Match: ip dscp af31 (26)
        0 packets, 0 bytes
        5 minute rate 0 bps
      Queueing
        Output Queue: Conversation 265
        Bandwidth 5 (%)
        Bandwidth 76 (kbps)Max Threshold 64 (packets)
        (pkts matched/bytes matched) 0/0
        (depth/total drops/no-buffer drops) 0/0/0

    Class-map: class-default (match-any)
      26597 packets, 2577503 bytes
      5 minute offered rate 0 bps, drop rate 0 bps
      Match: any
      Queueing
        Flow Based Fair Queueing
        Maximum Number of Hashed Queues 256
        (total queued/total drops/no-buffer drops) 0/0/0

I dont see anything here.

I tried getting the IP SLA to work and am unable to get it working. I searched cisco's site and found a step by step. It still wont work.

I would like to get it working as I believe it will provide me with great insight as to what and where things may being going amuck.
0
 
LVL 9

Expert Comment

by:Alex Bahar
Comment Utility
It seems you are correctly marking and classifying voice packets, and you are not dropping any voice packets on your WAN interface. If you can get IP SLA working, it will give you a good idea about how your service provider MPLS network is performing.
    Class-map: AutoQoS-VoIP-RTP-Trust (match-any)
      946717 packets, 60802272 bytes
      5 minute offered rate 25000 bps, drop rate 0 bps
      Match: ip dscp ef (46)
        946717 packets, 60802272 bytes
        5 minute rate 25000 bps
      Queueing
        Strict Priority
        Output Queue: Conversation 264
        Bandwidth 70 (%)
        Bandwidth 1075 (kbps) Burst 26875 (Bytes)
        (pkts matched/bytes matched) 8886/575780
        (total drops/bytes drops) 0/0
0
How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

 

Author Comment

by:wantingtolearn
Comment Utility
Any insights- step by step to get it working on 12.4? I'm quickly running out of hair
0
 
LVL 9

Expert Comment

by:Alex Bahar
Comment Utility
Have you checked the 12.4 IP SLA configuration guide? I suggest you to configure proactive monitoring and log onto a SYSLOG server.
See section for RTP configuration: http://www.cisco.com/en/US/docs/ios/12_4/ip_sla/configuration/guide/hsudpe.html
Section for proactive monitoring of jitter packet loss etc http://www.cisco.com/en/US/docs/ios/12_4/ip_sla/configuration/guide/hsthresh.html#wp1082288
0
 

Author Comment

by:wantingtolearn
Comment Utility
Now today, out of nowhere I am getting dropped packets in my Output Queue. I have made no changes to QOS

Here is the output:

ORD3825#sh queue s0/0/0:0
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 354
  Queueing strategy: Class-based queueing
  Output queue: 0/1000/64/354 (size/max total/threshold/drops)
     Conversations  0/7/256 (active/max active/max total)
     Reserved Conversations 1/1 (allocated/max allocated)
     Available Bandwidth 1 kilobits/sec

Not sure why this would be happening. This is a full T1 and there are at the most 5 people making calls at the same time.

0
 

Author Comment

by:wantingtolearn
Comment Utility
This output changes as well:

ORD3825#sh policy-map interface s0/0/0:0
 Serial0/0/0:0

  Service-policy output: AutoQoS-Policy-Trust

    Class-map: AutoQoS-VoIP-RTP-Trust (match-any)
      1714149 packets, 110033192 bytes
      5 minute offered rate 0 bps, drop rate 0 bps
      Match: ip dscp ef (46)
        1714149 packets, 110033192 bytes
        5 minute rate 0 bps
      Queueing
        Strict Priority
        Output Queue: Conversation 264
        Bandwidth 70 (%)
        Bandwidth 1075 (kbps) Burst 26875 (Bytes)
        (pkts matched/bytes matched) 20851/1334536
        (total drops/bytes drops) 0/0

    Class-map: AutoQoS-VoIP-Control-Trust (match-any)
      0 packets, 0 bytes
      5 minute offered rate 0 bps, drop rate 0 bps
      Match: ip dscp cs3 (24)
        0 packets, 0 bytes
        5 minute rate 0 bps
      Match: ip dscp af31 (26)
        0 packets, 0 bytes
        5 minute rate 0 bps
      Queueing
        Output Queue: Conversation 265
        Bandwidth 5 (%)
        Bandwidth 76 (kbps)Max Threshold 64 (packets)
        (pkts matched/bytes matched) 0/0
        (depth/total drops/no-buffer drops) 0/0/0

    Class-map: class-default (match-any)
      29565 packets, 3054678 bytes
      5 minute offered rate 0 bps, drop rate 0 bps
      Match: any
      Queueing
        Flow Based Fair Queueing
        Maximum Number of Hashed Queues 256
        (total queued/total drops/no-buffer drops) 0/370/0
0
 
LVL 9

Accepted Solution

by:
Alex Bahar earned 500 total points
Comment Utility
You are droping packets in the class-default. These are data packets.
0

Featured Post

Do You Know the 4 Main Threat Actor Types?

Do you know the main threat actor types? Most attackers fall into one of four categories, each with their own favored tactics, techniques, and procedures.

Join & Write a Comment

Hey there Heard about jingle, the add on for XMPP that enables point to point audio between two XMPP clients. No server config necessary. Actually quite a cool feature. However, how good is it if you can not use those voice capabilities to do a P…
Many of us in IT utilize a combination of roaming profiles and folder redirection to ensure user information carries over from one workstation to another; in my environment, it was to enable virtualization without needing a separate desktop for each…
Sending a Secure fax is easy with eFax Corporate (http://www.enterprise.efax.com). First, Just open a new email message.  In the To field, type your recipient's fax number @efaxsend.com. You can even send a secure international fax — just include t…
Internet Business Fax to Email Made Easy - With eFax Corporate (http://www.enterprise.efax.com), you'll receive a dedicated online fax number, which is used the same way as a typical analog fax number. You'll receive secure faxes in your email, fr…

743 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

15 Experts available now in Live!

Get 1:1 Help Now