Link to home
Start Free TrialLog in
Avatar of ahmzb1990
ahmzb1990

asked on

Unable to transfer a call between a SIP Trunk and Call Manager

Hi,

We are currently having an issue transferring a call between a SIP extn and a extn sitting on cisco call manager.

Background:
We are currently using Cisco call manager version: 7.1.5.31900-3
Approx 2 months ago we purchased a call centre software called "IPCM" which integrates with our CRM and our call manager. IPCM is SIP based. This program provides us with Call Queing/Dashbaords  soft phones etc.

 A SIP trunk was created in our call manager to route incoming calls over to our new "Call centre" server. A new extn range was also configured to allow us to make outgoing calls via the softphones:

Problem:
The issue that we are currently facing is call transfer.  We are unable to transfer a call from "IPCM" to a cisco phone.

i.e.
Using a IPCM (SIP) Softphone, I make a call to a number i.e. 0412 345 678
I pick this call up and attempt to transfer this through to a cisco extension i.e. 244
the cisco extension (244) rings and picks up. I then attempt to press the "Transfer" button. The mobile call automatically disconnects.

We have checked and confirmed that the Trunk allows all requests i.e. INVITE, REFER, BYE etc.

Further Comments:
- Using the same method above (in the example) I am able to succesfully transfer the call if i the call is transferred to a complete number i.e. 0293142-244 rather than the agents extension (244).


Any help will be appreciated.
Avatar of José Méndez
José Méndez

If you wish, collect a packet capture while reproducing the issue, and then upload it to cloudshark.org or attach it here, I can help you decode what happens from the CUCM side:

https://supportforums.cisco.com/docs/DOC-11599
Avatar of ahmzb1990

ASKER

Thanks for your response, will it be okay if we run the capture using wireshark?
Wouldn´t that be 100 times more difficult? Its fine by me if you capture the same traffic information, just wanted to save you time.
thanks okay - I will do it your way and post back
It sounds like you need transcode resources. You may have "media termination point" required checked under the SIP Trunk config on CM and then not have transcode resources in the defined "media resource group list" or may not have a MRGL defined at all.

User generated image
Hope this helps
Hi bfason,

I can confirm the medical resource group list has been configured. please see screenshot.

Thanks
trunk-config.jpg
Hi willlywilburwonka

Sorry for the delay in getting back- It has been pretty hectik here - I have attached to  a wire shark capture (as i already had the application installed on the server) and uploaded the capture to cloud shark. Please see url for capture below.

http://cloudshark.org/captures/bfd2b699b2d4

Here are some further details:

Ip Address for Call Manager: 192.168.20.20/21
Client IP (Original Caller) 10.10.20.89 (Extn: 611)
Destination Etn# 9253 (9 being the prefix that sends the call from IPCM through the trunk to call manager.

Please advise if you require any further info

Thanks
Using a IPCM (SIP) Softphone, I make a call to a number i.e. 0412 345 678
I pick this call up

At that point you have a call to the outside connected to your softphone. Is that softphone registered to Callmanager or is it handled by IPCM?

and attempt to transfer this through to a cisco extension i.e. 244
the cisco extension (244) rings and picks up. I then attempt to press the "Transfer" button. The mobile call automatically disconnects.

The transfer action is being performed on the softphone, not on the Cisco phone, correct? I just want to have a clear understanding of the call flow instead of assuming facts.

Now, the packet capture you gathered doesn't belong to the traffic monitored by the UCM. Please, please follow my suggestion in the link provided, its 1 command to generate the packet capture, and a few clicks to download it from the server once finished.

In the sniffer capture collected, I see only 3 INVITEs toward UCM's IP:

INVITE sip:00423246822@192.168.20.20
INVITE sip:00403367708@192.168.20.20
INVITE sip:204@192.168.20.20

All from a SIP engine called FR-SIPS/5.8.1.166

Kindly collect the capture from the UCM, and help me clarify why can't I see an INVITE to 9253. Is it maybe a digit manipulation into 204?
Thanks for your reply.  

At that point you have a call to the outside connected to your softphone. Is that softphone registered to Callmanager or is it handled by IPCM?

The softphone is handled by IPCM


The transfer action is being performed on the softphone, not on the Cisco phone, correct? I just want to have a clear understanding of the call flow instead of assuming facts.

This is correct - The transfer is being performed on the softphone belonging to IPCM

Now, the packet capture you gathered doesn't belong to the traffic monitored by the UCM. Please, please follow my suggestion in the link provided, its 1 command to generate the packet capture, and a few clicks to download it from the server once finished.

I do not have access to SSH for the call managers - I will need dig through and find out who the cisco contacts are and request the login credentials

In the sniffer capture collected, I see only 3 INVITEs toward UCM's IP:

INVITE sip:00423246822@192.168.20.20
INVITE sip:00403367708@192.168.20.20
INVITE sip:204@192.168.20.20

All from a SIP engine called FR-SIPS/5.8.1.166

Kindly collect the capture from the UCM, and help me clarify why can't I see an INVITE to 9253. Is it maybe a digit manipulation into 204?

I will do everything i can to collect the capture - the 9 prefix is used to tell IPCM to push the call through the trunk to the call manager and the prefix is stripped

Thanks
ok, so we should be seeing a INVITE sip:9253@192.168.20.20, or a INVITE sip:253@192.168.20.20

In the previous packet capture there was none.
HAH!

I found a call to extension 253! but the IP is not 192.168.20.20, its 192.168.20.21, I didn't identify the second IP you posted because it looked like  a subnet mask. So what its happening in that particular call, is that 192.168.20.25 is disconnecting it with a BYE message, but it happens within miliseconds, so I am not sure if that is your test call??

User generated image
Thanks for your response.

Thats correct, This was only test call.
could it be perhaps that the ipcm server (10.10.20.25) is disconnecting the call as it doesnt recognise that extn number as there was no prefix to tell IPCM to send the call back through the trunk to call manager during the transfer process?
Nope, I doubt it. The call is actually traversing over to CM and CM is responding to it just fine. The call is being set up for G711, could that be it? Is the other device G711 capable? Im sure its capable, but, is it allowed to use it?
I believe G711 is fine - I have confirmed this by ensuring the IPCM server and IPCM softphone has these options

Thanks
I guess you'll have to debug the IPCM server to understand why would it send the disconnection. At this point the server, since it handles the transfer, and the softphone are suscpects, but most likely it is something in the IPCM.

What model is it? Version?
Our IPCM Server is FrontRange IPCM Build 5.8.1.177. I have been in contact with frontrange (the support line for IPCM) and this is their reply. I have attached it to this message- Maybe this will help? What do you think?
IPCM-Call-Transfer-Issue.msg
Sorry, I can't read that fie format
Sorry I hope pdf is fine? Please see attachements

Kind Regards
Microsoft-Outlook---Memo-Style.pdf
Could you please check if the SIP trunk's Callmanager group, contains both 192.168.20.20 and 192.168.20.21? You will have to look at the SIP trunk's Device Pool, then go to the DP and see the CM group, then go into the CM group and check.
Also, we have a mismatch between what they see and what I see. They say UCM disconnects the call, but #1, the images don't show the full trace, and #2, I see a BYE message from IPCM.

Can you gather the packet capture that we talked about from the Callmanager servers as we discussed? Please run the capture on both servers simultaneously. Run the command first, then try the transfer and then stop the capture pressing CTRL-C
I believe it contains both - Please see attached screenshot -
CM.png
I will work on this on Monday - Fingers crossed - If all goes well will post back the capture -

Just another thought I had- If we look at it from there point of view for a moment (as per their email) is it possible that the issue could relate to the contact changes (from 611 to 250)? as per their screen shot in the wireshark capture.

The way this has been configured on our call manager is that anything that hits 250 goes through  the SIP trunk to the IPCM server. This is used for all our 1300 numbers etc for incoming calls.

If you think this is a probable cause do you have any suggestions to test this?

Thanks
Well, again, what they say will have any meaning when we see the type of disconnection sent by CM. So far, the captures do not show CM disconnecting the call, and if that was the case, CM will show a cause code in the BYE message to futher clarify its call processing.
still working on trying to access the SSH credentials - Our "Old" cisco maintainers our not much help. Is there a way of recovering these credentials - I do have an administrative login for the call manager portal - Those credentials don't work with ssh
Hi

I managed to get this debug from the call manager - I hope it helps - It appears to have everything in there - Along with a bunch of other calls for a 3 minute period. The details of the test i made are:

IPCM Softphone call to mobile: 00423246822
Call transfer extn to: 242

Please let me know if you require any more details

Thanks
ccm00000048.txt
Ok, at this point it is a good idea to start using Callmanager traces because I do see a 403 forbidden message for the incoming call to 242. Now, the traces are not showing all the details I would need, could you please verify the trace configuration settings to match the following for both nodes:

http://www.cisco.com/en/US/customer/products/sw/voicesw/ps556/products_tech_note09186a0080094e89.shtml
Will look into it in the next hour and post back. In the capture i posted there was definately a forbidden message?
Thanks
Here is a cleaner version of the capture attached- I have cleaned it up as much as possible to only show the number called and transfer

thanks
testcall.txt
My friend, please if you want my help, follow my advise on what to debug. Please don't clean up the trace, as I need to see all the info before the 403 forbidden to understand why it is generated. Just set up the trace filters as in my link, and reproduce the issue to finally provide the called and calling number. Thats it ;P
My appologies i will do this and post back when im back in the offoce tomorrow morning
Sound good mate.
Hi Mate,

I have gone through the link and made all the requested changes.

Please find attached all callmanager logs exported from both call managers.

The extn called from: 611 (But i believe it gets remasked to show 250 which is the IPCM Server)
Number called: (0)0423246822
Extn call tried to transfer to: 253

Thanks
amc-pub-01-CiscoCallManager-sdi-.zip
amc-pub-01-CiscoCallManager-sdl-.zip
amc-sub-01-CiscoCallManager-sdi-.zip
amc-sub-01-CiscoCallManager-sdl-.zip
Is the IPCM routing in a load balacing fashion? Like for example, is it sending 1 call to .20.21, then the next one to 20.20, and so forth?

I see only 1 call as described before in all 5 SDI files, which makes me think its the one you attempted to transfer. So here it is the analysis:

02/06/2013 09:11:29.711 CCM|//SIP/SIPUdp/wait_UdpDataInd: Incoming SIP UDP message size 1096 from 10.10.20.25:[5060]:
INVITE sip:253@192.168.20.21:5060 SIP/2.0

Then CM processes the call routing:

02/06/2013 09:11:29.714 CCM|Digit analysis: match(pi="2", fqcn="", cn="0611",plv="5", pss="AU-BLOCK-PRE_PT:SYD-SIP-ISDN_PT:PHONE_PT:SYD-VM_PT:SYD-EMERGENCY_PT:SYD-VMPORT_PT", TodFilteredPss="AU-BLOCK-PRE_PT:SYD-SIP-ISDN_PT:PHONE_PT:SYD-VM_PT:SYD-EMERGENCY_PT:SYD-VMPORT_PT", dd="253",dac="0")|
02/06/2013 09:11:29.714 CCM||PretransformCallingPartyNumber=0611
|CallingPartyNumber=0611
|DialingPartition=PHONE_PT
|DialingPattern=253

Finds where to ring the call and sends the RINGING over to IPCM:

02/06/2013 09:11:29.719 CCM|//SIP/SIPUdp/wait_SdlSPISignal: Outgoing SIP UDP message to 10.10.20.25:[5060]:
SIP/2.0 180 Ringing

Cisco phone picks up, and tells IPCM that the audio should be connected to an MTP device with IP 192.168.20.21 (CM) using G.711:

02/06/2013 09:11:32.499 CCM|//SIP/SIPUdp/wait_SdlSPISignal: Outgoing SIP UDP message to 10.10.20.25:[5060]:
SIP/2.0 200 OK
...
o=CiscoSystemsCCM-SIP 2000 1 IN IP4 192.168.20.21
s=SIP Call
c=IN IP4 192.168.20.20
t=0 0
m=audio 24700 RTP/AVP 0 101
a=rtpmap:0 PCMU/8000

IPCM replies corretly:

02/06/2013 09:11:32.505 CCM|//SIP/SIPUdp/wait_UdpDataInd: Incoming SIP UDP message size 576 from 10.10.20.25:[5060]:
ACK sip:253@192.168.20.21:5060 SIP/2.0

And then disconnects the call 100+ miliseconds later:

02/06/2013 09:11:35.617 CCM|//SIP/SIPUdp/wait_UdpDataInd: Incoming SIP UDP message size 364 from 10.10.20.25:[5060]:
BYE sip:253@192.168.20.21:5060 SIP/2.0

This is all in the files you provided, inside the SUB folder, you can find it yourself. That call was disconnected from IPCM. Cisco rang it just fine and even connected it but should have played a busy tone right away after picking it up.

What does the IPCM engineer has to say about this particular disconnection??
Thanks for the info mate,  I have a meeting with them tomorrow morning to discuss further.:( I will bring your findings to their attention and see what they have to say:_
Please just make sure to make them understand they are disconnecting the call, previously the engineer analyzed a call where CM disconnects with a 403 negative reply, but that we cannot see on a CM trace.

Could you please try several transfer attempts in a row with a few seconds between them, and then collect the same logs as before? Maybe we have 2 different disconnections, and if not, you can tell the IPCM people that you debugged 10 consecutive calls and they all disconnected from IPCM.
And if possible ran a packet capture during your tests, it will be easier for them to see the test calls in a capture than in a CM trace.
will definately do so and will post back - Thank you very much for your help
You are very welcome.
Hi Willy

I have spoken to the IPCM techs and apparently they do not know how to "Read" cisco debugs so were not very interested in looking at the call manager logs but the good news is they are still looking at the issue from the IPCM side.

Yesterday, I ran a wireshark capture with them (as per there request) but this time rather than making an outside call (i.e. to a mobile number) and transfer it to a cisco extension, we made a call directly to a users extn and tried to transfer this call to a different extn and experienced different behavior. If you are happy to look at the capture and call manager debugs, i will replicate the issue and post back?

Kind Regards
Actually, I don't feel very comfortable adding difficulties to the mix. Lets fully understand what is going on with the first scenario, and then we can compare to the other scenario, but not before completely comprehending the call flow and its disconnection.

I knew they wouldn't feel comfortable reading CM traces, reason why I suggested taking packet captures from the CM servers themselves. If you have any doubt on how to, please just let me know, but we gotta get familiar with this tool, its by far the best tool I know when troubleshooting networking issues.

Can we do that?? Lets say, run the capture on both CM servers, then reproduce the affected call 3 times in a row. Just as you would with the IPCM engineers, same steps. Stop the  capture and then pull them out. Would you feel ok if we setup a team viewer session later on tonight to go through this??
Hi Mate,

Sorry for the delay in getting back has been pretty hectik here - I am happy to do a teamviewer session -

The engineers got back to me and they had requested to perform a number of tests with them - We tried creating 2 IPCM extns and transferring between both extns with no issues - We also tried transferring a call through the SIP trunk to an external number (as previously tested) and this worked fine - Only issue is the obvious were we cannot transfer a call through the SIP trunk to a cisco extn. Th

they are still looking into this but i am more than happy to run captures -- Only issue is as mentioned previously i do not have the credentials to login throiugh SSH??
you can reset them very easily with physical access to the servers:

http://www.cisco.com/en/US/docs/voice_ip_comm/cucm/cucos/8_0_1/cucos/iptpch2.html#wp1053218
Thanks for the info, Just to clarify will i be resetting the administrator or security password? also will the call manager/s require a reboot? is it safe to do during business hours?
Administrator pass, no reboot, yes its safe

=D
Hi Mate

I have just reset the admin password and ran the util capture command  and uploaded it to wikisend  due to file size - cloudshark only allow upto 10mb.

http://wikisend.com/download/433768/packets.cap

The details of the test are:

Number Called (From IPCM) extn (669): 0423246822
Call Answered
Tried to transfer call from IPCM (extn 669) to a cisco phone (9)204

Thanks
Please tell me you got the CM traces from 20.20!

So far we can say that the test calls you are showing me, when directed to 20.21, are disconnected by the IPCM server, but when directed to 20.20, are disconnected by CUCM with the 403 Forbidden.

Can you please check if the SIP trunk has a SIP Security Profile with security settings on? Maybe a screenshot will be best.
Please download the updated capture from

http://wikisend.com/download/433768/packets.cap

the one i posted earlier wasnt "complete'

I did get the capture from 20.20

Thanks
Please find screenshot below of SIP trunk security settings for IPCM
SIPSecrityscreenshot.jpg
Yes, but did you get the CM traces ?? Remember we have had mismatches between what the other engineer was seeing and what we were seeing ? Now we agree on a disconnection scenario, so we need raw CM traces to see why the 403 message. Please also check the my other recommendation.
Yes thats correct i did get these from the call manager traces - were you able to view the screenshot for the sip security profile - What else would you send me to send through?
Yeah ok... lets see CM Traces for that call you captured on sniffer traces. The whole point here, and why we are jumping from sniffer traces to CM traces, is that some of the calls you have showed to me are failing 1 way, but others way another way. Now that we agree on the 403 disconnection, lets see why from the CM traces.

DId you see where I point out that some calls directed to 20.20 are disconnected by the IPCM server?
Yup I saw it, looks unsecure enough.... SHouldnt be failing cuz of that. Could you please go to Unified Reporting > Database Status report and generate that one and send a screenshot to me please?
It is too long to screenshot - i downloaded it as an xml - hope it is the right one

Thanks
UnifiedCMDatabaseStatus.xml
<comment style="good">All servers have a good replication status.</comment>

Thats good now lets the CM traces please SDI logs from 20.20 from around the time you made the last test.
sorry mate, how do i get these files?
oh you mean the call manager logs no probs just a sec
I went through the trace files and found the 403. I can't see why CM generates that event. There are not enough debug entries. I wonder if the filters are all turned on for that specific server. It was the 20.21 by the way.
I will go through and doible check. Should I debug both servers?
Yes please.
I have rerun the capture this morning on both Call Managers and uploaded to CloudShark

 (192.168.20.20)
http://cloudshark.org/captures/a2205a0464a2

(192.168.20.21)
http://cloudshark.org/captures/2c42b7a3db8b

I ran the capture using the following command on both servers:
utils network capture eth0 file packets count 100000 size all

The details of both test calls are:

Capture Started
Call made from IPCM Softphone (extn 611) To Mobile 0423246822
Tried to transfer call from IPCM Softphone (Extn 611) to Cisco Extn (9)253
Mobile call was automatically dropped
Capture stopped.


Thanks
Hey mate, great info. You did check the trace filter settings right? Made sure they were all turned on and set to detail lvel?
My apologies i did set it to high and select but must of not saved - I will rerun this and resend the logs. Sorry about that mate
Please find new logs attached wit updated logging

Details of call

Call made from extn 611 (IPCM)  to mobile 042326822
Mobile call answered
Attempted to transfer call to cisco extn 211
Mobile call immediately disconnected

Sorry for any inconvenience
amc-pub-01-CiscoCallManager-sdi-.zip
amc-pub-01-CiscoCallManager-sdl-.zip
amc-sub-01-CiscoCallManager-sdi-.zip
amc-sub-01-CiscoCallManager-sdl-.zip
ASKER CERTIFIED SOLUTION
Avatar of ahmzb1990
ahmzb1990

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
What was the solution? Was it an IPCM issue or a Cisco issue?
resolved