Cisco IP Phones Reset Randomly!!
Hello,
One of my clients is having a problem that is proving very difficult to track down. They have an inhouse VoIP solution using Cisco UC540 and SPA525G2 phones. The switch is a Cisco Cmall Business SF 300 48 port for PoE. The problem is that phones randomly reset dropping any active calls. They do not seem to reboot however from a power issue considering the phone reboot history is not showing the events and their uptime shows months since the last reboot. The power consumption on the switch is well below concerning and shows no errors. What does happen is that the phone re-registers. Here is a snippet of the log from the UC540 right after one such event:
13339: Jan 8 18:54:40.286: %IPPHONE-6-REG_ALARM: Name=SEP708105B39B90 Load=7.5.1a Last=Hard+Unknown
013340: Jan 8 18:54:40.598: %IPPHONE-6-UNREGISTER_ABNO
013341: Jan 8 18:54:40.598: %IPPHONE-6-REGISTER: ephone-4:SEP708105B39B90 IP:10.1.1.15 Socket:13 DeviceType:Phone has registered.
013344: Jan 8 18:55:52.418: %IPPHONE-6-REG_ALARM: Name=SEP708105B39D50 Load=7.5.1a Last=Hard+Unknown
013345: Jan 8 18:55:52.626: %IPPHONE-6-UNREGISTER_ABNO
I am really getting heat to address the issue but I don't have much more to go on than this log. The frustrating thing is that it happens randomly and it normally is just one phone at a time. I have found one similar post about disabling proxy arp on our firewall (ASA 5505) which I am tempted to do but I am not sure if this will affect our NAT or ACLs on the firewall.
https://supportforums.cisco.com/thread/2031857
Anyone have any possible solutions or leads?
One of my clients is having a problem that is proving very difficult to track down. They have an inhouse VoIP solution using Cisco UC540 and SPA525G2 phones. The switch is a Cisco Cmall Business SF 300 48 port for PoE. The problem is that phones randomly reset dropping any active calls. They do not seem to reboot however from a power issue considering the phone reboot history is not showing the events and their uptime shows months since the last reboot. The power consumption on the switch is well below concerning and shows no errors. What does happen is that the phone re-registers. Here is a snippet of the log from the UC540 right after one such event:
13339: Jan 8 18:54:40.286: %IPPHONE-6-REG_ALARM: Name=SEP708105B39B90 Load=7.5.1a Last=Hard+Unknown
013340: Jan 8 18:54:40.598: %IPPHONE-6-UNREGISTER_ABNO
013341: Jan 8 18:54:40.598: %IPPHONE-6-REGISTER: ephone-4:SEP708105B39B90 IP:10.1.1.15 Socket:13 DeviceType:Phone has registered.
013344: Jan 8 18:55:52.418: %IPPHONE-6-REG_ALARM: Name=SEP708105B39D50 Load=7.5.1a Last=Hard+Unknown
013345: Jan 8 18:55:52.626: %IPPHONE-6-UNREGISTER_ABNO
I am really getting heat to address the issue but I don't have much more to go on than this log. The frustrating thing is that it happens randomly and it normally is just one phone at a time. I have found one similar post about disabling proxy arp on our firewall (ASA 5505) which I am tempted to do but I am not sure if this will affect our NAT or ACLs on the firewall.
https://supportforums.cisco.com/thread/2031857
Anyone have any possible solutions or leads?
rauenpc
Cisco TAC. Troubleshooting is very difficult when using cisco small business products because of their GUI focus. I cringe every time I have to work on the smb product line...
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Check this out too.
https://supportforums.cisco.com/thread/2085983
Although mostly talks about teleworkers, maybe those firmware versions will help you out. Looks like they updated the UC560 code to and that seemed to resolve the issue. not sure if it'll apply to you but worth looking into.
https://supportforums.cisco.com/thread/2085983
Although mostly talks about teleworkers, maybe those firmware versions will help you out. Looks like they updated the UC560 code to and that seemed to resolve the issue. not sure if it'll apply to you but worth looking into.
ASKER
Thanks for the responses. The phones are on the same LAN with only two offsite. I moved a few of the phones that have been reported to reset directly to the UC bypassing the switch to see if they still reboot. I will check the link provided and consider firmware updates.
Another thought might be a fluctuation in power I.e. there is a bad/leaky feed/patch cable leading to a drop in power.
Wiggle the patch cable and see if the phone resets.
If you have the option, connect the phones to a different port on the switch.
Switching phones to see whether the issue follows the phone or remains with the run.
Wiggle the patch cable and see if the phone resets.
If you have the option, connect the phones to a different port on the switch.
Switching phones to see whether the issue follows the phone or remains with the run.
ASKER
I really don'tjj think it's a power issue as mentioned considering the phones do not report ever rebooting but rather just re-registering. To debug however I moved a few of the phones that I know have had the issue directly to the UC to rule out the switch as a culprit. I am currently monitoring them and have not received word of a reset yet.
Presumably all your phones have a similar firmware. If you could clone the configuration of the phone with this issue and see if the replacement encounters a similar issue.
ASKER
This just happened at another client who has a similar setup (UC540, SF 300 switch, SPA525G2 phones) however in this case ALL of the phones rebooted. I checked the switch and it shows it has been online for 55 days, the UC shows it has been up for 30 days, and none of the phones show an event in the reboot history. So to me this means they didn't lose power but rather lost ethernet or some other reason for re-registering. This is a HUGE problem!
With the all event, check whether DHCP expiration is what led to this.
Historically do you record what IPs each phone has versus what IP each phone has after the "reset"?
i.e. the DHCP server had issues in confirming the existing IPs or assigning new ones. A loss of IP or assignement of a new one, might be what caused the reset.
Updating the Software/firmware to the most current might be a cure in the latest situation. The former, something else might be going on.
Do you have an option to use SNMP to monitor the UC and the Switch?
Do you have an option to setup an SNMPTRAPD receiver and then configure UC and the switch to trap events?
Historically do you record what IPs each phone has versus what IP each phone has after the "reset"?
i.e. the DHCP server had issues in confirming the existing IPs or assigning new ones. A loss of IP or assignement of a new one, might be what caused the reset.
Updating the Software/firmware to the most current might be a cure in the latest situation. The former, something else might be going on.
Do you have an option to use SNMP to monitor the UC and the Switch?
Do you have an option to setup an SNMPTRAPD receiver and then configure UC and the switch to trap events?
ASKER
Both sites had DHCP leases set to 1 day so I have changed it to infinite for both. No I have not recorded the IPs but thats not a bad idea.
SNMP is an option on both the UC and the switch. I know the UC can do traps but I do not have any experience working with SNMP so not sure what to do to set it up and/or monitor it.
SNMP is an option on both the UC and the switch. I know the UC can do traps but I do not have any experience working with SNMP so not sure what to do to set it up and/or monitor it.
Cacti.net can poll and record certain performance data. Snmptrapd are for specific events. On the switch, you would want an event when a port status link changes.
Is the UC also used to provide PC data networking functionality or the data/voice networks are completely separated?
First thing to chek which I believe was suggested deals with the updates available. The setup is not recent and this began recently?
Is the UC also used to provide PC data networking functionality or the data/voice networks are completely separated?
First thing to chek which I believe was suggested deals with the updates available. The setup is not recent and this began recently?
ASKER
The UC is used as the gateway as each phone is supplying data from its pc port. The phones and data are on separate vlans however. Both sites are relatively new installs but the reboots haven't occurred until the past couple weeks an no updates have been done to prompt it. I will update firmware if thats the route to go next but the firmware used on everything was updated during the install which was two months ago for the second client and 4 months ago for the first.
The issue might be triggered by the PC data rather than phone related.
ASKER
I am not sure what you mean Arnold.
To give you an update, the initial client I reported this issue for has not had any of the reset problems since I moved some of the phones to the UC and off the switch along with some other changes including DHCP lifetime, disabling Green Ethernet on the switch, etc. However the second client had all of their phones reboot twice just yesterday even after these changes have been made.
To give you an update, the initial client I reported this issue for has not had any of the reset problems since I moved some of the phones to the UC and off the switch along with some other changes including DHCP lifetime, disabling Green Ethernet on the switch, etc. However the second client had all of their phones reboot twice just yesterday even after these changes have been made.
Try updating all your firmware.
Also, make sure your not pegging the processor with some debug commands. had that happen to me today... :(
Also, make sure your not pegging the processor with some debug commands. had that happen to me today... :(
ASKER
@agonza07
The processor on the UC I assume?
The processor on the UC I assume?
correct. you can do a show process and it'll show you cpu utilization.
ASKER
Yeah, i don't think that is the case as it happens when I am not onsite and have no debugs going on at the time. Thanks for the input though... i will be updating firmware on the switch tonight and will be taking some of the phones off the switch going directly to the UC540 to see if they still reboot.
Arghhh.
We had this problem. It turns out that we had more phones than licenses in our UC. Believe it or not, once we bought more licenses, our troubles ended. Check out licences VS the number of phones you are trying to use. I know it sounds strange.
We had this problem. It turns out that we had more phones than licenses in our UC. Believe it or not, once we bought more licenses, our troubles ended. Check out licences VS the number of phones you are trying to use. I know it sounds strange.
ASKER
Thanks but we have plenty of licenses as only 20 phones are connected. So far we have not had the resets since the switch firmware upgrade but I have been out of town for a week so I will be checking on things tomorrow.
ASKER
Ultimately it appears that the firmware update on the switch did the trick. Thanks!