dbrennecke
asked on
The BlackBerry Controller service terminated with service-specific error 5003 (0x138B)
Our BES server running with Exchange 2003 stopped sending emails last night.
The event log shows the following error:
The BlackBerry Controller service terminated with service-specific error 5003 (0x138B).
The BlackBerry Controller service is stopped and will not restart (and every time I try to start it, I get the sam eerror message above).
Any iodeas on this?
The event log shows the following error:
The BlackBerry Controller service terminated with service-specific error 5003 (0x138B).
The BlackBerry Controller service is stopped and will not restart (and every time I try to start it, I get the sam eerror message above).
Any iodeas on this?
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Yes.. Running SQL n the server...
And I have a slew of these msgs in the BES log:
[30000] (08/21 16:48:47.825):{0xFB4} [Alarm::ActivateAlarm] Queuing alarm: <N/A> | BlackBerry Controller (Application Event Log on INSITE-ISSVR2) | 08/21/2008 16:48:47 (6FFFC3AD) -> EventID:6fffc3adBlackBerry Agent Controller Stopped
[30000] (08/21 16:48:47.825):{0x8A0} Alarm::ThreadProc: Received an alarm message
And I have a slew of these msgs in the BES log:
[30000] (08/21 16:48:47.825):{0xFB4} [Alarm::ActivateAlarm] Queuing alarm: <N/A> | BlackBerry Controller (Application Event Log on INSITE-ISSVR2) | 08/21/2008 16:48:47 (6FFFC3AD) -> EventID:6fffc3adBlackBerry
[30000] (08/21 16:48:47.825):{0x8A0} Alarm::ThreadProc: Received an alarm message
ASKER
Restarted the server and it resolved itself.... nice.
ASKER
Not the exact solution... But it was all helpful information. Thanks for helping.
I found a solution for this problem on my server. My BES server also happened to be my backup DNS server. For some reason the DNS server service locks out a port that BES needs to start the controller service. Simply STOP the DNS Server service, start the BlackBerry Controller service, then restart the DNS Server service.
The problem disappeared for me instantly.
The problem disappeared for me instantly.
DNS thing worked for me as well.
Thanks
Rob
Thanks
Rob
DNS server service suggestion worked for me as well.
I have had this issue a couple of times and all I had to do is stop the DNS server service and start Blackberry Controller service. Then restart the DNS Server service.
That's really weird, the DNS server service worked for me as well. Thanks for the post!
Stopping DNS, start controller service and then start dns worked for me too. Weird problem as we have restarted this server many many times since Blackberry install.
Thanks for the tip (Experts Exchange is SO worth the money!)
Thanks for the tip (Experts Exchange is SO worth the money!)
Restart dns worked for me.
Mods please change the accepted answer on this issue.
Mods please change the accepted answer on this issue.
I actually made a quick little script using "net stop/start" to fix the Controller service if it fails on the 2nd Attempt. Great find for suggesting stopping the DNS service.
Did the Stop DNS trick, worked for me as well. Thanks William
Stopping the DNS service worked also for me and great idea with the script for 2nd attempt, have that in place now.
EE rocks! and it's members!
EE rocks! and it's members!
I just restarted the server (W2K3) and life was good!
The DNS server fix worked for me as well, simple fix after banging my head against the wall for half a day - the wall thanks you also! :)
Stopping DNS Server, then starting the BlackBerry Controller worked for me as well. I was getting ready to start a long call to RIM. Thank you all very much
Thankyou, the DNS trick got my Controller Service running too - but doesn't the Controller Service just make sure other services are running? ie. if the Controller Service itself is not running it shouldn't affect you getting your emails should it? Apparantly it must tho. I'd already wiped my BB and wiped and recreated my User Account on BPS, now I'm getting "An error occurred. Please contact your System Administrator". Hmmm, wish I read this post first!! Back to the drawing board.
Thanks, the DNS server suggestion worked for a server I was troubleshooting as well.
Funny, all these "thanks" here, to DuoDreamer, and he didn't even get s ingle point for his solution. And we all know only about 1% of people who find a solution that's already been discovered and discussed will comment back on it, so imagine the sheer number of people affected by this BlackBerry bug that DuoDreamer has actually helped!
DuoDreamer found the cause of this bug 8 months ago, yet RIM still have not released a patch for it... It's pretty ironic actually how the sole purpose of the Blackberry Controller Service is purely to check all the key Blackberry Services are running, and start them if they stop, yet a simple DNS bug causes the Controller service to fail, which for some reason also brings down the whole Blackberry service from working. They could just implement cytexone's solution.
Cytexone, could you post the script you wrote? Does the script stop the dns, start the controller, and start the dns, all in the one script?
Also, I was wondering if you made the Controller service a Dependancy of the DNS service, then the DNS service would send a start command to the controller service, before the DNS service started, wouldn't it? If that theory is sound, how do you make one service a dependancy of another?
Cheers guys, Jono.
DuoDreamer found the cause of this bug 8 months ago, yet RIM still have not released a patch for it... It's pretty ironic actually how the sole purpose of the Blackberry Controller Service is purely to check all the key Blackberry Services are running, and start them if they stop, yet a simple DNS bug causes the Controller service to fail, which for some reason also brings down the whole Blackberry service from working. They could just implement cytexone's solution.
Cytexone, could you post the script you wrote? Does the script stop the dns, start the controller, and start the dns, all in the one script?
Also, I was wondering if you made the Controller service a Dependancy of the DNS service, then the DNS service would send a start command to the controller service, before the DNS service started, wouldn't it? If that theory is sound, how do you make one service a dependancy of another?
Cheers guys, Jono.
Good to see some praise for my research. :)
A script to perform the ordered service restarts is as simple as going into the BB Controller service, on the "Recovery" tab you can choose to "Run a Program" on service failure. Simply point this to a batch file that contains
net stop dns
net start bbcontroller
net start dns
I'm pretty sure those service names aren't correct, but you can get the idea from there.
In fact, in the parameters for the "Run a Program" choice, you can even have it pump in the restart attempt count, which you can use to send yourself an email/msg stating it attempted x times.
For those of you with PowerShell experience, there is a ton of neat stuff you can do.
A script to perform the ordered service restarts is as simple as going into the BB Controller service, on the "Recovery" tab you can choose to "Run a Program" on service failure. Simply point this to a batch file that contains
net stop dns
net start bbcontroller
net start dns
I'm pretty sure those service names aren't correct, but you can get the idea from there.
In fact, in the parameters for the "Run a Program" choice, you can even have it pump in the restart attempt count, which you can use to send yourself an email/msg stating it attempted x times.
For those of you with PowerShell experience, there is a ton of neat stuff you can do.
net stop "DNS Server"
net start "BlackBerry Controller"
net start "DNS Server"
net start "BlackBerry Controller"
net start "DNS Server"
Ive recently started having the same issue, and the dns start/stop does not work for me.
Ive also tried the microsoft resolution on reserving the ports via registry and restarting. still no luck.
Any suggestions?
Ive also tried the microsoft resolution on reserving the ports via registry and restarting. still no luck.
Any suggestions?
unashamed:
Try disabling the DNS service and restarting the server. If the BB service fails to start again, it is unrelated to this specific issue.
Try disabling the DNS service and restarting the server. If the BB service fails to start again, it is unrelated to this specific issue.
Thanks very much to DuoDreamer. I spent about 4 solid hours trying to sort out a Blackberry Curve that wouldn't activate ... and then trying to solve why my Blackberry Controller service was stopped - your DNS suggestion fixed it for me straight away :-)
great post about the dns server thanks
Awesome, worked here too!
To ensure that the BlackBerry Controller service starts BEFORE the DNS service when the server is restarted, you can set the DNS service to depend on the BlackBerry Controller service by going to regedit, browsing to the DNS service, and entering BlackBerry Controller as a dependency.
HKEY_LOCAL_MACHINE\SYSTEM\ CurrentCon trolSet\Se rvices\DNS
In the DependOnService value, add a line that says
BlackBerry Controller
HKEY_LOCAL_MACHINE\SYSTEM\
In the DependOnService value, add a line that says
BlackBerry Controller
DNS worked here as well. Thanks for the update and script idea.
Cheers
Cheers
For the idea of making the DNS Service a dependency of the Blackberry Controller Service:
Keep in mind that if you restart/stop the Blackberry Controller service, the DNS Service will restart/stop also.
Keep in mind that if you restart/stop the Blackberry Controller service, the DNS Service will restart/stop also.
Thanks so Much cntdtech. Great Idea.
wow! good find! thanks!
Yep. DNS for me too. I just stopped DNS and started BES services in the right order. FYI, that is: router, controller, dispatcher, policy service, mds connection service, attachment service, sync service, alter.
.......... and yet again, another happy EE customer. Yes, definitely worth the money!
This "fix" has solved my dilemma too, thanks to DuoDreamer. And kudos to cntdtech for the dependency tip! Great work guys!
This "fix" has solved my dilemma too, thanks to DuoDreamer. And kudos to cntdtech for the dependency tip! Great work guys!
You can also use the follow fix:
HKEY_LOCAL_MACHINE\SYSTEM\ CurrentCon trolSet\Se rvices\Tcp ip\Paramet ers
On the Edit menu, point to New, and then click Multi-string Value.
Right-click the new value, click Rename, type ReservedPorts, and then press ENTER.
Double-click the ReservedPorts value, type the range of ports that you want to reserve(4070-4070 and 4071-4071) in this case , and then click OK.
HKEY_LOCAL_MACHINE\SYSTEM\
On the Edit menu, point to New, and then click Multi-string Value.
Right-click the new value, click Rename, type ReservedPorts, and then press ENTER.
Double-click the ReservedPorts value, type the range of ports that you want to reserve(4070-4070 and 4071-4071) in this case , and then click OK.
Great - thanks woodall01!!!
Finally a solution rather than a workaround - although DuoDreamer's and cytexone's suggestion of creating a .bat script to automate the workaround have been a fantastic short-gap solution for well over a year since RIM have never acknowledged their bug let alone released a patch for it.
So essentially the BB Controller needs ports 4070 and 4071 reserved from the DNS Server to work, so this regedit fix should permanantly resolve the issue without needing to stop/start services.
Note - I assume you could just add one line - "4070-4071" as the "x-y" is a range.
Also anyone else adding this String - check to see if there is a ReservedPorts string already otherwise you'll get a "The Specified Value Name already exists" error. So just add the 2 ports to the end of the existing string. (It existed on all 3 servers that I checked after getting this wonderful post by woodall01)
Finally a solution rather than a workaround - although DuoDreamer's and cytexone's suggestion of creating a .bat script to automate the workaround have been a fantastic short-gap solution for well over a year since RIM have never acknowledged their bug let alone released a patch for it.
So essentially the BB Controller needs ports 4070 and 4071 reserved from the DNS Server to work, so this regedit fix should permanantly resolve the issue without needing to stop/start services.
Note - I assume you could just add one line - "4070-4071" as the "x-y" is a range.
Also anyone else adding this String - check to see if there is a ReservedPorts string already otherwise you'll get a "The Specified Value Name already exists" error. So just add the 2 ports to the end of the existing string. (It existed on all 3 servers that I checked after getting this wonderful post by woodall01)
Thanks again guys - the workaround was really useful but the 'ReservedPorts' idea is perfect! Now I can remotely reboot clients' servers and just walk away, without having to keep my fingers crossed that the services will start properly the next time Windows loads.
EVEN MORE IMPORTANTLY than fixing the Blackberry Controller service, this registry update can also fix a bug in Small Business Server 2003 (and probably Windows Server 2003) whereby the IPSEC service might fail to start for exactly the same reason. This bug was introduced by a Windows Update! It's an intermittent problem but, when it occurs, the server will block ALL incoming and outgoing communications, meaning that client PCs can't log on to the network and you can't remotely control the server to troubleshoot it because it won't allow Internet traffic so you MUST go on-site to fix it straight away - what a nightmare!
I've made the changes to my registry and exported the registry key. If you copy the following lines into Notepad and save it as a .REG file, you have a one-click fix which will automatically make the changes for you. I've done this so that I can very quickly and easily apply the fix to all new Windows Server 2003 / Small Business Server 2003 servers that we install for clients.
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SYSTEM \CurrentCo ntrolSet\S ervices\Tc pip\Parame ters]
"ReservedPorts"=hex(7):31, 00,34,00,3 3,00,33,00 ,2d,00,31, 00,34,00,3 3,00,34,00 ,\
00,00,33,00,33,00,34,00,33 ,00,00,00, 34,00,30,0 0,37,00,30 ,00,2d,00, 34,00,30,\
00,37,00,31,00,00,00,34,00 ,35,00,30, 00,30,00,0 0,00,00,00
The exported registry value is in HEX, which is a bit annoying, but I can tell you that I've reserved the following ports: 1433-1434 for SQL Server, 3343 for Cluster Services (don't know why but it was already in my SBS2003 registry!), 4070-4071 for Blackberry Controller and 4500 for IPSEC. If you don't trust me (why would you?!), you can use these suggestions to manually update your own server's registry in Regedit and then, from the File menu, export the specific registry section to a .REG text file.
I've found this EE thread extremely useful so I hope that this helps to give something back to the community :-)
EVEN MORE IMPORTANTLY than fixing the Blackberry Controller service, this registry update can also fix a bug in Small Business Server 2003 (and probably Windows Server 2003) whereby the IPSEC service might fail to start for exactly the same reason. This bug was introduced by a Windows Update! It's an intermittent problem but, when it occurs, the server will block ALL incoming and outgoing communications, meaning that client PCs can't log on to the network and you can't remotely control the server to troubleshoot it because it won't allow Internet traffic so you MUST go on-site to fix it straight away - what a nightmare!
I've made the changes to my registry and exported the registry key. If you copy the following lines into Notepad and save it as a .REG file, you have a one-click fix which will automatically make the changes for you. I've done this so that I can very quickly and easily apply the fix to all new Windows Server 2003 / Small Business Server 2003 servers that we install for clients.
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SYSTEM
"ReservedPorts"=hex(7):31,
00,00,33,00,33,00,34,00,33
00,37,00,31,00,00,00,34,00
The exported registry value is in HEX, which is a bit annoying, but I can tell you that I've reserved the following ports: 1433-1434 for SQL Server, 3343 for Cluster Services (don't know why but it was already in my SBS2003 registry!), 4070-4071 for Blackberry Controller and 4500 for IPSEC. If you don't trust me (why would you?!), you can use these suggestions to manually update your own server's registry in Regedit and then, from the File menu, export the specific registry section to a .REG text file.
I've found this EE thread extremely useful so I hope that this helps to give something back to the community :-)
Any way we can give points to DuoDreamer for such a great solution / find?
This worked perfectly, thanks Mate!
This worked perfectly, thanks Mate!
http://www.blackberry.com/btsc/search.do?cmd=displayKC&docType=kc&externalId=KB16001
worked for me. stop DNS, start controller, start DNS again.
worked for me. stop DNS, start controller, start DNS again.
DNS trick worked for me also!
Thank you sir! Give um some extra points or something!
Thank you sir! Give um some extra points or something!
this solution worked for me also. SBS 2003 Standard box with Blackberry Professional Server Express - this just happened at night without rebooting my server - the controller service stopping. Thanks to DuoDreamer.
Amazing! DNS server fix worked for me as well! Thanks!
Wonderful solution.
This was a headache for a full day!
Thanks to all.
BTW, the solution to reserver the ports works great.
This was a headache for a full day!
Thanks to all.
BTW, the solution to reserver the ports works great.
DNS restart fixed it for me also
The DNS fixed mine as well.
The DNS worked for me as well, many, many thanks, also used your script.
That fixed me also. Thx!
Stopping and Starting DNS Server worked for me, if it happens again i will take a closer look at the reserved ports. This problem seems to relate to IPSec services
The DNS thing is a good work around but it takes another 5 minutes to actually resolve the problem for good
http://www.blackberry.com/btsc/microsites/search.do?cmd=displayKC&docType=kc&externalId=KB16001&sliceId=1&docTypeID=DT_SUPPORTISSUE_1_1&dialogID=3184111&stateId=wrong%20Id
http://www.blackberry.com/btsc/microsites/search.do?cmd=displayKC&docType=kc&externalId=KB16001&sliceId=1&docTypeID=DT_SUPPORTISSUE_1_1&dialogID=3184111&stateId=wrong%20Id
Are you running SQL on the same server? If so try here:
http://www.blackberry.com/btsc/dynamickc.do?externalId=KB03806&sliceId=SAL_Public&command=show&forward=nonthreadedKC&kcId=KB03806
Here's MS description of the error.
http://msdn.microsoft.com/en-us/library/ms681387(VS.85).aspx