Link to home
Start Free TrialLog in
Avatar of BuildingITC
BuildingITC

asked on

Veritas 9.0 - The Backup Exec job engine system service is not responding.

Please help!  We have done everything short of reinstalling.  

Scheduled backups fail intermittently with the following alert:

Backup Daily -- The job failed with the following error: The Backup Exec job engine system service is not responding.

We have seen other alerts that state that the hardware has become unavailable or the job was canceled by the NT System Authority.

We can clearly see that the services are running so this is puzzling.  We can stop and restart the services no problem and the same issue will reoccur.  We only have one logon account which is the System Logon Account using the administrator credentials which are not restricted.

We have a new Dell PowerEdge 2350 running Windows 2000 Server, Exchange 2000, Symantec Corp Ed Anti-Virus, and Filemaker Pro.  We have a Quantum VS80 external tape.  There are two backup jobs scheduled - Backup Daily which is an incremental that occurs Mon, Tues, Wed and Fri & Backup Full which is a full backup that occurs on Thursday evenings.  We have about 50GB of data.

If the backup is manually run (right click job and select Run Now) most of the time it will run successfully and on a rare occasion the scheduled backup will run successfully as well.  That is why this has been so difficult to troubleshoot.

Running the diag utility the only errors I can find are the following which do not tell me much:

[0776] 10/06/03 21:14:18 PvlMoverSession::Release()
       Job = {A7376CC4-CB49-4FC4-A15C-87B0BABAFF9A} "Backup Daily"
       Drive  = {6BECD4B1-8A71-4452-8963-14A2F8A8AC53} "BNCHMARK 1"
       Media  = {A410472F-81D5-4F2F-A4AF-2B5512AB9F08} "Monday"
       Error: The session has been prematurely released!

[6188] 10/08/03 21:22:06 PvlMoverSession::Release()
       Job = {A7376CC4-CB49-4FC4-A15C-87B0BABAFF9A} "Backup Daily"
       Drive  = {6BECD4B1-8A71-4452-8963-14A2F8A8AC53} "BNCHMARK 1"
       Media  = {379B5A47-332D-4330-90D5-EB22B7DFEB42} "Wednesday"
       Error: The session has been prematurely released!

[3212] 10/09/03 08:39:38 Adamm Log Stopped!
Avatar of chicagoan
chicagoan
Flag of United States of America image

Try Setting all the backup services to manual, removing the VS80, reboot and let the Operating System re-discover it. Run the Quantum diagnostics. If the drive checks out restart your backup services and see where you are.  

Check to see if Removable Media Manager Service is running, is was a problem with the previous version of Veritas and some drives.

ALso check you event logs for SCSI errors.
Avatar of BuildingITC
BuildingITC

ASKER

I checked and the "Removable Storage" service was running so I disabled it and set it to manual.  I will run a test backup to see what happens.

We will not be able to take the server down until this evening so I will try the other recomendations then and report our findings.  Thanks for your guidance.
Not sure about the Removable Storage Service on this version, ut I'd try it both ways if nothing else (like DFS) needs it.
It failed with the same error.  So for grins I stopped all the services through the Backup Exec Services Manager and then restarted them.  Then I ran the same backup job and it appears to be running.  The problem is that when the scheduled job is due I am confident it will fail with the same error.  It's almost like I get one use after stopping and starting the services.  I will test this theory and report back.
That job Completed with exceptions.  See below.

Job name             : Backup Daily
Job type               : Backup
Job status             : Completed with exceptions
Job log                 : C:\Program Files\VERITAS\Backup Exec\NT\Data\BEX00421.xml
Server name         : SERVER1
Selection list name : Backup Daily
Device name         : BNCHMARK 1
Target name         : All Drives (SERVER1)
Media set name     : Daily
Okay, that theory was correct.

Once that job completed successfully the tape ejected.  I inserted a new tape and ran an inventory job which ran fine.

I ran the same daily backup job and it failed with the same error.  I restarted the services then ran the job again and it ran fine.

I am only getting one use out of the services.  Very strange!  Help!
Perhaps the job is corrupted in the queue.
Can you run multiple manual jobs after starting the services?
The jobs that I am running by right clicking and selecting Run Now are all scheduled queued jobs.

I am on hold with Veritas right now to find out if I can get support from them.  Seems like the Microsoft support process so I am not positive calling them will help.
I sent them an email.  We will see what they come back with.  I created a new job just to rule out the corrupt data idea and and running it now.  Just want to add that the job does not fail immediately... it starts to run the backup scan for about 3 minutes before it fails.  Thanks again for your help.
Avatar of Duncan Meyers
This sounds very similar to a problem I had with BAckup Exec 9 - Jobs hang in pre-processing. Check tech-note ID 255242 at http://seer.support.veritas.com/docs/255242.htm. Its a simple registry fix: "The workaround for this issue is to disable the creation of the DR file by modifying the registry key: HKey_Local_Machine\Software\Veritas\Backup Exec\Engine\Misc\Disaster Recovery Installed. The value of this key must be modified from 1 to 0."


Hope this helps,

Duncan
Duncan,

Regretfully this does not apply as we are running 9.0 rev 4454 which address that issue.  Thanks for the info though.  Veritas replied recommending that I download and install the latest version of the MDAC from the Microsoft Website.  I installed 2.8 and rebooted the server.  We will see if this resolves the issue.  I doubt it but we will see.  Any other suggestions are very welcome.

Thanks!
Thanks Duncan that solved my issue at least.  This site (and the people) rock!

Brian
Brian, please come back to experts exchange and let us know if your problem was solved by closing your question or posting further comments.
This is not my question.  The registry fix Duncan posted solved my problem.  Since it was already here I did not have to ask a question.  Thanks!
Works for me...

Duncan
BuildingITC,

Did you ever get a resolution to your problem?  I got the same suggestion from Veritas about the MDAC 2.8 but after installing and rebooting we are still experiencing the exact same symptoms as you.  Let me kow if you have found resolution please.

Thanks,
Jeff
Yes - After going through and backing up each directory and mailbox ONE BY ONE we found that there was a CORRUPT DELETED FILES in one of the users mailboxs that was causing it to hang.  Veritas WAS NO HELP and it took us over 30 hours to finally resolve this issue (we where dealing with 80GB of data).  We did not bill the customer as it should have worked out of the box.  Live and learn.  Hope this helps.
I have discovered a bug in BEX v9.1 (HF-11) which causes the Job Engine (bengine.exe) to fail on the SECOND backup operation since the last time that the BEX services were restarted.  My backups are all local drives beng backed up onto another local disk volume, the OS is Windows XP.

The bug is concerned with the method chosen to back up the files.  If the chosen method is to use file date/time stamps, then the the Job Engine fails consistantly on the second backup operation.  If, however, the chosen method is to use the archive bit, then the Job Engine never fails and every backup works perfectly - even when running several at the same time.
ASKER CERTIFIED SOLUTION
Avatar of PashaMod
PashaMod

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Just wondered if anybody else has had this problem?

I've looked throught this thread and have updated my MDAC to 2.8 and tried all sorts of things. I'm basially getting the same problem as the original guy. The only differences are that the backup job which it fails on never even starts. No data is backed up and then all subsequent jobs fail, including that one. When I look at the diagnostics file I get basically the same printout as Building ITC in the original post.

[2256] 12/16/04 10:33:06 Device Discovery End:

[0544] 12/16/04 22:27:29 PvlMoverSession::Release()
       Job = {B3B51E4B-639E-4D90-8367-526335A3DBAF} "GBWAN001_Data"
       Drive  = {F4FA2EB3-5D47-4A37-8B3C-5F6FC141B6BA} "Drive 1"
       Media  = {848F2788-70B7-4689-AB2D-347380D4E2AD} "000009"
       Error: The session has been prematurely released!

[0544] 12/16/04 22:27:29 PvlMoverSession::Release()
       Job = {A26ED8A2-8F04-4B1F-877F-722B1F83442A} "GBWAN002_Data"
       Drive  = {7A705BAB-5203-4931-873A-E7AAE8A25F8E} "Drive 2"
       Media  = {F752EBD9-663E-49C6-B259-4F7F21D75700} "000032"
       Error: The session has been prematurely released!

The system event log entry for that time and date is:

The Backup Exec Job Engine service terminated unexpectedly.  It has done this 1 time(s).  The following corrective action will be taken in 0 milliseconds: Restart the service.


[1028] 12/20/04 22:28:07 PvlMoverSession::Release()
       Job = {A26ED8A2-8F04-4B1F-877F-722B1F83442A} "GBWAN002_Data"
       Drive  = {7A705BAB-5203-4931-873A-E7AAE8A25F8E} "Drive 2"
       Media  = {3467AA8D-8462-47BE-A8AF-2531BBA992D7} "000034"
       Error: The session has been prematurely released!

[1028] 12/20/04 22:28:07 PvlMoverSession::Release()
       Job = {B3B51E4B-639E-4D90-8367-526335A3DBAF} "GBWAN001_Data"
       Drive  = {F4FA2EB3-5D47-4A37-8B3C-5F6FC141B6BA} "Drive 1"
       Media  = {B2748AC5-ED96-44B2-B99B-96F51AE94D25} "000011"
       Error: The session has been prematurely released!

The system event log entry for that time and date is:

The Backup Exec Job Engine service terminated unexpectedly.  It has done this 1 time(s).  The following corrective action will be taken in 0 milliseconds: Restart the service.


The reason the service restarts itself is that was one of my attempted 'fixes' I tried - to set the service to immediately restart upon stopping. But this doesn't really work as BackupExec never continues working after the service is restarted. All the jobs fail.

Yes, I have noticed that the system fails on the same jobs - GBWAN001_Data and GBWAN002_Data but I cannot imagine that these are causing it because those backups are no diffrent to any others (they have pre/post jobs commands but so do many of our other backups) that we run. About 15 backup jobs run and get spanned across both drives in the library. Bit it's only at 22:2x that the service fails and the backup jobs fail

Any advice anybody could offer would be greatly appreciated.

I have Veritas v.9.1 4691
I have a Dell PowerEdge 1650 running W2K SP3
My Backup Library is a Powervault 132T With 2 LTO2 Drives

and my head is banging!

Cheers

Matt


I had a very similar problem, however I was able to move the backup software and drive to a new server that I had just rebuilt.  Good luck in finding a solution.