fancycarman
asked on
E9003 In ArcServe 2000
We have recently started getting errors on several of our ArvServe servers "E9003 Unable to execute command..., EC=FILE NOT FOUND" when we run Pre/Post commands. We have reviewed file/dir permissions, moved the files to root directories and the ArcServe program folder, checked the databases for corruption, saved off-deleted-recreated the jobs, even replaced the command with built-in Windows files; and ArcServe still generates this error message. The problem spread to another server over the weekend with no apparent cause.
ArcServe has been updated to the latest service pack and is running on Windows 2000 with SP4. Our domain-level 'backup' use has domain administrative rights, and we're not seeing any other issues related to that.
Can anyone suggest why ArcServe is failing to locate (and execute) locally stored files?
We're trying to avoid setting up dozens of MS scheduled tasks to initiate and remote disk snapshots in the interim period until we can get our new Enterprise backup solution (using NetBackup).
ArcServe has been updated to the latest service pack and is running on Windows 2000 with SP4. Our domain-level 'backup' use has domain administrative rights, and we're not seeing any other issues related to that.
Can anyone suggest why ArcServe is failing to locate (and execute) locally stored files?
We're trying to avoid setting up dozens of MS scheduled tasks to initiate and remote disk snapshots in the interim period until we can get our new Enterprise backup solution (using NetBackup).
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
As a test run a test job with a simple pre or post command that is easy to tell if it works such as "mkdir test".
Provide an example
such as the exact command used, file location, contents of file (if batch or script).
Provide an example
such as the exact command used, file location, contents of file (if batch or script).
ASKER
We had tried that before, and I just did again this morning. Made a simple folder (C:\scripts) with pretest.cmd and posttest.cmd files, one that calls Calc and one that opens notepad. Both work from Cmd prompt, but a new ArcServe job with these pre / post commands still fails with the same EC=FILE NOT FOUND message.
Pretest.cmd - Calc.exe
Posttest.cmd - Start Notepad.exe c:\cp.txt
ArcServer Log exerpt
20041020 090346 Run Backup Job Scheduled for 10/20/04 at 8:30 AM.
20041020 090346 457 Run Command: C:\scripts\pretest.cmd.
20041020 090347 457 E9003 Unable to execute command. (CMD=C:\scripts\pretest.cm
20041020 090347 457 Start Backup Operation. (QUEUE=1, JOB=17)
20041020 090347 457 Description: PRE/POST Test (cmd).
20041020 090524 TAPE Tape Engine finished formatting media....
20041020 090541 457 Backup Operation Successful.
20041020 090541 457 Run Command: C:\scripts\posttest.cmd.
20041020 090542 457 E9003 Unable to execute command. (CMD=C:\scripts\posttest.c
I'm considering re-installing SP5 in case one of the DLLs has been corrupted.
Pretest.cmd - Calc.exe
Posttest.cmd - Start Notepad.exe c:\cp.txt
ArcServer Log exerpt
20041020 090346 Run Backup Job Scheduled for 10/20/04 at 8:30 AM.
20041020 090346 457 Run Command: C:\scripts\pretest.cmd.
20041020 090347 457 E9003 Unable to execute command. (CMD=C:\scripts\pretest.cm
20041020 090347 457 Start Backup Operation. (QUEUE=1, JOB=17)
20041020 090347 457 Description: PRE/POST Test (cmd).
20041020 090524 TAPE Tape Engine finished formatting media....
20041020 090541 457 Backup Operation Successful.
20041020 090541 457 Run Command: C:\scripts\posttest.cmd.
20041020 090542 457 E9003 Unable to execute command. (CMD=C:\scripts\posttest.c
I'm considering re-installing SP5 in case one of the DLLs has been corrupted.
Did this work prior to SP5?
ASKER
Yes, the pre and post commands work on all servers after upgrading to SP5. The error just arbitrarily started occurring on three different servers at different times over about a four week period. Once the problem appears, it doesn't clear up.
Of note, there were no unusual messages in the ArcServe logs; only one of three had experienced some database corruption (a regular occurrence on all of our ArcServe servers) during the period between the last successful post command and the first failed pre command. To clear the database corruption, we stop the engine and delete all r*.* files from the \Database folder.
Of note, there were no unusual messages in the ArcServe logs; only one of three had experienced some database corruption (a regular occurrence on all of our ArcServe servers) during the period between the last successful post command and the first failed pre command. To clear the database corruption, we stop the engine and delete all r*.* files from the \Database folder.
Yes the old Raima VLDB is not the best. There are some utilites that can be used to maintain it.
Maybe Filemon will help. Filemon is a free utility from SystemInternals. FileMon monitors and displays file system activity on a system in real-time.
http://www.sysinternals.com/ntw2k/source/filemon.shtml
Setup a test job and submit it on hold.
Run Filemon
Set the job to active
Right after the Pre command fails stop Filemon logging.
It just might be the tool needed.
Maybe Filemon will help. Filemon is a free utility from SystemInternals. FileMon monitors and displays file system activity on a system in real-time.
http://www.sysinternals.com/ntw2k/source/filemon.shtml
Setup a test job and submit it on hold.
Run Filemon
Set the job to active
Right after the Pre command fails stop Filemon logging.
It just might be the tool needed.
ASKER
Filemon - great tool, lots of data. Ran the tool, ran the job, saved the log. Nothing specific jumps out at me so far. Summary of what I've found:
A. Two DLL's are missing from the system - SBSLICFN.DLL and asatgenc.dll (but I cannot locate them on any ArcServe server);
B. Just before entry re: "E9003 Unable to execute command", asrunjob.exe reads the QSYS.QJF file. This file is updated when the System Account is reset;
C. RDS.exe reads the most of the catalog.* files; then
D. ArcServe logs the error message
A. Two DLL's are missing from the system - SBSLICFN.DLL and asatgenc.dll (but I cannot locate them on any ArcServe server);
B. Just before entry re: "E9003 Unable to execute command", asrunjob.exe reads the QSYS.QJF file. This file is updated when the System Account is reset;
C. RDS.exe reads the most of the catalog.* files; then
D. ArcServe logs the error message
sbslicfn.dll - no idea - that might have been from some other application.
asatgenc.dll - is located in the ARCserve directory on my system at least on my system.
qsys.qjf is a control file for the job queue so it makes sense that it would be accessed.
Dissapointing that Filemon did not record the specific file access failure.
Mine is just a basic install of 11.1 so the missing file is not related to any option that I have installed and you don't. We could say that I have it because I'm running 11.1 but then it does not make any sense why your install would make a call for it, if it did not come with 2000. Perhaps there was a problem with the install of SP5. No that does not work either because you said it was working after SP5 was installed. So I don't know why your servers would not have that file.
Pick some other system and do an install of ARCserve there to see if it has that missing file.
asatgenc.dll - is located in the ARCserve directory on my system at least on my system.
qsys.qjf is a control file for the job queue so it makes sense that it would be accessed.
Dissapointing that Filemon did not record the specific file access failure.
Mine is just a basic install of 11.1 so the missing file is not related to any option that I have installed and you don't. We could say that I have it because I'm running 11.1 but then it does not make any sense why your install would make a call for it, if it did not come with 2000. Perhaps there was a problem with the install of SP5. No that does not work either because you said it was working after SP5 was installed. So I don't know why your servers would not have that file.
Pick some other system and do an install of ARCserve there to see if it has that missing file.
ASKER