Hard-drive Restored. But why are records missing from many DataFiles?

Hi Guys
I am running a Business-Basic application on Unix SCO Open-System 5.0.5
I had a computer crash 3 weeks ago where data on my hard disk became inaccesible
I contacted a recovery service who charged me a small fortune to try and restore the data within 24hrs.
After 24 hours they I got back the data which they claimed was 100% restored.
On close examination of the datafiles I noticed that many of them showed a very unusual corruption
A random number (5%-15%) of records were completely erased from each datafile.
The key-index was intact, but the data for a number of records showed in a hex-dump as hex nulls ($00$)
the data just disappeared.
the file size remained the same (datafile were not truncated). Once reindexed all datafile were fine, but without
the missing records.
Does anyone know what can cause this situation?
The HD recovery company claimed that this is exactly how they received the Hard-Drive
(I yet have to check that). But I doubt that.
In order for this to happen all these datafiles had to go through some kind of a rewrite process.
My system was crush during a cold Boot. Can this cause that?
My theory is that the data became corrupt because it was restored to a slightly different platform then the
platforn they were originally created in.
Unfortunately my knowledge of computers does not reach this depth and I need help from some of you who undoubtly
know more about these things then I do.

your input is very much appreciated
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

It depends on what the system was doing when it crashed. If the database application is runing, and someone is updating the database and the system
crash, it could cause data lost in your database. (the transactions, tmp file,
data in memory, swap is lost !)

>>the file size remained the same (datafile were not truncated). Once reindexed all datafile were fine, but without the missing records.

that's what you need to try when the system crash to fix soem of the database.
(very common for C-ISAM).
dory550Author Commented:

Thanks for your response
Nothing was open when the system crashed. I went through a regular shut-down procedure, had everyone log off  and I shut off the system, when promped it was safe to do so. I switched off the system,  waited  60 seconds and swiched it back on.  The computer switched on it loaded the scsi tape drive  but it could not find the scsi hard drive and promped so on the screen.
I  tried again but had the same results.
You might have hardware problem with the box.

When your system run on multi user mode, a lot of services are running and
the database are likely one of the running process. (most system configured
the database auto start at boot !).

You can try to boot up your system from the emerency FDs, and mount the HD (or mount the HD to another box) to check it out.

Have a look at /var/adm/messages file to see if there is any error messages.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Ultimate Tool Kit for Technology Solution Provider

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.

dory550Author Commented:

I am sorry if I did not make myself clear....

You are on the wrong track!
I do not need help recovering from a crash.
I am up and running. I had a crash 3 weeks ago. I do not have any problem now, except the lost data which was already  manually added  (to some extent).
I am just trying to find out under what circumstance can about 100 datafile located in the same directory become
partially  corrupt like this.
My system & data were fine until the cold boot
Is it possible that I caused that?
In other words. What would it take to recreate this ?
Why am I doing this post?
Good question!
Because if the "Express Data Recovery"  job  was  mishandled  I can get back my $20000.00
As I mentioned in my first post, the system crashed could caused problem with your data file. In most of the case, database is runing at startup, regardless you are using
it or not. I have seen systems with the similar problem, due to power failured or panic in the past. data recovery is a difficult task, and some time it is unrecoverable.

if you still have a original HD, have the /lost+found dir to see if any thing there, if
there are some files in the dir, the problem is cause by the system creashed for sure. It might not be the fault of the person who recover the data for you.

dory550Author Commented:

Again. Thank you for your input.

Let me shed some more light.
This application does not have one or two big databases where you create 100 of little tables, reports, forms etc.
This is an old BASIC application where each table has its own datafile like it was in the old days.  25 years ago when they created it.
As you see,  there was no Database running in the background.
It is mostly written in GL2 basic  (if you happened to know the term)  this application has about 600 Basic programs (none was damaged)
It has  about 500 different datafile (tables?) most of them are sequential files. which  were not effected at all.
Only structured datafiles (tables), the ones that maintain an index in the fileheader for Random-Access  were effected.
That is  why I find it so strange.
That is why I suspected that it was restored in a different disk environment by the recovery people, which may have
caused this files to be rewritten with slightly different algorithm. hence the lost data.


Why am I using such an old application?  because it is fully customized.  It does the job and  "if it ain't broken..."

My $0.02:

My intial take on it was did the recovery company have the right disk parameters to recover the data, but then you said the programs and some datafiles were recovered intact.  

Were the tables on a different slice? And have you asked the recovery people exactly how they did recover each type of data?  

I'm sure this isn't much help, but if you can learn more from the recovery people <innocently>, you might learn enough to negotiate a partial refund of your fee.  If you got most of your data, they'd be entitled to some of that exorbitant fee.

Good Luck.
I have done some data recovery for SCO boxes 10 years ago for the clients when I was workibg for a software company. Since I kown  how the
application works I reindexed all datafile (C-ISAM) before handing back to the
client, and had no problems.

You data recovery provider might not know anything about your application at
all, all they do is try to extract the data files out out your problem HD, sinece
they don't know your application, they can't make your data.

I'm not sure $20000 US is too much or not, I'm not in US.

BTW, any extra char (including null char) get into your datafile, can cause problem, you need to reindexed the file.
dory550Author Commented:

I thank you guys for your responses

The Recovery company had all the information.
I had to fill up a questioner covering every aspect of my system hardware and software.
The damaged files resided on the same folder as the undamaged ones.
No datafile was missing they were all there.
The damaged files caused an error only when a program was attempting to use an index-key which led to nowhere.
otherwise no error was produced. once re-indexed there were no more errors.  Data ofcourse remained missing.

So far I did not ask them about the recovery method. That is one of the reasons I have this post here and other places.  

I was trying to find out what conditions can reproduce this kind of behavior. Most of the response I am receiving here and  other posts deal with ways to correct the problem.


I think I have now enough data to make my decision, if I should dispute the charges or not.
Before I end this post.
I would like to thank all of you guys for your input.

>>The damaged files caused an error only when a program was attempting >>to use an index-key which led to nowhere.
>>I was trying to find out what conditions can reproduce this kind of behavior.
System crash which the index file is open. or maunally edit  the data file Or index file. (null chararters in file could also cause the problem.)
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.