Hard-drive Restored. But why are records missing from many DataFiles?

Posted on 2006-03-26
Last Modified: 2013-12-23

Hi Guys
I am running a Business-Basic application on Unix SCO Open-System 5.0.5
I had a computer crash 3 weeks ago where data on my hard disk became inaccesible
I contacted a recovery service who charged me a small fortune to try and restore the data within 24hrs.
After 24 hours they I got back the data which they claimed was 100% restored.
On close examination of the datafiles I noticed that many of them showed a very unusual corruption
A random number (5%-15%) of records were completely erased from each datafile.
The key-index was intact, but the data for a number of records showed in a hex-dump as hex nulls ($00$)
the data just disappeared.
the file size remained the same (datafile were not truncated). Once reindexed all datafile were fine, but without
the missing records.
Does anyone know what can cause this situation?
The HD recovery company claimed that this is exactly how they received the Hard-Drive
(I yet have to check that). But I doubt that.
In order for this to happen all these datafiles had to go through some kind of a rewrite process.
My system was crush during a cold Boot. Can this cause that?
My theory is that the data became corrupt because it was restored to a slightly different platform then the
platforn they were originally created in.
Unfortunately my knowledge of computers does not reach this depth and I need help from some of you who undoubtly
know more about these things then I do.

your input is very much appreciated
Question by:dory550
    LVL 38

    Expert Comment

    It depends on what the system was doing when it crashed. If the database application is runing, and someone is updating the database and the system
    crash, it could cause data lost in your database. (the transactions, tmp file,
    data in memory, swap is lost !)

    >>the file size remained the same (datafile were not truncated). Once reindexed all datafile were fine, but without the missing records.

    that's what you need to try when the system crash to fix soem of the database.
    (very common for C-ISAM).

    Author Comment


    Thanks for your response
    Nothing was open when the system crashed. I went through a regular shut-down procedure, had everyone log off  and I shut off the system, when promped it was safe to do so. I switched off the system,  waited  60 seconds and swiched it back on.  The computer switched on it loaded the scsi tape drive  but it could not find the scsi hard drive and promped so on the screen.
    I  tried again but had the same results.
    LVL 38

    Accepted Solution

    You might have hardware problem with the box.

    When your system run on multi user mode, a lot of services are running and
    the database are likely one of the running process. (most system configured
    the database auto start at boot !).

    You can try to boot up your system from the emerency FDs, and mount the HD (or mount the HD to another box) to check it out.

    Have a look at /var/adm/messages file to see if there is any error messages.

    Author Comment


    I am sorry if I did not make myself clear....

    You are on the wrong track!
    I do not need help recovering from a crash.
    I am up and running. I had a crash 3 weeks ago. I do not have any problem now, except the lost data which was already  manually added  (to some extent).
    I am just trying to find out under what circumstance can about 100 datafile located in the same directory become
    partially  corrupt like this.
    My system & data were fine until the cold boot
    Is it possible that I caused that?
    In other words. What would it take to recreate this ?
    Why am I doing this post?
    Good question!
    Because if the "Express Data Recovery"  job  was  mishandled  I can get back my $20000.00
    LVL 38

    Expert Comment

    As I mentioned in my first post, the system crashed could caused problem with your data file. In most of the case, database is runing at startup, regardless you are using
    it or not. I have seen systems with the similar problem, due to power failured or panic in the past. data recovery is a difficult task, and some time it is unrecoverable.

    if you still have a original HD, have the /lost+found dir to see if any thing there, if
    there are some files in the dir, the problem is cause by the system creashed for sure. It might not be the fault of the person who recover the data for you.


    Author Comment


    Again. Thank you for your input.

    Let me shed some more light.
    This application does not have one or two big databases where you create 100 of little tables, reports, forms etc.
    This is an old BASIC application where each table has its own datafile like it was in the old days.  25 years ago when they created it.
    As you see,  there was no Database running in the background.
    It is mostly written in GL2 basic  (if you happened to know the term)  this application has about 600 Basic programs (none was damaged)
    It has  about 500 different datafile (tables?) most of them are sequential files. which  were not effected at all.
    Only structured datafiles (tables), the ones that maintain an index in the fileheader for Random-Access  were effected.
    That is  why I find it so strange.
    That is why I suspected that it was restored in a different disk environment by the recovery people, which may have
    caused this files to be rewritten with slightly different algorithm. hence the lost data.


    Why am I using such an old application?  because it is fully customized.  It does the job and  "if it ain't broken..."

    LVL 6

    Expert Comment

    My $0.02:

    My intial take on it was did the recovery company have the right disk parameters to recover the data, but then you said the programs and some datafiles were recovered intact.  

    Were the tables on a different slice? And have you asked the recovery people exactly how they did recover each type of data?  

    I'm sure this isn't much help, but if you can learn more from the recovery people <innocently>, you might learn enough to negotiate a partial refund of your fee.  If you got most of your data, they'd be entitled to some of that exorbitant fee.

    Good Luck.
    LVL 38

    Expert Comment

    I have done some data recovery for SCO boxes 10 years ago for the clients when I was workibg for a software company. Since I kown  how the
    application works I reindexed all datafile (C-ISAM) before handing back to the
    client, and had no problems.

    You data recovery provider might not know anything about your application at
    all, all they do is try to extract the data files out out your problem HD, sinece
    they don't know your application, they can't make your data.

    I'm not sure $20000 US is too much or not, I'm not in US.

    BTW, any extra char (including null char) get into your datafile, can cause problem, you need to reindexed the file.

    Author Comment


    I thank you guys for your responses

    The Recovery company had all the information.
    I had to fill up a questioner covering every aspect of my system hardware and software.
    The damaged files resided on the same folder as the undamaged ones.
    No datafile was missing they were all there.
    The damaged files caused an error only when a program was attempting to use an index-key which led to nowhere.
    otherwise no error was produced. once re-indexed there were no more errors.  Data ofcourse remained missing.

    So far I did not ask them about the recovery method. That is one of the reasons I have this post here and other places.  

    I was trying to find out what conditions can reproduce this kind of behavior. Most of the response I am receiving here and  other posts deal with ways to correct the problem.


    I think I have now enough data to make my decision, if I should dispute the charges or not.
    Before I end this post.
    I would like to thank all of you guys for your input.

    LVL 38

    Expert Comment

    >>The damaged files caused an error only when a program was attempting >>to use an index-key which led to nowhere.
    >>I was trying to find out what conditions can reproduce this kind of behavior.
    System crash which the index file is open. or maunally edit  the data file Or index file. (null chararters in file could also cause the problem.)

    Featured Post

    Looking for New Ways to Advertise?

    Engage with tech pros in our community with native advertising, as a Vendor Expert, and more.

    Join & Write a Comment

    Don’t let your business fall victim to the coming apocalypse – use our Survival Guide for the Fax Apocalypse to identify the risks and signs of zombie fax activities at your business.
    Data center, now-a-days, is referred as the home of all the advanced technologies. In-fact, most of the businesses are now establishing their entire organizational structure around the IT capabilities.
    After creating this article (, I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…
    This video gives you a great overview about bandwidth monitoring with SNMP and WMI with our network monitoring solution PRTG Network Monitor ( If you're looking for how to monitor bandwidth using netflow or packet s…

    745 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    18 Experts available now in Live!

    Get 1:1 Help Now