Solved

QNAP NAS files corrupted and have no extension

Posted on 2015-01-12
4
602 Views
Last Modified: 2015-01-19
One of my clients has a QNAP 2-bay NAS that they store files on. They have 2x disks in a RAID1, it's an ext4 partition, shared using AFP with a couple of iMacs running Mac OSX Snow Leopard. They do backups to external hard disk from the NAS and rotate several disks.

Client accidentally pulled both hard disks from the NAS while it was running. Don't ask. The RAID1 array of course was destroyed.  We opted to purchase 2x new disks and restore from backup (the new disks are enterprise disks, the old ones were junky little consider WD Black drives so it was just an opportunity to do the upgrade), and preserve the original disks in case we need to attempt some kind of re-assembly of the RAID array.

We've suddenly discovered that MANY, but not all of the files on the NAS have no extension, and they are unreadable. I tried attaching a single member disk from the RAID1 array to read it's contents, and the same files have no extension on the member disk, either.

Looking in our backups going all the way back to August 2014, the files seem to be this way as well.

Most of the files were originally InDesign, Photoshop, PDFs, Word documents, and TTF font files.

It seems to me like this has been ongoing for a long time, but even very recent files that the designer was working on just before the Christmas holidays appear corrupted, and the designer swears up and down that those files were readable prior to the incident with the RAID1 disks getting yanked.

The files seem to be the right size and when I open the document in a hex editor I can see that the contents does look to be intact so far as I can tell. Obviously something is still wrong with the content of the file, though.  

We made some attempts to rename the file and put the correct extension back (e.g. putting a .docx extension onto a file that we knew was originally a document), but the programs stil couldn't open the file, Word thought the Doc file was corrupted. Although to be honest I don't really trust the designer's word for what format the various files were in.

Has anybody heard of this happening before? Any ideas what we can do to maybe figure out what happened, or analyze the files to see if there's a way to make them usable again?

Reading a bit on the Internet it seems that Mac computers sometimes neglect to put extensions on files and instead put marker data inside the file that identifies it's contents, but I'm not really sure how that stuff works.  Maybe someone can fill me in.


--- Edit ---

I examined the beginning of a couple of the files in a hex editor. Screenshots are attached. Maybe somebody else can make heads or tails of what the format of these files could be so that I can check if they are intact?

hex editor display of second file
hex editor display of first file
0
Comment
Question by:Frosty555
  • 2
4 Comments
 
LVL 47

Expert Comment

by:dlethe
ID: 40545947
You need a pro to take images of everything and reconstruct. Too many things have gone wrong, starting with the customer's actions.    The backups have problems and no doubt they've never done a restore to verify they're any good.

Call your favorite disaster recovery firm, and prepare your customer for a  $5000+ job.

You're making things worse.  NEVER use original data.  You should be working with a bit-level copy.  Renaming files changes not only files but metadata, inodes, logs, and more.   To be frank, you're in over your head.  Ontrack is one of the best companies out there.  Call them and hope they don't determine that your actions up to now made things worse.
0
 
LVL 31

Author Comment

by:Frosty555
ID: 40553970
Okay so I have an update, I've figured out what's wrong with the files.

On Mac, some files (especially Quark Express files) can have an extended attribute called "com.apple.FinderInfo" which identifies the type of the file. When the QNAP performs a backup, it fails to copy this attribute.

So basically, we have ten thousand files that are missing their extended attribute that identifies the file. Aside from that the files are intact. I was able to rename one Quark file and give it the .QXB extension and open it without any problems.

SO... the question now is, is there any recovery utilities out there that can analyze the contents of the file, and add the file extension or attribute back?
0
 
LVL 27

Accepted Solution

by:
serialband earned 500 total points
ID: 40554245
Which version of OSX is he running?  It doesn't sound like a newer one.  None of my current files have the com.apple.FinderInfo attribute.  The newer OSX just needs the file extension, just like Windows, to identify the file type.  They've deprecated the resource forks and I've never had to explicitly copy or set the xattr to back up any files.  Earlier OSX users were supposed to use ditto to backup, copy or zip up files with resource forks.  Apple did add extended attribute support to the built-in rsync command later on, without having to compile the fix.  He should just upgrade to newer hardware, to get the newer version of OSX that supports file name extensions.


It might be simple enough to write a script using the xattr command to set it if he has the files organized by folders already.

Excerpt from man xattr
This example copies the com.apple.FinderInfo attribute from the /usr directory to the MyDir directory:

           % xattr -px com.apple.FinderInfo /usr
           00 00 00 00 00 00 00 00 40 00 00 00 00 00 00 00
           00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
           % xattr -l MyDir
           % xattr -wx com.apple.FinderInfo \
             "`xattr -px com.apple.FinderInfo /usr`" MyDir
           % xattr -l MyDir
           com.apple.FinderInfo:
           00000000  00 00 00 00 00 00 00 00 40 00 00 00 00 00 00 00  |........@.......|
           00000010  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  |................|
           00000020

Open in new window

You can see more examples here:  http://gotofritz.net/blog/geekery/os-x-extended-attibutes/

If they're sorted by folders, you could just write a for loop on the command line to do this.
Set the file
for file in (List of Files) ; do xattr -wx com.apple.FinderInfo $file File_Attributes; done

Rename the file with the extension:
for file in (List of Files) ; do mv $file $file.extension; done

List of Files can be as easy as *, if you just cd to the folder and the files have no spaces in their name.

Once you've fixed the extended attributes, I would suggest using Apple's built-in Time Machine for backing up a Mac.  If he has an older Mac, that can't use Time Machine, even rsync -X or rsnaphot (with rsync -X specified), or ditto would probably work better than the default QNAP NAS backup.  I don't trust default NAS backup tools.  It's simpler to treat them as dumb external disks and use native software to do the backup.
0
 
LVL 31

Author Comment

by:Frosty555
ID: 40559072
We played with xattr for a little while and I determined that I could re-create the com.apple.FinderInfo and make files readable again, but identifying what value the FinderInfo attribute should have for which files was going to be pretty painful.

In the end, we managed to recover by having QNAP support help us re-assemble the original RAID1 disks (they needed to SSH into the NAS to do this). After the RAID was re-assembled and the NAS was rebooted, we basically had our original data back, including the extended attributes.

So we're back in business. But I'm very aware that if we had actually suffered a real hard disk failure we would have essentially lost a lot of data or at the very least had to spend a lot of time in re-assembling the data.

We are going to investigate different backup and storage options and try to find one that works a bit better than the QNAP. I agree now the QNAP NAS backup is not really all that great.
0

Featured Post

Maximize Your Threat Intelligence Reporting

Reporting is one of the most important and least talked about aspects of a world-class threat intelligence program. Here’s how to do it right.

Join & Write a Comment

In this article we have discussed the manual scenarios to recover data from Windows 10 through some backup and recovery tools which are offered by it.
First I will try to share a design of a Veeam Backup Infrastructure without Direct NFS Access backup. Note: Direct NFS Access backup transport mechanism is only available in Veeam v9 In above I try to design the Veeam Backup flow between i…
In this Micro Tutorial viewers will learn how to use Boot Corrector from Paragon Rescue Kit Free to identify and fix the boot problems of Windows 7/8/2012R2 etc. As an example is used Windows 2012R2 which lost its active partition flag (often happen…
To efficiently enable the rotation of USB drives for backups, storage pools need to be created. This way no matter which USB drive is installed, the backups will successfully write without any administrative intervention. Multiple USB devices need t…

747 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now