Improve company productivity with a Business Account.Sign Up

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 1396
  • Last Modified:

Unable to run fsck on ext4 on a 16 TB drive array

Hello I have a major issue with a Logical Volume.
I am running Fedora Core 11 64bit with a 16 TB ext4 volume.
The server had a hardware failure that caused the Volume to HARD crash on three occasions.
 
Now I am receiving These error messages " kernel: EXT4-fs error (device dm-3): ext4_lookup: deleted inode referenced: 6269694"

I looked up these errors and found the solution of these to be to run fsck on the volume.

I have tried to run fsck.ext for  on /dev/VolGroup001/Bigvol  and /dev/dm-3

Once I got the drive completely unmounted via a reboot with commented out of fstab
I ran e2fsck -pf /dev/VolGroup001/Bigvol  

This is the error that was returned
"e2fsck 1.41.9  Error determining size of the physical device: File too large"

My Question is this:First on Fedora Core 11 they don't have a YUM package for 1.14.11 . I I use a RPM from fedora12 or a general RPM to upgrade?

Second : When I am able to run fsck on my 16 TB drive what switches should I use and what will I lose and what will I gain?

Thanks for the help
0
pjinlaok
Asked:
pjinlaok
  • 5
  • 3
1 Solution
 
Pieter JordaanSenior Systems Administrator - Web hosting and Network Security.Commented:

Try fsck.ext4
0
 
pjinlaokAuthor Commented:
I have tried the fsck.ext4 command. When I run the command I actually get the result from e2fsck running as fsck.ext4.


It seems that there is a known bug in fsck.ext4 on large volumes for version 1.14.9. The fix is in 1.14.11 but I can't find a version for Fedora Core 11.

It is available on FC12 but I don't want to upgrade.

Any other thoughts?
0
 
Pieter JordaanSenior Systems Administrator - Web hosting and Network Security.Commented:

I wouldn't take any chances with different tools.

Rather boot with the FC12 CD in recovery mode, and run the repair.
0
Easily Design & Build Your Next Website

Squarespace’s all-in-one platform gives you everything you need to express yourself creatively online, whether it is with a domain, website, or online store. Get started with your free trial today, and when ready, take 10% off your first purchase with offer code 'EXPERTS'.

 
pjinlaokAuthor Commented:
That's a great idea. Have you seen this specific issue before?

Any idea on if fsck fixing the errors that I am receiving?

Aug 17 10:21:02 bigvolserver kernel: EXT4-fs error (device dm-3): ext4_lookup: deleted inode referenced: 6269689
Aug 17 10:21:02 bigvolserver kernel: EXT4-fs error (device dm-3): ext4_lookup: deleted inode referenced: 5908783
Aug 17 10:21:02 bigvolserver kernel: EXT4-fs error (device dm-3): ext4_lookup: deleted inode referenced: 6291470
Aug 17 10:21:02 bigvolserver kernel: EXT4-fs error (device dm-3): ext4_lookup: deleted inode referenced: 6269694
Aug 17 10:21:02 bigvolserver kernel: EXT4-fs error (device dm-3): ext4_lookup: deleted inode referenced: 6312512
Aug 17 10:21:02 bigvolserver kernel: EXT4-fs error (device dm-3): ext4_lookup: deleted inode referenced: 6269536
Aug 17 10:21:02 bigvolserver kernel: EXT4-fs error (device dm-3): ext4_lookup: deleted inode referenced: 6269689
Aug 17 10:21:02 bigvolserver kernel: EXT4-fs error (device dm-3): ext4_lookup: deleted inode referenced: 5908783
Aug 17 10:21:02 bigvolserver kernel: EXT4-fs error (device dm-3): ext4_lookup: deleted inode referenced: 6291470
bigvolserver
0
 
Pieter JordaanSenior Systems Administrator - Web hosting and Network Security.Commented:

I have never used an ext4 partition of over 8TB, so this is a new problem for me.

I have seen many corrupted inodes be delete after a power failure.
ext4 has an increased intolerance to unclean shutdowns.

Rather download Fedora Core 13 for the file system check. Some bugs were fixed on the latest release.

0
 
pjinlaokAuthor Commented:
I already have the fedora core 12 64 bit DVD so I tried that first. I was able to boot into the recovery console. So far I can't run fsck on /dev/VolGroup001/Bigvol . Error cannot find superblock.

I could tell that the 16 TB drive = /dev/sda
when I run lvdisplay it shows me /dev/VolGroup001/Bigvol
But ?volGoup001/Bigvol doesn't show up in dev to run the fsck against.

Can you give me the steps to fsck a LVM on a rescue disk?

I am downloading over a torrent fedora core 13 64 bit DVD which will take a few more hours.

I am going to attempt the FSCK with 12 and if it fails I will try the fedora 13 DVD.
0
 
pjinlaokAuthor Commented:
I am unable to FSCK the two ext4 LVM's from the Fedora Core 12 rescue disk.
When I boot into Rescue mode off of the FC12 DVD I receive an error when processing the large volumes. I get a message "error processing /de/sda Do you want to re-initialize the drive - Warning DATA LOSS WILL OCCUR when RE_INITIALIZING.

I am going to attempt to upgrade to FC12 and hopefully it won't break anything.

 
0
 
pjinlaokAuthor Commented:
I had attempted to boot off of the Ferdora Core 12 DVD and run fsck.ext4, but I received a error processing /dev/sda Skip - Read only - or Initialize - Warning - INITIALIZATION WILL CAUSE DATA LOSS.

 So when I booted the LVM /dev/VolGrou001/Bigvol is not available to do a fsck on.

I then Upgraded to Ferdora core 12 but it only has e2fsprogs V 1.14-9 and it still did not allow to do a fsck on the 16 TB volume. yum would not upgrade beyond this version.

I finally downloaded the latest source from http://e2fsprogs.sourceforge.net/

ran ./configure
make
make install

Bam! it worked! I had to run e2fsck 4 times before all the errors were fixed, but it fixed all the errors and I am back up and running.


0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Featured Post

The 14th Annual Expert Award Winners

The results are in! Meet the top members of our 2017 Expert Awards. Congratulations to all who qualified!

  • 5
  • 3
Tackle projects and never again get stuck behind a technical roadblock.
Join Now