Still celebrating National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

Unable to run fsck on ext4 on a 16 TB drive array

Posted on 2010-08-16
8
Medium Priority
?
1,349 Views
Last Modified: 2013-12-02
Hello I have a major issue with a Logical Volume.
I am running Fedora Core 11 64bit with a 16 TB ext4 volume.
The server had a hardware failure that caused the Volume to HARD crash on three occasions.
 
Now I am receiving These error messages " kernel: EXT4-fs error (device dm-3): ext4_lookup: deleted inode referenced: 6269694"

I looked up these errors and found the solution of these to be to run fsck on the volume.

I have tried to run fsck.ext for  on /dev/VolGroup001/Bigvol  and /dev/dm-3

Once I got the drive completely unmounted via a reboot with commented out of fstab
I ran e2fsck -pf /dev/VolGroup001/Bigvol  

This is the error that was returned
"e2fsck 1.41.9  Error determining size of the physical device: File too large"

My Question is this:First on Fedora Core 11 they don't have a YUM package for 1.14.11 . I I use a RPM from fedora12 or a general RPM to upgrade?

Second : When I am able to run fsck on my 16 TB drive what switches should I use and what will I lose and what will I gain?

Thanks for the help
0
Comment
Question by:pjinlaok
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 5
  • 3
8 Comments
 
LVL 11

Expert Comment

by:Pieter Jordaan
ID: 33453731

Try fsck.ext4
0
 

Author Comment

by:pjinlaok
ID: 33456435
I have tried the fsck.ext4 command. When I run the command I actually get the result from e2fsck running as fsck.ext4.


It seems that there is a known bug in fsck.ext4 on large volumes for version 1.14.9. The fix is in 1.14.11 but I can't find a version for Fedora Core 11.

It is available on FC12 but I don't want to upgrade.

Any other thoughts?
0
 
LVL 11

Expert Comment

by:Pieter Jordaan
ID: 33456513

I wouldn't take any chances with different tools.

Rather boot with the FC12 CD in recovery mode, and run the repair.
0
Free Backup Tool for VMware and Hyper-V

Restore full virtual machine or individual guest files from 19 common file systems directly from the backup file. Schedule VM backups with PowerShell scripts. Set desired time, lean back and let the script to notify you via email upon completion.  

 

Author Comment

by:pjinlaok
ID: 33456857
That's a great idea. Have you seen this specific issue before?

Any idea on if fsck fixing the errors that I am receiving?

Aug 17 10:21:02 bigvolserver kernel: EXT4-fs error (device dm-3): ext4_lookup: deleted inode referenced: 6269689
Aug 17 10:21:02 bigvolserver kernel: EXT4-fs error (device dm-3): ext4_lookup: deleted inode referenced: 5908783
Aug 17 10:21:02 bigvolserver kernel: EXT4-fs error (device dm-3): ext4_lookup: deleted inode referenced: 6291470
Aug 17 10:21:02 bigvolserver kernel: EXT4-fs error (device dm-3): ext4_lookup: deleted inode referenced: 6269694
Aug 17 10:21:02 bigvolserver kernel: EXT4-fs error (device dm-3): ext4_lookup: deleted inode referenced: 6312512
Aug 17 10:21:02 bigvolserver kernel: EXT4-fs error (device dm-3): ext4_lookup: deleted inode referenced: 6269536
Aug 17 10:21:02 bigvolserver kernel: EXT4-fs error (device dm-3): ext4_lookup: deleted inode referenced: 6269689
Aug 17 10:21:02 bigvolserver kernel: EXT4-fs error (device dm-3): ext4_lookup: deleted inode referenced: 5908783
Aug 17 10:21:02 bigvolserver kernel: EXT4-fs error (device dm-3): ext4_lookup: deleted inode referenced: 6291470
bigvolserver
0
 
LVL 11

Expert Comment

by:Pieter Jordaan
ID: 33457044

I have never used an ext4 partition of over 8TB, so this is a new problem for me.

I have seen many corrupted inodes be delete after a power failure.
ext4 has an increased intolerance to unclean shutdowns.

Rather download Fedora Core 13 for the file system check. Some bugs were fixed on the latest release.

0
 

Author Comment

by:pjinlaok
ID: 33470787
I already have the fedora core 12 64 bit DVD so I tried that first. I was able to boot into the recovery console. So far I can't run fsck on /dev/VolGroup001/Bigvol . Error cannot find superblock.

I could tell that the 16 TB drive = /dev/sda
when I run lvdisplay it shows me /dev/VolGroup001/Bigvol
But ?volGoup001/Bigvol doesn't show up in dev to run the fsck against.

Can you give me the steps to fsck a LVM on a rescue disk?

I am downloading over a torrent fedora core 13 64 bit DVD which will take a few more hours.

I am going to attempt the FSCK with 12 and if it fails I will try the fedora 13 DVD.
0
 

Author Comment

by:pjinlaok
ID: 33471027
I am unable to FSCK the two ext4 LVM's from the Fedora Core 12 rescue disk.
When I boot into Rescue mode off of the FC12 DVD I receive an error when processing the large volumes. I get a message "error processing /de/sda Do you want to re-initialize the drive - Warning DATA LOSS WILL OCCUR when RE_INITIALIZING.

I am going to attempt to upgrade to FC12 and hopefully it won't break anything.

 
0
 

Accepted Solution

by:
pjinlaok earned 0 total points
ID: 33489500
I had attempted to boot off of the Ferdora Core 12 DVD and run fsck.ext4, but I received a error processing /dev/sda Skip - Read only - or Initialize - Warning - INITIALIZATION WILL CAUSE DATA LOSS.

 So when I booted the LVM /dev/VolGrou001/Bigvol is not available to do a fsck on.

I then Upgraded to Ferdora core 12 but it only has e2fsprogs V 1.14-9 and it still did not allow to do a fsck on the 16 TB volume. yum would not upgrade beyond this version.

I finally downloaded the latest source from http://e2fsprogs.sourceforge.net/

ran ./configure
make
make install

Bam! it worked! I had to run e2fsck 4 times before all the errors were fixed, but it fixed all the errors and I am back up and running.


0

Featured Post

Learn how to optimize MySQL for your business need

With the increasing importance of apps & networks in both business & personal interconnections, perfor. has become one of the key metrics of successful communication. This ebook is a hands-on business-case-driven guide to understanding MySQL query parameter tuning & database perf

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

I previously wrote an article addressing the use of UBCD4WIN and SARDU. All are great, but I have always been an advocate of SARDU. Recently it was suggested that I go back and take a look at Easy2Boot in comparison.
Storage devices are generally used to save the data or sometime transfer the data from one computer system to another system. However, sometimes user accidentally erased their important data from the Storage devices. Users have to know how data reco…
This video Micro Tutorial explains how to clone a hard drive using a commercial software product for Windows systems called Casper from Future Systems Solutions (FSS). Cloning makes an exact, complete copy of one hard disk drive (HDD) onto another d…
Michael from AdRem Software outlines event notifications and Automatic Corrective Actions in network monitoring. Automatic Corrective Actions are scripts, which can automatically run upon discovery of a certain undesirable condition in your network.…

721 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question