Solved

CentOS 5 forcefsck loop

Posted on 2012-03-12
6
669 Views
Last Modified: 2012-03-14
A client's POS server was crashing intermittently, and as part of the diagnostic I ran fsck. Of course the problems were on the root drive, so we had to use forcefsck to make the system check on boot.

The problem now is that forcefsck on boot finds errors that need to be addressed manually (not automatically), but it's loading the FS in read-only.

End result is that the machine is stuck in a loop. It boots and runs fsck, fails out to read-only, and we can't delete the forcefsck command.

The only solution I've found online so far is to try booting via a CD (like damn small linux) and accessing the files from there - but the root file system is inaccessible when I boot from CD.

The server has 2 SSD drives running in RAID 1 with a hardware controller.
0
Comment
Question by:RKFcomputers
  • 3
  • 3
6 Comments
 
LVL 31

Accepted Solution

by:
farzanj earned 500 total points
Comment Utility
You can do linux rescue.  You can do it by using CentOS CD1 or DVD.  At the boot prompt

boot:

Type
linux rescue askmethod

this enables options to mount some volumes over the network or if you don't need network availability option, just type

linux rescue

And hit enter.

It then mounts your actual drive under /mnt/sysimage.  So now your /root is /mnt/sysimage/root
And you /boot is /mnt/sysimage/boot.

You can do fsck as well if you enable chroot jail.  This you can do by issuing command

cd /mnt/sysimage
chroot /mnt/sysimage
0
 

Author Comment

by:RKFcomputers
Comment Utility
@Farzanj Thanks! I'll try that today and see how it works out.
0
 
LVL 31

Expert Comment

by:farzanj
Comment Utility
Good luck!!  And for your reference, here is the documentation of various modes to boot, including rescue.  You can also boot at run level emergency, or single user or run level 1, all of these are different.

http://www.centos.org/docs/5/html/Installation_Guide-en-US/ch-rescuemode.html
0
Free Gift Card with Acronis Backup Purchase!

Backup any data in any location: local and remote systems, physical and virtual servers, private and public clouds, Macs and PCs, tablets and mobile devices, & more! For limited time only, buy any Acronis backup products and get a FREE Amazon/Best Buy gift card worth up to $200!

 

Author Comment

by:RKFcomputers
Comment Utility
I just received the server from the client this evening, and it's actually CentOS 6, not 5.

When I boot from a CentOS 6.1 DVD 1 I get the following options:
Install or upgrade...
Install system with basic video
Rescue installed system
Boot from local drive
Memory test

I chose Rescue Installed System from the menu, but it started an installation wizard that didn't resemble your instructions so I bailed. Is there a trick to get the command line boot like what you're describing?
0
 
LVL 31

Expert Comment

by:farzanj
Comment Utility
You chose the right thing.  It will start like a wizard.  Will ask you if you have network options like NFS, HTTP or FTP. It takes some time and then it would tell you where it mounted your current OS.  One thing that it would not do is trying to partition or trying to install software.  Other than these options, you should not worry.
0
 

Author Closing Comment

by:RKFcomputers
Comment Utility
Great answer, with prompt follow up
0

Featured Post

Better Security Awareness With Threat Intelligence

See how one of the leading financial services organizations uses Recorded Future as part of a holistic threat intelligence program to promote security awareness and proactively and efficiently identify threats.

Join & Write a Comment

Suggested Solutions

Title # Comments Views Activity
LDAP find expired users 8 41
What is filling up the server HDD? 13 49
server plus 2 42
AWS CLI issues with Tags 3 24
In my business, I use the LTS (Long Term Support) versions of Linux. My workstations do real work, and so I rarely have the patience to deal with silly problems caused by an upgraded kernel that had experimental software on it to begin with from a r…
Every server (virtual or physical) needs a console: and the console can be provided through hardware directly connected, software for remote connections, local connections, through a KVM, etc. This document explains the different types of consol…
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
Connecting to an Amazon Linux EC2 Instance from Windows Using PuTTY.

772 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

10 Experts available now in Live!

Get 1:1 Help Now