Solved

CentOS 5 forcefsck loop

Posted on 2012-03-12
6
691 Views
Last Modified: 2012-03-14
A client's POS server was crashing intermittently, and as part of the diagnostic I ran fsck. Of course the problems were on the root drive, so we had to use forcefsck to make the system check on boot.

The problem now is that forcefsck on boot finds errors that need to be addressed manually (not automatically), but it's loading the FS in read-only.

End result is that the machine is stuck in a loop. It boots and runs fsck, fails out to read-only, and we can't delete the forcefsck command.

The only solution I've found online so far is to try booting via a CD (like damn small linux) and accessing the files from there - but the root file system is inaccessible when I boot from CD.

The server has 2 SSD drives running in RAID 1 with a hardware controller.
0
Comment
Question by:RKFcomputers
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 3
6 Comments
 
LVL 31

Accepted Solution

by:
farzanj earned 500 total points
ID: 37714497
You can do linux rescue.  You can do it by using CentOS CD1 or DVD.  At the boot prompt

boot:

Type
linux rescue askmethod

this enables options to mount some volumes over the network or if you don't need network availability option, just type

linux rescue

And hit enter.

It then mounts your actual drive under /mnt/sysimage.  So now your /root is /mnt/sysimage/root
And you /boot is /mnt/sysimage/boot.

You can do fsck as well if you enable chroot jail.  This you can do by issuing command

cd /mnt/sysimage
chroot /mnt/sysimage
0
 

Author Comment

by:RKFcomputers
ID: 37714552
@Farzanj Thanks! I'll try that today and see how it works out.
0
 
LVL 31

Expert Comment

by:farzanj
ID: 37714636
Good luck!!  And for your reference, here is the documentation of various modes to boot, including rescue.  You can also boot at run level emergency, or single user or run level 1, all of these are different.

http://www.centos.org/docs/5/html/Installation_Guide-en-US/ch-rescuemode.html
0
Portable, direct connect server access

The ATEN CV211 connects a laptop directly to any server allowing you instant access to perform data maintenance and local operations, for quick troubleshooting, updating, service and repair.

 

Author Comment

by:RKFcomputers
ID: 37718269
I just received the server from the client this evening, and it's actually CentOS 6, not 5.

When I boot from a CentOS 6.1 DVD 1 I get the following options:
Install or upgrade...
Install system with basic video
Rescue installed system
Boot from local drive
Memory test

I chose Rescue Installed System from the menu, but it started an installation wizard that didn't resemble your instructions so I bailed. Is there a trick to get the command line boot like what you're describing?
0
 
LVL 31

Expert Comment

by:farzanj
ID: 37718273
You chose the right thing.  It will start like a wizard.  Will ask you if you have network options like NFS, HTTP or FTP. It takes some time and then it would tell you where it mounted your current OS.  One thing that it would not do is trying to partition or trying to install software.  Other than these options, you should not worry.
0
 

Author Closing Comment

by:RKFcomputers
ID: 37721317
Great answer, with prompt follow up
0

Featured Post

Best Practices: Disaster Recovery Testing

Besides backup, any IT division should have a disaster recovery plan. You will find a few tips below relating to the development of such a plan and to what issues one should pay special attention in the course of backup planning.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Linux Hosting 16 102
Let's Encrypt - Auto Renew Failure via Cron Job 1 61
CentOs root password/fsck issue 7 58
Cannot seems to get SSHD running CentOS 6.5 3 39
Hyper-convergence systems have taken the IT world by storm and have quickly started to change our point of view of how the data center should and could be architected. In this article, I’ll explain the benefits of employing a hyper-converged system …
Moving your enterprise fax infrastructure from in-house fax machines and servers to the cloud makes sense — from both an efficiency and productivity standpoint. But does migrating to a cloud fax solution mean you will no longer be able to send or re…
Get a first impression of how PRTG looks and learn how it works.   This video is a short introduction to PRTG, as an initial overview or as a quick start for new PRTG users.
How to Install VMware Tools in Red Hat Enterprise Linux 6.4 (RHEL 6.4) Step-by-Step Tutorial

738 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question