Solved

RHEL 5 : Can only log in as root

Posted on 2016-09-14
6
30 Views
Last Modified: 2016-09-21
I rebooted my RHEL5 server today.  After discovering I couldn't remote in using ssh, I checked the console and it appeared the system was in runlevel 1 for some reason.  I rebooted again, watched the boot process and couldn't see anything significant.  I checked the runlevel, set the default runlevel to 3 and rebooted again with no effect.  I can log in to the console as root.  But, I cannot log in as any other existing user.  The login fails with no feedback.  If I try to "su - " to the user, I get a "permission" denied to the user's home directory and a "permission denied" to "/bin/bash".  I have tried to troubleshoot by adding a new user and logging in.  Same effect.

The "runlevel" command yields "N 3" which I am pretty sure means it thinks it is running in level 3 now.

All mounts seem to be OK.  I have rebooted with "-F" to force fsck, no problems.  I can navigate to all mounted volumes and access files on all of them.

I'm stumped.  The system was running and accessible when I rebooted earlier.  It is possible I mucked something up earlier when I was working on something else.  But, I am not sure what that might be.

I have crept around the web and there is rarely any mention of this problem.  When I find something, the suggestion of using "strace" is prevalent (did it, didn't reveal any permission issues that I can tell)  or some version of reinstall OS is deployed.

Anybody have any suggestions?
0
Comment
Question by:ecsginc
6 Comments
 
LVL 11

Expert Comment

by:loftyworm
ID: 41798694
I am no *nix guru, but I got strong suspicions that you have rooted :(

Maybe check your root boot files and see if anything is amiss

My 2 cents
0
 
LVL 9

Expert Comment

by:Scott Silva
ID: 41798894
Is there any message about damaged initrd or anything? If you have damaged boot files a system will usually drop to a minimal root session if it can get that far...
0
 

Author Comment

by:ecsginc
ID: 41801768
All boot of the errors are permissions issues similar to the su errors.  I am now thinking that somehow some uid, gid or permission got inadvertently change.  At this point I am now keying in on the sticky bit.  Does anyone have a list of standard system directories and or files where the sticky bit needs to be set?
0
What Should I Do With This Threat Intelligence?

Are you wondering if you actually need threat intelligence? The answer is yes. We explain the basics for creating useful threat intelligence.

 

Accepted Solution

by:
ecsginc earned 0 total points
ID: 41802028
FYI - I solved my problem.  After scouring permissions and ownership until my eyes bled (all errors in /var/logs/dmesg and /ver/log/messages aligned with permissions issues) I threw a Hail Mary and executed "chmod 755 /".  prior to this it was more of 744.  That did the trick.

I am not sure why or how the permissions got changed (I am suspecting and errant scripting inadvertently executed as su/root with empty environment variables yielding "/" on a chmod operation).

Regardless, if anyone should end up on this thread with their own issues, changing the permissions on the root directory ("/") did the trick for me.
0
 

Author Closing Comment

by:ecsginc
ID: 41808401
I discovered the apparent cause of my original issue.
0
 
LVL 40

Expert Comment

by:omarfarid
ID: 41808426
If you have a backup for system taken before issue, then you can restore and you will get the old perms
0

Featured Post

Highfive + Dolby Voice = No More Audio Complaints!

Poor audio quality is one of the top reasons people don’t use video conferencing. Get the crispest, clearest audio powered by Dolby Voice in every meeting. Highfive and Dolby Voice deliver the best video conferencing and audio experience for every meeting and every room.

Join & Write a Comment

In this tutorial I will explain how to make squid prevent malwares in five easy steps: Squid is a caching proxy for the Web supporting HTTP, HTTPS, FTP, and more. It reduces bandwidth and improves response times by caching and reusing frequently-…
It’s 2016. Password authentication should be dead — or at least close to dying. But, unfortunately, it has not traversed Quagga stage yet. Using password authentication is like laundering hotel guest linens with a washboard — it’s Passé.
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…

757 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

18 Experts available now in Live!

Get 1:1 Help Now