Apache config got garbled, what happened?

Coming back from the weekend the apache driven svn isn't responding. Turns out apache isn't running. Turns out part of my first line in my httpd.conf looks like this:

BZh91AY&SY~Cxáò^@U^X^?÷ÿ³P@^?ÿÿÿÿÿÿÿÿÿÿÿB~@~@^B^F^@^@^@À~A^A^Q^@^H`q>}ëîövç^]O~@)õ@ÊëàZêWZ^Ez^@òë~U*@| ^E^@^@R

I've got 107 lines of similar junk. The question is now, what happened?
I can rebuild my config, I'm mainly worried if there's any risk I've been deliberately attacked, and compromised.
Should not have been a poweroutage, because as I recall grub is incorrectly configured and fails booting. Gonna confirm that when I can get physical access.

ssl_error_log ends with:
[Thu Jul 02 13:16:19 2009] [error] [client 193.12.19.73] Provider encountered an error while streaming a REPORT response.  [500, #0]
[Thu Jul 02 13:16:19 2009] [error] [client 193.12.19.73] A failure occurred while driving the update report editor  [500, #104]
[Thu Jul 02 13:16:19 2009] [error] [client 193.12.19.73] Error writing base64 data: Connection reset by peer  [500, #104]
LVL 6
letharionAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Jan SpringerCommented:
If the logs haven't been modified:

1) check /var/log/secure
2) last -23
3) ls -l /tmp
4) grep ftp /var/log/messages
5) grep sftp /var/log/secure

Also, download chkrootkit and run it on your machine:

http://www.chkrootkit.org/

Are you running iptables (or similar) for the server and modsecurity firewall for apache?
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
letharionAuthor Commented:
1) Does not exist
2) last -23 Shows loads of logins by me, and only me (+ reboot) which is expected. The logs also indicate that all connections came from the same IP, which is the only one that the hosters firewall allows access from.
3) Came up empty
4) Empty
5) See 1)

Posting chrootkit output below, I see nothing that obviously is wrong.

>Are you running iptables (or similar) for the server and modsecurity firewall for apache?
The security measures are:
1) Hardened toolchain + hardened kernel (Gentoo hardened profile)
2) The local firewall allows incoming connections from 1 single IP. No-one outside of my office (5 ppl) should even know that the server exists, and even less what adress (3 ppl) it can be reached at.

Primarily the second made me think I was pretty safe, so I haven't taken any other measures. I was gonna go for SELinux, but haven't taken the time to learn enough.
The server is generally up to date.
ROOTDIR is `/'
Checking `amd'... not found
Checking `basename'... not infected
Checking `biff'... not found
Checking `chfn'... not infected
Checking `chsh'... not infected
Checking `cron'... not infected
Checking `crontab'... not infected
Checking `date'... not infected
Checking `du'... not infected
Checking `dirname'... not infected
Checking `echo'... not infected
Checking `egrep'... not infected
Checking `env'... not infected
Checking `find'... not infected
Checking `fingerd'... not found
Checking `gpm'... not infected
Checking `grep'... not infected
Checking `hdparm'... not infected
Checking `su'... not infected
Checking `ifconfig'... not infected
Checking `inetd'... not tested
Checking `inetdconf'... not found
Checking `identd'... not found
Checking `init'... not infected
Checking `killall'... not infected
Checking `ldsopreload'... not infected
Checking `login'... not infected
Checking `ls'... not infected
Checking `lsof'... not infected
Checking `mail'... not infected
Checking `mingetty'... not found
Checking `netstat'... not infected
Checking `named'... not found
Checking `passwd'... not infected
Checking `pidof'... not infected
Checking `pop2'... not found
Checking `pop3'... not found
Checking `ps'... not infected
Checking `pstree'... not infected
Checking `rpcinfo'... not infected
Checking `rlogind'... not found
Checking `rshd'... not found
Checking `slogin'... not infected
Checking `sendmail'... not infected
Checking `sshd'... not infected
Checking `syslogd'... not tested
Checking `tar'... not infected
Checking `tcpd'... not infected
Checking `tcpdump'... not infected
Checking `top'... not infected
Checking `telnetd'... not found
Checking `timed'... not found
Checking `traceroute'... not found
Checking `vdir'... not infected
Checking `w'... not infected
Checking `write'... not infected
Checking `aliens'... no suspect files
Searching for sniffer's logs, it may take a while... nothing found
Searching for HiDrootkit's default dir... nothing found
Searching for t0rn's default files and dirs... nothing found
Searching for t0rn's v8 defaults... nothing found
Searching for Lion Worm default files and dirs... nothing found
Searching for RSHA's default files and dir... nothing found
Searching for RH-Sharpe's default files... nothing found
Searching for Ambient's rootkit (ark) default files and dirs... nothing found
Searching for suspicious files and dirs, it may take a while...
/usr/lib/locale/.keep_sys-libs_glibc-2.2 /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/Date/Manip/.packlist /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/Crypt/SSLeay/.packlist /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/LWP/.packlist /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/Net/.packlist /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/Test/Harness/.packlist /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/URI/.packlist /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/HTML/Tagset/.packlist /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/List/Util/.packlist /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/HTML-Tree/.packlist /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/Pod/Parser/.packlist /usr/lib/perl5/5.8.8/i686-linux/.packlist /usr/lib/.keep /lib/rcscripts/.keep /lib/rcscripts/awk/.keep /lib/rcscripts/sh/.keep /lib/rcscripts/net/.keep /lib/udev/state/.keep_sys-fs_udev-0 /lib/udev/devices/.keep_sys-fs_udev-0
 
Searching for LPD Worm files and dirs... nothing found
Searching for Ramen Worm files and dirs... nothing found
Searching for Maniac files and dirs... nothing found
Searching for RK17 files and dirs... nothing found
Searching for Ducoci rootkit... nothing found
Searching for Adore Worm... nothing found
Searching for ShitC Worm... nothing found
Searching for Omega Worm... nothing found
Searching for Sadmind/IIS Worm... nothing found
Searching for MonKit... nothing found
Searching for Showtee... nothing found
Searching for OpticKit... nothing found
Searching for T.R.K... nothing found
Searching for Mithra... nothing found
Searching for LOC rootkit... nothing found
Searching for Romanian rootkit... nothing found
Searching for Suckit rootkit... nothing found
Searching for Volc rootkit... nothing found
Searching for Gold2 rootkit... nothing found
Searching for TC2 Worm default files and dirs... nothing found
Searching for Anonoying rootkit default files and dirs... nothing found
Searching for ZK rootkit default files and dirs... nothing found
Searching for ShKit rootkit default files and dirs... nothing found
Searching for AjaKit rootkit default files and dirs... nothing found
Searching for zaRwT rootkit default files and dirs... nothing found
Searching for Madalin rootkit default files... nothing found
Searching for Fu rootkit default files... nothing found
Searching for ESRK rootkit default files... nothing found
Searching for rootedoor... nothing found
Searching for ENYELKM rootkit default files... nothing found
Searching for common ssh-scanners default files... nothing found
Searching for suspect PHP files... /usr/bin/find: `/var/tmp/portage/sys-libs/pam-1.0.4/temp/ccWVfwxd.c': No such file or directory
/usr/bin/find: `/var/tmp/portage/sys-libs/pam-1.0.4/temp/ccIHlSUg.o': No such file or directory
nothing found
Searching for anomalies in shell history files... nothing found
Checking `asp'... not infected
Checking `bindshell'... not infected
Checking `lkm'... chkproc: nothing detected
chkdirs: nothing detected
Checking `rexedcs'... not found
Checking `sniffer'... eth0: not promisc and no PF_PACKET sockets
Checking `w55808'... not infected
Checking `wted'... chkwtmp: nothing deleted
Checking `scalper'... not infected
Checking `slapper'... not infected
Checking `z2'... chklastlog: nothing deleted
Checking `chkutmp'...  The tty of the following user process(es) were not found
 in /var/run/utmp !
! RUID          PID TTY    CMD
! 1000        13773 pts/5  -bash
! root        13904 pts/5  su -
! root        14109 pts/5  -su
! root        14290 pts/5  screen -xRR
chkutmp: nothing deleted

Open in new window

0
Jan SpringerCommented:
So, did the SVN process mangle your configuration?
0
10 Tips to Protect Your Business from Ransomware

Did you know that ransomware is the most widespread, destructive malware in the world today? It accounts for 39% of all security breaches, with ransomware gangsters projected to make $11.5B in profits from online extortion by 2019.

letharionAuthor Commented:
I'm not sure what exactly you ask.
The httpd.conf is not part of any repo, so svn should not have been able to doing anything to it, if that's what you mean.
0
Jan SpringerCommented:
I wonder if you're running into an older bug of apache that mangles the httpd.conf.  What version of apache are you running?
0
letharionAuthor Commented:
>I wonder if you're running into an older bug of apache that mangles the httpd.conf.  What version of apache are you running?

Oh.
I'm gonna check.
Now I can't ssh in anymore... I get in but get 'booted' immediately.
It's "up to date" atleast, I'm fairly certain it's 2.2.11
0
Jan SpringerCommented:
If the machine is local and you can get log in, look at your logs that contain login information and updates.
0
letharionAuthor Commented:
Unfortunately it's not. I'm gonna have to wait until tomorrow
0
letharionAuthor Commented:
When I got to the computer, I was unable to log in locally too, and since this behaviour was _really_ wierd IMO, I decided to reinstall. As a bonus I got a new and fresh kernel 2.6.29.
I would have loved to dig through the logs and figure out what happened, but I just don't have the time.
0
letharionAuthor Commented:
Thanks for taking the time :)
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Apache Web Server

From novice to tech pro — start learning today.