Solved

Apache config got garbled, what happened?

Posted on 2009-07-06
10
344 Views
Last Modified: 2012-05-07
Coming back from the weekend the apache driven svn isn't responding. Turns out apache isn't running. Turns out part of my first line in my httpd.conf looks like this:

BZh91AY&SY~Cxáò^@U^X^?÷ÿ³P@^?ÿÿÿÿÿÿÿÿÿÿÿB~@~@^B^F^@^@^@À~A^A^Q^@^H`q>}ëîövç^]O~@)õ@ÊëàZêWZ^Ez^@òë~U*@| ^E^@^@R

I've got 107 lines of similar junk. The question is now, what happened?
I can rebuild my config, I'm mainly worried if there's any risk I've been deliberately attacked, and compromised.
Should not have been a poweroutage, because as I recall grub is incorrectly configured and fails booting. Gonna confirm that when I can get physical access.

ssl_error_log ends with:
[Thu Jul 02 13:16:19 2009] [error] [client 193.12.19.73] Provider encountered an error while streaming a REPORT response.  [500, #0]
[Thu Jul 02 13:16:19 2009] [error] [client 193.12.19.73] A failure occurred while driving the update report editor  [500, #104]
[Thu Jul 02 13:16:19 2009] [error] [client 193.12.19.73] Error writing base64 data: Connection reset by peer  [500, #104]
0
Comment
Question by:letharion
  • 6
  • 4
10 Comments
 
LVL 28

Accepted Solution

by:
Jan Springer earned 500 total points
ID: 24785459
If the logs haven't been modified:

1) check /var/log/secure
2) last -23
3) ls -l /tmp
4) grep ftp /var/log/messages
5) grep sftp /var/log/secure

Also, download chkrootkit and run it on your machine:

http://www.chkrootkit.org/

Are you running iptables (or similar) for the server and modsecurity firewall for apache?
0
 
LVL 6

Author Comment

by:letharion
ID: 24786165
1) Does not exist
2) last -23 Shows loads of logins by me, and only me (+ reboot) which is expected. The logs also indicate that all connections came from the same IP, which is the only one that the hosters firewall allows access from.
3) Came up empty
4) Empty
5) See 1)

Posting chrootkit output below, I see nothing that obviously is wrong.

>Are you running iptables (or similar) for the server and modsecurity firewall for apache?
The security measures are:
1) Hardened toolchain + hardened kernel (Gentoo hardened profile)
2) The local firewall allows incoming connections from 1 single IP. No-one outside of my office (5 ppl) should even know that the server exists, and even less what adress (3 ppl) it can be reached at.

Primarily the second made me think I was pretty safe, so I haven't taken any other measures. I was gonna go for SELinux, but haven't taken the time to learn enough.
The server is generally up to date.
ROOTDIR is `/'
Checking `amd'... not found
Checking `basename'... not infected
Checking `biff'... not found
Checking `chfn'... not infected
Checking `chsh'... not infected
Checking `cron'... not infected
Checking `crontab'... not infected
Checking `date'... not infected
Checking `du'... not infected
Checking `dirname'... not infected
Checking `echo'... not infected
Checking `egrep'... not infected
Checking `env'... not infected
Checking `find'... not infected
Checking `fingerd'... not found
Checking `gpm'... not infected
Checking `grep'... not infected
Checking `hdparm'... not infected
Checking `su'... not infected
Checking `ifconfig'... not infected
Checking `inetd'... not tested
Checking `inetdconf'... not found
Checking `identd'... not found
Checking `init'... not infected
Checking `killall'... not infected
Checking `ldsopreload'... not infected
Checking `login'... not infected
Checking `ls'... not infected
Checking `lsof'... not infected
Checking `mail'... not infected
Checking `mingetty'... not found
Checking `netstat'... not infected
Checking `named'... not found
Checking `passwd'... not infected
Checking `pidof'... not infected
Checking `pop2'... not found
Checking `pop3'... not found
Checking `ps'... not infected
Checking `pstree'... not infected
Checking `rpcinfo'... not infected
Checking `rlogind'... not found
Checking `rshd'... not found
Checking `slogin'... not infected
Checking `sendmail'... not infected
Checking `sshd'... not infected
Checking `syslogd'... not tested
Checking `tar'... not infected
Checking `tcpd'... not infected
Checking `tcpdump'... not infected
Checking `top'... not infected
Checking `telnetd'... not found
Checking `timed'... not found
Checking `traceroute'... not found
Checking `vdir'... not infected
Checking `w'... not infected
Checking `write'... not infected
Checking `aliens'... no suspect files
Searching for sniffer's logs, it may take a while... nothing found
Searching for HiDrootkit's default dir... nothing found
Searching for t0rn's default files and dirs... nothing found
Searching for t0rn's v8 defaults... nothing found
Searching for Lion Worm default files and dirs... nothing found
Searching for RSHA's default files and dir... nothing found
Searching for RH-Sharpe's default files... nothing found
Searching for Ambient's rootkit (ark) default files and dirs... nothing found
Searching for suspicious files and dirs, it may take a while...
/usr/lib/locale/.keep_sys-libs_glibc-2.2 /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/Date/Manip/.packlist /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/Crypt/SSLeay/.packlist /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/LWP/.packlist /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/Net/.packlist /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/Test/Harness/.packlist /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/URI/.packlist /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/HTML/Tagset/.packlist /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/List/Util/.packlist /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/HTML-Tree/.packlist /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/Pod/Parser/.packlist /usr/lib/perl5/5.8.8/i686-linux/.packlist /usr/lib/.keep /lib/rcscripts/.keep /lib/rcscripts/awk/.keep /lib/rcscripts/sh/.keep /lib/rcscripts/net/.keep /lib/udev/state/.keep_sys-fs_udev-0 /lib/udev/devices/.keep_sys-fs_udev-0
 
Searching for LPD Worm files and dirs... nothing found
Searching for Ramen Worm files and dirs... nothing found
Searching for Maniac files and dirs... nothing found
Searching for RK17 files and dirs... nothing found
Searching for Ducoci rootkit... nothing found
Searching for Adore Worm... nothing found
Searching for ShitC Worm... nothing found
Searching for Omega Worm... nothing found
Searching for Sadmind/IIS Worm... nothing found
Searching for MonKit... nothing found
Searching for Showtee... nothing found
Searching for OpticKit... nothing found
Searching for T.R.K... nothing found
Searching for Mithra... nothing found
Searching for LOC rootkit... nothing found
Searching for Romanian rootkit... nothing found
Searching for Suckit rootkit... nothing found
Searching for Volc rootkit... nothing found
Searching for Gold2 rootkit... nothing found
Searching for TC2 Worm default files and dirs... nothing found
Searching for Anonoying rootkit default files and dirs... nothing found
Searching for ZK rootkit default files and dirs... nothing found
Searching for ShKit rootkit default files and dirs... nothing found
Searching for AjaKit rootkit default files and dirs... nothing found
Searching for zaRwT rootkit default files and dirs... nothing found
Searching for Madalin rootkit default files... nothing found
Searching for Fu rootkit default files... nothing found
Searching for ESRK rootkit default files... nothing found
Searching for rootedoor... nothing found
Searching for ENYELKM rootkit default files... nothing found
Searching for common ssh-scanners default files... nothing found
Searching for suspect PHP files... /usr/bin/find: `/var/tmp/portage/sys-libs/pam-1.0.4/temp/ccWVfwxd.c': No such file or directory
/usr/bin/find: `/var/tmp/portage/sys-libs/pam-1.0.4/temp/ccIHlSUg.o': No such file or directory
nothing found
Searching for anomalies in shell history files... nothing found
Checking `asp'... not infected
Checking `bindshell'... not infected
Checking `lkm'... chkproc: nothing detected
chkdirs: nothing detected
Checking `rexedcs'... not found
Checking `sniffer'... eth0: not promisc and no PF_PACKET sockets
Checking `w55808'... not infected
Checking `wted'... chkwtmp: nothing deleted
Checking `scalper'... not infected
Checking `slapper'... not infected
Checking `z2'... chklastlog: nothing deleted
Checking `chkutmp'...  The tty of the following user process(es) were not found
 in /var/run/utmp !
! RUID          PID TTY    CMD
! 1000        13773 pts/5  -bash
! root        13904 pts/5  su -
! root        14109 pts/5  -su
! root        14290 pts/5  screen -xRR
chkutmp: nothing deleted

Open in new window

0
 
LVL 28

Expert Comment

by:Jan Springer
ID: 24786306
So, did the SVN process mangle your configuration?
0
Portable, direct connect server access

The ATEN CV211 connects a laptop directly to any server allowing you instant access to perform data maintenance and local operations, for quick troubleshooting, updating, service and repair.

 
LVL 6

Author Comment

by:letharion
ID: 24786674
I'm not sure what exactly you ask.
The httpd.conf is not part of any repo, so svn should not have been able to doing anything to it, if that's what you mean.
0
 
LVL 28

Expert Comment

by:Jan Springer
ID: 24786749
I wonder if you're running into an older bug of apache that mangles the httpd.conf.  What version of apache are you running?
0
 
LVL 6

Author Comment

by:letharion
ID: 24786801
>I wonder if you're running into an older bug of apache that mangles the httpd.conf.  What version of apache are you running?

Oh.
I'm gonna check.
Now I can't ssh in anymore... I get in but get 'booted' immediately.
It's "up to date" atleast, I'm fairly certain it's 2.2.11
0
 
LVL 28

Expert Comment

by:Jan Springer
ID: 24786832
If the machine is local and you can get log in, look at your logs that contain login information and updates.
0
 
LVL 6

Author Comment

by:letharion
ID: 24786952
Unfortunately it's not. I'm gonna have to wait until tomorrow
0
 
LVL 6

Author Comment

by:letharion
ID: 24795091
When I got to the computer, I was unable to log in locally too, and since this behaviour was _really_ wierd IMO, I decided to reinstall. As a bonus I got a new and fresh kernel 2.6.29.
I would have loved to dig through the logs and figure out what happened, but I just don't have the time.
0
 
LVL 6

Author Closing Comment

by:letharion
ID: 31600064
Thanks for taking the time :)
0

Featured Post

Free Tool: Site Down Detector

Helpful to verify reports of your own downtime, or to double check a downed website you are trying to access.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

I have seen several blogs and forum entries elsewhere state that because NTFS volumes do not support linux ownership or permissions, they cannot be used for anonymous ftp upload through the vsftpd program.   IT can be done and here's how to get i…
Over the last year I have answered a couple of basic URL rewriting questions several times so I thought I might as well have a stab at: explaining the basics, providing a few useful links and consolidating some of the most common queries into a sing…
Microsoft Active Directory, the widely used IT infrastructure, is known for its high risk of credential theft. The best way to test your Active Directory’s vulnerabilities to pass-the-ticket, pass-the-hash, privilege escalation, and malware attacks …

838 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question