Solved

Apache config got garbled, what happened?

Posted on 2009-07-06
10
333 Views
Last Modified: 2012-05-07
Coming back from the weekend the apache driven svn isn't responding. Turns out apache isn't running. Turns out part of my first line in my httpd.conf looks like this:

BZh91AY&SY~Cxáò^@U^X^?÷ÿ³P@^?ÿÿÿÿÿÿÿÿÿÿÿB~@~@^B^F^@^@^@À~A^A^Q^@^H`q>}ëîövç^]O~@)õ@ÊëàZêWZ^Ez^@òë~U*@| ^E^@^@R

I've got 107 lines of similar junk. The question is now, what happened?
I can rebuild my config, I'm mainly worried if there's any risk I've been deliberately attacked, and compromised.
Should not have been a poweroutage, because as I recall grub is incorrectly configured and fails booting. Gonna confirm that when I can get physical access.

ssl_error_log ends with:
[Thu Jul 02 13:16:19 2009] [error] [client 193.12.19.73] Provider encountered an error while streaming a REPORT response.  [500, #0]
[Thu Jul 02 13:16:19 2009] [error] [client 193.12.19.73] A failure occurred while driving the update report editor  [500, #104]
[Thu Jul 02 13:16:19 2009] [error] [client 193.12.19.73] Error writing base64 data: Connection reset by peer  [500, #104]
0
Comment
Question by:letharion
  • 6
  • 4
10 Comments
 
LVL 28

Accepted Solution

by:
Jan Springer earned 500 total points
ID: 24785459
If the logs haven't been modified:

1) check /var/log/secure
2) last -23
3) ls -l /tmp
4) grep ftp /var/log/messages
5) grep sftp /var/log/secure

Also, download chkrootkit and run it on your machine:

http://www.chkrootkit.org/

Are you running iptables (or similar) for the server and modsecurity firewall for apache?
0
 
LVL 6

Author Comment

by:letharion
ID: 24786165
1) Does not exist
2) last -23 Shows loads of logins by me, and only me (+ reboot) which is expected. The logs also indicate that all connections came from the same IP, which is the only one that the hosters firewall allows access from.
3) Came up empty
4) Empty
5) See 1)

Posting chrootkit output below, I see nothing that obviously is wrong.

>Are you running iptables (or similar) for the server and modsecurity firewall for apache?
The security measures are:
1) Hardened toolchain + hardened kernel (Gentoo hardened profile)
2) The local firewall allows incoming connections from 1 single IP. No-one outside of my office (5 ppl) should even know that the server exists, and even less what adress (3 ppl) it can be reached at.

Primarily the second made me think I was pretty safe, so I haven't taken any other measures. I was gonna go for SELinux, but haven't taken the time to learn enough.
The server is generally up to date.
ROOTDIR is `/'

Checking `amd'... not found

Checking `basename'... not infected

Checking `biff'... not found

Checking `chfn'... not infected

Checking `chsh'... not infected

Checking `cron'... not infected

Checking `crontab'... not infected

Checking `date'... not infected

Checking `du'... not infected

Checking `dirname'... not infected

Checking `echo'... not infected

Checking `egrep'... not infected

Checking `env'... not infected

Checking `find'... not infected

Checking `fingerd'... not found

Checking `gpm'... not infected

Checking `grep'... not infected

Checking `hdparm'... not infected

Checking `su'... not infected

Checking `ifconfig'... not infected

Checking `inetd'... not tested

Checking `inetdconf'... not found

Checking `identd'... not found

Checking `init'... not infected

Checking `killall'... not infected

Checking `ldsopreload'... not infected

Checking `login'... not infected

Checking `ls'... not infected

Checking `lsof'... not infected

Checking `mail'... not infected

Checking `mingetty'... not found

Checking `netstat'... not infected

Checking `named'... not found

Checking `passwd'... not infected

Checking `pidof'... not infected

Checking `pop2'... not found

Checking `pop3'... not found

Checking `ps'... not infected

Checking `pstree'... not infected

Checking `rpcinfo'... not infected

Checking `rlogind'... not found

Checking `rshd'... not found

Checking `slogin'... not infected

Checking `sendmail'... not infected

Checking `sshd'... not infected

Checking `syslogd'... not tested

Checking `tar'... not infected

Checking `tcpd'... not infected

Checking `tcpdump'... not infected

Checking `top'... not infected

Checking `telnetd'... not found

Checking `timed'... not found

Checking `traceroute'... not found

Checking `vdir'... not infected

Checking `w'... not infected

Checking `write'... not infected

Checking `aliens'... no suspect files

Searching for sniffer's logs, it may take a while... nothing found

Searching for HiDrootkit's default dir... nothing found

Searching for t0rn's default files and dirs... nothing found

Searching for t0rn's v8 defaults... nothing found

Searching for Lion Worm default files and dirs... nothing found

Searching for RSHA's default files and dir... nothing found

Searching for RH-Sharpe's default files... nothing found

Searching for Ambient's rootkit (ark) default files and dirs... nothing found

Searching for suspicious files and dirs, it may take a while...

/usr/lib/locale/.keep_sys-libs_glibc-2.2 /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/Date/Manip/.packlist /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/Crypt/SSLeay/.packlist /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/LWP/.packlist /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/Net/.packlist /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/Test/Harness/.packlist /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/URI/.packlist /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/HTML/Tagset/.packlist /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/List/Util/.packlist /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/HTML-Tree/.packlist /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/Pod/Parser/.packlist /usr/lib/perl5/5.8.8/i686-linux/.packlist /usr/lib/.keep /lib/rcscripts/.keep /lib/rcscripts/awk/.keep /lib/rcscripts/sh/.keep /lib/rcscripts/net/.keep /lib/udev/state/.keep_sys-fs_udev-0 /lib/udev/devices/.keep_sys-fs_udev-0
 

Searching for LPD Worm files and dirs... nothing found

Searching for Ramen Worm files and dirs... nothing found

Searching for Maniac files and dirs... nothing found

Searching for RK17 files and dirs... nothing found

Searching for Ducoci rootkit... nothing found

Searching for Adore Worm... nothing found

Searching for ShitC Worm... nothing found

Searching for Omega Worm... nothing found

Searching for Sadmind/IIS Worm... nothing found

Searching for MonKit... nothing found

Searching for Showtee... nothing found

Searching for OpticKit... nothing found

Searching for T.R.K... nothing found

Searching for Mithra... nothing found

Searching for LOC rootkit... nothing found

Searching for Romanian rootkit... nothing found

Searching for Suckit rootkit... nothing found

Searching for Volc rootkit... nothing found

Searching for Gold2 rootkit... nothing found

Searching for TC2 Worm default files and dirs... nothing found

Searching for Anonoying rootkit default files and dirs... nothing found

Searching for ZK rootkit default files and dirs... nothing found

Searching for ShKit rootkit default files and dirs... nothing found

Searching for AjaKit rootkit default files and dirs... nothing found

Searching for zaRwT rootkit default files and dirs... nothing found

Searching for Madalin rootkit default files... nothing found

Searching for Fu rootkit default files... nothing found

Searching for ESRK rootkit default files... nothing found

Searching for rootedoor... nothing found

Searching for ENYELKM rootkit default files... nothing found

Searching for common ssh-scanners default files... nothing found

Searching for suspect PHP files... /usr/bin/find: `/var/tmp/portage/sys-libs/pam-1.0.4/temp/ccWVfwxd.c': No such file or directory

/usr/bin/find: `/var/tmp/portage/sys-libs/pam-1.0.4/temp/ccIHlSUg.o': No such file or directory

nothing found

Searching for anomalies in shell history files... nothing found

Checking `asp'... not infected

Checking `bindshell'... not infected

Checking `lkm'... chkproc: nothing detected

chkdirs: nothing detected

Checking `rexedcs'... not found

Checking `sniffer'... eth0: not promisc and no PF_PACKET sockets

Checking `w55808'... not infected

Checking `wted'... chkwtmp: nothing deleted

Checking `scalper'... not infected

Checking `slapper'... not infected

Checking `z2'... chklastlog: nothing deleted

Checking `chkutmp'...  The tty of the following user process(es) were not found

 in /var/run/utmp !

! RUID          PID TTY    CMD

! 1000        13773 pts/5  -bash

! root        13904 pts/5  su -

! root        14109 pts/5  -su

! root        14290 pts/5  screen -xRR

chkutmp: nothing deleted

Open in new window

0
 
LVL 28

Expert Comment

by:Jan Springer
ID: 24786306
So, did the SVN process mangle your configuration?
0
 
LVL 6

Author Comment

by:letharion
ID: 24786674
I'm not sure what exactly you ask.
The httpd.conf is not part of any repo, so svn should not have been able to doing anything to it, if that's what you mean.
0
 
LVL 28

Expert Comment

by:Jan Springer
ID: 24786749
I wonder if you're running into an older bug of apache that mangles the httpd.conf.  What version of apache are you running?
0
Zoho SalesIQ

Hassle-free live chat software re-imagined for business growth. 2 users, always free.

 
LVL 6

Author Comment

by:letharion
ID: 24786801
>I wonder if you're running into an older bug of apache that mangles the httpd.conf.  What version of apache are you running?

Oh.
I'm gonna check.
Now I can't ssh in anymore... I get in but get 'booted' immediately.
It's "up to date" atleast, I'm fairly certain it's 2.2.11
0
 
LVL 28

Expert Comment

by:Jan Springer
ID: 24786832
If the machine is local and you can get log in, look at your logs that contain login information and updates.
0
 
LVL 6

Author Comment

by:letharion
ID: 24786952
Unfortunately it's not. I'm gonna have to wait until tomorrow
0
 
LVL 6

Author Comment

by:letharion
ID: 24795091
When I got to the computer, I was unable to log in locally too, and since this behaviour was _really_ wierd IMO, I decided to reinstall. As a bonus I got a new and fresh kernel 2.6.29.
I would have loved to dig through the logs and figure out what happened, but I just don't have the time.
0
 
LVL 6

Author Closing Comment

by:letharion
ID: 31600064
Thanks for taking the time :)
0

Featured Post

What is SQL Server and how does it work?

The purpose of this paper is to provide you background on SQL Server. It’s your self-study guide for learning fundamentals. It includes both the history of SQL and its technical basics. Concepts and definitions will form the solid foundation of your future DBA expertise.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Squid Connection Pools 3 46
SIP Trunk provider 20 116
how to monitor remote shell execution on linux 9 97
More Than One Website On Same DMZ Server 3 53
If you've heard about htaccess and it sounds like it does what you want, but you're not sure how it works... well, you're in the right place. Read on. Some Basics #1. It's a file and its filename is .htaccess (yes, with a dot in the front). #…
In Solr 4.0 it is possible to atomically (or partially) update individual fields in a document. This article will show the operations possible for atomic updating as well as setting up your Solr instance to be able to perform the actions. One major …
This Micro Tutorial hows how you can integrate  Mac OSX to a Windows Active Directory Domain. Apple has made it easy to allow users to bind their macs to a windows domain with relative ease. The following video show how to bind OSX Mavericks to …
Both in life and business – not all partnerships are created equal. As the demand for cloud services increases, so do the number of self-proclaimed cloud partners. Asking the right questions up front in the partnership, will enable both parties …

910 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

24 Experts available now in Live!

Get 1:1 Help Now