Solved

Apache config got garbled, what happened?

Posted on 2009-07-06
10
338 Views
Last Modified: 2012-05-07
Coming back from the weekend the apache driven svn isn't responding. Turns out apache isn't running. Turns out part of my first line in my httpd.conf looks like this:

BZh91AY&SY~Cxáò^@U^X^?÷ÿ³P@^?ÿÿÿÿÿÿÿÿÿÿÿB~@~@^B^F^@^@^@À~A^A^Q^@^H`q>}ëîövç^]O~@)õ@ÊëàZêWZ^Ez^@òë~U*@| ^E^@^@R

I've got 107 lines of similar junk. The question is now, what happened?
I can rebuild my config, I'm mainly worried if there's any risk I've been deliberately attacked, and compromised.
Should not have been a poweroutage, because as I recall grub is incorrectly configured and fails booting. Gonna confirm that when I can get physical access.

ssl_error_log ends with:
[Thu Jul 02 13:16:19 2009] [error] [client 193.12.19.73] Provider encountered an error while streaming a REPORT response.  [500, #0]
[Thu Jul 02 13:16:19 2009] [error] [client 193.12.19.73] A failure occurred while driving the update report editor  [500, #104]
[Thu Jul 02 13:16:19 2009] [error] [client 193.12.19.73] Error writing base64 data: Connection reset by peer  [500, #104]
0
Comment
Question by:letharion
  • 6
  • 4
10 Comments
 
LVL 28

Accepted Solution

by:
Jan Springer earned 500 total points
ID: 24785459
If the logs haven't been modified:

1) check /var/log/secure
2) last -23
3) ls -l /tmp
4) grep ftp /var/log/messages
5) grep sftp /var/log/secure

Also, download chkrootkit and run it on your machine:

http://www.chkrootkit.org/

Are you running iptables (or similar) for the server and modsecurity firewall for apache?
0
 
LVL 6

Author Comment

by:letharion
ID: 24786165
1) Does not exist
2) last -23 Shows loads of logins by me, and only me (+ reboot) which is expected. The logs also indicate that all connections came from the same IP, which is the only one that the hosters firewall allows access from.
3) Came up empty
4) Empty
5) See 1)

Posting chrootkit output below, I see nothing that obviously is wrong.

>Are you running iptables (or similar) for the server and modsecurity firewall for apache?
The security measures are:
1) Hardened toolchain + hardened kernel (Gentoo hardened profile)
2) The local firewall allows incoming connections from 1 single IP. No-one outside of my office (5 ppl) should even know that the server exists, and even less what adress (3 ppl) it can be reached at.

Primarily the second made me think I was pretty safe, so I haven't taken any other measures. I was gonna go for SELinux, but haven't taken the time to learn enough.
The server is generally up to date.
ROOTDIR is `/'
Checking `amd'... not found
Checking `basename'... not infected
Checking `biff'... not found
Checking `chfn'... not infected
Checking `chsh'... not infected
Checking `cron'... not infected
Checking `crontab'... not infected
Checking `date'... not infected
Checking `du'... not infected
Checking `dirname'... not infected
Checking `echo'... not infected
Checking `egrep'... not infected
Checking `env'... not infected
Checking `find'... not infected
Checking `fingerd'... not found
Checking `gpm'... not infected
Checking `grep'... not infected
Checking `hdparm'... not infected
Checking `su'... not infected
Checking `ifconfig'... not infected
Checking `inetd'... not tested
Checking `inetdconf'... not found
Checking `identd'... not found
Checking `init'... not infected
Checking `killall'... not infected
Checking `ldsopreload'... not infected
Checking `login'... not infected
Checking `ls'... not infected
Checking `lsof'... not infected
Checking `mail'... not infected
Checking `mingetty'... not found
Checking `netstat'... not infected
Checking `named'... not found
Checking `passwd'... not infected
Checking `pidof'... not infected
Checking `pop2'... not found
Checking `pop3'... not found
Checking `ps'... not infected
Checking `pstree'... not infected
Checking `rpcinfo'... not infected
Checking `rlogind'... not found
Checking `rshd'... not found
Checking `slogin'... not infected
Checking `sendmail'... not infected
Checking `sshd'... not infected
Checking `syslogd'... not tested
Checking `tar'... not infected
Checking `tcpd'... not infected
Checking `tcpdump'... not infected
Checking `top'... not infected
Checking `telnetd'... not found
Checking `timed'... not found
Checking `traceroute'... not found
Checking `vdir'... not infected
Checking `w'... not infected
Checking `write'... not infected
Checking `aliens'... no suspect files
Searching for sniffer's logs, it may take a while... nothing found
Searching for HiDrootkit's default dir... nothing found
Searching for t0rn's default files and dirs... nothing found
Searching for t0rn's v8 defaults... nothing found
Searching for Lion Worm default files and dirs... nothing found
Searching for RSHA's default files and dir... nothing found
Searching for RH-Sharpe's default files... nothing found
Searching for Ambient's rootkit (ark) default files and dirs... nothing found
Searching for suspicious files and dirs, it may take a while...
/usr/lib/locale/.keep_sys-libs_glibc-2.2 /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/Date/Manip/.packlist /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/Crypt/SSLeay/.packlist /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/LWP/.packlist /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/Net/.packlist /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/Test/Harness/.packlist /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/URI/.packlist /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/HTML/Tagset/.packlist /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/List/Util/.packlist /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/HTML-Tree/.packlist /usr/lib/perl5/vendor_perl/5.8.8/i686-linux/auto/Pod/Parser/.packlist /usr/lib/perl5/5.8.8/i686-linux/.packlist /usr/lib/.keep /lib/rcscripts/.keep /lib/rcscripts/awk/.keep /lib/rcscripts/sh/.keep /lib/rcscripts/net/.keep /lib/udev/state/.keep_sys-fs_udev-0 /lib/udev/devices/.keep_sys-fs_udev-0
 
Searching for LPD Worm files and dirs... nothing found
Searching for Ramen Worm files and dirs... nothing found
Searching for Maniac files and dirs... nothing found
Searching for RK17 files and dirs... nothing found
Searching for Ducoci rootkit... nothing found
Searching for Adore Worm... nothing found
Searching for ShitC Worm... nothing found
Searching for Omega Worm... nothing found
Searching for Sadmind/IIS Worm... nothing found
Searching for MonKit... nothing found
Searching for Showtee... nothing found
Searching for OpticKit... nothing found
Searching for T.R.K... nothing found
Searching for Mithra... nothing found
Searching for LOC rootkit... nothing found
Searching for Romanian rootkit... nothing found
Searching for Suckit rootkit... nothing found
Searching for Volc rootkit... nothing found
Searching for Gold2 rootkit... nothing found
Searching for TC2 Worm default files and dirs... nothing found
Searching for Anonoying rootkit default files and dirs... nothing found
Searching for ZK rootkit default files and dirs... nothing found
Searching for ShKit rootkit default files and dirs... nothing found
Searching for AjaKit rootkit default files and dirs... nothing found
Searching for zaRwT rootkit default files and dirs... nothing found
Searching for Madalin rootkit default files... nothing found
Searching for Fu rootkit default files... nothing found
Searching for ESRK rootkit default files... nothing found
Searching for rootedoor... nothing found
Searching for ENYELKM rootkit default files... nothing found
Searching for common ssh-scanners default files... nothing found
Searching for suspect PHP files... /usr/bin/find: `/var/tmp/portage/sys-libs/pam-1.0.4/temp/ccWVfwxd.c': No such file or directory
/usr/bin/find: `/var/tmp/portage/sys-libs/pam-1.0.4/temp/ccIHlSUg.o': No such file or directory
nothing found
Searching for anomalies in shell history files... nothing found
Checking `asp'... not infected
Checking `bindshell'... not infected
Checking `lkm'... chkproc: nothing detected
chkdirs: nothing detected
Checking `rexedcs'... not found
Checking `sniffer'... eth0: not promisc and no PF_PACKET sockets
Checking `w55808'... not infected
Checking `wted'... chkwtmp: nothing deleted
Checking `scalper'... not infected
Checking `slapper'... not infected
Checking `z2'... chklastlog: nothing deleted
Checking `chkutmp'...  The tty of the following user process(es) were not found
 in /var/run/utmp !
! RUID          PID TTY    CMD
! 1000        13773 pts/5  -bash
! root        13904 pts/5  su -
! root        14109 pts/5  -su
! root        14290 pts/5  screen -xRR
chkutmp: nothing deleted

Open in new window

0
 
LVL 28

Expert Comment

by:Jan Springer
ID: 24786306
So, did the SVN process mangle your configuration?
0
Microsoft Certification Exam 74-409

Veeam® is happy to provide the Microsoft community with a study guide prepared by MVP and MCT, Orin Thomas. This guide will take you through each of the exam objectives, helping you to prepare for and pass the examination.

 
LVL 6

Author Comment

by:letharion
ID: 24786674
I'm not sure what exactly you ask.
The httpd.conf is not part of any repo, so svn should not have been able to doing anything to it, if that's what you mean.
0
 
LVL 28

Expert Comment

by:Jan Springer
ID: 24786749
I wonder if you're running into an older bug of apache that mangles the httpd.conf.  What version of apache are you running?
0
 
LVL 6

Author Comment

by:letharion
ID: 24786801
>I wonder if you're running into an older bug of apache that mangles the httpd.conf.  What version of apache are you running?

Oh.
I'm gonna check.
Now I can't ssh in anymore... I get in but get 'booted' immediately.
It's "up to date" atleast, I'm fairly certain it's 2.2.11
0
 
LVL 28

Expert Comment

by:Jan Springer
ID: 24786832
If the machine is local and you can get log in, look at your logs that contain login information and updates.
0
 
LVL 6

Author Comment

by:letharion
ID: 24786952
Unfortunately it's not. I'm gonna have to wait until tomorrow
0
 
LVL 6

Author Comment

by:letharion
ID: 24795091
When I got to the computer, I was unable to log in locally too, and since this behaviour was _really_ wierd IMO, I decided to reinstall. As a bonus I got a new and fresh kernel 2.6.29.
I would have loved to dig through the logs and figure out what happened, but I just don't have the time.
0
 
LVL 6

Author Closing Comment

by:letharion
ID: 31600064
Thanks for taking the time :)
0

Featured Post

VMware Disaster Recovery and Data Protection

In this expert guide, you’ll learn about the components of a Modern Data Center. You will use cases for the value-added capabilities of Veeam®, including combining backup and replication for VMware disaster recovery and using replication for data center migration.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Forward apache log to Syslog-NG 7 82
IPA - how do I choose who can log into which servers? 1 70
Weird Samba Connectivity Issue... 7 39
Cpanel file manager 8 22
If you've heard about htaccess and it sounds like it does what you want, but you're not sure how it works... well, you're in the right place. Read on. Some Basics #1. It's a file and its filename is .htaccess (yes, with a dot in the front). #…
In Solr 4.0 it is possible to atomically (or partially) update individual fields in a document. This article will show the operations possible for atomic updating as well as setting up your Solr instance to be able to perform the actions. One major …
This tutorial gives a high-level tour of the interface of Marketo (a marketing automation tool to help businesses track and engage prospective customers and drive them to purchase). You will see the main areas including Marketing Activities, Design …
Established in 1997, Technology Architects has become one of the most reputable technology solutions companies in the country. TA have been providing businesses with cost effective state-of-the-art solutions and unparalleled service that is designed…

776 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question