Intrusion detection on windows servers - best practice approach needed


I'm responsible for several windows 2003 servers. After reading some security papers it turns out that I can never find out if a hacker was/is able to manipulate my servers.

This is my setup:

I offer a webservice run on tomcat under windows 2003. This webservice is delivered by a frontend apache server with mod_proxy under Linux debian. In front of the debian and windows server is a zone based firewall with just certain ports open. Both servers are not allowed to connect to the Internet. Tomcat on windows is run as restricted user, with write permissions to the log folder and the application folders.

Given the case a hacker is able to get a shell access on windows and is able to run an exploit to force a privilege escalation granting him administrator permissions. How can I prevent that or is that assumption too paranoid?

Let's say the attacker is after some documents I host on that server or using a connection to our mail server to send a few (spam) emails. Out firewall shuts down automatically in case there are to many connection attempts, so I'm not after a spam proxy monitor.

So I'm thinking about a session security-breach monitoring that is redirected to some loghost that might trigger an alert.
Tripwire might also be an approach, but I didn't find any free solutions for windows.

How do you deal with it?

LVL 27
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Dear Tolomir,

Your assumption is far from paranoid!
I can strongly recommend the Microsoft "Enterprise Security Best Practices":

Kind regards,
TolomirAdministratorAuthor Commented:
great thank you, any other opinions ?
btanExec ConsultantCommented:
A) How can I prevent that or is that assumption too paranoid?
- Malware exploit on application vulnerability so by keeping to latest patch it does help to certain extend but it is not a panacea. Probably whitelisting approach will be a better approach whereby only trusted appl can be run and at least (not always highest) privilege.

There are such incident of escalation, see the links for sharing.

But I felt that the blackhat typically can also go for the approach to capture sys admin (or your superuser) credential by deploying keglogger and thereafter used it to gain higher privilege.

Hence, to shore up security posture, suggest that remote administration (or login) is disabled, all server admininstrator (including other superuser) be two-factor authenticated, audit logs (esp the login, resource related) for server is turn on. Make use of the server security features such as DEP, Applocker (for execution control, close to whitelisting).

B) How do you deal with it?
The current threat landscape has evolved and data ex-filtration is definitely one key agenda item on the perpetrator action list. Server will need to have Host based intrusion detection (and best prevention) capability. Also at network  side, Security information and event management (SIEM) server should be considered to collate and correlate all logs from the firewall, IDS, server etc to heighten the situation awareness - but of course cost will be some challenges.

Probably can check out the below tools
i) OSSEC (Open Source Host-based Intrusion Detection System) -
> It performs log analysis, file integrity checking, policy monitoring, rootkit detection, real-time alerting and active response.

ii) Splunk -
> Splunk enables you to search, report, monitor and analyze streaming and historical data from any source. Now troubleshoot application problems and investigate security incidents in minutes instead of hours or days, monitor to avoid service degradation or outages, deliver compliance at lower cost and gain new business insights from your IT data.

iii) Bothunter -
> BotHunter will help you catch malware infections that go regularly undetected by antivirus systems and completely ignored by traditional intrusion detection systems.

Hope it helps

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Powerful Yet Easy-to-Use Network Monitoring

Identify excessive bandwidth utilization or unexpected application traffic with SolarWinds Bandwidth Analyzer Pack.

btanExec ConsultantCommented:
Just to add on in the areas of protecting document in the server, besides the HIPS, areas such as data loss prevention (DLP) and rights management protection can be another considerations. But they tends to be commercial product, not much free that to my knowledge.

Microsoft has the Software Restriction policy (for Win2K8, it is called as Applocker), RIght Management  System that you may easily google and find the links :)

As for DLP, there are leading products from couple of big player such as McAfee, Symantec, WebSense and RSA.
TolomirAdministratorAuthor Commented:
Thank you,
I will check on Monday.

I only got 2003 available, but I will consider 2008 in the next year or the next version.

I already found out that using windows firewall causes more problems than it's useful: it drops packets sent back from the oracle server occasionally. I don't like that.
TolomirAdministratorAuthor Commented:
Thank you both
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today

From novice to tech pro — start learning today.