Brute force attack on SBS 2003 - receiving hundreds of 529 errors in security log, no source IP address listed

Dave Messman
Dave Messman used Ask the Experts™
on
I've read through other EE answers and done some research to no avail so far.

I have an SBS 2003 box set up with an third party firewall with the regular ports forwarded - 4125, 443, 995, 25, 1723, etc.  I had port 80 open - but closed it when this started to see if it would help - it did not.  The server has been pretty solid for years.

For the past 90 minutes, I've gotten hundreds of 529 errors - invalid logins.  If the event log showed me the source IP address they were coming from, I'd black that IP address on the router, but the event log doesn't show me the IP address.  If there's another place to find it, I don't know where it is.  Any advice is appreciated.

Here are some examples of what I see in the event log:

EVENT #      86664586
EVENT LOG      Security
EVENT TYPE      Audit Failure
SOURCE      Security
CATEGORY      Logon/Logoff
EVENT ID      529
USERNAME      NT AUTHORITY\SYSTEM
COMPUTERNAME        Servername1
DATE / TIME        1/28/2010 9:15:10 AM
MESSAGE      Logon Failure:
Reason: Unknown user name or bad password
User Name: inna
Domain:
Logon Type: 3
Logon Process: Advapi
Authentication Package: MICROSOFT_AUTHENTICATION_PACKAGE_V1_0
Workstation Name: Servername1
Caller User Name: Servername1$
Caller Domain: Domain
Caller Logon ID: (0x0,0x3E7)
Caller Process ID: 3188
Transited Services: -
Source Network Address: -
Source Port: -

EVENT #      86658445
EVENT LOG      Security
EVENT TYPE      Audit Failure
SOURCE      Security
CATEGORY      Logon/Logoff
EVENT ID      529
USERNAME      NT AUTHORITY\SYSTEM
COMPUTERNAME        servername1
DATE / TIME        1/28/2010 8:38:19 AM
MESSAGE      Logon Failure:
Reason: Unknown user name or bad password
User Name: administrator
Domain:
Logon Type: 3
Logon Process: Advapi
Authentication Package: MICROSOFT_AUTHENTICATION_PACKAGE_V1_0
Workstation Name: Servername1
Caller User Name: Servername1$
Caller Domain: Domain
Caller Logon ID: (0x0,0x3E7)
Caller Process ID: 3188
Transited Services: -
Source Network Address: -
Source Port: -


________________________________________
EVENT #      86656220
EVENT LOG      Security
EVENT TYPE      Audit Failure
SOURCE      Security
CATEGORY      Logon/Logoff
EVENT ID      529
USERNAME      NT AUTHORITY\SYSTEM
COMPUTERNAME        Servername1
DATE / TIME        1/28/2010 8:09:48 AM
MESSAGE      Logon Failure:
Reason: Unknown user name or bad password
User Name: admin
Domain:
Logon Type: 3
Logon Process: Advapi
Authentication Package: MICROSOFT_AUTHENTICATION_PACKAGE_V1_0
Workstation Name: Servername1
Caller User Name: Servername1$
Caller Domain: DomainCaller Logon ID: (0x0,0x3E7)
Caller Process ID: 3188
Transited Services: -
Source Network Address: -
Source Port: -
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Top Expert 2013

Commented:
Most important is do you have password lock out policies and forced complex password policies set? i.e. if they choose a proper user name, they should be locked out with 6 wrong guesses for at least 15 minutes.

Do you use port 995? i.e. do remote users POP the e-mail from Exchange? Possibly, but that is not a common scenario, you may be able to close that.

Are you sure you do not have 3389 open? That is a common target.

inna account may indicate they are targeting Exchange.
Make sure you are not an open relay:
http://www.amset.info/exchange/smtp-openrelay.asp

Your router may have more logging capabilities that would allow you to locate the source IP.

Do you have a back up MX/mail service off site? If so I would close all ports for 20+ minutes. They usually go away.also make sure the router does not respond to Internet  (ICMP/ping) requests. This is how they often locate you.
Dave MessmanIT Consultant

Author

Commented:
there are many helpful things in there - I've adjusted the security policy for 6 wrong guesses and 15 minute lockouts, and I did that hours ago, but the it seems like the policy changes haven't taken effect yet as the same users are still unsuccessfully authenticating over and over.  I'll reboot the box tonight to see if that makes it take effect.  This has been going on for over 6 hours now - with continued failed attempts.

But what I'd love to do is fine the source of the attacks.  I'm nearly positive these are coming from RWW.  Port 3389 is not open and we're not an open relay.  This should be IIS logs somewhere with this info, right?  Where would that be?  It's not in C:\WINDOWS\system32\LogFiles\W3SVC1\ex100128.log.

Where else could I look?
Top Expert 2013

Commented:
-The only place I have seen the IP logged (before a connection is completed) is in the event viewer. Usually you will see it as an even a few before or after the 529 entries
-They can guess forever with a wrong user name, you can't stop that. The policy will only block them if it is a legitimate, existing user name. The policy should be enforced 5 minutes after you made the change. Did you make it in a policy that affects the server. You can test by running GPResult
-You should be able to get more logging information is from your router. You may have to add a PC running syslog software.
What make and model is the router?

Actually you should be able to capture it if you were to install wireshark on the SBS and capture packets.
http://www.wireshark.org/download.html
How to Generate Services Revenue the Easiest Way

This Tuesday! Learn key insights about modern cyber protection services & gain practical strategies to skyrocket business:

- What it takes to build a cloud service portfolio
- How to determine which services will help your unique business grow
- Various use-cases and examples

Dave MessmanIT Consultant

Author

Commented:
you're right.  When they were guessing with the wrong usernames, they just kept guessing - but when they guessed "root" - which I do have as an account - it did lock them out - so that is working.

My router/firewall is a Fortigate 50B - which as far as I can tell has relatively difficult to use logs.  I'll investigate those further.

In each of the brute force attackers attacks, it says caller process id is 3188.  I wonder what that is.  I tried to replicate it.  I logged into RWW with an invalid password and that event log showed caller process 11056.  And then I did an invalid VPN login and that came back as 1128.  Another thing to think about is what they are attacking and shutting that down . . .
Top Expert 2013

Commented:
I belive process ID 3188 is world wide web service.
You did close port 80 did you not?
Top Expert 2013

Commented:
I am quite certain the Fortigate would have logging capabilities, they are good units, however I am not familiar with them.
Dave MessmanIT Consultant

Author

Commented:
I did close port 80.  Tonight, I'm going to completely close the server to the outside world for a bit - to see if the hacker loses interest.  And then I'll enable the firewall policies one by one - and see which one he is exploiting.  I presume it's 443.  
Top Expert 2013
Commented:
I would get it disconnected as soon as possible. If compromised you will never be able to trust it again without a clean re-install. Do you need a hand with configuring the logging? I may be able to call upon a fellow Expert familiar with the Fortigate.
Dave MessmanIT Consultant

Author

Commented:
wow - that's a generous offer - if you can help me with Fortigate logging, that would be helpful.  There have been thousands of attempted logins today.  I want to do it ASAP - but of course it's a production box, so I need to be wary of my users.  I'm taking it down at 9:30 tonight.  At least, if they do find a legit username, they got locked out relatively quickly.
Top Expert 2013

Commented:
I'll ping a Fortigate guy and see if he can help. He is likely better with security issues than I as well.
Commented:
I manage numerous SBS networks and I see this all the time. If you allow legitimate users to log in from the Internet, malicious users will attempt same.

The record for failed logins in one night was 14,000 or so. If you have an external firewall or ISA, you can block the IP, but the novely of that wears off quick. There are just too many robots out there probing for vulnerable systems.
Dave MessmanIT Consultant

Author

Commented:
For the moment, things are resolved.  I disabled the ports at the firewall level one by one.  Interestingly, it was port 25 that the attacks were coming over.  I probably got 5000+ attacks from 8 am to 9:30 pm.  After blocking port 25 from all internet for 45 minutes, the attacks ceased when I opened port 25 again.

I wrote to the Fortinet people about how to look at their logs.  It seems like they want you to buy their separate appliance - fortianalyzer - to look at the logs.  The real solution here, of course, is to block the IP address of the attacker - so that's the course I'll pursue once log analysis techniques are made more clear.
Top Expert 2013

Commented:
If they were hitting port 25, as inna account suggested earlier, I would make sure you are protected against being a mail relay as per the link provided earlier.

Can you not have the Fortigate forward the logs to a syslog server? If so you can get free software to set that up on a PC.
As far as blocking the IP's that is near impossible they change IP's numerous times a day. I know a fellow that managed a large network and he had scripts to capture the IP's and he would add them to a blocked list every day. He said he would add 2 to 20 per day. It is a never ending battle.
As SMBGUY said most servers with open ports get hit 3 or 4 times a week, though I find the attacks most often try 3 common accounts like administrator with about 100 hits and then move on. I must say though the ones that are most often hit are those with 3389 open and those that have routers that reply to internet pings.

Haven't heard from the Fortigate fellow yet, but he has some new projects on the go and may be quite busy.
Thanks dmessman. And hopefully things will remain quiet.
Cheers!
--Rob

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial