Solved

Exchange Queues Huge internal spam

Posted on 2010-08-22
59
504 Views
Last Modified: 2013-11-30
our exchange box has queue well over 80,000.  i'm not to familiar with exchange and i'm in dire need of assistance.
0
Comment
Question by:DRSLT
  • 25
  • 24
  • 6
  • +1
59 Comments
 
LVL 76

Expert Comment

by:Alan Hardisty
ID: 33495691
0
 
LVL 76

Expert Comment

by:Alan Hardisty
ID: 33495697
Quick and dirty fix is to block SMTP in and out of your server by blocking TCP port 25 on the firewall until you know what you are dealing with.

If you block it in and out, you will limit the damage.
0
 
LVL 1

Author Comment

by:DRSLT
ID: 33495715
alanhardisty i have been reading your article.  i have done the steps for the auth relay attack, but I am not receiving a 1708 id with any users.  i don't see any postmaster.. here is an example of one of the senders ¤@¶gÅéÅç®ÄªGµL­­" <efqedjyzoud@yahoo.com

it is just a bunch of emails from that type of sender, but they are all different and random

it is exchange 2003.
0
Back Up Your Microsoft Windows Server®

Back up all your Microsoft Windows Server – on-premises, in remote locations, in private and hybrid clouds. Your entire Windows Server will be backed up in one easy step with patented, block-level disk imaging. We achieve RTOs (recovery time objectives) as low as 15 seconds.

 
LVL 11

Expert Comment

by:mattibutt
ID: 33495718
if you go into message queue select 100,00 delete all if they coming from same user block that
0
 
LVL 1

Author Comment

by:DRSLT
ID: 33495741
when i try to delete all the computer freezes and usually has to be restarted.  and it is not same user.. all different with random characters

ok i blocked smtp on firewall
0
 
LVL 11

Expert Comment

by:mattibutt
ID: 33495754
well if your server cant handle try deleting in 10000 groups and see what happen
0
 
LVL 76

Expert Comment

by:Alan Hardisty
ID: 33495758
How many users on your server?

Do you have any anti-spam software installed?

Are all the senders similar to yahoo (not postmaster).  If they are then you are definitely an authenticated relay.

I would download Vamsoft ORF (30-day trial).  Install it in log only mode and then monitor the logs.  You can then see when the sender is yahoo, then cross reference that to your security logs and you should be able to identify which account is being abused.

www.vamsoft.com

0
 
LVL 76

Expert Comment

by:Alan Hardisty
ID: 33495772
You can use my article for details of how to clear the queues and not manually : )Modify the SMTP Connector and change it to use a smarthost not DNS.  Add [99.99.99.99] as the smarthost IP, then change the default retry intervals on your SMTP virtual server to 1 minute and the timeouts etc to 1 minute.  After a while, all the emails will timeout and get deleted.
0
 
LVL 28

Expert Comment

by:sunnyc7
ID: 33495775
You have to use aqadmcli to delete your queue.
Use alans article above
Configure sender authentication on smtp virtual server

You can't delete spam queues with 10000 mails from gui.
You have to use aqadmcli

Also check if you are an open relay
www.mxtoolbox.com
 Enter your domain name and run smtp diagnostics
0
 
LVL 1

Author Comment

by:DRSLT
ID: 33495779
alan i would say around 1500 users

we have symantec endpoint protection, also barracuda spam filter

yes all senders are similar to yahoo.... yahoo.com, pchome.com.tw, hotmail.com, yam.com, etc NO postmaster from what i see

ok thanks i'll give that a shot.
0
 
LVL 1

Author Comment

by:DRSLT
ID: 33495790
220 c3cuda.drs-c3.com ESMTP (9dafa388d8b9b4705576684ad3984ab6)
     

 Not an open relay.
 0 seconds - Good on Connection time
 1.123 seconds - Good on Transaction time
 OK - 207.255.1.80 resolves to c3cuda.drs-c3.com
 OK - Reverse DNS matches SMTP Banner

Session Transcript:
HELO please-read-policy.mxtoolbox.com250 c3cuda.drs-c3.com Hello recover.mxtoolbox.com [64.20.227.133], pleased to meet you [62 ms]MAIL FROM: <supertool@mxtoolbox.com>250 Ok [62 ms]RCPT TO: <test@example.com>550 No such domain at this location (test@example.com) [686 ms]QUIT221 Bye [62 ms]
0
 
LVL 11

Expert Comment

by:mattibutt
ID: 33495794
if i was you i would contact  barracuda spam filter to stress that their filter product has failed miserably but like i said delete all the spam emails if you have 1500 users not sure why you cant delete 100 thousands emails dont freeze them just simply delete them
0
 
LVL 11

Expert Comment

by:mattibutt
ID: 33495799
its also kind of risky to maintain a system for 1500 users if you are not faimiliar with exchange to intermediate level if you solve this problem i would strongly encourage you to train yourself to intermediate level thanks
0
 
LVL 76

Expert Comment

by:Alan Hardisty
ID: 33495813
The Barracuda is not at fault.  Security has been breached because of a weak password.  I was only alerted to a server at a customers site this morning because of an alert on our daily monitoring software telling me of too many invalid login attempts.

Once this problem has been put to bed, you need to tighten up security by enabling account lockout after x invalid attempts, then force strong password and regular changes.  It is a pain in the butt, but better than a repeat of your current problem.

With 1,500 accounts - I won't suggest resetting ALL passwords ; )

Vamsoft will be your best friend and works alongside other anti-spam quite happily.
0
 
LVL 28

Expert Comment

by:sunnyc7
ID: 33495820
To delete messages in all queues
Download aqadmcli from here
ftp://ftp.microsoft.com/pss/Tools/Exchange%20Support%20Tools/Aqadmcli/aqadmcli.exe

aqadmcli "delmsg flags=ALL"

To delete by particular sender
delmsg flags=SENDER,sender=user@domain.com

To delete all messages in a specific queue
aqadmcli "queueaction domain.com, qa=MSGACTION,ma=DEL,flags=ALL"


b) Also check sembee's article here
http://www.amset.info/exchange/smtp-openrelay.asp
0
 
LVL 1

Author Comment

by:DRSLT
ID: 33495825
alan thanks for your help.. we already have acct lockout after x attempts and force strong password and reg changes.

i have installed vamsoft and configured and it is running now
0
 
LVL 11

Expert Comment

by:mattibutt
ID: 33495828
question is after vamsoft is spam stopped you wont see that until whole queue is empty and smpt is enabled
0
 
LVL 76

Expert Comment

by:Alan Hardisty
ID: 33495833
@mattibutt - your comments are not helping.
0
 
LVL 11

Expert Comment

by:mattibutt
ID: 33495845
ok i wont post anymore on this question alan
0
 
LVL 1

Author Comment

by:DRSLT
ID: 33495856
well vamsoft has over 3060 already...

vamsoft.docx
0
 
LVL 28

Expert Comment

by:sunnyc7
ID: 33495873
DRSLT - did you try deleting the queue with aqadmcli ?
0
 
LVL 1

Author Comment

by:DRSLT
ID: 33495882
sunny working on that now.. trying to figure out the command for deleteing a queue..  if i do a delete all will that just delete off my exchange box?
0
 
LVL 28

Expert Comment

by:sunnyc7
ID: 33495905
No. It will only delete messages waiting to be sent in your queue.
0
 
LVL 1

Author Comment

by:DRSLT
ID: 33495910
ok thanks ran it now
0
 
LVL 1

Author Comment

by:DRSLT
ID: 33495914
alan can you give me anythign to go off with that screenshot?
0
 
LVL 76

Accepted Solution

by:
Alan Hardisty earned 500 total points
ID: 33495917
Okay - Vamsoft is busy!
Can you stop the Simple Mail Transport Service service, then wait for 5 minutes, then restart the service.
Reason for this is to give the server a chance to breathe and to allow Vamsoft to start monitoring again and to kill any existing open connections.
Once restarted, any new attempt to send via your server will be logged by Vamsoft and the very first new entry in th elogs can be used to cross-reference your security event log.  From there, you should be able to identify the breached account.
Once identified, reset the password, restart Simple Mail Transfer Protocol service and then monitor Vamsoft again.  Hopefully that should be then end of the spam.  If it is, then tme to clean up using whatever method you prefer although Sunny's suggestion of aqadmcli is probably the quickest (I must update my article).
0
 
LVL 76

Expert Comment

by:Alan Hardisty
ID: 33495922
No. Follow the above advice and cross reference the first new entry in Vamsoft after a 5 minute stop of SMTP service.
0
 
LVL 1

Author Comment

by:DRSLT
ID: 33495926
smtp is stopped.
0
 
LVL 76

Expert Comment

by:Alan Hardisty
ID: 33495937
Coffee time : )
0
 
LVL 76

Expert Comment

by:Alan Hardisty
ID: 33495942
Make sure you record the time when you start SMTP again.
Then anything after that time in Vamsoft / Security Event logs is new and can be used to resolve the problem.
0
 
LVL 76

Expert Comment

by:Alan Hardisty
ID: 33495961
Well - from the logs, you have Saudi Arabia, China and the Ukraine all abusing your server!
Nice to be internationally abused ; )
0
 
LVL 1

Author Comment

by:DRSLT
ID: 33495976
ok i'm going to start smtp here.. in event logs am i looking under Security or Application?
0
 
LVL 76

Expert Comment

by:Alan Hardisty
ID: 33495993
Security - but cross reference the login attempts to the start of the spam you see running through Vamsoft.
So, if the first spam from gmail / yam / yahoo etc as the sender arrives at 18:10:23pm (current time my end), then look in the Security Event Log for a login at that time or just before and then check the user account name.
Change the password for that account, restart the SMTP service (possibly with a 5 minute gap), then see if Vamsoft shows more spam arriving.
0
 
LVL 1

Author Comment

by:DRSLT
ID: 33496045
well it looks like the first acct that was ussed was SYSTEM from the exchange box
0
 
LVL 76

Expert Comment

by:Alan Hardisty
ID: 33496071
Are you sure - that is not an account with a password you can reset as far as I am aware?
0
 
LVL 1

Author Comment

by:DRSLT
ID: 33496080
first message was 13:21:27 and system showed up at 13:21:25 and 13:21:28 and next user didn't show until 13:21:36
0
 
LVL 76

Expert Comment

by:Alan Hardisty
ID: 33496089
What about before 13:21:25?
0
 
LVL 1

Author Comment

by:DRSLT
ID: 33496095
there is this one acct that kept showing... i changed his password and stopped smtp..  i'm going to repeat procedures and see what happens
0
 
LVL 76

Expert Comment

by:Alan Hardisty
ID: 33496102
An account other than System?
0
 
LVL 1

Author Comment

by:DRSLT
ID: 33496108
k just did it and again system keeps showing

13:36:39 first message

13:36:37 - system
13:36:42 - rrock -change his passwd
13:36:54 - System
13:36:55 - erosen
0
 
LVL 76

Expert Comment

by:Alan Hardisty
ID: 33496112
Hmmm.  Not sure what changing the password of System account will do to the server but sounds like it needs to be done.
0
 
LVL 1

Author Comment

by:DRSLT
ID: 33496148
could it be possible that it could be another exchange box in our company that is causing this and not acutally here at our location?
0
 
LVL 76

Expert Comment

by:Alan Hardisty
ID: 33496159
Based on the IP Addresses - probably not.
But - to be sure - can you install Vamsoft on those in Log Only mode and see if they are receiving mail from the same senders and passing it on to you?
Also - you can enable Vamsoft to block and deny access to the IP Addresses that are listed as the senders.
How many Exchange servers?
0
 
LVL 1

Author Comment

by:DRSLT
ID: 33496198
no can't install on other servers.. we have a lot, big corporation.  right now i am going through and getting all the ip's from sender and i'm going to block them in firewall
0
 
LVL 76

Expert Comment

by:Alan Hardisty
ID: 33496223
I would add the entire IP Address block for the ISP, not just the single IP.  You may find others locating and abusing you.
If you need help, let me know the list and I'll let you have the address block.
0
 
LVL 1

Author Comment

by:DRSLT
ID: 33496253
i was just going to block the entire class A.

you recommend differently?
0
 
LVL 1

Author Comment

by:DRSLT
ID: 33496276
ok i have them all in our firewall being blocked.. is there a good way to tell its working?  because our queue is still growing in ESM
0
 
LVL 76

Expert Comment

by:Alan Hardisty
ID: 33496291
Your queue may still grow - they got 130,000 spam into a server I managed with an authenticated relay before I could close it down.
Use Sunny's aqadcmli to zap the queues.
0
 
LVL 76

Expert Comment

by:Alan Hardisty
ID: 33496293
I would lookup the IP address and then note down the address block.  You may block innocent parties if you just select the entire A class.
0
 
LVL 1

Author Comment

by:DRSLT
ID: 33496295
i zapped the queues and they are back to 9200 again.. but this time vamsoft isn't showing anything really happening.. it has 35
0
 
LVL 1

Author Comment

by:DRSLT
ID: 33496300
alan this queue is called Internet-[192.168.100.190] (smtp connector)  is it supposed to be there... i'm thinking maybe not?
0
 
LVL 76

Expert Comment

by:Alan Hardisty
ID: 33496308
Quite normal.  If Vamsoft is showing nothing - nothing new is coming in, just the backlog of stuff due to being overwhelmed.  Perfectly normal.
Let them build up again, zap the queue, let them build up again, zap and repeat until the queue is empty.
0
 
LVL 76

Expert Comment

by:Alan Hardisty
ID: 33496313
What sits at that IP on your internal network?
0
 
LVL 1

Author Comment

by:DRSLT
ID: 33496317
that is our cuda
0
 
LVL 76

Expert Comment

by:Alan Hardisty
ID: 33496322
Then you probably are sending all outbound mail to your Barracuda rather than sending out directly via DNS.
If that is the case, then don't delete or you may not get any mail going anywhere.
0
 
LVL 1

Author Comment

by:DRSLT
ID: 33496352
no we are not sending emails to our cuda.. we send our via DNS.. so it shouldnt be there.

which the leads me to that conclusion.  I just zapped queues and now there is nothing and looks normal operation. and now no more internet queue
0
 
LVL 1

Author Comment

by:DRSLT
ID: 33496364
thanks for all the help Alan.
0
 
LVL 76

Expert Comment

by:Alan Hardisty
ID: 33496371
Good news.  Keep a tight monitor on Vamsoft and the queues.
If it should happen again - you know what to do and to look for.
Vamsoft will stop working after 30 days : (   But - for $239 per server you could keep it if you like it.  Personally - I love it for it's simplicity, it's spam killing abilities and the logs, not to mention the price.
0
 
LVL 76

Expert Comment

by:Alan Hardisty
ID: 33496373
You are very welcome - thanks for the points.
If you get issues again - just re-post in this question and I'll pick up again.
0

Featured Post

Free Tool: Site Down Detector

Helpful to verify reports of your own downtime, or to double check a downed website you are trying to access.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

All of the resources available today make learning a new digital media easier than ever-- if you know where to begin. This is a clear, simple guide to a few of the basic digital art mediums and how to begin learning them on your own.
This article aims to explain the working of CircularLogArchiver. This tool was designed to solve the buildup of log file in cases where systems do not support circular logging or where circular logging is not enabled
The basic steps you have just learned will be implemented in this video. The basic steps are shown to configure an Exchange DAG in a live working Exchange Server Environment and manage the same (Exchange Server 2010 Software is used in a Windows Ser…
The viewer will learn how to successfully download and install the SARDU utility on Windows 8, without downloading adware.

840 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question