Solved

Exchange Queues Huge internal spam

Posted on 2010-08-22
59
499 Views
Last Modified: 2013-11-30
our exchange box has queue well over 80,000.  i'm not to familiar with exchange and i'm in dire need of assistance.
0
Comment
Question by:DRSLT
  • 25
  • 24
  • 6
  • +1
59 Comments
 
LVL 76

Expert Comment

by:Alan Hardisty
Comment Utility
0
 
LVL 76

Expert Comment

by:Alan Hardisty
Comment Utility
Quick and dirty fix is to block SMTP in and out of your server by blocking TCP port 25 on the firewall until you know what you are dealing with.

If you block it in and out, you will limit the damage.
0
 
LVL 1

Author Comment

by:DRSLT
Comment Utility
alanhardisty i have been reading your article.  i have done the steps for the auth relay attack, but I am not receiving a 1708 id with any users.  i don't see any postmaster.. here is an example of one of the senders ¤@¶gÅéÅç®ÄªGµL­­" <efqedjyzoud@yahoo.com

it is just a bunch of emails from that type of sender, but they are all different and random

it is exchange 2003.
0
 
LVL 11

Expert Comment

by:mattibutt
Comment Utility
if you go into message queue select 100,00 delete all if they coming from same user block that
0
 
LVL 1

Author Comment

by:DRSLT
Comment Utility
when i try to delete all the computer freezes and usually has to be restarted.  and it is not same user.. all different with random characters

ok i blocked smtp on firewall
0
 
LVL 11

Expert Comment

by:mattibutt
Comment Utility
well if your server cant handle try deleting in 10000 groups and see what happen
0
 
LVL 76

Expert Comment

by:Alan Hardisty
Comment Utility
How many users on your server?

Do you have any anti-spam software installed?

Are all the senders similar to yahoo (not postmaster).  If they are then you are definitely an authenticated relay.

I would download Vamsoft ORF (30-day trial).  Install it in log only mode and then monitor the logs.  You can then see when the sender is yahoo, then cross reference that to your security logs and you should be able to identify which account is being abused.

www.vamsoft.com

0
 
LVL 76

Expert Comment

by:Alan Hardisty
Comment Utility
You can use my article for details of how to clear the queues and not manually : )Modify the SMTP Connector and change it to use a smarthost not DNS.  Add [99.99.99.99] as the smarthost IP, then change the default retry intervals on your SMTP virtual server to 1 minute and the timeouts etc to 1 minute.  After a while, all the emails will timeout and get deleted.
0
 
LVL 28

Expert Comment

by:sunnyc7
Comment Utility
You have to use aqadmcli to delete your queue.
Use alans article above
Configure sender authentication on smtp virtual server

You can't delete spam queues with 10000 mails from gui.
You have to use aqadmcli

Also check if you are an open relay
www.mxtoolbox.com
 Enter your domain name and run smtp diagnostics
0
 
LVL 1

Author Comment

by:DRSLT
Comment Utility
alan i would say around 1500 users

we have symantec endpoint protection, also barracuda spam filter

yes all senders are similar to yahoo.... yahoo.com, pchome.com.tw, hotmail.com, yam.com, etc NO postmaster from what i see

ok thanks i'll give that a shot.
0
 
LVL 1

Author Comment

by:DRSLT
Comment Utility
220 c3cuda.drs-c3.com ESMTP (9dafa388d8b9b4705576684ad3984ab6)
     

 Not an open relay.
 0 seconds - Good on Connection time
 1.123 seconds - Good on Transaction time
 OK - 207.255.1.80 resolves to c3cuda.drs-c3.com
 OK - Reverse DNS matches SMTP Banner

Session Transcript:
HELO please-read-policy.mxtoolbox.com250 c3cuda.drs-c3.com Hello recover.mxtoolbox.com [64.20.227.133], pleased to meet you [62 ms]MAIL FROM: <supertool@mxtoolbox.com>250 Ok [62 ms]RCPT TO: <test@example.com>550 No such domain at this location (test@example.com) [686 ms]QUIT221 Bye [62 ms]
0
 
LVL 11

Expert Comment

by:mattibutt
Comment Utility
if i was you i would contact  barracuda spam filter to stress that their filter product has failed miserably but like i said delete all the spam emails if you have 1500 users not sure why you cant delete 100 thousands emails dont freeze them just simply delete them
0
 
LVL 11

Expert Comment

by:mattibutt
Comment Utility
its also kind of risky to maintain a system for 1500 users if you are not faimiliar with exchange to intermediate level if you solve this problem i would strongly encourage you to train yourself to intermediate level thanks
0
 
LVL 76

Expert Comment

by:Alan Hardisty
Comment Utility
The Barracuda is not at fault.  Security has been breached because of a weak password.  I was only alerted to a server at a customers site this morning because of an alert on our daily monitoring software telling me of too many invalid login attempts.

Once this problem has been put to bed, you need to tighten up security by enabling account lockout after x invalid attempts, then force strong password and regular changes.  It is a pain in the butt, but better than a repeat of your current problem.

With 1,500 accounts - I won't suggest resetting ALL passwords ; )

Vamsoft will be your best friend and works alongside other anti-spam quite happily.
0
 
LVL 28

Expert Comment

by:sunnyc7
Comment Utility
To delete messages in all queues
Download aqadmcli from here
ftp://ftp.microsoft.com/pss/Tools/Exchange%20Support%20Tools/Aqadmcli/aqadmcli.exe

aqadmcli "delmsg flags=ALL"

To delete by particular sender
delmsg flags=SENDER,sender=user@domain.com

To delete all messages in a specific queue
aqadmcli "queueaction domain.com, qa=MSGACTION,ma=DEL,flags=ALL"


b) Also check sembee's article here
http://www.amset.info/exchange/smtp-openrelay.asp
0
 
LVL 1

Author Comment

by:DRSLT
Comment Utility
alan thanks for your help.. we already have acct lockout after x attempts and force strong password and reg changes.

i have installed vamsoft and configured and it is running now
0
 
LVL 11

Expert Comment

by:mattibutt
Comment Utility
question is after vamsoft is spam stopped you wont see that until whole queue is empty and smpt is enabled
0
 
LVL 76

Expert Comment

by:Alan Hardisty
Comment Utility
@mattibutt - your comments are not helping.
0
 
LVL 11

Expert Comment

by:mattibutt
Comment Utility
ok i wont post anymore on this question alan
0
 
LVL 1

Author Comment

by:DRSLT
Comment Utility
well vamsoft has over 3060 already...

vamsoft.docx
0
 
LVL 28

Expert Comment

by:sunnyc7
Comment Utility
DRSLT - did you try deleting the queue with aqadmcli ?
0
 
LVL 1

Author Comment

by:DRSLT
Comment Utility
sunny working on that now.. trying to figure out the command for deleteing a queue..  if i do a delete all will that just delete off my exchange box?
0
 
LVL 28

Expert Comment

by:sunnyc7
Comment Utility
No. It will only delete messages waiting to be sent in your queue.
0
 
LVL 1

Author Comment

by:DRSLT
Comment Utility
ok thanks ran it now
0
 
LVL 1

Author Comment

by:DRSLT
Comment Utility
alan can you give me anythign to go off with that screenshot?
0
 
LVL 76

Accepted Solution

by:
Alan Hardisty earned 500 total points
Comment Utility
Okay - Vamsoft is busy!
Can you stop the Simple Mail Transport Service service, then wait for 5 minutes, then restart the service.
Reason for this is to give the server a chance to breathe and to allow Vamsoft to start monitoring again and to kill any existing open connections.
Once restarted, any new attempt to send via your server will be logged by Vamsoft and the very first new entry in th elogs can be used to cross-reference your security event log.  From there, you should be able to identify the breached account.
Once identified, reset the password, restart Simple Mail Transfer Protocol service and then monitor Vamsoft again.  Hopefully that should be then end of the spam.  If it is, then tme to clean up using whatever method you prefer although Sunny's suggestion of aqadmcli is probably the quickest (I must update my article).
0
 
LVL 76

Expert Comment

by:Alan Hardisty
Comment Utility
No. Follow the above advice and cross reference the first new entry in Vamsoft after a 5 minute stop of SMTP service.
0
 
LVL 1

Author Comment

by:DRSLT
Comment Utility
smtp is stopped.
0
 
LVL 76

Expert Comment

by:Alan Hardisty
Comment Utility
Coffee time : )
0
How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

 
LVL 76

Expert Comment

by:Alan Hardisty
Comment Utility
Make sure you record the time when you start SMTP again.
Then anything after that time in Vamsoft / Security Event logs is new and can be used to resolve the problem.
0
 
LVL 76

Expert Comment

by:Alan Hardisty
Comment Utility
Well - from the logs, you have Saudi Arabia, China and the Ukraine all abusing your server!
Nice to be internationally abused ; )
0
 
LVL 1

Author Comment

by:DRSLT
Comment Utility
ok i'm going to start smtp here.. in event logs am i looking under Security or Application?
0
 
LVL 76

Expert Comment

by:Alan Hardisty
Comment Utility
Security - but cross reference the login attempts to the start of the spam you see running through Vamsoft.
So, if the first spam from gmail / yam / yahoo etc as the sender arrives at 18:10:23pm (current time my end), then look in the Security Event Log for a login at that time or just before and then check the user account name.
Change the password for that account, restart the SMTP service (possibly with a 5 minute gap), then see if Vamsoft shows more spam arriving.
0
 
LVL 1

Author Comment

by:DRSLT
Comment Utility
well it looks like the first acct that was ussed was SYSTEM from the exchange box
0
 
LVL 76

Expert Comment

by:Alan Hardisty
Comment Utility
Are you sure - that is not an account with a password you can reset as far as I am aware?
0
 
LVL 1

Author Comment

by:DRSLT
Comment Utility
first message was 13:21:27 and system showed up at 13:21:25 and 13:21:28 and next user didn't show until 13:21:36
0
 
LVL 76

Expert Comment

by:Alan Hardisty
Comment Utility
What about before 13:21:25?
0
 
LVL 1

Author Comment

by:DRSLT
Comment Utility
there is this one acct that kept showing... i changed his password and stopped smtp..  i'm going to repeat procedures and see what happens
0
 
LVL 76

Expert Comment

by:Alan Hardisty
Comment Utility
An account other than System?
0
 
LVL 1

Author Comment

by:DRSLT
Comment Utility
k just did it and again system keeps showing

13:36:39 first message

13:36:37 - system
13:36:42 - rrock -change his passwd
13:36:54 - System
13:36:55 - erosen
0
 
LVL 76

Expert Comment

by:Alan Hardisty
Comment Utility
Hmmm.  Not sure what changing the password of System account will do to the server but sounds like it needs to be done.
0
 
LVL 1

Author Comment

by:DRSLT
Comment Utility
could it be possible that it could be another exchange box in our company that is causing this and not acutally here at our location?
0
 
LVL 76

Expert Comment

by:Alan Hardisty
Comment Utility
Based on the IP Addresses - probably not.
But - to be sure - can you install Vamsoft on those in Log Only mode and see if they are receiving mail from the same senders and passing it on to you?
Also - you can enable Vamsoft to block and deny access to the IP Addresses that are listed as the senders.
How many Exchange servers?
0
 
LVL 1

Author Comment

by:DRSLT
Comment Utility
no can't install on other servers.. we have a lot, big corporation.  right now i am going through and getting all the ip's from sender and i'm going to block them in firewall
0
 
LVL 76

Expert Comment

by:Alan Hardisty
Comment Utility
I would add the entire IP Address block for the ISP, not just the single IP.  You may find others locating and abusing you.
If you need help, let me know the list and I'll let you have the address block.
0
 
LVL 1

Author Comment

by:DRSLT
Comment Utility
i was just going to block the entire class A.

you recommend differently?
0
 
LVL 1

Author Comment

by:DRSLT
Comment Utility
ok i have them all in our firewall being blocked.. is there a good way to tell its working?  because our queue is still growing in ESM
0
 
LVL 76

Expert Comment

by:Alan Hardisty
Comment Utility
Your queue may still grow - they got 130,000 spam into a server I managed with an authenticated relay before I could close it down.
Use Sunny's aqadcmli to zap the queues.
0
 
LVL 76

Expert Comment

by:Alan Hardisty
Comment Utility
I would lookup the IP address and then note down the address block.  You may block innocent parties if you just select the entire A class.
0
 
LVL 1

Author Comment

by:DRSLT
Comment Utility
i zapped the queues and they are back to 9200 again.. but this time vamsoft isn't showing anything really happening.. it has 35
0
 
LVL 1

Author Comment

by:DRSLT
Comment Utility
alan this queue is called Internet-[192.168.100.190] (smtp connector)  is it supposed to be there... i'm thinking maybe not?
0
 
LVL 76

Expert Comment

by:Alan Hardisty
Comment Utility
Quite normal.  If Vamsoft is showing nothing - nothing new is coming in, just the backlog of stuff due to being overwhelmed.  Perfectly normal.
Let them build up again, zap the queue, let them build up again, zap and repeat until the queue is empty.
0
 
LVL 76

Expert Comment

by:Alan Hardisty
Comment Utility
What sits at that IP on your internal network?
0
 
LVL 1

Author Comment

by:DRSLT
Comment Utility
that is our cuda
0
 
LVL 76

Expert Comment

by:Alan Hardisty
Comment Utility
Then you probably are sending all outbound mail to your Barracuda rather than sending out directly via DNS.
If that is the case, then don't delete or you may not get any mail going anywhere.
0
 
LVL 1

Author Comment

by:DRSLT
Comment Utility
no we are not sending emails to our cuda.. we send our via DNS.. so it shouldnt be there.

which the leads me to that conclusion.  I just zapped queues and now there is nothing and looks normal operation. and now no more internet queue
0
 
LVL 1

Author Comment

by:DRSLT
Comment Utility
thanks for all the help Alan.
0
 
LVL 76

Expert Comment

by:Alan Hardisty
Comment Utility
Good news.  Keep a tight monitor on Vamsoft and the queues.
If it should happen again - you know what to do and to look for.
Vamsoft will stop working after 30 days : (   But - for $239 per server you could keep it if you like it.  Personally - I love it for it's simplicity, it's spam killing abilities and the logs, not to mention the price.
0
 
LVL 76

Expert Comment

by:Alan Hardisty
Comment Utility
You are very welcome - thanks for the points.
If you get issues again - just re-post in this question and I'll pick up again.
0

Featured Post

6 Surprising Benefits of Threat Intelligence

All sorts of threat intelligence is available on the web. Intelligence you can learn from, and use to anticipate and prepare for future attacks.

Join & Write a Comment

This process describes the steps required to Import and Export data from and to .pst files using Exchange 2010. We can use these steps to export data from a user to a .pst file, import data back to the same or a different user, or even import data t…
Learn to move / copy / export exchange contacts to iPhone without using any software. Also see the issues in configuration of exchange with iPhone to migrate contacts.
The viewer will learn how to create multiple layers to apply various filters and how to delete areas from each layer’s filter.
An overview on how to enroll an hourly employee into the employee database and how to give them access into the clock in terminal.

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

9 Experts available now in Live!

Get 1:1 Help Now