Help, hijacked Exchange Server generating spam

I have an Exchange 5.5 sp 3 Server running on NT4 at work that seems to have been hijacked.

When checking a complaint by a user about their mail not getting to the destination in a timely manner, I discovered a IMS (Internet Mail Service) Out Bound Queue full of SPAM (ie the standard "viagra", "get out of debt", "porn chick" stuff, addressed to aol, hotmail etc).

After eliminating all other sources, ie scanning all PC's for viruses, disconnecting parts of the network, etc, I have discovered that the server seems to be creating the spam itself. This is based on the fact that the server is now totally disconnected from the network (ethernet cable unplugged), the queue has been deleted, and as soon as I restart the IMS, within a minute there are a 1000 email messages in the queue.

Its not being Relayed, since now not connected to network and internet connection has been closed. Anti Relay setup has worked in the past, and has not been changed.

I have run a virus scan on the machine, but found nothing.

When I stop the IMS, the messages stop being generated, as soon as I start it again, the messages start being generated.

Any idea's PLEASE !
tes053198Asked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Exchange_AdminCommented:
Stop the IMS.
Rename the IMCDATA folder on all drives to IMCDATA.OLD.
Create an IMCDATA folder wherever you renamed one.
Start the IMS.

Are you sure that your server is configured not to relay?
Double check this.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
tes053198Author Commented:
Thanks for suggestion.

I've done this several times, each time I re-start the IMS service, the "empty" Out folder starts to fill with messages, even though the server is disconnected from the network.
Exchange_AdminCommented:
"I re-start the IMS service, the "empty" Out folder starts to fill with "

Don't just delete the files in the IMCDATA\OUT folder.
Follow the EXACT steps I gave you earlier.

Maximize Customer Retention with Superior Service

The IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy for valuable how-to assets including sample agreements, checklists, flowcharts, and more to help build customer satisfaction and retention.

BembiCEOCommented:
Have look to the other subfoldsers, I'm not sure if Ex5.5. had a queue forlder. Otherwise, let the server run (unplugged), and delete the messages from time to time, until it stops from itself.

Use one of the Relay-Test web site you ensure, that your server is really closed for relay,

see
http://openrbl.org
http://www.ordb.org
http://njabl.org

and others
tes053198Author Commented:
Exchange_Admin, thanks for your assistance so far.

I have followed your instructions exactly.

When I restarted the IMS service the system immediately began to create spam messages in the "out" folder. However after about a minute and 500 or so emails, it stopped. No more messages have been created (note: still disconnected from the internet).

Seems to me that it has simply cleared a backlog of messages.

Now on the relaying issue ...

 - In the Routing tab of the Properties diag of the IMS, I have "Reroute incoming SMTP mail" checked since I have external users and internal users at remote sites over slow links using POP.
 - In the Routing list box I have "Send To" set to our domain name, and "Route To" set to "<inbound>". No other items in the list.
 - Under Routing Restrictions for this I have both the "Hosts and clients that sucessfully authenticate" and the "Hosts and clients with these IP addresses" checked.
I have no IP addresses entered in the list box directly under the "Hosts and clients with these IP addresses" check box.

From my reading of various articles, this should secure the relay.
I have had it tested in the past and found it to be secure using these settings, to the extent that I cannot even redirect mail from "my" users accounts to external email addresses (which I have wanted to do).

If these settings are not correct, please advise where I have gone wrong ???

Cleanup ...

I have been using "Find" and various phrases from the spam mails to identify and delete them from the various copies of "out" folders (slow progress with 48,000 messages).

What should I do with the "valid" messages after I have deleted all the spam messages. Can I simply drop these into the "imcdata\out", or should I stop the service first, remove the "queue.dat" file, drop in the "valid" emails, and then re-start the IMS service ???


Obviously I want to resolve any outstanding issues before I restore the Internet connection, and run the risk of having a repeat of this problem.

Thanks
Exchange_AdminCommented:
"What should I do with the "valid" messages after I have deleted all the spam messages. Can I simply drop these into the "imcdata\out", or should I stop the service first, remove the "queue.dat" file, drop in the "valid" emails, and then re-start the IMS service ???"

Just drag and drop the messages into the IMCDATA\PICKUP folder. Exchange will determine if they are inbound or outbound messages.

Have you checked your Exchange server's relay status by trying to TELNET into port 25 from outside your network?
LadislaoCommented:
I have exactly the same problem as Tes under exactly the same versions of NT4 and Exchange, so far nothing has solved this problem, I have tested my server in open relay test websites and so far nothing, I'm not relaying any email. But even disconnected from the network as soon as the IMC is started it begins to generate about 1 email per second! and the OutBound Messages Queue gets full with 6000 to 7000 emails.

Any other solution is welcomed...

Thanks
danichCommented:
Those messages that were filling the outbound queues; did they have an originator of <>? If so those are NDRs back to the spammers informing them of their failed relay attempt.

One thing I have seen sporadic reports on is that if Guest is enabled then anyone can connect to the IMS; I think this was only for Ex2K though.
LadislaoCommented:
The messages that are filling the outbound queues have an email address as originator.
And I have the Guest account disabled of course.

A MS engineer toll me that I should delete the IMC connector and then re-create it again with teh same configuration, do you think this would solve the problem?

Thanks
BembiCEOCommented:
- In the Routing tab of the Properties diag of the IMS, I have "Reroute incoming SMTP mail" checked since I have external users and internal users at remote sites over slow links using POP.

Yes, in the dialog below, you have set your domain names and everything else is relayed. Relay has nothing to do with POP3. If users connect to the EXCH by POP3 botocoll, they access their mailbox. The option above affects SMTP, not POP3.

- In the Routing list box I have "Send To" set to our domain name, and "Route To" set to "<inbound>". No other items in the list.
See above, this determines, what is handled as inbound, everything else will be relayed.

- Under Routing Restrictions for this I have both the "Hosts and clients that sucessfully authenticate" and the "Hosts and clients with these IP addresses" checked.

This may restrict the relay, as far as you do not use a pop-poll programm to get mails from your provider, this program runs under local permissions and therefore is logged on. If spam is sent to your providers mailbox and you pull it to your server, the sender (for EXCH) is your server itself and therefore relay is aloowed.
tes053198Author Commented:
Exchange_Admin,

Sorry I haven't responded for a while, been busy sorting thru 50,000 odd spam messages looking for the valid ones.

I have re-connected my internet connection and tested the relay thru several relay checking sites, all OK. I am also see-ing attempted relay's being denied in the application logs (I'll logging every thing for a while), and I've tried telnet-ing to port 25. It would all seem secure to me.

My big concern is where did the spam come from (if the relay was secure) ? and how do I know I'm not going to get a repeat, since I've changed nothing ?

I restored the connection to port 80 closed at the firewall (so could use OWA) yesterday afternoon, and overnight got 60 or so messages, all from <test@yahoo.com>, queued in the out folder. I suspect that these were using some kind of exploit of the IIS and OWA.
I've installed the latest cumulative patches for IIS, but I suspect there is still some weakness there, as I have had one such message since.

Also, when moving "valid" messages to the pickup folder, I get ...
"The file LQ59NP1R in the pickup directory could not be parsed to get the originator and  recipients due to error 0x80004005. It either contains no recipients, or it is not a valid  822 message. It will be moved to the Archive directory."
any idea's to fix this ?

danich,

Some did have originator of <>, but mostly (99%) had somename@earthlink.com , hotmail.com, tripod.com, etc as originator, and targetted at same kinds of email addresses.

Bembi,

We are not using a provider other than to carry traffic, mail comes directly to the public IP of our firewall, and is then routed to the exchange server inside our network. Does this change your answer ?
danichCommented:
Deleting and recreating the IMS won't change a thing.

The one about file LQ59NP1R is spurious; indicates the spam message was crafted incorrectly; some spammer trying to be cute resulting in an invalid message. Ignore it.

You said you have the IP addresses checked, but what addresses did you enter?
BembiCEOCommented:
> Does this change your answer ?
1 + 2 NO
3.) If this is the resulting condition: YES. But it depend from your configuration. If you allow relay (you have), you must be sure, that all rules together allow only the relay for users, who have a valid explicit log on to your system. The best way is to use one of the relay check web sites (see my mail above), they will send a few anonymous mails with different recipient formats to your server. If one of the mails comes back, your relay is open. Whenever one of these mails passes your EX, everybody can do it.

I can not see a reason for an open relay. As far as all repeipant mailboxes are on your server (POP3 doesn't matter), there is nothing to realy. One reason for relay may be, if you have a second EX outside your Domain (inside your domain, you can use direct connectors) with active mailboxes of the same domain.
ahmedbahgatCommented:
It seems to me that your server is infected with mass mailing worm, the problem there are some smart worms that will not be detecetd by AV software, I recommend using a variety od different AV software , the best is Trend Micro for Exchange, and the majority offers free online scan

if the problem is not a worm and continue to happen then Exmerge the server mailboxes, export all public folders to pst files using outlook client then format and rebuild "pain in the neck but this will gurantee the elimination of the problem" however you might try to remove the IMS then reinstalling prior to doing the big job


cheers
tes053198Author Commented:
Sorry for taking a while to close off this question.

In the end I hired in a local expert who has secured the server via a number of patches and registry updates, seems it was an open relay after all.

Thanks for your assistance
LadislaoCommented:
Hi tes

I'm still having some problems with my Exchange server and those problems are the same that you solved, can you tell me the patches and registry updates that you applyed?

Thanks tes
fargonCommented:
I was having the same problem with one of my clients and found that someone with an IP address in China had guessed on of the passwords/user names and was authenticating to the server thus allowing him/her to attempt spam which failed and just filled up the outgoing mail queue.  The only way I figured this out was by using a packet sniffer and changing administrative and user passwords.  Once the passwords were changed the Exchange Server 5.5 SP4 on a Small Business Server Version 4.5 started logging the following message.

Event ID: 4183
Source: MSExchangeIMC
Type: Error
Catagory: SMTP Interface Events

Description"
"Authentication attempt (AUTH LOGIN) from xxx.xx.xxx.xxx.xx as \webmaster failed: LogonUser () call failed with error: A required privilege is not held by the client."

Hope that this is of help!
prosewallCommented:
I think that you will find that the problem was an exploit of the IMC that is common to the SMTP connector in W2k and NT4 as well that incorrectly authenticated NTLM authentication in the IMC.  

If a valid account anme was given, ti seem that the IMC failed to even check to see ifthe password was correct.

For Exchange 5.5 this is a post SP4 patch.

The info can be found at
http://www.securityfocus.com/bid/4205/info/
and
http://www.microsoft.com/technet/treeview/default.asp?url=/technet/security/bulletin/MS02-011.asp

IMC version for Exchange 5.5 should be at least 05.05.2655.55 to avoid this problem.
P
tomn2tsrCommented:
For what it's worth, I had a client who had many/all of the stated symptoms.  I performed the following, and so far have had no additional unauthorized UCEs.

I changed the password to a difficult password of the username for the service account for Exchange.

I then changed the Service Account of Exchange itself and created a "difficult" password (combo of caps, lowercase and punctuation).

I applied 3 Post-SP4 patches from Microsoft's website.

I disabled System Attendant, rebooted, renamed ALL imcdata to imcdata-old, recreated imc-data wherever I renamed, reenabled System Attendant, rebooted and finally, all my queues were empty.

I also enable Maximum Diagnostics at the server level on the Directory Service Properties in Exchange Admin for the MAPI Interface Logs.

I have seen some Event Id 4183s now, as a result of the maximum diagnostics, that show someone trying to attach to the server to start some more UCE.

I believe
1)  that the spammers exploited the vulnerabilities in Exchange that were present before the application of the aforementioned fixes.  This stopped the attacks.
2)  Recreating empty IMCDATA folders cleared up my queues.

A number of my remediation steps came from other posts found here on experts-exchange.
KNThomasonCommented:
How do I implement OWA on Exchange 2003 for external access.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Exchange

From novice to tech pro — start learning today.