Troublshooting IMF...

We have the IMF v2 configured on our Exchange servers and I configured my copy of Outlook to show the SCL of each message.

Recently I noticed that the SCL wasn't being populated so I added a few counters to the performance monitors of our exchange servers and noticed that NO messages are being scanned by the IMF.

I checked that the appropriate reg settings were present, installed the latest IMF update and made sure that the custom XML file was saved in Unicode but still nothing.

Other than adding the counters to the performance monitor and checking the reg settings I'm not sure what else I can do to troubleshoot this.

Can anyone point me in the right direction?


Cheers,
Danny
LVL 1
LFMSupportAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

LeeDerbyshireCommented:
Did you used to have IMF v1 installed?  If so, did you uninstall it before installing SP2 ?
LFMSupportAuthor Commented:
No, the IMF v1 was not installed.

Version 2 was installed as part of the SP2 upgrade.

Danny
LeeDerbyshireCommented:
There are two places that you need to supply configuration information to get IMF v2 working:

1. Define SCL thresholds on the IMF tab on the Global Settings/Message Delivery Properties .

2. Enable filtering on the SMTP Virtual Server. Servers/Server/Protocols/SMTP/Default SMTP Virtual Server.  Click Advanced, click Edit, check Apply Inteligent Message Filter.

Did you check both of these locations?  Many people overlook the second, assuming that it's already enabled.
Angular Fundamentals

Learn the fundamentals of Angular 2, a JavaScript framework for developing dynamic single page applications.

LFMSupportAuthor Commented:
Both of those are set.

This has been working at one point.

Also, it might be worth mentioning that our servers are in a FE/BE config. Do all the servers need IMF enabling on their SMTP connectors or can we get away with just the FE server?

Danny
LeeDerbyshireCommented:
You only need it on your FE server.  Which registry settings have you changed?  I'm not aware of having to change anything in the registry to get IMF working.
LFMSupportAuthor Commented:
The registry settings that I set are the following:

HKEY_LOCAL_MACHINE\Software\Microsoft\Exchange\ContentFilter\ArchiveSCL <<-- This keeps the SCL rating with Archived Spam
HKEY_LOCAL_MACHINE\Software\Microsoft\Exchange\ContentFilter\checkauthSessions <<-- Makes sure even authenticated sessions are checked.

The last one I added as a troubleshooting step to no avail

Danny
LeeDerbyshireCommented:
Are you collecting all your mail from an IP address (maybe at your ISP) that has been added to the Global Accept list in Connection Filtering?

How about temporarily renaming the custom weights XML file (so that it doesn't load it) and then stop/starting the SMTP service.

Which perfmon counter did you add, 'Messages scanned for UCE/sec' , or one of the other ones, 'Total messages assigned an SCL rating of x' ?
LFMSupportAuthor Commented:
We are accepting mail from a number of IP addresses. They are the addresses of our AV provider.

I've actually "lost" the IMF counters from PerfMon so I'm going to reinstall SP2 to see where that leaves me.

Danny
LeeDerbyshireCommented:
The perfmon counters do disappear when you stop the SMTP service, but should start to re-appear when messages start to come into any Virtual SMTP Server that has IMF enabled.  If the counters have not appeared after a long time, than that is a sure sign that something is wrong.
LeeDerbyshireCommented:
And by 'disappear'  I mean they actually disappear from the list of available performance objects, not just the graph.
LFMSupportAuthor Commented:
I've reinstalled SP2 and the counters have reappeared - they did disappear from the list of available performance counters.

I've scheduled the server for a reboot later tonight so I'll check it again tomorrow morning.

Dan
LFMSupportAuthor Commented:
Morning!

The FE Exchange Server still doesn't seem to be scanning emails. One curious thing though... the BE servers are scanning mail fine and in some cases blocking perfectly valid email but thats another issue!

Danny
LeeDerbyshireCommented:
I don't suppose there's any chance that mail is going direct to your BE server(s) is there?  Your MX records definitely point to the FE?
LFMSupportAuthor Commented:
All mail is definitly going to the FE server.

Dan
LeeDerbyshireCommented:
And the FE is configured to relay the messages to a Connector, or an SMTP Virtual Server on the BE?
LFMSupportAuthor Commented:
All that was done to make the server a front end server was the checkbox in server properties was checked.

From what I've read, that is all thats required really. So yeah, I gues it is configured to relay messages to recipients on the BE servers.

Dan
LeeDerbyshireCommented:
In ESM, on the Front-end server.  Do you have an SMTP connector in the Connectors container?
LFMSupportAuthor Commented:
Yes we do, we have one for each site so they can send there own outbound mail and all inbound mail handled by the FE server
LeeDerbyshireCommented:
Right.  And does it have the BE server as a local bridgehead, or is it configured in some other way to send mail to the BE, rather than itself?  I know it doesn't make sense to relay mail to itself, but if it is not going through the FE Virtual Server, then it won't get scanned, because the VS is where you tell it to do the scanning.  If you have a connector, then it won't use the VS.
LFMSupportAuthor Commented:
The local bridgehead server on the FE server is its own SMTP virtual server.

Do I need to put the BE servers on there?
LeeDerbyshireCommented:
I wouldn't suggest changing anything just yet, in case it breaks your setup, but if the server has itself listed in its Global Accept list (Message Delivery properties), then that might make it bypass scanning.
LeeDerbyshireCommented:
One thing you might try.  Enable logging on the FE VS, and make sure that messages are actually going through it.
LFMSupportAuthor Commented:
The FE server isn't listed in the Global Accept list but it's IP addresses are on the Allow Relay list for it's own SMTP server.

The server also acts as a web server and that was the only way to get it sending emails to external recipients...

Danny
LFMSupportAuthor Commented:
I'll set the logging now.
LFMSupportAuthor Commented:
Just logging for a couple of minutes shows a shed load of inbound mail hitting the server. Is there something in particular I need to look for or was that to make sure it was receiving external mail rather than one of the BE servers?
LeeDerbyshireCommented:
It was just to make sure that it was going through there.  The thing is, though, I've never heard of IMF not working.  There may be something in the setup that is upsetting it, though.  It's a long time since I used an SMTP connector, though.  Is there a particular reason that you use the connector like that, instead of just letting the VS handle the incoming mail directly?
LFMSupportAuthor Commented:
When the servers were originally configured that was the configuration MS recommended.

Isn't the SMTP connector only used for external mail while the VS is used for the internal traffic?
LeeDerbyshireCommented:
The VS can handle both, and usually does.  You'd use a connector for special cases, I think, like to an external server that belongs to the same company, but is in another routing group.  You may have something like that set up.  Maybe your VS really is handling all the mail, and only some of it is going through the connector.  Can you tell from the configs what it is doing?
LFMSupportAuthor Commented:
The reason we have a connector(s) is becuase we connect to our AV providers mail servers for sending outbound mail and for receiving inbound mail.

Does that make sense?

Danny
LeeDerbyshireCommented:
I'd have thought you could do that without the connector - but if you've been told to do it that way, then it's best to leave it, I guess.  If the VS is logging SMTP transactions, then they are still going through that anyway, and should be getting scanned.  Have the IMF perfmon counters reappeared in the drop-down list yet?

Do you get any of these Events logged:

http://www.microsoft.com/technet/prodtechnol/exchange/guides/IMFDeploy/55645ac9-28c2-4e63-a7b8-861f8ae6c052.mspx
LeeDerbyshireCommented:
Look for the 7515 event, especially.
LFMSupportAuthor Commented:
The IMF perfmon counters have reappeared and have logged 2 emails scanned all day.

I just checked the event logs and we got a 7514 Event about ten minutes ago. The doc you linked too suggests uninstalling the IMF and reinstalling it - how do I do that, re service pack it?

Dan
LeeDerbyshireCommented:
Yes, I don't think you reinstall IMF now without doing the whole SP2.  You've already tried that, though, haven't you?
LFMSupportAuthor Commented:
Yeah, I tried that last night.

I can always try it again - won't do any harm!
LeeDerbyshireCommented:
Are you sure that IMF v1 was never on the server?  I found a few mentions of the 7514 on the Web, and several of them mention the fact that SP2 was installed before IMF v1 was removed.  Do you see a separate entry for the Intelligent Message Filter in Add/Remove Programs?  Another issue that someone discussed mentions applying a downloaded IMF update from last year that was meant for v1 onto a server with SP2/v2.
LeeDerbyshireCommented:
LFMSupportAuthor Commented:
I already checked the ANSI/Unicode encoding of the XML file - we had that problem previously. One thing I'm unsure of though is the first line of the XML file. We have:

"<?xml version="1.0" encoding="UTF-16" ?> "

but I've also seen:

"<?xml version="1.0" encoding="UTF-8" ?> "

Danny

LeeDerbyshireCommented:
I have UTF-16 , and it works.
LFMSupportAuthor Commented:
it's unlikley to be that then....

I'll install the latest update for the IMF to see if that gives it a kick...
LeeDerbyshireCommented:
Did you check your Add/Remove programs to see if IMF v1 was ever on the server?  Or are you 100% sure that it wasn't?
LFMSupportAuthor Commented:
It wasn't listed in Add/Remove so I can only assume it wasn't installed.
LeeDerbyshireCommented:
I think there was a problem with uninstalling IMF whereby it didn't work if you tried to uninstall it while logged in under a different account to the one that installed it.  It depends on how many different people log into your server, I suppose, and how many other people are likely to install things on your server.

By the way, if the IP address of that AV vendor is in the Global Accept list in Message Delivery Options, then that could very well make all your incoming mail bypass IMF.
LFMSupportAuthor Commented:
You said:

"By the way, if the IP address of that AV vendor is in the Global Accept list in Message Delivery Options, then that could very well make all your incoming mail bypass IMF."

Would I therefore be better adding the AV vendor IP addresses to the virtual server rather than the Global Accept list? Is that possible?

LeeDerbyshireCommented:
I'm not sure that you need to enter it in any list.  It should just come in, and get routed.  Unless you were told otherwise?
LFMSupportAuthor Commented:
I'm thinking from a security POV, we can't have machines on the internet connecting directly to our mail server - only the AV vendor servers.
LeeDerbyshireCommented:
Oh, right.  Then you could add the AV server to the accept list in Connection Control on the Virtual Server (and deny everything else).  I assume that right now you have it in the Accept list in Message Delivery\Connection Filtering?
LFMSupportAuthor Commented:
That's right yeah
LeeDerbyshireCommented:
I don't think that is going to achieve your aim of refusing connections from other servers, rather, it will make the server always accept mail from that particular source.  I would suggest removing it from the list, and adding it to Connection Control in the VS instead, at least as a test for a few minutes.  If the AV server somehow can't get through (because of some other configuration we've overlooked), it will still keep trying to deliver the messages until it succeeds again.  Without the IP addresses in Connection Control, I think you'll find that you're accepting connections from other servers, too.
LFMSupportAuthor Commented:
OK, so to be clear.

I'm removing the AV vendor IP's from the Global Accept list in Global Settings --> Message Delivery and adding them to SMTP VS --> Access --> Connection control

Do I need to add the IP's of the BE servers to the Connection Control settings?

Dan
LeeDerbyshireCommented:
Yes, I would try that.  Write down what you do, so that you can undo it if necessary.  I would add the BE servers, too.  You could always experiment with removing them some other time (when it's quiet), if you like, but I think your BE servers will need to send via the FE.
LFMSupportAuthor Commented:
I'll give it a go and post back the results.

I'll add the BE servers too but they use an SMTP connector to send their own mail so it shouldn't have too much impact.
LFMSupportAuthor Commented:
I switched the AV vendor addresses from the Global Accept List to the SMTP VS accept list and after a bit of fiddling with the XML file (UTF & encoding) the perfmon counters are now ticking up on the Total Messages Scanned for UCE counter.

Thats it. Thats the only IMF based counter ticking over. So whilst mail is being scanned, no mail is being given any SCL rating.

Is there anything else I could try?

Danny
LeeDerbyshireCommented:
One of those Event IDs indicated an inability to assign an SCL

http://www.microsoft.com/technet/prodtechnol/exchange/guides/IMFDeploy/55645ac9-28c2-4e63-a7b8-861f8ae6c052.mspx

Do you see anything like that now?
LFMSupportAuthor Commented:
There are no events in the App log or the Sys log.

Can the event logging be turned up on the IMF?

Danny
LeeDerbyshireCommented:
Not that I know of.  Which counters are you showing, all of these?

Total Messages an SCL Rating of 0
Total Messages an SCL Rating of 1
Total Messages an SCL Rating of 2
...
Total Messages an SCL Rating of 9
LFMSupportAuthor Commented:
Those are the ones, along with the other IMF counters

Danny
LeeDerbyshireCommented:
This is a bit of  long shot, but is there any chance that one of the counters is making the y-axis scale so large, that the individual counters don't appear to have got off the base line?
LFMSupportAuthor Commented:
No, I have the counters in Report Mode.

Worth a try though :)
LeeDerbyshireCommented:
So it is scanning the messages, but not a single SCL has been applied?  And the Archive folder is empty?
LFMSupportAuthor Commented:
indeed...
LeeDerbyshireCommented:
Which folders do you now have under Exchsrvr\bin ?  Mine looks like this, do you have the same version numbers, or anything different?

MSCFv2
  6.5.7765.0
  6.5.7770.0
  6.5.7780.0
LFMSupportAuthor Commented:
I have those same folders and then the DLL, Dat and XML files in the root of the MSCFv2 folder.

I emptied the XML file and restarted SMTP. It logged this error:

Event Type:      Error
Event Source:      MSExchangeTransport
Event Category:      SMTP Protocol
Event ID:      7514
Date:            16/02/2006
Time:            09:44:28
User:            N/A
Computer:      LFM-WEBSERVER
Description:
An error occurred while loading Microsoft Exchange Intelligent Message Filter.  The error code is 0x80070570.


The Event ID is the same as you've mentioned previously but the error code is different...
LFMSupportAuthor Commented:
Also, since I moved the AV Vendor addresses on to the the SMTP VS's I also had to add each server to each other server (does that make sense!?)

We also had relay permission configured for a couple of web servers and had to add those addresses too.

Does that sound right?
LeeDerbyshireCommented:
I think 0x80070570 means ERROR_FILE_CORRUPT .  I wonder which file it refers to, though.  It could be the DLL, or the XML, or maybe something else.  I would try removing the registry entry for the XML file first, and restart SMTP.  If that made no difference, get a copy of Filemon.exe from www.sysinternal.com , and start it logging while you start up SMTP.  See if it reveals any file access errors.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
LFMSupportAuthor Commented:
Which registry entry for the XML file do I remove?
LeeDerbyshireCommented:
There isn't one is there?  I forgot.  Just try moving the file out of the folder while restarting SMTP, and see if you still get the Event 7514 .
LFMSupportAuthor Commented:
OK, we're getting somewhere!

As of the last SMTP Server service restart the IMC has started giving out SCL ratings!

Which services do I need to stop to roll back the IMF to a previous version? I was going to roll back to a previous version and then re run the latest update but it tells me it can't update the DLL becuase it's in use.

Dan
LeeDerbyshireCommented:
There's a description of how to roll back here:

http://support.microsoft.com/?kbid=907747

Basically a regsvr32 and an iisreset.
LFMSupportAuthor Commented:
Did that and rolled back the version but the latest update still tells me the IMF installation is corrupt.... Never mind, I can always post another question about that in the future if I don't figure it out.

Do you reckon if I copy the XML file back to the IMF folder it'll spaz out again?
LeeDerbyshireCommented:
Only one way to find out, I guess.  First, though, I would try opening it in Notepad, and then do a Save As.  Make sure that Unicode format is selected at the bottom.  ANSI upsets it.
LFMSupportAuthor Commented:
Nice one - I can always come mithering again later!

You've been a MASSIVE help, thanks very much

Dan
LeeDerbyshireCommented:
You're too kind.  I still don't feel as though you're completely sorted yet, but I think maybe you could do with some other opinions.  Not many people are going to look at a thread this long.
LFMSupportAuthor Commented:
I agree that it's not quite totally sorted but I'm happy to sort the XML file myself now you got the IMF actually scanning email!
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Exchange

From novice to tech pro — start learning today.