Link to home
Start Free TrialLog in
Avatar of barryberkov
barryberkov

asked on

Duplicate messages received in Outlook 2003

There is some kind of issue with Outlook 2003 repeatedly downloading the same message.   We leave messages on the POP server and with Outlook Express, there is no issue.   However with Outlook 2003 certain messages get downloaded repeatedly.  I've read through the knowledge base articles and some related stuff here but have not yet seen a good explanation or solution when the user has multiple POP3 accounts set up.   Note also that we have installed the service pack for Office.   Prior versions of Outlook don't seem to have the same problem or to the same degree.  Any comments or suggestions would be appreciated.

BFB
Avatar of sunray_2003
sunray_2003
Flag of United States of America image

Ever since you started using outlook 2003 and this POP3 account , you got duplicate messages or recently it started ?

Have you tried to recreate the email account and did it help ?

Also try to recreate your profile and see if there is any change

SR
Hi sunray_2003,

Sunray, we had simular problem and used the MS Tech article to solve.
http://support.microsoft.com/default.aspx?scid=kb;en-us;284404

God Bless
Avatar of barryberkov
barryberkov

ASKER

The reason this is a difficult question is that 1) I did recreate the email account and the problem came back and 2) I need to check both POP3 accounts so turning off one as suggested by the knowledgebase article isn't a good option.   What's particularly weird is that it appears as though not all messages get duplicated -- only certain ones.   I'm surprised that there haven't been more reports of this since in our company, we got three new computers with Office 2003 and everyone started having this problem to varying degrees.  

BFB
BFB,

Do you see a pattern in what type of emails get duplicated .. whether emails having attchments or email coming from certain domain names ?

Does getting duplicated emails random or whenever that specific type of email come through , it gets duplicated ?

SR
Are you sure that one of the other accounts don't  forward based on server side rules, or client rules??

I had this once with a new client who had a 9 fold email replicaiton issue,    Problem was the previous consultants setup each to forward to another,   Was STUPID!!  but none the less, problem solved...

Try to Download from one at a time, see if the same message comes from the same server, or another...

Secondly, see if the account removes messages from the server and also if it leaves copies, what does it do with them....

Have a good one, hope this helps to quech the fire...

CW
I'm not sure anyone is going to be able to solve this -- as it really seems to be a serious bug in Outlook 2003.   I deleted the mail account and recreated it.   Once it downloaded all the messages left on the server, the problem started up again -- some messages get downloaded multiple times while others do not.  in Outlook Express, there is absolutely no problem.

Barry
ASKER CERTIFIED SOLUTION
Avatar of RRRRyan
RRRRyan

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
This problem also occurs on OL 2002 even with SP3 which is the recommneded fix.

I recently installed win XP + SP2 and the Office XP 2002 + SP3 on three PCs all suffer with the same problems.

I run several E-mails some on same POP3 and some separate, there is certainly a pattern but I cannot get my head around it.

I have noted the following.

1. I imported my .PST files from OL 2000 - which had previously worked well (Never impecably)

2. All accounts are set to Leave on server for 5 days

3. Some e-mails were not been deleted by OL - even though they have timed out, these are the ones that keep getting downloaded. Once deleted manually (using the Mail2web service)they do not get re-downloaded.

4. I run my collect every 10 mins and after one duplicate download, then all seems OK ......... that is, until the NEXT DAY IS CLOCKED UP, then the problem starts again.

5. The problem seems to have started with the inclusion of "concurrent e-mail collection" which was not in OL 2000, whether there is any relationship I do not know.

6. This issue has been around a long time and is a serious one, which MS do not appear to have either recognisied or properly diagnosed. I have seen many repors on this on the internet but none have come up with an adequate solution.

7. How does one bring this to the attention of MS (disabling an account is not acceptable when running a business with multiple client contact addresses)  



Wow, I just can't believe that this problem is completely unfixed and apparently unrecognised by MS - is the status of this issue still the same? Are there any workarounds?

We have a client now who's secretary is running Outlook 2003 attaching to a Unix based POP3 mailbox and is receiving multiple copies of the same email, sometimes up to 15 copies of each one, all with the same message ID. She already receives loads of email, so multiple copies are a real pain. Outlook is set up to use a single POP3 account and this problem has only started just this week after about two weeks of plain sailing since installation. Patching with the latest Office updates has not fixed the problem and messages are not set to be left on the server. Would it be worth playing with any profiles or POP3 status files to see if we can get back to the good ol' days?

Zyxel - your notes are very good - I agree with your opinion that messages stay on the server and are downloaded multiple times maybe as a result of the "concurrent e-mail collect" not locking or terminating correctly. Is there even a hack to disable this 'feature'.

All the best,

R
This has subsided somewhat for me. I still have two machines checking separately and leave the messages 3 days on the server. The phenomenon went away without explanation. I have recreated the profiles on both machines and deleted and recreated the account on the linux server. On occasion we receive an email that keeps downloading but it is tied to a specific email and once deleted from the server goes away. It is completely a mystery to me, one that I'm happy to say is not haunting me at present.
To follow up on my previous posting .......

Recently I did a complete system re-install which included .........

1. XP + SP2
2. Office 2003 +SPs
3. A completely new PST files for Inbox and Archive/backup

and let this run for a few days (I have about 12 POP3 accounts many sharing the same Server / Domain name)

This worked fine ................... So, I (concurrently) opened my previous PST file and "moved" all e-mails to the new PST file.

This was 3 weeks ago and "so far" everything is working fine.

I do get a few duplicates on one account ..... but ONLY from one sender (as part of a daily mailshot) so I am assuming that there are infact two copies being sent. Otherwise "touch wood" all is well.

(This compares with, typically, dozens of new, plus all the ones, duplicated, that I had just deleted !!! a real mess)

There is one well known underlying problem with POP3 accounts ..... there is no mechanism to flag up when an e-mail has been download (an left on the server) and so we must rely on Outlook to keep an accurate record.

My "feeling" is that if your PST gets corrupted, in some obsure way (which Outlook cannot fix), then this mechanism is screwed up and the only solution is to do as I did, creat a new PST file from scratch and then move wanted (historic) e-mails to that PST file. The moved e-mails are either cleaned or the corrupted info is stored in some other part of the PST File.

I did use a duplication file remover but I think this actually agravated the problem
I was also using a SPAM removal program
Again "A feeling" is that as these are moving e-mails around the PST file "links or records were getting corrupted"

Another issue, I noticed that the E-mail download was being slowed down or even stalling, then timing out so it was not completed by the time the next download was initiated, at such time, Outlook was trying to do two or even three concurrent downloads of the same information ........ This I think was a major problem.

More on this is a moment

Now that I've reinstalled every thing the full download takes no more than a couple on minutes (I often have large attachments) and typically 10-15 seconds when none ..... so definitely no overlap.

WRT to the Slow / stalled Downloads I was also having a lot of problems with DNS lookups in Internet Explorer, which showed up as either corrupted, incomplete, or slow to load pages ..... I never go to the bottom of this and perhaps something within Windows was screwed up. (With the new install all is fine)

I did install a local DNS cache on a small stand alone PC to work for my local network, this seemed to help, but did not fully cure the situation.

With the new re-installation Web page down load is now near instantaneous (512k DSL) (I am still using my local DNS server) but the DNS lookup also works for the E-mail send and receive once it has cached a domain name. (You should not need this as it is simply "icing on the cake" for internet speed)

For now, I am keeping my "fingers tightly crossed" that the problem has gone away.

I hope these notes are of some use and help you sort out your issues.

P.



 






A couple of addition observations

WRT: "Another issue, I noticed that the E-mail download was being slowed down or even stalling, then timing out so it was not completed by the time the next download was initiated, at such time, Outlook was trying to do two or even three concurrent downloads of the same information ........ This I think was a major problem."

This was confirmed to some degree, when I disabled the Automatic download and left the PC time to full complete e-mail downloads. When I did this there were no longer concurrent downloads from the same e-mail account and the duplication of e-mails was reduced considerably, however, it did not stop it happening again the next day. So Outlook was still not updating the Date Log ..... Some what confirmed by .....

I also noted that e-mails older than my five day time-out were not getting deleted from the server and could start to appear several days later. To sort this I had to use Mail2web and delete the emails off the server.

Again this appears to be a "E-mail status tracking problem".

Thinking back, I think that the PST file got corrupted due to the way Outlook closes down, in that it does not do a propper "finish what it's doing, close the PST files and then Exit" ..... It just exited, even if in the middle of an E-mail Download.

I was using the ICON on the lower right as an indication that the E-mail Download was completed, whereas in reality it was NOT ...... I have a little ustility (DU Meter) that monitors the volume of data passing over my LAN connection and I could see that Outlook was still uploading / downloading e-mails long after it said it had finished.

Probably, repeated shutting down of the open PST files by Outlook had messed them up big-time.

I have now set Outlook to do an E-mail "Send/Receive on Exit" ...... seems to force a nice clean PST closure and Outlook Shutdown is properly controlled.

To summerise:

There appear to be many symptoms associated with OUTLOOK duplication, but the underlying issue related to Outlook seems to lie in corrupted PST files (all of the other Fixes are "red-herrings").

1. If possible reinstall Windows and Outlook (or at least Outlook)

2. Creat new PST files (for Mail and Archive)

3. Creat new E-mail Accounts

4. Check that your Internet access works Well and no obvious issues with DNS lookup (Slow / corrupted web pages etc)

5. If needed "MOVE" wanted e-mails from Old PST files to New PST file

6. Minimise the size of PST files by regular Archiving. Large PST files are slow to load and nore likely to get corrupted. I suggest to Start a new Archive PST if greater than 500Mb.

7. Set Outlook to  do an E-mail "Send/Receive on Exit" (this is in the Tools/ E-mail options)

8. Do NOT collect e-mails more frequently than 15 mins (DSL) (possibly longer for Dial Up)

9. Install DU Meter to check that E-mail Send/Receive has actually stopped when Outlook says it has and before Exiting Outlook.

I think that is all and should give some serious clues as to what is really going on.

It certainly worked for me ...... after I'd exhasted ALL of the other "Fixes" offered by MS and everyone else.

P.


 
Thanks for the feedback RRRRyan and zyxel...

Some of the symptoms you mention are certainly happening on the client's machine - such as "E-mail download [] being slowed down or even stalling", there are problems shutting down Outlook and errors with the mail received icon.

Can you just confirm how you disabled "Automatic download"...? Do you mean that you disabled automatic send and receive? (We have no copy of Outlook 2003 to play with at this office - (un?)luckily?!) I certainly like your idea of doing a send and receive at the end to clear everything up... Is there any way we can turn off the concurrent 'feature' of Outlook?

I know that the client secretary gets annoyed when things don't work at FULL SPEED! If she finds that Outlook is slow, she shuts it down and starts again (corrupting the PST mid-download?)... and then when it comes back up again it can obviously have trouble working out which files it has seen and which it hasn't, starting the whole duplication behaviour.

Anyway, at least there is some work in there for the company... it just doesn't look good for us if we can't fix *every* problem... It can be pretty difficult to explain that there's nothing we can do about M$'s programming!

ps. I think that zyxel should get some points for the massive and informative answers... anyone out there with the points stick want to beat zy with it?

J
Hi Again,

To Clarify .... (I was getting a bit brain dead by the time I wrote my notes)

1. YES, For Automatic downlaod ...... I meant Automatic Send / Receive

    I don't know any way to turn off the concurrent feature (it was added after OL2000 to speed up downloading) - I don't belive this is a fundamental issue, there are other demons at work.

2. The Issues are for Outlook 2000, through 2003 so should apply to all versions (they all use the same PST file system ...... I will try this out on 2002 (which is installed on another PC) using the same "problem PST files" which I have kep safe on my server for further investigation)

3. Tell the secratary ..... NEVER NEVER EVER TERMINATE OUTLOOK DURING DOWNLOADS ..... this is what appears to cause the problem in the first place. If downloads are slow, then the problem is with Windows Internet Settings and MUST be fixed first, even if it means a clean install of Windows (which is not a bad thing from time to time) as files get updated, and corrupted and setting get changed (often with no user intervention)

4. Tedious as it is I am inclined suggest that you do a clean install of windows & office / outook. I suggest XP AND SP2 plust ALL incremental fixes and Outlook plus all fixes.

For simplicity, I suggest you simply remove the existing HD and keep as it is. Put in a new one (they are now so cheap) ............ and then this can be set as a slave and important files copied across onto the new drive.

Most people don't really need more than 40 Gb ..... Split this into TWO equal partitions.

All working files should, in all seriousness, if company critical, be saved on a (remote) RAID 5 server ..... this also minimises the build up of crap on the local H/D.

Once setup DO NOT USE THE OLD PST FILES, but set up a completely clean install of outlook .... PST, E-mails, Rules .... everything. And do all the checks I suggested above. If all is well - run it for a few days at least, if all is well BACKUP the NEW PST file (exit Outlook and simply copy the files to a safe place) Then open Outlook and then additionally open the OLD PST file(s) and move all important e-mails to the NEW PST file. Continue to run Outlook, if all is well ..... your a winner. This is a bit tedious but it does seem to wok.

5. FINALLY get a copy of SECONDCOPY2000 ..... $10
and set this up to backup the PST files to the second partition allong with PREVIOUS REVISIONS of the PST .... say 5, so should a problem arise, you can copy-back an earlier version and just re-download recent e-mails ....... this is how I do it.

There is also a Nice little BACKUP UTILITY that can be added to OUTLOOK to do this for you, but (I think) is only available for OL2003 .... I'll check. This works in the background and will do it once every day. .......... However, still use SC2000 to make historical BUs, I have installed this but as yet not used it in anger as I prefere SC2000.

Incidently SC2000 can be used to BU all other work files to a safe place, another drive, LAN location, Server etc. I've been using it for a few years and it is absolutely reliable, highly configurable and extremely easy to set up ..... use the advanced setting.

Finally .......... please don't beat me with a stick ...... It might hurt ;-) .............

and besides, I've seen so many people with this problem on various BBs it seems only fair to try and put a few people out of their misery, (like mine) ..... I just hope MS get a handle on this and fix it once and for all ..... or a least put out a buletin explaining what is really going on.

Personaly I think MS wish we'd all migrate to Outlook Express, and only use Outlook for its Server / IMAP applications. But Hey ! why should we throw away some very usefull features that are NOT included OE.

Here endeth the forth lesson .............. !!!!

P.





Great info again, Zyxel!

I'm just going to respond to your points one by one and let you know what I think with reference to the client's secretary's machine (OL2003).

1. Automatic send and receive now turned off - had been set for 5 mins. I wonder whether OL starts more downloading threads if the first ones haven't finished and the time out is complete... hmm.

2. We are working with OL2003 here (fully patched) on Win2K SP4.

3. Just phoned the secretary and shouted "NEVER NEVER EVER TERMINATE OUTLOOK DURING DOWNLOADS" down the phone as requested. (I thought that an email was not going to be direct enough - even if it did get received fifteen times!)

4. Regarding a clean install of Office AND OL... I'm slightly concerned in this case that this is overkill. This windows install is only about a month old - the secretary corrupted her registry and we were not able to recover it for less than the cost of a full re-install. I have however, scheduled a cleaning of Office as you suggest.

5. Points about backups noted - we have a system in place, but haven't dug into the saved PST files as yet.

An extra point - As soon as it's receiving email OL hangs - you can't shut outlook during downloads... The mouse cursor disappears, the screen stops refreshing - standard windows locking behaviour. This can, apparently also effect Word - which is strange.

My main concern is that I could send someone to work on this machine all morning - clean everything down - leave it working fine... Then, something happens strange with the network connection or the server, connection is lost and OL corrupts/gets confused again and we are back to the same situation. See, the issue is that we don't have a fix - just a way of working around... and since we don't know what starts this behaviour, it would be irresponsible to charge a client to put the system back to somewhere where it could be ruined again. It's just a shame, because, as you say, OL does have some nifty stuff that the client likes.

Anyhow, thanks for all the info - I hope that M$ fix this soon - I know that their techsupport people don't even have support documents/resources for this problem, so it's pretty deep rooted.

J
Hi Again Again Again ............................ Again !!! :-)

A few (brief) comments (WRT your para nos.)

1.  5 mins is definitely too often (I recall a note from MS to this effect somewhere and from my own experiences it is as well) and yes it will start new threads if the previous Send/Receive sequence has not finished and it just gets worse as it can't keep up ..... I've seen over 50 concurrent threads before now ........... Arrrgggggg !!!

2. GoooooooooooodD

3. Excellllllllllllentttttt ............. "the man's got balls !!!"   ............ "Errr Yuh .... so what's new !!"  (no ladies watching I hope)

4. Ah .... OK that should be OK, in this case just create new PSTs, Rules and E-mail accnts
and if ok then move from OLD pst to NEW pst (as stated earlier)

5. Nice ;-) ...... many don't  ........ the BU's should help to quickly "rewind" if Seccy does something silly .... etc.

OL should NOT hang the PC even if slow ...... something IS screwed up here ..... needs investigation , NOTE: however if PSTs are BIG they can take a while (minutes) to reload after OL has been terminated incorrectly (for any reason) Another reason to Creat new PSTs. and Achive OLD e-mails ..... This causght me out and Guess What I used to "Kill" OL until I realised the effect.

OL can (optionally) use WORD for editing and if OL is playing up then it Locks up Word as well (not smart) ...... you can switch OFF this option.

Just to finally re-itterate (this should take about an hour)

1. Create new PSTs and copy only e-mails across

2. Minimise the size of the PSTs (for faster loading) - normall the file is indexed and quick but once i is broken this has to be re-indexed and can take upto 5 mins, during which time everything grinds to a halt.

As, no matter what else, the PSTs MUST be cleansed

Cheers

P.
 
 

Hi guys & gals,

I have just proved the problem & my suggested fixes .................. it is definitely corrupted PST files caused by Outlook Crashes

Today it all happend (according to expectations) ........... Arggggg .........  but Hey ...... 5 mins and I'm "back to Good" again ............... It's nice to be able to prove my own theories !!!

Sequence of events

1. I added a new e-mail account (one I'd realised was forgotten after the last fixes)

2. It started to download all the new e-mails from the new address and for some odd reason partway through Outlook Hung (I had oppenend a new e-mail and some other things were active) anyway outlook created an error message and closed abruptly part way through the Send/Receive sequence.

My gut feeling was that the e-mail I opened (which was a bit dubious)  tried to do something suspect (This one account DOES NOT HAVE A/V or Spam filtering at the server end) so maybe there was some dubious HTML code / attachement ...... I am not going to risk opening it again to find out !!!

3. I restarted OL and (guess what) ................. Yep !  ......  Duplicate emails ............ A lot of them ....

4. Cure ........... Close OL, then Copy my backed up PST files from last night (now you did start to make B/Us ..... didn't you ???)  In place the now corrupted files ...........

I then restarted OL and Hey Presto, e-mails download correctly (including those from the new account) and all is well with the world again !!!

What more can I say ................  The proof of the pudding is in screwing up again !!!

QED

P.

PS. .............  
OH yes that OUTLOOK BACKUP Program / add-on I mentioned earlier is called PFBACKUP which (contrary to my memory) works with 2000, 2002, 2003 and is available form M$ Office Download Centre as PFBACKUP.EXE and is a mere 175Kb. ................ Get it, Install it, Use it, every day .............

P.P.P.P>P>>>>>>>>> ------------------------------- >>                                  Ouch !!!

Yes it's me again ............

One more thing I have discovered, pretty ovious, but none the less a potential issue.

If you down load e-mails using the ADMIN account and the emails have another user name

for example

Junk@yourdomain.com

but receive on you account

admin@yourdomain.com

Then Outlook is unable to delete the "Junk" email as it thinks it should be "Admin" and niether flags it up as received or deletes it from the server.

I found this out as I have a few Phantom emails which I collect using the Admin account (helps me sort out subscriptions and helps me track if a subscription has passed on my e-mail address because I can see immedaitely where it came from)  and I noticed that these e-mails do often get duplicated. Whereas valid e-mails do not.

Bye ............. !
Well, I wussed out yesterday and moved the problem email accounts off Outlook and onto Thunderbird - although interestingly TB was unable to import OL's email because... wait for it... there was some form of corruption in the PST file! At some point we shall have to go back and move the e-email to a clean PST and then import it again.

Now - I'm interested in your last note about multiple emails coming to one address... The secretary receives emails from four different addresses into one email account - I wonder if that was a contributing factor.

Anyhow, if M$ ever releases any patches for these undocumented 'features' it would be great if the first person to pick them up could post here so that we get a thread email notification.

All the best,

J
Hi  "J"

Just a few of thoughts ......

1. Thunderbird, I use a "portable" version which lives on a USB drive and allows me to plug into any passing PC with and internet connection and a USB port. This way I can manage my e-mails from most palces ..... Thunderbird seems very robust, but unfortunately it lacks many of the useful features of OL but does offer other very useful ones.

2. Now it is very interesting that Thunderbirds sees the OL PST file as corrupted and Outlook doesn't ...... that's kind of strange ...... This may well also cause / contribute to very slow collection/sending of E-mails while OL is "struggling" to get around the "busted" file.

3 Can you explain in more detail your comment "The secretary receives emails from four different addresses into one email account" as I can't visualise what you mean by this.

Cheers

P.
You noted that only emails linked to certain sender accounts were being duplicated; this is probably due to a timeout issue in the sender's copy of Outlook.  

I ran into the same problem with Outlook 2007 and 2003 running on XP.  The issue seems to be linked to the "server timeout" value on the sender's copy of Outlook -- and a bug in the Outlook resend function.  Anything that delays the send process can trip the limit, e.g. emails that have large file attachments, a slow wireless connection, slow email server response, and delays caused by antivirus scans.  This can occur whether the user is linked to an Exchange server, Unix email server or simply connecting to the Web to via an ISP.

The best solution appears to be to increase the server timeout value -- to 5+ minutes if you have a fast connection, or more if the connection and throughput are slower.   (Select your main email account and pick edit/change, then "More Settings" then the "Advanced" tab.)

I hope this is helpful!