The president of one of my business clients keeps getting into this situation - he checks his email (hosted by an outside firm running Linux servers) from both home and laptop PCs using Outlook 2003. We have Outlook on both machines set to 'leave emails on the server' for 7 days. So he sees the emails on both machines (in case he leaves it running on the home machine during the day, he still gets the emails at the office)...
Once in a while he keeps getting the same couple emails hundreds or thousands of times! He rants at me, I log into his web mail account and delete the offending couple emails.
The hosting company says Outlook is notorius for this when you want to leave mail on the server / check from 2 locations and that he should check mail via the web or only with one PC.... I've never experienced this with any other clients of mine.