[Okta Webinar] Learn how to a build a cloud-first strategyRegister Now

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 1517
  • Last Modified:

Exchange 2003: Single users mail suddenly being held in the local delivery queue

Greetings,

Exchange is a wonderful beast! I have an Exchange server running SBS 2003, with Exchange 2003. We have about 25 e-mail clients on the domain (single domain, server controls DHCP). All the clients are on the same side of the firewall as the server, and are joined to the domain.

Suddenly one of the client stopped receiving new mail in his inbox. All the other clients are receiving mail normally. The effected client can send but can't receive. If I go to the Exchange System manager and open the queues the local delivery queue shows the state as "retry" and the number of messages in the queue are 15 (and climbing).

I tried restarting the client computer (no success) and I tried restating the server (same result). I also tried deleting the first mail message in the queue, BTW all the messages in the delivery queue are for the same recipient from different sources. I checked the send/receive threshold for the user and there are no problems there.

What am I missing here?

Telefunken
0
telefunken
Asked:
telefunken
  • 22
  • 22
  • 19
  • +1
2 Solutions
 
setasoujiroCommented:
do you get any errors when you hit send/receive in outlook?
0
 
telefunkenAuthor Commented:
No-No errors to report in Outlook.

Regards

Telefunken
0
 
Hendrik WieseCommented:
Maybe check if his mailbox limit has not been reached?
0
VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

 
telefunkenAuthor Commented:
Hendrik,

Yeah I checked that. His mailbox is quite large, but it hasn't reached the send/receive threshold yet. He's not even at the warning level yet.

T=regards,

Telefunken
0
 
telefunkenAuthor Commented:
Perhaps another clue to what the problem is....in the Exchange System Manager under First Storage Group>Mailbox Store>Logons   the effected user has/had six pages of logons indicated at 12:33, the time I restarted the server. Other users only have a few. ??????????

Telefunken
0
 
telefunkenAuthor Commented:
OK the issue is taking on new dimension, now I have 2 users effected by the problem. Local delivery is not taking place.

Hellllllllllllllllllllllp!!!

Telefunken
0
 
setasoujiroCommented:
try disabling cached mode in outlook and see if that helps?
0
 
Hendrik WieseCommented:
Has anything changed on your connectors? Was an update recently deployed?

I am only on mobile at the moment, as soon as I get behind my laptop I will be able to assist you better.
0
 
Hendrik WieseCommented:
Is it possible that your mail store has reached it's limit?
0
 
setasoujiroCommented:
sorry, disabling cached modewont help....
indeed check if your store does not have any issues. Can you also check if there are any event logs?
0
 
telefunkenAuthor Commented:
Perhaps a bit of back story would be useful here. A week ago we had a catastrophic failure of our primary array, resulting in a new installation of the server software followed by a restoration of the latest backup. After establishing a connection to exchange, users had the ability to send but not receive. I deleted each users OST file on their respective client computers and then re-syncronized the users Outlook client with the Exchange server. So far so good, all clients were able to connect from this point (2 days ago) and resume normal service.....until now.

So far 2 users incoming mail are stuck in the loacl delivery queue......

Any ideas?

Telefunken
0
 
setasoujiroCommented:
ok this might have been pointed out from the start... :)
now are there any logs in the Exchange server event log?
0
 
telefunkenAuthor Commented:
Event id 2102

Process MAD.EXE (PID=4112). All Domain Controller Servers in use are not responding:
0
 
Hendrik WieseCommented:
You can try to narrow down the problem by running the exchange best practice (ExBPT). But as requested by seta, please check the logs and post the errors?
0
 
setasoujiroCommented:
ok it seems you have some other issues then just the mail.
Any other logs?
0
 
Hendrik WieseCommented:
Stop the mad.exe process and start scanning the machine for viruses
0
 
Hendrik WieseCommented:
What was the initial cause of the crash before?
0
 
Hendrik WieseCommented:
Also restart you PDC to see if it starts responding. Because without a DC you have no exchange!
0
 
setasoujiroCommented:
can you verify that the exchange system attendnt service is running?
0
 
setasoujiroCommented:
do NOT stop the MAD.exe, this is part of exchange...
0
 
Hendrik WieseCommented:
I would say restart the exchange and domain controller just to get the users back up, and then we can go through the logs. Ending the mad.exe process and then going into services and starting all exchange services would help. But the restart of the servers would be your best bet for now.
0
 
telefunkenAuthor Commented:
Wow where do I begin, how about I post a copy of the application event logs? This is a tab delimited version of the event log. I know it's not ideal but it's a start. There are many minor issues to be worked out, right now the exchange DB is my concern. Application-Event-log-10-05-11.txt

Telefunken
0
 
Hendrik WieseCommented:
Try stopping symantec on the server?
0
 
setasoujiroCommented:
ok your datastore is limited to 18 GB, and there was already 15GB in it on the ttime of logging, so i think you should look there
0
 
telefunkenAuthor Commented:
Hendrik,

I think you are on to something! What's the total limit on an Exchange 2003 store 16GB??? If so that may be it, the total of the priv1.stm, priv1.edb, pub1.stm and pub1.edb files is 16.2GB. If this is the problem how can I shrink the store? If a user archives older mail will that reduce the size of the DB?

Telefunken
0
 
Hendrik WieseCommented:
To shrink the size of the store files you must delete and then purge the extra mailboxes, and then take the DB offline and run the offline defrag with eseutil:

http://support.microsoft.com/kb/328804 
0
 
setasoujiroCommented:
of shrinking, try to raise the limit first:
http://support.microsoft.com/kb/912375  because i wouldn't start defraggingt etc since you're not in a healthy environment for the moment :)
0
 
Hendrik WieseCommented:
0
 
Hendrik WieseCommented:
Yes you could increase the limits in the registry.
0
 
Hendrik WieseCommented:
Try the FixIt posted by seta and let us know if it works?
0
 
telefunkenAuthor Commented:
I increased the size of the DB using the registry editor, and I archived old items on the effected users computer. I still haven't been able to get the incoming mail to the users mailbox.
0
 
telefunkenAuthor Commented:
On a side note I decided to remove a recent re-installation of Symantec AV because the Live update wasn't working correctly. During the uninstall the installer just kept looping and now I can't stop it. I get the following error in the Application log Event Id 1001 I can't shut down the installer no matter what I do.
0
 
telefunkenAuthor Commented:
This is Event Id 1216

The Exchange store 'First Storage Group\Public Folder Store (BISCUIT)' is limited to 18 GB. The current physical size of this database (the .edb file and the .stm file) is <1 GB. If the physical size of this database minus its logical free space exceeds the limit of 18 GB, the database will be dismounted on a regular basis.

So I can see that the DB size was correctly adjusted to 18GB
0
 
telefunkenAuthor Commented:
I successfully got the AV uninstalled and stopped the MSI from running....thank god for small miracles, one less thing to deal with.

Telefunken
0
 
Hendrik WieseCommented:
Did you manage to get your AD's to respond?
0
 
setasoujiroCommented:
You should post the content of the events in the future, not only the ID, this is very generic ;-)
did you indeed get your domain controllers to respond?
0
 
telefunkenAuthor Commented:
Sorry for the brevity of the responses earlier, as I'm sure you can understand I'm running around trying to archive user content on some of the larger mailboxes before attempting to take the store offline and defrag the DB. I was really pressed for time when I posted the Event Id 1001 above, if you are referring to the Event Id 1216, what more information would you like? I'll get you anything you need.

The situation remains the same at this time. I have six clients whose incoming mail is being held in the queue for local delivery. I don't know if it's by chance, but they happen to be the users with the largest mailboxes. None of them have exceeded the individual limits for their mailboxes.

Does this still sound like a DB size issue? I was planning on taking the DB offline later today, and running the esutil utility on it to defrag the DB. Is this still the right course of action?

Telefunken
0
 
telefunkenAuthor Commented:
Event Id 2114

Process INETINFO.EXE (PID=2500). Topology Discovery failed, error 0x80040a02.

Event Id 9152

Microsoft Exchange System Attendant reported an error '0x8007203a' in its DS Monitoring thread.

Event Id 2102

Process MAD.EXE (PID=1940). All Domain Controller Servers in use are not responding:
biscuit.GCI.local

Event Id 8026

LDAP Bind was unsuccessful on directory BISCUIT for distinguished name ''. Directory returned error:[0x51] Server Down.    

Event id 9098

The MAD Monitoring thread was unable to read its configuration from the DS, error '0x80004005'.

Event Id 1022

Logon Failure on database "First Storage Group\Mailbox Store (BISCUIT)" - Windows 2000 account GCI\Ed; mailbox /o=GCI/ou=first administrative group/cn=Recipients/cn=Ed.
Error: -2147221231

I get the above Event Id for 9 different clients

Event Id 9662

There was an error obtaining the Unsolicited Commercial Email default filter level from the directory. The error code was 0x80004005. The value remains at 8.

Those are the Event Id # and their descriptions currently popping up in the Application log since the last restart of the Server.

Telefunken

0
 
setasoujiroCommented:
It seems your domain is pretty messed up from what i can see.
I wouldn't even know where to look from here.
check this out and look at the last comment and run the wizard as stated:

http://forums.msexchange.org/m_150498400/mpage_1/key_/tm.htm#150498411
0
 
Hendrik WieseCommented:
Try installing the latest service pack if you haven't already done so!
0
 
telefunkenAuthor Commented:
Seta I tried that registry edit....no luck.

Telefunken
0
 
telefunkenAuthor Commented:
Hendrik,

I'm on SP2 for both the MS Server 2003 and the Exchange Server 2003

Telefunken
0
 
setasoujiroCommented:
What happens when you try and deliver mail to the user over telnet from the local lan?
0
 
telefunkenAuthor Commented:
Seta-Thanks for the link to the hotfix unfortunately I'm on SP2, the fix relates to SP1. As for telnet, I'm afraid I'm showing my inexperience here. I'll try anything you ask if you can guide me through it. I've never used telnet before. Care to walk me through it?

Telefunken
0
 
setasoujiroCommented:
ok, download putty from here:http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html
-then open it and enter your server's IP
-choose telnet radio button, and enter port 25 in the port field.
-press enter
-you should see a black shell with some white text with your servername in it somewhere
-enter the following sequence of commands:

helo hotmail.com --> press enter
mail from: someemailaddress@hotmail.com  --> enter
rcpt to: <oneofproblemrecipients@yourdomain.com>  -->enter
data:    --> enter
subject: some bla bla   --> enter
enter some message -->enter
.  --> enter

normally you should see "message queued for delivery


0
 
telefunkenAuthor Commented:
seta

results from telnet.

message is in the local delivery queue, holding for retry like all the other mail at this point.

Thanks for the lesson.

Telefunken
0
 
Hendrik WieseCommented:
Look at the following link, I hope this helps: http://www.thecyberwolfe.com/blog/?p=665
0
 
setasoujiroCommented:
no problem, but that shows that it is indeed accepting the messages, but not putting them in the mailbox.
euhm, would it be an option for you to have a teamviewer session to the server to check together?
mail me at seta.soujiro2070@gmail.com in case you want to proceed this way :)
0
 
Hendrik WieseCommented:
Following the link that I posted, they suggest that the first mailbox that had the problem would be the course.
0
 
telefunkenAuthor Commented:
Hendrik,

I read the article earlier, and tried the reg edit potion with no effect. Do you feel that I must delete the users mailbox in order to make this process work? If so I've never used exmerge before and would need some guidance.

Telefunken
0
 
Hendrik WieseCommented:
Hi,

Yes you would need to delete the mailbox and also all the emails after using exmerge.

For a guide on how to use it please refer to: http://www.petri.co.il/brick_level_backup_of_mailboxes_by_using_exmerge.htm
0
 
setasoujiroCommented:
Telefunken, just to be sure, did you try the regkey regarding queing?
because that seems promising ...
0
 
Hendrik WieseCommented:
Hi,

As per the link I posted earlier, please try it and let me know if it works *holding thumbs*:

To fix the user’s mailbox, I exported their account to a .pst using Exmerge (instructions on how to use Exmerge). I then deleted the user’s mailbox and created a new one, adding back in all of their aliases. This got new mail flowing again without too much fuss. The mail in the local delivery queue, however, was still stuck.

To get this going to the right mailbox, we had to ‘recategorize’ the existing mail to the new mailbox.

    Stop the SMTP service.
    Open Regedit and navigate to the following key:
    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\SmtpSvc
    Creat a subkey named “Queuing”
    Select Queuing and on right hand pane create new dword value with name “ResetMessageStatus¿ with value 1 in hexadecimal.
    Restart the SMTP service – check the queues and see if mail is being delivered. Also check the account mailbox at this point.
    Once all the mails are delivered to that user, stop the SMTP service and change the value for ¿ResetMessageStatus¿ to 0 and start the SMTP service.


Reference: http://www.thecyberwolfe.com/blog/?p=665
0
 
setasoujiroCommented:
We tried this yesterday but not yet did the reg fix, as all of the users are not yet exported.
However upon recreating the mailbox the mail started flowing again (new mail that is, not the ones stuck in queue)
0
 
Hendrik WieseCommented:
Was all the emails queued for delivery to the initial user that had the issue deleted from the queue, as suggested in the article?
0
 
setasoujiroCommented:
no , it was for serveral users so it could be that those other mails were holding the queue, but as said there was no reg fix applied at the time so i can't say anymore :)
0
 
Hendrik WieseCommented:
Cool, so we will wait for telefunken to do the reg change and check if it works.

Cheers!!!
0
 
Sikhumbuzo NtsadaCommented:
Hi this is not necessarily a fix but to enlighten you on mail flow - maybe it can bring light to your problem.

See file attached.


How-to-use-SMTP-queues-to-troubl.pdf
0
 
telefunkenAuthor Commented:
Update: Thanks to all for all the great input, and a special thanks to Seta for all his help yesterday. I am happy to report that things are moving forward. As Seta has already reported, we (he) used exmerge to export the primary users mailbox, then deleted and purged said users mailbox which got new mail flowing again. I applied the registry fix and voila... mail stuck in the  local delivery queue for this user flowed on to the users mailbox. The downside to all this is that the PST that was created with exmerge had errors ;( I ran scanpst.exe on the PST file and repaired and then opened the data file in the users mailbox. Theres a lot of mail in it, but what a mess. He won't be too happy, but his new mail is flowing. I still have a few effected users, for whom I will need to repeat this process, but I think it should go better now that I know what needs to be done.

@Santasi24 Thanks for the link I'll read it.

@HendrikWiese Thanks for the persistence, I'm going to reread some of your posts now that some of the pressure is off. The link you gave me seems to be right on the money as far as outlining the fix.

@Setasoiiro Thanks again man you stepped up and really helped me out in a jam. I appreciate it.

I'll report back later today hopefully and report on the rest of the process. With any luck I can move on to other less pressing issues. (of which there are many)

Telefunken
0
 
Hendrik WieseCommented:
Very glad that things are back on track. Good luck with the remaining mailboxes.

Enjoy your day.

Cheers!!!
0
 
Sikhumbuzo NtsadaCommented:
True dat
0
 
telefunkenAuthor Commented:
I have a question to confirm my understanding of Event Id 1221. When it reports the amount of free space available after the online defragmentation, is that the amount of space within the current DB or is it the amount of available space before the maximum size of the DB is reached?

Telefunken
0
 
telefunkenAuthor Commented:
OK I am happy to report all is well ! Mail is flowing again and nothing is stuck in the queue (lets hope it stays that way!) I have to say this was tough when it came to awarding points. Hendrik you ultimately had the correct answer, but I have to also award points to Sata for his assistance above and beyond the call. Together we were able to follow the directions Hendrik gave and with a few bumps in the road related to the corrupted mailboxes, we got it up and running again for all users.

Thanks again EE and all it's Gurus and Masters for all the help and support,

Telefunken
0
 
setasoujiroCommented:
Glad you got it sorted, now you can relax a little ;) just think about what we discussed for the infra. ;)

Take care
0

Featured Post

Simplify Active Directory Administration

Administration of Active Directory does not have to be hard.  Too often what should be a simple task is made more difficult than it needs to be.The solution?  Hyena from SystemTools Software.  With ease-of-use as well as powerful importing and bulk updating capabilities.

  • 22
  • 22
  • 19
  • +1
Tackle projects and never again get stuck behind a technical roadblock.
Join Now