• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 10577
  • Last Modified:

Exchange 2010 Transport rule stopped unexpectedly


setup is
server1 CAS HUB + not in use MB
Server2  MB
Server3 MB

suddenly i found all staff outlook not sending or receiving emails but it shows connected and i found that server1 Microsoft exchange transport rule stopped

the event viewer showed the following errors in order

1- Event ID: 490

edgetransport (4704) Transport Mail Database: An attempt to open the file "C:\Program Files\Microsoft\Exchange Server\V14\TransportRoles\data\Queue\trntmp.log" for read / write access failed with system error 32 (0x00000020): "The process cannot access the file because it is being used by another process. ".  The open file operation will fail with error -1032 (0xfffffbf8).

For more information, click http://www.microsoft.com/contentredirect.asp.

2- Event ID:413

edgetransport (4704) Transport Mail Database: Unable to create a new logfile because the database cannot write to the log drive. The drive may be read-only, out of disk space, misconfigured, or corrupted. Error -1032.

For more information, click http://www.microsoft.com/contentredirect.asp.

3- Event ID 492

edgetransport (4704) Transport Mail Database: The logfile sequence in "C:\Program Files\Microsoft\Exchange Server\V14\TransportRoles\data\Queue\" has been halted due to a fatal error.  No further updates are possible for the databases that use this logfile sequence.  Please correct the problem and restart or restore from backup.

For more information, click http://www.microsoft.com/contentredirect.asp.

4- Event ID: 17019

Transport Mail Database: A database operation has encountered an I/O error. The Microsoft Exchange Transport service is shutting down. Exception details: Microsoft.Exchange.Isam.IsamLogWriteFailException: Failure writing to log file (-510)
   at Microsoft.Exchange.Isam.JetInterop.MJetDelete(MJET_TABLEID tableid)
   at Microsoft.Exchange.Isam.Interop.MJetDelete(MJET_TABLEID tableid)
   at Microsoft.Exchange.Transport.Storage.DataRow.MaterializeDelete(DataTableCursor cursor)
   at Microsoft.Exchange.Transport.Storage.DataRow.MaterializeToCursor(DataTableCursor cursor, Func`1 checkpointCallback)

5- Event ID: 17106

Transport Mail Database: MSExchangeTransport has detected a critical storage error, updated the registry key (SOFTWARE\Microsoft\ExchangeServer\v14\Transport\QueueDatabase) and as a result, will attempt self-healing after process restart.

6- Event ID: 7001

The service will be stopped. Reason: A failure occurred in a transport database operation.

7- Event ID 1040

 The existing worker process HasExisted value before calling CloseProcess is True

8- Event ID 1022

Worker process with process ID 4704 requested the service to be stopped.

9- Event ID 1033

A worker process encountered transient problems and requested to be restarted in 5 minutes.

10- Event ID 14039

The most recent group metrics file was generated at 10/5/2011 7:29:09 PM, which is older than 1/6/2013 8:59:52 AM.
Full path: C:\Program Files\Microsoft\Exchange Server\V14\ClientAccess\GroupMetrics\GroupMetrics-2011-10-05T19-29-09.bin

11 - Event ID 1034

The service is trying to restart the worker process manager.

12- Event ID 12025

Transport service is disconnecting performance counters with process lifetime from their old process.

13 - Event ID 12028

The process with process ID 4704 is holding the performance counter active remote delivery queue length Value=1 SpinLock=0 Lifetime=Type: 1 ProcessId: 4704 StartupTime: 130018404418906250 from instance _total(CFBEE918) RefCount=0 SpinLock=0 Offset=32 and category MSExchangeTransport Queues while running processes are: Processes running are:
984 LogonUI
1964 exfba
4524 svchost
976 svchost
8328 uiSeAgnt
1364 w3wp
1752 aqagent
1356 coreServiceShell
8644 mmc
6468 TrustedInstaller
164 svchost
364 smss
560 svchost
3316 MSExchangeMailSubmission
8248 mmc
10208 mmc
8040 taskhost
5476 w3wp
6460 sppsvc
8484 TimounterMonitor
548 csrss
1924 MSExchangeADTopologyService
716 lsm
1724 svchost
8160 svchost
7968 explorer
684 svchost
2108 svchost
2424 MSExchangeThrottling
11956 powershell
3540 Microsoft.Exchange.ProtectedServiceHost
8208 conhost
3176 MSExchangeMailboxReplication
1112 svchost
2096 MsExchangeFDS
3940 Microsoft.Exchange.Search.ExSearch
9776 dllhost
5244 mad
4652 vmms
2484 winvnc4
8392 HPSizingToolUpdateProcess
708 lsass
2212 RaidServ
1688 schedul2
6808 iexplore
700 services
1288 spoolsv
892 svchost
5064 store
4632 svchost
7980 msdtc
8496 iexplore
3640 msexchangerepl
3836 Microsoft.Exchange.RpcClientAccess.Service
8000 dwm
2452 svchost
7720 csrss
1856 miniwinagent
3428 TmListen
2240 sqlwriter
8472 TrueImageMonitor
7324 svchost
4008 Microsoft.Exchange.ServiceHost
9720 taskeng
6960 w3wp
4592 conhost
1636 conhost
1044 sqlservr
1436 uiWatchDog
1580 conhost
2024 msftesql
4584 SMEX_Master
1628 coreFrameworkHost
2912 Microsoft.Exchange.AntispamUpdateSvc
600 csrss
1764 w3wp
4184 MSExchangeTransport
4380 MSExchangeTransportLogSearch
636 winlogon
1816 inetinfo
7524 svchost
2136 svcGenericHost
2596 Microsoft.Exchange.AddressBook.Service
8308 schedhlp
7124 rdpclip
2196 conhost
3360 mmc
5344 w3wp
812 svchost
8888 w3wp
8296 mmc
2188 SMEX_SystemWatcher
608 wininit
2576 winvnc4
4544 svcGenericHost
2376 winlogon
996 svchost
2964 Microsoft.Exchange.EdgeSyncSvc
4 System
396 SMSvcHost
2292 MSExchangeMailboxAssistants
0 Idle

13- Event ID 1002

The service is trying to stop.

14- Event ID 1003

The service has stopped.

15 - Event ID 1000

The service is trying to start.

16 - Event ID 12025 again
17 - Event ID 12028 again
18 - Event ID 17104

Transport Mail Database: MSExchangeTransport has detected a critical storage error but failed to complete the desired recovery action on Move due to error Cannot move source directory: C:\Program Files\Microsoft\Exchange Server\V14\TransportRoles\data\Queue to destination directory: C:\Program Files\Microsoft\Exchange Server\V14\TransportRoles\data\Queue\Queue.old.
after that i started the service manually

i noticed couple of things

the w3wp.exe service memory usage on top with more than 1200K
and when i shutdown the trend micro agent the server performance increase

any idea what is the reason and how i avoid same to happen again ????
  • 9
  • 5
  • 4
  • +2
2 Solutions
Bruno PACIIT ConsultantCommented:

Looks really like your C: drive is full on the Transport server...

Have a good day.
F_A_H_DAuthor Commented:
150GB is free
Manpreet SIngh KhatraSolutions Architect, Project LeadCommented:
I would say stop the Transport service move the Transport database and restart the Service .... could be that the transport DB is corrupted

- Rancy
Has Powershell sent you back into the Stone Age?

If managing Active Directory using Windows Powershell┬« is making you feel like you stepped back in time, you are not alone.  For nearly 20 years, AD admins around the world have used one tool for day-to-day AD management: Hyena. Discover why.

F_A_H_DAuthor Commented:

since i started the service manually and i stopped the trendmicro the serve is ok plus the event viewer has no more errors

so do u think still i should move the DB as u suggest ?
Bruno PACIIT ConsultantCommented:
Other possible causes :

1) Someone has changed the ACLs on the Exchange Directories and removed the "SYSTEM" account from the ACL... Exchange services run under local system account and must have full access on all its files.
2) An antivirus software block access to files while it is scanning them. All the Exchange files should be excluded from scanning.
F_A_H_DAuthor Commented:

i feel that number 2 is correct as the event ID 490 mentioned The process cannot access the file because it is being used by another process. " so the files might be conducted in scan process
F_A_H_DAuthor Commented:
i add to the AV rules to except .EVT, .EVTX, .LOG from the scanning
any more extension i should add ?
Two MS articles on Exchange AV exclusions - the first has a complete list of files, location sand file extensions:


Manpreet SIngh KhatraSolutions Architect, Project LeadCommented:
Do we have Exclusions to not scan any Exchange related stuff ?

DB, Logs, v14 or any other locations ?
the Extention to the files (.edb .log .jrs etc etc)

- Rancy
F_A_H_DAuthor Commented:
i have configured exclusion for the following extensions
.EVT, .EVTX, .LOG, .CONFIG, .DIA, .WSB, .CHK, .JRS, .LOG, .EDB, .JSL, .QUE, .LZX, .CI, .WID, .001, .DIR, .000, .002, .CFG, .GRXML, .DSC, .BIN, .XML, .AVC, .DT, .LST, .CAB, .FDB, .MDB, .FDM, .PPL, .IDE, .SET, .DA1, .KEY, .V3D, .DAT, .KLB, .VDB, .DEF, .KLI, .VDM

per the http://blogs.technet.com/b/davmcg/archive/2012/02/04/exchange-server-2010-and-antivirus-exclusions.aspx  instruction

so is that enough ?
Bruno PACIIT ConsultantCommented:

Usually I add an exclusion on the whole directory that contains any databases. I don't use extensions.

Mailbox databases are easy to locate looking at the database settings.
About Transport database if you did not move it it should be located in "...\Exchange Server\V14\TransportRoles\data\Queue".
Usually I exclude the whole "data" directory from AV scanning.

Have a good day.
F_A_H_DAuthor Commented:
same happened just now ... same time of yesterday !!!

Guys ... i think its AV issue so i will uninstall it and install the latest version its trendmicro actually
plus i didnt apply any exchange service pack i will apply SP1 and see

if i applied SP1 to the CAS , i will have to apply it for the other MB servers ??
Bruno PACIIT ConsultantCommented:
Hi again,

Having the same Service Pack level on all Exchange servers is best practice.
You can have CAS servers with a higher SP level than mailbox servers but having mailbox servers with a higher level than CAS is not supported.
CAS servers should always have the same or higher SP level than mailbox servers.
Neil RussellTechnical Development LeadCommented:

You dont keep upto date then on exchange service packs? You should be on SP2 UD5 by now.
F_A_H_DAuthor Commented:

issue solved by removing the trend micro and install the latest client version

thank you very much
Manpreet SIngh KhatraSolutions Architect, Project LeadCommented:
Awesome .... hope this will work better and cause no issues

- Rancy
F_A_H_DAuthor Commented:
Rancy .. you sleep ?
Manpreet SIngh KhatraSolutions Architect, Project LeadCommented:
No me awake please let me know your query

- Rancy
F_A_H_DAuthor Commented:
see after i removed the MB rule from the server was holding all rules the below error ... same error comes in another server when i removed the CAS HUB rules it was holding all rules !!

MSExchangecommon 106
Performance counter updating error. Counter name is Unique servers queried, category name is MSExchange Active Manager Client. Optional code: 3. Exception: The exception thrown is : System.InvalidOperationException: The requested Performance Counter is not a custom counter, it has to be initialized as ReadOnly.
   at System.Diagnostics.PerformanceCounter.Initialize()
   at System.Diagnostics.PerformanceCounter.set_RawValue(Int64 value)
   at Microsoft.Exchange.Diagnostics.ExPerformanceCounter.set_RawValue(Int64 value)
Last worker process info : System.ArgumentException: Process with an Id of 4296 is not running.
   at System.Diagnostics.Process.GetProcessById(Int32 processId, String machineName)
   at Microsoft.Exchange.Diagnostics.ExPerformanceCounter.GetLastWorkerProcessInfo()
Processes running while Performance counter failed to update:
588 wininit
1572 miniwinagent
1768 Microsoft.Exchange.AntispamUpdateSvc
388 svchost
584 svchost
976 svchost
580 csrss
8064 TrustedInstaller
7864 svchost
7272 rdpclip
9436 mmc
4904 MSExchangeTransportLogSearch
5516 svchost
172 svchost
9036 conhost
528 csrss
8640 w3wp
8048 vmms
7848 Microsoft.Exchange.RpcClientAccess.Service
9028 Microsoft.Exchange.AddressBook.Service
2132 winvnc4
356 svchost
7244 taskhost
344 smss
7628 HPSizingToolUpdateProcess
9492 MSExchangeADTopologyService
3188 w3wp
8408 dllhost
7416 mmc
10764 w3wp
7808 mmc
10368 MSExchangeMailboxReplication
7608 msftesql
11088 uiWatchDog
1688 exfba
6028 uiSeAgnt
6808 svchost
8184 sppsvc
696 lsm
1088 svchost
7588 schedhlp
7192 mmc
5812 conhost
10736 EdgeTransport
688 lsass
7732 TrueImageMonitor
7384 explorer
680 services
872 svchost
10720 w3wp
1460 aqagent
5596 winlogon
1260 spoolsv
6972 svchost
2636 TmListen
3028 Microsoft.Exchange.ProtectedServiceHost
1056 sqlwriter
616 winlogon
4200 svcGenericHost
1440 svchost
2540 conhost
5772 inetinfo
7740 TimounterMonitor
792 svchost
2024 RaidServ
6552 coreFrameworkHost
640 mmc
4380 msdtc
6152 SMEX_Master
7332 dwm
10876 w3wp
10284 Microsoft.Exchange.EdgeSyncSvc
8904 svchost
8508 Microsoft.Exchange.ServiceHost
824 csrss
1016 svchost
1556 MSExchangeTransport
6924 svchost
1800 SMSvcHost
1404 schedul2
2536 coreServiceShell
10460 MsExchangeFDS
1592 winvnc4
5168 w3wp
964 LogonUI
4 System
1776 sqlservr
3548 conhost
0 Idle
Performance Counters Layout information: FileMappingNotFoundException for category MSExchange Active Manager Client : Microsoft.Exchange.Diagnostics.FileMappingNotFoundException: Cound not open File mapping for name : Global\netfxcustomperfcounters.1.0msexchange active manager client
   at Microsoft.Exchange.Diagnostics.FileMapping.OpenFileMapping(String name, Boolean writable)
   at Microsoft.Exchange.Diagnostics.FileMapping..ctor(String name, Boolean writable)
   at Microsoft.Exchange.Diagnostics.PerformanceCounterMemoryMappedFile.Initialize(String fileMappingName, Boolean writable)
   at Microsoft.Exchange.Diagnostics.ExPerformanceCounter.GetAllInstancesLayout(String categoryName)
Manpreet SIngh KhatraSolutions Architect, Project LeadCommented:
Thats Strange



add RPC Client Access performance counters
Start Exchange Management Shell
Add-PSSnapin Microsoft.Exchange.Management.PowerShell.Setup
New-PerfCounters -DefinitionFileName "C:\Program Files\Microsoft\Exchange Server\V14\Setup\Perf\RpcClientAccessPerformanceCounters.xml"  

- Rancy
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Featured Post

Problems using Powershell and Active Directory?

Managing Active Directory does not always have to be complicated.  If you are spending more time trying instead of doing, then it's time to look at something else. For nearly 20 years, AD admins around the world have used one tool for day-to-day AD management: Hyena. Discover why

  • 9
  • 5
  • 4
  • +2
Tackle projects and never again get stuck behind a technical roadblock.
Join Now