Microsoft Exchange Information Store service won't start.

Try dcdiag and netdiag to see if these tools can pinpoint the failure.

As always with SBS 2003, might be worth running the SBS BPA, it may pick up on the issue that is causing the failure.  It's likely to be one particular aspect of AD which is causing the failures and hence the Exchange failures.

Are you in a position where you can restore from a recent backup?  Isn't a 'fix' per se but may be the quickest way to get you back up and running.

Check this kb and follow the suggestions there.
http://support.microsoft.com/kb/935676

thanks

@sunnyc7 is that good for Exchange 2003 as well?

I should also mention that Exchange is not running in cluster mode. Its installed on only one machine.

Good point.

No.

I'd also suggest checking the health of the Exchange Information Store database with ESEUTIL (but please use with caution and make sure you have backups) as it is quite likely to be in a 'dirty' state after the unexpected shutdown.

ye thats what i read on other "forums" as well. Been making backup for last 2 hours, not done yet. my MBDATA folder is around 90 gig. could that also be harmful?

and if someone would provide some tips regarding ESEUTIL id really appreciate it. thank you for the heads up on being cautious

Can you stop - site replication
Check if RPC is running under local system account.
check if any other Automatic service is not starting.

Check SMTP and IIS Admin service also

how did this issue start ?

For some reason I overlooked one vital point.

** Windows 2003 SBS Server
Windows 2003 SBS runs with Exchange 2003 Standard edition
Exchange 2003 standard has a database size limit of 75 GB
http://www.petri.co.il/change_store_size_limits_ex2003_sp2.htm

Next steps for you:
a) run an offline defrag using
eseutil /d "c:\program files\microsoft\exchange\.....\priv1.edb"

b) then run
isinteg -s SERVERNAME -fix -test ALLTESTS

Guides
http://www.computerperformance.co.uk/exchange2003/exchange2003_isinteg.htm
http://www.computerperformance.co.uk/exchange2003/exchange2003_eseutil.htm

a) you need to make sure the registry entry for database size is 75 GB
b) Then you need to reduce the whitespace in the database and try to bring down the size.

c) remaining > you need to exmerge the data out and get it less than 75 GB

You have a prime case of migrating to SBS 2008.

thanks

Site Replication is stopped
RPC is NOT running under Local Account, its running under NT Authority\NetworkService

List of Automatice Services not started:

SMTP (Error 87: The parameter is incorrect).
Perfomance Logs and Alerts ( I think this is normal)

Microsft Exchange System Attendant
Microsoft Exchange Routing Engine
Microsoft Exchange POp3
Microsoft Exchange MTA Stacks
Microsoft Exchange Information Store
Microsoft Exchange IMAP4
Certificate Services

IIS Admin is running under Local Account

The contents of the MDBDATA folder are more important than the folder itself - the private and public EDB file sizes are ones to check individually.  Sunny is right though, if these files are collectively making up 90GB this is huge for SBS and even if you have Exchange SP2 and are up to the full 75GB limit this is the end of the road size-wise.  Definitely an offline defrag is called for in that case.

A good resource for ESEUTIL is also http://technet.microsoft.com/en-us/library/bb123761%28EXCHG.65%29.aspx 

anti-mhz

Stores wont mount till you get to 75 GB
Also you should check your database size in registry from here

http://www.petri.co.il/change_store_size_limits_ex2003_sp2.htm

Yes, P&L is normally stopped, it starts and stops as and when it's needed

Also word of caution

eseutil /p
or eseutil /d

runs at 5-6 GB per hr. For 90 GB - you are looking at overnight job.

Out of curiosity, did dcdiag or netdiag show up any configuration problems?

My edm. + .stm are 36.2 GB
The 90 GIG I mentioned are .edm. + .stm + 5MB log files.
the registry entry for 75 gb has been applied at some point before today's fiasco.



"a) run an offline defrag using
eseutil /d "c:\program files\microsoft\exchange\.....\priv1.edb"

b) then run
isinteg -s SERVERNAME -fix -test ALLTESTS"


still applies?

Hang on, so priv1.edb and pub1.edb are both less than 36GB?  Well that's not so bad.

And, yes, pretty much it still applies, you probably won't get far until the Exchange DB is in good health.  

You could however try ESEUTIL /r (recovery) followed by ESEUTIL /g (integrity check) first - I must say I've seldom had cause to use this tool lately, Sunnyc might disagree and I'd welcome his opinion but this might be safer than starting with the defrag, particularly if DB size isn't really an issue.

ok if your EDB + STM is 36 GB

All you need to do = Take a Good Full Backup.
what backup software are you using.

When you configure your backup, make sure you Flush the logs.
http://www.msexchange.org/tutorials/Exchange-2003-Backup-Restore-NTBACKUP.html

Take a normal full backup.

I'd run this first

eseutil /mh "c:\program files\......\priv1.edb"

substitute correct path to priv1.edb

copy paste the output here for /mh

thanks

C:\Program Files\Exchsrvr\bin>eseutil /mh "C:\Program Files\Exchsrvr\MDBDATA\pri
v1.edb"

Microsoft(R) Exchange Server Database Utilities
Version 6.5
Copyright (C) Microsoft Corporation. All Rights Reserved.

Initiating FILE DUMP mode...
         Database: C:\Program Files\Exchsrvr\MDBDATA\priv1.edb

        File Type: Database
   Format ulMagic: 0x89abcdef
   Engine ulMagic: 0x89abcdef
 Format ulVersion: 0x620,11
 Engine ulVersion: 0x620,11
Created ulVersion: 0x620,9
     DB Signature: Create time:11/23/2008 14:30:56 Rand:3452743 Computer:
         cbDbPage: 4096
           dbtime: 108962739 (0x67ea3b3)
            State: Clean Shutdown
     Log Required: 0-0 (0x0-0x0)
   Streaming File: Yes
         Shadowed: Yes
       Last Objid: 16873
     Scrub Dbtime: 0 (0x0)
       Scrub Date: 00/00/1900 00:00:00
     Repair Count: 0
      Repair Date: 00/00/1900 00:00:00
 Old Repair Count: 0
  Last Consistent: (0x3020,DB,51)  08/23/2010 08:54:31
      Last Attach: (0x2C67,12D,89)  07/01/2010 22:09:29
      Last Detach: (0x3020,DB,51)  08/23/2010 08:54:31
             Dbid: 1
    Log Signature: Create time:11/23/2008 14:30:55 Rand:3443651 Computer:
       OS Version: (5.2.3790 SP 2)

Previous Full Backup:
        Log Gen: 2662-2662 (0xa66-0xa66)
           Mark: (0xA66,D1C,7F)
           Mark: 02/16/2009 04:59:27

Previous Incremental Backup:
        Log Gen: 0-0 (0x0-0x0)
           Mark: (0x0,0,0)
           Mark: 00/00/1900 00:00:00

Current Full Backup:
        Log Gen: 0-0 (0x0-0x0)
           Mark: (0x0,0,0)
           Mark: 00/00/1900 00:00:00

Current Shadow copy backup:
        Log Gen: 0-0 (0x0-0x0)
           Mark: (0x0,0,0)
           Mark: 00/00/1900 00:00:00

     cpgUpgrade55Format: 0
    cpgUpgradeFreePages: 0
cpgUpgradeSpaceMapPages: 0

       ECC Fix Success Count: none
   Old ECC Fix Success Count: none
         ECC Fix Error Count: none
     Old ECC Fix Error Count: none
    Bad Checksum Error Count: none
Old bad Checksum Error Count: none

Operation completed successfully in 3.250 seconds.

Select allOpen in new window

Previous Full Backup:
        Log Gen: 2662-2662 (0xa66-0xa66)
           Mark: (0xA66,D1C,7F)
           Mark: 02/16/2009 04:59:27

I will go with a full backup first.
what backup software did you use to backup ?

If you dont have an - you can use ntbackup.

shutdown status is - clean shutdown. I think you're good as far as EDB is concerned.
we will still run a repair and defrag and isinteg to fix errors - after the full backup.

thanks

Curious date on that log as he said he was currently running a full backup.

SunnyC7 I'll submit to your superior Exchange knowledge here - part of me thinks it's an AD issue but I'd like to see the outcome of ESEUTIL.  

Anti-mhz can I suggest also looking in the vsi1 folder and ensuring that chkdsk hasn't removed any of the folders (such as queue).  Unlikely to be the answer but worth a try.

comphil - it can be an AD issue, we dont know that yet.

anti-mhz
Are there any other errors in event logs

start > run > eventvwr
Check under application

Please post back

thanks

eseutil probably only reports NTbackup, not third party solutions like the one im using.

Since I posted Event Logs in the first post, I have experienced lots of Application logs

Source: Userenv
Eventid: 1053
Description:

Windows cannot determine the user or computer name. (There are no more endpoints available from the endpoint mapper. ). Group Policy processing aborted.

Where would I find "vs1" folder? I did a search on local discs with no luck

"vsi1"

c:\Program Files\Microsoft Exchange Server\Mailroot\vsi 1

What I have is

C:\Program Files\Exchsrvr\Mailroot\vsi 1\BadMail
C:\Program Files\Exchsrvr\Mailroot\vsi 1\PickUp
C:\Program Files\Exchsrvr\Mailroot\vsi 1\PickUp

Queue is timestamped (date modified) 08/23/10 - would that say anything?

as far as I can tell none of these folders have anything in them.

then these folders are fine.

*

C:\Program Files\Exchsrvr\Mailroot\vsi 1\BadMail
C:\Program Files\Exchsrvr\Mailroot\vsi 1\PickUp
C:\Program Files\Exchsrvr\Mailroot\vsi 1\Queue

"we will still run a repair and defrag and isinteg to fix errors "

what order would u recommend?

as im looking at overnight job

First
Take a full backup and flush all the logs.
verify that the backup is good

Then do a eseutil /mh "C:\Program Files\Exchsrvr\mdbdata\priv1.edb"
It should show last full backup as today's date

SECOND
eseutil /p "C:\Program Files\Exchsrvr\mdbdata\priv1.edb"
eseutil /d "C:\Program Files\Exchsrvr\mdbdata\priv1.edb"
isinteg -s SERVERNAME -fix -test ALLTESTS

Run the last step till you get 0 errors.

something threw me off about the eseutil /p

C:\Program Files\Exchsrvr\bin>eseutil /p "C:\Program Files\Exchsrvr\mdbdata\priv
1.edb"

Microsoft(R) Exchange Server Database Utilities
Version 6.5
Copyright (C) Microsoft Corporation. All Rights Reserved.

Initiating REPAIR mode...
        Database: C:\Program Files\Exchsrvr\mdbdata\priv1.edb
  Streaming File: C:\Program Files\Exchsrvr\mdbdata\priv1.STM
  Temp. Database: TEMPREPAIR3836.EDB

Checking database integrity.

                     Scanning Status (% complete)

          0    10   20   30   40   50   60   70   80   90  100
          |----|----|----|----|----|----|----|----|----|----|-------|

C:\Program Files\Exchsrvr\bin>

I have console power again?

nvm I see ESE eventID: eseutil (3836) The database engine 6.05.7638.0002 started.

probably going to see end of process as well

The hard drive on which you are running the repair should have 150% of free space.

if your EDB is 36 GB, you'd need to move the EDB to another HDD which has a 50GB free space.
I used to run this from an external HDD and then copy it back to mdbdata.

Take a full backup first.
running eseutil without that is very risky.

Then verify if the backup has flushed and committed the logs using eseutil /mh

Is backup via NTBackup absolutely necessary? as opposed to backup made by a third party solution? I could never get ntbackup to work right on this server.  Can I just flush .log files (MBDATA folder) by hand?

NO WAY ! Flush the logs.

You have to take backup via a Exchange Aware Backup Solution - like BackupExec
NTBackup is free
hence the easiest one.

Backup is absolutely necessary before you do this.

I mean no way you should flush the logs @

on second reading it reads like No way > (go ahead and flush the logs...)

logs are flushed.

would I need to restore from backup and repeat eseutil

i should also ask if chkdsk has any effect. something tellin me it does. would you recommend running/not running chkdsk at startup. the machine does ask me to.

you dont need to.

a) First try to mount the database and start information store and other services.

lets see if the services are starting ?
Can you restart the sbs box ?

at the moment the eseutil /d is about 50% . restarting the box at this point might be the worst idea ever. nothing i can do right now but watch.

is it normal for eseutil /d  to stay still at 50%? its been about 2 hours


isinteg -s SERVERNAME -fix -test ALLTESTS - would that take as much time as /p or /d?

no
isinteg takes much less time,

Also you have to repeat isinteg till you get ZERO errors.
First pass may end with a few errors.

eseutil /d runs at 5-6 GB per hour.
you can do the math for your EDB

what would I doif eseutil /d is at 50% freeze

Go to the drive where you are running this.
See if the drive has 0 - 50 MB space left.

if so - then delete the temp files created during eseutil /p or eseutil /d
and run it again.

You can stop the process by CTRL+Z

got my fingers crossed

How's it going man ? your eseutil has been running for over 30 hrs now :)

first try never got past 50%. lettin it run again hoping second time's the charm.

C:\Program Files\Exchsrvr\bin>isinteg -s SERVER -fix -test ALLTESTS
Error: unable to get databases status from server. The reason could be either wr
ong server name or networking problems. Isinteg quits now.

http://support.microsoft.com/kb/182081
Microsoft KB reads:

Running
When you run Isinteg on a computer that is running Exchange Server 2003, Isinteg first checks to see whether the MSExchangeIS service is started. If the MSExchangeIS service is not started, the following message is displayed, and Isinteg stops:
Error: unable to get databases status from server. The reason could be either wrong server name or networking problems. Isinteg quits now.

this applies directly

what would I do next?

Please make sure Exchange Info. Store is mounted. > services

start  > run > services.msc

Start all exchange services.

at the moment error remains
Error 1068: The dependency service or group failed to start. Some of the othe Exchange services like Microsoft Exchange Routing Engine service

Can you start the dependency groups first.
Check their dependency's too.

Also start Simple Mail Transfer protocol / IIS Admin etc.

Dependencies won't start/SMTP service wont start.

I followed this KB: http://support.microsoft.com/kb/327649

as 87 paramter is incorrect was most common error

HKEY_LOCAL_MACHINE\Software\Microsoft\Rpc\Internet\Ports

While i expected to see a range
as specified

"In the Multi-String Editor dialog box, increase the range of specified ports. For example, if the current range is from 10000-15000, which allocates 5,000 ports, you can change the range to 10000-20000 to allocate 10,000 ports.For additional information, click the article number below to view the article in the Microsoft Knowledge Base:
250367  (http://support.microsoft.com/kb/250367/EN-US/ ) INFO: Configuring Microsoft Distributed Transaction Coordinator (DTC) to Work Through a Firewall "

value that i saw for Ports was: "4125"

ive got no clue what that means

could a "4125" limit outgoing ports , if it was range from 1-4125? instances provided by MS are 5000,10000

Can you post the error logs. What errors did you find and what patches did you apply to fix it ?

Windows is attempting to start the following service on Local Computer...

Microsoft Exchange Information Store

Could not start the Microsoft Exchange Information Store service on Local Computer.
Error: 1068: The dependency service or group failed to start.




Windows is attempting to start the following service on Local Computer...

Microsoft Exchange Routing Engine: Error 87: The parameter is incorrect



Windows is attempting to start the following service on Local Computer...

Simple Mail Transfer Protocol (SMTP) service on Local Computer.
Error 87: The parameter is incorrect.


Windows is attempting to start the following service on Local Computer...
Microsoft Exchange MTA stacks

Could not start the Microsoft Exchange MTA stacks service on Local Computer.
Error 1068: The dependency service or group failed to start.



Event Logs > Application

Source: Service Control Manager
EventID:7023
Description: The Simple Mail Transfer Protocol (SMTP) service terminated with the following error:
The parameter is incorrect.


Source: service Control Manager
Eventid: 7001
Description: The Microsoft Exchange Information Store service depends on the Microsoft Exchange System Attendant service which failed to start because of the following error:
The operation completed successfully.

Source; Service Control Manager
Eventid:  7001
Description: The Microsoft Exchange Event service depends on the Microsoft Exchange Information Store service which failed to start because of the following error:
The dependency service or group failed to start.

Source: Service Control Manager
EventID: 7023
Description: The Microsoft Exchange Routing Engine service terminated with the following error:
The parameter is incorrect.

Source: service Control Manager:
EventId; 7001
Description:
The Microsoft Exchange MTA Stacks service depends on the Microsoft Exchange System Attendant service which failed to start because of the following error:
The operation completed successfully.

At this moment I dont know what the source of the problem is. so im not patching nothing.

Hold on.
Will post back

Check if rpc proxy is present here
c:/windows/system32/rpcproxy

Also
Can you restart the exchange server.

Lets see what we get after the restart.
I hope all the RPC issues will be resolved.

C:\WINDOWS\system32\rpcproxy has a rpcproxy.dll file
 is this normal?

yes thats wher eit should be

please restart exchange and lets see if it starts the services.

if theres no way to restart exchange without restarting the DC, this is going to have wait until weekend.

Whats happening brother

about to restart.

wow. who could have thought

http://support.microsoft.com/kb/327649 -


If you do not have to specify which ports MSDTC uses, you can resolve this problem by deleting the following registry key:
HKEY_LOCAL_MACHINE\Software\Microsoft\Rpc\Internet

   1. Click Start, click Run, type regedt32 in the Open box, and then click OK. This starts Registry Editor.
   2. In Registry Editor, expand HKEY_LOCAL_MACHINE.
   3. Under HKEY_LOCAL_MACHINE, double-click Software, double-click Microsoft, and then double-click Rpc.
   4. Under RPC, select Internet.
   5. On the Edit menu, click Delete to delete the key.
   6. On the File menu, click Exit to close Registry Editor.
   7. Click Start, and then click Shut Down. Select Restart in the list, and then click OK to restart the computer.


All my exchange services are back up!

*who would have thought errrrrr

All services up ? is Exchange mail flow ok ?

all services up. mail ingoing/outgoing - ok

Cool.
Did these tests pass
Test for inbound / outbound SMTP

www.testexchangeconnectivity.com/

Testing Inbound SMTP Mail flow for domain user@domain.com
  Inbound SMTP mail flow was verified successfully.


Performing Outbound SMTP Test
  Outbound SMTP Test Successful

ok.
You are good to go.

Time to enjoy your weekend.

500 points should go to sunnc7. I made a mistake

Hi anti-mhz.
If you found any of my posts helpful in solving your problem, you should mark that as an answer and then close the question.
Otherwise if you close the question - that means you did not find any help here.

You can make these changes from your end of the EE interface.

thanks

I will ask a moderator to help you close this. No issues.
Go enjoy your weekend man.

Ok. The question has been opened. You can select any answer and allocate points to that.

thanks

hi anti-mhz
Is the issue resolved ?
the question shows as still open.

If you found any of the above comments helpful, you can select that and mark it as an answer.

thanks

Thanks for the points :)