Exchange 2010 Database Dismounted, Corrupted

Had a serious power surge combined with an outage.

When all came back online, I had several I.T. issues. The big one was our Exchange database dismounted. Using Exchange 2010 SP3, RU4.

I ran the eseutil /mh command and found it was a dirty shutdown.
Then I ran eseutil /p to repair the database.
Database repair completed successfully.
I then ran a chkdisk /R on the server itself (but not within eseutil - should I?).
Database still would not mount, so I moved all the logs to another folder. Then the database mounted.

Next, the Transport Service would not stay started. To fix this I renamed the SenderReputation folder, restarted the Transport Service, and it worked.

Now - many issues. In my e-mail I lost all my inbox items,  another user is getting sending errors "Event ID 10030" MSExchangeIS Mailbox Store "mismatch was detected between a view of the folder and the actual contents." Still other users are having odd issues here and there.

My diagnosis is that my database shell is corrupt and I need to create a new one. What do you guys think?

If you think this is what I need to do, I'd appreciate any reliable links on how this is best done.

Thanks everyone!
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Alan HardistyCo-OwnerCommented:
If you repair a database, you should defragment it afterwards, then after the defragmentation, you should run isinteg too (at least twice), to integrity check the database.

The command is isinteg -s servername -fix -test alltests.

The store needs to be dismounted to run isinteg.

SECC_ITAuthor Commented:
I am looking it up, but can you tell me the command to run defrag within eseutil?

Do I have to dismount the db to defrag it?

Can users be online during the defrag?
Alan HardistyCo-OwnerCommented:
Eseutil /d (similar to the repair - eseutil /p)

The database will need to be dismounted.

You will need 110% of the database size in free disk space to successfully run the defrag, or you can add the /t switch to point the temp file to a different drive with sufficient space.

Your Guide to Achieving IT Business Success

The IT Service Excellence Tool Kit has best practices to keep your clients happy and business booming. Inside, you’ll find everything you need to increase client satisfaction and retention, become more competitive, and increase your overall success.

SECC_ITAuthor Commented:
My database is 33 gigabytes. I have limited time to do this - I can start at noon today and have all day tomorrow.

Roughly how long will the defrag take?

Roughly how long will the integrity check take?

How do I know if I need to run the integrity check twice?

Should I move everything to a new database shell after all this? I have about 50 users. About how long would that take?
Alan HardistyCo-OwnerCommented:
The defrag runs at about 4-6gb per hour depending on hardware.

You could always setup a new database and move the mailboxes from the corrupt store to the new store which would leave any corruption behind and mean only the mailboxes you are moving are down, but if they are all having problems, it may be easier to defrag and integrity check the database.

Once isinteg shows 0 errors and 0 fixes you know it has finished.
SECC_ITAuthor Commented:
If I run the defrag, then the integrity check, and end up with 0 errors and fixes, then I definitely have no reason to move to a new database, correct?

Roughly, how long will the integrity check take on a 33GB database with about 50 boxes?
Alan HardistyCo-OwnerCommented:
Correct - no need to move after the defrag and integrity check.

A best guess would be about an hour, but don't quote me on that ;)

SECC_ITAuthor Commented:
Okay I am starting this today since we closed at noon.

THANK YOU so much, Alan - I love EE!!!

Going to keep this open just in case questions/issues come up during my New Year's fun...
Alan HardistyCo-OwnerCommented:
No problems. I'm out partying tonight, but will check in just in case.

Out for lunch tomorrow too, but will also check in.

Have a fun one.

SECC_ITAuthor Commented:
Going to start the defrag, then head out. I can do the rest remotely. Going partying, too!

I really, really appreciate your help.

Happy New Year - keep safe!
Alan HardistyCo-OwnerCommented:
No problems - apologies in advance if any future postings are the ramblings of a drunk person partying or a person with a hangover feeling sorry for themselves tomorrow ;)

Happy New Year to you too - May your Exchange Databases be eternally happy from 2014 onwards :)

SECC_ITAuthor Commented:
Defrag went fine and very quickly.

When I ran the isinteg command I received this error:

Isinteg: The request to cancel verification process failed.
Isinteg: The request to stop verification process failed.

I found a technet post that said this:

Run the below commands and in this order (my server name is exchange).

c:\>isinteg -s exchange -fix -test rowcounts
c:\>isinteg -s exchange -test alltests

But I received the same error.

Am googling like crazy now...
Alan HardistyCo-OwnerCommented:
Is the store dismounted?

Where did you run the command from?  Did you get to choose the database to integrity check?

SECC_ITAuthor Commented:
Store IS dismounted. Yes, I chose the db and Y to continue. I ran the command from the management shell.
Alan HardistyCo-OwnerCommented:
As an Administrator (right-click the Exchange Management Shell and choose Run As Administrator)?
SECC_ITAuthor Commented:
Yes, as administrator. Also tried running it from regular command line, again, as administrator. Additional info on the error:  

"The verification has probably been interrupted. Please review the log file for more information."

Um. Where's the log file?

I also have a public folder DB, that IS online, but I'm not working on that. I definitely chose the offline exchange db.

I see in my event log that activesync is still making requests. Let me stop some services and try again.
Alan HardistyCo-OwnerCommented:
Seems that you need to use:

New-MailboxRepairRequest -Database "Database Name" -CorruptionType ProvisionedFolder,SearchFolder,AggregateCounts,Folderview

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
SECC_ITAuthor Commented:
So I need to remount the database to run that command?
Alan HardistyCo-OwnerCommented:
Yep - seems so!

Never done isinteg before on Exchange 2010 - so used to doing it for 2003 databases that they snuck in a new command on me (thanks MS).
SECC_ITAuthor Commented:
Okay, I remounted it, then ran that command. It returned a table of sorts with headers: RequestID, Mailbox, ArchiveMailbox, Database, Server. There is a sequence of numbers listed under RequestID, nothing under Mailbox, False under the ArchiveMailbox (we don't use them), the DB name under Database, and the server name under Server.

Then it just showed the c: prompt again. Confused, I ran the command again, and got a warning "unable to submit online integrity check request for database ... maximum allowed databaset ask exceeded."

So, what this tells me - This is a tool that works while the db is online, so it is working away in the background. How will I know the results? I assume there's a log somewhere or it will be in the event viewer.

Anyway, I'm not sure what's happening now so I am going to research that command some more and see what I can learn.

Thanks again for your help - this has been a great learning experience! (Still is....)
Alan HardistyCo-OwnerCommented:
Event logs should show you what is happening.

Me too!
SECC_ITAuthor Commented:
I found the events that pertain to my issue and it IS working on the background as I thought. I am out bringing in the new year as exchange works away...
SECC_ITAuthor Commented:
Like an idiot I forgot to save my event logs and they got rewritten - I know it was fixing the mailboxes because I did see a bunch of 10062 events last night.

Now I want to make sure Exchange is healthy. I found this script:

It is run from PowerShell. Do you trust the site above to write reliable scripts? I changed my execution policy so it will run, but am leery.
Alan HardistyCo-OwnerCommented:
Yep - I'd trust the site.  It's by Paul Cunningham MVP.

Can't see what the script does as I'm on my iPhone.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today

From novice to tech pro — start learning today.