Want to protect your cyber security and still get fast solutions? Ask a secure question today.Go Premium

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 356
  • Last Modified:

Problems with Exchange database recovery

Guys,

I'm looking for a little help.  Last Wednesday, we had a UPS failure that triggered a dirty shutdown of Exchange.  I run two 64-bit servers on the backend for mailboxes (one for Students and one for Faculty/Staff).  The Fac/Staff box came up fine with no issues.  The student server is another story.  By the way, our backups were not valid so I can't go that direction (another story another day).

When it came back up, there were two storage groups with databases that would not mount.  My storage groups and DB's are arranged by year, so the 2010-11 and 2011-12 mailstores were the ones affected.  After following several threads and links here, I was finally able to recover and restore the 2011-12 SG and mailstore.  It is fine.  A couple of passes with eseutil to repair and defrag.

Attempted the same process on the 2010-11 box.  At first, all seemed OK.  But we started getting reports from students that messages in their accounts were empty or they couldn't access it at all after logging in via OWA.  I ran the database recovery tool under EMC and it fixed a handful, but others remained problematic.  Now it appears that it hasn't completed the recovery due to lack of disk space.  I now have three .edb files with a Temp#### prefix.  The current 2010_11.edb file is 42GB in it's current storage group.  The temp edb files are scattered randomly across other drives.

Question:  Are all 3 needed for a full recovery?  They are 40GB, 30GB, and 7.5GB in size.  I'm assuming that they are all from the same 2010_12.edb file, but stopped in various stages of repair/defrag due to lack of drive space.  Should I re-run eseutil /d against them all or what do you advise?  I know I need 110% of free space to run the tool through it's completion.

I'm heading out for a few hours but will check back in later tonight.  Thanks in Advance!

Ben
0
bwhorton
Asked:
bwhorton
  • 6
  • 6
  • 4
  • +1
3 Solutions
 
Wonko_the_SaneCommented:
Now I am not sure what was discussed in the other threads, but some of this does not add up, at least I don't quite understand it.
You can more than likely get rid of the temp edbs, but if you can keep them somewhere I'd suggest to do thta, even though they are probably useless.

I don't understand right now how your students could access this store at all if the recovery/repair never completed, because then the DB wouldn't mount at all. If your database doesn't mount because it's not in a clean shutdown state and you don't have the logs then you might have to run eseutil /p (repair). If it does mount however and data is missing then you may be out of luck already - the advice here is to restore from backup, but if you don't have one you're in trouble.

You can try the "New-MailboxRepairRequest" cmdlet on the mailboxes having issues, this will try to fix logical corruption at the mailbox level:
http://technet.microsoft.com/en-us/library/ff625226.aspx

Also check the event logs for any clues on what might be wrong.
0
 
Wonko_the_SaneCommented:
Sorry, just saw it's Exchange 2007.

The tool that might help you is ISINTEG, not New-mailboxrepairrequest.
0
 
bwhortonAuthor Commented:
Thanks Wonko.  I'm off-site ATM but will attempt a point of clarification.  The students could access it (well most of them) when I remounted it to test access and usability.  Of course I cant run eseutil or isinteg while it's mounted.  Sorry if my description above was incomplete.  -ben
0
Has Powershell sent you back into the Stone Age?

If managing Active Directory using Windows Powershell® is making you feel like you stepped back in time, you are not alone.  For nearly 20 years, AD admins around the world have used one tool for day-to-day AD management: Hyena. Discover why.

 
Wonko_the_SaneCommented:
OK, if it was mounted already I suggest:

- Try and run a full backup. This should show if there is any physical corruption in the DB. If there is, you may have to run eseutil again - you will get some entries in the application log telling you to restore from backup if this is the case.

- Take the DB offline and make a copy of the DB file + log files.

- run isinteg (alltests)

Obviously all this is going to take some time.
0
 
Deepu ChowdaryCommented:
Yes as said above take a full backup and run Eseutil & isinteg.
Its suggestable to take a copy of db files while doing anything with eseutil
The command for isinteg is isinteg -s servername -fix - test alltests.
0
 
bwhortonAuthor Commented:
Running a full backup now.  Will check in the morning when I get in and see if there is any physical corruption.  I made a copy of the db and log files last Thursday when all of this started.  I'll copy the most current db and log files after this back up finishes tomorrow a.m. and then check.  Will re-run eseutil and isinteg, then post the results here.  It may be tomorrow afternoon before I post another update, so stay tuned.  

Thanks guys,

Ben
0
 
Deepu ChowdaryCommented:
Welcome bwhorton :)
0
 
lucid8Commented:
Yep you are on the right track regarding isinteg, i.e. you should ALWAYS run this post eseutil /P and only thing to clarify and add value here is that you want to run isinteg -s servername -fix - test alltests multiple times until you experience no errors corrected.  You will have to run this at least twice since the first pass will correct a certain number of errors for sure.  Subsequent passes may or may not find and correct errors, however if it does then repeat until you get 0 errors found and corrected.  Also dont worry about warnings, just errors.
0
 
bwhortonAuthor Commented:
I haven't forgot about you but the backup is pushing 18 hours now.  Not sure why that is; most days it is done within 4-6 hours.  Still on track to perform the above suggestions and will post the results here.

Thanks,

Ben
0
 
lucid8Commented:
1. Are you doing just a DB based backup of the exchange server or are you doing a mailbox level backup as well?

2.  How big is the log files folder?

3. Any chance you are having hardware problems with the disk etc?  assume you have checked the event viewer?
0
 
bwhortonAuthor Commented:
Backup complete.  It was a full backup, complete with mb level.
Log files for that storage group is only 7MB
Not seeing any hardware issues.  It's on a RAID5 EMC SAN.

Ready to run isinteg after school is out.  I'm going to dismount the affected db; however, there are a number of other db's on this server that are not affected.  Looking at the syntax (isinteg -s servername -fix - test alltests), it appears to run against the entire server???  Just looking for a point of clarification.  Please confirm before I proceed.

thanks
0
 
lucid8Commented:
No it only runs against the specified db

The backup took so long because you did a mb level backup as well and those take allot more time and resources


0
 
Wonko_the_SaneCommented:
It will detect which DBs are available and ask which one you want to fix. Since it cannot touch mounted DBs there is no reason to worry, just dismount the problem DB first.
0
 
bwhortonAuthor Commented:
Ok, since mid-terms are ongoing at the moment, I'm going to do this tonight when I get home.  I will update as I make progress.  Thank you all for your patience and assistance thus far.

Ben
0
 
lucid8Commented:
Excellent
0
 
bwhortonAuthor Commented:
All appears to be good.  A EE subscription is the best money one can spend.  Kudos to Wonko the Sane, Exchange9, and lucid8 for their combined solution and clarification.

Thanks,

Ben
0
 
lucid8Commented:
Glad it all worked out and thanks for the points
0
 
lucid8Commented:
Glad it all worked out and thanks for the points
0

Featured Post

Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

  • 6
  • 6
  • 4
  • +1
Tackle projects and never again get stuck behind a technical roadblock.
Join Now