Installing Service Pack 1 on Win 2003 and SP2 on Exchange

Is it possible to move this question to Windows 2003 Server area? I accidentally posted it here.

Most servers i've upgraded went good. There where no problems.
But i had 1 server which wouldn't boot after the sp1 update. I thought it had something to do with the sql server.
So i think it's very important to make a backup. Maybe you can make a mirror on a second disk first and disconnect that one before upgrading.

Ok thanks. Do you know if we can just update just Exchange to SP2 (after updating to SP1 with win2003 std) and run it among other win2003 servers without SP1?

Yes, you can back it up with the built in back up. What size is the data on your system drive?

I'm running Exchange 2003 on a W2K3 server but it already had service pack 1. I didn't have any problems with the Exchange SP, but my guess is if you were going to have one of them cause you any issues, it would the Exchange SP.

Yes, you can run a W2K3 server with SP1 among others without. Are there any particular reasons you haven't updated your servers?

A free tool along with a cheap tool (or free if you don't care about licensing) are great for situations such as this, as well as many others. Get something like Windows xpe and ghost, or something like them. XPE allows you to boot the operating system into RAM from a disc. Plus it's small enough to fit diagnostic utilities and other tools, such as ghost, on the disc with it. You can boot to your network with access to all drives as long as they are not physically damaged. Even if they're physically damaged, you can run repair tools that might help. XPE and tools like it are usually free.

Ghost is not too expensive. You can just ghost the volume, make your changes, installs, or whatever then boot to XPE and ghost the image back if something goes wrong.

You should look at putting WSUS on one of your servers for updates. One of the nice features about IMF v2.0 is that it is updated by WSUS now.

Hello Cptn,
The reason why it hasn't been updated is because I'm new to this company and this is the way they gave everything to me. They are getting a lot of spam but don't wanna use any paid solution. So i thought of upgrading to Exchange SP2 to new IMF but win2003 didn't have SP1 and exch SP2 needs that.

I guess my question is : Is it ok to run just the mail server with SP1 (for win 2003) and SP2 (for Exchange) and leave the other servers as it is (for now).
Once I set Acronis up and buy a new NAS Unit upgrade other servers. (We don't have enough space to even do a full back with Acronis). And I'll look into XPE and ghost.

You can allways uninstall the servicepack via the control panel, Add/Remove software if the server doesn't run as expected after you have installed it. I've done that without problems a few times. So it isn't absolutley necessary to use an imaging tool. I'd just make sure you have rebooted the server before applying SP1.

If you want me to move the Q to the Windows 2003 area, I can do that for you? Just yell and I'll move it!

rindi,
PE Storage

Yes, it is definitely ok to run one with SP1 with others that are not running it. The only time I've ever had a SP blue screen a server was with NT 4 and it was because I didn't read the "readme" first. It was an issue with Compaq servers and it was fixed, but it was totally my fault.

You shouldn't have any problems upgrading to SP1. As rindi stated, you can uninstall it as long as you check the option to save the backup data of what it replaces. But, in the slim chance that it blue screens or won't come back up, there is a process for booting the recovery console and "undoing" what a service pack or update did. Doing it the manual was just takes some time. Are these generic type machines or are they real servers?

Also, if you have a dedicated internet connection, you should keep your servers up to date. Something like XPE is great for MANY situations, not just this one. I can give you some links for it if you like.

How big is your system partition on the machine you're updating? Do you have enough room on another server to at least use the built in backup? Back it up to another network drive. If the server bombs (and I seriously doubt it will), you can set it back up by creating one small partition to reinstall the operating system and one the same as the system partition in it now. Reinstall the operating system, restore the info back to the other partition and change the boot.ini.

Thanks again Cptn and Rindi.

The C drive on mail server is 20GB (13 free) and D is 116GB (80 free) and that is the one I'm trying to install SP1 and SP2 for IMF. The problem is I don't have enough space to backup the entire server (say I ghost it or use Acronis), ideally I would like to store all the data as well (since its the mail server). I will need atleast 45gb to do that and this is the only server that has enough space.

Yes these are real servers (Dell PowerEdge 2650s, 2850s, rack mounted). I can definitely back System State. Is there a way to backup Exchange using win backup (I have tried it once, it took hours and hours to do it).

They are running Veritas, there are so many failed backups I'm not confident to use. I guess I'm in a bit of soup. If there are links please do post it, that would helpful. Another weekend gone. I can definitely build the system partition on a different machine (SQL Server machine). Do you have any links for it? Also please give me links to back up this server to an external drive (is it possible? to have an external drive attached to my laptop on the network and store it there?) I'm sorry I'm asking too many stupid questions. Thanks for all the replies.

ntbackup backs up exchange, but depending on the size and how well it is uzsed this can take a long time. If you have the exchange agents with BE, then that shouldn't be a problem either. Since BE bombs out during exchange backup, I suspect so will ntbackup. This is probably due to a corrupt exchange store, so you may want to use the exchange tools to first check the store's integrity and repair it. These tools should also work before upgrading the exchange SP's but it's probably better to patch exchange first.

My guess on the Veritas is that you're trying to do brick level backups (backing up each individual mailbox)? If so, you do NOT need to be doing that. If you are and they are failing, "Enable single instance backup of message attachments" is probably checked. In any case, with Exchange 2003, you have the "Recovery Storage Group" feature so there is really no need to do brick level backups.

I wouldn't use the builtin backup to backup Exchange. I was assmuing you didn't have Exchange installed on your system partition. And if it's only a 20GB partition, my guess is you don't? If you do, your information store is probably pretty small. If you're using the Standard version of Exchange (versus the Enterprise version), the limitation of the info store's size is 16 or 18GB. That shouldn't take to long to backup.

If Exchange is on another partitiion, you don't need to worry about it. Just backup the system partition. SP1 is not going to do anything to any other partitions. As for external devices, as long as the machine can see it, you can back up to it with the built in back up or Veritas.

Veritas is not the greatest product in the world, but it will definitely do. What Veritas product are you using? Do you have the Exchange client for it? Do you have the IDR client? I've done full recoverys using both Backup Exec and Net Backup. It's not the best, but if you get everything set right, it will work.

You don't need to build a different partition on a seperate machine if you have room. If you don't have room, you won't be able to create another partition anyway. If you do have room, I would suggest you use Veritas to back the system partition on your Exchange server up to the SQL server. This will go a LOT faster than backing up to tape.

I've had MANY battles with several versions of Veritas. I'm sure your issues with it can be fixed, but this is not the DB for it. If you would like to discuss it away from here, I can give you my email address to discuss them.

Windows PE / XPE can be downloaded at Sourceforge: http://sourceforge.net/project/showfiles.php?group_id=126922&package_id=140740
Once you have XPE, go here to find many different ways to plug other apps into it: http://oss.netfarm.it/winpe/

Thanks Rindi. The exchange has only 30 gb of data. But youa are saying its not ok to run an exchange backup overnight using ntbackup without running Isinteg tool right?

"These tools should also work before upgrading the exchange SP's but it's probably better to patch exchange first."  
Or are you saying that its ok to run SP2 first before doing all this? But I wanted to back it up before installing SP2.

Oh, and BTW, the question about your servers... You're not going to have some weird hardware issue with Dell's. If you are, they will be known and well documented as they are popular servers. It's always nerve racking to do something that has the potential of damaging a server, but the bottom line is that I would bet $1000 against 1$ that you won't have any problems. I would bet that the odds of the servers performing better are much greater than it messing them up. If I were going to worry about either of these, it would the the Exchange SP2 that I would worry about, lol.

Even if you were using older tape drives, like Mammoth technology or something, the backup of a 30GB info store should take less than hour. On drives newer than a couple of years, even less. You must have Veritas set to back up each individual mailbox?

If your Exchange is 30GB, then it's not on the system drive right? If you're worried about backing up the server before you do this, then you only need to worry about the system drive if Exchange is elsewhere. If you have Veritas, just do a full backup of the system drive/state. That shouldn't take to long. Even if you used ntbackup, no more than 2 hours for 20GB or less.

I don't know what the recommended method is, but I didn't clean my database up before I installed SP2 for Exchange. I think Rindi was suggesting that the first concern should be to patch Exchange. If so, I would definitely agree with that.

An SP is the least of things to worry about crashing your server. It sounds like your first concern should be that you are getting good backups. My second concern would be patching all the servers OS. It sounds like excessive spam is the least of your worrys right now.  

Thanks a lot Cptn (Captain is it?) No its not on the system drive.
So I will backup  1. System State 2. OS drive and then install SP1 on win 2003 and install sp2 for exchange and see how it goes.

I just checked Veritas, i guess the backups are fine but it fails because of open files and veritas doesn't come with VSS (its an add on, something called advanced file open). I'm getting 150GB worth of byte count.

Yes Spam is least of my worries but my boss and other VPs think otherwise.

Is it possible and ok to do a backup of the server to an external drive? Will it be useful?

Thanks again for all the help, I really appreciate it!!

For exchange the open file agent won't work, for that you need the exchange agent, and also the remote agent needs to be running.

And yes, upgrading exchange will not do anything to the store, so I would patch exchange, then run the Isinteg tool. The backup could also be taking a long time because of virii contained in the mails. It may help if you turn the virus scanner off while backing up. With Ntbackup you don't need a special agent to backup exchange, it is builtin.

Is it imperative that we run Isinteg tool after patching exchange? Will it be of any consequence if I don't do Isinteg?

I'm suspecting a broken database which this would help repair. For that reason (or a virus) the backup is probably not working. Patching exchange won't repair the database...

Ok I understand that. Virus meaning, something somebody's mailbox?

Yes. Virii inside mails often don't get cleaned out properly, and when these get processed while backing up or even by the AV software while backing up, this can cause the backup to fail.

You're backup is probably working. A known issue with Veritas is that it reports a job as failed if it skipped a file. The job that backs up your Information store through the Exchange client should be a job all by istelf so that when a skipped file somewhere else causes the "failed", you'll know your Exchange data was succesful. It could be possible that you're backups are taking so long because of a corrupt database, but I'm guessing that it's a problem that is another common issue with Veritas. Look at your backup job for Exchange. If you are backing up individual mailboxes (brick level backups) then your backups of Exchange will take MUCH longer. In Veritas products, backing up the Exchange databases is done the way Microsoft intended for it to be done. Backing up individual mailboxes is something Veritas (or other backup applications) added. I don't think Backup Exec allows you to back the mailboxes up the same way that Exchange stores them. In other words, it backs up single instances of multiple attachments. The information store only has one copy of something even if 500 people have it. Backup Exec copies everything in a persons mailbox so that it can restore individual boxes or emails. So that one instance of the attachment in your database is backed up 500 times in that job. I have a 26MB database in my server. A brick level backup for me takes a more than 24 hours and is usually around 400 to 500GB where as the backup of Exchange databases takes about 30 to 40 minutes. Net Backup allows you to do brick level backups similar to the way Exchange stores things and this only takes a couple of hours and is only around 30GB. But with 2003 and the Recovery Storage Group, you do not need to do brick level backups at all.

So, with all that said (yes I know I'm long winded), Rindi is correct. You could have a corrupted database or your virus software is slowing it down. My guess it's showing failed because files in windows system folders were skipped, or maybe pst files if people store them on your servers.

She is also correct about backing up with Veritas. If you have the Exchange client for Veritas, you have all you need. Backup Exec uses an Advanced Open File plugin, but it still skips some files. We had it when we used Backup Exec but didn't use it. It created more overhead than it's worth. I would bet your skipped file messages are similar to what ours were. It's skpping some Windows system files or a few application files that are open. The Exchange client is getting what it needs in case you have to recovery and the SQL client is getting what it needs for recovery. Unless you're using IDR (Intelligent Disaster Recover), and you have to do a complete restore, you're most likely (and probably better off) going to install the operating system first anyway. Applications can easily be reinstalled as well. The data that can't be reinstalled is the important thing.

The only files of real concern to us that were getting skipped were PST files. But, the job hasn't actually failed, it's just skipped some files locked by something else. There is a registry setting that will make those jobs not showed as failed if they only failed because of that. The information will still be in log so that you can see what was skipped and the job will say something liked "Succesful with exceptions". If it fails for any other reason, it will still showed as failed though.

Not sure which Veritas product you're using, but if you have Net Backup, VSS is the alternative to Advanced Open File and doesn't require any additional add-ons. But it only works on 2003 servers because it's using Shadow Copy for snapshots. For other servers, you need to add-on and they call it VSP (Virtual Snapshot Provider).

Yes, it is very possible to backup to an external drive. Depending on the amount of information, company policies, etc, many people backup to external drives as their main backup procedure. The big question will be how it connects and how fast data can be written to it. But, the speed a drive can write to tape is typically the bottleneck in a backup process anyway. If your drives are older, it can sometimes even be possible to get faster throughput on something connected via USB (but not likely).

If you're virus detection software didn't catch an infected mail before it was delivered to an inbox, then it won't most likely won't catch it until someone acutally tries to run or open it and Veritas will still back it up. Even if there is nothing infected though, real time scanning on your mail server will slow the backup job WAY down if you're doing brick level backups.

If you're using Backup Exec with no Advanced Open File plugin, you're jobs are reporting failed because of skipped files. Look at your Exchange backup log. Are any of the skipped files Exchange database files (.edb, .stm,  .log files that begin with E000xxxx.log) or any other files in the \Exchsrvr\ directory?

Following is one such file from Veritas Job Log errors. We are using Veritas backup exec ver9.1 revision 4691. Actually I just checked we are using VSS.

Access denied to file XYZ [xyz]Top of Information StoreInboxMail Delivery (failure xyz@companyname.com).
Access denied to file ABC [abc]Top of Information StoreDeleted ItemsMail Delivery (failure abc@companyname.com).
Access denied to file XYZ1[xyz1]Top of Information StoreDeleted ItemsFwd: Warning again.
Access denied to file XYZ1 [xyz1]Top of Information StoreDeleted ItemsMail Delivery (failure xyz1@companyname.com).
Directory not found. Can not backup directory George Wenger [gwenger] and its subdirectories.
Access denied to file Somebody [mhcalvin]FinderUnread MailYou cannot do that!.
Access denied to file Somebody [mhcalvin]FinderUnread MailMail Delivery (failure Somebody@companyname.com).
Access denied to file Somebody [Somebody]FinderUnread MailRe: FW: Tr: Fwd: Prière.
Access denied to file Somebody [Somebody]FinderUnread MailNotice again.
Access denied to file Somebody [Somebody]FinderUnread MailMail Delivery (failure Somebody@companyname.com).
Access denied to file Somebody [Somebody]FinderUnread Mailimportant message.
Access denied to file Somebody [Somebody]FinderUnread Mailthe truth?.
Access denied to file Somebody [Somebody]Top of Information StoreInboxYou cannot do that!.
Access denied to file Somebody [Somebody]Top of Information StoreInboxMail Delivery (failure Somebody@companyname.com).
Access denied to file Somebody [Somebody]Top of Information StoreInboxRe: FW: Tr: Fwd: Prière.
Access denied to file Somebody [Somebody]Top of Information StoreInboxNotice again.
Access denied to file Somebody [Somebody]Top of Information StoreInboxMail Delivery (failure Somebody@companyname.com).
Access denied to file Somebody [Somebody]Top of Information StoreInboximportant message.
Access denied to file Somebody [Somebody]Top of Information StoreInboxthe truth?.
Directory not found. Can not backup directory Robin Baumer [rbaumer] and its subdirectories.
Access denied to file Susana Caley [scaley]Top of Information StoreDeleted ItemsFwd: image.jpg.

We use Symantec and we also have Symantec Mail Security for Exchange. I'll check and see if the Quarantine folder is being backed up or not. Thanks very much for the speedy responses Rindi.

First off, this indicates that you are doing brick level backups.

The ones that say "Can not backup directory George Wenger [gwenger] and its subdirectories" happen when the user account is disabled or no longer exists in AD but still exists in Exchange.

There are a few things that can be going on with the others. It can be that the service account being used by BE does not have proper rights to these accounts. This can be checked by logging on to a computer with the service account, opening Outlook and attempting to open "mhcalvin" or "Somebody's" through the "Open other Users Inbox" feature. If it won't let you, then check the rights of the service account on these accounts. If this is the problem, you should see messages for these same people on every backup log. I doubt it's this, but if it is, you can easily fix it.

It can be that these mailboxes are tagged with "hide account from address lists" checked. For whatever reason, sometimes Backup Exec can't attach to mailboxes that have this feature enabled. Same holds true for this situation usually too. If this is the problem, you should see messages for these same people in all of this job's logs. Doubt it's this either, but this is also easily checked/fixed.

Being that these messages are the same ones in different people's boxes, it's more than likely your anti-virus software is locking them to scan at the time BE is trying to back them up or the service account has reached the max number of attachments and emails it's allowed to access at one time. Make sure scheduled full scans and scheduled backups are not running into each other. The way anti-spam software is configured can create this too, but I assume you don't have any since install spam filters is the ultimate goal here? If it's your anti-virus software, this can be tested by turning it off for a backup job.

There is also a known issue with Exchange 2003 that has to do with the limit of attachments and messages that one account can access at a single time. I think it's 100 attachments and 250 messages? But there is a pretty simple registry edit that will fix this.

My guess is that your virus software is locking them at the same time BE tries to back them up. I have a link somewhere to the KB artickle that tells you everything that should be on your exclude list for your Exchange 2003 server's virus scanner. I think we have gotten way off topic now though (sorry rindi). I'll dig and find them if you need them. Just email me directly at {Email Address removed by rindi, PE Storage}

Sorry rindi, didn't know that. Thank you.

no problem

Well I guess I could create new rule with SYmantec with all the key words from spam. But again i don't how to eliminate spam with images in it (imf can't do that either I think). Thanks for the link.

I really would like to thank both of you for the execellent support so far. Now how can I up the points and share it between both? And I guess I have enough info for now to get started at least. Will do that tomorrow and will let you guys know. Thanks!!

500 points is all you can assign. There is a "Split Points" in the box where you can post new comments, and then you just select those answers you want to accept as the most helpfull.

Give all the points to rindi for being a board admin that let us get waaaaay of topic I think. Thanks again rindi.

Symantec Mail Security's content filter rules will quarantine mail based on attachments as well as text or sender. Do some kind of cost benefit analysis to show the cost of you dealing with it and what it's costing the company versus the cost of subscribing to a filter service. Trying to build a list yourself is an all day affair if you have many users. Also, if your off for the weekend and an important email comes to the boss with an attachment or quarantined for something else, it sits there waiting to be approved for 2 or 3 days.

Mail Security gives you a way of subscribing to a filter service. Then all known spam will be rejected/deleted and you don't have to worry about it. We use N2H2/Centian and have been happy with them. http://www.securecomputing.com/index.cfm?skey=22&menu=products