SCR not replaying logs -- ReplayLogQueueLength keeps growing

Having an issue setting up SCR between a NY server and a UK server.Logs are sent from NY to UK just fine, but they don't get rolled up into the database. I set this up about a week ago and the number of transaction logs keeps growing and growing.  Not sure which is the chicken and which is the egg, but when I do a get-storagegroupcopystatus -standbymachine, the LatestFullBackupTime parameter is blank (even though the backup are run regularly and logs are flushed on the source as expected). I've tried several things including:

*  disable and re-enabling SCR
* changing the replaylagtime from the default to 30 seconds
* restarting the replication service
* installing the lastest rollup (sp1 rollup 4) on both machines
* reboot each machine

Nothing seems to have any effect. Event Log doesn't seem to be giving me any information either. Below are the output of get-storagegroupcopystatus and test-replicationhealth commands. Any help you be greatly appreciated:

get-storagegroupcopystatus -identity "svr01exch\first storage group" -standbymachine svr02exch

Identity                         : SVR01EXCH\First Storage Group
StorageGroupName                 : First Storage Group
SummaryCopyStatus                : Healthy
NotSupported                     : False
NotConfigured                    : False
Disabled                         : False
ServiceDown                      : False
Failed                           : False
Initializing                     : False
Resynchronizing                  : False
Seeding                          : False
Suspend                          : False
CCRTargetNode                    :
FailedMessage                    :
SuspendComment                   :
CopyQueueLength                  : 3
ReplayQueueLength                : 2753
LatestAvailableLogTime           : 10/23/2008 6:38:33 PM
LastCopyNotificationedLogTime    : 10/23/2008 6:38:33 PM
LastCopiedLogTime                : 10/23/2008 6:37:21 PM
LastInspectedLogTime             : 10/23/2008 6:37:21 PM
LastReplayedLogTime              : 10/18/2008 3:55:34 PM
LastLogGenerated                 : 693557
LastLogCopyNotified              : 693555
LastLogCopied                    : 693554
LastLogInspected                 : 693554
LastLogReplayed                  : 690801
LatestFullBackupTime             :
LatestIncrementalBackupTime      :
LatestDifferentialBackupTime     :
LatestCopyBackupTime             :
SnapshotBackup                   :
SnapshotLatestFullBackup         :
SnapshotLatestIncrementalBackup  :
SnapshotLatestDifferentialBackup :
SnapshotLatestCopyBackup         :
OutstandingDumpsterRequests      : {}
DumpsterServersNotAvailable      :
DumpsterStatistics               :
IsValid                          : True
ObjectState                      : Unchanged

Server          Check                      Result     Error
------          -----                      ------     -----
SVR02EXCH       ReplayService              Passed
SVR02EXCH       SGCopySuspended            Passed
SVR02EXCH       SGCopyFailed               Passed
SVR02EXCH       SGInitializing             Passed
SVR02EXCH       SGCopyQueueLength          Passed
SVR02EXCH       SGReplayQueueLength        Passed
SVR02EXCH       SGStandbyReplayLag         *FAILED*   Standby Continuous Replic
                                                      ation for storage group '
                                                      SVR01EXCH\First Storage G
                                                      roup' on computer 'SVR02E
                                                      XCH' has configured repla
                                                      y lag time of 00:00:30 an
                                                      d a replay queue length o
                                                      f 2748. However, log repl
                                                      ay is actually behind by
                                                      a period of 733337.18:23:
                                                      20.4279554 which is great
                                                      er by more than the Failu
                                                      re window of 01:00:00. Ei
                                                      ther log replay is slow o
                                                      r is not making progress.
                                                       Please investigate furth

Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Waiting for your reply on the other forum
dinkum305Author Commented:
Regarding the size of the database: This database is 80GB. I have another 80GB database I want to replicate as well after I get past this issue.

Ok to get over this issue change the ReplayLagTime cmdlet to assingn 15 - 30 minutes and the TruncateLagTime cmdlet with 60 mins (this matches the failure point time).

If it does not work - you may want to suspend and resume storage group copy. Restarting the Replication service might not be a good idea at the moment.
Problems using Powershell and Active Directory?

Managing Active Directory does not always have to be complicated.  If you are spending more time trying instead of doing, then it's time to look at something else. For nearly 20 years, AD admins around the world have used one tool for day-to-day AD management: Hyena. Discover why

Well I would go and Seed the Target. See a half updated DB on target would not be a good idea and to keep it like that for long even worse.
Else try this.....
  • disable-storageggroupcopy -standbymachine YOURSERVER
  • When you are prompted to delete files in target, ignore that and do not delete.
  • Enable SCR again .... enable-storagegroupcopy
  • See if that initiates the Relay
dinkum305Author Commented:
Thanks both,

I just entered the following commands:

Disable-StorageGroupCopy -identity "svr01exch\first storage group" -StandbyMachine svr02exch

Enable-StorageGroupCopy -identity "svr01exch\first storage group" -StandbyMachine svr02exch -ReplayLagTime 0.0:15:0 -TruncationLagTime 0.1:0:0

Will see if that helps. Is there anything I should do to prime it to replay (such as running a backup on the source) or should it start playing the logs into the target db on it's own?

Thanks again
Personally i would recommend re-seeding however, i wanted to provide this option if my plan A fails. However, now since i have mentioned this - if my planA fails - simply re-seed however keep the same counters fixed when you work with blank DB on target.

There isn't anything that needs to be done on source server. Relax.
dinkum305Author Commented:
get-storagegroupcopystatus now yields:

Identity                         : CIF01EXCH\First Storage Group
StorageGroupName                 : First Storage Group
SummaryCopyStatus                : Failed
FailedMessage                    : Log file action LogCopy failed for storage g
                                   roup CIF01EXCH\First Storage Group. Reason:
                                   The handle is invalid.
CopyQueueLength                  : 0
ReplayQueueLength                : 0
LastLogGenerated                 : 693709
LastLogCopyNotified              : 0
LastLogCopied                    : 0
LastLogInspected                 : 0
LastLogReplayed                  : 0
IsValid                          : True
ObjectState                      : Unchanged

Any other ideas or am I forced to reseed? (Not an easy task for 80 GB over a T1.)
If its possible - please dismount stores on source - copy all the log files to a different directory from source storage group - this ensures we do not have any log files except our sweet little 80Gb database file.

Now, remount your stores - and work with the same command again.

If a dismount and remount is not possible try removing logs except for last 20 odd files.

then work with the same command again.

Else, if it doesn't work - re-seed is what i visualize further.
dinkum305Author Commented:
Can't dismount, but I was able to copy all the log files except for e00.log. I put them in the log directory on the target, but I'm getting a different error now. I think its telling me which log has the invalid handle

Event Type:      Error
Event Source:      ESE
Event Category:      General
Event ID:      481
Date:            10/23/2008
Time:            11:45:44 PM
User:            N/A
Computer:      SVR02EXCH
Microsoft.Exchange.Cluster.ReplayService (5320) Log Verifier e00 63835064: An attempt to read from the file "\\SVR01EXCH\1fd53a2e-3679-4740-94bb-c8af43ab2327$\E00000A95C5.log" at offset 593920 (0x0000000000091000) for 65536 (0x00010000) bytes failed after 1 seconds with system error 6 (0x00000006): "The handle is invalid. ".  The read operation will fail with error -1022 (0xfffffc02).  If this error persists then the file may be damaged and may need to be restored from a previous backup.

For more information, click

I re-copied that one log from the source and now I'm back to the "handle is invalid" point I was before. I'm a bit concerned that I have corruption on the source database.

-1022 normally comes along when database is not being able to be read properly - could be various reason - disk / third party applications / high RPC perf issue. Plenty.

I guess let us not waste time and do a reseed by removing all those database and log files from target, fresh reseed makes sense. i know it sounds a bit frustrating but thats how E2k7 is - if it works its great if not its nasty

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
dinkum305Author Commented:
OK, well thanks for your assistance on this. Probably looking at a week at least to reseed, but I feel a little better about it knowing that it can't be helped.

I'm actually in the process of seeding the second database, hopefully I'll be ready to setup SCR for that this weekend. Maybe I'll have better luck.
Wish you all the very best.
dinkum305Author Commented:
Well, I setup SCR for the other store and it seems to be behaving properly. Hopefully I will be done seeding this weekend and I can retry with the original store.
How do you qualify strange - could you pen down some words - that would be helpful.
:-) !
dinkum305Author Commented:
Not sure what you mean by that comment.
"Well, I setup SCR for the other store and it seems to be behaving properly"

I am not even sure what does this mean, just wanted to ask you - what's wrong ??
Well, chill it should be fine :-)
dinkum305Author Commented:
Was just giving an update to the thread in case it would help any lurkers.

The original issue was that the transaction logs weren't replaying on the target. You advised me to re-seed which I'm doing. But, in the meantime, I've got a second database seeded ... I setup SCR and I'm not seeing the original issue. I'm hopeful that when the other database is done seeding this weekend, I'll be all set.
Ah ok the second store is seeded and no errors till now, i thought the second one has also gone to the saved...i hope your weekend doesn't get spoiled and the first store behaves as good as the second.
You should be.....and yes thanks for the would 100% help others too
I had a similar problem. I have solved it having restarted services Microsoft Exchange Replication Service. Good luck!
dinkum305Author Commented:
After reseeding, everything looks good. A bit of a pain, but alls well that ends well
Wow that is such a good end to such a long story.

It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.