Link to home
Start Free TrialLog in
Avatar of posae
posae

asked on

DFS Replication Affected by Crypto Locker

Hello,

we have (2) 2008 servers using DFS replication.  We recently took over this account and do not have experience with DFS.  

A server at location (B) had all the files on it encrypted by a virus.  It then began replicating to the server at location (A) which is the main Domain Controller (SBS 2008).  we stopped the DFS replication quickly so that the server in location (A) only had 50 files replicate to it.

we have snapshots of how the files looked prior to the virus.  

What i need to do is restore all the files in location (B) with the good files we have in location (A).  Office B is 2 hours away and they do not have a very fast internet connection.  Right now DFS is disabled

Is there any to copy the good files at site (A) to a usb drive and take them to Site (B) so that we can restore those files and then turn on DFS?  Or do we need to re-push all files from site (A) to site (B) using DFS?

THanks!
ASKER CERTIFIED SOLUTION
Avatar of John
John
Flag of Canada image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
SOLUTION
Avatar of Bembi
Bembi
Flag of Germany image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of posae
posae

ASKER

thanks bembi.  we found the machine encrypting everything and got it off the network.  basically the bad site (B) or let's call canada office has all files encrypted.  The other office (A) called US office is ok.  

There is 65 GB of data (approx 200k files).  I can delete the data on the canada site as we have a good backup of the data.  the issue is getting the canada site up - to -date with the backup data.  

If i delete all the data on the canada side, that would delete the US Side.  i have the data to put back in the US Side.  Can i then take that same data up to the Canada site on USB or do i have to initialize the Canada folders using replication from the start?
If i delete all the data on the canada side, that would delete the US Side.    <-- Yes, it should, but check the US side for any files.

Can i then take that same data up to the Canada site on USB or do i have to initialize the Canada folders using replication from the start?  <-- After you have checked, you should probably initialize the Canada folders and restore the data. You do not want the synchronization to do something screwy because it thinks it has a truckload of data somewhere.
Avatar of posae

ASKER

Thanks John.  the US office is the main server and the location of the backup.  Right now, the US server is clean and has the up-to-date version of the data.

The only thing i have left to do is fix the Canada copies and to do this is the most efficient way.  I appreciate your help
If you have two copies of the same thing on two sites, they will appear different: Clean data on US, Restored Data on Canada. That may cause issues.

Can you restore from US to Canada ? That is just overwrite the Canada site?  I am not certain about that.
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
You basically have to go through a prestaging, transferring the 65gb back from site a to site b and then restoring the files there, then reestablishes dfs replication from A to B only. In this case the DFS will run differential comparison ......
What will happen any files changed on A will replicate to B, any files modified on B will not replicate out.
You can use dfs management to generate a report with site B as the reference to see what and how many files will replicate back once B to A replication is activated...
Avatar of posae

ASKER

Thanks for all the comments.  I have spent much time on this with no success.  DFS replication group had 3 folders replicating.

Users
US
Canada

The primary was the US server for all.  The virus hit the canadian server.  I deleted all data in the US and Canada folders on the Canadian server.  All data on the US server is clean.  I deleted the US and Canada replication folders from the primary server.  The Users folder is still replicating successfully and i tested.

I am trying to recreate the replication folders in DFS but no matter how i add them, it is not successful.  do i have to create new shares/folders to get this to work?
What do you mean you deleted folders?
You need to kickout the Canada node from the replication group of the folders where crypto took hold.
Cleanup the Canada side, delete/restore the data from backup.
Read the Canada node back into the replication group.

A deletion while replication is established will still be "pending" even though you disabled Canada to us replication, the us to Canada will trigger a delete to match the one you made.



You mah have to use the dfsutil to clearup the dfs replication/space.
Did you check the Dcs sysvol to make sure it was not compromised by the crypto......

You may have to use different folders to reestablish I.e. Instead of using US, Canada, try using

You would also need to prestage (restore files from backup) before establishing replication to minimize the amount of data that needs to go from US to Canada..........
Avatar of posae

ASKER

Thanks arnold.  in the DFS mgt, i removed the US and Canada folders from the replication group.  right-clicked and deleted.  after i did that, I deleted all the data on the Canada member.  As mentioned before, the users replication is working perfectly, so i am trying to add a new entry for the US folder (smaller) in the replication group.  I am going into DFS Mgt console, going to the existing replication group with the one Users folder replication definition and adding a new one on the primary (US) server.  after i add, the folder does not show up in the DFS Mgt console on the Canadian server.  

My goal it to just set up the syncing and re-send the data from the US site to the canadian site understanding that it will take forever.
What do you mean you re adding a new folder to a replication group?
Usually, each folder should have its own replication group. (Or at least that is how I have been configuring them)

You should create the folders on the respective servers setting the permissions (security) that you want domain based groups, ....
Making changes after the folders are populated with files, will trigger replication events.............

Why not transfer use the 65GB backup drive out there, restore it and reestablish replication?

Network saturation might render the office ............ Causing you more trouble that you have now.

Bleeping computers have a writeup on GPO with description on how to reduce the crypto virus attack vector for infiltrating your system/s by using software restriction policy denying users rights to run exe, zips and exe's from zips/..... From unwanted locations..........
Avatar of posae

ASKER

Thanks.  i was able to set up the folders in the replication group.  about 10GB of the 60GB replicated last night.  Now i am noticing that it is not sending the rest.  I created a test document on the one site and and it replicated immediately to the other, but there is still 50 GB of data to go.  Is there any way to see what specific folders are not syncing to the second site?
It has to create all the references in the staging folder under DFSRPrivate hidden folder within the replication folder.
folder
      dfsrprivate\STaging
Check event log on both sides dfs.....backups stop replications.
Avatar of posae

ASKER

ok...so the one new folder is finished and all is well there.

i added the 3rd and final folder to the US server.  After about a half hour or so, the folder that i added to the US server is not showing in the list on the Canada server.  I did a dfsrdiag pollad /member:US from the canada server but the replication folder is not showing on the canadian server dfs mgt

I checked the logs on the US server and it looks like the us server crawled the local folder to replicate and then when it tried to contact the Canada servers content set, it did not find anything:

20151219 10:18:16.907 1548 SYSM  4445 Migration::SysVolMigration::DeleteRoMember [MIG] Current global state is 'Start'
20151219 10:19:06.236 1548 DOWN  3991 DownstreamTransport::EstablishSession Failed on connId:{1F6EBB71-201D-4907-B6D9-C98BA2C602B4} csId:{792BBEA9-3474-4883-BB77-931C8A1D1E1A} rgName:Stetron Replication Error:
+      [Error:9027(0x2343) DownstreamTransport::EstablishSession downstreamtransport.cpp:3984 1548 C A failure was reported by the remote partner]
+      [Error:9028(0x2344) DownstreamTransport::EstablishSession downstreamtransport.cpp:3984 1548 C The content set was not found]
20151219 10:19:06.236 1548 INCO  3722 InConnection::RestartSession Retrying establish contentset session. connId:{1F6EBB71-201D-4907-B6D9-C98BA2C602B4} csId:{792BBEA9-3474-4883-BB77-931C8A1D1E1A} csName:Markham After Crypto - Added on SBS
20151219 10:19:06.236 1548 INCO   850 [WARN] SessionTask::Step (Ignored) Failed, should have already been processed. Error:
+      [Error:9027(0x2343) InConnection::EstablishSession inconnection.cpp:3813 1548 C A failure was reported by the remote partner]
+      [Error:9027(0x2343) DownstreamTransport::EstablishSession downstreamtransport.cpp:4005 1548 C A failure was reported by the remote partner]
+      [Error:9027(0x2343) DownstreamTransport::EstablishSession downstreamtransport.cpp:3984 1548 C A failure was reported by the remote partner]
+      [Error:9028(0x2344) DownstreamTransport::EstablishSession downstreamtransport.cpp:3984 1548 C The content set was not found]
20151219 10:21:02.956 1548 DOWN  3991 DownstreamTransport::EstablishSession Failed on connId:{1F6EBB71-201D-4907-B6D9-C98BA2C602B4} csId:{792BBEA9-3474-4883-BB77-931C8A1D1E1A} rgName:Stetron Replication Error:
+      [Error:9027(0x2343) DownstreamTransport::EstablishSession downstreamtransport.cpp:3984 1548 C A failure was reported by the remote partner]
+      [Error:9028(0x2344) DownstreamTransport::EstablishSession downstreamtransport.cpp:3984 1548 C The content set was not found]
20151219 10:21:02.956 1548 INCO  3722 InConnection::RestartSession Retrying establish contentset session. connId:{1F6EBB71-201D-4907-B6D9-C98BA2C602B4} csId:{792BBEA9-3474-4883-BB77-931C8A1D1E1A} csName:Markham After Crypto - Added on SBS
20151219 10:21:02.956 1548 INCO   850 [WARN] SessionTask::Step (Ignored) Failed, should have already been processed. Error:
+      [Error:9027(0x2343) InConnection::EstablishSession inconnection.cpp:3813 1548 C A failure was reported by the remote partner]
+      [Error:9027(0x2343) DownstreamTransport::EstablishSession downstreamtransport.cpp:4005 1548 C A failure was reported by the remote partner]
+      [Error:9027(0x2343) DownstreamTransport::EstablishSession downstreamtransport.cpp:3984 1548 C A failure was reported by the remote partner]
+      [Error:9028(0x2344) DownstreamTransport::EstablishSession downstreamtransport.cpp:3984 1548 C The content set was not found]
20151219 10:23:02.216 1548 DOWN  3991 DownstreamTransport::EstablishSession Failed on connId:{1F6EBB71-201D-4907-B6D9-C98BA2C602B4} csId:{792BBEA9-3474-4883-BB77-931C8A1D1E1A} rgName:Stetron Replication Error:
+      [Error:9027(0x2343) DownstreamTransport::EstablishSession downstreamtransport.cpp:3984 1548 C A failure was reported by the remote partner]
+      [Error:9028(0x2344) DownstreamTransport::EstablishSession downstreamtransport.cpp:3984 1548 C The content set was not found]
20151219 10:23:02.216 1548 INCO  3722 InConnection::RestartSession Retrying establish contentset session. connId:{1F6EBB71-201D-4907-B6D9-C98BA2C602B4} csId:{792BBEA9-3474-4883-BB77-931C8A1D1E1A} csName:Markham After Crypto - Added on SBS
20151219 10:23:02.216 1548 INCO   850 [WARN] SessionTask::Step (Ignored) Failed, should have already been processed. Error:
+      [Error:9027(0x2343) InConnection::EstablishSession inconnection.cpp:3813 1548 C A failure was reported by the remote partner]
+      [Error:9027(0x2343) DownstreamTransport::EstablishSession downstreamtransport.cpp:4005 1548 C A failure was reported by the remote partner]
+      [Error:9027(0x2343) DownstreamTransport::EstablishSession downstreamtransport.cpp:3984 1548 C A failure was reported by the remote partner]
+      [Error:9028(0x2344) DownstreamTransport::EstablishSession downstreamtransport.cpp:3984 1548 C The content set was not found]

I have 2 folders replicating perfectly but this new one is not.
Voice it time, as dfs relies on the ad so it might take the ad dfs addition to replicate, and then the dfs replication will kick in.  Probably 15 minutes should finalize the connection, and begin replication.

Backups, turn off dfs replication.....severing the connection between partners.
The amount of space dedicated to staging may be impacting pergormance if set too low.  You might want to raise the value to minimize disk IO for staging area frequent cleanup, if you have the space, make the staging area as the largest single file....for the duration of the sync.
Oh, only one side has content, that might be the notice is Canada has no content, meaning everything the US side has needs to be catalogued, staged and sent.