Solved

Smart Sync or Binary Diff Tool

Posted on 2010-09-01
6
697 Views
Last Modified: 2012-05-10
We have the need to transfer copies of our builds to India for testing. These builds are about 4G in total but have huge amounts which do not change between files.
So the build contains a combination of many files and big CABs (or equivalents) . Some of the binaries might only change by a few bytes for the version stamping.

=> Is there a copy or sync tool out there which is really smart and can build remotely (either directly or by a copy and process) a copy of a file area based on a "seed" area. This seed area on the remote site would simply be a copy of the previous build ? It would need to recognise the common parts in the binary files.
Context : Windows Server 2003 / 2008

Is this essentially Microsofts DFS Replication ?

Now a normal sync tool would be file for the stand alone files that don;t change but hopeless for a very large binary which only has say 2% of the file changed. It would have no choice but to copy the large binary file.

Thanks
0
Comment
Question by:greg_roberts
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 2
6 Comments
 
LVL 3

Accepted Solution

by:
AndyEastham earned 125 total points
ID: 33596434
Hi,

Have you looked a rsync?
http://www.samba.org/rsync/

Or for windows:
http://www.aboutmyip.com/AboutMyXApp/DeltaCopy.jsp

Andy
0
 
LVL 18

Expert Comment

by:BigSchmuh
ID: 33596493
Most international development team use some Repository tool ( like "SubVersioN" ) that allows :
-any "client" to replicate locally all (or a branch) of it (using CheckIn/CheckOut methods to avoid data write conflicts)
-versioning all files in the Repository
-a global Repository backup

You may although use a replication system like DFS to synchronize a directory in 2(+) locations but it requires sharing some Domain info

If your build does change all files at every new build, both Win2008 R2 DFS and SVN should allow for a differential/incremental evaluation...that is to be tested anyway
0
 
LVL 18

Expert Comment

by:BigSchmuh
ID: 33596521
SVN (SubVersioN)
   http://en.wikipedia.org/wiki/Apache_Subversion
On Windows, I have been using http://www.visualsvn.com/server (and its VisualSVN client) with success
0
Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 

Author Comment

by:greg_roberts
ID: 33601586
Andy
Thanks. i looked at rsync but could not run. No csyxxx.dll (some library dll to convert between windows to linux)

BigSchmuh
We use TFS 2080 for our source control and subversion is more of a source control tool than a replication tool.

Normal "sync" tools result in the 4G build needing 2G of b/w to copy. I am hoping that a block level trial will reduce this dramatically. Was trialing R1Soft but can't get the s/w to work. This has block level copy feature.
0
 
LVL 18

Assisted Solution

by:BigSchmuh
BigSchmuh earned 125 total points
ID: 33602245
My suggestion is to try using SVN (a source control tool) to replicate a directory locally:
-costless
-should allow for some incremental updates (Unsure with your binaries)
0
 

Author Closing Comment

by:greg_roberts
ID: 33602265
Got rsync working using the url within the url for cwRsync but haven't got all the options right as i end up with the same sync transfer count as other tools (even using --inplace). Suspect issue with file times being used insead of CRC of file content.

As builds have complete binaries and files, exe, pdb, dlls , cab files etc i can't see how the SVN solution will help but thats for contrinuting.
0

Featured Post

Simplifying Server Workload Migrations

This use case outlines the migration challenges that organizations face and how the Acronis AnyData Engine supports physical-to-physical (P2P), physical-to-virtual (P2V), virtual to physical (V2P), and cross-virtual (V2V) migration scenarios to address these challenges.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Dropbox for Free 5 91
EMC vplex question 1 64
logical drive not on preferred path 3 79
Foreign Configurations found on adapter PERC H710, Raid 5 2 54
Having issues meeting security compliance criteria because of those pesky USB drives? Then I can help you! This article will explain how to disable USB Mass Storage devices in Windows Server 2008 R2.
Create your own, high-performance VM backup appliance by installing NAKIVO Backup & Replication directly onto a Synology NAS!
This video teaches viewers how to encrypt an external drive that requires a password to read and edit the drive. All tasks are done in Disk Utility. Plug in the external drive you wish to encrypt: Make sure all previous data on the drive has been …
This tutorial will walk an individual through the process of installing the necessary services and then configuring a Windows Server 2012 system as an iSCSI target. To install the necessary roles, go to Server Manager, and select Add Roles and Featu…
Suggested Courses

734 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question