asked on

Storage query

Hi,

I am planning to purchase a a Dell dedupe storage appliance to copy 16 TB of data ( from NetApp SVMs which acts as file servers) using Robocopy or EMCopy (as the Veeam backup currently not supporting to backup from NetApp SVM file servers, CIFS volumes) in a weekly basis. I want to retain the backup for a month. When I do this using Robocopy, could you let me know whether the storage appliance dedupe apply or not on the backed up files.

Kind regards

Benny Shtark

Hi,

I can only speak about dedupe in general or dedupe for netapp, but i guess for dell or other vendors it works similarly.

Basicly, when you enable Dedupe on a volume, you specify schedule when you want it to run.
when running, it will scan all blocks of the volume (in case of netapp its 4K blocks), and create "table of hashes", then it will search for identical hashes of each block.
when found, it will "delete" duplicate blocks and create a "pointer" (like symbolic link in linux) to first unique block, thus freeing up space.

Its important to understand that dedupe doesnt "always works", it effect can only be seen after "scan" was complete.
Also, the process takes memory and cpu resources of the storage, so access to deduped volumes during the "deduplication scan" can be slower.

the greatest effect for deduplication can be seen on volumes holding simmilar virtual machines (like vmware).
its NOT recommended to use deduplication if you have a lot of compressed files, as compression is messing up all the blocks (they become too random),
also, not recommended to use dedup on volumes for databases, as it can make database slow (during the deduplication scan), especially on large DBs.

Its totally fine to use it on "backup" machine where you do not really care if deduplication scan is running in background every now and then..

Now about your question with robocopy:

If you copy over backup files using robocopy to a dedup-enabled volume, just make sure that files are not encrypted or compressed (just raw backups), then the dedup will take over and free up much disk space by eliminating duplicate blocks.

I have no experience with dedup on Dell, but on netapp it works really nice and very safe too.

check out this to learn more

cheers

AKA

ASKER

Thanks Benny... HP Storeonce and Dell Data Domain storage appliances are using inline deduplication which works really well when we use Veeam backup solution. My query is how the dedupe works when using Robocopy as the storage is not using post-process deduplication.

Regards,

ASKER CERTIFIED SOLUTION

Benny Shtark

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial