Solved

iSCSI LUN Backup Using Synology NAS Over Internet - Slow Speeds

Posted on 2015-01-13
12
1,960 Views
Last Modified: 2015-01-20
I am trying to get an off-site backup solution running for my company. I am using an iSCSI LUN on a Synology RackStation for data storage at my office, and I have another Synology RackStation at my home. The LUN is thin provisioned for 2TB, but we're currently only using ~25GB.

I opened up the required TCP ports on my home firewall (873 & 3260), setup the backup on the NAS at my office, and initiated the first backup. I have an upstream speed of ~8Mbps at my office, and downstream of 50Mbps at my home (both are business class connections), and the resource monitor of each NAS is showing ~1MB/s for the transfer speed, so it's correct. In theory, it should take a little over 7 hours to complete the backup.

After about 12 hours, I checked the status and only 6% was completed. I thought that maybe since the LUN is provisioned for 2TB, it was trying to transfer 2TB of data, but that would be over 120GB of data in 12 hours, and there's no way my connection could upload 10GB per hour. I checked the resource monitor, and it was transferring at about 20KB/s. I tested the network connections at my home and office, and both were fine. I decided to stop the transfer and try it again. It started out again at ~1MB/s, but after about 6 hours, it was back down to ~20KB/s.

I'm at a loss of what the problem could be. I am now about an hour into another attempt for the backup. It's still reading between 700KB/s and 1.3MB/s for the transfer speed, but the backup dialog on the NAS at my office is still says 0%. If it has been transferring at ~1MB/s for an hour, it should be somewhere around 14% completed.

Hopefully someone has some experience with Synology or LUN backups and can give me some insight. I am using a SonicWALL TZ 215 firewall at my office and an ASUS RT-AC87U at my home. Each RackStation has 2x 1GbE ports, which I have bonded and given a single static IP address.
0
Comment
Question by:steven_theck
  • 5
  • 3
  • 2
  • +2
12 Comments
 
LVL 118

Assisted Solution

by:Andrew Hancock (VMware vExpert / EE MVE)
Andrew Hancock (VMware vExpert / EE MVE) earned 167 total points
ID: 40549869
Why not use Amazon S3 Storage or Amazon Glacier Backup both supported on the Synology NAS.

Amazon will have much faster data speeds into their datacenter, than your home solution!

and the cost of storage is very inexpensive.
0
 
LVL 45

Assisted Solution

by:Craig Beck
Craig Beck earned 167 total points
ID: 40549870
An ISP I use in the UK (among others) has a good table which shows what to expect...

http://www.zen.co.uk/business/online-data-backup/backup-solutions/data-transfer-times.aspx

That indicates that a 10Mbps line would be able to shift around 100GB of data in 24hours, so your expectation is correct.

It may be that one of the ISPs/connections is rate-limiting certain protocols between certain hours.  Does either ISP enforce any traffic shaping, etc?

Also, I would advise that you look at encrypting the iSCSI traffic if you're not already, especially if it's company data and there is a chance of personal details being transmitted across the internet.  You could get hugely fined if there is a data-protection breach and I think that many would consider this to be a breach in itself anyway.
0
 
LVL 30

Assisted Solution

by:pgm554
pgm554 earned 166 total points
ID: 40550198
What's the backup program?
You might want to backup the NAS in the office first,then set it up for backup at the house using a block level (rysnc) method so that only the delta changes are replicated over the WAN.

https://www.synology.com/en-us/knowledgebase/tutorials/461
0
 

Accepted Solution

by:
steven_theck earned 0 total points
ID: 40550276
Let me answer everyone's posts, and then I'll provide the information I discovered today.

@Andrew Hancock, I have actually never looked into S3 or Glacier. If I'm understanding it correctly, we would pay $0.25/month for 25GB of storage, and nothing more as long as we don't retrieve the data, which would be $0.09/GB for retrieval. If that's correct, then that's an absolute steal and I'll probably end up using Glacier as an additional backup option.

@craigbeck, I have the same ISP at my home and office, and they are both business-class connections that do not throttle. When the transfer rates are very low, I am still obtaining full speeds performing other actions. And I forgot to mention, but yes, the data being sent is encrypted.

@pgm554, the backup software is built into the Synology NAS. It is essentially a block-level method that splits the LUN into 4GB files.

Today, I attempted to backup the LUN to a folder on the local NAS. It took between 1-2 hours to complete the backup, but it was successful. I opened the directory on the NAS that it backed everything up to, and it contained 512 4GB files. That's 2TB of data. However, when I looked at the Storage Manager on the NAS, it shows 26GB of the volume being used.

I'm not extremely familiar with iSCSI LUNs and how the block-level backups are done, but I assume that it must make each 4GB block initially and changes will only be made to the blocks that contain data. So even though the blocks show up as 4GB files, most of the data is zeros. Yet when I transfer the data, the remote host sees it as 2TB of data. I'm sure my logic is not completely accurate here, but I decided to test it out using Andrew's suggestion of Amazon Glacier. I created an AWS account, setup the Glacier backup, and the progress within the Glacier app showed that it was attempting to back up 2.00TB of data.

So now, I'm going to create a new iSCSI LUN with a much lower provisioned capacity (maybe 50GB), copy the data to the new LUN, and try the backup again tonight to see how long it takes to complete. For 50GB at 8Mbps, it should take a little over 14 hours. And on the NAS, I can always increase the provisioned capacity of the LUN at any time. I'll report back tomorrow with the status.
0
 
LVL 118
ID: 40550290
I have actually never looked into S3 or Glacier. If I'm understanding it correctly, we would pay $0.25/month for 25GB of storage, and nothing more as long as we don't retrieve the data, which would be $0.09/GB for retrieval. If that's correct, then that's an absolute steal and I'll probably end up using Glacier as an additional backup option.

Correct, that's why I don't understand why you are messing around with any other solutions!

You cannot beat or compete with Amazon!
0
 

Author Comment

by:steven_theck
ID: 40550330
@Andrew Hancock, the main reason I'm trying to get the Synology-to-Synology solution working is because I already had both NAS devices in place (at my office for file storage and at home for my personal storage), so why not use them? Also, I prefer to have a physical backup available. I'm currently backing up our data daily to USB drives, but the Synology-to-Synology backup would give me an automated physical backup.
0
Give your grad a cloud of their own!

With up to 8TB of storage, give your favorite graduate their own personal cloud to centralize all their photos, videos and music in one safe place. They can save, sync and share all their stuff, and automatic photo backup helps free up space on their smartphone and tablet.

 
LVL 118
ID: 40550368
less chance Amazon will be destroyed or house burgled, and they have more bandwidth!
0
 
LVL 16

Expert Comment

by:Gerald Connolly
ID: 40550593
I think those numbers from Zen are a little over optimistic. They seem to assume that a 10Mbps = 1.2MBps so gives 5GBph, they obviously haven't heard of the 10 bits per Byte overhead of a comms link, or the fact it's really really hard to drive a link at 100% for an extended period.
Getting 2GBph would be a result!

50GB over a 8Mb link is max 800KBps = 17.5 hours, probably more like 25-30 hours
0
 

Assisted Solution

by:steven_theck
steven_theck earned 0 total points
ID: 40551821
The offsite backup to my home Synology RackStation successfully completed, and the average data transfer rate was about 9Mbps. I now have a backed up folder on my home NAS with 15 4GB files (I increased the provisioned capacity to 60GB). I deleted a few unneeded files from the iSCSI target and I'm performing another backup to see if it's performing incremental or full backups each time. Thanks everyone for your suggestions.
0
 
LVL 45

Expert Comment

by:Craig Beck
ID: 40551915
@Gerald - actually it's not far off, and they're conservative, and your calculation is slightly off too :-)

Of course, those figures are aimed at what they estimate you should expect when backing up to their Cloud using one of their circuits, but I think practically doubling the predicted time is a bit too harsh.

Also, given that this scenario uses the same ISP throughout I think we can ignore some of the other issues which may affect this, such as irregular QoS markings or different traffic policing policies between providers, etc.
0
 

Author Comment

by:steven_theck
ID: 40552481
Just wanted to post an update that the Synology iSCSI LUN backup does not perform a full backup every time. It will only update the 4GB blocks that have had changes made to them. I deleted and added quite a few files to the iSCSI target, only 3 of the 4GB blocks had updated modified dates, and it completed in about 3 hours.
0
 

Author Closing Comment

by:steven_theck
ID: 40559349
Best solution: Synology LUN Backup will backup the thin provisioned capacity of the LUN, not just the storage space currently being used.
0

Featured Post

How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

Join & Write a Comment

This article is an update and follow-up of my previous article:   Storage 101: common concepts in the IT enterprise storage This time, I expand on more frequently used storage concepts.
Don’t let your business fall victim to the coming apocalypse – use our Survival Guide for the Fax Apocalypse to identify the risks and signs of zombie fax activities at your business.
This tutorial will walk an individual through the steps necessary to enable the VMware\Hyper-V licensed feature of Backup Exec 2012. In addition, how to add a VMware server and configure a backup job. The first step is to acquire the necessary licen…
To efficiently enable the rotation of USB drives for backups, storage pools need to be created. This way no matter which USB drive is installed, the backups will successfully write without any administrative intervention. Multiple USB devices need t…

744 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now