Link to home
Start Free TrialLog in
Avatar of SKYENERGY2020
SKYENERGY2020Flag for Canada

asked on

How bulid my own offsite back up solution

I have an server and I have some 2 TB external had disks ,I want to use them as offsite back up to store data . and I want to generate money of providing data back service form some clients

Things I want to know


1- What is best software to manage the back up
2 What  type of hardware and software setup I need  to do in order to utilize the old server and external hard disk to  back up the date from remote location
 
3- The software I want  should has the  ability to generate report  about the size of backed up  data and used internet traffic
 


Thank you
Avatar of serialband
serialband
Flag of Ukraine image

Don't do it yourself.  Since you have to ask, you're not going to do it correctly.

1.  That's entirely subjective.  It depends what you're backing up.  Windows, Linux, OS X, Unix?  What needs to be backed up?  VMs, whole disk images, files?

2.  Are you running it as a "cloud" service?  Are you running it yourself?  Where is it located?  What sort of security do you have for your site?  2TB is not a lot of disk space these days.

3.  That depends on the tool you're using.
There are some open source SAN tools available so you could use one of those to turn your system into cloud storage.

*However* you may be reinventing the wheel. Many existing cloud providers would let you sign up as a reseller. Takes the onus off you to maintain the hardware.
1- What is best software to manage the back up.

To answer this you'll first require mentioning what OS variations you'll be backing up + if you'll be restoring backups or providing the backup files to clients.

For example...

a) if you're backing up LXD containers, you'll use lxc export (which creates a specialized tarball).

b) For Linux systems, you'll likely generate a tarball using xz or zstd compression.

c) For Windows, you'll create zip files.

d) For WordPress, if you're providing clients access to backup files, you might produce .zip files so they can restore on Linux or Macs or Windows.

So you'll start by answering all these questions to determine best backup software.

2 What  type of hardware and software setup I need  to do in order to utilize the old server and external hard disk to  back up the date from remote location.

a) Hardware - any will do.

b) Software - determined by how you answered #1 questions.

Tip: Best backup approach will be to rsync all file changes (which will be very few) to an external machine, then run your backups on an external machine against a complete clone of all the files. This ensures compression + file I/O occurs on a backup machine rather than a production machine.
 
3- The software I want  should has the  ability to generate report  about the size of backed up  data and used internet traffic.

Pretty much impossible.

a) You can easily determine the raw file size of any backup archive file, by summing all file sizes. To guess at backup archive file size after compression is near impossible.

b) If you use the approach above of maintaining a complete tree of files on your backup machine, then just doing an rsync to sync new files + minor file changes, bandwidth taken for this process is barely measurable, in most cases.
Tried finding a recent EE thread about this + the EE search function... leaves much to be desired, so I'll outline how I do backups here.

1) Run a separate backup machine at home (if you have a fast connection) or online using a storage servers.

For example, an 48TB RAID storage server runs around $150 USD/month, which provides roughly 36TB space available.

2) The point in running a backup machine is to ensure no performance effects on production sites + to optimize backup speed.

3) On the backup machine build a directory hierarchy. I use this format...

# This is where rsync pulls any new files or file changes
/scratch/$machine/$site

# This is where final tarballs live
/tarballs/$timestamp.$site.tar.zst

Open in new window


4) On backup machine, do a remote login to all other machines to do a database dump into each site's DocumentRoot directory, then rsync over any file changes (which will include site database dump).

Note: This might relate to sites or applications. Your database dump might decode wp-config.php files for WordPress sites or pickup databases through some other method, like a dblist.yaml file located in every directory where a database might be deposited.

5) By using rsync, a massive site, say 10G+ in size might only take a few seconds to sync, because most files will remain the same.

6) At this point, you can spin off tar + zstd in background to create either a full backup or incremental backup.

7) Repeat #4-#6 for every site/machine/application to backup.

Makes no difference how many tar + zstd processes you queue up. They may take hours to finish.

Since all compression + file I/O occurs on the backup machine, there's no effect on any production machines.

Also, since tarball is created on the backup machine, there's no step to move massive, multi-Gig files around a network.

8) If you feel more comfortable having multiple backup machines running (I do), you can setup any number of machines you like. Just have all the other backup machines rsync against the first backup machine. This ensures every single backup on every backup machine includes the exact same files.
1. Check out Resilio for something easy. Or check StorageCraft for a proper backup solution.

2. I would suggest you setup a VPN with each client to perform their backups so that you don't expose the data openly to the Internet.

3. Reporting you will only get from proper backup software but even with that I don't think any have the ability to report on internet traffic since most backup software are designed to work locally and have no need to report on network traffic. StorageCraft might have a solution for you though. Talk to them for specifics.
5) By using rsync, a massive site, say 10G+ in size might only take a few seconds to sync, because most files will remain the same.
Only after the initial sync which may take hours/days
Veeam Reporting tools work well with veeam backups for reporting. You haven't stated what you are backing up.
As a managed service provider/Cloud Service Provider you will need  approx 10x the space of the average clients size.
5 yearly backups/12 monthly backups/4 weekly backus/7 daily backups on 2 separate storage boxes, synced to the cloud.
Yearly/monthly can go to tape. You can rent racks in 2 geographically separate data centers
c) For Windows, you'll create zip files.
You're behind the times.  That's only for people that don't know any better.

Backups are one thing, but how about security?  How are you going to keep your copy of the data safe?  You'll need your own backups.  You'll also need to secure your facility from data exfiltration.  There's a lot more you need to do than to just run a RAID or NAS.  It's going to be more expensive if you do it yourself on such a small scale.
Avatar of SKYENERGY2020

ASKER

Thank you  all for your  replay ,the operating system that I want to backup is windows servers 2012 and 2016  .Another thing why I decided to do offsite back up by myself  instead of taking  reseller program with another companies that provide the same  service  ,because almost  of company  in my province have poor customer services and charge high price for back up comparing  to other international companies  and also I need something fast access to backup  when the disaster happened .In addition  ,the regulation  and law preventing of hosting Medical data overseas


Thank you
Then you're going to have to solve the questions I've asked about how you are going to secure your data.
As serialband suggested, just make sure your final backup files are only accessible via something like HTTPS with required login.

I've seen many backups leaked due to poor security.

Best to consider your security first, the build on top of rock solid security.
HTTPS just secures the connection.  I'm talking about data on the server.  Who has access?  How secure is the building?  How do you control who has physical access?  Do you have generators or UPS to keep the systems running in case of a power failure?  Do you have upgrade plans?  Do you have a maintenance plan?

Is the data encrypted?  If you do use https, are you only using TLS 1.2?  How is that password stored?  Do you have a firewall?  Do you have protection against the user leaking their own password?  Will you have 2FA?  Do you have backups of those backup files?

It's not the earlier days of cloud services, when dropbox was easily hacked and leaked user data.  You need to support much more security than just https.  Even back then dropbox should have been asking those, but they never hired security minded people at the beginning, just devs to rush things out to market.
you could use one of  the cloudberry tools to map a cloud file store to your machine and then use your existing backup tools to backup to that share   .. unfortunately windows backup will always create full backups.
https://www.cloudberrylab.com/drive.aspx $80 USD per server (one time purchase)
You should really only need to back up user data, unless it's a critical server or VM.

You can dedup user file backups with HardLink Backup: https://www.lupinho.net/en/hardlinkbackup/
VMs should be backed up with Veeam or something similar: https://www.veeam.com
Physical systems need other backups, such as shadowcraft: https://www.storagecraft.com
I do what you are thinking, I use Whole Sale Backup.  So it goes like this, you have an off site box, or asure one, runs server, then you load their program on it.  Now you are a better Carbonite.  So on the client stations you load the client side, config it and you are off and going.  If you have  a static IP address and a fast download, your home will work.  

I have like a hundered boxes all being backed up to me.  It's all encrypted, hipa compliant and has an amazing web interface you can check out.

You can put your logo on the side they have, they have billing codes that I don't understand AND a kick ass tech support, an American that works with you, will remote in and is super nice.

I think it's like 150 per month and I get a hundred or so clients, the data size is not limited at all.

Whats great too is if you have to restore a large amout of data, you do it at your house, just connect a usb drive, use their password, decrypt the data and you are off.

wholesalebackup.com
(800) 624-9561
mesa-backup.JPG
Oh and yes, it does great reporting, I just read that.  It also uses VSS so it backs up open and locked files.  Can email your reports.
This question needs an answer!
Become an EE member today
7 DAY FREE TRIAL
Members can start a 7-Day Free trial then enjoy unlimited access to the platform.
View membership options
or
Learn why we charge membership fees
We get it - no one likes a content blocker. Take one extra minute and find out why we block content.