Link to home
Create AccountLog in
Avatar of Rabbit80
Rabbit80Flag for United Kingdom of Great Britain and Northern Ireland

asked on

Cloning a remote server (Ubuntu) into a local VM

We have a server held in a datacenter a few hundred miles away.

We need to clone the hard disks machine to a local server (We have a matching machine locally) in order to run some testing on it without affecting the live server. The question is, how can we do this?

The server is running Ubuntu 10.04 and has 3 hard disks (sda, sdb, sdc) - it boots from sdc.

I have no issue with sda and sdb as I can use dd to clone them, but I'm not sure how to go about cloning sdc.

My questions are..

1) Best way to clone over ssh using dd - preferably compressing the stream to make the transfer quicker (About 120GB to transfer)?
2) Best way to clone the main hard disk (sdc) over ssh (About 80GB used). I cannot shut the server down and boot from a live CD or usb stick?

Thanks
SOLUTION
Avatar of Pepe2323
Pepe2323

Link to home
membership
Create an account to see this answer
Signing up is free. No credit card required.
Create Account
Avatar of Rabbit80

ASKER

Thanks for that!

Will this work for the OS drive?
Avatar of Pepe2323
Pepe2323

Yes but you will need to boot the target server with a live CD linux version

For example if your plan is to have like an image of the disk on a third server you can do this:

This will copy the entire drive into a file on the remote machine:

$ dd if=/dev/sda | ssh user@server 'dd of=sda.img'

To restore a local drive from the image on the server, reverse the command:

$ ssh user@server 'dd if=sda.img' | dd of=/dev/sda

As you can see is very similar as i described previously
That's the point I was trying to make in my original post.. I cannot boot the server from a live CD as a) we cannot take it offline for that long and b) we don't have easy or convenient physical access to it.
I thought you ware talking about the source server and not target server.

So in that case how you plan to put the information on the target server if there is not a Os runnig ??
Ah - I see.. Reading comprehension failure on my part!

I thought you were talking about booting the source server from the Live CD!

Will dd create a suitable copy from the source if it is live?
correct  dd is raw copy of the entry disk so if you are doing a clone copy the target disk also need to be identical  as source disk
SOLUTION
Link to home
membership
Create an account to see this answer
Signing up is free. No credit card required.
Create Account
As an additional note, if this is a production web server odds are you aren't updating much on it except logs and possibly the DB (though that should really be housed in another machine, or at least be snapshot capable).

Rsync will only pull the changed files on subsequent runs. You could probably just use rsync's --exclude option to skip /var/log (and your DB storage directory if necessary), as the existing logs will have no practical impact on the test server's performance. If a subsequent run returns no changes then no need to go any further.

If your database is on the same machine and is not running on a snapshot-capable partition (1) change that when you can and (2) you can always perform a binary export, import it, and recreate the database locally.
Thanks Ghost - you confirmed what I was thinking which is why I asked the question in the first place. Any guidance on using rsync? Its not a tool I have used before.
ASKER CERTIFIED SOLUTION
Avatar of serialband
serialband
Flag of Ukraine image

Link to home
membership
Create an account to see this answer
Signing up is free. No credit card required.
Create Account
Have split the points as there were 2 parts to the question. Thanks guys - your help is appreciated!
I suppose I don't really care much about the points, but considering I answered the question I would have appreciated at least a bit of credit as an assist...

I guess that's what happens when I take the wife out for date night, heh.

So long as your problem's solved that's what's really important ;)
Sorry Ghost - I just assumed it was you that posted the final solution.. didn't look at the name! (confession - its nearly 4am and i'm a little blurry eyed!)

Thanks again for your help.
Ouch, late night. Get some sleep. But in the morning, perhaps you'll remember that RA button up top can summon a mod to change your selections... if you care about karma and the like ;)

The experts love to help, but the "street cred" is a nice reward. Be kind, rewind =)
Points reassigned and Ghost given credit!

Thanks to everyone!
Thank you, and I mean that with all honesty.

I've become a little too accustomed to askers bailing out on questions (in other forums) as soon as a workable solution is posted. Being able to scan through the solutions and quickly pick out the useful answers is an invaluable time saver for most users.

The attention to completion from both sides is what I appreciate about EE =)
@Ghost

It wasn't my intention to come in to take the points.  I'm not even really here for points except to keep a minimum to be able to read the solutions to the other questions.  I sometimes just respond after a solution is accepted just to provide information.

You'll also find that up to half the questions become neglected.  They've only recently deleted some very old neglected questions (over 10 years) when they updated the interface a short while ago to allow the mods to more easily get to them.  I got a few emails about them a while back.
@serialband

I know that wasn't your intent. I catch up on people's histories pretty quickly when I see their names pop up a few times in the same questions I'm interested in =)

I don't have any interest in the points or even attaining/maintaining Expert status for this account. My interests are solely placed in helping people get their problems solved, getting to know the community, and maintaining incentive for others to do the same.

I've been around... I know a good number of the newer Experts really do care about the points, and more importantly, those searching for answers want to find them quickly and get on with their day.

Ensuring the right solutions are flagged as answers / assists facilitates both, and I think it's beneficial to occasionally remind the community about keeping a tidy house so we can all enjoy it -- and make the the CVs' jobs a little easier at the same time ;)
I just simply appreciate that all you guys took the time to help. . It would have taken me a lot longer otherwise. It was simply a mistake on my part not realising we had a third expert in the conversation else I would have credited each of you - I had some other unrelated issues which required urgent attention hence the late night. . And its a pretty easy mistake to make.

The copy job is going to happen over the weekend although I have tested the methods laid out here and now have the confidence to perform the task.

Thanks again to all of you.
Well if you do run into any problems with the data migration, I'm positive there will be someone here that can help.

Best of luck to you =)