Solved

Using 'tar' for High Availability/Cloning

Posted on 2010-11-10
26
548 Views
Last Modified: 2013-11-15
I have a ubuntu server running. I'm using 'tar' to clone or to have high availability of this server. I followed the tutorial from the link click here. I tried this installing a new server of same version. When I extracted the tar(tar -xvpzf ~/clone.tgz -C /) on the destination(new), at the end it's ending with the following output similar to the below(don't know if it's error).

 
tar: var/run: time stamp 2010-11-09 17:09:11 is 7335.159880406 s in the future
tar: var/spool/postfix/usr/lib/zoneinfo: time stamp 2010-11-09 17:08:26 is 7290.159730037 s in the future
tar: var/lib: time stamp 2010-11-09 17:27:51 is 8455.159349527 s in the future
tar: usr/bin: time stamp 2010-11-09 17:28:02 is 8466.159254097 s in the future
tar: usr/share/sgml: time stamp 2010-11-09 17:27:47 is 8451.158909506 s in the future
tar: usr/share/man/man7: time stamp 2010-11-09 17:27:50 is 8454.158393583 s in the future
tar: usr/share/man/man1: time stamp 2010-11-09 17:28:02 is 8466.158166556 s in the future
tar: usr/share/man/man8: time stamp 2010-11-09 17:27:51 is 8455.158057701 s in the future
tar: usr/share/omf/time-admin: time stamp 2010-11-09 17:27:52 is 8456.157830449 s in the future
tar: usr/share/omf/gnome-system-tools: time stamp 2010-11-09 17:27:52 is 8456.157751897 s in the future
tar: usr/share/omf/shares-admin: time stamp 2010-11-09 17:27:52 is 8456.157671741 s in the future
tar: usr/share/omf/users-admin: time stamp 2010-11-09 17:27:52 is 8456.15759237 s in the future
tar: usr/share/omf: time stamp 2010-11-09 17:27:52 is 8456.157516024 s in the future
tar: usr/share/doc/libpolkit-gtk-1-0: time stamp 2010-11-09 17:27:51 is 8455.156837457 s in the future
tar: usr/share/doc/libgtk2.0-bin: time stamp 2010-11-09 17:27:50 is 8454.156531212 s in the future
tar: usr/share/doc/libpolkit-agent-1-0: time stamp 2010-11-09 17:27:51 is 8455.156432743 s in the future
tar: usr/share/doc/policykit-1: time stamp 2010-11-09 17:27:51 is 8455.156218227 s in the future
tar: usr/share/doc/libgconf2-4: time stamp 2010-11-09 17:27:48 is 8452.156101033 s in the future
tar: usr/share/doc/libgtk2.0-0: time stamp 2010-11-09 17:27:49 is 8453.155923584 s in the future
tar: usr/share/doc/libpolkit-backend-1-0: time stamp 2010-11-09 17:27:51 is 8455.155741929 s in the future
tar: usr/share/doc/gconf2: time stamp 2010-11-09 17:27:48 is 8452.155530924 s in the future
tar: usr/share/doc: time stamp 2010-11-09 17:27:52 is 8456.155372777 s in the future
tar: usr/share/gnome/help/time-admin/ca/figures: time stamp 2010-11-09 17:27:52 is 8456.155290694 s in the future
tar: usr/share/gnome/help/time-admin/ca: time stamp 2010-11-09 17:27:52 is 8456.155210771 s in the future
tar: usr/share/gnome/help/time-admin/es/figures: time stamp 2010-11-09 17:27:52 is 8456.155130511 s in the future
tar: usr/share/gnome/help/time-admin/es: time stamp 2010-11-09 17:27:52 is 8456.155049168 s in the future
tar: usr/share/gnome/help/time-admin/cs/figures: time stamp 2010-11-09 17:27:52 is 8456.154967741 s in the future
tar: usr/share/gnome/help/time-admin/cs: time stamp 2010-11-09 17:27:52 is 8456.154888469 s in the future
tar: usr/share/gnome/help/time-admin/it/figures: time stamp 2010-11-09 17:27:52 is 8456.154808997 s in the future
tar: usr/share/gnome/help/time-admin/it: time stamp 2010-11-09 17:27:52 is 8456.154676381 s in the future
tar: usr/share/gnome/help/time-admin/ru/figures: time stamp 2010-11-09 17:27:52 is 8456.154592231 s in the future
tar: usr/share/gnome/help/time-admin/ru: time stamp 2010-11-09 17:27:52 is 8456.154512497 s in the future
tar: usr/share/gnome/help/time-admin/C: time stamp 2010-11-09 17:27:52 is 8456.154433419 s in the future
tar: usr/share/gnome/help/time-admin/sv/figures: time stamp 2010-11-09 17:27:52 is 8456.154351042 s in the future
tar: usr/share/gnome/help/time-admin/sv: time stamp 2010-11-09 17:27:52 is 8456.153015355 s in the future
tar: usr/share/gnome/help/time-admin/fr/figures: time stamp 2010-11-09 17:27:52 is 8456.152923013 s in the future
tar: usr/share/gnome/help/time-admin/fr: time stamp 2010-11-09 17:27:52 is 8456.152842126 s in the future
tar: usr/share/gnome/help/time-admin/el/figures: time stamp 2010-11-09 17:27:52 is 8456.152735113 s in the future
tar: usr/share/gnome/help/time-admin/el: time stamp 2010-11-09 17:27:52 is 8456.152653645 s in the future
tar: usr/share/gnome/help/time-admin/nl/figures: time stamp 2010-11-09 17:27:52 is 8456.152572431 s in the future
tar: usr/share/gnome/help/time-admin/nl: time stamp 2010-11-09 17:27:52 is 8456.152493321 s in the future
tar: usr/share/gnome/help/time-admin/zh_CN/figures: time stamp 2010-11-09 17:27:52 is 8456.152412151 s in the future
tar: usr/share/gnome/help/time-admin/zh_CN: time stamp 2010-11-09 17:27:52 is 8456.152330911 s in the future
tar: usr/share/gnome/help/time-admin/en_GB/figures: time stamp 2010-11-09
---------------------------------------------
---------------------------------------------
---------------------------------------------

Open in new window


I'm using following command to create a tar file of the specified directories on the source system.

tar -cvzf ~/clone.tgz --exclude ~/clone.tgz --exclude /etc/hosts --exclude /etc/hostname --exclude /etc/udev/ --exclude /etc/network/interfaces --exclude /etc/resolv.conf  /etc /home /opt /tmp /usr /var /mnt

- Should I've to include any more directory like bin or lib? - Please suggest me
- Should I've to exclude any directory?, like I had a network device(eth0) problem(failed to start up eth0). So in the above command I've excluded "/etc/udev/" and after this I felt this was fine. Like this, is there any thing I've to exclude from /etc/ or from any directory that I've included? - Please suggest me.
0
Comment
Question by:Thyagaraj03
  • 13
  • 10
  • +2
26 Comments
 
LVL 1

Expert Comment

by:FirstMM
ID: 34101005
1. tar: var/run: time stamp 2010-11-09 17:09:11 is 7335.159880406 s in the future

This error basically means that you are attempting to restore the files onto a system where the time is in front of the archive.  Tar is attempting to write the time stamps as it extracts files but cannot because the system time is behind the archive.
7335 seconds is only about 2 hours 25 minutes so you should find it works properly soon.

You might need to check the computer time on both systems if they are at the same location.

Mikey
0
 

Author Comment

by:Thyagaraj03
ID: 34101031
@FirtsMM:

I checked now and both the systems have same time set.

Any more help...
0
 
LVL 14

Expert Comment

by:small_student
ID: 34101065
Seriously if you want to clone your system there are many other simpler ways

1- CloneZilla

http://clonezilla.org/

2- rsync your system to the new system


I also noticed that you have installed a second system and then copied the tgz file to it and extracted it. This means you are copying data
If you really want to do it the TAR way then I suggest excluding

/tmp
/var/lock

Another thing you can do is test with each dir first to find which one actually is breaking
0
 
LVL 14

Expert Comment

by:fmarzocca
ID: 34101094
The best way to have an HA server is yo use rsync:
http://everythinglinux.org/rsync/
0
 

Author Comment

by:Thyagaraj03
ID: 34101208
@small_student:
If it's a cloud server I could not use Clonezilla as I don't have direct access
Thanks for tar suggestions. Won't that error be any problem?


@fmarzocca:
Yes, after the tar is successful I'll be using 'rsync' with ssh combination. You could say that I can directly use the rsync instead tar but as I make lot of mistakes(for rsync, if I wrongly specify source and dest. exchanged) I thought of using tar and then rsync.
Please give my rsync with ssh for synchronization(incremental backup) between old and new servers of the specified directories.
0
 

Author Comment

by:Thyagaraj03
ID: 34109834
Please help me!
0
 
LVL 14

Expert Comment

by:small_student
ID: 34109890
rsync -avP /home 192.168.x.y:/home

I am assuming that the other server IP is 192.168.x.y

In case you are afraid to overwrite something at the destination BACKUP the /home at destination.

If all went well then do the same for the other dirs /usr /var ...etc

You should not sync /tmp /sys /proc

Also dont sync /var/run or /var/log becasue they are specific for your system
0
 
LVL 3

Expert Comment

by:berniep
ID: 34110489
I see from your tar that you're including the "-z" compression option;
you can do this also with rsync, eg: rsync -avzP -e ssh /home ...
0
 

Author Comment

by:Thyagaraj03
ID: 34110719
@small_student:
I created a directory under user1 on the original system and used "rsync -avP /home/user1 192.168.2.3:/hom/user1". But the directory is not created under user1 on 192.168.2.3. What I'm doing wrong. I think it's a incremental backup(what I wanted) may be I've to use full bkp with rsync, like so?. I don' t know.
0
 
LVL 14

Expert Comment

by:small_student
ID: 34110914
If this is your exact syntax then you have misspelled the home in your destination.

rsync -avP /home/user1 192.168.2.3:/hom/user1

If not, what error do your get? where was the directory created??

0
 

Author Comment

by:Thyagaraj03
ID: 34111077
No, that was a typo in my post. Now its working. But I think it's not a incremental one. I created a directory under /usr on my orginal machine and I used your rsync command, it started syncing all the files and directories under /usr. It's a time consuming.
0
 
LVL 14

Expert Comment

by:small_student
ID: 34111190
No it is Incremental , please post the command you typed, it has to do with / put at source and destination.

0
 

Author Comment

by:Thyagaraj03
ID: 34111288
This is the command I used on my orginal computer:
rsync -avP /usr 192.168.2.100:/usr
The output was like the below:

usr/share/webmin/acl/edit_acl.cgi
        2358 100%    2.71kB/s    0:00:00 (xfer#51755, to-check=1103/61974)
usr/share/webmin/acl/edit_group.cgi
        4018 100%    4.56kB/s    0:00:00 (xfer#51756, to-check=1102/61974)
usr/share/webmin/acl/edit_pass.cgi
        1642 100%    1.86kB/s    0:00:00 (xfer#51757, to-check=1101/61974)
usr/share/webmin/acl/edit_rbac.cgi
         797 100%    0.90kB/s    0:00:00 (xfer#51758, to-check=1100/61974)
usr/share/webmin/acl/edit_sync.cgi
        1147 100%    1.30kB/s    0:00:00 (xfer#51759, to-check=1099/61974)
usr/share/webmin/acl/edit_unix.cgi
0
What Is Threat Intelligence?

Threat intelligence is often discussed, but rarely understood. Starting with a precise definition, along with clear business goals, is essential.

 
LVL 14

Accepted Solution

by:
small_student earned 500 total points
ID: 34111509
Ok now if you run this

rsync -avP /usr 192.168.3.3:/usr

A directory named usr will be placed under /usr do at 192.168.3.3 you will have

/usr/usr

Therfore you will see that it is doing it all over again.

But if you run

rsync -avP /usr/ 192.168.3.3:/usr

This will increment the destination /usr

0
 

Author Comment

by:Thyagaraj03
ID: 34118291
When I used "rsync -avP /usr/ 192.168.2.100:/usr" for the first time I think it was doing the full sync and the following is the output(o/p is not from starting)

share/doc/testdisk/html/dyn.css
          75 100%    0.09kB/s    0:00:00 (xfer#94, to-check=1239/45191)
share/doc/testdisk/html/ejecuci_n_con_scripts.html
       13197 100%   15.51kB/s    0:00:00 (xfer#95, to-check=1238/45191)
share/doc/testdisk/html/ejecucion_con_scripts.html
       13306 100%   15.56kB/s    0:00:00 (xfer#96, to-check=1237/45191)
share/doc/testdisk/html/ejecutando_testdisk.html
       10202 100%   11.89kB/s    0:00:00 (xfer#97, to-check=1236/45191)
share/doc/testdisk/html/ejemplos_de_recuperaci_n_de_datos.html
       17517 100%   19.78kB/s    0:00:00 (xfer#98, to-check=1235/45191)
share/doc/testdisk/html/en.png



I pressed ctrl+c and again I used the ur rsync command now I think It's doing incremental and below is the o/p of this:

root@192.168.2.100's password:
sending incremental file list
share/python-support/
share/tomcat6/
share/tomcat6/webapps/
triple/
triple/test
           0 100%    0.00kB/s    0:00:00 (xfer#1, to-check=0/96600)

sent 2153942 bytes  received 13331 bytes  84991.10 bytes/sec
total size is 1469981901  speedup is 678.26


The directory 'triple' and the file 'test', I created them on original system and is present on the destination.

What you say?. what I've to do?.


0
 
LVL 14

Expert Comment

by:small_student
ID: 34118564
That's it you have it working correctly now

By the way you can add the --stats option to give you full statistics

rsync -avP --stats ....etc

Now I guess you can clone the partitions you wanted to clone in the first place
0
 

Author Comment

by:Thyagaraj03
ID: 34119232

- Could you change and  give me above rsync command so that it syncs only specified directories and even for excluding some files/directories from them.
spcified dir's:/usr /lib /etc ...
excludes: /etc/fstab /etc/udev/
- Finally I want to schedule rsync so that it won't prompt for the password(I think it will prompt as ssh is used)
0
 

Author Comment

by:Thyagaraj03
ID: 34135249
While using rsync -avP /etc/ 192.168.3.3:/etc I would like to exclude some files like /etc/udev/*, /etc/network/interfaces ...
0
 
LVL 14

Expert Comment

by:small_student
ID: 34135266
Sorry I have been busy, I will look through this tonight as I am a little bit busy at the moment, sorry for the delay, its actually a matter of syntax but I dont remember it exactly at the moment, will post tonight hopefully.
0
 

Author Comment

by:Thyagaraj03
ID: 34135303
it's ok. I know masters will be busy always. Take your time
0
 
LVL 14

Expert Comment

by:small_student
ID: 34135471
Here it is

 rsync -avP --exclude 'udev' --exclude 'network/interfaces' /etc/ 192.168.3.3:/etc

The files/folders you want to exclude are relevant to the folder you are syncing so you dont put the /etc in our example.

Here is a detailed link which also exaplains the --exclude-from options in case you have many files/dir to exclude

http://articles.slicehost.com/2007/10/10/rsync-exclude-files-and-folders
0
 

Author Comment

by:Thyagaraj03
ID: 34135696
Yeah, almost everything is done but one. Before posting, I have already followed the link u suggested. But wrongly used like --exclude '/etc/udev/' which misleaded me.
Is it possible to synchronize multiple directories in one single rsync command with excludes?. If not possible let's close this here itself.
0
 
LVL 14

Expert Comment

by:small_student
ID: 34135878
You can specify multiple sources but one destinatoin, you can try the following

rsync -avP --exclude 'whateveryouwant' /etc /home 192.168.3.3:/

Note that I removed the / from the sources because they are syncing to the / on the destination.

Try this and see how it goes, it might seem like a full at the beggining then the second time it will be incremental. test with small dir like /etc and /home, if all was fine then add the others.

If you are not comfortable with this you can make them in separate statements.
0
 

Author Comment

by:Thyagaraj03
ID: 34141842
You mean to say like this 'rsync -avP --exclude 'hosts' --exclude 'user1' /etc/ /home 192.168.3.3:/'
where 'user1' is a user account under /home.
0
 

Author Comment

by:Thyagaraj03
ID: 34142724
If I have two files with the same name let 'test' under /etc/ and /home, then how do I exclude only /etc/test file if I use the following:
'rsync -avP --exclude 'hosts' --exclude 'user1' /etc/ /home 192.168.3.3:/'.
0
 
LVL 14

Expert Comment

by:small_student
ID: 34143720
Actually I have not tried it, give it a try and let me know
0

Featured Post

Why You Should Analyze Threat Actor TTPs

After years of analyzing threat actor behavior, it’s become clear that at any given time there are specific tactics, techniques, and procedures (TTPs) that are particularly prevalent. By analyzing and understanding these TTPs, you can dramatically enhance your security program.

Join & Write a Comment

In order for businesses to be compliant with certain information security laws in some countries, you need to be able to prove that a user (which user it was becomes important to the business to take action against the user after an event has occurr…
Are you sitting there reading this and wondering how to get started with Linux? It almost seems like picking the right Linux distribution is about like picking the right college or buying a new car if you read some of the article out there. Relax… l…
Here's a very brief overview of the methods PRTG Network Monitor (https://www.paessler.com/prtg) offers for monitoring bandwidth, to help you decide which methods you´d like to investigate in more detail.  The methods are covered in more detail in o…
This video explains how to create simple products associated to Magento configurable product and offers fast way of their generation with Store Manager for Magento tool.

747 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

13 Experts available now in Live!

Get 1:1 Help Now