Solved

Replication of files from Primary and the Secondary Linux servers

Posted on 2016-10-09
14
130 Views
Last Modified: 2016-10-26
Hallo Friends,
I have two Red Hat 7 virtual servers. One is Primary and the other one is a Secondary. Need to have  replication of files from Primary and the Secondary.  Please suggest few options and methodologies for doing the same. Believe one is SRDF but teh support team doesn't support that.

Thanks
0
Comment
Question by:flextron
  • 5
  • 5
  • 2
  • +2
14 Comments
 
LVL 28

Expert Comment

by:Jan Springer
ID: 41837398
I use rsync and it works very well.
0
 

Author Comment

by:flextron
ID: 41837422
Thanks Jan....
Are there any steps available or any resource URL where it can be found.
0
 
LVL 28

Expert Comment

by:Jan Springer
ID: 41837428
1) install rsync
     depending upon your initial installation, it may already be installed
     do a "which rsync" to check
     or install as root "yum install rsync"

2) create a script of what needs to be synced

3) as root, generate public and private key on the originating server

4) put the public key into the .ssh/authorized_keys folder for root on the other server

5) create a cron job to do sync

I can help with any or all of this.
0
Connect further...control easier

With the ATEN CE624, you can now enjoy a high-quality visual experience powered by HDBaseT technology and the convenience of a single Cat6 cable to transmit uncompressed video with zero latency and multi-streaming for dual-view applications where remote access is required.

 

Author Comment

by:flextron
ID: 41837451
Hallo Jan
When I do the command "which rsync" I get : /usr/bin/rsync  
Guess that means it is installed already.

Not sure how to do step 2 and 5.
The Folders which needs to by synced is /data
0
 
LVL 28

Accepted Solution

by:
Jan Springer earned 250 total points
ID: 41837481
update iptables or firewalld (whichever you're using) to restrict ssh by only authorized IPs/subnets.  this is extremely critical as I am going to walk you through using keys as root to login in to the other server.

install fping
       
      yum install fping

verify that its installation location (needed for the script below)

      which fping

create your ssh keys and enter through everything, you'll need it passwordless

        cd /root
        ssh-keygen

copy the public key information

        cat .ssh/id_rsa.pub

ssh to the other server, insert the public key and save

        ssh root@IP_of_secondary_server
        ssh-keygen
        cd .ssh
        vi authorized_keys
 
now you should be able to ssh as root to the other server without requiring a password.  the first time that you do so, you'll be prompted to accept the fingerprint.  Do so and know that, under normal circumstances, you should never see that message again.  if you do, beware of a man in the middle attack.

     vi /usr/local/bin/syncfiles

a typical script would look like this

     #!/bin/bash

     RESULT=`/sbin/fping 192.168.0.34`

     if [ "$RESULT" == "192.168.0.34 is alive" ]; then

             /usr/bin/rsync -av --delete-after -e ssh /data/ root@192.168.0.34/data
     fi

     exit 0

And a cron job

    vi /etc/cron.d/syncfiles

Insert this line and save.  change this to whatever frequency  you need

    30 0 * * * root /usr/local/bin/syncfiles > /dev/null 2>&1

systemctl restart crond
0
 

Author Comment

by:flextron
ID: 41837560
Hallo Jan,
The server is not connected to internet. Only intranet,
Hence yum install won't work . !
Also, can you please explain a bit more about teh cron part
0
 
LVL 28

Expert Comment

by:Jan Springer
ID: 41837569
cron is a service that runs jobs at defined times.

do you have the original DVD?  

you can install from that.
0
 
LVL 37

Assisted Solution

by:ArneLovius
ArneLovius earned 250 total points
ID: 41837575
@flextron, do you mean this  for SRDF ?

As Jan has said, rsync is _the_ application for unidirectional replication between different systems across virtually any type of link, depending on the size of the filesystem (number of files and directories) the time it takes rsync to "crawl" the filesystem may be an issue.

Just as an alternative idea, zfs has "zfs send" and "zfs receive" which like rsync can also be used to unidirectionally replicate a filesytem, and as these can also be used for snapshots, for very large filesystems, this can be considerably faster than rsync for keeping a replica up to date. The issue being that this only works on a zfs filesystem...
0
 
LVL 27

Expert Comment

by:skullnobrains
ID: 41838143
you can also use gluster which will be a little slower than a regular fs but will replicate in near real-time and replicates in both directions

you can use this doc
https://gluster.readthedocs.io/en/latest/Quick-Start-Guide/Quickstart/
- their first 2 steps make you create a dedicated partition for gluster but you can use an existing directory
- note that you should not use the directory behind gluster directly but rather mount your glusterfs on both hosts on separate directories. if that's unclear and you're interested, i'll elaborate.
- the key to the setup is in step 5 : "replia 2" instructs gluster to keep 2 copies of each file and since you only have 2 hosts, you end up with mirroring

--

there are many other options such as replicating fusefs or stuff based on inotify

i've used unison with good results as well. it's only better compared to rsync if you need bidirectional sync

zfs will provide block-level replication ( and a crazy number of other features you probably won't use if your server is an existing virtual linux server ). note that it is much more efficient with small files than traditional block-based replications.

rsync is mainly interesting for it's versioning capabilities. but note that rsync is not suitable for frequently syncing a huge number of files.
0
 
LVL 40

Expert Comment

by:noci
ID: 41838172
drbd might be an option on the block level if needed.
http://www.drbd.org

If you need synchronous replication from primary to secondary then drbd is the only option.
0
 
LVL 27

Expert Comment

by:skullnobrains
ID: 41838193
gluster and a wealth of other options allow synchronous replications between 2 hosts
obviously with synchronous replication comes latency
0
 

Author Comment

by:flextron
ID: 41838371
@Jan Springer: IS fping absolutely necessary. Will tnsping won't work ?
0
 
LVL 28

Expert Comment

by:Jan Springer
ID: 41838374
Any ping that you can specify a count of one, returns a one-line answer and the answer is consistent will work.
0
 

Author Closing Comment

by:flextron
ID: 41861554
Sorry for the delay ...but yes.. this is a way of keeping things in sync. Other way I believe would be having an enterprise scheduler like COntrol M or Tidal to have a job which copies the  files from Prod to DR on a nightly basis.
0

Featured Post

Easy, flexible multimedia distribution & control

Coming soon!  Ideal for large-scale A/V applications, ATEN's VM3200 Modular Matrix Switch is an all-in-one solution that simplifies video wall integration. Easily customize display layouts to see what you want, how you want it in 4k.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Weird Samba Connectivity Issue... 7 52
Error Message during CentOS 7 Minimal Install 3 58
windows 7 starter missing password 21 89
Upgrade BIOS / EUFI at Scale 4 55
Linux users are sometimes dumbfounded by the severe lack of documentation on a topic. Sometimes, the documentation is copious, but other times, you end up with some obscure "it varies depending on your distribution" over and over when searching for …
I. Introduction There's an interesting discussion going on now in an Experts Exchange Group — Attachments with no extension (http://www.experts-exchange.com/discussions/210281/Attachments-with-no-extension.html). This reminded me of questions tha…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…

840 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question