Link to home
Start Free TrialLog in
Avatar of pride-it
pride-it

asked on

Copying files from the Production server to a target database server

As part of the new ECO Migration process, we will migrate the Production database Agile ECOs to a target test database.  Part of this new process will involve copying files from the Production server (mesa) to a target database server - in this case UAT2 (extapuata).  We are hoping you can write a Unix script to take care of the file copying.  This Unix script should exist on the Production Server and will be scheduled to run once a day.  The specifics of the script are below:
The source directory for the files will be on mesa in /n01/applprod/PROD11i/appl/ilink/11.5.0/processed.
The target directory for the files will be on extapuata in /n01/uat2/prod11iappl/ilink/11.5.0/incoming.
The files you would need to copy will all begin with a 'TO' and have a extension of 'AXML'.  For example TO00001.AXML.
The number of files in the source directory will keep growing.  You don't need to copy them all.  You will have to use the following logic to determine if a file needs to be copied or not:
If the specific file in the source directory, say TO00001.AXML, exists on the target server in the  /n01/uat2/prod11iappl/ilink/11.5.0/processed directory, then you don't need to copy that file - it has already been processed.  
If it doesn't exist in that directory, then you need to check if the file exists on the target server in the  /n01/uat2/prod11iappl/ilink/11.5.0/incoming directory.  If it does exist there, then you don't want to copy that file - it is already there waiting to be processed.  
If it doesn't exist  in /n01/uat2/prod11iappl/ilink/11.5.0/incoming , then you need to copy that file to that directory.
You will need to make sure the files which are copied have the correct permissions and ownership/grouping so that Agile can process them.  Currently on mesa the files are put in the /n01/applprod/PROD11i/appl/ilink/11.5.0/incoming directory as agile:oaa.  When Agile processes the files, it moves them to the /n01/applprod/PROD11i/appl/ilink/11.5.0/processed directory as applprod:oaa.  When you are copying the files from the mesa  /n01/applprod/PROD11i/appl/ilink/11.5.0/processed directory and putting them in extapuata /n01/uat2/prod11iappl/ilink/11.5.0/incoming, you will have to change their ownership/grouping from applprod:oaa to agile:oaa.
When you are writing this script, keep in mind that these files may need to be moved to more than one target database server.  Initially we are only migrating the ECO files from Production to UAT2.  In the near future we may be migrating them from Production to SIT2 or TST or some other database also.  So make your script generic enough that it can be used for multiple target databases or easy enough to modify if a different version of the script is needed for each target database.  In your script, the source database will ALWAYS be Production (mesa).  Just the target database may change.
Avatar of arnold
arnold
Flag of United States of America image

Is this a task you were assigned and you need assistance?
Please post what you have come up with so far.
I'm unfamiliar with the database you mentioned could you provide additional detail?
Depending on the capability of the database, do you have an option to replicate the database? Mirror the database, backup the database?
ASKER CERTIFIED SOLUTION
Avatar of sentner
sentner
Flag of United States of America image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of pride-it
pride-it

ASKER

Will this work if this is not a SAMBA share?
Hello pride-it, did you try the suggestions we gave you and did you get any errors?

Rsync in essence is just a smart 'remote copy' utility from 1 system to another (remote) system.
rsync does not require SAMBA or any other remote mounting.  It uses its own protocol.  You do need to install the rsync daemon on the server side though, so this may depend on whether you have root access.
Rsync works great, but I still need to check two locations on the destination server to see if the files exist in these locations and then only run the rsync if the files do not exist is these location on destination server.
The two dirctories I have to check for the files are:
n01/uat2/prod11iappl/ilink/11.5.0/processed, /n01/uat2/prod11iappl/ilink/11.5.0/incoming
So far I have this as my script:

rsync -avvvvz -e "ssh -i /home/rsync/.ssh/rsync-key" /n01/applprod/PROD11i/appl/ilink/11.5.0/processed/TO*.AXML rsync@extapuata:/n01/uat2/prod11iappl/ilink/11.5.0/incoming/
/home/scripts