Solved

UNIX script question- Plse help guru's

Posted on 2006-06-27
9
646 Views
Last Modified: 2013-12-06
Hi,
 
I'm in the midst of writing a UNIX script that sftp's files to an external host and am stuck with a problem. The problem is that the files created on my server as a order number that correlates to a sequence of directories on the remote host which is where the file should be ftp'ed.

Example:
My server filename:
myserver> /home/ 572011 O4MP ORDS 188406 V01 DT 20060608154548.txt

The 4th field (188406 ) in my filename correlates to a directory on the remote server where the file should be store. So on the remote server we should be sftp'ing this file to path

Remote Server:
remoteserver> /Test Documents/TestOrders/180001-190000/188001-189000/188401-188500/188406

The remote server has about 30+ directories and sub-directory ranges created to facilitate this file transfer. Remote server directory listings as range level and sub directories follow:
 100001-110000                  110001-120000
180001-190000                  270001-280000                  300001-310000                  320001-330000
330001-340000                  350001-360000                  370001-380000                  380001-390000
390001-400000                  400001-410000                  410001-420000                  420001-430000
430001-440000                  440001-450000                  450001-460000                  460001-470000
470001-480000                  480001-490000                  490001-500000                  500001-510000
510001-520000                  520001-530000                  530001-540000                  540001-550000
550001-560000                  560001-570000                  570001-580000                  580001-590000
590001-600000                  600001-610000                  

So another example would be that if we received a filename:
 /home/ 572011 O4MP ORDS 588406 V01 DT 20060608154548.txt (Note: 588406 is the directory to be stored on remote site as explained above)

How can I make the script smart enough for it to look at the forth fiels in my file name sftp it on the remote server path:
remoteserver> /Test Documents/TestOrders/580001-590000/588001-589000/588401-588500/588406

Your assistance would be greatly appreciated.

Thanks.
0
Comment
Question by:budrito12
  • 6
  • 3
9 Comments
 

Author Comment

by:budrito12
ID: 16998080
Please note this is for #!/bin/sh, #!/bin/csh or #!/bin/ksh and based on HP-UX 11i
0
 
LVL 4

Expert Comment

by:JulieBouillon
ID: 16999641
This should you permit to do the trick:

---8<---
#!/bin/ksh

FILE="/home/ 572011 O4MP ORDS 588406 V01 DT 20060608154548.txt"

typeset -i DIR3=$(echo $FILE | cut -f5 -d' ')
typeset -i DIR2=$(echo $DIR3 | cut -c1-3)
typeset -i DIR1=$(echo $DIR3 | cut -c1-2)

echo "/Test Documents/TestOrders/${DIR1}0001-$((DIR1+1))0000/${DIR2}001-$((DIR2+1))000/${DIR3}"
--->8---

Hope it helps.
0
 

Author Comment

by:budrito12
ID: 17006027
O.K very clever indeed and am very impressed...it works. Thanks heaps I'm also not just after solutions so can you please be kind enough to explain how and what your code does.

Once again thanks for helping out.
0
 
LVL 4

Expert Comment

by:JulieBouillon
ID: 17007123

>FILE="/home/ 572011 O4MP ORDS 588406 V01 DT 20060608154548.txt"
Here you'll loop through your dir for each files. Since I just post a proof of concept a assigned a fixed values.

>typeset -i DIR3=$(echo $FILE | cut -f5 -d' ')
We create an "integer" which contains the fifth fields of $FILE. Fields are separated by blanks.
So now $DIR3 contains 572011.

>typeset -i DIR2=$(echo $DIR3 | cut -c1-3)
Another "integer" which contains the first 3 characters of $DIR3.
So now $DIR2 contains 572.

>typeset -i DIR1=$(echo $DIR3 | cut -c1-2)
The last "integer" we will used and which contains the first two characters of $DIR3.
So now $DIR1 contains 57.

>echo "/Test Documents/TestOrders/${DIR1}0001-$((DIR1+1))0000/${DIR2}001-$((DIR2+1))000/${DIR3}"
Here we construct the string.
The form ${DIR1} is the same as $DIR1 but the name of the variable must be enclosed by curly brackets. In fact if we wrote $DIR10001 instead of ${DIR1}0001, ksh would have tried to print out the content of a variable named DIR10001 and the results would have been an empty string.
The form $((DIR+1)) is the way we can do some basic arithmetics in ksh. In our cas, ksh is printing the result of 57+1 -> 58.


ksh is an amazing shell which allows you to make some elaborated stuff. I encourage you to familiarize yourself with ksh scripting:
 - http://quong.best.vwh.net/shellin20/
 - http://www.bolthole.com/solaris/ksh.html
0
6 Surprising Benefits of Threat Intelligence

All sorts of threat intelligence is available on the web. Intelligence you can learn from, and use to anticipate and prepare for future attacks.

 

Author Comment

by:budrito12
ID: 17014960
Many thanks for the above explanation. I have wriiten the below script incorportaing your directory calculation but seem to have an issue with the output. Basically for every file in directory "/home/filedir" find the remote directory name (as per your given algorithm) and sftp that file. Note that filename as given above has spaces and the output concatenates all the files into one commandline

SCRIPT:
#!/bin/ksh
tmp0=/tmp/a
INPUTFILE=/tmp/inputfile
cd /home/filedir
ls > $tmp0 2>/dev/null
FILE="/home/filedir/`cat $tmp0`"

#Number range algorithm
for filename in `cat $tmp0`
do
typeset -i DIR3=$(echo $FILE | cut -f4 -d' ')
typeset -i DIR2=$(echo $DIR3 | cut -c1-3)
typeset -i DIR1=$(echo $DIR3 | cut -c1-2)
done

TARGETOUT="/Test Documents/TestOrders/${DIR1}0001-$((DIR1+1))0000/${DIR2}001-$((DIR2+1))000/${DIR3}"

for filename in `cat $tmp0`
do
echo "cd $TARGETOUT" >> $INPUTFILE
echo "put "$FILE"" >> $INPUTFILE
echo "quit" >> $INPUTFILE
done

sftp -b $INPUTFILE testuser@10.242.XX.XX          > $tmp1 2>&1

Output of the above program is:
put /home/filedir/147258 12BL NOTE 259783 V01 DT 20060605161329.txt 457898 MISC NOTE 259793 V01 DT 20060609153346.txt 5
72011 O4MP ORDS 259789 V01 DT 20060608153450.txt
quit
cd /Test Documents/TestOrders/250001-260000/259001-260000/259783
put /home/filedir/147258 12BL NOTE 259783 V01 DT 20060605161329.txt 457898 MISC NOTE 259793 V01 DT 20060609153346.txt 5
72011 O4MP ORDS 259789 V01 DT 20060608153450.txt
quit
cd /Test Documents/TestOrders/250001-260000/259001-260000/259783
put /home/filedir/147258 12BL NOTE 259783 V01 DT 20060605161329.txt 457898 MISC NOTE 259793 V01 DT 20060609153346.txt 5
72011 O4MP ORDS 259789 V01 DT 20060608153450.txt
quit
..........................and repetitive....

Also note that the PUT wont work as the filename has space so treats each space as a filename. I've tried putting "" around the filename but can't seem to get it working.

Can you please advise where I'm going wrong.

Thanks
0
 
LVL 4

Accepted Solution

by:
JulieBouillon earned 125 total points
ID: 17015229
Try this one:
#!/bin/ksh

SOURCEDIR=/home/filedir

# let's go where our file resides
cd $SOURCEDIR

# we send each file found in our dir
for FILE in *
do
   # compute the destination directory
   typeset -i DIR3=$(echo "$FILE" | cut -f4 -d' ')
   typeset -i DIR2=$(echo $DIR3 | cut -c1-3)
   typeset -i DIR1=$(echo $DIR3 | cut -c1-2)
   TARGETDIR="/Test Documents/TestOrders/${DIR1}0001-$((DIR1+1))0000/${DIR2}001-$((DIR2+1))000/${DIR3}"

   # file transfer (I use scp instead of sftp, scp is as secure as is sftp, look at the man page for more details)
   scp "$SOURCEDIR/$FILE"  "testuser@10.242.XX.XX:$TARGETDIR"
   if [[ $? -ne 0 ]] ; then
      print "An error occured while transfering file $SOURCEDIR/$FILE"
   fi
done
0
 

Author Comment

by:budrito12
ID: 17028992
Thanks for the above, I forgot to mention that in actual fact there was one other directory that I had to ftp to which I missed out but based on your explanation provided I managed to fix the issue. The script now looks like below but still has one outstanding issue. The TARGETOUT dir has a space (i.e Test Documents), I've tried putting in single & double quotes but it still does'nt cd into correct directory. I get the error message "scp: ambiguous target", any ide3s? Thanks for all your help to date.

#!/bin/ksh

SOURCEDIR=/home/filedir

# let's go where our file resides
cd $SOURCEDIR

# we send each file found in our dir
for FILE in *
do

# compute the destination directory
typeset -i DIR4=$(echo $FILE | cut -f4 -d' ')
typeset -i DIR3=$(echo $DIR4 | cut -c1-4)
typeset -i DIR2=$(echo $DIR4 | cut -c1-3)
typeset -i DIR1=$(echo $DIR4 | cut -c1-2)

TARGETDIR="/Test Documents/TestOrders/${DIR1}0001-$((DIR1+1))0000/${DIR2}001-$((DIR2+1))000/${DIR3}01-$
((DIR3+1))00/${DIR4}"


# file transfer
scp "$SOURCEDIR/$FILE"  "Test@10.242.xxx.xxx:$TARGETDIR"
echo "$SOURCEDIR/$FILE"  "ProjectMeerkat@10.242.xxx.xxx:$TARGETDIR"
if [[ $? -ne 0 ]] ; then
print "An error occured while transfering file $SOURCEDIR/$FILE"
fi
done
0
 

Author Comment

by:budrito12
ID: 17029022
please ignor the above, I've managed to fix the problem. Once again thanks for your help and explanation and I will award you the points.

The fix:
I put single quotes encapsulating $TARGETDIR...see below.

scp "$SOURCEDIR/$FILE"  "Test@10.242.xxx.xxx:'$TARGETDIR'"
echo "$SOURCEDIR/$FILE"  "ProjectMeerkat@10.242.xxx.xxx:'$TARGETDIR'"

rgds
0
 

Author Comment

by:budrito12
ID: 17087293
Hi,

Another question regarding the above, the 4th field in the filename has to be 6 digits and numeric. If it happens to be greater then  6+ digits how can I put a check in the script to move any filename with the 4th field greater then 6 digits to an error directory.

thanks
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

Many people tend to confuse the function of a virus with the one of adware, this misunderstanding of the basic of what each software is and how it operates causes users and organizations to take the wrong security measures that would protect them ag…
Every server (virtual or physical) needs a console: and the console can be provided through hardware directly connected, software for remote connections, local connections, through a KVM, etc. This document explains the different types of consol…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…

708 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

18 Experts available now in Live!

Get 1:1 Help Now