Need help with shell script to call a perl program to run automatically

Posted on 2011-10-19
Medium Priority
Last Modified: 2012-05-12
Hi guys,

I need help on the attached shell script. The idea is when a csv file is dropped into a specific directory, the shell script would call the perl script to process that file. After the file has been processed, the input file would be moved to another folder.

When i run the shell script manually i.e  typing ./do_reconciliation.sh in terminal, the shell script works but it's not doing what I intended it to do at the moment which is to get it to run if the  .csv file is present in the folder.

Help would be much appreciated.




umask 0002

for file in `find "$BASEDIR/input/" -name "*.csv" -printf "%f\n"` ; do

	/u/xi6505/pronto/cus/imports/reconciliation_2.pl $BASEDIR/input/$file	
	mv $BASEDIR/input/$file $BASEDIR/processed/$file


Open in new window

Question by:Jason_Sutiono

Accepted Solution

icenick earned 500 total points
ID: 36998157

First of all, I would change the for loop to "for file in *.conf" which takes less time than find.

After that, I would put the script in crontab to run each 1 minute or 5 minutes to keep up with the new csv files coming into the main directory.

I would add PATH variable to the script so crontab can find the commands and run successfully.

I hope this helps.

Good Luck!

Expert Comment

ID: 36998168
In order to search for csv files in a specific dir you should have sth like this:

for file in /path/to/*.csv

The above will search recursively for all files with extensions .csv under /path/to/.

Good Luck!
LVL 68

Expert Comment

ID: 36998303

your "find" command in line 7 will display the whole path of the "*.csv" file(s) found, i.e. $file will contain something like


The parameter which you're giving to the perl script contains the full path one more time, so the script will see something like


which is obviously wrong.

Call the perl script simply like this and your first part should be fine:

/u/xi6505/pronto/cus/imports/reconciliation_2.pl $file

The "mv" part is slightly more complex:

mv $file $BASEDIR/processed/$(basename $file)

i.e. take only the filename out of the "find" result (that's what "basename" does) and append it to the base directory plus the "processed" subdirectory.


LVL 38

Expert Comment

by:Gerwin Jansen, EE MVE
ID: 37001156
Some general tips that will help you debugging your scripts:

- add "echo" to lines that execute other scripts or perform actions
- test your script by calling it with the "-xv" option to see what it is doing

Doing the above would get you this:

bash -xv ./do_reconciliation.sh 

+ BASEDIR=/tmp

umask 0002
+ umask 0002

for file in `find "$BASEDIR" -name "*.csv" -printf "%f\n" 2>/dev/null`
	echo /u/xi6505/pronto/cus/imports/reconciliation_2.pl $BASEDIR/input/$file
	echo mv $BASEDIR/input/$file $BASEDIR/processed/$file

find "$BASEDIR" -name "*.csv" -printf "%f\n" 2>/dev/null
++ find /tmp -name '*.csv' -printf '%f\n'
+ for file in '`find "$BASEDIR" -name "*.csv" -printf "%f\n" 2>/dev/null`'
+ echo /u/xi6505/pronto/cus/imports/reconciliation_2.pl /tmp/input/test2.csv
/u/xi6505/pronto/cus/imports/reconciliation_2.pl /tmp/input/test2.csv
+ echo mv /tmp/input/test2.csv /tmp/processed/test2.csv
mv /tmp/input/test2.csv /tmp/processed/test2.csv
+ for file in '`find "$BASEDIR" -name "*.csv" -printf "%f\n" 2>/dev/null`'
+ echo /u/xi6505/pronto/cus/imports/reconciliation_2.pl /tmp/input/test.csv
/u/xi6505/pronto/cus/imports/reconciliation_2.pl /tmp/input/test.csv
+ echo mv /tmp/input/test.csv /tmp/processed/test.csv
mv /tmp/input/test.csv /tmp/processed/test.csv

Open in new window


Author Comment

ID: 37015427
Thanks to you guys, managed to get it to work now. Cheers

Featured Post

Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Email validation in proper way is  very important validation required in any web pages. This code is self explainable except that Regular Expression which I used for pattern matching. I originally published as a thread on my website : http://www…
Recently, an awarded photographer, Selina De Maeyer (http://www.selinademaeyer.com/), completed a photo shoot of a beautiful event (http://www.sintjacobantwerpen.be/verslag-en-fotoreportage-van-de-sacramentsprocessie-door-antwerpen#thumbnails) in An…
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…
This video shows how to set up a shell script to accept a positional parameter when called, pass that to a SQL script, accept the output from the statement back and then manipulate it in the Shell.
Suggested Courses

840 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question