Need help with shell script to call a perl program to run automatically

Posted on 2011-10-19
Last Modified: 2012-05-12
Hi guys,

I need help on the attached shell script. The idea is when a csv file is dropped into a specific directory, the shell script would call the perl script to process that file. After the file has been processed, the input file would be moved to another folder.

When i run the shell script manually i.e  typing ./ in terminal, the shell script works but it's not doing what I intended it to do at the moment which is to get it to run if the  .csv file is present in the folder.

Help would be much appreciated.




umask 0002

for file in `find "$BASEDIR/input/" -name "*.csv" -printf "%f\n"` ; do

	/u/xi6505/pronto/cus/imports/ $BASEDIR/input/$file	
	mv $BASEDIR/input/$file $BASEDIR/processed/$file


Open in new window

Question by:Jason_Sutiono
    LVL 7

    Accepted Solution


    First of all, I would change the for loop to "for file in *.conf" which takes less time than find.

    After that, I would put the script in crontab to run each 1 minute or 5 minutes to keep up with the new csv files coming into the main directory.

    I would add PATH variable to the script so crontab can find the commands and run successfully.

    I hope this helps.

    Good Luck!
    LVL 7

    Expert Comment

    In order to search for csv files in a specific dir you should have sth like this:

    for file in /path/to/*.csv

    The above will search recursively for all files with extensions .csv under /path/to/.

    Good Luck!
    LVL 68

    Expert Comment


    your "find" command in line 7 will display the whole path of the "*.csv" file(s) found, i.e. $file will contain something like


    The parameter which you're giving to the perl script contains the full path one more time, so the script will see something like


    which is obviously wrong.

    Call the perl script simply like this and your first part should be fine:

    /u/xi6505/pronto/cus/imports/ $file

    The "mv" part is slightly more complex:

    mv $file $BASEDIR/processed/$(basename $file)

    i.e. take only the filename out of the "find" result (that's what "basename" does) and append it to the base directory plus the "processed" subdirectory.


    LVL 37

    Expert Comment

    by:Gerwin Jansen
    Some general tips that will help you debugging your scripts:

    - add "echo" to lines that execute other scripts or perform actions
    - test your script by calling it with the "-xv" option to see what it is doing

    Doing the above would get you this:

    bash -xv ./ 
    + BASEDIR=/tmp
    umask 0002
    + umask 0002
    for file in `find "$BASEDIR" -name "*.csv" -printf "%f\n" 2>/dev/null`
    	echo /u/xi6505/pronto/cus/imports/ $BASEDIR/input/$file
    	echo mv $BASEDIR/input/$file $BASEDIR/processed/$file
    find "$BASEDIR" -name "*.csv" -printf "%f\n" 2>/dev/null
    ++ find /tmp -name '*.csv' -printf '%f\n'
    + for file in '`find "$BASEDIR" -name "*.csv" -printf "%f\n" 2>/dev/null`'
    + echo /u/xi6505/pronto/cus/imports/ /tmp/input/test2.csv
    /u/xi6505/pronto/cus/imports/ /tmp/input/test2.csv
    + echo mv /tmp/input/test2.csv /tmp/processed/test2.csv
    mv /tmp/input/test2.csv /tmp/processed/test2.csv
    + for file in '`find "$BASEDIR" -name "*.csv" -printf "%f\n" 2>/dev/null`'
    + echo /u/xi6505/pronto/cus/imports/ /tmp/input/test.csv
    /u/xi6505/pronto/cus/imports/ /tmp/input/test.csv
    + echo mv /tmp/input/test.csv /tmp/processed/test.csv
    mv /tmp/input/test.csv /tmp/processed/test.csv

    Open in new window


    Author Comment

    Thanks to you guys, managed to get it to work now. Cheers

    Featured Post

    What Is Threat Intelligence?

    Threat intelligence is often discussed, but rarely understood. Starting with a precise definition, along with clear business goals, is essential.

    Join & Write a Comment

    Attention: This article will no longer be maintained. If you have any questions, please feel free to mail me. Please see for the updated article. It is avail…
    Why Shell Scripting? Shell scripting is a powerful method of accessing UNIX systems and it is very flexible. Shell scripts are required when we want to execute a sequence of commands in Unix flavored operating systems. “Shell” is the command line i…
    Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
    This video shows how to set up a shell script to accept a positional parameter when called, pass that to a SQL script, accept the output from the statement back and then manipulate it in the Shell.

    754 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    16 Experts available now in Live!

    Get 1:1 Help Now