Solved

How to run a bash/perl script that gets triggered when a file has been received in a directory

Posted on 2011-09-04
3
460 Views
Last Modified: 2012-05-12
Hi Team,

  I have a requirement wherein a third party machine will be SCP-ing  a log file into a user directory on my mySQL server.   I need a mechanism wherein if the file "arrives" it would kick off a script that would process the log file and eventually upload it to my MySQL database.

  While google searching,  I keep stumbling on "inotify" and "fschange".   In my case, it looks like inotify is simpler to implement,...although for my very simple requirement, even  a regularly firing (1 minute interval) cron job will do.

  The problem with the inotify google searches i stumble into is that they just keep referencing the same inotify syntax and no one actually gives a real world example of how to use it.

In my application, the received file could be quite large and could take up to a minute to transfer over, so I need a script that actually checks to see if the file stops growing....and then...start parsing it.

  Any suggestions on how I can implement this would be much appreciated.

  Thanks very much.

0
Comment
Question by:rleyba828
3 Comments
 
LVL 38

Accepted Solution

by:
wesly_chen earned 500 total points
ID: 36480855
1. Tun cron job every 3 minutes
*/3 * * * *  /path-to/check_arrive.sh  

2. Assume the  file is dir   /home/user/upload/
/path-to/check_arrive.sh  
#!/bin/bash

UPLOLAD_DIR=/home/user/upload/

for FILE in `find $UPLOAD_DIR -maxdepth 1 -type f -mmin -3`  # find the file created in past 3 minutes
do
  lsof  $FILE   # check if the file still lock by any process
  LOCK=$?
  if [ "$LOCK" -ne "0" ]   # if the file is free
  then
      <do something>    # better move the file to other folder
  fi
done

Open in new window

0
 
LVL 109

Expert Comment

by:Ray Paseur
ID: 36481348
Have you considered using an email pipe?
0
 

Author Comment

by:rleyba828
ID: 36486216
Hi Wesly,

   Thanks for this.  I will try out your script when I get home tonight so I can test on some LAB machine. I will update you with what I find.

Ray,

   I searched for some links on e-mail pipe.   In our situation though, the receiving box is behind a firewall and is not designed to receive e-mails or have an e-mail account set.  They are allowing inbound ssh only, unfortunately.  But thanks for the idea.
0

Featured Post

VMware Disaster Recovery and Data Protection

In this expert guide, you’ll learn about the components of a Modern Data Center. You will use cases for the value-added capabilities of Veeam®, including combining backup and replication for VMware disaster recovery and using replication for data center migration.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

In the distant past (last year) I hacked together a little toy that would allow a couple of Manager types to query, preview, and extract data from a number of MongoDB instances, to their tool of choice: Excel (http://dilbert.com/strips/comic/2007-08…
This article discusses how to create an extensible mechanism for linked drop downs.
The viewer will learn how to dynamically set the form action using jQuery.
The viewer will learn how to create a basic form using some HTML5 and PHP for later processing. Set up your basic HTML file. Open your form tag and set the method and action attributes.: (CODE) Set up your first few inputs one for the name and …

772 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question