Solved

How to run a bash/perl script that gets triggered when a file has been received in a directory

Posted on 2011-09-04
3
455 Views
Last Modified: 2012-05-12
Hi Team,

  I have a requirement wherein a third party machine will be SCP-ing  a log file into a user directory on my mySQL server.   I need a mechanism wherein if the file "arrives" it would kick off a script that would process the log file and eventually upload it to my MySQL database.

  While google searching,  I keep stumbling on "inotify" and "fschange".   In my case, it looks like inotify is simpler to implement,...although for my very simple requirement, even  a regularly firing (1 minute interval) cron job will do.

  The problem with the inotify google searches i stumble into is that they just keep referencing the same inotify syntax and no one actually gives a real world example of how to use it.

In my application, the received file could be quite large and could take up to a minute to transfer over, so I need a script that actually checks to see if the file stops growing....and then...start parsing it.

  Any suggestions on how I can implement this would be much appreciated.

  Thanks very much.

0
Comment
Question by:rleyba828
3 Comments
 
LVL 38

Accepted Solution

by:
wesly_chen earned 500 total points
Comment Utility
1. Tun cron job every 3 minutes
*/3 * * * *  /path-to/check_arrive.sh  

2. Assume the  file is dir   /home/user/upload/
/path-to/check_arrive.sh  
#!/bin/bash

UPLOLAD_DIR=/home/user/upload/

for FILE in `find $UPLOAD_DIR -maxdepth 1 -type f -mmin -3`  # find the file created in past 3 minutes
do
  lsof  $FILE   # check if the file still lock by any process
  LOCK=$?
  if [ "$LOCK" -ne "0" ]   # if the file is free
  then
      <do something>    # better move the file to other folder
  fi
done

Open in new window

0
 
LVL 108

Expert Comment

by:Ray Paseur
Comment Utility
Have you considered using an email pipe?
0
 

Author Comment

by:rleyba828
Comment Utility
Hi Wesly,

   Thanks for this.  I will try out your script when I get home tonight so I can test on some LAB machine. I will update you with what I find.

Ray,

   I searched for some links on e-mail pipe.   In our situation though, the receiving box is behind a firewall and is not designed to receive e-mails or have an e-mail account set.  They are allowing inbound ssh only, unfortunately.  But thanks for the idea.
0

Featured Post

How to improve team productivity

Quip adds documents, spreadsheets, and tasklists to your Slack experience
- Elevate ideas to Quip docs
- Share Quip docs in Slack
- Get notified of changes to your docs
- Available on iOS/Android/Desktop/Web
- Online/Offline

Join & Write a Comment

This article discusses four methods for overlaying images in a container on a web page
Since pre-biblical times, humans have sought ways to keep secrets, and share the secrets selectively.  This article explores the ways PHP can be used to hide and encrypt information.
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
The viewer will learn how to dynamically set the form action using jQuery.

772 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now