?
Solved

How to run a bash/perl script that gets triggered when a file has been received in a directory

Posted on 2011-09-04
3
Medium Priority
?
472 Views
Last Modified: 2012-05-12
Hi Team,

  I have a requirement wherein a third party machine will be SCP-ing  a log file into a user directory on my mySQL server.   I need a mechanism wherein if the file "arrives" it would kick off a script that would process the log file and eventually upload it to my MySQL database.

  While google searching,  I keep stumbling on "inotify" and "fschange".   In my case, it looks like inotify is simpler to implement,...although for my very simple requirement, even  a regularly firing (1 minute interval) cron job will do.

  The problem with the inotify google searches i stumble into is that they just keep referencing the same inotify syntax and no one actually gives a real world example of how to use it.

In my application, the received file could be quite large and could take up to a minute to transfer over, so I need a script that actually checks to see if the file stops growing....and then...start parsing it.

  Any suggestions on how I can implement this would be much appreciated.

  Thanks very much.

0
Comment
Question by:rleyba828
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
3 Comments
 
LVL 38

Accepted Solution

by:
wesly_chen earned 2000 total points
ID: 36480855
1. Tun cron job every 3 minutes
*/3 * * * *  /path-to/check_arrive.sh  

2. Assume the  file is dir   /home/user/upload/
/path-to/check_arrive.sh  
#!/bin/bash

UPLOLAD_DIR=/home/user/upload/

for FILE in `find $UPLOAD_DIR -maxdepth 1 -type f -mmin -3`  # find the file created in past 3 minutes
do
  lsof  $FILE   # check if the file still lock by any process
  LOCK=$?
  if [ "$LOCK" -ne "0" ]   # if the file is free
  then
      <do something>    # better move the file to other folder
  fi
done

Open in new window

0
 
LVL 111

Expert Comment

by:Ray Paseur
ID: 36481348
Have you considered using an email pipe?
0
 

Author Comment

by:rleyba828
ID: 36486216
Hi Wesly,

   Thanks for this.  I will try out your script when I get home tonight so I can test on some LAB machine. I will update you with what I find.

Ray,

   I searched for some links on e-mail pipe.   In our situation though, the receiving box is behind a firewall and is not designed to receive e-mails or have an e-mail account set.  They are allowing inbound ssh only, unfortunately.  But thanks for the idea.
0

Featured Post

Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Checking the Alert Log in AWS RDS Oracle can be a pain through their user interface.  I made a script to download the Alert Log, look for errors, and email me the trace files.  In this article I'll describe what I did and share my script.
Nothing in an HTTP request can be trusted, including HTTP headers and form data.  A form token is a tool that can be used to guard against request forgeries (CSRF).  This article shows an improved approach to form tokens, making it more difficult to…
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
This tutorial will teach you the core code needed to finalize the addition of a watermark to your image. The viewer will use a small PHP class to learn and create a watermark.
Suggested Courses

771 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question