[2 days left] What’s wrong with your cloud strategy? Learn why multicloud solutions matter with Nimble Storage.Register Now

x
?
Solved

shell script update database with the file name....

Posted on 2014-02-14
11
Medium Priority
?
387 Views
Last Modified: 2014-05-06
Hello, I have a linux ubuntu server and at times there are new files added to a folder. I would like to have shell script that will check this folder periodically and then update the folder name to the database ( if the name does not exist in the db already). Can someone let me know how I can do this? Thanks for the help.

A
0
Comment
Question by:aej1973
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
11 Comments
 
LVL 111

Expert Comment

by:Ray Paseur
ID: 39861226
Uhh, can you please tell us a little more about the business rules here?  Why are you asking for a shell script?  What role does that play in your application logic and workflow?

It's relatively easy to detect changes in a folder.  PHP has scandir().  You can store a copy of the output and compare the current scandir() output to the stored copy to see if anything changed.  You can go a little further by checking filemtime() values for the contents of the folder.
0
 

Author Comment

by:aej1973
ID: 39861301
Hello Ray, thanks for getting back to me. My business logic is as follows:

1.There are times that a user will upload an audio file to my server. This upload is not done via any form but through a telephone.

2.Since the upload is not done via a form I need to check to see if any new audio file has been uploaded to the folder ( there is a specific folder that the audio file gets uploaded to).

3.If a new audio file has been uploaded, then I need to upload the name of the audio file to a mysql database. The audio file is in the format aaaa@bbbbb.net. I will need to split the string at the '@'sign and store the 'aaaa' portion as one field and the 'bbbbb' portion as another field.

Thanks again for the help.

Regards,
a
0
 
LVL 111

Expert Comment

by:Ray Paseur
ID: 39861354
Makes sense.  Have you considered using a CRON job?  You can fire up the CRON job once per minute and it will take only a fraction of a second to check for new files.  That would seem to be more efficient than manually running something from the command line.
0
Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

 

Author Comment

by:aej1973
ID: 39861361
True, that is what I am working on now.
0
 
LVL 111

Expert Comment

by:Ray Paseur
ID: 39861435
Good.  You can usually set something like that up with cPanel or similar.  Are you planning on keeping all of the uploaded files in the same directory or moving them to a different permanent store?
0
 

Author Comment

by:aej1973
ID: 39861451
just keep them in the same directory.
0
 
LVL 111

Expert Comment

by:Ray Paseur
ID: 39861521
OK, here is what I would do.  

When you use scandir() you get back an array.  You can use json_encode() to turn this array into a string.  You can use md5() to make a hash from the string.  Md5() is idempotent, meaning that it will always create the same hash from the same input string.  You can store this hash and the  JSON string in a data base table.  If the newly made hash is the same as the old hash, there have been no changes to the directory.  If there is a difference, you can compare the contents of the current scandir() to the old scandir() and detect the differences.  That should tell you what is new in the directory.

You might also want to consider the possibility that there could be a name collision in the directory.  Not sure if that could happen, but it's worth a moment's thought since it might cause data loss.
0
 
LVL 2

Expert Comment

by:c_kedar
ID: 39864072
Shell command 'find' is well suited for this purpose.
Use -mtime options of find, something like 'find -mtime -10' to get list of files created/updated in last 10 minutes and process the output of the command to update db.

Setup cron to run this script every 10 minutes.
0
 
LVL 111

Expert Comment

by:Ray Paseur
ID: 39864968
Maybe worth a test; I've never used iwatch, but the blog post about it sounds credible.
0
 
LVL 27

Accepted Solution

by:
skullnobrains earned 2000 total points
ID: 39886543
iwatch / inotify is nice but not sufficient because if your script stops at some point, it will not pickup changes that happened in the meantime

it would be much easier if you moved the file to another directory after processing them

but if you need to keep them in the same dir, as far as i know, mtime works in days and not minutes. it seems more reasonable to use the -newer predicate and store where you stopped in a different file. there are many other possible options.

something like this should do (written directly in ee so may be typos)

DIR=/your/watch/dir
touch /var/run/upload_checks_
sleep 1

values=$(
if test -f /var/run/upload_checks
then find $DIR -newer /var/run/upload_checks
else find $DIR
fi | sed 's:.*/\([^/@]*\)@\([^.]\)\..*$:,("\1","\2"):' | tr -d "\n" | cut 2-
)

echo 'insert ignore into TABLE (cola,colb) VALUES '$values | mysql -u USER -p PASS -h HOST  \
&& mv /var/run/upload_checks_ /var/run/upload_checks

Open in new window


make sure your db has a uniq index on the combined 2 fields
0

Featured Post

[Webinar] Lessons on Recovering from Petya

Skyport is working hard to help customers recover from recent attacks, like the Petya worm. This work has brought to light some important lessons. New malware attacks like this can take down your entire environment. Learn from others mistakes on how to prevent Petya like worms.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Recently, an awarded photographer, Selina De Maeyer (http://www.selinademaeyer.com/), completed a photo shoot of a beautiful event (http://www.sintjacobantwerpen.be/verslag-en-fotoreportage-van-de-sacramentsprocessie-door-antwerpen#thumbnails) in An…
Introduction This article is intended for those who are new to PHP error handling (https://www.experts-exchange.com/articles/11769/And-by-the-way-I-am-New-to-PHP.html).  It addresses one of the most common problems that plague beginning PHP develop…
The viewer will learn how to look for a specific file type in a local or remote server directory using PHP.
In a recent question (https://www.experts-exchange.com/questions/29004105/Run-AutoHotkey-script-directly-from-Notepad.html) here at Experts Exchange, a member asked how to run an AutoHotkey script (.AHK) directly from Notepad++ (aka NPP). This video…
Suggested Courses

649 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question