Still celebrating National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

Deleting (cffile) files where the related id does not exist- Coldfusion

Posted on 2014-03-06
9
Medium Priority
?
373 Views
Last Modified: 2014-03-13
I need help figuring out how to delete files in a folder that are no longer needed.
They are image files, which relate to active real estate listings.
When the listing is no longer active I wish to delete this images which are associated to the listing.
My database table only has active listings, it doesn't not have old, deleted or expired.
When my table gets updated (every 20 min) any non actives will be purged from the table, when this occurs I want to be able to delete the associated images from the server..

the naming convention is:
property id = propertyID
Images = IMG_propertyID_1, IMG_propertyID_2, IMG_propertyID_3...


Any help with this would be greatly appreciated.
0
Comment
Question by:izweig
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 6
  • 3
9 Comments
 
LVL 52

Expert Comment

by:_agx_
ID: 39911341
The basic steps are simple, but your approach will depends a lot on your current process, how the images are stored, total number of files, etc.. Conceptually:

1. before the records are purged, run a query to grab properties that will be deleted.
2. Loop through the property id's. For each one, use <cfdirectory> with a filter ie  filter="IMG_#currentPropertyID#*" to locate the associated files. Then loop through the results and delete each file. Something along these lines (not tested)

          <cfloop query="deletedProperties" ...>
                 <cfdirectory action="list"
                        directory="..."
                        filter="IMG_#deletedProperties.propertyID#*"
                        type="file"
                        listinfo="name"
                        name="filesToDelete"  ...>
                 <cfloop query="filesToDelete">
                       <cfset fullPath = filesToDelete.directory &"/"& filesToDelete.name>
                       <cffile action="delete" file="#fullPath#"
                 </cfloop>
          </cfloop>

Keep in mind it won't scale well if you have a large number of images to delete OR a bunch of images all stored in a single directory.  If that's the case, you may want to offload the cleanup process to a scheduled task. Run every X minutes and have the script process a batch of N items at a time (50,100, whatever...). That approach is more scalable and ensures the script won't timeout due to processing a massive number of files.

Before you purge the properties, move the id's to a separate table, say PropertiesToDelete. When the script runs, grab the first N items:

              SELECT PropertyID FROM PropertiesToDelete LIMIT 50

The loop through the id's and find the files (just like above). Only add a step - After the physical files are deleted - delete the propertyID record from the table - so it won't be processed again.

HTH
0
 

Author Comment

by:izweig
ID: 39911355
Thanks, I like this approach.
My problem is I do not know  when these records will be deleted.
The data is fed to my server and updated every 20 min.

That is my challenge.. I need to:
match the ID from the table to the file names,
where  'no matching ID'  action  delete file

Does this make sense?
Thanks!
0
 
LVL 52

Expert Comment

by:_agx_
ID: 39911470
(Method 1)
Well ideally you want to compare the old and new data feeds to determine which property id's were deleted.  Rather than just truncating and repopulating the table. Once you identify the deleted id's, the rest is simple. Example:

Current Data ID's: 1,2,3,5,6,7
New Data  ID's:     1,5,6,8,14
Deleted ID's:         2,3,7    <=== removed

I don't know how you're processing the new data, but typically you feed new data into a temp table. Then use JOIN's to identify new-changed-or deleted records. I remember posting an answer about something similar.. let me see if I can find it...

(Method 2)
Failing that, the only way to figure out which files to delete would be to compare all records in the db with all files on disk, and figure out what's missing.  That could be Very time consuming since you have to examine *everything*. Best not to do that if you can avoid it. That sounds like what you're talking about below. ie Check all the files against all db records - and figure out which files to delete, ie


That is my challenge.. I need to:
match the ID from the table to the file names,
where  'no matching ID'  action  delete file

Keep in mind you might need to do that once - just to get the ball rolling, but going forward it's better to do the detection - when the properties are removed ie Method 1. , instead of after the fact.
0
Ransomware: The New Cyber Threat & How to Stop It

This infographic explains ransomware, type of malware that blocks access to your files or your systems and holds them hostage until a ransom is paid. It also examines the different types of ransomware and explains what you can do to thwart this sinister online threat.  

 
LVL 52

Accepted Solution

by:
_agx_ earned 1500 total points
ID: 39912777
(EDIT)

I don't know how you're processing the new data, but typically you feed new data into a temp table. Then use JOIN's to identify new-changed-or deleted records. I remember posting an answer about something similar.. let me see if I can find it...

I can't seem to find it ... but with the "old" and "new" data in separate tables, a simple OUTER JOIN will identify the properties that no longer exist in the "new" feed ie The ones you want to delete

         SELECT  old.PropertyID
         FROM    OldDataTable old LEFT JOIN NewDataTable nw
                        ON nw.PropertyID = old.PropertyID
         WHERE  nw.PropertyID IS NULL

I'm making some assumptions about your app. Like I mentioned earlier, the specifics and exact will really depend on how your current process works. ie How is your new data is imported, how many images, how they're stored etc....
0
 
LVL 52

Expert Comment

by:_agx_
ID: 39927472
@izweig - Why the  "B" grade? A "B" means the answer was incomplete. The answers provided above were very thorough given you only provided a general outline of your process.  As I mentioned several times, to provide specific code - we need specific information about your process. ie "How is your new data is imported, how many images, how they're stored etc.... ".  You should not assign B grades unless you've provided all of the necessary information and responded to all of the questions.
0
 

Author Comment

by:izweig
ID: 39927485
Does B mean incomplete?
B means Good, according to the form, I thought your solution was good.
Would you like an A?
0
 
LVL 52

Expert Comment

by:_agx_
ID: 39927513
It's more of an FYI. It may not very clear from the "answer" screen but generally

"A" means => answered well
"B" means => incomplete/partially answered
"C" means => poorly answered.

With most experts its a point of pride to provide solid and thorough answers. So a "B" grade is often seen as indicating they didn't fully answer the question. Sometimes it happens, but usually asker and expert exchange more questions until the expert has given them a complete answer they were satisfied with.

(Edit) Again, it's a matter of perception ... (yes, we experts do take our fields seriously ;-) So just an FYI for future grading.
0
 

Author Comment

by:izweig
ID: 39927532
I see what you are saying. This is not clear in the grading form.
0
 
LVL 52

Expert Comment

by:_agx_
ID: 39927565
Yes, I spend more time answering than asking, so I'm not as familiar w/the answer form, but figured it's probably not as clear as it could be.  (Edit) There used to be an FAQ on it. Not sure if it's still around after the redesign.  Just figured I'd give you a heads up for the future. Anyway, thanks for responding :)  Well back to the grind for me...
0

Featured Post

Enroll in September's Course of the Month

This month’s featured course covers 16 hours of training in installation, management, and deployment of VMware vSphere virtualization environments. It's free for Premium Members, Team Accounts, and Qualified Experts!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In this series, we will discuss common questions received as a database Solutions Engineer at Percona. In this role, we speak with a wide array of MySQL and MongoDB users responsible for both extremely large and complex environments to smaller singl…
By, Vadim Tkachenko. In this article we’ll look at ClickHouse on its one year anniversary.
In this video, Percona Solution Engineer Dimitri Vanoverbeke discusses why you want to use at least three nodes in a database cluster. To discuss how Percona Consulting can help with your design and architecture needs for your database and infras…
In this video, Percona Solutions Engineer Barrett Chambers discusses some of the basic syntax differences between MySQL and MongoDB. To learn more check out our webinar on MongoDB administration for MySQL DBA: https://www.percona.com/resources/we…
Suggested Courses

705 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question