?
Solved

C# Performance Issues With Deleting Files

Posted on 2010-04-01
7
Medium Priority
?
346 Views
Last Modified: 2013-11-10
I've millions of files (<20 bytes each file though) sitting in a directory and I need to be able clean them up by date and file types. I’ve developed a quick console program using C#/2.0 basically I get the files using Directory.GetFiles() method with search options. But this appears to be performing well on directories with few hundred or so files only. What’s the best way to accomplish this task. I need to clean up files from time to time without putting lot of pressure on production servers. Thanks.
0
Comment
Question by:ExcUsr2008
7 Comments
 
LVL 12

Expert Comment

by:geowrian
ID: 29422369
I could be wrong about how the getFiles method works, but based on what I've previously seen, Windows actually pulls the information about all files before applying any filters. So even if your filter only covers 10 of a hundred thousand files in a folder, it still has to iterate the info on all 100,000 files first.

I remember when there was a discussion about slow performance with the randomization seed Firefox developers used, which was based on the attributes of one of the last files in a directory - which took significantly longer as the number files grew, despite only pulling one file. That said, this was C++ and not .Net, but the impression I got was it was a Windows issue.
0
 
LVL 7

Expert Comment

by:lucky_james
ID: 29424661
you can directly try issuing the DOS commands.
0
 
LVL 4

Accepted Solution

by:
ong-hh earned 2000 total points
ID: 29426688
try using windows search API, it provides the feature to index the folders.

http://www.microsoft.com/windows/products/winfamily/desktopsearch/choose/windowssearch4.mspx
0
The new generation of project management tools

With monday.com’s project management tool, you can see what everyone on your team is working in a single glance. Its intuitive dashboards are customizable, so you can create systems that work for you.

 
LVL 3

Expert Comment

by:leonov_alex
ID: 29426954
0
 
LVL 12

Expert Comment

by:lazyberezovsky
ID: 29440903
Another workaround is to put files in directories named by date.
0
 
LVL 2

Expert Comment

by:dlmichener
ID: 29479107
Any reason it has to be in managed code?  How about a PowerShell or even a BAT file that built the list.  Once the list is generated from the script, then C# could do the dirty work.  Or just do it all in the script.
0
 

Author Comment

by:ExcUsr2008
ID: 29676648
It appears that the efficient way is to index files and consume windows search API functions.
I thank everyone for posting ideas/methods/solutions.
0

Featured Post

Never miss a deadline with monday.com

The revolutionary project management tool is here!   Plan visually with a single glance and make sure your projects get done.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Make the most of your online learning experience.
The SignAloud Glove is capable of translating American Sign Language signs into text and audio.
In this fourth video of the Xpdf series, we discuss and demonstrate the PDFinfo utility, which retrieves the contents of a PDF's Info Dictionary, as well as some other information, including the page count. We show how to isolate the page count in a…
Progress

601 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question