C# Performance Issues With Deleting Files

I've millions of files (<20 bytes each file though) sitting in a directory and I need to be able clean them up by date and file types. I’ve developed a quick console program using C#/2.0 basically I get the files using Directory.GetFiles() method with search options. But this appears to be performing well on directories with few hundred or so files only. What’s the best way to accomplish this task. I need to clean up files from time to time without putting lot of pressure on production servers. Thanks.
ExcUsr2008Asked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

geowrianCommented:
I could be wrong about how the getFiles method works, but based on what I've previously seen, Windows actually pulls the information about all files before applying any filters. So even if your filter only covers 10 of a hundred thousand files in a folder, it still has to iterate the info on all 100,000 files first.

I remember when there was a discussion about slow performance with the randomization seed Firefox developers used, which was based on the attributes of one of the last files in a directory - which took significantly longer as the number files grew, despite only pulling one file. That said, this was C++ and not .Net, but the impression I got was it was a Windows issue.
0
lucky_jamesCommented:
you can directly try issuing the DOS commands.
0
ong-hhCommented:
try using windows search API, it provides the feature to index the folders.

http://www.microsoft.com/windows/products/winfamily/desktopsearch/choose/windowssearch4.mspx
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
The Ultimate Tool Kit for Technolgy Solution Provi

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy for valuable how-to assets including sample agreements, checklists, flowcharts, and more!

leonov_alexCommented:
0
lazyberezovskyCommented:
Another workaround is to put files in directories named by date.
0
dlmichenerCommented:
Any reason it has to be in managed code?  How about a PowerShell or even a BAT file that built the list.  Once the list is generated from the script, then C# could do the dirty work.  Or just do it all in the script.
0
ExcUsr2008Author Commented:
It appears that the efficient way is to index files and consume windows search API functions.
I thank everyone for posting ideas/methods/solutions.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Programming

From novice to tech pro — start learning today.