Solved

How do I quickly delete a large file on an Ext3 volume?

Posted on 2009-07-13
14
1,503 Views
Last Modified: 2013-12-21
On a Ubuntu 8.04LTS box, I have some rather large files (100GB and larger) that I need to delete on a daily basis.  RM'ing the files take an inordinate amount of time.  Like in the minutes range.  Is there another method of deleting these large files that wouldn't take so long?
FYI, security isn't an issue in this particular matter, so we don't need to worry with wiping the disk, or anything.
Thanks in advance!
0
Comment
Question by:Digital_Skream
  • 5
  • 5
  • 4
14 Comments
 
LVL 4

Expert Comment

by:samierana
ID: 24842767
Would you like to use the idea of adding a new Hard disk and use it to store the said data and instead of deleting just format that drive. it will not take much time. you can use script to do the work.
adding and formating help can be found here:
http://www.ehow.com/how_5068221_format-linux-hard-drives.html
0
 
LVL 1

Author Comment

by:Digital_Skream
ID: 24842788
That isn't feasible in this case.  Logistics prevent such a path.
0
 
LVL 43

Expert Comment

by:ravenpl
ID: 24842872
There's no faster way, sorry. The filesystemhas to update it's on-disk structures, and it has to last - at least under ext3. Consider changing filesystem type.
0
Microsoft Certification Exam 74-409

Veeam® is happy to provide the Microsoft community with a study guide prepared by MVP and MCT, Orin Thomas. This guide will take you through each of the exam objectives, helping you to prepare for and pass the examination.

 
LVL 43

Expert Comment

by:ravenpl
ID: 24842887
If You don't need the space immediately freed, You could run the rm (unlink) in the background...
0
 
LVL 1

Author Comment

by:Digital_Skream
ID: 24842925
Yah, I would need the space freed to create another file via script.
What would be a better filesystem?  I like the robustness of ext3, but the only drawback for me is the lengthy delete times required.  
0
 
LVL 4

Expert Comment

by:samierana
ID: 24842927
Using GUI is one way. It will immediate delete it but will go to Trash. Then you can empty trash later.
If you want your server not to be much busy during this deleting process you can use the following command:
ls | grep .| xargs rm
where "." is the name of the file/partial name of files.
0
 
LVL 1

Author Comment

by:Digital_Skream
ID: 24842946
If it is in Trash, would the space be available for use?  Or would that just be like moving a file from one directory to another?
0
 
LVL 43

Accepted Solution

by:
ravenpl earned 500 total points
ID: 24842973
large files? try ext4 with it's extents (ext3 has fixed extents of 4K size) - the smaller number of extents the faster file deletion.
0
 
LVL 43

Expert Comment

by:ravenpl
ID: 24843026
There's many webpages comparing filesystems performance
http://www.t2-project.org/zine/4/
http://linuxgazette.net/102/piszcz.html
But those tests are rather run on small files...
0
 
LVL 4

Expert Comment

by:samierana
ID: 24843289
space will not be available while files are in trash.
0
 
LVL 4

Expert Comment

by:samierana
ID: 24843325
Check this link out. if it works for you:
http://linux.die.net/man/1/fastrm
0
 
LVL 1

Author Comment

by:Digital_Skream
ID: 24845529
Fastrm sounds very interesting, but I am unable to find the source/binaries/package anywhere for ubuntu.
0
 
LVL 43

Expert Comment

by:ravenpl
ID: 24846446
fastrm will not work for You. It's little faster when deleting large amount of files from single directory or directories tree.
for single large file it has to perform the same operations as regular rm.

fastrm used to be part of "inn" news server package - inn keeps very large number of small files - where fastrm was little faster...
0
 
LVL 1

Author Comment

by:Digital_Skream
ID: 24850815
Then it sounds to me like I need to research a different filesystem, and that would be my only option.
0

Featured Post

Secure Your Active Directory - April 20, 2017

Active Directory plays a critical role in your company’s IT infrastructure and keeping it secure in today’s hacker-infested world is a must.
Microsoft published 300+ pages of guidance, but who has the time, money, and resources to implement? Register now to find an easier way.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Users are often faced with high disk consumption without really knowing where the largest amount of data resides. Disk Usage Analyzer (aka Baobab) is is a graphical, menu-driven application to analyse disk usage in any Gnome environment and can e…
In my business, I use the LTS (Long Term Support) versions of Linux. My workstations do real work, and so I rarely have the patience to deal with silly problems caused by an upgraded kernel that had experimental software on it to begin with from a r…
In a recent question (https://www.experts-exchange.com/questions/29004105/Run-AutoHotkey-script-directly-from-Notepad.html) here at Experts Exchange, a member asked how to run an AutoHotkey script (.AHK) directly from Notepad++ (aka NPP). This video…
How to Install VMware Tools in Red Hat Enterprise Linux 6.4 (RHEL 6.4) Step-by-Step Tutorial

730 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question