?
Solved

How do I quickly delete a large file on an Ext3 volume?

Posted on 2009-07-13
14
Medium Priority
?
1,517 Views
Last Modified: 2013-12-21
On a Ubuntu 8.04LTS box, I have some rather large files (100GB and larger) that I need to delete on a daily basis.  RM'ing the files take an inordinate amount of time.  Like in the minutes range.  Is there another method of deleting these large files that wouldn't take so long?
FYI, security isn't an issue in this particular matter, so we don't need to worry with wiping the disk, or anything.
Thanks in advance!
0
Comment
Question by:Digital_Skream
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 5
  • 5
  • 4
14 Comments
 
LVL 4

Expert Comment

by:samierana
ID: 24842767
Would you like to use the idea of adding a new Hard disk and use it to store the said data and instead of deleting just format that drive. it will not take much time. you can use script to do the work.
adding and formating help can be found here:
http://www.ehow.com/how_5068221_format-linux-hard-drives.html
0
 
LVL 1

Author Comment

by:Digital_Skream
ID: 24842788
That isn't feasible in this case.  Logistics prevent such a path.
0
 
LVL 43

Expert Comment

by:ravenpl
ID: 24842872
There's no faster way, sorry. The filesystemhas to update it's on-disk structures, and it has to last - at least under ext3. Consider changing filesystem type.
0
Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 43

Expert Comment

by:ravenpl
ID: 24842887
If You don't need the space immediately freed, You could run the rm (unlink) in the background...
0
 
LVL 1

Author Comment

by:Digital_Skream
ID: 24842925
Yah, I would need the space freed to create another file via script.
What would be a better filesystem?  I like the robustness of ext3, but the only drawback for me is the lengthy delete times required.  
0
 
LVL 4

Expert Comment

by:samierana
ID: 24842927
Using GUI is one way. It will immediate delete it but will go to Trash. Then you can empty trash later.
If you want your server not to be much busy during this deleting process you can use the following command:
ls | grep .| xargs rm
where "." is the name of the file/partial name of files.
0
 
LVL 1

Author Comment

by:Digital_Skream
ID: 24842946
If it is in Trash, would the space be available for use?  Or would that just be like moving a file from one directory to another?
0
 
LVL 43

Accepted Solution

by:
ravenpl earned 2000 total points
ID: 24842973
large files? try ext4 with it's extents (ext3 has fixed extents of 4K size) - the smaller number of extents the faster file deletion.
0
 
LVL 43

Expert Comment

by:ravenpl
ID: 24843026
There's many webpages comparing filesystems performance
http://www.t2-project.org/zine/4/
http://linuxgazette.net/102/piszcz.html
But those tests are rather run on small files...
0
 
LVL 4

Expert Comment

by:samierana
ID: 24843289
space will not be available while files are in trash.
0
 
LVL 4

Expert Comment

by:samierana
ID: 24843325
Check this link out. if it works for you:
http://linux.die.net/man/1/fastrm
0
 
LVL 1

Author Comment

by:Digital_Skream
ID: 24845529
Fastrm sounds very interesting, but I am unable to find the source/binaries/package anywhere for ubuntu.
0
 
LVL 43

Expert Comment

by:ravenpl
ID: 24846446
fastrm will not work for You. It's little faster when deleting large amount of files from single directory or directories tree.
for single large file it has to perform the same operations as regular rm.

fastrm used to be part of "inn" news server package - inn keeps very large number of small files - where fastrm was little faster...
0
 
LVL 1

Author Comment

by:Digital_Skream
ID: 24850815
Then it sounds to me like I need to research a different filesystem, and that would be my only option.
0

Featured Post

7 Extremely Useful Linux Commands for Beginners

Just getting started with Linux? Here's a quick start guide that has 7 commands that we believe will come in handy.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In my business, I use the LTS (Long Term Support) versions of Linux. My workstations do real work, and so I rarely have the patience to deal with silly problems caused by an upgraded kernel that had experimental software on it to begin with from a r…
This article will explain how to establish a SSH connection to Ubuntu through the firewall and using a different port other then 22. I have set up a Ubuntu virtual machine in Virtualbox and I am running a Windows 7 workstation. From the Ubuntu vi…
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
How to Install VMware Tools in Red Hat Enterprise Linux 6.4 (RHEL 6.4) Step-by-Step Tutorial
Suggested Courses

800 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question