Solved

How do I quickly delete a large file on an Ext3 volume?

Posted on 2009-07-13
14
1,509 Views
Last Modified: 2013-12-21
On a Ubuntu 8.04LTS box, I have some rather large files (100GB and larger) that I need to delete on a daily basis.  RM'ing the files take an inordinate amount of time.  Like in the minutes range.  Is there another method of deleting these large files that wouldn't take so long?
FYI, security isn't an issue in this particular matter, so we don't need to worry with wiping the disk, or anything.
Thanks in advance!
0
Comment
Question by:Digital_Skream
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 5
  • 5
  • 4
14 Comments
 
LVL 4

Expert Comment

by:samierana
ID: 24842767
Would you like to use the idea of adding a new Hard disk and use it to store the said data and instead of deleting just format that drive. it will not take much time. you can use script to do the work.
adding and formating help can be found here:
http://www.ehow.com/how_5068221_format-linux-hard-drives.html
0
 
LVL 1

Author Comment

by:Digital_Skream
ID: 24842788
That isn't feasible in this case.  Logistics prevent such a path.
0
 
LVL 43

Expert Comment

by:ravenpl
ID: 24842872
There's no faster way, sorry. The filesystemhas to update it's on-disk structures, and it has to last - at least under ext3. Consider changing filesystem type.
0
Don't Cry: How Liquid Web is Ensuring Security

WannaCry is just the start. Read how Liquid Web is protecting itself and its customers against new threats.

 
LVL 43

Expert Comment

by:ravenpl
ID: 24842887
If You don't need the space immediately freed, You could run the rm (unlink) in the background...
0
 
LVL 1

Author Comment

by:Digital_Skream
ID: 24842925
Yah, I would need the space freed to create another file via script.
What would be a better filesystem?  I like the robustness of ext3, but the only drawback for me is the lengthy delete times required.  
0
 
LVL 4

Expert Comment

by:samierana
ID: 24842927
Using GUI is one way. It will immediate delete it but will go to Trash. Then you can empty trash later.
If you want your server not to be much busy during this deleting process you can use the following command:
ls | grep .| xargs rm
where "." is the name of the file/partial name of files.
0
 
LVL 1

Author Comment

by:Digital_Skream
ID: 24842946
If it is in Trash, would the space be available for use?  Or would that just be like moving a file from one directory to another?
0
 
LVL 43

Accepted Solution

by:
ravenpl earned 500 total points
ID: 24842973
large files? try ext4 with it's extents (ext3 has fixed extents of 4K size) - the smaller number of extents the faster file deletion.
0
 
LVL 43

Expert Comment

by:ravenpl
ID: 24843026
There's many webpages comparing filesystems performance
http://www.t2-project.org/zine/4/
http://linuxgazette.net/102/piszcz.html
But those tests are rather run on small files...
0
 
LVL 4

Expert Comment

by:samierana
ID: 24843289
space will not be available while files are in trash.
0
 
LVL 4

Expert Comment

by:samierana
ID: 24843325
Check this link out. if it works for you:
http://linux.die.net/man/1/fastrm
0
 
LVL 1

Author Comment

by:Digital_Skream
ID: 24845529
Fastrm sounds very interesting, but I am unable to find the source/binaries/package anywhere for ubuntu.
0
 
LVL 43

Expert Comment

by:ravenpl
ID: 24846446
fastrm will not work for You. It's little faster when deleting large amount of files from single directory or directories tree.
for single large file it has to perform the same operations as regular rm.

fastrm used to be part of "inn" news server package - inn keeps very large number of small files - where fastrm was little faster...
0
 
LVL 1

Author Comment

by:Digital_Skream
ID: 24850815
Then it sounds to me like I need to research a different filesystem, and that would be my only option.
0

Featured Post

Why Off-Site Backups Are The Only Way To Go

You are probably backing up your data—but how and where? Ransomware is on the rise and there are variants that specifically target backups. Read on to discover why off-site is the way to go.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This document is written for Red Hat Enterprise Linux AS release 4 and ORACLE 10g.  Earlier releases can be installed using this document as well however there are some additional steps for packages to be installed see Metalink. Disclaimer: I hav…
Recently, an awarded photographer, Selina De Maeyer (http://www.selinademaeyer.com/), completed a photo shoot of a beautiful event (http://www.sintjacobantwerpen.be/verslag-en-fotoreportage-van-de-sacramentsprocessie-door-antwerpen#thumbnails) in An…
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
In a recent question (https://www.experts-exchange.com/questions/29004105/Run-AutoHotkey-script-directly-from-Notepad.html) here at Experts Exchange, a member asked how to run an AutoHotkey script (.AHK) directly from Notepad++ (aka NPP). This video…

696 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question