Solved

How do I quickly delete a large file on an Ext3 volume?

Posted on 2009-07-13
14
1,487 Views
Last Modified: 2013-12-21
On a Ubuntu 8.04LTS box, I have some rather large files (100GB and larger) that I need to delete on a daily basis.  RM'ing the files take an inordinate amount of time.  Like in the minutes range.  Is there another method of deleting these large files that wouldn't take so long?
FYI, security isn't an issue in this particular matter, so we don't need to worry with wiping the disk, or anything.
Thanks in advance!
0
Comment
Question by:Digital_Skream
  • 5
  • 5
  • 4
14 Comments
 
LVL 4

Expert Comment

by:samierana
Comment Utility
Would you like to use the idea of adding a new Hard disk and use it to store the said data and instead of deleting just format that drive. it will not take much time. you can use script to do the work.
adding and formating help can be found here:
http://www.ehow.com/how_5068221_format-linux-hard-drives.html
0
 
LVL 1

Author Comment

by:Digital_Skream
Comment Utility
That isn't feasible in this case.  Logistics prevent such a path.
0
 
LVL 43

Expert Comment

by:ravenpl
Comment Utility
There's no faster way, sorry. The filesystemhas to update it's on-disk structures, and it has to last - at least under ext3. Consider changing filesystem type.
0
 
LVL 43

Expert Comment

by:ravenpl
Comment Utility
If You don't need the space immediately freed, You could run the rm (unlink) in the background...
0
 
LVL 1

Author Comment

by:Digital_Skream
Comment Utility
Yah, I would need the space freed to create another file via script.
What would be a better filesystem?  I like the robustness of ext3, but the only drawback for me is the lengthy delete times required.  
0
 
LVL 4

Expert Comment

by:samierana
Comment Utility
Using GUI is one way. It will immediate delete it but will go to Trash. Then you can empty trash later.
If you want your server not to be much busy during this deleting process you can use the following command:
ls | grep .| xargs rm
where "." is the name of the file/partial name of files.
0
 
LVL 1

Author Comment

by:Digital_Skream
Comment Utility
If it is in Trash, would the space be available for use?  Or would that just be like moving a file from one directory to another?
0
Top 6 Sources for Identifying Threat Actor TTPs

Understanding your enemy is essential. These six sources will help you identify the most popular threat actor tactics, techniques, and procedures (TTPs).

 
LVL 43

Accepted Solution

by:
ravenpl earned 500 total points
Comment Utility
large files? try ext4 with it's extents (ext3 has fixed extents of 4K size) - the smaller number of extents the faster file deletion.
0
 
LVL 43

Expert Comment

by:ravenpl
Comment Utility
There's many webpages comparing filesystems performance
http://www.t2-project.org/zine/4/
http://linuxgazette.net/102/piszcz.html
But those tests are rather run on small files...
0
 
LVL 4

Expert Comment

by:samierana
Comment Utility
space will not be available while files are in trash.
0
 
LVL 4

Expert Comment

by:samierana
Comment Utility
Check this link out. if it works for you:
http://linux.die.net/man/1/fastrm
0
 
LVL 1

Author Comment

by:Digital_Skream
Comment Utility
Fastrm sounds very interesting, but I am unable to find the source/binaries/package anywhere for ubuntu.
0
 
LVL 43

Expert Comment

by:ravenpl
Comment Utility
fastrm will not work for You. It's little faster when deleting large amount of files from single directory or directories tree.
for single large file it has to perform the same operations as regular rm.

fastrm used to be part of "inn" news server package - inn keeps very large number of small files - where fastrm was little faster...
0
 
LVL 1

Author Comment

by:Digital_Skream
Comment Utility
Then it sounds to me like I need to research a different filesystem, and that would be my only option.
0

Featured Post

Free Trending Threat Insights Every Day

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

Join & Write a Comment

The purpose of this article is to fix the unknown display problem in Linux Mint operating system. After installing the OS if you see Display monitor is not recognized then we can install "MESA" utilities to fix this problem or we can install additio…
The purpose of this article is to demonstrate how we can upgrade Python from version 2.7.6 to Python 2.7.10 on the Linux Mint operating system. I am using an Oracle Virtual Box where I have installed Linux Mint operating system version 17.2. Once yo…
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
This video discusses moving either the default database or any database to a new volume.

763 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now