Solved

Ideal max files/directories per directory

Posted on 2004-08-04
4
971 Views
Last Modified: 2013-12-15
Hi guys,

I'm using Red Hat Enterprise and would like to know what the ideal maximum number of files/directories per directory is. Currently we have a directory with over 7000 directories in it. For various reasons we now have to restructure this directory, and an important factor in deciding how to do this will be the ideal number of files/directories per directory.

A few years ago I was doing a similar project but on a Sun Solaris system, and the sys admin there told me the ideal max inodes per directory was about 200.

If anyone knows what the figure would be for Red Hat Enterprise, your advise would be very much appreciated.

Rangi
0
Comment
Question by:rangi500
  • 2
4 Comments
 
LVL 3

Expert Comment

by:pYrania
Comment Utility
I don't know where the so called sys admin got his information from, but there is no such limit.

I've seen performant filesystems with a lot more files/directories within another directory.
0
 
LVL 40

Expert Comment

by:jlevie
Comment Utility
I don't know if there is an "ideal number" of nodes within a directory for Linux or Solaris, but 7000 in one directory seems like a lot. I suspect that the the "ideal number" is largely determined by the way those nodes are used. An application that only accessed the directory at startup wouldn't suffer as much as one that was continuous accessing the directory. For the later apps breaking the single large directory in to some sort of hashed structure would be a big help.
0
 
LVL 2

Author Comment

by:rangi500
Comment Utility
Thanks for your feedback guys.

pYrania, apparenly there was an 'optimal' number of files per directory - perhaps a 'most efficient' number?

jlevie, yes we are planning to create a better structure, possibly hashed, but one of the main things we need to know to make the decision on how we do it is how many directories to put in each directory. If 7000, sounds like a lot, what sounds like "not a lot" - 1000? 100?

Some facts that may or may not be relevant:

   - the folders are read from much more than they are written to
   - the folders make up about 500GB of data
   - our sys admin has said the 7000 dirs in one dir may be responsible for crashing our backup software

Thanks,

Rangi
0
 
LVL 40

Accepted Solution

by:
jlevie earned 250 total points
Comment Utility
It probably doesn't matter much that the access is mostly read, nor should it matter as to the total data size, with respect to directory size. What would matter most is number of file opens per unit of time. A good hash scheme reduces the time required to locate a file for reading and as a bonus reduces the directory overhead associated with file creation.

> - our sys admin has said the 7000 dirs in one dir may be responsible for crashing our backup software

I don't know what you are using for backups, but I've never had any problems with Solaris ufsdump or Legato Networker with far larger directories than that. I discourage people from creating really big directories, but sometimes they still do ('til I find out about it).
0

Featured Post

Highfive Gives IT Their Time Back

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

Suggested Solutions

Title # Comments Views Activity
IPA client Config problems 2 96
Virtual Box issue 6 49
BASH script to modify crontab? 3 19
linux boot fsck problem 3 43
This is the error message I got (CODE) Error caused by incompatible libmp3lame 3.98-2 with ffmpeg I've googled this error message and found out sometimes it attaches this note "can be treated with downgrade libmp3lame to version 3.97 or 3.98" …
Little introduction about CP: CP is a command on linux that use to copy files and folder from one location to another location. Example usage of CP as follow: cp /myfoder /pathto/destination/folder/ cp abc.tar.gz /pathto/destination/folder/ab…
Connecting to an Amazon Linux EC2 Instance from Windows Using PuTTY.
This demo shows you how to set up the containerized NetScaler CPX with NetScaler Management and Analytics System in a non-routable Mesos/Marathon environment for use with Micro-Services applications.

763 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

9 Experts available now in Live!

Get 1:1 Help Now