Link to home
Start Free TrialLog in
Avatar of Eprs_Admin
Eprs_AdminFlag for Austria

asked on

NETAPP maxdirsize

Hi Experts,

I have a question regarding a NETAPP storage.
I have the unit FAS2552.
Our ERP system generates a lot of files each day and they are stored in an archive folder on the NETAPP.
Now we have 1.6 million files in it.
On the NETAPP there is a limit which is called MAXDIRSIZE. This is sometimes a problem and I have to increase the limit.

Is it possible to switch it off ?
Can you explain the MAXDIRSIZE ?
What does it mean this value ?
Which value I need for 3 million files in one folder ?
Avatar of Eprs_Admin
Eprs_Admin
Flag of Austria image

ASKER

Who can help me here ?
And why it is neglected ?
SOLUTION
Avatar of robocat
robocat

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Thanks a lot.
May you answer my questions ?

Is it possible to switch it off ?
Can you explain the MAXDIRSIZE ?
What does it mean this value ?
Which value I need for 3 million files in one folder ?
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
if you want to increase you can use, but once set you cannot revert back.  Like Robocat stated this can lead to performance issues and other unwanted things.

First I run just to list all the vols and the current limit.
maxfiles

Open in new window


Volume FirmData: maximum number of files is currently 31876689 (5724589 used).
Volume LIT_IMG01: maximum number of files is currently 93535556 (88124169 used).
once you have the volume name you want to increase the size in small increments.  .  I normally do 10% of the current.
ie.  LIT_IMG01 is set to 93,535,556.   I would increase it to 102,889,111.  This will round to the nearest block value.
maxfiles LIT_IMG01 102889111

Open in new window

ok thanks so far
As far as I know, the maxfiles value is the total amount of files on the volume.
This is not my problem.

My problem is the maxdirsize, the files per folder.
What options do you have to reduce the number of files, or to sub-divide these into multiple folders?  Those may be good questions to ask your application vendor if you don't know the answers.

If you absolutely must support this large number of files in a single directory, you may want to explore configuring your Net APP to support a SAN instead of NAS connection.  (Most NetApp systems these days support both.)  I've had good results in the past with large numbers of files in a single SAN volume.  That may be a good question for you to ask Net App.
It looks like the Author is asking what is the max size of a file or folder not the actual number of files per volume
Avatar of robocat
robocat

Maxdirsize is a volume option (vol options maxdirsize).

The number is not the number of files but the maximum (technical) size of a directory in KBytes.  So it's not easy to calculate how big it must be for a certain number of files.

Increasing it without a real understanding of what this means or why the limit is there can lead to severe performance issues, and can not be undone.

So do not increase this limit, but try to move files into subfolders. About 100K files is a reasonable maximum per folder.
I spoke to my vendor of the EPR system but they say it is hard to change.
So I think this takes time and I do not have time.
Because we write each day 2000 files into this archiv folder.
NETAPP told me to create a new volume just for this archive and disable UNC path.
And maxdirsize should not bigger than 3% of the RAM in the controller.
Can you tell how to find out, how much RAM I have on the controller ?
>I spoke to my vendor of the EPR system but they say it is hard to change.

The ERP system doesn't need to do this or even be aware of this. This is actually quite a common problem with many ERP systems that is easily solved, assuming these files can be archived (audit files, log files, ...).

You can start by moving the files manually. Move all files that are no longer actively used to a few subfolders (at least two). Say all files older than a month?

This automatically clears the problem and the original folder will have room again for the months to come. Each subfolder that you create can itself contain as many files as the original folder, no need to increase maxdirsize.

Next, you can automate the archiving of these files, eg. by scripting or using some kind of archival software.
If your ERP system is using a relational database then the files generated are usually kept there for recovery which basically means that you restore your last backup and assuming you do not have your transaction logs backed up then your ERP has kept logs for point and time recovery.  Now realistically, will you ever restore your ERP from 6 months ago, last month or even from 2 days ago?  The answer is resounding NO.  Therefore, these are archive files and what you could do is create a policy to keep at the most one week of files and if you are really paranoid then ensure they files are backed up.
Hi ROBOCAT,

ok I understand.
But when I move the files to another folder, the ERP system cannot find the articles anymore.
So this is also not an option.
Because in this archive folder they save all pictures for our articles.
ASKER CERTIFIED SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
many thanks so far.