Eprs_Admin
asked on
NETAPP maxdirsize
Hi Experts,
I have a question regarding a NETAPP storage.
I have the unit FAS2552.
Our ERP system generates a lot of files each day and they are stored in an archive folder on the NETAPP.
Now we have 1.6 million files in it.
On the NETAPP there is a limit which is called MAXDIRSIZE. This is sometimes a problem and I have to increase the limit.
Is it possible to switch it off ?
Can you explain the MAXDIRSIZE ?
What does it mean this value ?
Which value I need for 3 million files in one folder ?
I have a question regarding a NETAPP storage.
I have the unit FAS2552.
Our ERP system generates a lot of files each day and they are stored in an archive folder on the NETAPP.
Now we have 1.6 million files in it.
On the NETAPP there is a limit which is called MAXDIRSIZE. This is sometimes a problem and I have to increase the limit.
Is it possible to switch it off ?
Can you explain the MAXDIRSIZE ?
What does it mean this value ?
Which value I need for 3 million files in one folder ?
ASKER
And why it is neglected ?
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Thanks a lot.
May you answer my questions ?
Is it possible to switch it off ?
Can you explain the MAXDIRSIZE ?
What does it mean this value ?
Which value I need for 3 million files in one folder ?
May you answer my questions ?
Is it possible to switch it off ?
Can you explain the MAXDIRSIZE ?
What does it mean this value ?
Which value I need for 3 million files in one folder ?
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
if you want to increase you can use, but once set you cannot revert back. Like Robocat stated this can lead to performance issues and other unwanted things.
First I run just to list all the vols and the current limit.
Volume FirmData: maximum number of files is currently 31876689 (5724589 used).
Volume LIT_IMG01: maximum number of files is currently 93535556 (88124169 used).
once you have the volume name you want to increase the size in small increments. . I normally do 10% of the current.
ie. LIT_IMG01 is set to 93,535,556. I would increase it to 102,889,111. This will round to the nearest block value.
First I run just to list all the vols and the current limit.
maxfiles
Volume FirmData: maximum number of files is currently 31876689 (5724589 used).
Volume LIT_IMG01: maximum number of files is currently 93535556 (88124169 used).
once you have the volume name you want to increase the size in small increments. . I normally do 10% of the current.
ie. LIT_IMG01 is set to 93,535,556. I would increase it to 102,889,111. This will round to the nearest block value.
maxfiles LIT_IMG01 102889111
ASKER
ok thanks so far
ASKER
As far as I know, the maxfiles value is the total amount of files on the volume.
This is not my problem.
My problem is the maxdirsize, the files per folder.
This is not my problem.
My problem is the maxdirsize, the files per folder.
What options do you have to reduce the number of files, or to sub-divide these into multiple folders? Those may be good questions to ask your application vendor if you don't know the answers.
If you absolutely must support this large number of files in a single directory, you may want to explore configuring your Net APP to support a SAN instead of NAS connection. (Most NetApp systems these days support both.) I've had good results in the past with large numbers of files in a single SAN volume. That may be a good question for you to ask Net App.
If you absolutely must support this large number of files in a single directory, you may want to explore configuring your Net APP to support a SAN instead of NAS connection. (Most NetApp systems these days support both.) I've had good results in the past with large numbers of files in a single SAN volume. That may be a good question for you to ask Net App.
It looks like the Author is asking what is the max size of a file or folder not the actual number of files per volume
Maxdirsize is a volume option (vol options maxdirsize).
The number is not the number of files but the maximum (technical) size of a directory in KBytes. So it's not easy to calculate how big it must be for a certain number of files.
Increasing it without a real understanding of what this means or why the limit is there can lead to severe performance issues, and can not be undone.
So do not increase this limit, but try to move files into subfolders. About 100K files is a reasonable maximum per folder.
The number is not the number of files but the maximum (technical) size of a directory in KBytes. So it's not easy to calculate how big it must be for a certain number of files.
Increasing it without a real understanding of what this means or why the limit is there can lead to severe performance issues, and can not be undone.
So do not increase this limit, but try to move files into subfolders. About 100K files is a reasonable maximum per folder.
ASKER
I spoke to my vendor of the EPR system but they say it is hard to change.
So I think this takes time and I do not have time.
Because we write each day 2000 files into this archiv folder.
So I think this takes time and I do not have time.
Because we write each day 2000 files into this archiv folder.
ASKER
NETAPP told me to create a new volume just for this archive and disable UNC path.
And maxdirsize should not bigger than 3% of the RAM in the controller.
Can you tell how to find out, how much RAM I have on the controller ?
And maxdirsize should not bigger than 3% of the RAM in the controller.
Can you tell how to find out, how much RAM I have on the controller ?
>I spoke to my vendor of the EPR system but they say it is hard to change.
The ERP system doesn't need to do this or even be aware of this. This is actually quite a common problem with many ERP systems that is easily solved, assuming these files can be archived (audit files, log files, ...).
You can start by moving the files manually. Move all files that are no longer actively used to a few subfolders (at least two). Say all files older than a month?
This automatically clears the problem and the original folder will have room again for the months to come. Each subfolder that you create can itself contain as many files as the original folder, no need to increase maxdirsize.
Next, you can automate the archiving of these files, eg. by scripting or using some kind of archival software.
The ERP system doesn't need to do this or even be aware of this. This is actually quite a common problem with many ERP systems that is easily solved, assuming these files can be archived (audit files, log files, ...).
You can start by moving the files manually. Move all files that are no longer actively used to a few subfolders (at least two). Say all files older than a month?
This automatically clears the problem and the original folder will have room again for the months to come. Each subfolder that you create can itself contain as many files as the original folder, no need to increase maxdirsize.
Next, you can automate the archiving of these files, eg. by scripting or using some kind of archival software.
If your ERP system is using a relational database then the files generated are usually kept there for recovery which basically means that you restore your last backup and assuming you do not have your transaction logs backed up then your ERP has kept logs for point and time recovery. Now realistically, will you ever restore your ERP from 6 months ago, last month or even from 2 days ago? The answer is resounding NO. Therefore, these are archive files and what you could do is create a policy to keep at the most one week of files and if you are really paranoid then ensure they files are backed up.
ASKER
Hi ROBOCAT,
ok I understand.
But when I move the files to another folder, the ERP system cannot find the articles anymore.
So this is also not an option.
Because in this archive folder they save all pictures for our articles.
ok I understand.
But when I move the files to another folder, the ERP system cannot find the articles anymore.
So this is also not an option.
Because in this archive folder they save all pictures for our articles.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
many thanks so far.
ASKER