Avatar of mjbegin
mjbeginFlag for United States of America asked on

Extremely slow response opening a folder with thousands of sub-folders.

I have a client who is going paperless. They are scanning medical documents in a folder. Properties of the folder shows it is 50GB large, 176046 folders, and 78,092 files. I don't understand how the folder count got so skewered. I know the actual root folder has 11,000 folders in it. Each root folder can have 6 sub-folders and each of those sub-folders has many more sub-folders. I do not believe there are 176046 folders with only 78092 files. Can propertes of a root folder lie?

Anyway, opening the root folder takes 1-3 minutes on all their desktops. I can see the NIC Icon light stay on solid the entire wait time. Opening up the folder on the server takes only a few seconds. I have a managed switch and looked at all the ports. All ports look good.

Not sure what else to look at. Any feedback will be appreciated.

Mike Begin
Windows Server 2003Windows XPSwitches / Hubs

Avatar of undefined
Last Comment
mjbegin

8/22/2022 - Mon
Mal Osborne

Is the drive running NTFS?  FAT32 is way slower at these sort of operations.   Tried running CHKDSK on the drive?
fruhj

Windows networking is really slow at enumerating folders.
I assume you have shared the root, which when they open it, has to display 11,000 entries?
This is too much.

A different method of organization is needed.
ASKER
mjbegin

Malmensa, All drives are running NTFS.

Fruhj, I agree this might not be the best solution, but what else would you suggest. They are only a third of the wy through scanning the medical docs. I never thought it would be this many folders/docs.
Experts Exchange is like having an extremely knowledgeable team sitting and waiting for your call. Couldn't do my job half as well as I do without it!
James Murphy
fruhj

Theres got to be a way to break it down initially - maybe have 26 initial folders (A->Z) or something to limit the initial directory scan.

Any alternates such as FTP would also be plagued by an 11,000 node initial directory listing.
ASKER
mjbegin

Fruhj,

I was thinking of breaking the folder into several smaller folders. Something like A-E, F-H, etc. All the folders are named patient last name, first name. But if they are only a third of the way thru the scanning process to make them paperless, then I may have led them down the wrong path. Ultimately, this will be a huge amount of folders, sub-folders, and files that Microsoft will have issues with. Someone suggested I crank up a Linux machine and store all these folders on it, share it on the network, and map a drive from all the desktops to it. I feel this might not work since the desktops still need to open the root folder and display 11,000 folders.

If you or anyone else knows of a better solution, or think Linux might work, I will appreciate the feedback.

Mike
fruhj

I agree with you about linux.
so long as windows is your destination, you're going to want to limit things.

If the files are scanned in any way that has data attached to them, you can always use that data to move things around later.
for example, I have clients that scan and name files according to a convention - in your case, if it's last name_firstname, then it would be easy to create top level folders for a-z
you could still do sub folders under that.

other ideas are top folders of the year, or year and month (but not the date, 365x50 years and you're right back in the same place!
Get an unlimited membership to EE for less than $4 a week.
Unlimited question asking, solutions, articles and more.
ASKER CERTIFIED SOLUTION
mjbegin

Log in or sign up to see answer
Become an EE member today7-DAY FREE TRIAL
Members can start a 7-Day Free trial then enjoy unlimited access to the platform
Sign up - Free for 7 days
or
Learn why we charge membership fees
We get it - no one likes a content blocker. Take one extra minute and find out why we block content.
See how we're fighting big data
Not exactly the question you had in mind?
Sign up for an EE membership and get your own personalized solution. With an EE membership, you can ask unlimited troubleshooting, research, or opinion questions.
ask a question