50,000 files in a file share across a WAN = slow search

Does anyone know a way to optimize windows searching through folders across a WAN?  The client is a small 3 computer network hosting their files in my data center across a VPN.  The internet pipe for them is 20 MBit down and 5 MBit up.  They also are using Branch Cache (which is working nicely - but apparently not for this!) for all file shares.

Things my client isn't willing to use:
1.  Document management system (unless open source  / free)
2.  FOLDER STRUCTURE.  Man.. they don't want to organize files into folders.  It's 50k of loose doc and pdf files.  Messy, but its their business model.

When in MSWord, searching for files by name is painful for them.  It takes a really long time to find what they're looking for.  Is there any way to make windows cache and hold windows file names for frequently used folders?

Topology:
Routers:  Fortigate IPSEC VPN Site-to-site
Workstations:  Windows 7 Ultimate domain joined
Server:  Server 2008 R2
DNS: AD integrated


Thanks!
LVL 2
TimFarrenAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

JohnBusiness Consultant (Owner)Commented:
I hope you (they) did not expect different. Situation normal for that kind of model.

The practical way over VPN is to make a subset of files on each local machine and then Sync Back Pro to sync files at the end of the day (or twice a day).  That way the files are local and searching through the subset will be fast.

Otherwise, it is probably as fast as it is going to go.
0
TimFarrenAuthor Commented:
Update... I'm looking into OpenKM.  Anyone used it before?
http://openkm.com/en/

John - these files are shared.  I would prefer to have them read / write to the server at all times to keep things real-time and up-to-date.  It all works very fast opening files because of the branch cache feature enabled on the 2008 server and workstations, but searching through filenames is what is slow.  If that could be improved - then all is well.
0
David Johnson, CD, MVPOwnerCommented:
50K files is a lot to go through even on a local machine.
0
Redefining Cyber Security w/ AI & Machine Learning

The implications of AI and machine learning in cyber security are massive and constantly growing, creating both efficiencies and new challenges across the board. Join our webinar on Sept. 21st to learn more about leveraging AI and machine learning to protect your business.

TimFarrenAuthor Commented:
David - Yes.  Agreed.  This is the reason for my post.  Trying to find creative ways to fix the problem.  I've already suggested they break their files into smaller groups by using folders - but they don't want to do this.
0
Larry Struckmeyer MVPCommented:
"Doctor... It hurts when I do this....."

you know the rest.  <g>
0
TimFarrenAuthor Commented:
I've installed something called DocFetcher that seems to be doing the trick quite nicely.  It's indexed all 56,000 documents and allows for quick search and retrieval.  By the looks of the comments above, I may have to choose my own comment as the solution.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Bryant SchaperCommented:
Do you have indexing enable on the filesystem?

Searching large volumes of files is painful, but how long are we talking about.  We have one massive folder with hundreds or thousands of subfolders and 1.5 million files, over the WAN we can search the first folder tier quick it has 50k+, the rest  of the folders take some time, a few minutes.

That said how does the underlying disk IO look, could you be bottlenecking at the storage tier?
0
TimFarrenAuthor Commented:
Bryant - the indexing - client side or server side?  This is a mapped network drive.
0
TimFarrenAuthor Commented:
I was looking for a 3rd party indexing option basically.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Windows Networking

From novice to tech pro — start learning today.