• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 1154
  • Last Modified:

Storing a Large Number of Files in SharePoint & how to Manage them

Hey Everyone,

I was hoping to start some dialogue about SharePoint 2010 and the new Document Management capibilities that come along with it.

I work at a mid-sized business with 40+ employees with our business producing quite a few new documents a day (100+) mainly comprised of emails, word Documents and/or PDFs.

We're using file shares to store these files but should like to move these files into SharePoint the thing is we are finding it hard to get information on what's required for SharePoint to store our exisitng 750GB+ (400,000+) doucments.

Most of the articles online seem to be focusing on the business benefits of storing documents in SharePoint, which we are already feelng pretty confident that we understand, just not so confident on the idea of how SharePoint stores documents in a SQL database. We dont have experience managing databases 750GB+ or databases where transaction logs can bloat so significantly if a user was to upload a large scanned contract.

Does anyone have any experience dealing or using SharePoint as their primary document management system, storing and managing similar amounts of documents? We are a two man IT team and need to approach these things with caution.
0
tdilbert
Asked:
tdilbert
  • 2
1 Solution
 
ImaCircularSawCommented:
SharePoint stores files in it's content database as a binary stream.  There is the option to hold files in a BLOB on a file system that's managed by the database server, however, I would only recommend this for large media files.

The best thing with SharePoint is that you can off-load most of the migration work to users.  You should help them build an appropriate document storage structure then they should be able to do the document migration themselves.  This can be done on a per document basis or you can view a document library in windows explorer and drag/drop a number of files at a time.

If you have some more specific questions about your concerns I could answer them.

One word of warning, there is very rarely a real reason for using folder inside document libraries, try to avoid using them now as later down the line they'll cause problems.
0
 
Justin SmithSr. System EngineerCommented:
Nothing is required for SharePoint to handle all those files, it will do that with ease.  However, it doesn't make it a good idea in all situations.

As Saw mentioned above, all those files (BLOBs) will go straight into a SQL database.  SQL wasn't designed to hold or stream unstructured data files.  It CAN do it, but it wasn't designed to.  It's much better to offload those to file servers and let SQL manage the metadata.  This offload is done through a technology called Remote BLOB Storage.  This is a new feature of SQL Server 2008 R2.

If you do decide to go the RBS route, I would suggest buying an enterprise level product to act as your RBS provider.  I have had great luck with Metalogix StoragePoint.  And with this product, it would allow you to leave the files where they are, and "catalog" them into SharePoint.  This means they would live where they currently are, but SharePoint thinks they have been imported.  You can set a target document library, set metadata, and run the import.....while the files stay where they are.  Pretty powerful.

There are also some free RBS providers out there, that do the job, but aren't as robust.  The File Extender from AvePoint is probably the most popular.
0
 
tdilbertAuthor Commented:
This is the sort of answer IT departments of a lot of companies are looking for. I think the functionality of SharePoint merits purchasing and investing in the product. However, as the business side of a company look at some of the other capabilities they start saying "why don't we do our document management in SharePoint".

Unfortunately we find that Microsoft spends more time advertising that SharePoint can handle millions of files, but not such a good job of documenting how and what involved resource wise (people, software and time). It would seem that the internet is riddled with a lot of SharePoint enthusiasts that have more faith than experience in this area.
I am going to look into StoragePoint, thank you ACH1LLES.
0
 
Justin SmithSr. System EngineerCommented:
No problem.  If you need assistance implementing the RBS solution, I'm available.  My email address is in my profile.
0

Featured Post

 [eBook] Windows Nano Server

Download this FREE eBook and learn all you need to get started with Windows Nano Server, including deployment options, remote management
and troubleshooting tips and tricks

  • 2
Tackle projects and never again get stuck behind a technical roadblock.
Join Now