TEEDA
asked on
Folders vs metadata for SharePoint Records Management
We are in the process of planning a Records Management System. Right now, we are using 2007, but we do plan to move to 2010 in early 2012.
My own approach to folders in SharePoint has always been a big NO - don't use them. However, a colleague has brought up a very valid concern. She is in charge of a site where multiple users upload files monthly, and has run into the issue of files being overwritten because of a duplicate file name. If the files were uploaded into segregated folders, it would significantly reduce the chances of overwriting existing files (in her site, at least)..
The area I will be creating will hold very sensitive information about students - this information may be subject to legal requests and so e-discovery is of paramount importance. Since many of the students will have the same type of document attached to them (medical diagnosis letters, psychologists reports) the danger of overwriting is of huge concern.
Initially, my thought was to have the documents uploaded with some required metadata - for instance, each student has a unique student number, so that would be a required field. But as my colleague pointed out, that will not prevent a file called Psych Report for student #001 overwriting Psych Report for student #006. Since multiple users in multiple locations will be uploading files, file naming conventions are unlikely to be followed.
Any help in solving this dilemma would be most appreciated. Thanks for your time.
My own approach to folders in SharePoint has always been a big NO - don't use them. However, a colleague has brought up a very valid concern. She is in charge of a site where multiple users upload files monthly, and has run into the issue of files being overwritten because of a duplicate file name. If the files were uploaded into segregated folders, it would significantly reduce the chances of overwriting existing files (in her site, at least)..
The area I will be creating will hold very sensitive information about students - this information may be subject to legal requests and so e-discovery is of paramount importance. Since many of the students will have the same type of document attached to them (medical diagnosis letters, psychologists reports) the danger of overwriting is of huge concern.
Initially, my thought was to have the documents uploaded with some required metadata - for instance, each student has a unique student number, so that would be a required field. But as my colleague pointed out, that will not prevent a file called Psych Report for student #001 overwriting Psych Report for student #006. Since multiple users in multiple locations will be uploading files, file naming conventions are unlikely to be followed.
Any help in solving this dilemma would be most appreciated. Thanks for your time.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
@sharepointguru14: Because a document library is actually a specialized list and a folder is actually a list item that can contain other list items yes there is some permissions that can be applied, however, overall it is not like managing a file system and doesn't have the same full features because for example you've simply separated the storage of the files from the display which is a good solution and I've done the same for a EE type system at my last employer (that was far superior to the EE technology)
I'm avoiding proposing a technical solution requiring programming at this point simply because the asker hasn't shown any sign of having that type of technical expertise. For example, the asker didn't mention SharePoint Designer or Visual Studio
Remember, SharePoint 2010 can now do some amazing things with only InfoPath and the web UI
Have you actually used the Content Organizer in SharePoint 2010?
I'm avoiding proposing a technical solution requiring programming at this point simply because the asker hasn't shown any sign of having that type of technical expertise. For example, the asker didn't mention SharePoint Designer or Visual Studio
Remember, SharePoint 2010 can now do some amazing things with only InfoPath and the web UI
Have you actually used the Content Organizer in SharePoint 2010?
Yep content organizer is pretty good if you get the rules setup correctly. I didn't bother suggesting it because you did. I was simply curious as to your take on why you suggest that permissions could only be handled at the root with subsites and document libraries being the endpoint, but maybe I was misunderstanding.
@sharepointguru14: I do often promote the idea of separating storage from display with SharePoint but in this case I think it would be a lot of work and might not be required. If you read my first comment I didn't say you can't apply permissions to folders. I said it simply doesn't work the same or as well. Cheers
ASKER
The content routing sounds interesting, but I may have to get this started in 2007 and migrate to 2010 next year (another issue that worries me).
I do have SharePoint Designer and often use it for workflows. Since we have up to 800 students in any one year in this system (a subset of our total student population), each of whom could have multiple records uploaded, I was very reluctant to go the folder route. It would mean that someone would have to create a folder every time a new student was added, and there would be nothing to prevent a document being uploaded to the wrong folder. In addition, the retention requirement for these records is 25 years, so we could end up with 250,000 or more documents in this one system. We will definitely have metadata added to every upload regardless of whether we use folders or not (lots of metadata!).
So as per sharepointguru14's suggestion, I could use a workflow to fire on new item creation that appends a randomly generated (or sequential, such as date/time) suffix to the file name. That way, no matter how many times a file called Psych Report is uploaded, previous records with that file name would have already been renamed with the suffix and would not be overwritten.
I think I have a solution!! Many thanks!!
I do have SharePoint Designer and often use it for workflows. Since we have up to 800 students in any one year in this system (a subset of our total student population), each of whom could have multiple records uploaded, I was very reluctant to go the folder route. It would mean that someone would have to create a folder every time a new student was added, and there would be nothing to prevent a document being uploaded to the wrong folder. In addition, the retention requirement for these records is 25 years, so we could end up with 250,000 or more documents in this one system. We will definitely have metadata added to every upload regardless of whether we use folders or not (lots of metadata!).
So as per sharepointguru14's suggestion, I could use a workflow to fire on new item creation that appends a randomly generated (or sequential, such as date/time) suffix to the file name. That way, no matter how many times a file called Psych Report is uploaded, previous records with that file name would have already been renamed with the suffix and would not be overwritten.
I think I have a solution!! Many thanks!!
OK, conversion from 2007 to 2010 might not go well if you aren't careful. I actually attended a invite only "SharePoint Deep Dive" at Microsoft HQ in Redmond and was told by their top experts that in-place upgrades were only designed for installations of SharePoint on "Small Business Server"
Basically a new install of SP 2010 is better than an upgraded one because the 2007 features cannot be fully upgraded.
If you are going to go the workflow route how are you going to handle multiple versions of the same document?
I'd go straight to 2010 and save yourself work in the long run.
Basically a new install of SP 2010 is better than an upgraded one because the 2007 features cannot be fully upgraded.
If you are going to go the workflow route how are you going to handle multiple versions of the same document?
I'd go straight to 2010 and save yourself work in the long run.
ASKER
Oh, boy. I think I'm between a rock and a hard place on the SP versions decision.
Versioning - the only document we need versioning on is one that will be an online (probably InfoPath) form - it will edited online and past versions kept. Everything else is an outside consult and therefore will have to be scanned and uploaded with required metadata.
Many thanks to you both for your help. It is much appreciated.
Versioning - the only document we need versioning on is one that will be an online (probably InfoPath) form - it will edited online and past versions kept. Everything else is an outside consult and therefore will have to be scanned and uploaded with required metadata.
Many thanks to you both for your help. It is much appreciated.
If you have the option of going 2010 up front I would highly recommend that. Anytime you can avoid a migration it is worth the extra time or effort. In place upgrades are really never the best idea migrations always seem to be the better route.
TedBilly I was referring to your comment right after you started that it doesn't work like a file system where you say:
" You can only control access and permissions at the root, so if you want to mimic the behaviour of a file share you need to use subsites for the folder structure with the document library as the leaf folder (end point) where files can be uploaded."
Which is what confused me as you can handle permissions at the folder and file level within a SharePoint library.
TedBilly I was referring to your comment right after you started that it doesn't work like a file system where you say:
" You can only control access and permissions at the root, so if you want to mimic the behaviour of a file share you need to use subsites for the folder structure with the document library as the leaf folder (end point) where files can be uploaded."
Which is what confused me as you can handle permissions at the folder and file level within a SharePoint library.
About 25 years ago when non-mainframe computers began to enter the workplace I was working in Civil engineering. That industry generates a lot of paperwork and all the paperwork is legally binding. That is why professional engineers have to sign so much of the work. The documentation would have to go through many revisions and because file systems were still somewhat primitive (and haven't improved much) a project would 20 people created many duplicate file names even with a naming convention. Some of the projects had as many as 80,000 documents! The technology I've used many versions of DOS, UNIX, Novell, Windows, custom web applications and finally SharePoint.
As I've found and you've likely found, users will not honor naming conventions and sometimes will ignore file hierarchies and dump files anywhere with any name.
There is an issue with SharePoint document libraries that is relevant here if you want to mimic a file share. Permissions and control of folders doesn't work the same as a file share. You can only control access and permissions at the root, so if you want to mimic the behaviour of a file share you need to use subsites for the folder structure with the document library as the leaf folder (end point) where files can be uploaded.
So instead of:
http://sharepoint.domain.com/documentlibrary/folder/folder/folder/
you use
http://sharepoint.domain.com/folder/folder/folder/documentlibrary
However, that is ignoring very powerful features in SharePoint.
First off, use MetaData no matter what. It will allow you to force users to apply appropriate context to the documents which might be missing from the filename if the user enters silly names. Metadata is used by search and can be managed by custom workflows or any programmatic solutions you want to use to manipulate the documents.
Another choice you have is a custom list with documents as attachments. That method will still allow Metadata and will also allow duplicate filenames!
If your team has the expertise custom workflows can be used with Metadata to manage and move documents uploaded by users and correct naming conventions. It's tricky but we've done it in MOSS 2007.
However, this new feature in SharePoint might work for you:
http://blogs.technet.com/b/speschka/archive/2009/10/30/sharepoint-2010-content-organizer-part-1-a-cool-new-feature-for-managing-your-content.aspx