Want to protect your cyber security and still get fast solutions? Ask a secure question today.Go Premium

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 321
  • Last Modified:

Storing Large Amounts of Files into Folders - Efficiently - and Naming Convention

I have an application that will allow users to upload photos. Potentially, my application can reach into the 100,000+ user range. I was wondering if anyone knows an efficient way to create the file structure, or naming convention, to hold the photos. I've seen some examples based on UserIDs, Dates, and/or Splitting up the PhotoID into sections, but I'm still not very clear on these techniques. I was also wondering how many files I should store in each folder so that the files can be access on my website efficiently. I've read anywhere betweek 100 to 1,000 files per folder.

Right now I am thinking about storing the photo file on a web server and using Sql Server to store the file reference and file info (i.e. FileID, UserID, Date, FileName, FolderStructure, and perhaps a DomainID to prepare for situations where I will have to use multiple servers to store the photos).

Any help and insight on the topic will be appreciated. Thanks!
0
Skytide
Asked:
Skytide
  • 3
  • 2
1 Solution
 
Justin_WCommented:
Based on the info above, I would say that it is probably a bad idea to store the images directly on the file system for such a large number of users/files.

You should probably store the files in a binary/image/BLOB column in a DB (e.g. SQL Server) rather than storing them directly on the filesystem. You could also store the filename, user, etc. in other columns (and index them) to provide efficient access to each file without worrying about folder names, etc. Storing them in a DB would also bypass any potential problems resulting from limits on filename/path lengths (max is 256 in Windows IIRC).
0
 
SkytideAuthor Commented:
I would prefer to store the images on a web server since I don't want my database to become too large. I also might have to store images across servers in the future, so I want to avoid having to extract the images from the database when that happens. As a note too, the photos aren't sensitive, so I'm not really worried about the security of them.

Most (but not all) of the articles and discussions I have read have also discourages storing images in blobs. Here is one: http://www.sqlteam.com/item.asp?ItemID=986 However, it's a really confusing topic since it seems like one could go either way.
0
 
Justin_WCommented:
>> Most (but not all) of the articles and discussions I have read have also discourages storing images in blobs.

There were good reasons for this in SQL Server 7.0 and earlier, but SQL Server 2000 and later have resolved those issues
0
 
SkytideAuthor Commented:
Anymore input on this? I want to keep things open in case I decide to use a Content Delivery Network, so I won't be storing the images in the database.
0
 
Justin_WCommented:
Well, as I mentioned, file systems maximum length limits on filenames. Therefore, you may want to store full metadat info (full filename, dates, folders, etc.) in a DB, and use a GUID (or other reasonably short unique string) as the file system filename. That way, you could easily store and access full file data (including for reporting), but wouldn't have to worry about excessive string parsing or filenames. Also, you could always convert the filenames to something else later using a simple custom app.
0

Featured Post

What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

  • 3
  • 2
Tackle projects and never again get stuck behind a technical roadblock.
Join Now