• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 150
  • Last Modified:

Best practices for sharing large amounts of data

I work for a city government and our police department uses video systems that often result in large video files that need to be sent to district attorney's offices, other lawyers, etc. They had previously been burning data DVD's but now with HD video they're running into files that are larger than DVD's 4.7 gb limit. The PD wanted to invest in a bunch of blu-ray burners and we advised against it since most lawyers and other offices most likely won't have blu-ray players to read the disks, not to mention that optical drives are more or less obsolete or soon will be. Other suggestions have been to use flash drives (purchased in bulk for cheap), setup a local FTP site and let them maintain logins for it, cloud sharing, etc.

So my question to throw out there to all you experts is this - What would you consider to be the best practice for sharing large data files? Some individual files may go as large as 20 gigs in size. I'm looking for pros/cons, cost estimates, etc.
William Fulks
William Fulks
4 Solutions
Tom CieslikIT EngineerCommented:
You can convert video filed to most efficient mp4 format and share using FTP server or create VPN tunnel and let them download files from share folder where you'll sent them link to.
It's always problem with that big files
Flash drives will become expensive if you'll calculate costs per year.
Dr. KlahnPrincipal Software EngineerCommented:
Given that Blu-Ray players are selling from $30 to $80 now, I would not consider that a block to using Blu-Ray discs.  That kind of money is petty cash these days.

Side note:  It's going to be a long time before optical drives are obsolete while Microsoft distributes Windows and Windows software on them.

The public should not be footing the bill for this.  "Reasonable and customary charges" are made for FOIA and other document copying.  Video should be no different.  If someone wants 20 GB of video, sock him for a 32 GB USB drive at cost plus 10%, plus tech time.  Everybody has a budget.
rojoshoRTCC-III Level-2 SupportCommented:
Hey William,

Good comments from my colleagues.  My first question (OK, concern) is how secure do these files have to be?  Flash Drives, DVD and Blu-Ray are only the latest form of what we once called 'Sneakernet' and it is still as unsecured today as it was back in the 80's and 90's.  So if security is an issue, then setting up a VPN to a ftp server as suggested by Tom, would be the preferred way to ensure that the persons with the right 'need to know' are allowed (Key word) access to this information.
> We have to remember that we are not just discussing transporting files/data, but that these files are sensitive and may be covered under the HIPAA and Patriots Act guidelines.
> As will all removal media, the data/files/needs **always** out grows the current technology - think about... 8" Floopy to 3.5", to CD, to DVD, to Blu-Ray, to Flash Drive.... "the beat goes on"....
> With that said, that should give you the justification to budget the true equipment and configuration that your department will need.

Forgive me, my degree is in ISA....I cant help my self.

Simple Misconfiguration =Network Vulnerability

In this technical webinar, AlgoSec will present several examples of common misconfigurations; including a basic device change, business application connectivity changes, and data center migrations. Learn best practices to protect your business from attack.

Gerald ConnollyCommented:
if you stick with the physical route (aka sneakernet) using USB flash drives dont forget to encrypt the data and send the key via an alternative route (eg email)

NB if you are going to use an FTP site use encryption as well, but be aware of the time it may take for you to upload and the recipient to download e.g a 20GB file over a 100Mb/s link will probably take 2-3 hours to upload
rojoshoRTCC-III Level-2 SupportCommented:

EXCELLENT points - Encryption would be a must if sneaker-net is one of your options - but remember that 85% of all security issues are caused by human error.

Definitely a case for AWS.
If you are from US, AWS has a gov region that covers your regulation needs. Even without the gov region, all Hipaa and SOC and ISO are well addressed.
Technically speaking, you will upload the relevant files to a secure S3 bucket, either all of them or on demand, and create authorised links for playback or for download.
It will be the most economical way, with proper security in place. I suggest that you talk with an Amazon solutions architect or an amazon partner about it.
William FulksSystems Analyst & WebmasterAuthor Commented:
Thanks everyone for the input. I'll let this hang out a couple more days to see if any more replies come in. This whole thing has turned into another discussion (at work) about laws regarding evidence storage, etc. I was told by one person that evidence in murder cases must be stored for 75 years, other cases have different periods of time, etc. Interesting stuff to think about from an IT side as more stuff goes totally digital.
William FulksSystems Analyst & WebmasterAuthor Commented:
Thanks y'all!
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Featured Post

The Firewall Audit Checklist

Preparing for a firewall audit today is almost impossible.
AlgoSec, together with some of the largest global organizations and auditors, has created a checklist to follow when preparing for your firewall audit. Simplify risk mitigation while staying compliant all of the time!

Tackle projects and never again get stuck behind a technical roadblock.
Join Now