Best practices for sharing large amounts of data

Posted on 2016-12-08
Last Modified: 2017-01-10
I work for a city government and our police department uses video systems that often result in large video files that need to be sent to district attorney's offices, other lawyers, etc. They had previously been burning data DVD's but now with HD video they're running into files that are larger than DVD's 4.7 gb limit. The PD wanted to invest in a bunch of blu-ray burners and we advised against it since most lawyers and other offices most likely won't have blu-ray players to read the disks, not to mention that optical drives are more or less obsolete or soon will be. Other suggestions have been to use flash drives (purchased in bulk for cheap), setup a local FTP site and let them maintain logins for it, cloud sharing, etc.

So my question to throw out there to all you experts is this - What would you consider to be the best practice for sharing large data files? Some individual files may go as large as 20 gigs in size. I'm looking for pros/cons, cost estimates, etc.
Question by:William Fulks
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
LVL 23

Assisted Solution

by:Tom Cieslik
Tom Cieslik earned 83 total points
ID: 41918396
You can convert video filed to most efficient mp4 format and share using FTP server or create VPN tunnel and let them download files from share folder where you'll sent them link to.
It's always problem with that big files
Flash drives will become expensive if you'll calculate costs per year.
LVL 28

Accepted Solution

Dr. Klahn earned 251 total points
ID: 41918401
Given that Blu-Ray players are selling from $30 to $80 now, I would not consider that a block to using Blu-Ray discs.  That kind of money is petty cash these days.

Side note:  It's going to be a long time before optical drives are obsolete while Microsoft distributes Windows and Windows software on them.

The public should not be footing the bill for this.  "Reasonable and customary charges" are made for FOIA and other document copying.  Video should be no different.  If someone wants 20 GB of video, sock him for a 32 GB USB drive at cost plus 10%, plus tech time.  Everybody has a budget.

Assisted Solution

rojosho earned 83 total points
ID: 41919764
Hey William,

Good comments from my colleagues.  My first question (OK, concern) is how secure do these files have to be?  Flash Drives, DVD and Blu-Ray are only the latest form of what we once called 'Sneakernet' and it is still as unsecured today as it was back in the 80's and 90's.  So if security is an issue, then setting up a VPN to a ftp server as suggested by Tom, would be the preferred way to ensure that the persons with the right 'need to know' are allowed (Key word) access to this information.
> We have to remember that we are not just discussing transporting files/data, but that these files are sensitive and may be covered under the HIPAA and Patriots Act guidelines.
> As will all removal media, the data/files/needs **always** out grows the current technology - think about... 8" Floopy to 3.5", to CD, to DVD, to Blu-Ray, to Flash Drive.... "the beat goes on"....
> With that said, that should give you the justification to budget the true equipment and configuration that your department will need.

Forgive me, my degree is in ISA....I cant help my self.

Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

LVL 17

Assisted Solution

by:Gerald Connolly
Gerald Connolly earned 83 total points
ID: 41920268
if you stick with the physical route (aka sneakernet) using USB flash drives dont forget to encrypt the data and send the key via an alternative route (eg email)

NB if you are going to use an FTP site use encryption as well, but be aware of the time it may take for you to upload and the recipient to download e.g a 20GB file over a 100Mb/s link will probably take 2-3 hours to upload

Expert Comment

ID: 41920717

EXCELLENT points - Encryption would be a must if sneaker-net is one of your options - but remember that 85% of all security issues are caused by human error.

LVL 33

Expert Comment

ID: 41921091
Definitely a case for AWS.
If you are from US, AWS has a gov region that covers your regulation needs. Even without the gov region, all Hipaa and SOC and ISO are well addressed.
Technically speaking, you will upload the relevant files to a secure S3 bucket, either all of them or on demand, and create authorised links for playback or for download.
It will be the most economical way, with proper security in place. I suggest that you talk with an Amazon solutions architect or an amazon partner about it.
LVL 15

Author Comment

by:William Fulks
ID: 41921418
Thanks everyone for the input. I'll let this hang out a couple more days to see if any more replies come in. This whole thing has turned into another discussion (at work) about laws regarding evidence storage, etc. I was told by one person that evidence in murder cases must be stored for 75 years, other cases have different periods of time, etc. Interesting stuff to think about from an IT side as more stuff goes totally digital.
LVL 15

Author Closing Comment

by:William Fulks
ID: 41955769
Thanks y'all!

Featured Post

Threat Trends for MSPs to Watch

See the findings.
Despite its humble beginnings, phishing has come a long way since those first crudely constructed emails. Today, phishing sites can appear and disappear in the length of a coffee break, and it takes more than a little know-how to keep your clients secure.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Determining the an SCCM package name from the Package ID
This article helps those who get the 0xc004d307 error when trying to rearm (reset the license) Office 2013 in a Virtual Desktop Infrastructure (VDI) and/or those trying to prep the master image for Microsoft Key Management (KMS) activation. (i.e.- C…
Finding and deleting duplicate (picture) files can be a time consuming task. My wife and I, our three kids and their families all share one dilemma: Managing our pictures. Between desktops, laptops, phones, tablets, and cameras; over the last decade…
In this video, viewers will be given step by step instructions on adjusting mouse, pointer and cursor visibility in Microsoft Windows 10. The video seeks to educate those who are struggling with the new Windows 10 Graphical User Interface. Change Cu…

691 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question