Best practices for sharing large amounts of data

Posted on 2016-12-08
Last Modified: 2017-01-10
I work for a city government and our police department uses video systems that often result in large video files that need to be sent to district attorney's offices, other lawyers, etc. They had previously been burning data DVD's but now with HD video they're running into files that are larger than DVD's 4.7 gb limit. The PD wanted to invest in a bunch of blu-ray burners and we advised against it since most lawyers and other offices most likely won't have blu-ray players to read the disks, not to mention that optical drives are more or less obsolete or soon will be. Other suggestions have been to use flash drives (purchased in bulk for cheap), setup a local FTP site and let them maintain logins for it, cloud sharing, etc.

So my question to throw out there to all you experts is this - What would you consider to be the best practice for sharing large data files? Some individual files may go as large as 20 gigs in size. I'm looking for pros/cons, cost estimates, etc.
Question by:William Fulks
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
LVL 22

Assisted Solution

by:Tom Cieslik
Tom Cieslik earned 83 total points
ID: 41918396
You can convert video filed to most efficient mp4 format and share using FTP server or create VPN tunnel and let them download files from share folder where you'll sent them link to.
It's always problem with that big files
Flash drives will become expensive if you'll calculate costs per year.
LVL 27

Accepted Solution

Dr. Klahn earned 251 total points
ID: 41918401
Given that Blu-Ray players are selling from $30 to $80 now, I would not consider that a block to using Blu-Ray discs.  That kind of money is petty cash these days.

Side note:  It's going to be a long time before optical drives are obsolete while Microsoft distributes Windows and Windows software on them.

The public should not be footing the bill for this.  "Reasonable and customary charges" are made for FOIA and other document copying.  Video should be no different.  If someone wants 20 GB of video, sock him for a 32 GB USB drive at cost plus 10%, plus tech time.  Everybody has a budget.

Assisted Solution

rojosho earned 83 total points
ID: 41919764
Hey William,

Good comments from my colleagues.  My first question (OK, concern) is how secure do these files have to be?  Flash Drives, DVD and Blu-Ray are only the latest form of what we once called 'Sneakernet' and it is still as unsecured today as it was back in the 80's and 90's.  So if security is an issue, then setting up a VPN to a ftp server as suggested by Tom, would be the preferred way to ensure that the persons with the right 'need to know' are allowed (Key word) access to this information.
> We have to remember that we are not just discussing transporting files/data, but that these files are sensitive and may be covered under the HIPAA and Patriots Act guidelines.
> As will all removal media, the data/files/needs **always** out grows the current technology - think about... 8" Floopy to 3.5", to CD, to DVD, to Blu-Ray, to Flash Drive.... "the beat goes on"....
> With that said, that should give you the justification to budget the true equipment and configuration that your department will need.

Forgive me, my degree is in ISA....I cant help my self.

Back Up Your Microsoft Windows Server®

Back up all your Microsoft Windows Server – on-premises, in remote locations, in private and hybrid clouds. Your entire Windows Server will be backed up in one easy step with patented, block-level disk imaging. We achieve RTOs (recovery time objectives) as low as 15 seconds.

LVL 17

Assisted Solution

by:Gerald Connolly
Gerald Connolly earned 83 total points
ID: 41920268
if you stick with the physical route (aka sneakernet) using USB flash drives dont forget to encrypt the data and send the key via an alternative route (eg email)

NB if you are going to use an FTP site use encryption as well, but be aware of the time it may take for you to upload and the recipient to download e.g a 20GB file over a 100Mb/s link will probably take 2-3 hours to upload

Expert Comment

ID: 41920717

EXCELLENT points - Encryption would be a must if sneaker-net is one of your options - but remember that 85% of all security issues are caused by human error.

LVL 33

Expert Comment

ID: 41921091
Definitely a case for AWS.
If you are from US, AWS has a gov region that covers your regulation needs. Even without the gov region, all Hipaa and SOC and ISO are well addressed.
Technically speaking, you will upload the relevant files to a secure S3 bucket, either all of them or on demand, and create authorised links for playback or for download.
It will be the most economical way, with proper security in place. I suggest that you talk with an Amazon solutions architect or an amazon partner about it.
LVL 14

Author Comment

by:William Fulks
ID: 41921418
Thanks everyone for the input. I'll let this hang out a couple more days to see if any more replies come in. This whole thing has turned into another discussion (at work) about laws regarding evidence storage, etc. I was told by one person that evidence in murder cases must be stored for 75 years, other cases have different periods of time, etc. Interesting stuff to think about from an IT side as more stuff goes totally digital.
LVL 14

Author Closing Comment

by:William Fulks
ID: 41955769
Thanks y'all!

Featured Post

Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Concerto Cloud Services, a provider of fully managed private, public and hybrid cloud solutions, announced today it was named to the 20 Coolest Cloud Infrastructure Vendors Of The 2017 Cloud  (…
Learn how to PXE Boot both BIOS & UEFI machines with DHCP Policies and Custom Vendor Classes
This video teaches viewers how to encrypt an external drive that requires a password to read and edit the drive. All tasks are done in Disk Utility. Plug in the external drive you wish to encrypt: Make sure all previous data on the drive has been …
This is used to tweak the memory usage for your computer, it is used for servers more so than workstations but just be careful editing registry settings as it may cause irreversible results. I hold no responsibility for anything you do to the regist…

726 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question