[Last Call] Learn about multicloud storage options and how to improve your company's cloud strategy. Register Now


Best practices for sharing large amounts of data

Posted on 2016-12-08
Medium Priority
Last Modified: 2017-01-10
I work for a city government and our police department uses video systems that often result in large video files that need to be sent to district attorney's offices, other lawyers, etc. They had previously been burning data DVD's but now with HD video they're running into files that are larger than DVD's 4.7 gb limit. The PD wanted to invest in a bunch of blu-ray burners and we advised against it since most lawyers and other offices most likely won't have blu-ray players to read the disks, not to mention that optical drives are more or less obsolete or soon will be. Other suggestions have been to use flash drives (purchased in bulk for cheap), setup a local FTP site and let them maintain logins for it, cloud sharing, etc.

So my question to throw out there to all you experts is this - What would you consider to be the best practice for sharing large data files? Some individual files may go as large as 20 gigs in size. I'm looking for pros/cons, cost estimates, etc.
Question by:William Fulks
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
LVL 24

Assisted Solution

by:Tom Cieslik
Tom Cieslik earned 332 total points
ID: 41918396
You can convert video filed to most efficient mp4 format and share using FTP server or create VPN tunnel and let them download files from share folder where you'll sent them link to.
It's always problem with that big files
Flash drives will become expensive if you'll calculate costs per year.
LVL 29

Accepted Solution

Dr. Klahn earned 1004 total points
ID: 41918401
Given that Blu-Ray players are selling from $30 to $80 now, I would not consider that a block to using Blu-Ray discs.  That kind of money is petty cash these days.

Side note:  It's going to be a long time before optical drives are obsolete while Microsoft distributes Windows and Windows software on them.

The public should not be footing the bill for this.  "Reasonable and customary charges" are made for FOIA and other document copying.  Video should be no different.  If someone wants 20 GB of video, sock him for a 32 GB USB drive at cost plus 10%, plus tech time.  Everybody has a budget.

Assisted Solution

rojosho earned 332 total points
ID: 41919764
Hey William,

Good comments from my colleagues.  My first question (OK, concern) is how secure do these files have to be?  Flash Drives, DVD and Blu-Ray are only the latest form of what we once called 'Sneakernet' and it is still as unsecured today as it was back in the 80's and 90's.  So if security is an issue, then setting up a VPN to a ftp server as suggested by Tom, would be the preferred way to ensure that the persons with the right 'need to know' are allowed (Key word) access to this information.
> We have to remember that we are not just discussing transporting files/data, but that these files are sensitive and may be covered under the HIPAA and Patriots Act guidelines.
> As will all removal media, the data/files/needs **always** out grows the current technology - think about... 8" Floopy to 3.5", to CD, to DVD, to Blu-Ray, to Flash Drive.... "the beat goes on"....
> With that said, that should give you the justification to budget the true equipment and configuration that your department will need.

Forgive me, my degree is in ISA....I cant help my self.

Get your Conversational Ransomware Defense e‑book

This e-book gives you an insight into the ransomware threat and reviews the fundamentals of top-notch ransomware preparedness and recovery. To help you protect yourself and your organization. The initial infection may be inevitable, so the best protection is to be fully prepared.

LVL 17

Assisted Solution

by:Gerald Connolly
Gerald Connolly earned 332 total points
ID: 41920268
if you stick with the physical route (aka sneakernet) using USB flash drives dont forget to encrypt the data and send the key via an alternative route (eg email)

NB if you are going to use an FTP site use encryption as well, but be aware of the time it may take for you to upload and the recipient to download e.g a 20GB file over a 100Mb/s link will probably take 2-3 hours to upload

Expert Comment

ID: 41920717

EXCELLENT points - Encryption would be a must if sneaker-net is one of your options - but remember that 85% of all security issues are caused by human error.

LVL 33

Expert Comment

ID: 41921091
Definitely a case for AWS.
If you are from US, AWS has a gov region that covers your regulation needs. Even without the gov region, all Hipaa and SOC and ISO are well addressed.
Technically speaking, you will upload the relevant files to a secure S3 bucket, either all of them or on demand, and create authorised links for playback or for download.
It will be the most economical way, with proper security in place. I suggest that you talk with an Amazon solutions architect or an amazon partner about it.
LVL 15

Author Comment

by:William Fulks
ID: 41921418
Thanks everyone for the input. I'll let this hang out a couple more days to see if any more replies come in. This whole thing has turned into another discussion (at work) about laws regarding evidence storage, etc. I was told by one person that evidence in murder cases must be stored for 75 years, other cases have different periods of time, etc. Interesting stuff to think about from an IT side as more stuff goes totally digital.
LVL 15

Author Closing Comment

by:William Fulks
ID: 41955769
Thanks y'all!

Featured Post

Understanding Web Applications

Without even knowing it, most of us are using web applications on a daily basis. Gmail and Yahoo email, Twitter, Facebook, and eBay are used by most of us daily—and they are web applications. We often confuse these web applications tools for websites.  So, what is the difference?

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

When asking a question in a forum or creating documentation, screenshots are vital tools that can convey a lot more information and save you and your reader a lot of time
Why do some people recommend buying business VoIP from an ISP? What are the benefits to my company? What are the costs?
With the advent of Windows 10, Microsoft is pushing a Get Windows 10 icon into the notification area (system tray) of qualifying computers. There are many reasons for wanting to remove this icon. This two-part Experts Exchange video Micro Tutorial s…
Windows 10 is mostly good. However the one thing that annoys me is how many clicks you have to do to dial a VPN connection. You have to go to settings from the start menu, (2 clicks), Network and Internet (1 click), Click VPN (another click) then fi…

650 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question