Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people, just like you, are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
Solved

Best practices for sharing large amounts of data

Posted on 2016-12-08
9
95 Views
Last Modified: 2017-01-10
I work for a city government and our police department uses video systems that often result in large video files that need to be sent to district attorney's offices, other lawyers, etc. They had previously been burning data DVD's but now with HD video they're running into files that are larger than DVD's 4.7 gb limit. The PD wanted to invest in a bunch of blu-ray burners and we advised against it since most lawyers and other offices most likely won't have blu-ray players to read the disks, not to mention that optical drives are more or less obsolete or soon will be. Other suggestions have been to use flash drives (purchased in bulk for cheap), setup a local FTP site and let them maintain logins for it, cloud sharing, etc.

So my question to throw out there to all you experts is this - What would you consider to be the best practice for sharing large data files? Some individual files may go as large as 20 gigs in size. I'm looking for pros/cons, cost estimates, etc.
0
Comment
Question by:William Fulks
9 Comments
 
LVL 20

Assisted Solution

by:Tom Cieslik
Tom Cieslik earned 83 total points
ID: 41918396
You can convert video filed to most efficient mp4 format and share using FTP server or create VPN tunnel and let them download files from share folder where you'll sent them link to.
It's always problem with that big files
Flash drives will become expensive if you'll calculate costs per year.
0
 
LVL 26

Accepted Solution

by:
Dr. Klahn earned 251 total points
ID: 41918401
Given that Blu-Ray players are selling from $30 to $80 now, I would not consider that a block to using Blu-Ray discs.  That kind of money is petty cash these days.

Side note:  It's going to be a long time before optical drives are obsolete while Microsoft distributes Windows and Windows software on them.

The public should not be footing the bill for this.  "Reasonable and customary charges" are made for FOIA and other document copying.  Video should be no different.  If someone wants 20 GB of video, sock him for a 32 GB USB drive at cost plus 10%, plus tech time.  Everybody has a budget.
1
 
LVL 7

Assisted Solution

by:rojosho
rojosho earned 83 total points
ID: 41919764
Hey William,

Good comments from my colleagues.  My first question (OK, concern) is how secure do these files have to be?  Flash Drives, DVD and Blu-Ray are only the latest form of what we once called 'Sneakernet' and it is still as unsecured today as it was back in the 80's and 90's.  So if security is an issue, then setting up a VPN to a ftp server as suggested by Tom, would be the preferred way to ensure that the persons with the right 'need to know' are allowed (Key word) access to this information.
> We have to remember that we are not just discussing transporting files/data, but that these files are sensitive and may be covered under the HIPAA and Patriots Act guidelines.
> As will all removal media, the data/files/needs **always** out grows the current technology - think about... 8" Floopy to 3.5", to CD, to DVD, to Blu-Ray, to Flash Drive.... "the beat goes on"....
> With that said, that should give you the justification to budget the true equipment and configuration that your department will need.

Forgive me, my degree is in ISA....I cant help my self.

Rojosho
0
U.S. Department of Agriculture and Acronis Access

With the new era of mobile computing, smartphones and tablets, wireless communications and cloud services, the USDA sought to take advantage of a mobilized workforce and the blurring lines between personal and corporate computing resources.

 
LVL 17

Assisted Solution

by:Gerald Connolly
Gerald Connolly earned 83 total points
ID: 41920268
if you stick with the physical route (aka sneakernet) using USB flash drives dont forget to encrypt the data and send the key via an alternative route (eg email)

NB if you are going to use an FTP site use encryption as well, but be aware of the time it may take for you to upload and the recipient to download e.g a 20GB file over a 100Mb/s link will probably take 2-3 hours to upload
0
 
LVL 7

Expert Comment

by:rojosho
ID: 41920717
Gerald,

EXCELLENT points - Encryption would be a must if sneaker-net is one of your options - but remember that 85% of all security issues are caused by human error.

Rojosho
0
 
LVL 33

Expert Comment

by:shalomc
ID: 41921091
Definitely a case for AWS.
If you are from US, AWS has a gov region that covers your regulation needs. Even without the gov region, all Hipaa and SOC and ISO are well addressed.
Technically speaking, you will upload the relevant files to a secure S3 bucket, either all of them or on demand, and create authorised links for playback or for download.
It will be the most economical way, with proper security in place. I suggest that you talk with an Amazon solutions architect or an amazon partner about it.
0
 
LVL 14

Author Comment

by:William Fulks
ID: 41921418
Thanks everyone for the input. I'll let this hang out a couple more days to see if any more replies come in. This whole thing has turned into another discussion (at work) about laws regarding evidence storage, etc. I was told by one person that evidence in murder cases must be stored for 75 years, other cases have different periods of time, etc. Interesting stuff to think about from an IT side as more stuff goes totally digital.
0
 
LVL 14

Author Closing Comment

by:William Fulks
ID: 41955769
Thanks y'all!
0

Featured Post

Free Tool: Port Scanner

Check which ports are open to the outside world. Helps make sure that your firewall rules are working as intended.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article aims to explain the working of CircularLogArchiver. This tool was designed to solve the buildup of log file in cases where systems do not support circular logging or where circular logging is not enabled
When we purchase storage, we typically are advertised storage of 500GB, 1TB, 2TB and so on. However, when you actually install it into your computer, your 500GB HDD will actually show up as 465GB. Why? It has to do with the way people and computers…
This Micro Tutorial will teach you how to reformat your flash drive. Sometimes your flash drive may have issues carrying files so this will completely restore it to manufacturing settings. Make sure to backup all files before reformatting. This w…
This is an introductory video for CloudBerry Managed Backup. You will learn how to sign up with the service and get started in a few minutes.

791 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question