Need Suggestion: Cloud-Based Storage With Media Streaming & Connection Options
Hey Guys -
I used to have an extensive media collection for which I've spent a ton of time and resources ripping from physical to digital media. Currently, I have 20+tb which is mostly comprised of my ripped Blu-Ray movies. All media is stored on my home "server" on a single 26tb+ partition which is made up of x10 physical drives (x9 SATA & x1 SSD) binded together using StableBit DrivePool (works great :)). Unfortunately, I still have more to go and each time I must add storage, it's costing more. For the last upgrade, I not only purchased another 4tb drive, but also a SATA card plus upgraded my power supply from a 650W to 1000W. Besides the cost, nothing I have is backed up! At the end of this post is my current setup.
I play all of my media using a few methods. Most local playing on TV is via a different dedicated computer running Kodi (XBMC). The second most used playback is Plex which sends the media out in many ways.
My Question
When thinking about the above situation recently, I thought it may be a good solution to purchase cloud-based storage and start moving media there. I'd need at least a few TBs to start, but would also need for the service to offer certain features to make it compatible with my setup. The need for local sync also wouldn't be required or used. Below is the feature list I'd need:
Required Features for Cloud Storage to Work
I considered a shared workspace which is basically like renting a VM in the cloud, but the plans for ones I've seen so far weren't storage focused therefore even their highest tiered plans only offered 1tb max.
So, any suggestions - or - alternative ideas for tackling this issue which would be cost effective?
Thanks!
Current Setup
Server
- Intel i7 3770K / 16gb RAM
- Windows 8.1 x64 Update 1 (Fully Patched) (Do bit of gaming and other stuff on it, too)
- x3 Partitions
- C:\ - 258gb SSD - OS & Primary App Installs
- D:\ - 26tb Drive Pool - Media & other storage (x1 SSD used for pool landing zone & x9 SATA drives)
- E:\ - 500gb Hybrid Drive - App/Game Installs & VMs
- Media - Most videos are in MKV if they are 720p or higher.Standard def are mostly in MP4. All media is organized, properly tagged, and named. Plex Server installed locally
Kodi System (Main TV)
- Intel i5 / 8gb RAM
- Windows 7 x64 (Use instead of dedicated linux install since I also run other apps including Plex Home Theater, EventGhost, and Flirc dongle
- Kodi 14.2 (Newest)
- Connects to media over ethernet via SMB / UNC Shares
Kodi System (Bedroom TV)
- Raspberry Pi 2
- 900MHz quad-core ARM processor / 1gb RAM
- OSMC RC2
- Connects to media over wi-fi via SMB / UNC Shares
Thanks again guys!
I used to have an extensive media collection for which I've spent a ton of time and resources ripping from physical to digital media. Currently, I have 20+tb which is mostly comprised of my ripped Blu-Ray movies. All media is stored on my home "server" on a single 26tb+ partition which is made up of x10 physical drives (x9 SATA & x1 SSD) binded together using StableBit DrivePool (works great :)). Unfortunately, I still have more to go and each time I must add storage, it's costing more. For the last upgrade, I not only purchased another 4tb drive, but also a SATA card plus upgraded my power supply from a 650W to 1000W. Besides the cost, nothing I have is backed up! At the end of this post is my current setup.
I play all of my media using a few methods. Most local playing on TV is via a different dedicated computer running Kodi (XBMC). The second most used playback is Plex which sends the media out in many ways.
My Question
When thinking about the above situation recently, I thought it may be a good solution to purchase cloud-based storage and start moving media there. I'd need at least a few TBs to start, but would also need for the service to offer certain features to make it compatible with my setup. The need for local sync also wouldn't be required or used. Below is the feature list I'd need:
Required Features for Cloud Storage to Work
Ability to have multiple TBs of space to purchase
Preferably multiple connection protocols. To get it to work with Plex, I'd probably have to be able to map a drive to it either using native Windows options (think you can map FTP or using other tools with r/w access. Kodi (XBMC) is much more flexible and offers many more options for media source over network
Not have to worry about licensing issues. I won't be sharing the data with anyone but don't want to chance DCMA issues even though I own the physical media
Optional - Offer some sort of Linux Desktop / interface where I can install / run Plex locally to media and/or Python scripts. This would actually be the ideal solution since it would manage media much faster - however - would also cost more I'm sure...
I considered a shared workspace which is basically like renting a VM in the cloud, but the plans for ones I've seen so far weren't storage focused therefore even their highest tiered plans only offered 1tb max.
So, any suggestions - or - alternative ideas for tackling this issue which would be cost effective?
Thanks!
Current Setup
Server
- Intel i7 3770K / 16gb RAM
- Windows 8.1 x64 Update 1 (Fully Patched) (Do bit of gaming and other stuff on it, too)
- x3 Partitions
- C:\ - 258gb SSD - OS & Primary App Installs
- D:\ - 26tb Drive Pool - Media & other storage (x1 SSD used for pool landing zone & x9 SATA drives)
- E:\ - 500gb Hybrid Drive - App/Game Installs & VMs
- Media - Most videos are in MKV if they are 720p or higher.Standard def are mostly in MP4. All media is organized, properly tagged, and named. Plex Server installed locally
Kodi System (Main TV)
- Intel i5 / 8gb RAM
- Windows 7 x64 (Use instead of dedicated linux install since I also run other apps including Plex Home Theater, EventGhost, and Flirc dongle
- Kodi 14.2 (Newest)
- Connects to media over ethernet via SMB / UNC Shares
Kodi System (Bedroom TV)
- Raspberry Pi 2
- 900MHz quad-core ARM processor / 1gb RAM
- OSMC RC2
- Connects to media over wi-fi via SMB / UNC Shares
Thanks again guys!
David
It is real simple. Use a flavor of open Solaris with the ZFS file system. Hot snapshot backup: online expansion; compression; dedup: mix and match any drive sizes; high data integrity and error tolerance; Google ZFS and you will be sold. Also no raid controller as the O/S does that
ASKER
Thanks for your reply...
I'm not sure I follow you. Have experience in many Linux distros, but never used Solaris before. So is what you are proposing is the setup to use once I establish / find an online storage solution or is an online storage solution? Could you please elaborate?
Thanks
I'm not sure I follow you. Have experience in many Linux distros, but never used Solaris before. So is what you are proposing is the setup to use once I establish / find an online storage solution or is an online storage solution? Could you please elaborate?
Thanks
You have a few hurdles to jump over before you can use any cloud based solution.
Cloud storage is really expensive for a private user. Amazon S3, which technically doesn't fit your specs as SMB is not supported, is today the best value for money, and it will cost you between $680 - $850 a month.
Amazon EBS provide block storage that you can mount from a running instance. To get a decent performance and capacity, you will need 2 SSD drives of 13TB each at a total of $2600 a month.
You also need a server instance, consumer windows is usually not supported so you need a Linux or Windows 2012 DC edition. A Large (m3.xlarge) Linux instance with 15GB ram is $204 a month.
Thinking about getting this setup from another cloud service?
You will find that you can get significant discounts (even at Amazon), but you will have to commit for 12, 24 or even 36 months.
Try Leaseweb - they offer leased servers with a month by month option, and have an online calculator for all of your options.
Then there is the question of performance....
SMB/CIFS is a very "chatty" protocol, and doesn't like non LAN connections.
Don't expect the same playback quality you have today. As a workaround, you can setup a Wowza or other streaming server in the cloud where your media library is, and have your own private Mytube.
Cloud storage is really expensive for a private user. Amazon S3, which technically doesn't fit your specs as SMB is not supported, is today the best value for money, and it will cost you between $680 - $850 a month.
Amazon EBS provide block storage that you can mount from a running instance. To get a decent performance and capacity, you will need 2 SSD drives of 13TB each at a total of $2600 a month.
You also need a server instance, consumer windows is usually not supported so you need a Linux or Windows 2012 DC edition. A Large (m3.xlarge) Linux instance with 15GB ram is $204 a month.
Thinking about getting this setup from another cloud service?
You will find that you can get significant discounts (even at Amazon), but you will have to commit for 12, 24 or even 36 months.
Try Leaseweb - they offer leased servers with a month by month option, and have an online calculator for all of your options.
Then there is the question of performance....
SMB/CIFS is a very "chatty" protocol, and doesn't like non LAN connections.
Don't expect the same playback quality you have today. As a workaround, you can setup a Wowza or other streaming server in the cloud where your media library is, and have your own private Mytube.
No, my suggestion is to repurpose your server with this O/S, but with a decent SAS/SATA JBOD controller. A cloud solution isn't for you due to the cost and bandwidth you'll need to stream video. BUT purchase a online backup solution that has unlimited capacity to serve as your disaster recovery repository.
Look at napp-it for a GUI appliance overlay that will make setup/configuration easy. You'll have NFS, SMB (windows filesystems), even apple time machine and iSCSI if you want.
Look at napp-it for a GUI appliance overlay that will make setup/configuration easy. You'll have NFS, SMB (windows filesystems), even apple time machine and iSCSI if you want.
ASKER
I'll try these out and post back - Thanks
This question needs an answer!
Become an EE member today
7 DAY FREE TRIALMembers can start a 7-Day Free trial then enjoy unlimited access to the platform.
View membership options
or
Learn why we charge membership fees
We get it - no one likes a content blocker. Take one extra minute and find out why we block content.