Solved

Any feedback about desktop grid storage (storage cloud on desktop) ?

Posted on 2011-09-16
15
540 Views
Last Modified: 2013-11-14
Dear experts,

I was wondering if any of you had plan or are already using some storage solution on a desktop grid ?

Desktop grid are known since the SETI project but this project was a computational desktop grid (CPU share) where I am looking for a storage one (HDD share).

Any experience related to a "centralized use" based on your managed desktops available storage space and availability ?

Thanks for reading anyway
Best regards
0
Comment
Question by:BigSchmuh
  • 7
  • 5
  • 2
  • +1
15 Comments
 
LVL 70

Expert Comment

by:KCTS
ID: 36552419
Are you talking about something like DropBox ? http://www.dropbox.com/
0
 
LVL 18

Author Comment

by:BigSchmuh
ID: 36553227
No, I am not looking for a centralized online unsafe storage like Dropbox, neither for a centralized backup like BackBlaze.
I am looking for a distributed safe long-term storage, I thought of this storage to be distributed on a Desktop grid.
0
 
LVL 17

Expert Comment

by:Gerald Connolly
ID: 36554278
Are you looking to use the spare storage on a bunch of existing workstations or just that you need some storage on your LAN.

If its is the former then it will not be reliable, if its the latter then just buy some NAS boxes
0
Optimizing Cloud Backup for Low Bandwidth

With cloud storage prices going down a growing number of SMBs start to use it for backup storage. Unfortunately, business data volume rarely fits the average Internet speed. This article provides an overview of main Internet speed challenges and reveals backup best practices.

 
LVL 18

Author Comment

by:BigSchmuh
ID: 36554757
I am looking to ise spare storage on workstation...and yes it will be more reliable than a centralized solution. That's exactly what Internet was designed for...
0
 
LVL 17

Expert Comment

by:Gerald Connolly
ID: 36554898
Well even if there were some software out there to allow you to harvest the spare capacity on workstations it actually less reliable than using centralised storage, mainly because you do not have any control over it. What happens if the user of the workstation decides to reboot, or switch it off and who would be responsible for virus checking and the security of the data stored? Who is backing it up etc etc.

And using a NAS solution doesnt necessarily need to be centralised.
0
 
LVL 18

Author Comment

by:BigSchmuh
ID: 36554918
Desktop grid storage obviously requires 3 copy minimum, 5 may be more than enough if the dispatch allows to avoid using correlated nodes.
 
Do you know any software able to dispatch on many NAS ?
0
 
LVL 17

Accepted Solution

by:
Gerald Connolly earned 250 total points
ID: 36555664
Although there are several distributed NAS systems available such as Panasas and Gluster, none that i am aware of will make use of the spare capacity on a bunch of workstations.

Even if you had the physical security of multiple copies of the data which would seriously restrict the available space, you would still have the problem of the security of the data.
0
 
LVL 18

Author Comment

by:BigSchmuh
ID: 36556315
Data security is not a problem :
-all files should be splitted in small blocs and send all around using bittorrent capability
-all files are encrypted with the main catalog unaware of their key

Restricting the space ? 100GB per workstation and 1000 workstations means about 100TB which are 20TB useable...
0
 
LVL 17

Expert Comment

by:Gerald Connolly
ID: 36556373
You seemed to have got all your own answers!

What software are you going to use to run this farm?
0
 
LVL 12

Assisted Solution

by:marcustech
marcustech earned 250 total points
ID: 36556477
Sounds like a nightmare!  I would expect that the overhead and duplication on this sort of distributed file system would run the network ragged - have you got any ideas on what the overhead will be like?
0
 
LVL 18

Author Comment

by:BigSchmuh
ID: 36557213
I don't have any package ready for that...I am wondering if all those possible features (bittorrent, encryption, catalog, ...) make "storage desktop grid" some sense or if this is not a good idea at all.

Regarding the network, P2P exchange aren't that bad and appears "flat" on the grid (vs centralized exchanges). No new bottleneck are expected, it takes the available bandwidth and that's all.
0
 
LVL 17

Expert Comment

by:Gerald Connolly
ID: 36557318
@marcustech, i agree with that, and it is going to be real difficult to manage as well. Its probably going to be more trouble than its worth.

I know it sounds like a good idea, but the fact that there isnt any software around to actually do this might be a clue!

Another problem might be that if a users workstation starts slowing down because of all the extra disk and network traffic, i suspect that the user will start killing processes to stop it happening! Once the user community find out what's going on you might have a revolt on your hands!  :-)
0
 
LVL 12

Assisted Solution

by:marcustech
marcustech earned 250 total points
ID: 36558862
Dropbox use what they call 'Lan Sync' when you have shared folders in the cloud they replicate changes peer to peer between workstations on the same subnet - http://www.dropbox.com/help/137  That sounds similar to what you're after, but there's not a lot of detail on how it's actually implemented.
0
 
LVL 18

Author Comment

by:BigSchmuh
ID: 36573442
Any more feedbacks, bottlenecks, open-source project or software that you could suggest ?
0
 
LVL 18

Author Closing Comment

by:BigSchmuh
ID: 36598068
Thanks for trying !
0

Featured Post

Enterprise Mobility and BYOD For Dummies

Like “For Dummies” books, you can read this in whatever order you choose and learn about mobility and BYOD; and how to put a competitive mobile infrastructure in place. Developed for SMBs and large enterprises alike, you will find helpful use cases, planning, and implementation.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Ever notice how you can't use a new drive in Windows without having Windows assigning a Disk Signature?  Ever have a signature collision problem (especially with Virtual Machines?)  This article is intended to help you understand what's going on and…
Create your own, high-performance VM backup appliance by installing NAKIVO Backup & Replication directly onto a Synology NAS!
This video teaches viewers how to encrypt an external drive that requires a password to read and edit the drive. All tasks are done in Disk Utility. Plug in the external drive you wish to encrypt: Make sure all previous data on the drive has been …
This Micro Tutorial will teach you how to reformat your flash drive. Sometimes your flash drive may have issues carrying files so this will completely restore it to manufacturing settings. Make sure to backup all files before reformatting. This w…

789 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question