Solved

Any feedback about desktop grid storage (storage cloud on desktop) ?

Posted on 2011-09-16
15
533 Views
Last Modified: 2013-11-14
Dear experts,

I was wondering if any of you had plan or are already using some storage solution on a desktop grid ?

Desktop grid are known since the SETI project but this project was a computational desktop grid (CPU share) where I am looking for a storage one (HDD share).

Any experience related to a "centralized use" based on your managed desktops available storage space and availability ?

Thanks for reading anyway
Best regards
0
Comment
Question by:BigSchmuh
  • 7
  • 5
  • 2
  • +1
15 Comments
 
LVL 70

Expert Comment

by:KCTS
ID: 36552419
Are you talking about something like DropBox ? http://www.dropbox.com/
0
 
LVL 18

Author Comment

by:BigSchmuh
ID: 36553227
No, I am not looking for a centralized online unsafe storage like Dropbox, neither for a centralized backup like BackBlaze.
I am looking for a distributed safe long-term storage, I thought of this storage to be distributed on a Desktop grid.
0
 
LVL 16

Expert Comment

by:Gerald Connolly
ID: 36554278
Are you looking to use the spare storage on a bunch of existing workstations or just that you need some storage on your LAN.

If its is the former then it will not be reliable, if its the latter then just buy some NAS boxes
0
 
LVL 18

Author Comment

by:BigSchmuh
ID: 36554757
I am looking to ise spare storage on workstation...and yes it will be more reliable than a centralized solution. That's exactly what Internet was designed for...
0
 
LVL 16

Expert Comment

by:Gerald Connolly
ID: 36554898
Well even if there were some software out there to allow you to harvest the spare capacity on workstations it actually less reliable than using centralised storage, mainly because you do not have any control over it. What happens if the user of the workstation decides to reboot, or switch it off and who would be responsible for virus checking and the security of the data stored? Who is backing it up etc etc.

And using a NAS solution doesnt necessarily need to be centralised.
0
 
LVL 18

Author Comment

by:BigSchmuh
ID: 36554918
Desktop grid storage obviously requires 3 copy minimum, 5 may be more than enough if the dispatch allows to avoid using correlated nodes.
 
Do you know any software able to dispatch on many NAS ?
0
 
LVL 16

Accepted Solution

by:
Gerald Connolly earned 250 total points
ID: 36555664
Although there are several distributed NAS systems available such as Panasas and Gluster, none that i am aware of will make use of the spare capacity on a bunch of workstations.

Even if you had the physical security of multiple copies of the data which would seriously restrict the available space, you would still have the problem of the security of the data.
0
IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

 
LVL 18

Author Comment

by:BigSchmuh
ID: 36556315
Data security is not a problem :
-all files should be splitted in small blocs and send all around using bittorrent capability
-all files are encrypted with the main catalog unaware of their key

Restricting the space ? 100GB per workstation and 1000 workstations means about 100TB which are 20TB useable...
0
 
LVL 16

Expert Comment

by:Gerald Connolly
ID: 36556373
You seemed to have got all your own answers!

What software are you going to use to run this farm?
0
 
LVL 12

Assisted Solution

by:marcustech
marcustech earned 250 total points
ID: 36556477
Sounds like a nightmare!  I would expect that the overhead and duplication on this sort of distributed file system would run the network ragged - have you got any ideas on what the overhead will be like?
0
 
LVL 18

Author Comment

by:BigSchmuh
ID: 36557213
I don't have any package ready for that...I am wondering if all those possible features (bittorrent, encryption, catalog, ...) make "storage desktop grid" some sense or if this is not a good idea at all.

Regarding the network, P2P exchange aren't that bad and appears "flat" on the grid (vs centralized exchanges). No new bottleneck are expected, it takes the available bandwidth and that's all.
0
 
LVL 16

Expert Comment

by:Gerald Connolly
ID: 36557318
@marcustech, i agree with that, and it is going to be real difficult to manage as well. Its probably going to be more trouble than its worth.

I know it sounds like a good idea, but the fact that there isnt any software around to actually do this might be a clue!

Another problem might be that if a users workstation starts slowing down because of all the extra disk and network traffic, i suspect that the user will start killing processes to stop it happening! Once the user community find out what's going on you might have a revolt on your hands!  :-)
0
 
LVL 12

Assisted Solution

by:marcustech
marcustech earned 250 total points
ID: 36558862
Dropbox use what they call 'Lan Sync' when you have shared folders in the cloud they replicate changes peer to peer between workstations on the same subnet - http://www.dropbox.com/help/137  That sounds similar to what you're after, but there's not a lot of detail on how it's actually implemented.
0
 
LVL 18

Author Comment

by:BigSchmuh
ID: 36573442
Any more feedbacks, bottlenecks, open-source project or software that you could suggest ?
0
 
LVL 18

Author Closing Comment

by:BigSchmuh
ID: 36598068
Thanks for trying !
0

Featured Post

Top 6 Sources for Identifying Threat Actor TTPs

Understanding your enemy is essential. These six sources will help you identify the most popular threat actor tactics, techniques, and procedures (TTPs).

Join & Write a Comment

Sometimes the best way to deal with an infected computer is to boot from external media and run your tools from there.  The reason you may wish to do this really depends on the infection.  Some malware is so recalcitrant that no matter what you do i…
AWS Glacier is Amazons cheapest storage option and is their answer to a ‘Cold’ storage service.  Customers primarily use this service for archival purposes and storage of infrastructure backups.  Its unlimited storage potential and low storage cost …
This tutorial will walk an individual through the process of installing the necessary services and then configuring a Windows Server 2012 system as an iSCSI target. To install the necessary roles, go to Server Manager, and select Add Roles and Featu…
This Micro Tutorial will teach you how to reformat your flash drive. Sometimes your flash drive may have issues carrying files so this will completely restore it to manufacturing settings. Make sure to backup all files before reformatting. This w…

708 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

18 Experts available now in Live!

Get 1:1 Help Now