Is this possible?  Synchronization of a 160GB iTunes folder via VPN to another location and another RAID5 array.

Posted on 2006-05-21
Last Modified: 2010-04-27
DISCLAIEMR:  This is going to be for a record label owner, so stability and performance should be the focus over price.  I'd like options between 5-20K including drives.  If this is an unreasonable budget, advise me on what the least expensive proven solution would be.  If you know a reseller for a product, are a reseller for a solution, or have installed something like this before, I'll put you through your paces and you can have this client.

I have an end-user with an Apple notebook using iTunes off of a 500GB LaCie Firewire drive.  The current size of his collection is 160GB.  When he moved his library from his internal drive to an external one, the time that it took iTunes to begin playing a song increased by a few fractions of a second.  This delay was slight, and he had to live with it because of his need for increased space, but he can't live with anything near a 1 second delay between the time he presses play in iTunes and the time that sound is emitted from his speakes.  I mention this only to shed light on the quality of service we need to attain with the solution.

This LaCie external drive is only as reliable as the two drives in RAID0 inside it.  If one fails, his data is toast.  He backs up this drive to another two LaCie drives (one he keeps at home, one he keeps at the office) but he forgets to backup alot and this means he can lose a weeks worth of information.  In recent months, he's only maintained the backup at home, further illustrating how the backup being a manual one is a work-flow problem.

The client doesn't want to lug the firewire drive home each night in order to access his music at home.  I recommended that he get a RAID5 array at the office.  I don't know if this will be a full blown computer or just a NAS with hot-swap bays, but I want to get him off of a RAID0 solution that could die at any moment.  At home, he needs to get an identical RAID5 solution.  Then we need to synchronize these during the course of the day.  If he downloads 200MB of music in one day, it will need to be there when he gets home.  iTunes shouldn't be able to tell the difference between one RAID5 at the office and the other RAID5 at the house.  Now, if gigabit network isn't going to have the response time of Firewire 400 or 800, we can try to get a NAS that lets us plug into it with firewire and access as a removable HD.  I don't think that this will be needed, but I'm open to doing anything.  What can't happen is that he loses the quick "click and it plays" routine that he has right now.  This is step one.

Once we do this I'd like a way to automate the sync of a portable copy to take with him on the road if he ever needs it.  Some setup like this, the NAS's that we choose have a USB/Firewire port and we stick one of his LaCie's in there.  We run software that sync the drive to the NAS so that all three copies are up to date.  If he does work at home it sync to the office NAS and then sync to the removable HD attached to the NAS.  If he makes changes at the office they sync the the removable HD and the NAS at home.  If he goes on the road and downloads some music, when he plugs it back in, it sync to the office NAS and then that office NAS sync to the home NAS.

Is this possible or is there any other solution whereby he can access his music everywhere?  I was looking at X Drive, online storage, Windows 2003 DFS, Second Copy and other solutions and I'm open to anything.  It looks like a 300GB drive hosted somewhere on the internet isn't feasible because he's have latency and bandwidth problems.  He is on the road very VERY rarely (read: never).  We mainly are doing this to give him fault tolerance and save him the hassle of lugging a firewire drive around.  He does not have full use of his body, so this is something that is worthwhile for him and the company he owns.

I'm thinking of upgrading his internet to 4.5MBit synchronous service at the office if it would help.  I'd do this at his home if it was necessary as well.  I could even budget to get the $1399 6.0Mbit service.  I just need it to work great, not good, but great.  Twenty thousand is a lot to spend on an iTunes directory but it's got to work absolutely flawlessly for that amount, and there can't be something just as good at 8 or 10K.

Question by:jd_netcentral
    LVL 87

    Accepted Solution

    You do need a fast internet connection either way to get it to work properly, so 4.5MB sync noth sides is the first thing to do. when he has that, a tool like rsync can be used to synchronize the data.
    LVL 9

    Assisted Solution

    Wow... he must love his music... a lot !!

    I would suggest the following solution as a way forward. The one concern I have is latency for click and play speed. This would need to be checked but it does provide all other requirements. I am confident that it will be quick enough, but ANY external array will be slower than a direct attached disk.

    1. Laptop with Firewire drive for local storage. This is a Offline Copy of the central files.
    2. Server with DAS RAID-1 300GB drives running W2003 DFS at both locations
    3. Set up DFS replication between the two servers and virtualise the storage.
    4. Set up laptop to use offline files, pointing to the virtualised storage.
    5. You would probably need to upgrade the WAN connection but not sure if 4.5 - 6 would make that much difference.

    Initially, you would need to use your Lacie Drive to transfer data between home & office to reduce the impact on the WAN first time sync. From then on, you will be able to do synchronous updates between home & office.

    By using Offline Folders on the laptop, you retain your click and play speed and also provides you with mobility for road trips. Each time the laptop connects to the Home/Office network it will sync with the DFS storage.

    If you include a VPN at the office, you can also sync updates on the road if you download 200MB of new music while away and it will be ready when you get home. However, this would be VERY slow during the sync process.

    Cost would be 2 servers with storage (4 drives). Approx. 10K I suspect.

    Hope this helps
    LVL 3

    Assisted Solution

    I once worked at a high school as systemadmin and we became involved with an ADSL-pilot. The national media company would like to open it's archive of old movies to schools and offered a double 8 Mbit ADSL line and a SGI media-server for schools that would like to participate in the project. In turn, the teachers should integrate the media to their lessons.

    During the night, the server got synchronised over the ADSL-line to a server at the media-company.

    At daytime, the server was ready to stream fullscreen video over the 100 Mbit network to several clients at once. And we could use the (idle) ADSL-line for our internet-connectivity.

    So now back to your situation...
    I would suggest to place an small server at home connected to an 100 or 1000 Mbit network.  You can equip this server with a RAID-set if you want to. You can connect the home-server with the corperate network using a VPN- or SSH- tunnel and use a tool like rsync (as Rindi suggested) or DFS (IPKON's idea) to synchronise. This can be done during daytime as the user is only on one location at the time. So when he is downloading something, the synchronizing-job will be uploading. So they won't byte eachother. At least, not very havily. And if the upload is not fast enough, it does not really matter as the user will spend some time travelling.

    On this point, we've got two servers with an identical filesystem. Now we have to trick Itunes in thinking that both servers are the same as Itunes maintains it's own database. First assign a driveletter (for ex. X:) to the media-share and setup Itunes that it does not alter the filesystem in any way. (Otherwise Itunes will rearrange the files, changing filenames, locations and so on). Also make sure that ripped or downloaded (itunes) files are also stored on drive X. Now have Itunes run an inventory of the entire share. Once you're ready, disconnect from X and move to the other (home) network. Connect to the new fileserver, again assigning X: as driveletter. If everything is fine, you should be able to play the entire content.

    Of course you want to keep it user-friendly so you might want to create 3 seperate batchfiles 'home', 'work' and 'stop' that perform the right 'net use' commands.

    PS: Afaik, the LaCie drives can also run in RAID1 (mirroring) mode. So that would make an 250 Gb volume which is still enough for the 160 Gb media.

    Write Comment

    Please enter a first name

    Please enter a last name

    We will never share this with anyone.

    Featured Post

    How to run any project with ease

    Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
    - Combine task lists, docs, spreadsheets, and chat in one
    - View and edit from mobile/offline
    - Cut down on emails

    If you have a USB Drive that is not recognized by Windows the problem is usually that you have too many network drives or other drives that occupy all the drive letters D: E: or F: which is the normal drive letter of a usb drive. The way to correct …
    Ever notice how you can't use a new drive in Windows without having Windows assigning a Disk Signature?  Ever have a signature collision problem (especially with Virtual Machines?)  This article is intended to help you understand what's going on and…
    This video teaches viewers how to encrypt an external drive that requires a password to read and edit the drive. All tasks are done in Disk Utility. Plug in the external drive you wish to encrypt: Make sure all previous data on the drive has been …
    This tutorial will walk an individual through the process of installing the necessary services and then configuring a Windows Server 2012 system as an iSCSI target. To install the necessary roles, go to Server Manager, and select Add Roles and Featu…

    761 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    12 Experts available now in Live!

    Get 1:1 Help Now