funaroma
asked on
UNISON to Cygwin File/Folder/Path too long
We're using UNISON to back up a Windows/cygwin server using SSH to a FreeBSD machine. One folder has many subfolders and file names that are too long. Changing file and folder names is not an option. UNISON fails to mirror this folder, with an error indicating that the file name is too long. Is there a way to use UNISON to back up this directory, but have it use the Windows 8.3 filenames instead of the long ones the users have created?
gheist
Is there any problem with FreeBSD ???
ASKER
As usual, I don't understand your question or what direction you are trying to take. Can you be more specific? Are you asking about the destination FreeBSD server? Are you asking why the source server is not FreeBSD? Do you have anything useful to offer?
I suppose you could zip/rar the folders before backing them up with unison; you could probably script the whole thing.
Where do you get the error about long filenames when using unison? on the windows box or on the freebsd box?
Where do you get the error about long filenames when using unison? on the windows box or on the freebsd box?
ASKER
The error occurs during the cron job on the FreeBSD server, which attempts to pull the data from the remote windows server.
I did think about zipping the entire folder in question before sending it over the wire, but then I lose the benefits of the unison sync process, where only the files/parts of the files that have changed are sent over. If I zip the entire directory, the zip file would be sent each and every time the cron job runs, and the zip file is likely to be at least a couple gigs in size, if not larger.
I'm a little stumped about this filename/path size limitation, and find it difficult to accept that there's no other way around it. It's as if Windows can do something that *nix cannot, and that's contrary to everything i've heard about *nix solutions. Then again, I'm not sure exactly where the limitation lies, so I'm probably just talking out my arse.
Any other ideas? I'm upping the points on this.
I did think about zipping the entire folder in question before sending it over the wire, but then I lose the benefits of the unison sync process, where only the files/parts of the files that have changed are sent over. If I zip the entire directory, the zip file would be sent each and every time the cron job runs, and the zip file is likely to be at least a couple gigs in size, if not larger.
I'm a little stumped about this filename/path size limitation, and find it difficult to accept that there's no other way around it. It's as if Windows can do something that *nix cannot, and that's contrary to everything i've heard about *nix solutions. Then again, I'm not sure exactly where the limitation lies, so I'm probably just talking out my arse.
Any other ideas? I'm upping the points on this.
Windows also has a filename length limitation -- i think it's 128 characters, but i'm not sure about that.
What's the local directory that the file is being saved to?
What's the local directory that the file is being saved to?
ASKER
the local directory is a separate hard drive in FreeBSD, i.e.
/DriveName/MirrorFolder/Da
If Windows had such a filename/path limitation, how could the users have created, and continue to create, files and folders deep within the current path?
I read that Cygwin seems to have the 260 character limit... I really hope there's a way around it, but it's starting to look like there is not.
/DriveName/MirrorFolder/Da
If Windows had such a filename/path limitation, how could the users have created, and continue to create, files and folders deep within the current path?
I read that Cygwin seems to have the 260 character limit... I really hope there's a way around it, but it's starting to look like there is not.
What about you build with Interix/SFU ???
> If Windows had such a filename/path limitation
It takes full or relative path, and only some functions are subject to this limit.
> If Windows had such a filename/path limitation
It takes full or relative path, and only some functions are subject to this limit.
ASKER
How does Microsoft's SFU differ from Cywin? Are there major caveats or reasons not to go with SFU?
As with all M$ products it requires multiple reboots when installed, and seeds FUD that cron is insecure ...
ASKER
LOL...
Are you saying it requires reboots DURING the install, or that after installation it is unstable enough to require reboots over time?
It may work for what I'm using it for, so I'll take a look. Do you have any links to the security issue(s) that may be prevalent in this install, and do properly functioning firewalls/packet fitlers prevent compromise?
Are you saying it requires reboots DURING the install, or that after installation it is unstable enough to require reboots over time?
It may work for what I'm using it for, so I'll take a look. Do you have any links to the security issue(s) that may be prevalent in this install, and do properly functioning firewalls/packet fitlers prevent compromise?
It requires one reboot after install.
First it asks you to insert raw password file, then it simply shows you the message that UNIX cron scheduler is insecure and thus disabled, no need to dig passwd file or worry about messages. Interix is based on OpenBSD somewhere around 3.1, which never used always vulnerable vixie-cron
I suggest you use it only to compile.
Same sould be done with latest cygwin toolset.
If one of resulting binaries works like you need - use it. Otherwise stay with official build
First it asks you to insert raw password file, then it simply shows you the message that UNIX cron scheduler is insecure and thus disabled, no need to dig passwd file or worry about messages. Interix is based on OpenBSD somewhere around 3.1, which never used always vulnerable vixie-cron
I suggest you use it only to compile.
Same sould be done with latest cygwin toolset.
If one of resulting binaries works like you need - use it. Otherwise stay with official build
Depends on whether your problem is FILEname too long, or PATHname too long. How long are the actual FILEnames you're synchronizing?
I suspect the issue is that you're backing up from a folder that's close to the root and you have verrry deeply nested subdirectories. More info, please - the actual output of an error, complete with the full pathname of the file it's harfing on, would be most helpful.
I suspect the issue is that you're backing up from a folder that's close to the root and you have verrry deeply nested subdirectories. More info, please - the actual output of an error, complete with the full pathname of the file it's harfing on, would be most helpful.
dunno; funaroma just sort of quit responding to T/S requests.
ASKER
Ack, I'm sorry.... no, we have not resolved this yet and I didn't realize there had been more activity on this. I think I've been missing the email notices and have been so busy I've not even checked back here using a browser lately. Please accept my apologies!!
We'd rather not use "Services for Unix", but instead just solve the problem using our current software if possible. Even though SFU is free, I'm not familiar enough with it compared to cygwin.
jrsystemsnet, both the file names AND the path names are very long. I don't have the error message at the moment -- I'm on a different project and will need some time to get back to this particular issue. In the meantime, if you have any other solutions based on BOTH being long, I'm all ears. I've recommended to the client that they start revising how they name these things -- they are ridiculous even WITHOUT this issue, and I don't know how anybody's getting work done with folders and files nested so deep. Even though it gives them a means of finding their info using the "tree", it must take forever to actually get to the files they want to use.
We'd rather not use "Services for Unix", but instead just solve the problem using our current software if possible. Even though SFU is free, I'm not familiar enough with it compared to cygwin.
jrsystemsnet, both the file names AND the path names are very long. I don't have the error message at the moment -- I'm on a different project and will need some time to get back to this particular issue. In the meantime, if you have any other solutions based on BOTH being long, I'm all ears. I've recommended to the client that they start revising how they name these things -- they are ridiculous even WITHOUT this issue, and I don't know how anybody's getting work done with folders and files nested so deep. Even though it gives them a means of finding their info using the "tree", it must take forever to actually get to the files they want to use.
funaroma, there is a limit to how long the combined pathname plus filename can be - if memory serves, perhaps 128 characters? you can work around this by not backing up from close to the root of the server being backed up - for example, if you back up c:\very\long\folder\path\e
I hope you follow that, it's a little muddy in the telling, but I'm not sure how to state it more clearly.
I hope you follow that, it's a little muddy in the telling, but I'm not sure how to state it more clearly.
ASKER
Hi again,
I do follow, I just don't know how to resolve it for our situation, as it would mean we have to manually script the backup of many subfolders, and keep track of any new ones. I think the client is just going to have to get more realistic with the naming convention they are using, or we'll have to zip the entire tree in full every now and then, and with all changed files each day, and pull those over. I don't see any other way to do it... but we should be able to script/schedule a winzip-type app to do it...
I do follow, I just don't know how to resolve it for our situation, as it would mean we have to manually script the backup of many subfolders, and keep track of any new ones. I think the client is just going to have to get more realistic with the naming convention they are using, or we'll have to zip the entire tree in full every now and then, and with all changed files each day, and pull those over. I don't see any other way to do it... but we should be able to script/schedule a winzip-type app to do it...
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
jr, I have accepted your answer as it contains the most definitive choices for me, in addition to possibly zipping the directory structure in full once, and then maintaining differential or incremental zip files that can be easily synchronized using UNISON. I appreciate your ever-useful expertise and willingness to help... you are great!
For rsync - have a look at www.rsnapshot.org
Or google cwrsync, which is an all-in-one rsync-for-windows packaging that doesn't require cygwin at all (it actually bundles a couple of key cygwin dlls in with rsync itself. VERY handy for quick-and-dirty win32 stuff)