sftp client in Redhat Linux supports append mode ?

sunhux
sunhux used Ask the Experts™
on

I have a Redhat Linux 4.5 which is sftp client & it is the party initiating
sftp connection (get & put or mget or mput) to a remote Windows server.

Questions:

1) ftp can support append mode.  I read somewhere that Linux (but not sure
   which flavor or version) can support append mode too if we apply a patch.
   Which patch is that & where can I download it (give exact url  / link)

2) what if on my Linux box end , can I set a data file into "append" mode?
    something like   ' chattr +a datafile '

3) sftp does not have "prompt" command, so does it by default support
    no prompting when we issue mget or mput ?
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
President
Commented:
Hi,

1) What do you mean by that? What is that you want ?  To append a file or to resume if it the download closes in between ?? It has nothing to do with Linux. FTP is an application program (ftp daemon) which supports resume etc.

2) As I told above file resume / append etc are not the responsibility of the Linux itself. These are features that applications may or may not provide so OS commands would not help you to achieve what you want. You should be configuring your application to support these.

3) though sftp does not support the "prompt" command like ftp it has a special switch called -b which you include it as a parameter and supply a  filename then he scp will read the list of files to transfer from a file instead of the console so that you'll be able to automatize file downloads.

Cheers,
K.
Top Expert 2007

Commented:
1. FTP supports resume (not aware of append).   I'm not aware of any patch that adds resume capabilities to SFTP.

2. Setting chattr +a file, will simply prevent you overwriting the file.  I won't make it automatically append.
 
3.  Note that mget and mput are aliases for get and put.  I've never encountered an sftp client that prompts you when uploading/downloading multiple files.
Top Expert 2015
Commented:
You can use rsync over ssh on instable links (cwrsync is a win client)
OWASP: Forgery and Phishing

Learn the techniques to avoid forgery and phishing attacks and the types of attacks an application or network may face.

Author

Commented:

> 1) What do you mean by that? What is that you want ?  To append a file or to resume if it the
> download closes in between ?? It has nothing to do with Linux. FTP is an application program
> (ftp daemon) which supports resume etc.

In my environment, if a file is existing, we don't want to overwrite but to append : that's what
the append mode is currently doing.  If a file is existing, it means it's not been processed yet,
so I'll need to append, otherwise get/put will overwrite & we'll lose data.

I'm not talking about unstable links or resuming a broken transfer

Author

Commented:


Just issue ftp in Windows command prompt & issue help & you'll see there's an optiion  for
'append'.  I'll need this feature in my Redhat Linux sftp & I think I saw somewhere there's
a patch to enable this feature, so let me know where to get this patch

Author

Commented:

Perhaps an equivalent patch for Redhat as in the following:
   http://fixunix.com/openssh/176045-sftp-append-patch.html


I'll need a freeware or free solution.


Btw, the remote Windows box runs Openssh for Windows 3.8p1-1 running on Windows 2003
- does it supports ssh1 or ssh2?
Top Expert 2015

Commented:
Are you constructing a log file this way?
APPEND just like ASCII mode is a lot of anachronism in FTP

Author

Commented:

> Are you constructing a log file this way?
In a way but I needed this not for logfile but for datafile


> APPEND just like ASCII mode is a lot of anachronism in FTP

regardless, I'll need a solution that's equivalent to "Append" as our
current FTP scripts / files transfer make use of Append but we're
required to move to sftp & I'll need a workaround to substitute for
Append.  Since you mentioned "ASCII", that's one of the issue I'll
be facing but I'll have to write scripts to do "dos2unix" or use Perl
scripts to strip/insert the CR characters at the end of each line
(someone else has given me solutions in EE on this "ASCII" thingy
 so that's why I didn't raise it here )
Top Expert 2015

Commented:
You can login via ssh and cat >file

ssh user@host 'cat >file'

Author

Commented:

or for Append purposes,
ssh user@host 'cat >>file'  ??

Can give me specific equivalents if I'm issuing from the Linux box (sftp client)
get = ...
put = ...
or the ssh example with cat you gave is only for put to the Windows box ?

PS: bear in mind the remote box is Windows, so should it be
  ssh user@host 'type >>file'  ??
Top Expert 2015

Commented:
there is no append in sftp protocol.

you can use ssh login instead.

Author

Commented:

Pardon my slow-wittedness,  so

ssh user@host 'cat >>file'  ??   = put (with append)

What about get, how's the ssh command syntax like?
Kerem ERSOYPresident
Commented:
While append is a valid comand in FTP protocol there's not a correspondoing commnad in sftp. This means that SFTP is unusable for your purposes. But it is always possible to copy a file over ssh. Since this would be done over SSH it would be as safe as sftp and it would support appending.

The commands should be :

ssh user@remotehost 'cat remotefile' >> localfile

so you're on target host running SSH to the source system and running 'cat remotefile' over the source sytem and redirecting the output to a localfile.

But this is cumulative. Say you the contents of the remotefile is:
111111
22222
33333
44444

and when you execute the command the contents of the localfile would be
111111
22222
33333
44444

But when you run it a second time the contents would be:
111111
22222
33333
44444
111111
22222
33333
44444

and so on. So you'll need to truncate the source file (remotefile) after it has been transferred. IS tis what you want? then you might use this command to truncate the remote file:

ssh root@remotehost 'cat remotefile && > remotefile' >> localfile  

This will connect to remote host. Send the contents of remotefile to the localhost and truncate the remotefile and redirect the contents to localfile.

Cheers,
K.

Author

Commented:
Thanks for providing the exact commands.  So

ssh user@remotehost 'cat remotefile' >> localfile                          = get with Append?

ssh root@remotehost 'cat remotefile && > remotefile' >> localfile  = put with Append?

Author

Commented:

& one other request, can you also help insert "dos2unix" & "unix2dos", bearing
in mind the box which I'm initiating the ssh from is a Linux & the remotehost is a
Windows  box

Author

Commented:

I mean help insert "dos2unix" or "unix2dos" into the ssh command examples you've provided

I'm not good at this, but correct me if I'm wrong, I think it's

ssh user@remotehost 'cat remotefile | dos2unix' >> localfile

ssh root@remotehost 'cat remotefile|unix2dos && > remotefile' >> localfile
Kerem ERSOYPresident
Commented:
> ssh user@remotehost 'cat remotefile' >> localfile                          = get with Append?

> ssh root@remotehost 'cat remotefile && > remotefile' >> localfile  = put with Append?

This method is a one way execution connects to remote prints the remote file redirects the input to a localfile. The difference is the second is truncating the file after displaying the contents so that it won't get listed again in a consecutive run. They both do the Get. onsy source file is truncated in the second one (the command "> remotefile" does this)
Kerem ERSOYPresident
Commented:
Nope you should use:

ssh user@remotehost 'cat remotefile' | dos2unix >> localfile                = get with append

or

ssh root@remotehost 'cat remotefile && > remotefile' | dos2unix >> localfile   = get and ttuncate the source file with append

Author

Commented:

How does the "ssh user@remotehost ......" command look like if I want it
to achieve the equivalent of  put with 'Append'  & 'dos2unix'  (the initiating
box is Linux while the remote server is a Windows, ie I transfer a file from
the Linux box to the remote Windows box) ?
Kerem ERSOYPresident
Commented:
AS I've told you earlier :

ssh user@remotehost 'cat remotefile' | dos2unix >> localfile  
does this leaving the Windows Log as is.

ssh root@remotehost 'cat remotefile && > remotefile' | dos2unix >> localfile  
does this (clearing the windows log after transfer)

Cheers,
K.
Kerem ERSOYPresident

Commented:
If you initiate the connection from linux and transferring a file from windows you'll need to use dos2unix

If other  way round you need to use unix2dos..

Please tray and see for yourself.

Cheers,
K.

Author

Commented:

One more question:

> You can use rsync over ssh on instable links
What's the exact syntax like if we were to use rsync+ssh+Append mode with dos2unix or unix2dos
(kindly give the equivalents for put & get initiated from the Linux box to the Windows remote host)
Kerem ERSOYPresident

Commented:
what rsync does is to keep files over source and destination files/directories synchronised. It does not work as ftp. I don't know it is what you want.

Here's a link from EE that I've participated a while ago about rsync and SSH:


http://www.experts-exchange.com/OS/Linux/Q_22675834.html


Cheers,
K.

Author

Commented:
> http://www.experts-exchange.com/OS/Linux/Q_22675834.html

Yep, the link is quite close to what's needed, just that it's short of a specific
command/example for implementing ASCII & Append modes (ie dos2unix &
unix2dos  and  >> ).  Could you provide?

Author

Commented:

Thanks.  I awarded points for responses that
give specific codes/examples.

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial