const71
asked on
COMPRESS FILES ON THE FLY USING FTP
Is it possible to compress data on the fly as it is FTP'ed from an IBM Mainframe to a Windows FTP Server. We
are trying to avoid the extra step of having to WINZIP the very large file on the FTP sever once the upload is done
from the HOST.
Is there a command in FTP that allows compression on the fly?
are trying to avoid the extra step of having to WINZIP the very large file on the FTP sever once the upload is done
from the HOST.
Is there a command in FTP that allows compression on the fly?
ASKER
thanks for link..unfortunately the ftp client used is that which comes with OS/390 Mainframe
There is a couple of ways we handle this.
have a CRON job zip the file before it is FTP'd
You will obviously have to change your FTP script to only send *.zip files.
We have a cron job that runs every couple of minutes that says
if file.txt exists than compress it in our case to file.tar
if file.tar exists then ftp it to server x
the ftp script that runs as a cron, looks like
ftp -n <<SCRIPT
open 192.168.1.1
user anonymous abc@companyxxx.com
put /home/exceed/interface/rep orts/dlr_b ackorder.t ar dlr_backorder.tar
put /home/exceed/interface/rep orts/dlr_C ustGroup.t ar dlr_CustGroup.tar
put /home/exceed/interface/rep orts/dlr_m odels.tar dlr_models.tar
put /home/exceed/interface/rep orts/dlr_p rice.tar dlr_price.tar
put /home/exceed/interface/rep orts/dlr_q oh.tar dlr_qoh.tar
put /home/exceed/interface/rep orts/dlr_r eserved.ta r dlr_reserved.tar
put /home/exceed/interface/rep orts/dlr_s alesorder. tar dlr_salesorder.tar
quit
SCRIPT
exit
:
Hope this leads you in the right direction
Cheers
Ian
have a CRON job zip the file before it is FTP'd
You will obviously have to change your FTP script to only send *.zip files.
We have a cron job that runs every couple of minutes that says
if file.txt exists than compress it in our case to file.tar
if file.tar exists then ftp it to server x
the ftp script that runs as a cron, looks like
ftp -n <<SCRIPT
open 192.168.1.1
user anonymous abc@companyxxx.com
put /home/exceed/interface/rep
put /home/exceed/interface/rep
put /home/exceed/interface/rep
put /home/exceed/interface/rep
put /home/exceed/interface/rep
put /home/exceed/interface/rep
put /home/exceed/interface/rep
quit
SCRIPT
exit
:
Hope this leads you in the right direction
Cheers
Ian
Hi again,
Wasn't too sure if you'd know what I meant by tar.
tar is the unix ( in our case AIX ) "equivalent" of zip.
Winzip opens *.tar files with no problem so end user does not need tar.
In unix if you go man tar you will get the syntax.
man cron gives the syntax for the "scheduled task"
Cheers again
Ian
Wasn't too sure if you'd know what I meant by tar.
tar is the unix ( in our case AIX ) "equivalent" of zip.
Winzip opens *.tar files with no problem so end user does not need tar.
In unix if you go man tar you will get the syntax.
man cron gives the syntax for the "scheduled task"
Cheers again
Ian
Const71,
A small note: Parkerig has a good idea. However, a cursory review of the TAR document is that TAR DOES NOT compress files (in fact the documentation available at the OpenGroup specifically recommends running COMPRESS against the archive built by TAR). TAR is merely a packager of multiple files into a single archive.
The idea, however, is reasonable. A script can run the appropriate version of ZIP to compress the files. They can then be transmitted as BINARY files by FTP (the binary is important here, attempting to move binary files between disparate systems will not work reliably without the BINARY specification).
The FTP protocol RFC does not, to my recollection, make any provisions for "on the fly" compression.
I hope that the preceding is helpful.
- Bob (aka RLGSC)
A small note: Parkerig has a good idea. However, a cursory review of the TAR document is that TAR DOES NOT compress files (in fact the documentation available at the OpenGroup specifically recommends running COMPRESS against the archive built by TAR). TAR is merely a packager of multiple files into a single archive.
The idea, however, is reasonable. A script can run the appropriate version of ZIP to compress the files. They can then be transmitted as BINARY files by FTP (the binary is important here, attempting to move binary files between disparate systems will not work reliably without the BINARY specification).
The FTP protocol RFC does not, to my recollection, make any provisions for "on the fly" compression.
I hope that the preceding is helpful.
- Bob (aka RLGSC)
Howdy,
Bob thanks for info.
I have just tested the other inbuilt options - DO NOT RUN without readning man pack or man compress first !
(1) pack - not winzip compatiable
(2) compress - not winzip compatiable.
This lead me on a search for a unix ZIP.
the below URL has a number of zip executables for various OS
http://www.gzip.org/#exe
http://www.gzip.org/ - hompage so you can get FAQ, source and compile etc. if required.
Cheers to all
Ian
Bob thanks for info.
I have just tested the other inbuilt options - DO NOT RUN without readning man pack or man compress first !
(1) pack - not winzip compatiable
(2) compress - not winzip compatiable.
This lead me on a search for a unix ZIP.
the below URL has a number of zip executables for various OS
http://www.gzip.org/#exe
http://www.gzip.org/ - hompage so you can get FAQ, source and compile etc. if required.
Cheers to all
Ian
if you are running a dial up modem, say as 56k each end, in reality you are compressing the files in midstream about as best as you can, "on the fly", and .zip'ing it will only be improving the intial storage requirements at the receiving end
among my prior efforts, I just set up a batch file to pick up and move the file desired for xmit, and pkzip was suffiicient, running on schedule (of when the file available). Script would copy (or move) file to directory for sending (or for pickup). The xmit scheduler itself ran dependent on whether the directory was empty or had content
among my prior efforts, I just set up a batch file to pick up and move the file desired for xmit, and pkzip was suffiicient, running on schedule (of when the file available). Script would copy (or move) file to directory for sending (or for pickup). The xmit scheduler itself ran dependent on whether the directory was empty or had content
ASKER
I think I am leaning towards compressing the file on the host...FTPing the file to a windows server...but the question concerning compatibility remains. Is it possible to decompress a file that was compressed on the IBM host on the windows platform?
actually, gzip compression is understood by winzip. If you gzip'd the file prior to ftp, then ftp the file using binary mode, winzip will be able to uncompress the file.
Just wondering if you considered secure copy as it is available in most *nix platforms as well as windows and the -C option compresses automatically.
Just wondering if you considered secure copy as it is available in most *nix platforms as well as windows and the -C option compresses automatically.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Try http://www.webcom.com/help/ftp/compression.shtml