Solved

COMPRESS FILES ON THE FLY USING FTP

Posted on 2004-03-31
10
5,296 Views
Last Modified: 2013-11-29
Is it possible to compress data on the fly as it is FTP'ed from an IBM Mainframe to a Windows FTP Server. We
are trying to avoid the extra step of having to WINZIP the very large file on the FTP sever once the upload is done
from the HOST.


Is there a command in FTP that allows compression on the fly?
0
Comment
Question by:const71
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
10 Comments
 
LVL 49

Expert Comment

by:sunray_2003
ID: 10727842
Not sure if this can help

Try  http://www.webcom.com/help/ftp/compression.shtml
0
 

Author Comment

by:const71
ID: 10727933
thanks for link..unfortunately the ftp client used is that which comes with OS/390 Mainframe
0
 
LVL 6

Expert Comment

by:parkerig
ID: 10728050
There is a couple of ways we handle this.
have a CRON job zip the file before it is FTP'd
You will obviously have to change your FTP script to only send *.zip files.

We have a cron job that runs every couple of minutes that says

if file.txt exists than compress it in our case to file.tar
if file.tar exists then ftp it to server x

the ftp script that runs as a cron, looks like

ftp -n <<SCRIPT                                                                
open 192.168.1.1                                                                  
user anonymous abc@companyxxx.com                                                      
put /home/exceed/interface/reports/dlr_backorder.tar dlr_backorder.tar          
put /home/exceed/interface/reports/dlr_CustGroup.tar dlr_CustGroup.tar          
put /home/exceed/interface/reports/dlr_models.tar dlr_models.tar                
put /home/exceed/interface/reports/dlr_price.tar dlr_price.tar                  
put /home/exceed/interface/reports/dlr_qoh.tar dlr_qoh.tar                      
put /home/exceed/interface/reports/dlr_reserved.tar dlr_reserved.tar            
put /home/exceed/interface/reports/dlr_salesorder.tar dlr_salesorder.tar        
quit                                                                            
SCRIPT                                                                          
exit                                                                            
:



Hope this leads you in the right direction

Cheers
Ian
0
Easy, flexible multimedia distribution & control

Coming soon!  Ideal for large-scale A/V applications, ATEN's VM3200 Modular Matrix Switch is an all-in-one solution that simplifies video wall integration. Easily customize display layouts to see what you want, how you want it in 4k.

 
LVL 6

Expert Comment

by:parkerig
ID: 10728128
Hi again,

Wasn't too sure if you'd know what I meant by tar.
tar is the unix ( in our case AIX ) "equivalent" of zip.
Winzip opens *.tar files with no problem so end user does not need tar.
In unix if you go man tar you will get the syntax.
man cron gives the syntax for the "scheduled task"

Cheers again
Ian
0
 
LVL 8

Expert Comment

by:RLGSC
ID: 10728307
Const71,

A small note: Parkerig has a good idea. However, a cursory review of the TAR document is that TAR DOES NOT compress files (in fact the documentation available at the OpenGroup specifically recommends running COMPRESS against the archive built by TAR). TAR is merely a packager of multiple files into a single archive.

The idea, however, is reasonable. A script can run the appropriate version of ZIP to compress the files. They can then be transmitted as BINARY files by FTP (the binary is important here, attempting to move binary files between disparate systems will not work reliably without the BINARY specification).

The FTP protocol RFC does not, to my recollection, make any provisions for "on the fly" compression.

I hope that the preceding is helpful.

- Bob (aka RLGSC)
0
 
LVL 6

Expert Comment

by:parkerig
ID: 10728439
Howdy,
Bob thanks for info.

I have just tested the other inbuilt options - DO NOT RUN without readning man pack or man compress first !

(1) pack - not winzip compatiable
(2) compress - not winzip compatiable.

This lead me on a search for a unix ZIP.

the below URL has a number of zip executables for various OS

http://www.gzip.org/#exe
http://www.gzip.org/ - hompage so you can get FAQ, source and compile etc. if required.

Cheers to all
Ian
0
 
LVL 24

Expert Comment

by:SunBow
ID: 10728527
if you are running a dial up modem, say as 56k each end, in reality you are compressing the files in midstream about as best as you can, "on the fly", and .zip'ing it will only be improving the intial storage requirements at the receiving end

among my prior efforts, I just set up a batch file to pick up and move the file desired for xmit, and pkzip was suffiicient, running on schedule (of when the file available). Script would copy (or move) file to directory for sending (or for pickup). The xmit scheduler itself ran dependent on whether the directory was empty or had content
0
 

Author Comment

by:const71
ID: 10729428
I think I am leaning towards compressing the file on the host...FTPing the file to a windows server...but the question concerning compatibility remains. Is it possible to decompress a file that was compressed on the IBM host on the windows platform?
0
 
LVL 10

Expert Comment

by:Kavar
ID: 10729713
actually, gzip compression is understood by winzip.  If you gzip'd the file prior to ftp, then ftp the file using binary mode, winzip will be able to uncompress the file.

Just wondering if you considered secure copy as it is available in most *nix platforms as well as windows and the -C option compresses automatically.
0
 
LVL 6

Accepted Solution

by:
parkerig earned 500 total points
ID: 10729937
Hi,
As mentioned above compatibility no problem, make sure you do a binary ftp - add switch to info I gave above.

Another alternative to FTP is SAMBA

www.samba.org

You can then use any Windows zip programme.

Cheers
Ian
0

Featured Post

PeopleSoft Has Never Been Easier

PeopleSoft Adoption Made Smooth & Simple!

On-The-Job Training Is made Intuitive & Easy With WalkMe's On-Screen Guidance Tool.  Claim Your Free WalkMe Account Now

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
SCCM, SCOM or Something Else 6 65
Simple Router Management, Subnets and VLANs e.g. RV0xx 7 76
Ping and real time 48 84
Find computer name from username 9 107
If you're not part of the solution, you're part of the problem.   Tips on how to secure IoT devices, even the dumbest ones, so they can't be used as part of a DDoS botnet.  Use PRTG Network Monitor as one of the building blocks, to detect unusual…
I had an issue with InstallShield not being able to use Computer Browser service on Windows Server 2012. Here is the solution I found.
Internet Business Fax to Email Made Easy - With  eFax Corporate (http://www.enterprise.efax.com), you'll receive a dedicated online fax number, which is used the same way as a typical analog fax number. You'll receive secure faxes in your email, f…
This video gives you a great overview about bandwidth monitoring with SNMP and WMI with our network monitoring solution PRTG Network Monitor (https://www.paessler.com/prtg). If you're looking for how to monitor bandwidth using netflow or packet s…

730 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question