Solved

gzip results of remote find and gunzip locally in mone command

Posted on 2011-03-14
10
653 Views
Last Modified: 2013-11-05
Hi,

I'm trying to pipe the results of a find command on a remote server through gzip to compress them and then gunzip them on a local server in a one-liner, if possible:

something like:
ssh remotesite "find /archive/userfiles/ -type f -mtime -1 | cat {} gzip -9 | gunzip > /local/folder"

Open in new window


Cheers
0
Comment
Question by:bwgexperts
  • 7
  • 2
10 Comments
 
LVL 31

Expert Comment

by:farzanj
ID: 35133434
You cannot do file transfer in one liner.  You either need to cat it and this way you may save it

ssh remotesite "find /archive/userfiles/ -type f -mtime -1 -exec  cat {} " > /local/folder/file




0
 

Author Comment

by:bwgexperts
ID: 35133494
Thanks for the reply, but this gut seems to think you can gzip & gunzip in the once command

http://www.commandlinefu.com/commands/view/4859/scp-with-compression.

What I need to do is plug in a find command into this structure.

Thanks again.
0
 
LVL 31

Expert Comment

by:farzanj
ID: 35133550
Ok,

Try this then
 
ssh remotesite "find /archive/userfiles/ -type f -mtime -1 -exec  gunzip -c1 {} " | gunzip -c > /local/folder/file

Open in new window

0
Master Your Team's Linux and Cloud Stack!

The average business loses $13.5M per year to ineffective training (per 1,000 employees). Keep ahead of the competition and combine in-person quality with online cost and flexibility by training with Linux Academy.

 
LVL 31

Assisted Solution

by:farzanj
farzanj earned 167 total points
ID: 35133560
Or this one.

I left ; in the above one

ssh remotesite "find /archive/userfiles/ -type f -mtime -1 -exec  cat {} \; | gunzip -c1 {} " | gunzip -c > /local/folder/file
0
 
LVL 12

Assisted Solution

by:mccracky
mccracky earned 83 total points
ID: 35133635
I don't think you can do it in a "one-liner" and you don't really need to gzip and gunzip it as ssh has compression built in.  In two lines you could:

1.  ssh -fC user@remotesite "find /archive/userfiles/ -type f -mtime -1 > ~/filelist
2.  scp -C user@remotesite:filelist /local/folder

The "-C" option of ssh and scp turns on gzip compression for the transfers.
0
 
LVL 31

Expert Comment

by:farzanj
ID: 35133671
The example that you had quoted had only one file to be transferred.  I did not follow the same logic but for more files, it may be harder to do it in a single line.  As I said before, first you may need two commands as also shown by mccracky.
0
 
LVL 31

Expert Comment

by:farzanj
ID: 35134135
Your DNS is not configured right.

I could not find your MX record.


dig mx www.mxtoolbox.com

; <<>> DiG 9.7.0-P2-RedHat-9.7.0-5.P2.el6 <<>> mx www.mxtoolbox.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 17720
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0

;; QUESTION SECTION:
;www.mxtoolbox.com.            IN      MX

;; AUTHORITY SECTION:
mxtoolbox.com.            5      IN      SOA      ns1.zoneedit.com. soacontact.zoneedit.com. 1133805690 14400 7200 1073741824 3600

;; Query time: 31 msec
;; SERVER: 192.168.2.2#53(192.168.2.2)
;; WHEN: Mon Mar 14 20:05:27 2011
;; MSG SIZE  rcvd: 95


FOR EXAMPLE

dig mx juno.com

; <<>> DiG 9.7.0-P2-RedHat-9.7.0-5.P2.el6 <<>> mx juno.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 8211
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 2

;; QUESTION SECTION:
;juno.com.                  IN      MX

;; ANSWER SECTION:
juno.com.            5      IN      MX      10 mx.dca.untd.com.
juno.com.            5      IN      MX      10 mx.vgs.untd.com.

;; ADDITIONAL SECTION:
mx.vgs.untd.com.      5      IN      A      64.136.52.37
mx.dca.untd.com.      5      IN      A      64.136.44.37

;; Query time: 27 msec
;; SERVER: 192.168.2.2#53(192.168.2.2)
;; WHEN: Mon Mar 14 20:07:32 2011
;; MSG SIZE  rcvd: 109
0
 
LVL 31

Accepted Solution

by:
farzanj earned 167 total points
ID: 35134143
I send a mail on your root account. I used IP address.  Did you get any mail on your root account?

How to check?

Just type mail after becoming root
0
 
LVL 31

Expert Comment

by:farzanj
ID: 35134149
Please check using a local machine.  Otherwise it would take for ever to troubleshoot.  This question relates to EMAIL not troubleshooting the entire network.
0
 

Author Closing Comment

by:bwgexperts
ID: 36452779
Cheers.
0

Featured Post

Free Tool: Subnet Calculator

The subnet calculator helps you design networks by taking an IP address and network mask and returning information such as network, broadcast address, and host range.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Guacamole cut and paste issue 3 97
VMware Workstation 12 Player 16 133
LogmeIn using Linux Ubuntu 16.04 6 64
How to change the nameserver on Ubuntu Server 6 50
The purpose of this article is to fix the unknown display problem in Linux Mint operating system. After installing the OS if you see Display monitor is not recognized then we can install "MESA" utilities to fix this problem or we can install additio…
Fine Tune your automatic Updates for Ubuntu / Debian
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…
This demo shows you how to set up the containerized NetScaler CPX with NetScaler Management and Analytics System in a non-routable Mesos/Marathon environment for use with Micro-Services applications.

830 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question