Solved

gzip results of remote find and gunzip locally in mone command

Posted on 2011-03-14
10
647 Views
Last Modified: 2013-11-05
Hi,

I'm trying to pipe the results of a find command on a remote server through gzip to compress them and then gunzip them on a local server in a one-liner, if possible:

something like:
ssh remotesite "find /archive/userfiles/ -type f -mtime -1 | cat {} gzip -9 | gunzip > /local/folder"

Open in new window


Cheers
0
Comment
Question by:bwgexperts
  • 7
  • 2
10 Comments
 
LVL 31

Expert Comment

by:farzanj
ID: 35133434
You cannot do file transfer in one liner.  You either need to cat it and this way you may save it

ssh remotesite "find /archive/userfiles/ -type f -mtime -1 -exec  cat {} " > /local/folder/file




0
 

Author Comment

by:bwgexperts
ID: 35133494
Thanks for the reply, but this gut seems to think you can gzip & gunzip in the once command

http://www.commandlinefu.com/commands/view/4859/scp-with-compression.

What I need to do is plug in a find command into this structure.

Thanks again.
0
 
LVL 31

Expert Comment

by:farzanj
ID: 35133550
Ok,

Try this then
 
ssh remotesite "find /archive/userfiles/ -type f -mtime -1 -exec  gunzip -c1 {} " | gunzip -c > /local/folder/file

Open in new window

0
 
LVL 31

Assisted Solution

by:farzanj
farzanj earned 167 total points
ID: 35133560
Or this one.

I left ; in the above one

ssh remotesite "find /archive/userfiles/ -type f -mtime -1 -exec  cat {} \; | gunzip -c1 {} " | gunzip -c > /local/folder/file
0
 
LVL 12

Assisted Solution

by:mccracky
mccracky earned 83 total points
ID: 35133635
I don't think you can do it in a "one-liner" and you don't really need to gzip and gunzip it as ssh has compression built in.  In two lines you could:

1.  ssh -fC user@remotesite "find /archive/userfiles/ -type f -mtime -1 > ~/filelist
2.  scp -C user@remotesite:filelist /local/folder

The "-C" option of ssh and scp turns on gzip compression for the transfers.
0
Enterprise Mobility and BYOD For Dummies

Like “For Dummies” books, you can read this in whatever order you choose and learn about mobility and BYOD; and how to put a competitive mobile infrastructure in place. Developed for SMBs and large enterprises alike, you will find helpful use cases, planning, and implementation.

 
LVL 31

Expert Comment

by:farzanj
ID: 35133671
The example that you had quoted had only one file to be transferred.  I did not follow the same logic but for more files, it may be harder to do it in a single line.  As I said before, first you may need two commands as also shown by mccracky.
0
 
LVL 31

Expert Comment

by:farzanj
ID: 35134135
Your DNS is not configured right.

I could not find your MX record.


dig mx www.mxtoolbox.com

; <<>> DiG 9.7.0-P2-RedHat-9.7.0-5.P2.el6 <<>> mx www.mxtoolbox.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 17720
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0

;; QUESTION SECTION:
;www.mxtoolbox.com.            IN      MX

;; AUTHORITY SECTION:
mxtoolbox.com.            5      IN      SOA      ns1.zoneedit.com. soacontact.zoneedit.com. 1133805690 14400 7200 1073741824 3600

;; Query time: 31 msec
;; SERVER: 192.168.2.2#53(192.168.2.2)
;; WHEN: Mon Mar 14 20:05:27 2011
;; MSG SIZE  rcvd: 95


FOR EXAMPLE

dig mx juno.com

; <<>> DiG 9.7.0-P2-RedHat-9.7.0-5.P2.el6 <<>> mx juno.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 8211
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 2

;; QUESTION SECTION:
;juno.com.                  IN      MX

;; ANSWER SECTION:
juno.com.            5      IN      MX      10 mx.dca.untd.com.
juno.com.            5      IN      MX      10 mx.vgs.untd.com.

;; ADDITIONAL SECTION:
mx.vgs.untd.com.      5      IN      A      64.136.52.37
mx.dca.untd.com.      5      IN      A      64.136.44.37

;; Query time: 27 msec
;; SERVER: 192.168.2.2#53(192.168.2.2)
;; WHEN: Mon Mar 14 20:07:32 2011
;; MSG SIZE  rcvd: 109
0
 
LVL 31

Accepted Solution

by:
farzanj earned 167 total points
ID: 35134143
I send a mail on your root account. I used IP address.  Did you get any mail on your root account?

How to check?

Just type mail after becoming root
0
 
LVL 31

Expert Comment

by:farzanj
ID: 35134149
Please check using a local machine.  Otherwise it would take for ever to troubleshoot.  This question relates to EMAIL not troubleshooting the entire network.
0
 

Author Closing Comment

by:bwgexperts
ID: 36452779
Cheers.
0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Using 'screen' for session sharing, The Simple Edition Step 1: user starts session with command: screen Step 2: other user (logged in with same user account) connects with command: screen -x Done. Both users are connected to the same CLI sessio…
The purpose of this article is to demonstrate how we can use conditional statements using Python.
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
This demo shows you how to set up the containerized NetScaler CPX with NetScaler Management and Analytics System in a non-routable Mesos/Marathon environment for use with Micro-Services applications.

863 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

18 Experts available now in Live!

Get 1:1 Help Now