Want to win a PS4? Go Premium and enter to win our High-Tech Treats giveaway. Enter to Win

x
?
Solved

gzip results of remote find and gunzip locally in mone command

Posted on 2011-03-14
10
Medium Priority
?
668 Views
Last Modified: 2013-11-05
Hi,

I'm trying to pipe the results of a find command on a remote server through gzip to compress them and then gunzip them on a local server in a one-liner, if possible:

something like:
ssh remotesite "find /archive/userfiles/ -type f -mtime -1 | cat {} gzip -9 | gunzip > /local/folder"

Open in new window


Cheers
0
Comment
Question by:bwgexperts
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 7
  • 2
10 Comments
 
LVL 31

Expert Comment

by:farzanj
ID: 35133434
You cannot do file transfer in one liner.  You either need to cat it and this way you may save it

ssh remotesite "find /archive/userfiles/ -type f -mtime -1 -exec  cat {} " > /local/folder/file




0
 

Author Comment

by:bwgexperts
ID: 35133494
Thanks for the reply, but this gut seems to think you can gzip & gunzip in the once command

http://www.commandlinefu.com/commands/view/4859/scp-with-compression.

What I need to do is plug in a find command into this structure.

Thanks again.
0
 
LVL 31

Expert Comment

by:farzanj
ID: 35133550
Ok,

Try this then
 
ssh remotesite "find /archive/userfiles/ -type f -mtime -1 -exec  gunzip -c1 {} " | gunzip -c > /local/folder/file

Open in new window

0
Nothing ever in the clear!

This technical paper will help you implement VMware’s VM encryption as well as implement Veeam encryption which together will achieve the nothing ever in the clear goal. If a bad guy steals VMs, backups or traffic they get nothing.

 
LVL 31

Assisted Solution

by:farzanj
farzanj earned 501 total points
ID: 35133560
Or this one.

I left ; in the above one

ssh remotesite "find /archive/userfiles/ -type f -mtime -1 -exec  cat {} \; | gunzip -c1 {} " | gunzip -c > /local/folder/file
0
 
LVL 12

Assisted Solution

by:mccracky
mccracky earned 249 total points
ID: 35133635
I don't think you can do it in a "one-liner" and you don't really need to gzip and gunzip it as ssh has compression built in.  In two lines you could:

1.  ssh -fC user@remotesite "find /archive/userfiles/ -type f -mtime -1 > ~/filelist
2.  scp -C user@remotesite:filelist /local/folder

The "-C" option of ssh and scp turns on gzip compression for the transfers.
0
 
LVL 31

Expert Comment

by:farzanj
ID: 35133671
The example that you had quoted had only one file to be transferred.  I did not follow the same logic but for more files, it may be harder to do it in a single line.  As I said before, first you may need two commands as also shown by mccracky.
0
 
LVL 31

Expert Comment

by:farzanj
ID: 35134135
Your DNS is not configured right.

I could not find your MX record.


dig mx www.mxtoolbox.com

; <<>> DiG 9.7.0-P2-RedHat-9.7.0-5.P2.el6 <<>> mx www.mxtoolbox.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 17720
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0

;; QUESTION SECTION:
;www.mxtoolbox.com.            IN      MX

;; AUTHORITY SECTION:
mxtoolbox.com.            5      IN      SOA      ns1.zoneedit.com. soacontact.zoneedit.com. 1133805690 14400 7200 1073741824 3600

;; Query time: 31 msec
;; SERVER: 192.168.2.2#53(192.168.2.2)
;; WHEN: Mon Mar 14 20:05:27 2011
;; MSG SIZE  rcvd: 95


FOR EXAMPLE

dig mx juno.com

; <<>> DiG 9.7.0-P2-RedHat-9.7.0-5.P2.el6 <<>> mx juno.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 8211
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 2

;; QUESTION SECTION:
;juno.com.                  IN      MX

;; ANSWER SECTION:
juno.com.            5      IN      MX      10 mx.dca.untd.com.
juno.com.            5      IN      MX      10 mx.vgs.untd.com.

;; ADDITIONAL SECTION:
mx.vgs.untd.com.      5      IN      A      64.136.52.37
mx.dca.untd.com.      5      IN      A      64.136.44.37

;; Query time: 27 msec
;; SERVER: 192.168.2.2#53(192.168.2.2)
;; WHEN: Mon Mar 14 20:07:32 2011
;; MSG SIZE  rcvd: 109
0
 
LVL 31

Accepted Solution

by:
farzanj earned 501 total points
ID: 35134143
I send a mail on your root account. I used IP address.  Did you get any mail on your root account?

How to check?

Just type mail after becoming root
0
 
LVL 31

Expert Comment

by:farzanj
ID: 35134149
Please check using a local machine.  Otherwise it would take for ever to troubleshoot.  This question relates to EMAIL not troubleshooting the entire network.
0
 

Author Closing Comment

by:bwgexperts
ID: 36452779
Cheers.
0

Featured Post

Visualize your virtual and backup environments

Create well-organized and polished visualizations of your virtual and backup environments when planning VMware vSphere, Microsoft Hyper-V or Veeam deployments. It helps you to gain better visibility and valuable business insights.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Little introduction about CP: CP is a command on linux that use to copy files and folder from one location to another location. Example usage of CP as follow: cp /myfoder /pathto/destination/folder/ cp abc.tar.gz /pathto/destination/folder/ab…
Have you ever been frustrated by having to click seven times in order to retrieve a small bit of information from the web, always the same seven clicks, scrolling down and down until you reach your target? When you know the benefits of the command l…
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
How to Install VMware Tools in Red Hat Enterprise Linux 6.4 (RHEL 6.4) Step-by-Step Tutorial
Suggested Courses

609 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question