Solved

gzip results of remote find and gunzip locally in mone command

Posted on 2011-03-14
10
652 Views
Last Modified: 2013-11-05
Hi,

I'm trying to pipe the results of a find command on a remote server through gzip to compress them and then gunzip them on a local server in a one-liner, if possible:

something like:
ssh remotesite "find /archive/userfiles/ -type f -mtime -1 | cat {} gzip -9 | gunzip > /local/folder"

Open in new window


Cheers
0
Comment
Question by:bwgexperts
  • 7
  • 2
10 Comments
 
LVL 31

Expert Comment

by:farzanj
ID: 35133434
You cannot do file transfer in one liner.  You either need to cat it and this way you may save it

ssh remotesite "find /archive/userfiles/ -type f -mtime -1 -exec  cat {} " > /local/folder/file




0
 

Author Comment

by:bwgexperts
ID: 35133494
Thanks for the reply, but this gut seems to think you can gzip & gunzip in the once command

http://www.commandlinefu.com/commands/view/4859/scp-with-compression.

What I need to do is plug in a find command into this structure.

Thanks again.
0
 
LVL 31

Expert Comment

by:farzanj
ID: 35133550
Ok,

Try this then
 
ssh remotesite "find /archive/userfiles/ -type f -mtime -1 -exec  gunzip -c1 {} " | gunzip -c > /local/folder/file

Open in new window

0
Comprehensive Backup Solutions for Microsoft

Acronis protects the complete Microsoft technology stack: Windows Server, Windows PC, laptop and Surface data; Microsoft business applications; Microsoft Hyper-V; Azure VMs; Microsoft Windows Server 2016; Microsoft Exchange 2016 and SQL Server 2016.

 
LVL 31

Assisted Solution

by:farzanj
farzanj earned 167 total points
ID: 35133560
Or this one.

I left ; in the above one

ssh remotesite "find /archive/userfiles/ -type f -mtime -1 -exec  cat {} \; | gunzip -c1 {} " | gunzip -c > /local/folder/file
0
 
LVL 12

Assisted Solution

by:mccracky
mccracky earned 83 total points
ID: 35133635
I don't think you can do it in a "one-liner" and you don't really need to gzip and gunzip it as ssh has compression built in.  In two lines you could:

1.  ssh -fC user@remotesite "find /archive/userfiles/ -type f -mtime -1 > ~/filelist
2.  scp -C user@remotesite:filelist /local/folder

The "-C" option of ssh and scp turns on gzip compression for the transfers.
0
 
LVL 31

Expert Comment

by:farzanj
ID: 35133671
The example that you had quoted had only one file to be transferred.  I did not follow the same logic but for more files, it may be harder to do it in a single line.  As I said before, first you may need two commands as also shown by mccracky.
0
 
LVL 31

Expert Comment

by:farzanj
ID: 35134135
Your DNS is not configured right.

I could not find your MX record.


dig mx www.mxtoolbox.com

; <<>> DiG 9.7.0-P2-RedHat-9.7.0-5.P2.el6 <<>> mx www.mxtoolbox.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 17720
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0

;; QUESTION SECTION:
;www.mxtoolbox.com.            IN      MX

;; AUTHORITY SECTION:
mxtoolbox.com.            5      IN      SOA      ns1.zoneedit.com. soacontact.zoneedit.com. 1133805690 14400 7200 1073741824 3600

;; Query time: 31 msec
;; SERVER: 192.168.2.2#53(192.168.2.2)
;; WHEN: Mon Mar 14 20:05:27 2011
;; MSG SIZE  rcvd: 95


FOR EXAMPLE

dig mx juno.com

; <<>> DiG 9.7.0-P2-RedHat-9.7.0-5.P2.el6 <<>> mx juno.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 8211
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 2

;; QUESTION SECTION:
;juno.com.                  IN      MX

;; ANSWER SECTION:
juno.com.            5      IN      MX      10 mx.dca.untd.com.
juno.com.            5      IN      MX      10 mx.vgs.untd.com.

;; ADDITIONAL SECTION:
mx.vgs.untd.com.      5      IN      A      64.136.52.37
mx.dca.untd.com.      5      IN      A      64.136.44.37

;; Query time: 27 msec
;; SERVER: 192.168.2.2#53(192.168.2.2)
;; WHEN: Mon Mar 14 20:07:32 2011
;; MSG SIZE  rcvd: 109
0
 
LVL 31

Accepted Solution

by:
farzanj earned 167 total points
ID: 35134143
I send a mail on your root account. I used IP address.  Did you get any mail on your root account?

How to check?

Just type mail after becoming root
0
 
LVL 31

Expert Comment

by:farzanj
ID: 35134149
Please check using a local machine.  Otherwise it would take for ever to troubleshoot.  This question relates to EMAIL not troubleshooting the entire network.
0
 

Author Closing Comment

by:bwgexperts
ID: 36452779
Cheers.
0

Featured Post

Master Your Team's Linux and Cloud Stack!

The average business loses $13.5M per year to ineffective training (per 1,000 employees). Keep ahead of the competition and combine in-person quality with online cost and flexibility by training with Linux Academy.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
issue in getting eth0 IP in oracale virtual box Linux VM 4 54
Need BIOS update Linux for MSI X99A motherboard. 4 41
wipe a usb using python 5 48
android secure ftp 3 38
If you have a server on collocation with the super-fast CPU, that doesn't mean that you get it running at full power. Here is a preamble. When doing inventory of Linux servers, that I'm administering, I've found that some of them are running on l…
Little introduction about CP: CP is a command on linux that use to copy files and folder from one location to another location. Example usage of CP as follow: cp /myfoder /pathto/destination/folder/ cp abc.tar.gz /pathto/destination/folder/ab…
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:

809 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question