Solved

automate sftp without key authentication and expect

Posted on 2010-08-18
22
1,729 Views
Last Modified: 2013-11-17
Here is the requirement and want to know whether i would be able to do it or not:

I want to automate the sftp to get a file from a remote server without key authentication and expect. can anybody provide any script for this to achieve?
0
Comment
Question by:kalyan_arjun
  • 10
  • 7
  • 3
  • +2
22 Comments
 
LVL 34

Expert Comment

by:Shreedhar Ette
ID: 33471928
Hi,

Refer this:
http://cpanforum.com/threads/3329

It might help,
Shree
0
 

Author Comment

by:kalyan_arjun
ID: 33471945
The link you had sent us linked to another one which i am not able to access. And moreover, it seems like he is using expect with the perl. Like I said, i am looking for a script without key authentication and expect.
Anyway, Thanks for the quick response.
0
 
LVL 7

Expert Comment

by:lewisg
ID: 33472047
This is what I have used before http://troy.jdmz.net/rsync/index.html

Works GREAT!
0
 

Author Comment

by:kalyan_arjun
ID: 33472169
Hi lewis, The link you had sent me is really a nice but it mentioned that it needs to have the keys on the remote host(check Configuring remotehost section). This will not work for me.

Let me be more specific on my requirement.
I dont have any permissions/authority to create a file on the remote host and remote directory. A file would be placed daily and im looking for a script to get that file from the remote host.
I am able to do sftp@remotehost and get that file after entering the password. I want to acheive this in the script without paswordkey and expect.

Lewis, I appreciate for the link that you had sent and is really a nice article regarding the secuirty tip where you can mention the ipadreeses that would allow the sftp. Thanks!
0
 
LVL 7

Expert Comment

by:lewisg
ID: 33472233
If your system has autoexpect here is a slick scripting solution:

http://dbaspot.com/forums/shell/196804-novice-needs-help-using-expect-automate-sftp.html

0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 33472789
No way.
There is no possibility in sftp to provide a password on the commandline or to have it read from a file or stdin or the like.
You will have to go with key authentication or with something like expect.
ssh-agent will not work either, as it relies on keys and not system passwords.
wmp
 
0
 
LVL 26

Expert Comment

by:arober11
ID: 33478537
Do you have php installed?  As the SFTP sub-system permits password authentication, see: http://www.phpbuilder.com/manual/function.ssh2-sftp.php
0
 
LVL 26

Expert Comment

by:arober11
ID: 33526860
Along similiar lines, if Java is your language of choice, the mindterm java library and SFTPCopyFile.java example provide the functionality you require:

Usage: SFTPCopyFile <server:port> <username> <password> to|from <src_file> <dst_file>

See: http://www.appgate.com/index/products/mindterm/
0
 

Author Comment

by:kalyan_arjun
ID: 33551987
i was able to do sftp with the expect as there were no options until I use another language. Thanks everybody for your suggestions.

And one more thing is, i need to get the latest file from the other server. Eveyday one file is copied to the remote server and i need to get the latest file from the remote server:

sftp> latest_file=`ls -ltr | tail -1 | awk '{print $9}'
Invalid command.
It seems like only the sftp server is installed on this server

but ls -ltr works and it will give a list of files. Not sure what the other vendor has it. Is there anyway i could do it?
0
 

Author Comment

by:kalyan_arjun
ID: 33551991

#!/usr/local/bin/expect

#DATE1=`date "+%m%d%y%H%M%S"'
#LOGFILE= /ul/batch/logs/log_$DATE1.log
spawn sftp abc@hostname.com
expect "password:"
send "pass\n";

expect sftp>
send "get abc.txt\r"
expect sftp>
send "exit\r"

expect eof
0
 

Author Comment

by:kalyan_arjun
ID: 33555636
Any help with this or do i need to open another question for this?
0
IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

 
LVL 7

Expert Comment

by:lewisg
ID: 33559643
It looks like the question was answered with information that appears on the dbaspot.com.

As for getting the newest file that sounds like a new question.

SFTP has a fairly limited command set and since you can't run anything on the remote machine you might try capturing a list of file names to your machine and then figuring out what the latest file is. What do the file names look like?
0
 

Author Comment

by:kalyan_arjun
ID: 33564448

#!/usr/local/bin/expect

#DATE1=`date "+%m%d%y%H%M%S"'
#LOGFILE= /ul/batch/logs/log_$DATE1.log
spawn sftp abc@hostname.com
expect "password:"
send "pass\n";

expect sftp>
send "ls /ul/test\n"
expect "No Such File" {send "exit\n"}\
expect sftp>
send "get abc.txt\r"
expect sftp>
send "exit\r"
expect eof

I tried using exit and quit and neither of them are working. Even though the file is not there, it is executing the next steps.

I am running with an agent where it looks for an exit code 0 for successful and 1 for failure. I am looking for a script for this..
0
 
LVL 26

Expert Comment

by:arober11
ID: 33577535
Hi, you now appear to be using "expect" but  your question: "automate sftp without key authentication and expect"
states you require a solutions that does not use "expect", have you changed your mind?
0
 

Author Comment

by:kalyan_arjun
ID: 33578227
sorry I should have changed it. Now,i am using expect for this sftp.Like, i mentioned, i want to quit or exit if the file is not there.
0
 
LVL 26

Expert Comment

by:arober11
ID: 33579238
Replace your:

expect "No Such File" {send "exit\n"}\


with:

expect "No such" { send "exit\r"
                           exit 2}
0
 

Author Comment

by:kalyan_arjun
ID: 33580160
still not working..
0
 
LVL 26

Expert Comment

by:arober11
ID: 33598740
Sorry forgot it's FORMAT sensitive, you need a blank line after the "{", try:

expect "No such" {
                           send "exit\r";
                           exit 2
                         }

If still not working add a "-d" debug flag to first line e.g.

#!/usr/local/bin/expect -d

Should show you were you going wrong, ELSE try the following, which works:


#!/bin/sh
# Name:    sftp_get_using_pwd.sh
# Purpose: SFTP test and pull a remote file, using password authentication.
#
EXPECT_DIR="/usr/local/bin/"
LOG_DIR="/tmp"
LOG_FILE="no_key-sftp"
DEST_DIR="."
SFTP_USER="abcdef"
SFTP_PASS="xxxxxx"
SFTP_HOST="some.host"
SFTPEXIT=0
tstFile="/ul/test"

# ----- Procedures -----
sftp_batch() {

$EXPECT_DIR/expect 2>&1 > "${LOG_DIR}/${LOG_FILE}" <<EOF
set timeout 100
spawn  /usr/bin/sftp $SFTP_USER@$SFTP_HOST
log_file ${LOG_DIR}/${LOG_FILE}-sftp
expect "Connecting to $SFTP_HOST..." {
  expect {
           -re ".*assword:" {
                              send -- "$SFTP_PASS\r";
                              expect {
                                       "Password:" {
                                                       send_user "ERROR SFTP password invalid\n";
                                                       exit 1
                                                      }
                                       "sftp> " {
                                                       send "ls $tstFile\r";
                                                       expect {
                                                                "No such file or directory" {
                                                                              send_user "ERROR No such File: $tstFile\n";
                                                                              send "quit\r";
                                                                              exit 8
                                                                             }
                                                                "sftp> " {
                                                                        send "get $tstFile $DEST_DIR\r";
                                                                        expect "sftp>" {
                                                                                        send "quit\r";
                                                                                        exit
                                                                                      }
                                                                        send_user "ERROR SFTP get failed OR timed out\n";
                                                                        send "quit\r";
                                                                        exit 7
                                                                        }
                                                            }
                                                        send_user "ERROR SFTP listing failed\n";
                                                        send "quit\r";
                                                        exit 6
                                                   }
                                     }
                           }
       }
       send_user "ERROR SFTP connection failed\n"
       exit 3
}
send_user "ERROR SFTP initialization failed\n"
exit 4
EOF
SFTPEXIT=$?
0
 
LVL 26

Expert Comment

by:arober11
ID: 33598774
Woops, cut'n'paste failure, the whole script is below:


#!/bin/sh
# Name:    sftp_get_using_pwd.sh
# Purpose: SFTP test and pull a remote file, using password authentication.
#
EXPECT_DIR="/usr/local/bin/"
LOG_DIR="/tmp"
LOG_FILE="no_key-sftp"
DEST_DIR="."
SFTP_USER="abcdef"
SFTP_PASS="xxxxxx"
SFTP_HOST="some.host"
SFTPEXIT=0
tstFile="/ul/test"

# ----- Procedures -----
sftp_batch() {

$EXPECT_DIR/expect 2>&1 > "${LOG_DIR}/${LOG_FILE}" <<EOF
set timeout 100
spawn  /usr/bin/sftp $SFTP_USER@$SFTP_HOST
log_file ${LOG_DIR}/${LOG_FILE}
expect "Connecting to $SFTP_HOST..." {
  expect {
           -re ".*assword:" {
                              send -- "$SFTP_PASS\r";
                              expect {
                                       "Password:" {
                                                       send_user "ERROR SFTP password invalid\n";
                                                       exit 1
                                                      }
                                       "sftp> " {
                                                       send "ls $tstFile\r";
                                                       expect {
                                                                "No such file or directory" {
                                                                              send_user "ERROR No such File: $tstFile\n";
                                                                              send "quit\r";
                                                                              exit 8
                                                                             }
                                                                "sftp> " {
                                                                        send "get $tstFile $DEST_DIR\r";
                                                                        expect "sftp>" {
                                                                                        send "quit\r";
                                                                                        exit
                                                                                      }
                                                                        send_user "ERROR SFTP get failed OR timed out\n";
                                                                        send "quit\r";
                                                                        exit 7
                                                                        }
                                                            }
                                                        send_user "ERROR SFTP listing failed\n";
                                                        send "quit\r";
                                                        exit 6
                                                   }
                                     }
                           }
       }
       send_user "ERROR SFTP connection failed\n"
       exit 3
}
send_user "ERROR SFTP initialization failed\n"
exit 4
EOF
SFTPEXIT=$?
}

# ----- Main -----
echo "ls" > $CMD_FILE
sftp_batch
echo "----------------"
echo "Returned $SFTPEXIT"
echo "--"
cat "${LOG_DIR}/${LOG_FILE}"
0
 

Author Comment

by:kalyan_arjun
ID: 33601057
This script works like a charm. I am trying to remove the files after it is downloaded where i am unable to do it. Could you throw some light on this?
0
 
LVL 26

Accepted Solution

by:
arober11 earned 500 total points
ID: 33601428
Not the safest method, but you could just chnage the following lines:

                                                                        expect "sftp>" {
                                                                                        send "quit\r";
                                                                                        exit
                                                                                      }
                                                                        send_user "ERROR SFTP get failed OR timed out\n";

to:
                                                                        expect "sftp>" {
                                                                                               send "rm $tstFile\r";
                                                                                               expect "sftp>" {
                                                                                                                     send "quit\r";
                                                                                                                     exit
                                                                                                                    }
                                                                                              send_user "ERROR SFTP remove failed OR timed out\n";
                                                                                              exit 5
                                                                                              }
                                                                        send_user "ERROR SFTP get failed OR timed out\n";
0
 

Author Closing Comment

by:kalyan_arjun
ID: 33708881
perfect solution and it worked like a charm...
0

Featured Post

How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

Join & Write a Comment

Suggested Solutions

Using libpcap/Jpcap to capture and send packets on Solaris version (10/11) Library used: 1.      Libpcap (http://www.tcpdump.org) Version 1.2 2.      Jpcap(http://netresearch.ics.uci.edu/kfujii/Jpcap/doc/index.html) Version 0.6 Prerequisite: 1.      GCC …
Imagine a situation that you have installed SSL (http://en.wikipedia.org/wiki/Secure_Sockets_Layer) Certificate on your Cisco ASA (Cisco Adaptive Security Appliance) firewall. Installation of SSL certificate on ASA is an another topic for which you …
This video shows how to set up a shell script to accept a positional parameter when called, pass that to a SQL script, accept the output from the statement back and then manipulate it in the Shell.
Viewers will learn how to properly install and use Secure Shell (SSH) to work on projects or homework remotely. Download Secure Shell: Follow basic installation instructions: Open Secure Shell and use "Quick Connect" to enter credentials includi…

743 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

13 Experts available now in Live!

Get 1:1 Help Now