uisng expect with sftp

I have some small script that logs into some sftp machine and asks us for login, i want to store the password and send it to the server using expet, this is our current code:

sftp user@ip > ./ftp_exti 2> ./ftp_error <<END_SCRIPT

 cd ../../a/b/
 ls -l
 cd ../../c/d/
 ls -l
 quit

END_SCRIPT

Open in new window



I want  to do something like this, but this is not working:


#!/usr/bin/expect -f

spawn sftp user@ip > ./ftp_exti 2> ./ftp_error <<END_SCRIPT

expect "*password"

send  -- "pass\r"

expect eof

 cd ../../a/b/
 ls -l
 cd ../../c/d/
 ls -l
 quit

END_SCRIPT

Open in new window

LVL 1
celticianAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

woolmilkporcCommented:
I assume public key authentication is still not an option?

Anyway.

You can't mix bash's "here" documents and expect. Your scripts runs entirely under expect, and expect doesn't know anything about "here" documents.

So you should do everything with expect, perhaps like this (example, modifed for my environment):

#!/usr/bin/expect -f

spawn sftp -o pubkeyauthentication=no localhost
expect "password"
send  -- "mypass\r"

expect "sftp>"
send --  "cd ../../\r"
expect "sftp>"
send --  "ls -l\r"
expect "sftp>"
send -- "cd /\r "
expect "sftp>"
send -- "ls -l\r"
expect "sftp>"
send -- "quit\r"

expect eof

exit
0
celticianAuthor Commented:
nope, public key is not an option here for many reasons

what do you mean with sftp -o pubkeyauthentication=no localhost??

where do i set the target server name here??
0
celticianAuthor Commented:
The first script is working, it only requires me to enter manually the password, i just want to set it to be sent using expet, changing the least possible things to the first script
0
Cloud Class® Course: SQL Server Core 2016

This course will introduce you to SQL Server Core 2016, as well as teach you about SSMS, data tools, installation, server configuration, using Management Studio, and writing and executing queries.

celticianAuthor Commented:
This is really weird, now when executing the script im getting this error:

ksh: sftpER_v1.sh:  not found


Looks like bash is not installed??
0
woolmilkporcCommented:
Please, let's proceed step by step.

1)

... -o pubkeyauthentication=no localhost

is, as I wrote. a modifcation required in my environment for testing. I have public key authentication everywhere, so I must switch it off voluntarily. I don't need a userid because it's the same on source and target, the more if using localhost.

sftp user@remotehost

 is the version you should use.

2)

When i wrote "You can't mix bash's "here" documents and expect" I was talking about the second script you posted. It will not work!

3)

>> now when executing the script << Which script, please?

"ftpER_v1.sh" looks as if two or more strings were overlaid. This can happen with expect and storing a result in a variable if the variable is not quoted. I don't think this is related to bash in any way.

But again, which script gives this "ftpER_v1.sh:  not found" result? And did you try my "expect only" suggestion?
0
celticianAuthor Commented:
When trying the all expect version you suggested me, its when i get that error, this is the modified code im executing:

#!/usr/bin/expect -f

spawn sftp user@ip > ./ftpexit 2> ./ftperror<<END_SCRIPT 

expect "user@ip's password:"

send -- "pass\r"


expect "sftp>"
send -- "cd ../../c/d/\r"
expect "sftp>"
send -- "ls -l\r"
expect "sftp>"
send -- "cd ../../a/b/\r"
expect "sftp>"
send -- "ls -l\r"
expect "sftp>"
send -- "quit\r"

expect eof


END_SCRIPT

Open in new window


i get this error:

ksh: script_name.sh:  not found

(the real name of the script is written up, but i shouldn't have written it).

anyway, the chmod to the file has been set to 777 too, just i case this might have to do.

What im i doing wrong?
0
woolmilkporcCommented:
This is wrong:

spawn sftp user@ip > ./ftpexit 2> ./ftperror <<END_SCRIPT

and this:

END_SCRIPT

As I wrote, expect cannot handle heredocs, and it cannot perform output redirection with ">" or ">>"

#!/usr/bin/expect -f

spawn sftp user@ip 

expect "user@ip's password:"

send -- "pass\r"

expect "sftp>"
send -- "cd ../../c/d/\r"
expect "sftp>"
send -- "ls -l\r"
expect "sftp>"
send -- "cd ../../a/b/\r"
expect "sftp>"
send -- "ls -l\r"
expect "sftp>"
send -- "quit\r"

expect eof

exit

Open in new window

If you must write a log file inside the script, please let me know. It's feasible!

And don't start the script with "sh scriptname".
It is not a shell script, it's an expect script (or better, expect is the shell, not bash or ksh).
Start it just with "./scriptname"
0
celticianAuthor Commented:
Ill check tomorrow, today i don't have access to this resource :(
0
celticianAuthor Commented:
Ok, this script is being called from another bigger one, and the main script has to read some values from these logs, thats why i should have them avaiable with that name for the main script...

I would like not to change anything in the main script if possible. This is how this is script is being invoked in the main one and how these logs are used:

  rm salidaftp
  rm errorftp
  ./sftpER_v1.sh
  chmod 777 salidaftp
  chmod 777 errorftp

ferror=`cat ./salidaftp | grep Error_SP_R_00001_$yesterday | grep -v ".done" | cut -c34-42`

Open in new window


where "/sftpER_v1.sh" is the our script

So isn't there any way of creating these logs just like they are being created now? (lets say with some other new external script to be called from the expect) ??
0
woolmilkporcCommented:
Why not just a tiny change to the main script:

  rm salidaftp
  rm errorftp
  ./sftpER_v1.sh > ./salidaftp 2>./errorftp
  chmod 777 salidaftp
  chmod 777 errorftp
ferror=`cat ./salidaftp | grep Error_SP_R_00001_$yesterday | grep -v ".done" | cut -c34-42`


The separation of stdout and stderr into two files will probably not work, because expect would combine both streams into one, but you can try. If in doubt use just one file:

 ./sftpER_v1.sh > ./salidaftp 2>&1

--------------

With a more comprehensive change to the main script we could make it much shorter:

ferror=`./sftpER_v1.sh 2>&1| grep Error_SP_R_00001_$yesterday | grep -v ".done" | cut -c34-42`

That's all, no "rm", no "chmod", no "cat"
--------------

OK, if you really don't want to change anything in the main script we must return our script to bash and embed the expect code via heredoc.
Please note: The heredoc is used to steer expect from bash, not to steer sftp from expect, as in your original attempt.

I admit, the below looks a bit weird, but it's the only simple way to achieve your goal I'm aware of.

#!/bin/bash

echo $(

/usr/bin/expect <<EOF

spawn sftp user@ip 

expect "user@ip's password:"

send -- "pass\r"

expect "sftp>"
send -- "cd ../../c/d/\r"
expect "sftp>"
send -- "ls -l\r"
expect "sftp>"
send -- "cd ../../a/b/\r"
expect "sftp>"
send -- "ls -l\r"
expect "sftp>"
send -- "quit\r"

expect eof

exit

EOF
) > ./salidaftp 2>./errorftp

Open in new window


Again, stdout and stderr of sftp can probably no longer be separated. "errorftp" will collect expect errors (if any), not sftp errors.
0
celticianAuthor Commented:
you seem to be correct, it doesn't accept two streams.

IM trying your second solutio with the changes to the main script, i hope it works... i will be able to test it in a few hours...

i dont mind making changes to the main script but i just didn't want to make it too difficult.
0
celticianAuthor Commented:
OK i have been able to check and i must be doing something wrong, it is not working as it should

it should be showing how many error are there in some file (i thinK) but it prints nothing

i compare it to the working script (previous version, prior to implementing the expect) and it should return some number, here it prints nothing.

The code is now:


Main script:

  ./sftpER_v1.sh > ./salidaftp 2>&1


ferror=`./sftpER_v1.sh 2>&1 | grep Error_SP_R_00001_$yesterday | grep -v ".done" | cut -c34-42`

Open in new window


Expect script:

#!/usr/bin/expect -f

spawn sftp username@machine

expect "username@machine's password:"

send -- "pass\r"



expect "sftp>"
send -- "cd ../../a/b/\r"
expect "sftp>"
send -- "ls -l\r"
expect "sftp>"
send -- "cd ../../c/d/\r"
expect "sftp>"
send -- "ls -l\r"
expect "sftp>"
send -- "quit\r"

expect eof

Open in new window




The log after execution leaves the next info, it seems it's not recognizing the file where we are calling the expec script:

machine:userName:/home/userName/scripts$ cat salidaftp
mainScript.sh[46]: ./sftpER_v1.sh:  not found

Open in new window

0
woolmilkporcCommented:
Three possible reasons:

1) The script  has a name other than sftpER_v1.sh

2) The location of sftpER_v1.sh is not the current directory (the one where you're positioned when starting the main script)

3) expect is not in /usr/bin or is not installed at all. Run "which expect".  What do you see?

Note: The "Main script" you posted must not call the script sftpER_v1.sh twice - line 1 is wrong!
0
celticianAuthor Commented:
Sorry for the time i took to reply, as i said before i dont have access to this resource everyday. Im back to this this week.

Replying to your reasons:

1)I have double checked and the script name is the same.

2)The location is the same directory for both scripts

3) expect is installed here: /usr/local/bin/expect

I have deleted the first call to the script sftpER_v1.sh leaving now only:

ferror=`./sftpER_v1.sh 2>&1 | grep Error_SP_R_00001_$yesterday | grep -v ".done" | cut -c34-42`
0
woolmilkporcCommented:
>> expect is installed here: /usr/local/bin/expect <<

And that's the reason for the "not found" error!

Change the first line of sftpER_v1.sh to

#!/usr/local/bin/expect -f
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
celticianAuthor Commented:
Now it works partially, the part it wasn't working, (the main script getting the file size from the file read by the second script). But ill post that as another question.

So the trouble was that.

Thank you!!!!!!
0
celticianAuthor Commented:
Very helpful!!
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Unix OS

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.