FTP to SCP shell script

you need to set trust (no password) between the systems:

http://waelchatila.com/2005/06/06/1118124232757.html
http://www.cvrti.utah.edu/~dustman/no-more-pw-ssh/

OR

use expect:

https://www.experts-exchange.com/questions/23459740/Running-command-or-script-on-remote-system.html

If you set trust between the systems, then run

scp username@server1/tmp/scptest.txt /path/on/your/server

for expect, please see link provided

There's no problem in using the keys  - Add your public key into your 'authorized_keys' and leave the keys where they are (all  files commonly in your~/.ssh directory.
And - take care to have correct permissions - 0600 for your private key and and the 'authorized keys' !
 
wmp

woolmilkporc, i tried doing the following:
1. generated the keys for the same ID in both servers.
2. updated the same in 'authorized_keys'
3. chmod 600...
4. scp user@server1:/tmp/file .

It didnt promt for a password but the file is not transfered. Further, i think the .cshrc is called. Cud u give me an insight of what is happening.

Did you get an error message?
Try scp -v user@server1:/tmp/file .

to see a bit more.
 
I don't know whether .cshrc gets called, but I do think it wouldn't matter.
 
wmp
 

This is what generally happens for authentication (from ssh man page):

          The file $HOME/.ssh/authorized_keys lists the public keys
          that are permitted for logging in.  When the user logs in,
          the ssh program tells the server which key pair it would
          like to use for authentication.  The server checks if this
          key is permitted, and if so, sends the user (actually the
          ssh program running on behalf of the user) a challenge, a
          random number, encrypted by the user's public key.  The
          challenge can only be decrypted using the proper private
          key.  The user's client then decrypts the challenge using
          the private key, proving that he/she knows the private key
          but without disclosing it to the server.

I tried scp-ing with -v . I do see some debug statements. No obvious errors. But not sure about the following lines shown below.

host1:/tmp>scp -v /tmp/scptest.txt user@server1:/home/user/anish/scptry
........
debug1: Entering interactive session.
debug1: Sending command: scp -v -t /home/user/anish/scptry
Welcome Ramesh... from cshrc This time is Tue Nov 11 08:14:44 EST 2008
host1:/tmp> debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
debug1: channel 0: free: client-session, nchannels 1
debug1: fd 0 clearing O_NONBLOCK
debug1: fd 1 clearing O_NONBLOCK
debug1: Transferred: stdin 0, stdout 0, stderr 0 bytes in 1.2 seconds
debug1: Bytes per second: stdin 0.0, stdout 0.0, stderr 0.0
debug1: Exit status 0

What is the Interactive session about in the middle? and 'Welcome Ramesh... from cshrc This time is Tue Nov 11 08:14:44 EST 2008' is from my .cshrc.

Long story short. Transfer still not working.

can you try sftp interactivity to see if you can login and get the file?

The 'interactive session ..' message is normal.
But there really seems to be a problem with your .cshrc.
I'am a bit astonished about the 'host1:/tmp> ' prompt suddenly appearing.
Could you please, just for testing, rename your .cshrc and try again?
If the copy then succeeds, please post your .cshrc. Let's look what strange things might be in there ...
 
wmp

Renaming .cshrc and scp-ing did the trick. I got it transfered .Thanks a lot but give me a solutions to fix my .cshrc also. And why is my .cshrc called in between? My .cshrc is ....

echo "Welcome Ramesh... from cshrc This time is `date`"
source /home/algo/MRS_ENVIRONMENT/common.cshrc
set prompt="`hostname`:`pwd`> "
alias cd 'cd \!*;set prompt="`hostname`:`pwd`> "'

setenv ASPEED_ROOT /prod/alg/linux_vendor/aspeed
setenv ASPEED_WB_HOST xxxxxxxxx
setenv ASPEED_CONFIG /prod/alg/linux_vendor/aspeed/config/config.user.txt

what do you have in /home/algo/MRS_ENVIRONMENT/common.cshrc ?

I do have one. I don think i will be able share the whole file. if you want me to check for a particular call or some thing specific, i can check and get back to you. But why is my .cshrc called?

when a csh is run (which is needed for login) it reads .login and .cshrc and execute commands found in them.

Can you mask info in the file and share rest of it?

I think .login is executed only in an interactive shell, in contrast to .cshrc. Could you place all commands which produce terminal output (echo ...) there instead of having them in .cshrc?
Sure wee need to know whether  /home/algo/MRS_ENVIRONMENT/common.cshrc
does produce terminal output, too.

Yes you are right. It is because of the echos. I tried commending echos and transfer works fine for me. but i will not be able to do the same in production. i will have to come up with some creative mechanism to transfer with all the echos or bypass .cshrc call. Can you help me out with this ?

Did you already try to put the echos in a .login file and eliminate them in .cshrc?

I didnt put the echos anywhere. i removeed it and the transfer worked. Now, the prod admin will have the same copy of .cshrc. I will not be able to ask em to change his .cshrc inorder to run my script.

did you try to use sftp ?

If you can't change anything at all in your environment, and if you must continue using csh (ksh works very much better for this case), and if your transfer must be ssh-based, I fear there will be no other solution as the one omarfarid is keeping to suggest - sftp

When i try to sftp, i get the following message:
connecting to server1...
Received message too long 1466264675

But if i commend out the following lines(first two lines) in cshrc, i am able to sftp.
echo "Welcome Ramesh... from cshrc This time is `date`"
source /home/algo/MRS_ENVIRONMENT/common.cshrc

So, go and see your 'prod admin' and have a discussion with /her/him about .cshrc etc.

ok. Let me have a discussion. But do inform me in case you can find a way out without altering .cshrc. Thanks.

I think I found a solution to this problem. Here, I do a test to see if .cshrc is working with terminal (which means an interactive / login session) or not.

tty | grep /dev > /dev/null
if ( $status == 0 ) then
echo hi
endif

I tested this with sftp and scp, and it is working fine.

here you just put your statements between the if and endif

Sorry, but I forgot to say put this in .cshrc file

I would have suggested a similar thing (e.g. testing $prompt), but since Franklinraj22 doesn't want to modify .cshrc ...

Hi woolmilkporc,

Thank you for the comment, the question in not whither we want to change or not, but it is what to change.

The test I am doing will not affect the application requirements, it is just a check to avoid the echo statements that is causing the problem.

Thanks,

Thanks omarfarid. the solution you gave did work. But again as woolmilkporc said, it wud be better if i can find a way out without editing my .cshrc.
In the scenario i am in, if my solution incudes a change in my .cshrc then I will have to ask all the users to change their .cshrc files including the production admin's. I will end up seeking for too many permission and aprovals.

I'm note sure if it will work, but you can try

scp -n /tmp/scptest.txt user@server1:/home/user/anish/scptry

Sorry, ignore my last suggestion.

Are you going to talk to your application provider / developer to see if changes can be done in .cshrc? if yes, then you need to change all users' .cshrc files.

How many users really need to run scp?
In the question I read 'scripts that use FTP ' - normally such scripts are only run by a few 'utility' userids. Modify .cshrc of those users only, or better, move them to ksh.
 
wmp

What I deal with is a overnight batch system. This batch runs hundreds of scripts and some of them uses ftp and this is to be changed to scp.
Now let me tell why i should change all .cshrc files. Each developer has a user ID and an environment similar to  production in the development server. This development environment can be resynced to any paricular date, make changes and full batch test can be conducted with the developer's userID. This full batch test results have to be send for approval for any task. Here is my need to change all .cshrc files of all the users. Else he/she will not be able to run the full batch test which happens for most of the tests.

Hi FranklinRaj22

If my solution remains the only one, how come it was not accepted as the solution or part of the solution !