Solved

Rsync fails on Cisco 5505

Posted on 2009-07-04
21
1,719 Views
Last Modified: 2013-12-15
Hi,
I have a setup as per the attached diagram (note:IPs have been changed for security purposes).
The two servers are running RedHat Linux and Server 2 is set to synchronize with Server 1 using rsync. However rsync fails to transfer the files. Rsync uses SSH, which connect and authenticates well, but after that it just stops.
I have tried using sftp and scp between the two sites but they stop as well.
Using just ssh, I am able to connect between the two sites, but listing a long directory on lists the first few lines. Then I would have to reset the connection.

Below is a session I tried using scp with debug on.

I don't want to put anyone off-track, but I suspect that the "switch" on the ASA5505 is blocking rsync and other similar traffic. Could this be the case?

Thanks
Carmelo
 
linux-h4ti:~ # scp -v -C root@192.168.15.2:/boot/vmlinuz .

Executing: program /usr/bin/ssh host 192.168.15.2, user root, command scp -v -f /boot/vmlinuz

OpenSSH_5.0p1, OpenSSL 0.9.8g 19 Oct 2007

debug1: Reading configuration data /etc/ssh/ssh_config

debug1: Applying options for *

debug1: Connecting to 192.168.15.2 [192.168.15.2] port 22.

debug1: Connection established.

debug1: permanently_set_uid: 0/0

debug1: identity file /root/.ssh/id_rsa type -1

debug1: identity file /root/.ssh/id_dsa type -1

debug1: Remote protocol version 2.0, remote software version OpenSSH_5.0

debug1: match: OpenSSH_5.0 pat OpenSSH*

debug1: Enabling compatibility mode for protocol 2.0

debug1: Local version string SSH-2.0-OpenSSH_5.0

debug1: SSH2_MSG_KEXINIT sent

debug1: SSH2_MSG_KEXINIT received

debug1: kex: server->client aes128-cbc hmac-md5 zlib@openssh.com

debug1: kex: client->server aes128-cbc hmac-md5 zlib@openssh.com

debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent

debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP

debug1: SSH2_MSG_KEX_DH_GEX_INIT sent

debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY

debug1: Host '192.168.15.2' is known and matches the RSA host key.

debug1: Found key in /root/.ssh/known_hosts:3

debug1: ssh_rsa_verify: signature correct

debug1: SSH2_MSG_NEWKEYS sent

debug1: expecting SSH2_MSG_NEWKEYS

debug1: SSH2_MSG_NEWKEYS received

debug1: SSH2_MSG_SERVICE_REQUEST sent

debug1: SSH2_MSG_SERVICE_ACCEPT received

debug1: Authentications that can continue: publickey,keyboard-interactive

debug1: Next authentication method: publickey

debug1: Trying private key: /root/.ssh/id_rsa

debug1: Trying private key: /root/.ssh/id_dsa

debug1: Next authentication method: keyboard-interactive

Password:

debug1: Enabling compression at level 6.

debug1: Authentication succeeded (keyboard-interactive).

debug1: channel 0: new [client-session]

debug1: Entering interactive session.

debug1: Sending environment.

debug1: Sending env LANG = POSIX

debug1: Sending env LC_CTYPE = en_US.UTF-8

debug1: Sending command: scp -v -f /boot/vmlinuz

Sending file modes: C0644 2106968 vmlinuz

Sink: C0644 2106968 vmlinuz

vmlinuz                                         0%    0     0.0KB/s - stalled -^C

debug1: channel 0: free: client-session, nchannels 1

debug1: fd 0 clearing O_NONBLOCK

debug1: fd 1 clearing O_NONBLOCK

Killed by signal 2.

linux-h4ti:~ #

Open in new window

SRV.png
0
Comment
Question by:cleversol
  • 11
  • 10
21 Comments
 
LVL 13

Expert Comment

by:3nerds
ID: 24785654
What is the Private link and were does it connect to? Was this added as an additional VLAN on the ASA's? Do you have the base license for the ASA's or are they upgraded.

Also make sure that port 873 is open for rsync.

Regards,

3nerds
0
 

Author Comment

by:cleversol
ID: 24788432
The private link belongs to the ISP and I was told to regard it as a direct connect cable between the two sites. It is connected on the switch on the ASA5505 possibly on ethernet 0/2 (the servers are connected on ethernet 0/1 and the Internet is connected on ethernet 0/0).
I was told by the person who setup the ASA5505 that the traffic between ethernet 0/1 and ethernet 0/2 is not restricted in any way.

Of note: Prior to changing to this configuration, both ASA5505 were connected to the Internet (but with different public IPs). The direct link was not in place and rsync used to function properly over the internet.

The only changes made since then were:
1) Both ASA5505 now have the same public IP (but one is disconnected)
2) The direct link has been added
3) The keys for setting up ssh have been changed to reflect the new IPs

I hope this sheds some hint on what may be messing with the communications.

On a final note ... port 873??? Which of the participating equipment do you think is interfering with port 873?

Thanks and best regards
Carmelo
0
 

Author Comment

by:cleversol
ID: 24788674
Some more fresh information:
I have just tried using WinSCP to copy a file to Server2 while connected to the LAN using cisco vpn connected to ASA5505 on site 1. I managed to copy a file without a problem between my laptop and Server 2.

I have done the same thing with Server 1 and again there was no problem copying files using WinSCP to Server 1.

Thanks and regards
Carmelo
0
 
LVL 13

Expert Comment

by:3nerds
ID: 24788688
Couple questions:

in an ASA5505 the interfaces don't make alot of difference it is specifically related to what vlan they are assigned to. There is a restriction in the base license the limits vlan functionality. How these ports are configured on the back end will determine what to look at next.

873 is the rsync port, it would need to be open in both directions.

2 things if you paid to have this setup make them come nback and fix it!!

the other is if you want futher help I am going to need to see the scrubbed configs.

Regards,

3nerds
0
 

Author Comment

by:cleversol
ID: 24788773
Thanks for a quick response. Here is the configuration of one of the ASA5505.

Carmelo
: Saved

:

ASA Version 8.0(3)

!

hostname Qormi-Side

domain-name worldofbets.com

enable password P/Ib.c5zgNP.q9EV encrypted

names

!

interface Vlan1

 description Internal Network

 nameif inside

 security-level 50

 ip address 192.168.15.101 255.255.255.0

!

interface Vlan2

 nameif outside

 security-level 0

 ip address 217.10.10.15 255.255.255.240

!

interface Ethernet0/0

 switchport access vlan 2

!

interface Ethernet0/1

!

interface Ethernet0/2

!

interface Ethernet0/3

!

interface Ethernet0/4

!

interface Ethernet0/5

!

interface Ethernet0/6

!

interface Ethernet0/7

!

passwd 2KFQnbNIdI.2KYOU encrypted

boot system disk0:/asa803-k8.bin

ftp mode passive

clock timezone CEST 1

clock summer-time CEDT recurring last Sun Mar 2:00 last Sun Oct 3:00

dns server-group DefaultDNS

 domain-name domain.com

same-security-traffic permit inter-interface

same-security-traffic permit intra-interface

object-group service VPN udp

 port-object eq 1194

object-group protocol TCPUDP

 protocol-object udp

 protocol-object tcp

access-list outside_access_in extended permit tcp any any eq ssh

access-list outside_access_in extended permit udp any any object-group VPN

access-list outside_access_in extended permit icmp any any

access-list inside_access_in extended permit ip any any

pager lines 24

logging enable

logging asdm informational

logging debug-trace

mtu inside 1500

mtu outside 1500

no failover

icmp unreachable rate-limit 1 burst-size 1

asdm image disk0:/asdm-603.bin

no asdm history enable

arp timeout 14400

global (outside) 1 interface

static (inside,outside) 217.10.10.15 192.168.15.2 netmask 255.255.255.255

static (outside,inside) 192.168.15.2 217.10.10.15 netmask 255.255.255.255

access-group inside_access_in in interface inside

access-group outside_access_in in interface outside

route outside 0.0.0.0 0.0.0.0 217.10.10.15 1

route inside 172.16.1.0 255.255.255.0 192.168.15.1 1

route inside 192.168.15.2 255.255.255.255 192.168.15.1 1

timeout xlate 3:00:00

timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 icmp 0:00:02

timeout sunrpc 0:10:00 h323 0:05:00 h225 1:00:00 mgcp 0:05:00 mgcp-pat 0:05:00

timeout sip 0:30:00 sip_media 0:02:00 sip-invite 0:03:00 sip-disconnect 0:02:00

timeout uauth 0:05:00 absolute

dynamic-access-policy-record DfltAccessPolicy

http server enable

http 172.16.1.0 255.255.255.0 inside

http 192.168.15.0 255.255.255.0 inside

http 217.168.162.53 255.255.255.255 outside

http 217.10.10.105 255.255.255.255 outside

no snmp-server location

no snmp-server contact

snmp-server enable traps snmp authentication linkup linkdown coldstart

sysopt noproxyarp inside

telnet timeout 5

ssh timeout 5

console timeout 0

dhcpd auto_config outside

!
 

no threat-detection basic-threat

threat-detection statistics port

threat-detection statistics protocol

threat-detection statistics access-list

!

!

prompt hostname context

Cryptochecksum:8bccb174ca0e900ce27786223b63150f

: end

asdm image disk0:/asdm-603.bin

no asdm history enable

Open in new window

0
 
LVL 13

Expert Comment

by:3nerds
ID: 24788910
So help me clear this setup in my head a bit. What is the Default Gateway of server B?


I have just tried using WinSCP to copy a file to Server2 while connected to the LAN using cisco vpn connected to ASA5505 on site 1. I managed to copy a file without a problem between my laptop and Server 2 -->> What address is you VPN client connecting to?


Possible  to see the other asa config as it appears to have alot more to it?

Regards,

3nerds
0
 

Author Comment

by:cleversol
ID: 24788990
Default route on server B is 192.168.15.101

The whole setup is designed for a manual failover. When something goes wrong on server A, the ISP is requested to disconnect the ASA5505 on site A and connect the one on site B to the Internet. Server B is mapped to the same public IP on ASA on site B as Server A is mapped on ASA at site A.

As I connect using cisco vpn on ASA at site A, I am unable to access the configuration on that ASA. But I believe that it is the almost the same as the one at site B save for IP mapping and the vpn setup.

Thanks for all your efforts
Carmelo
0
 
LVL 13

Expert Comment

by:3nerds
ID: 24789160
That for stating what I was assuming, good to have the details.

To be honest I have never plugged 2 asa 5505s in together like a switch to switch connection just never thought of it to be honest.

Couple of things that make me go hmm are as follows

1. I assume internet works everywhere because you didn't state that it was not working along with the fact that you can connect to both servers from VPN makes me think that is it fine. But the line that I question is that you have the following:

global (outside) 1 interface
 but you don't have the additional Nat statments to go along with is. This will not break rsync and is just an observation.

What should happen is the traffic needs to go out server hits asaB jumps to asaA and then talks to server1. The traffic flowing back takes the reverse course.

What make me pause on this is that what is really happening is it is hitting the first asa at site b, traveling from the port to vlan1 making a uturn and traveling out another port to asaA it repeats the process here. I am wondering if your snag is in the way an asa handles traffic and if it is getting hung up along the way.

Going to have to think about that one for a bit.

3nerds
0
 

Author Comment

by:cleversol
ID: 24789198
Thanks for all you help so far.
It is an unconventional setup, but in designing it we thought we could failover within a few minutes.

If a solution is not found, I will try to connect the servers directly through a second ethernet card available on the servers and which currently is set down. Having said that I prefer not to add any more configurations to the servers, as they were operating properly in the past.

Anyway, I will look forward to your response.

Thanks once again
Best regards
Carmelo
0
 
LVL 13

Expert Comment

by:3nerds
ID: 24795006
Cleversol,

I don't have 2 asa 5505s here to test this for you so I can't say for sure that it will work or not. I am a little superised that server B can browse the internet with it having a DG of .101 which is across the private link. When the asa says in its route to go out the outside locally connected interface to reach the internet.

I think that you are running into some of the limitations with u-turn traffic on the asa. Not to say it may not work eventually I am just not sure what additional pieces may need to be added to get the traffic flowing seamlessly. You initial thought that the asa switch was blocking is sort of right but it relates more to the fact that the asa is not a router and has problems with traffic flowing in and out of the same interface. There are work arounds but the ones that I know about don't really fit with what you are doing.

The only thought I had as a test was to shutdown vlan2 and test to see what happens. not sure it that is possibly confusing it at all. If it sdid help then you would just have to turn it up in a failure.

As a side not some of you nat pieces apear to be missing from this config should you ever need to fail over.

Good Luck,

3nerds
0
6 Surprising Benefits of Threat Intelligence

All sorts of threat intelligence is available on the web. Intelligence you can learn from, and use to anticipate and prepare for future attacks.

 
LVL 13

Expert Comment

by:3nerds
ID: 24795023
I am a little superised that server B can browse the internet with it having a DG of .101 which is across the private link. --> mis typed should read.

I am a little superised that server B can browse the internet with it having a DG of .101 which puts the internet across the private link. Would have thought it would need an address of .1 to work.

Regards,

3nerds
0
 

Author Comment

by:cleversol
ID: 24798162
Hello 3nerds, thanks for your comments.
Server B cannot access the Internet as it is in standby mode.  We had to change to this configuration because when it was connected to the Internet we were having double logs updates on the program running on server A. Server B becomes connected to the Internet only when the link on ASA B is connected to the Internet and after ASA A is disconnected.

We have created a manual fail over setup. In case of server A failure, techs at the ISP will be instructed to disconnect one ASA and connect the other one.

I will try to solve the problem by skipping the ASA and connect the two servers directly through the second nic card (presently unused) on each server.

I hope it works and I will come back here and let you know. If it works, then the problem would surely be with the ASA's.

Regards
Carmelol
0
 
LVL 13

Expert Comment

by:3nerds
ID: 24798388
I hope it goes well. Sorry I didn't have a better answer.

Regards,

3nerds
0
 

Author Comment

by:cleversol
ID: 24824105
Here's some more information.
I connected both servers directly using the link the ISP has provided and ....
The same problem persisted.

I have no idea what could be the problem, it simply does not make any sense.

Regards
Carmelo
0
 
LVL 13

Expert Comment

by:3nerds
ID: 24825768
Have you asked the ISP, it is possible they are blocking something.

3nerds
0
 

Author Comment

by:cleversol
ID: 24827070
The ISP promises that there are no controls on the interconnecting line.

Carmelo
0
 
LVL 13

Expert Comment

by:3nerds
ID: 24842608
Have you tested this link with any services other then rsync?

3nerds
0
 

Author Comment

by:cleversol
ID: 24842729
In fact I did and it turned out that the link cannot handle large packets. Any ping with a payload larger than 1468 bytes is silently discarded. I am still waiting for the ISP to solve the problem. I intend to let you know what is the result of a fix.

Thanks for your continued support
Carmelo
0
 
LVL 13

Expert Comment

by:3nerds
ID: 24842807
Glad to hear you are getting some answers.

Good Luck,

3nerds
0
 

Author Comment

by:cleversol
ID: 24885195
This question has been resubmitted with additional information here:
http://www.experts-exchange.com/Software/Internet_Email/File_Sharing/SSH_Telnet/Q_24580958.html

as it turned out that the problem is a server problem.

Thanks for all your help
Carmelo
0
 

Accepted Solution

by:
cleversol earned 0 total points
ID: 24932655
One final submission here and the solution.
The problem was solved just yesterday.
It was discovered that the Interlink does not support an MTU of 1500. The MTU was set to 1492.
Then both servers' MTU was set to the same value.

Everything started working fine after that.
Regards
Carmelo
0

Featured Post

Free Trending Threat Insights Every Day

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

Join & Write a Comment

Suggested Solutions

Title # Comments Views Activity
shrewsoft VPN client and DNS 6 46
PC upgrade to Linux Mint 7 33
cisco VIRL 3 13
nagios 1 5
Little introduction about CP: CP is a command on linux that use to copy files and folder from one location to another location. Example usage of CP as follow: cp /myfoder /pathto/destination/folder/ cp abc.tar.gz /pathto/destination/folder/ab…
I recently attended Cisco Live! in Las Vegas, a conference that boasted over 28,000 techies in attendance, and a week of hands-on learning hosted by a solid partner with which Concerto goes to market.  Every year, Cisco displays cutting-edge technol…
Get a first impression of how PRTG looks and learn how it works.   This video is a short introduction to PRTG, as an initial overview or as a quick start for new PRTG users.
This demo shows you how to set up the containerized NetScaler CPX with NetScaler Management and Analytics System in a non-routable Mesos/Marathon environment for use with Micro-Services applications.

706 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

18 Experts available now in Live!

Get 1:1 Help Now