Solved

Rsync fails on Cisco 5505

Posted on 2009-07-04
21
1,772 Views
Last Modified: 2013-12-15
Hi,
I have a setup as per the attached diagram (note:IPs have been changed for security purposes).
The two servers are running RedHat Linux and Server 2 is set to synchronize with Server 1 using rsync. However rsync fails to transfer the files. Rsync uses SSH, which connect and authenticates well, but after that it just stops.
I have tried using sftp and scp between the two sites but they stop as well.
Using just ssh, I am able to connect between the two sites, but listing a long directory on lists the first few lines. Then I would have to reset the connection.

Below is a session I tried using scp with debug on.

I don't want to put anyone off-track, but I suspect that the "switch" on the ASA5505 is blocking rsync and other similar traffic. Could this be the case?

Thanks
Carmelo
 
linux-h4ti:~ # scp -v -C root@192.168.15.2:/boot/vmlinuz .
Executing: program /usr/bin/ssh host 192.168.15.2, user root, command scp -v -f /boot/vmlinuz
OpenSSH_5.0p1, OpenSSL 0.9.8g 19 Oct 2007
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug1: Connecting to 192.168.15.2 [192.168.15.2] port 22.
debug1: Connection established.
debug1: permanently_set_uid: 0/0
debug1: identity file /root/.ssh/id_rsa type -1
debug1: identity file /root/.ssh/id_dsa type -1
debug1: Remote protocol version 2.0, remote software version OpenSSH_5.0
debug1: match: OpenSSH_5.0 pat OpenSSH*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_5.0
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-cbc hmac-md5 zlib@openssh.com
debug1: kex: client->server aes128-cbc hmac-md5 zlib@openssh.com
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Host '192.168.15.2' is known and matches the RSA host key.
debug1: Found key in /root/.ssh/known_hosts:3
debug1: ssh_rsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,keyboard-interactive
debug1: Next authentication method: publickey
debug1: Trying private key: /root/.ssh/id_rsa
debug1: Trying private key: /root/.ssh/id_dsa
debug1: Next authentication method: keyboard-interactive
Password:
debug1: Enabling compression at level 6.
debug1: Authentication succeeded (keyboard-interactive).
debug1: channel 0: new [client-session]
debug1: Entering interactive session.
debug1: Sending environment.
debug1: Sending env LANG = POSIX
debug1: Sending env LC_CTYPE = en_US.UTF-8
debug1: Sending command: scp -v -f /boot/vmlinuz
Sending file modes: C0644 2106968 vmlinuz
Sink: C0644 2106968 vmlinuz
vmlinuz                                         0%    0     0.0KB/s - stalled -^C
debug1: channel 0: free: client-session, nchannels 1
debug1: fd 0 clearing O_NONBLOCK
debug1: fd 1 clearing O_NONBLOCK
Killed by signal 2.
linux-h4ti:~ #

Open in new window

SRV.png
0
Comment
Question by:cleversol
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 11
  • 10
21 Comments
 
LVL 13

Expert Comment

by:3nerds
ID: 24785654
What is the Private link and were does it connect to? Was this added as an additional VLAN on the ASA's? Do you have the base license for the ASA's or are they upgraded.

Also make sure that port 873 is open for rsync.

Regards,

3nerds
0
 

Author Comment

by:cleversol
ID: 24788432
The private link belongs to the ISP and I was told to regard it as a direct connect cable between the two sites. It is connected on the switch on the ASA5505 possibly on ethernet 0/2 (the servers are connected on ethernet 0/1 and the Internet is connected on ethernet 0/0).
I was told by the person who setup the ASA5505 that the traffic between ethernet 0/1 and ethernet 0/2 is not restricted in any way.

Of note: Prior to changing to this configuration, both ASA5505 were connected to the Internet (but with different public IPs). The direct link was not in place and rsync used to function properly over the internet.

The only changes made since then were:
1) Both ASA5505 now have the same public IP (but one is disconnected)
2) The direct link has been added
3) The keys for setting up ssh have been changed to reflect the new IPs

I hope this sheds some hint on what may be messing with the communications.

On a final note ... port 873??? Which of the participating equipment do you think is interfering with port 873?

Thanks and best regards
Carmelo
0
 

Author Comment

by:cleversol
ID: 24788674
Some more fresh information:
I have just tried using WinSCP to copy a file to Server2 while connected to the LAN using cisco vpn connected to ASA5505 on site 1. I managed to copy a file without a problem between my laptop and Server 2.

I have done the same thing with Server 1 and again there was no problem copying files using WinSCP to Server 1.

Thanks and regards
Carmelo
0
Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 13

Expert Comment

by:3nerds
ID: 24788688
Couple questions:

in an ASA5505 the interfaces don't make alot of difference it is specifically related to what vlan they are assigned to. There is a restriction in the base license the limits vlan functionality. How these ports are configured on the back end will determine what to look at next.

873 is the rsync port, it would need to be open in both directions.

2 things if you paid to have this setup make them come nback and fix it!!

the other is if you want futher help I am going to need to see the scrubbed configs.

Regards,

3nerds
0
 

Author Comment

by:cleversol
ID: 24788773
Thanks for a quick response. Here is the configuration of one of the ASA5505.

Carmelo
: Saved
:
ASA Version 8.0(3)
!
hostname Qormi-Side
domain-name worldofbets.com
enable password P/Ib.c5zgNP.q9EV encrypted
names
!
interface Vlan1
 description Internal Network
 nameif inside
 security-level 50
 ip address 192.168.15.101 255.255.255.0
!
interface Vlan2
 nameif outside
 security-level 0
 ip address 217.10.10.15 255.255.255.240
!
interface Ethernet0/0
 switchport access vlan 2
!
interface Ethernet0/1
!
interface Ethernet0/2
!
interface Ethernet0/3
!
interface Ethernet0/4
!
interface Ethernet0/5
!
interface Ethernet0/6
!
interface Ethernet0/7
!
passwd 2KFQnbNIdI.2KYOU encrypted
boot system disk0:/asa803-k8.bin
ftp mode passive
clock timezone CEST 1
clock summer-time CEDT recurring last Sun Mar 2:00 last Sun Oct 3:00
dns server-group DefaultDNS
 domain-name domain.com
same-security-traffic permit inter-interface
same-security-traffic permit intra-interface
object-group service VPN udp
 port-object eq 1194
object-group protocol TCPUDP
 protocol-object udp
 protocol-object tcp
access-list outside_access_in extended permit tcp any any eq ssh
access-list outside_access_in extended permit udp any any object-group VPN
access-list outside_access_in extended permit icmp any any
access-list inside_access_in extended permit ip any any
pager lines 24
logging enable
logging asdm informational
logging debug-trace
mtu inside 1500
mtu outside 1500
no failover
icmp unreachable rate-limit 1 burst-size 1
asdm image disk0:/asdm-603.bin
no asdm history enable
arp timeout 14400
global (outside) 1 interface
static (inside,outside) 217.10.10.15 192.168.15.2 netmask 255.255.255.255
static (outside,inside) 192.168.15.2 217.10.10.15 netmask 255.255.255.255
access-group inside_access_in in interface inside
access-group outside_access_in in interface outside
route outside 0.0.0.0 0.0.0.0 217.10.10.15 1
route inside 172.16.1.0 255.255.255.0 192.168.15.1 1
route inside 192.168.15.2 255.255.255.255 192.168.15.1 1
timeout xlate 3:00:00
timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 icmp 0:00:02
timeout sunrpc 0:10:00 h323 0:05:00 h225 1:00:00 mgcp 0:05:00 mgcp-pat 0:05:00
timeout sip 0:30:00 sip_media 0:02:00 sip-invite 0:03:00 sip-disconnect 0:02:00
timeout uauth 0:05:00 absolute
dynamic-access-policy-record DfltAccessPolicy
http server enable
http 172.16.1.0 255.255.255.0 inside
http 192.168.15.0 255.255.255.0 inside
http 217.168.162.53 255.255.255.255 outside
http 217.10.10.105 255.255.255.255 outside
no snmp-server location
no snmp-server contact
snmp-server enable traps snmp authentication linkup linkdown coldstart
sysopt noproxyarp inside
telnet timeout 5
ssh timeout 5
console timeout 0
dhcpd auto_config outside
!
 
no threat-detection basic-threat
threat-detection statistics port
threat-detection statistics protocol
threat-detection statistics access-list
!
!
prompt hostname context
Cryptochecksum:8bccb174ca0e900ce27786223b63150f
: end
asdm image disk0:/asdm-603.bin
no asdm history enable

Open in new window

0
 
LVL 13

Expert Comment

by:3nerds
ID: 24788910
So help me clear this setup in my head a bit. What is the Default Gateway of server B?


I have just tried using WinSCP to copy a file to Server2 while connected to the LAN using cisco vpn connected to ASA5505 on site 1. I managed to copy a file without a problem between my laptop and Server 2 -->> What address is you VPN client connecting to?


Possible  to see the other asa config as it appears to have alot more to it?

Regards,

3nerds
0
 

Author Comment

by:cleversol
ID: 24788990
Default route on server B is 192.168.15.101

The whole setup is designed for a manual failover. When something goes wrong on server A, the ISP is requested to disconnect the ASA5505 on site A and connect the one on site B to the Internet. Server B is mapped to the same public IP on ASA on site B as Server A is mapped on ASA at site A.

As I connect using cisco vpn on ASA at site A, I am unable to access the configuration on that ASA. But I believe that it is the almost the same as the one at site B save for IP mapping and the vpn setup.

Thanks for all your efforts
Carmelo
0
 
LVL 13

Expert Comment

by:3nerds
ID: 24789160
That for stating what I was assuming, good to have the details.

To be honest I have never plugged 2 asa 5505s in together like a switch to switch connection just never thought of it to be honest.

Couple of things that make me go hmm are as follows

1. I assume internet works everywhere because you didn't state that it was not working along with the fact that you can connect to both servers from VPN makes me think that is it fine. But the line that I question is that you have the following:

global (outside) 1 interface
 but you don't have the additional Nat statments to go along with is. This will not break rsync and is just an observation.

What should happen is the traffic needs to go out server hits asaB jumps to asaA and then talks to server1. The traffic flowing back takes the reverse course.

What make me pause on this is that what is really happening is it is hitting the first asa at site b, traveling from the port to vlan1 making a uturn and traveling out another port to asaA it repeats the process here. I am wondering if your snag is in the way an asa handles traffic and if it is getting hung up along the way.

Going to have to think about that one for a bit.

3nerds
0
 

Author Comment

by:cleversol
ID: 24789198
Thanks for all you help so far.
It is an unconventional setup, but in designing it we thought we could failover within a few minutes.

If a solution is not found, I will try to connect the servers directly through a second ethernet card available on the servers and which currently is set down. Having said that I prefer not to add any more configurations to the servers, as they were operating properly in the past.

Anyway, I will look forward to your response.

Thanks once again
Best regards
Carmelo
0
 
LVL 13

Expert Comment

by:3nerds
ID: 24795006
Cleversol,

I don't have 2 asa 5505s here to test this for you so I can't say for sure that it will work or not. I am a little superised that server B can browse the internet with it having a DG of .101 which is across the private link. When the asa says in its route to go out the outside locally connected interface to reach the internet.

I think that you are running into some of the limitations with u-turn traffic on the asa. Not to say it may not work eventually I am just not sure what additional pieces may need to be added to get the traffic flowing seamlessly. You initial thought that the asa switch was blocking is sort of right but it relates more to the fact that the asa is not a router and has problems with traffic flowing in and out of the same interface. There are work arounds but the ones that I know about don't really fit with what you are doing.

The only thought I had as a test was to shutdown vlan2 and test to see what happens. not sure it that is possibly confusing it at all. If it sdid help then you would just have to turn it up in a failure.

As a side not some of you nat pieces apear to be missing from this config should you ever need to fail over.

Good Luck,

3nerds
0
 
LVL 13

Expert Comment

by:3nerds
ID: 24795023
I am a little superised that server B can browse the internet with it having a DG of .101 which is across the private link. --> mis typed should read.

I am a little superised that server B can browse the internet with it having a DG of .101 which puts the internet across the private link. Would have thought it would need an address of .1 to work.

Regards,

3nerds
0
 

Author Comment

by:cleversol
ID: 24798162
Hello 3nerds, thanks for your comments.
Server B cannot access the Internet as it is in standby mode.  We had to change to this configuration because when it was connected to the Internet we were having double logs updates on the program running on server A. Server B becomes connected to the Internet only when the link on ASA B is connected to the Internet and after ASA A is disconnected.

We have created a manual fail over setup. In case of server A failure, techs at the ISP will be instructed to disconnect one ASA and connect the other one.

I will try to solve the problem by skipping the ASA and connect the two servers directly through the second nic card (presently unused) on each server.

I hope it works and I will come back here and let you know. If it works, then the problem would surely be with the ASA's.

Regards
Carmelol
0
 
LVL 13

Expert Comment

by:3nerds
ID: 24798388
I hope it goes well. Sorry I didn't have a better answer.

Regards,

3nerds
0
 

Author Comment

by:cleversol
ID: 24824105
Here's some more information.
I connected both servers directly using the link the ISP has provided and ....
The same problem persisted.

I have no idea what could be the problem, it simply does not make any sense.

Regards
Carmelo
0
 
LVL 13

Expert Comment

by:3nerds
ID: 24825768
Have you asked the ISP, it is possible they are blocking something.

3nerds
0
 

Author Comment

by:cleversol
ID: 24827070
The ISP promises that there are no controls on the interconnecting line.

Carmelo
0
 
LVL 13

Expert Comment

by:3nerds
ID: 24842608
Have you tested this link with any services other then rsync?

3nerds
0
 

Author Comment

by:cleversol
ID: 24842729
In fact I did and it turned out that the link cannot handle large packets. Any ping with a payload larger than 1468 bytes is silently discarded. I am still waiting for the ISP to solve the problem. I intend to let you know what is the result of a fix.

Thanks for your continued support
Carmelo
0
 
LVL 13

Expert Comment

by:3nerds
ID: 24842807
Glad to hear you are getting some answers.

Good Luck,

3nerds
0
 

Author Comment

by:cleversol
ID: 24885195
This question has been resubmitted with additional information here:
http://www.experts-exchange.com/Software/Internet_Email/File_Sharing/SSH_Telnet/Q_24580958.html

as it turned out that the problem is a server problem.

Thanks for all your help
Carmelo
0
 

Accepted Solution

by:
cleversol earned 0 total points
ID: 24932655
One final submission here and the solution.
The problem was solved just yesterday.
It was discovered that the Interlink does not support an MTU of 1500. The MTU was set to 1492.
Then both servers' MTU was set to the same value.

Everything started working fine after that.
Regards
Carmelo
0

Featured Post

Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Concerto Cloud Services, a provider of fully managed private, public and hybrid cloud solutions, announced today it was named to the 20 Coolest Cloud Infrastructure Vendors Of The 2017 Cloud  (http://www.concertocloud.com/about/in-the-news/2017/02/0…
For months I had no idea how to 'discover' the IP address of the other end of a link (without asking someone who knows), and it drove me batty. Think about it. You can't use Cisco Discovery Protocol (CDP) because it's not implemented on the ASAs.…
This demo shows you how to set up the containerized NetScaler CPX with NetScaler Management and Analytics System in a non-routable Mesos/Marathon environment for use with Micro-Services applications.
Both in life and business – not all partnerships are created equal. As the demand for cloud services increases, so do the number of self-proclaimed cloud partners. Asking the right questions up front in the partnership, will enable both parties …
Suggested Courses

738 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question