Second Remote Site Does Not Work via pFsense IPsec for phase 2

Chris Krueger
Chris Krueger used Ask the Experts™
on
Using pFsense at our main site and at a remote site. They both work on phase 1 and phase two over IPsec. Added a second IPsec phase 1 and phase 2 connection between a second remote site and only phase 1 tunnel connects, I cannot access any ips on the main site from my new remote site.

Is there an issue with trying to have two remote sites using the same remote IP subnet at the main site?
Main_Site_01_IPsec_Tunnels.png
Main_Site_01_IPsec_Status.png
Working_Remote_Site_01_IPsec_Tunnel.png
Working_Remote_Site_01_IPsec_Status.png
Failing_Remote_Site_02_IPsec_Tunnel.png
Failing_Remote_Site_02_IPsec_Status.png
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
nociSoftware Engineer
Distinguished Expert 2018

Commented:
Remote IP the same is the problem.....
(To many details missing from question).....

so say remote both are 192.168.10.0/24  

Then th pfSense firewall  needs to decide for each packet:    should i take the first or the second tunnel.
If you bring down the first and enable the 2nd, then the second tunnel will work and the first will fail.

The basic assumption on the IP protcol stack is that each address  define a UNIQUE system. So no doubles exist.... ( NAT make this a bit special) as some systems can hide behind one address. Systems behind NAT do not exist. (From a network point of view).

The best way to fix this is to change the remote address ranges to "relatively unique" ip ranges...
ie. set up one site as 192.168.42.0/24  and another as 192.168.57.0/24
SouljaSr.Net.Eng
Top Expert 2011

Commented:
If I am reading your question right. "Remote being the same" you are referring to the remote network from the spoke site perspectives? If so, that is not an issue. But to insure this is what you mean, is this your setup?

Addressing is for examples:

MAIN SITE                                    REMOTE 1
192.168.1.0                                  192.168.2.0
                                                      REMOTE 2
                                                      192.168.3.0
nociSoftware Engineer
Distinguished Expert 2018

Commented:
Try to avoid the common networks like (192.168.0/24, 192.168.1/0/24, 192.168.100.0/24, 192.168.254.0/24 , 192.168.255.0/24)  or 192.168.178.0/24... (default for Frtizbox).
Python 3 Fundamentals

This course will teach participants about installing and configuring Python, syntax, importing, statements, types, strings, booleans, files, lists, tuples, comprehensions, functions, and classes.

Author

Commented:
N. Spears, yes I believe we are talking the same thing. I am posting screen shots of the configuration for ease of conversation. Thank you.

Author

Commented:
Uploaded/Attached screen shots of the VPN configuration data.
SouljaSr.Net.Eng
Top Expert 2011

Commented:
For the new site do you have the pfsense to automatically NoNat for it, or have you added a NoNat rule for for the new vpn on each side?

Author

Commented:
We have not done NoNat on either remote site. This morning, I noticed I could no longer ping the working site and vise versa till I disconnected the non working IPsec tunnel, then the working site worked with ping again to 172.30.1.1 and 172.30.100.1.

When I say"working"vs not working, both remote connections establish a tunnel, only one can access 172.30.1.0/24 resources. The other one stops communicating with 172.30.1.0/24 resources at main campus "hub" location.
SouljaSr.Net.Eng
Top Expert 2011

Commented:
What is your IPSEC log showing for the new tunnel?

Author

Commented:
From "not working" remote pfsense box:

Time
Process
PID
Message
Mar 4 21:12:19
charon

00[CFG] loaded IKE secret for %any 216.21.169.55
Mar 4 21:12:19
charon

00[CFG] opening triplet file /usr/local/etc/ipsec.d/triplets.dat failed: No such file or directory
Mar 4 21:12:19
charon

00[CFG] loaded 0 RADIUS server configurations
Mar 4 21:12:19
charon

00[LIB] loaded plugins: charon unbound aes des blowfish rc2 sha2 sha1 md4 md5 random nonce x509 revocation constraints pubkey pkcs1 pkcs7 pkcs8 pkcs12 pgp dnskey sshkey ipseckey pem openssl fips-prf curve25519 xcbc cmac hmac curl attr kernel-pfkey kernel-pfroute resolve socket-default stroke vici updown eap-identity eap-sim eap-md5 eap-mschapv2 eap-dynamic eap-radius eap-tls eap-ttls eap-peap xauth-generic xauth-eap whitelist addrblock counters
Mar 4 21:12:19
charon

00[JOB] spawning 16 worker threads
Mar 4 21:12:19
ipsec_starter
95576
charon (95670) started after 40 ms
Mar 4 21:12:19
charon

04[CFG] received stroke: add connection 'bypasslan'
Mar 4 21:12:19
charon

04[CFG] conn bypasslan
Mar 4 21:12:19
charon

04[CFG] left=%any
Mar 4 21:12:19
charon

04[CFG] leftsubnet=192.168.200.0/24
Mar 4 21:12:19
charon

04[CFG] right=%any
Mar 4 21:12:19
charon

04[CFG] rightsubnet=192.168.200.0/24
Mar 4 21:12:19
charon

04[CFG] dpddelay=30
Mar 4 21:12:19
charon

04[CFG] dpdtimeout=150
Mar 4 21:12:19
charon

04[CFG] sha256_96=no
Mar 4 21:12:19
charon

04[CFG] mediation=no
Mar 4 21:12:19
charon

04[CFG] added configuration 'bypasslan'
Mar 4 21:12:19
charon

04[CFG] received stroke: route 'bypasslan'
Mar 4 21:12:19
ipsec_starter
95576
'bypasslan' shunt PASS policy installed
Mar 4 21:12:19
charon

15[CFG] received stroke: add connection 'con1000'
Mar 4 21:12:19
charon

15[CFG] conn con1000
Mar 4 21:12:19
charon

15[CFG] left=24.94.188.174
Mar 4 21:12:19
charon

15[CFG] leftsubnet=172.30.226.0/24
Mar 4 21:12:19
charon

15[CFG] leftauth=psk
Mar 4 21:12:19
charon

15[CFG] leftid=24.94.188.174
Mar 4 21:12:19
charon

15[CFG] right=216.21.169.55
Mar 4 21:12:19
charon

15[CFG] rightsubnet=172.30.1.0/24
Mar 4 21:12:19
charon

15[CFG] rightauth=psk
Mar 4 21:12:19
charon

15[CFG] rightid=216.21.169.55
Mar 4 21:12:19
charon

15[CFG] ike=aes256-sha256-modp2048!
Mar 4 21:12:19
charon

15[CFG] esp=aes128gcm128-sha256-modp2048,aes128gcm96-sha256-modp2048,aes128gcm64-sha256-modp2048!
Mar 4 21:12:19
charon

15[CFG] dpddelay=10
Mar 4 21:12:19
charon

15[CFG] dpdtimeout=60
Mar 4 21:12:19
charon

15[CFG] dpdaction=3
Mar 4 21:12:19
charon

15[CFG] sha256_96=no
Mar 4 21:12:19
charon

15[CFG] mediation=no
Mar 4 21:12:19
charon

15[CFG] keyexchange=ikev2
Mar 4 21:12:19
charon

15[CFG] added configuration 'con1000'
Mar 4 21:12:19
charon

14[CFG] received stroke: route 'con1000'
Mar 4 21:12:19
charon

14[CFG] configured proposals: ESP:AES_GCM_16_128/NO_EXT_SEQ, ESP:AES_GCM_12_128/NO_EXT_SEQ, ESP:AES_GCM_8_128/NO_EXT_SEQ
Mar 4 21:12:19
charon

14[CHD] CHILD_SA con1000{1} state change: CREATED => ROUTED
Mar 4 21:12:19
ipsec_starter
95576
'con1000' routed
Mar 4 21:12:32
charon

11[CFG] vici client 1 connected
Mar 4 21:12:32
charon

12[CFG] vici client 1 registered for: list-sa
Mar 4 21:12:32
charon

12[CFG] vici client 1 requests: list-sas
Mar 4 21:12:32
charon

10[CFG] vici client 1 disconnected
Mar 4 21:12:39
charon

07[CFG] vici client 2 connected
Mar 4 21:12:39
charon

08[CFG] vici client 2 registered for: list-sa
Mar 4 21:12:39
charon

07[CFG] vici client 2 requests: list-sas
Mar 4 21:12:39
charon

06[CFG] vici client 2 disconnected

Mar 4 21:12:19
charon

15[CFG] added configuration 'con1000'
Mar 4 21:12:19
charon

14[CFG] received stroke: route 'con1000'
Mar 4 21:12:19
charon

14[CFG] configured proposals: ESP:AES_GCM_16_128/NO_EXT_SEQ, ESP:AES_GCM_12_128/NO_EXT_SEQ, ESP:AES_GCM_8_128/NO_EXT_SEQ
Mar 4 21:12:19
charon

14[CHD] CHILD_SA con1000{1} state change: CREATED => ROUTED
Mar 4 21:12:19
ipsec_starter
95576
'con1000' routed
Mar 4 21:12:32
charon

11[CFG] vici client 1 connected
Mar 4 21:12:32
charon

12[CFG] vici client 1 registered for: list-sa
Mar 4 21:12:32
charon

12[CFG] vici client 1 requests: list-sas
Mar 4 21:12:32
charon

10[CFG] vici client 1 disconnected
Mar 4 21:12:39
charon

07[CFG] vici client 2 connected
Mar 4 21:12:39
charon

08[CFG] vici client 2 registered for: list-sa
Mar 4 21:12:39
charon

07[CFG] vici client 2 requests: list-sas
Mar 4 21:12:39
charon

06[CFG] vici client 2 disconnected
Mar 4 21:21:09
charon

12[KNL] interface usbus1 appeared
Mar 4 21:21:09
charon

12[KNL] interface usbus1 deactivated
Mar 4 21:21:09
charon

12[KNL] interface usbus1 disappeared
Mar 4 21:24:35
charon

09[KNL] creating acquire job for policy 24.94.188.174/32|/0 === 216.21.169.55/32|/0 with reqid {1}
Mar 4 21:24:35
charon

09[IKE] <con1000|1> queueing IKE_VENDOR task
Mar 4 21:24:35
charon

09[IKE] <con1000|1> queueing IKE_INIT task
Mar 4 21:24:35
charon

09[IKE] <con1000|1> queueing IKE_NATD task
Mar 4 21:24:35
charon

09[IKE] <con1000|1> queueing IKE_CERT_PRE task
Mar 4 21:24:35
charon

09[IKE] <con1000|1> queueing IKE_AUTH task
Mar 4 21:24:35
charon

09[IKE] <con1000|1> queueing IKE_CERT_POST task
Mar 4 21:24:35
charon

09[IKE] <con1000|1> queueing IKE_CONFIG task
Mar 4 21:24:35
charon

09[IKE] <con1000|1> queueing IKE_AUTH_LIFETIME task
Mar 4 21:24:35
charon

09[IKE] <con1000|1> queueing CHILD_CREATE task
Mar 4 21:24:35
charon

09[IKE] <con1000|1> activating new tasks
Mar 4 21:24:35
charon

09[IKE] <con1000|1> activating IKE_VENDOR task
Mar 4 21:24:35
charon

09[IKE] <con1000|1> activating IKE_INIT task
Mar 4 21:24:35
charon

09[IKE] <con1000|1> activating IKE_NATD task
Mar 4 21:24:35
charon

09[IKE] <con1000|1> activating IKE_CERT_PRE task
Mar 4 21:24:35
charon

09[IKE] <con1000|1> activating IKE_AUTH task
Mar 4 21:24:35
charon

09[IKE] <con1000|1> activating IKE_CERT_POST task
Mar 4 21:24:35
charon

09[IKE] <con1000|1> activating IKE_CONFIG task
Mar 4 21:24:35
charon

09[IKE] <con1000|1> activating CHILD_CREATE task
Mar 4 21:24:35
charon

09[IKE] <con1000|1> activating IKE_AUTH_LIFETIME task
Mar 4 21:24:35
charon

09[IKE] <con1000|1> initiating IKE_SA con1000[1] to 216.21.169.55
Mar 4 21:24:35
charon

09[IKE] <con1000|1> IKE_SA con1000[1] state change: CREATED => CONNECTING
Mar 4 21:24:35
charon

09[CFG] <con1000|1> configured proposals: IKE:AES_CBC_256/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_2048
Mar 4 21:24:35
charon

09[CFG] <con1000|1> sending supported signature hash algorithms: sha256 sha384 sha512 identity
Mar 4 21:24:35
charon

09[ENC] <con1000|1> generating IKE_SA_INIT request 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) N(FRAG_SUP) N(HASH_ALG) N(REDIR_SUP) ]
Mar 4 21:24:35
charon

09[NET] <con1000|1> sending packet: from 24.94.188.174[500] to 216.21.169.55[500] (464 bytes)
Mar 4 21:24:39
charon

09[IKE] <con1000|1> retransmit 1 of request with message ID 0
Mar 4 21:24:39
charon

09[NET] <con1000|1> sending packet: from 24.94.188.174[500] to 216.21.169.55[500] (464 bytes)
Mar 4 21:24:46
charon

09[IKE] <con1000|1> retransmit 2 of request with message ID 0
Mar 4 21:24:46
charon

09[NET] <con1000|1> sending packet: from 24.94.188.174[500] to 216.21.169.55[500] (464 bytes)
Mar 4 21:24:59
charon

09[IKE] <con1000|1> retransmit 3 of request with message ID 0
Mar 4 21:24:59
charon

09[NET] <con1000|1> sending packet: from 24.94.188.174[500] to 216.21.169.55[500] (464 bytes)
Mar 4 21:25:22
charon

09[IKE] <con1000|1> retransmit 4 of request with message ID 0
Mar 4 21:25:22
charon

09[NET] <con1000|1> sending packet: from 24.94.188.174[500] to 216.21.169.55[500] (464 bytes)

Author

Commented:
Status of IKE charon daemon (strongSwan 5.7.1, FreeBSD 11.2-RELEASE-p6, amd64):
  uptime: 4 minutes, since Mar 04 21:12:20 2019
  worker threads: 11 of 16 idle, 5/0/0/0 working, job queue: 0/0/0/0, scheduled: 0
  loaded plugins: charon unbound aes des blowfish rc2 sha2 sha1 md4 md5 random nonce x509 revocation constraints pubkey pkcs1 pkcs7 pkcs8 pkcs12 pgp dnskey sshkey ipseckey pem openssl fips-prf curve25519 xcbc cmac hmac curl attr kernel-pfkey kernel-pfroute resolve socket-default stroke vici updown eap-identity eap-sim eap-md5 eap-mschapv2 eap-dynamic eap-radius eap-tls eap-ttls eap-peap xauth-generic xauth-eap whitelist addrblock counters
Listening IP addresses:
  24.94.188.174
  192.168.200.1
  172.30.226.1
Connections:
Security Associations (0 up, 0 connecting):
  no match
 
 
 
  Shunted Connections:
   bypasslan:  192.168.200.0/24|/0 === 192.168.200.0/24|/0 PASS
Routed Connections:
     con1000{1}:  ROUTED, TUNNEL, reqid 1
     con1000{1}:   172.30.226.0/24|/0 === 172.30.1.0/24|/0
Security Associations (0 up, 0 connecting):
  none

Author

Commented:
Sorry everyone, I accidentally marked this solved last night.
SouljaSr.Net.Eng
Top Expert 2011

Commented:
What is this in the log?:

04[CFG] leftsubnet=192.168.200.0/24
Mar 4 21:12:19
charon

04[CFG] right=%any
Mar 4 21:12:19
charon

04[CFG] rightsubnet=192.168.200.0/24
Mar 4 21:12:19
charon

Author

Commented:
That is my lan interface I have a separate interface they is 172.30.226.0 that I was attempting to use for the vpn

Author

Commented:
I have a Wan Lan and Remote as three separate interfaces on the firewall the lan is 192.168.200.0 and the remote is 172.30.226.0 I have dhcp on lan and remote figure I could hook up pc to remote and isolate myself from my lan and still get vpn IPsec back to main hub
nociSoftware Engineer
Distinguished Expert 2018

Commented:
If left & right subnet are equal why would it transport data..., it looks like the tunnel is the same as a local interface.. One of which (tunnel/local) won't work.

netstat -rn  will show how packets get routed....

Author

Commented:
All:
I plan on getting back to this this evening or next. Thank you for all the help so far, more details to come.

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial