projects
asked on
centos7 firewalld udp ports
Running nmap from a remote location shows those ports are closed? What's going on and how do I fix it.
In public.xml
In public.xml
<port protocol="udp" port="20128"/>
<port protocol="udp" port="20129"/>
<port protocol="udp" port="20130"/>
<port protocol="udp" port="20131"/># firewall-cmd --zone=public --list-all
public (default, active)
interfaces: enp1s0f1
sources:
services: dhcpv6-client http https
ports: 20128/udp 20130/udp 20129/udp 20131/udp
masquerade: no
forward-ports:
icmp-blocks:
rich rules:
***SNIP***$ nmap -p 50128 -sU -P0 x.x.x.x
Host is up (0.00031s latency).
PORT STATE SERVICE
50128/udp closed unknown
$ nmap -p 50129 -sU -P0 x.x.x.x
Host is up (0.00034s latency).
PORT STATE SERVICE
50129/udp closed unknown# nmap -p 50120 -sU -P0 x.x.x.x
PORT STATE SERVICE
50120/udp filtered unknownASKER
# firewall-cmd --zone=public --list-all
public (default, active)
interfaces: enp1s0f1
sources:
services: dhcpv6-client http https
ports: 20128-20131/udp
masquerade: no
forward-ports:
icmp-blocks:
rich rules:
rule family="ipv4" source address="x.x.x.x/32" port port="3306" protocol="tcp" accept
rule family="ipv4" source address="x.x.x.x/32" port port="3128" protocol="tcp" accept
I'm puzzled. Your open ports are in the 20000 range and your nmap is in the 50000 range.
ASKER
Sorry, they are actually all in the 50xxx range.
According to the snippet above, they have not been added.
ASKER
Sorry for the confusion, I simply obfuscated the ports for the post, they are all in the same range, the rule and the test.
I cannot go back to edit now so just keep in mind both rules and testing are the same ports no matter what is shown in this question.
I cannot go back to edit now so just keep in mind both rules and testing are the same ports no matter what is shown in this question.
How about you private message me the actual information along with any public IPs and let me take a look. I'll answer up here with no private information. I'll be unavailable for a couple of hours but will check as soon as I can.
ASKER
I just use random ports when posting unless they are common such as http/s.
It's exactly as I posted, only the ports were changed. Let's just start from here.
So, as you wanted;
# firewall-cmd --zone=public --list-all
public (default, active)
interfaces: enp1s0f1
sources:
services: dhcpv6-client http https
ports: 50128-50131/udp
masquerade: no
forward-ports:
icmp-blocks:
rich rules:
rule family="ipv4" source address="x.x.x.x/32" port port="3306" protocol="tcp" accept
rule family="ipv4" source address="x.x.x.x/32" port port="3128" protocol="tcp" accept
It's exactly as I posted, only the ports were changed. Let's just start from here.
So, as you wanted;
# firewall-cmd --zone=public --list-all
public (default, active)
interfaces: enp1s0f1
sources:
services: dhcpv6-client http https
ports: 50128-50131/udp
masquerade: no
forward-ports:
icmp-blocks:
rich rules:
rule family="ipv4" source address="x.x.x.x/32" port port="3306" protocol="tcp" accept
rule family="ipv4" source address="x.x.x.x/32" port port="3128" protocol="tcp" accept
ASKER
Seems maybe 'closed unknown' doesn't mean much as I am seeing another application able to connect using those ports.
On the other hand, some cannot and they are getting failed.
From any location on the Internet, I can ping the server and get normal responses.
From some LANs I am testing from, all ping requests with ascending TTL values return "ICMP Unreachable" rather than "Timed Out".
What could be changing those "Timed Out" responses (type 11) to "Unreachable" (type 3).
Could it be the machine I am testing from as the network is a bit strange, the NIC is a "vEthernet ... Virtual Switch" due to the Hyper Visor.
On the other hand, some cannot and they are getting failed.
From any location on the Internet, I can ping the server and get normal responses.
From some LANs I am testing from, all ping requests with ascending TTL values return "ICMP Unreachable" rather than "Timed Out".
What could be changing those "Timed Out" responses (type 11) to "Unreachable" (type 3).
Could it be the machine I am testing from as the network is a bit strange, the NIC is a "vEthernet ... Virtual Switch" due to the Hyper Visor.
ASKER
Watching communications using tcpdump and seeing;
19:47:14.793213 IP (tos 0x0, ttl 1, id 2289, offset 0, flags [DF], proto UDP (17), length 38)
x.x.x.82.46223 > x.x.x.13.50128: UDP, length 10
19:47:14.802103 IP (tos 0x0, ttl 2, id 2290, offset 0, flags [DF], proto UDP (17), length 38)
x.x.x.82.33611 > x.x.x.13.50128: UDP, length 10
19:47:14.807118 IP (tos 0x0, ttl 3, id 2291, offset 0, flags [DF], proto UDP (17), length 38)
x.x.x.82.51475 > x.x.x.13.50128: UDP, length 10
19:47:14.813121 IP (tos 0x0, ttl 4, id 2292, offset 0, flags [DF], proto UDP (17), length 38)
x.x.x.82.33021 > x.x.x.13.50128: UDP, length 10
19:47:14.829141 IP (tos 0x0, ttl 5, id 2293, offset 0, flags [DF], proto UDP (17), length 38)
x.x.x.82.49140 > x.x.x.13.50128: UDP, length 10
Not 100% sure of what the output is since I don't use tcpdump very much. Seems to be full two way communications no?
19:47:14.793213 IP (tos 0x0, ttl 1, id 2289, offset 0, flags [DF], proto UDP (17), length 38)
x.x.x.82.46223 > x.x.x.13.50128: UDP, length 10
19:47:14.802103 IP (tos 0x0, ttl 2, id 2290, offset 0, flags [DF], proto UDP (17), length 38)
x.x.x.82.33611 > x.x.x.13.50128: UDP, length 10
19:47:14.807118 IP (tos 0x0, ttl 3, id 2291, offset 0, flags [DF], proto UDP (17), length 38)
x.x.x.82.51475 > x.x.x.13.50128: UDP, length 10
19:47:14.813121 IP (tos 0x0, ttl 4, id 2292, offset 0, flags [DF], proto UDP (17), length 38)
x.x.x.82.33021 > x.x.x.13.50128: UDP, length 10
19:47:14.829141 IP (tos 0x0, ttl 5, id 2293, offset 0, flags [DF], proto UDP (17), length 38)
x.x.x.82.49140 > x.x.x.13.50128: UDP, length 10
Not 100% sure of what the output is since I don't use tcpdump very much. Seems to be full two way communications no?
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
I'm not 100% what should be listening for this. I believe it's just a server sided icmp or something. I'm not sure since the dev is not around.
netstat -an | grep 501
Shows nothing.
You've given me a lead that I didn't have before with the above and by telling me that the results show you there is no response.
netstat -an | grep 501
Shows nothing.
You've given me a lead that I didn't have before with the above and by telling me that the results show you there is no response.
If you expect a connection to a local port, then the service must be listening on that port.
An empty netstat suggests that the service/application is not running and/or failed to start.
An empty netstat suggests that the service/application is not running and/or failed to start.
ASKER
The app is definitely running but we're wondering if the php composer is missing something or got messed up when it was updated.
Is there some manual way of opening a port without a service, maybe a small script, in order to test for a connection and response.
Is there some manual way of opening a port without a service, maybe a small script, in order to test for a connection and response.
There is. I don't do programming to open a socket, but that's what you need.
ASKER
So, on the server, I am running the following;
nc -l -u 50127
On a remote, I am trying to reach that port but nothing responds.
nc -u x.x.x.x 50127
This must be a firewall issue or something else on the server blocking.
nc -l -u 50127
On a remote, I am trying to reach that port but nothing responds.
nc -u x.x.x.x 50127
This must be a firewall issue or something else on the server blocking.
Can you run a wireshark (not tcpdump) in verbose mode? If you want to post that privately, I would rather not deal with obfuscated information.
Or you can IM me the public IP and I can run wireshark at my end.
Or you can IM me the public IP and I can run wireshark at my end.
ASKER
I'm assuming you mean running wireshark on the server, which I'll have to install.
However, as I mentioned, I've got nc opened to a port which is not responding. I also tested on one of the ports I'm actually trying to use, which is allowed in firewalld but no response.
I cannot share the public IP, sorry. I work for others and even privately, they are very unwilling to share such info.
However, as I mentioned, I've got nc opened to a port which is not responding. I also tested on one of the ports I'm actually trying to use, which is allowed in firewalld but no response.
I cannot share the public IP, sorry. I work for others and even privately, they are very unwilling to share such info.
ASKER
Hang on now, using nc is not doing anything, no response what so ever, even to ports 80/443, standard ports.
I then used nmap instead;
# nmap -sU -p 50127-50131 x.x.x.x
Starting Nmap 6.40 ( http://nmap.org ) at 2016-08-19 11:14 MST
Nmap scan report for xx (x.x.x.x)
Host is up (0.032s latency).
PORT STATE SERVICE
50127/udp filtered unknown
50128/udp closed unknown
50129/udp open|filtered unknown
50130/udp closed unknown
50131/udp closed unknown
Note that only ports 50128-50131 are actually configured in the firewall.
I then used nmap instead;
# nmap -sU -p 50127-50131 x.x.x.x
Starting Nmap 6.40 ( http://nmap.org ) at 2016-08-19 11:14 MST
Nmap scan report for xx (x.x.x.x)
Host is up (0.032s latency).
PORT STATE SERVICE
50127/udp filtered unknown
50128/udp closed unknown
50129/udp open|filtered unknown
50130/udp closed unknown
50131/udp closed unknown
Note that only ports 50128-50131 are actually configured in the firewall.
50129 is the only port open but is restricted either in a firewall rule that I can't see or the app itself.
ASKER
I posted the firewall rule at the top of this. Ports 50128-50131 are configured.
What I'm not understanding is that some connections are actually reaching those ports while most are not.
In my tcpdump, what is it that shows you there is no reply?
I'm looking at the captures of those which did get a response and they don't look any different.
I ran the same command and am seeing different results.
PORT STATE SERVICE
50127/udp open|filtered unknown
50128/udp closed unknown
50129/udp open|filtered unknown
50130/udp open|filtered unknown
50131/udp closed unknown
Is tcpdump only seeing the port open when it is actually in use?
What I'm not understanding is that some connections are actually reaching those ports while most are not.
In my tcpdump, what is it that shows you there is no reply?
I'm looking at the captures of those which did get a response and they don't look any different.
I ran the same command and am seeing different results.
PORT STATE SERVICE
50127/udp open|filtered unknown
50128/udp closed unknown
50129/udp open|filtered unknown
50130/udp open|filtered unknown
50131/udp closed unknown
Is tcpdump only seeing the port open when it is actually in use?
ASKER
I went thought this once before but cannot find the notes on it. It was something pretty simple and nothing to do with the communications/app but with the server I think.
Just checked again, different.
PORT STATE SERVICE
50127/udp open|filtered unknown (this one is NOT in the firewall config)
50128/udp open|filtered unknown
50129/udp open|filtered unknown
50130/udp closed unknown
50131/udp open|filtered unknown
Just checked again, different.
PORT STATE SERVICE
50127/udp open|filtered unknown (this one is NOT in the firewall config)
50128/udp open|filtered unknown
50129/udp open|filtered unknown
50130/udp closed unknown
50131/udp open|filtered unknown
A quick test is to turn the firewall off, run your test, and turn it back on. If it's open 30-60 seconds, your risk is minimized.
ASKER
Yes, I also tried that and it made no difference. That's why I think it's something with the server itself. SElinux is disabled, there is no other firewall software yet these ports seem to function a little randomly.
Without firewalld and selinux, it has to be the application. Have you checked the configuration?
ASKER
Yes, I've been monitoring logs and trying different things for days. It's why I posted the question, because I'm out of ideas and wondering if this is somehow firewalld related.
If you need outside assistance, I can Teamviewer in and take a look. All of my work is automatically presumed non-disclosure.
ASKER
I appreciate that and will keep it in mind and will pass it on.
I also appreciate the help you've given me in trying to solve this, thanks. I'll keep digging and see if I can come up with any new leads.
I also appreciate the help you've given me in trying to solve this, thanks. I'll keep digging and see if I can come up with any new leads.
ASKER
Still, the part I don't understand is why some connections seem to be able to reach those ports while others cannot. That is odd.
I think I need to monitor in real time as you mentioned my tcpdump log is showing only incoming connections, not outgoing.
I was asking what it is that shows you this, is it the > traffic in the log I posted?
Does my firewall config look right to you? I posted it above.
I think I need to monitor in real time as you mentioned my tcpdump log is showing only incoming connections, not outgoing.
I was asking what it is that shows you this, is it the > traffic in the log I posted?
Does my firewall config look right to you? I posted it above.
ASKER
I'm testing using wireshark on a remote that needs to connect to those server ports.
I keep seeing port unreachable yet I've also disabled the firewall.
This again leads to the app not working right but I've even used an older version which I know for sure worked fine.
Still looking for leads.
I keep seeing port unreachable yet I've also disabled the firewall.
This again leads to the app not working right but I've even used an older version which I know for sure worked fine.
Still looking for leads.
ASKER
Didn't know you could use iptables to also view the rules.
PS: I know I'm using the term 'connection' but I do mean state of the socket/s.
PS: I know I'm using the term 'connection' but I do mean state of the socket/s.
# iptables -L --line-numbers -n
Chain INPUT (policy ACCEPT)
num target prot opt source destination
1 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
2 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
3 INPUT_direct all -- 0.0.0.0/0 0.0.0.0/0
4 INPUT_ZONES_SOURCE all -- 0.0.0.0/0 0.0.0.0/0
5 INPUT_ZONES all -- 0.0.0.0/0 0.0.0.0/0
6 ACCEPT icmp -- 0.0.0.0/0 0.0.0.0/0
7 REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited
Chain FORWARD (policy ACCEPT)
num target prot opt source destination
1 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
2 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
3 FORWARD_direct all -- 0.0.0.0/0 0.0.0.0/0
4 FORWARD_IN_ZONES_SOURCE all -- 0.0.0.0/0 0.0.0.0/0
5 FORWARD_IN_ZONES all -- 0.0.0.0/0 0.0.0.0/0
6 FORWARD_OUT_ZONES_SOURCE all -- 0.0.0.0/0 0.0.0.0/0
7 FORWARD_OUT_ZONES all -- 0.0.0.0/0 0.0.0.0/0
8 ACCEPT icmp -- 0.0.0.0/0 0.0.0.0/0
9 REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited
Chain OUTPUT (policy ACCEPT)
num target prot opt source destination
1 OUTPUT_direct all -- 0.0.0.0/0 0.0.0.0/0
Chain FORWARD_IN_ZONES (1 references)
num target prot opt source destination
1 FWDI_public all -- 0.0.0.0/0 0.0.0.0/0 [goto]
2 FWDI_public all -- 0.0.0.0/0 0.0.0.0/0 [goto]
Chain FORWARD_IN_ZONES_SOURCE (1 references)
num target prot opt source destination
Chain FORWARD_OUT_ZONES (1 references)
num target prot opt source destination
1 FWDO_public all -- 0.0.0.0/0 0.0.0.0/0 [goto]
2 FWDO_public all -- 0.0.0.0/0 0.0.0.0/0 [goto]
Chain FORWARD_OUT_ZONES_SOURCE (1 references)
num target prot opt source destination
Chain FORWARD_direct (1 references)
num target prot opt source destination
Chain FWDI_public (2 references)
num target prot opt source destination
1 FWDI_public_log all -- 0.0.0.0/0 0.0.0.0/0
2 FWDI_public_deny all -- 0.0.0.0/0 0.0.0.0/0
3 FWDI_public_allow all -- 0.0.0.0/0 0.0.0.0/0
Chain FWDI_public_allow (1 references)
num target prot opt source destination
Chain FWDI_public_deny (1 references)
num target prot opt source destination
Chain FWDI_public_log (1 references)
num target prot opt source destination
Chain FWDO_public (2 references)
num target prot opt source destination
1 FWDO_public_log all -- 0.0.0.0/0 0.0.0.0/0
2 FWDO_public_deny all -- 0.0.0.0/0 0.0.0.0/0
3 FWDO_public_allow all -- 0.0.0.0/0 0.0.0.0/0
Chain FWDO_public_allow (1 references)
num target prot opt source destination
Chain FWDO_public_deny (1 references)
num target prot opt source destination
Chain FWDO_public_log (1 references)
num target prot opt source destination
Chain INPUT_ZONES (1 references)
num target prot opt source destination
1 IN_public all -- 0.0.0.0/0 0.0.0.0/0 [goto]
2 IN_public all -- 0.0.0.0/0 0.0.0.0/0 [goto]
Chain INPUT_ZONES_SOURCE (1 references)
num target prot opt source destination
Chain INPUT_direct (1 references)
num target prot opt source destination
Chain IN_public (2 references)
num target prot opt source destination
1 IN_public_log all -- 0.0.0.0/0 0.0.0.0/0
2 IN_public_deny all -- 0.0.0.0/0 0.0.0.0/0
3 IN_public_allow all -- 0.0.0.0/0 0.0.0.0/0
Chain IN_public_allow (1 references)
num target prot opt source destination
1 ACCEPT tcp -- x.x.x.x 0.0.0.0/0 tcp dpt:3306 ctstate NEW
2 ACCEPT tcp -- x.x.x.x 0.0.0.0/0 tcp dpt:3128 ctstate NEW
3 ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpts:50128:50131 ctstate NEW
Chain IN_public_deny (1 references)
num target prot opt source destination
Chain IN_public_log (1 references)
num target prot opt source destination
Chain OUTPUT_direct (1 references)
num target prot opt source destinationASKER
Here is something I got from the dev.
50128: testType 0: Traceroute
50129: testType 1: Ping
50130: testType 2: Ping
50131: testType 3: Traceroute
They are using non standard ports for the above. Could the problem be related to an icmp type that is not being allowed? Something by the server since even disabling firewall makes no difference.
# firewall-cmd --get-icmptypes
destination-unreachable echo-reply echo-request parameter-problem redirect router-advertisement router-solicitation source-quench time-exceeded
From a remote wireshark session
10673 142.356335 x.x.x.x 192.168.1.205 ICMP 80 Destination unreachable (Port unreachable)
50128: testType 0: Traceroute
50129: testType 1: Ping
50130: testType 2: Ping
50131: testType 3: Traceroute
They are using non standard ports for the above. Could the problem be related to an icmp type that is not being allowed? Something by the server since even disabling firewall makes no difference.
# firewall-cmd --get-icmptypes
destination-unreachable echo-reply echo-request parameter-problem redirect router-advertisement router-solicitation source-quench time-exceeded
From a remote wireshark session
10673 142.356335 x.x.x.x 192.168.1.205 ICMP 80 Destination unreachable (Port unreachable)
ASKER
I'm doing this test from a location where an app communicating with the server is running.
Nmap cannot ever 'see' those ports since there is no service but it can see the state if something is connecting to them at the very moment of the test from the same network. From another network, you'll always see closed.
Your mention of this is the lead I needed to know where the problem was.
Nmap cannot ever 'see' those ports since there is no service but it can see the state if something is connecting to them at the very moment of the test from the same network. From another network, you'll always see closed.
Your mention of this is the lead I needed to know where the problem was.
firewall-cmd --zone=public --list-all
(or whatever your zone name is)