FireBall
asked on
hard perl script
we are senind the given output below to a subroutine in evey one second.
Than we want to create automated ip table rules with checking thesses
if a destination (side of 255.255.255.255) is in more then 1000+ lines then we want to let the software decide
* if the source port (3.rd column) is a criteria
* if data (4th column) is a criteria
* if destination port is a criteria
* AND TTL check (If attack comes from same machine there must be one ttl value but if it comes from 2 machines ?)
then it will create iptables rules as this
iptables -A input -d (destination ip) -m ttl --ttl-eq=TTLNumber (if ttl is a criteria) --sport xxxx (if source port criteria) --dport yyyy (if destination port criteria) --match string --algo kmp --hex-string '|f4 6d 04 25 b2 02 00 0a|' (if packet is a criteria) -j DROP
how should we do that ?
Than we want to create automated ip table rules with checking thesses
if a destination (side of 255.255.255.255) is in more then 1000+ lines then we want to let the software decide
* if the source port (3.rd column) is a criteria
* if data (4th column) is a criteria
* if destination port is a criteria
* AND TTL check (If attack comes from same machine there must be one ttl value but if it comes from 2 machines ?)
then it will create iptables rules as this
iptables -A input -d (destination ip) -m ttl --ttl-eq=TTLNumber (if ttl is a criteria) --sport xxxx (if source port criteria) --dport yyyy (if destination port criteria) --match string --algo kmp --hex-string '|f4 6d 04 25 b2 02 00 0a|' (if packet is a criteria) -j DROP
how should we do that ?
1433301992474 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301992503 - 41.26.121.86. 51594 0001000000000000067178737a6b6603 37.123.100.8. 53 233 66
1433301992532 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301992565 - 185.9.156.26. 62218 4e495431006500008802fd66d3000000 185.9.156.13. 9987 128 62
1433301992594 - 185.9.156.26. 62218 670afe6f00000000229d748b45aa7bef 185.9.156.13. 9987 128 212
1433301992640 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301992673 - 213.238.170.199. 137 000100000000000020454a4543454e45 213.238.170.255. 137 128 78
1433301992706 - 213.238.170.199. 137 00010000000000002045434641454d46 213.238.170.255. 137 128 78
1433301992736 - 213.238.170.199. 137 00010000000000002045474545464846 213.238.170.255. 137 128 78
1433301992764 - 213.238.170.199. 137 00010000000000002045424549454e45 213.238.170.255. 137 128 78
1433301682696 - 213.238.170.199. 137 0001000000000000204649464a454b45 213.238.170.255. 137 128 78
1433301682746 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301682778 - 213.238.170.199. 137 00010000000000002046424550454946 213.238.170.255. 137 128 78
1433301682808 - 157.56.205.8. 40021 00010000000000000a636961686f7374 37.123.100.220. 53 112 60
1433301682837 - 191.234.67.56. 44197 00010000000000000a636961686f7374 37.123.100.220. 53 48 60
1433301682866 - 157.56.205.9. 29205 00010000000000000a636961686f7374 37.123.100.220. 53 112 60
1433301682895 - 37.123.103.4. 64210 00010000000000000477706164000001 224.0.0.252. 5355 1 50
1433301682925 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301682954 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301682984 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301683012 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301683066 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301683109 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301683139 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301683173 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301683202 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301683232 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301683261 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301683308 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301683340 - 57.177.212.60. 17824 0001000000000000026d740737373737 37.123.100.8. 53 239 60
1433301683370 - 37.123.103.4. 63257 00010000000000000477706164000001 224.0.0.252. 5355 1 50
1433301683399 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301683437 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301683480 - 157.56.205.9. 14975 00010000000000000a636961686f7374 37.123.100.220. 53 111 60
1433301683510 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301683540 - 37.123.103.4. 63257 00010000000000000477706164000001 224.0.0.252. 5355 1 50
1433301683569 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301683598 - 37.123.96.154. 137 00010000000000002046484641454245 37.123.96.159. 137 128 78
1433301683632 - 18.92.167.121. 51761 00010000000000000273720377777705 37.123.100.8. 53 233 62
1433301683662 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301683691 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301683720 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301683749 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301683778 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301683807 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301683836 - 061632. 246284 000000000100020222022300672c52 (hlim UDP 00 128
1433301683869 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301683899 - 37.123.103.4. 137 00010000000000002046484641454245 37.123.103.15. 137 128 78
1433301482893 - 185.9.159.186. 137 00010000000000002045504644454e45 185.9.159.255. 137 128 78
1433301482920 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301482934 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301482947 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301482960 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301482984 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301482997 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301483010 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301483023 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301483058 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301483073 - 14.211.48.149. 19515 00010000000000000a656e6978736a69 37.123.100.8. 53 237 68
1433301483102 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301483115 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301483127 - 185.9.156.26. 62218 4e495431006500008802fd66d3000000 185.9.156.13. 9987 128 62
1433301483140 - 185.9.156.26. 62218 670afe6f00000000229d748b45aa7bef 185.9.156.13. 9987 128 212
1433301483153 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301483166 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301483178 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301483191 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301483213 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301483228 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301483241 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301483254 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301483266 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301483279 - 91.19.112.252. 9508 00010000000000001067627364736467 37.123.100.8. 53 236 77
1433301483291 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301483304 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301483316 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301483329 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301483341 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301483354 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301483366 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301483379 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301483391 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301483404 - 37.123.103.4. 137 00010000000000002046484641454245 37.123.103.15. 137 128 78
1433301483416 - 178.20.225.235. 54513 0500000000000000f8be000013000000 255.255.255.255. 48887 128 75
1433301483433 - 93.158.178.83. 21776 0001000000000000037777770c726f6d 37.123.100.2. 53 51 66ASKER
Destination mean destination ip address
255.255.255.255 mean in the given logs first line you will see it. I tryed to describe that column
If destination has 1000+ log in sended log to sub function
Sub function will check and detect how to block this attack
255.255.255.255 mean in the given logs first line you will see it. I tryed to describe that column
If destination has 1000+ log in sended log to sub function
Sub function will check and detect how to block this attack
So check for 1000+ repeated values in column 6, and find a set of other column values common to those rows?
Wouldn't the destination always be a common value?
Wouldn't the destination always be a common value?
ASKER
Depending on attack structure
Packet should be unique or maybe not
Source should be spood or maybe not
Ttl must be one if attack come from a source but if it comes from 2 different source ? So we need group the destination with ttl
Source port should be uniqe or maybe not
packet size should be unique or maybe not
Most prof attacks make everything unique so the only criteria is ttl and dest. Ip
But that should block too many people , but what if hacker lost a point like same packets or like packetsize ....etc we will add it to rule
Packet should be unique or maybe not
Source should be spood or maybe not
Ttl must be one if attack come from a source but if it comes from 2 different source ? So we need group the destination with ttl
Source port should be uniqe or maybe not
packet size should be unique or maybe not
Most prof attacks make everything unique so the only criteria is ttl and dest. Ip
But that should block too many people , but what if hacker lost a point like same packets or like packetsize ....etc we will add it to rule
Which column is Packet? which column is Source? What does a spood look like? If Destination is unique, how can it be 1000+?
If we only block unique attacks, does that mean that a repeated attack will never get blocked?
If we only block unique attacks, does that mean that a repeated attack will never get blocked?
ASKER
Or if attacks are never repeated, then why bother blocking anything we've seen before?
ASKER
of course if server gets 1500 logs 1500 of them should not be an attack . 800 of them maybe attackler log
So after grouping by TTL and destination it will check for the density of other columns same values for ex. our limit is %80 for aaccepting as an attack (because %20 should be real traffic from the same ttl)
IT will be detect that
from same ttl and to same dest. software found that %80 packet sample is same %2 source port same %1 source ip same %90 packet size same ....etc.
so if it is more then %80 will be a criteria for created iptables rule
So after grouping by TTL and destination it will check for the density of other columns same values for ex. our limit is %80 for aaccepting as an attack (because %20 should be real traffic from the same ttl)
IT will be detect that
from same ttl and to same dest. software found that %80 packet sample is same %2 source port same %1 source ip same %90 packet size same ....etc.
so if it is more then %80 will be a criteria for created iptables rule
ASKER
That scripts loop in one second so after attack start it blocks max. in 1 second and release back the rule in 5 minutes.
instead of manuelly checking that would be perfecct.
we have some enterprise products like juniper srx 3k , 5k , mx480 ...etc.
Any of this systems are not valuable on udp attacks because they work in session logic and can not decide spoof udp so we will bring a transparent bridge between router and firewall which will clean up the dirty traffic
and we are always under attack %98 of this attacks should be catched by this filter
If this attack hits and goes in to firewall it locks all the system
If we limit the firewall udp threshold it blocks real connections this is a type of clean up work of traffic
instead of manuelly checking that would be perfecct.
we have some enterprise products like juniper srx 3k , 5k , mx480 ...etc.
Any of this systems are not valuable on udp attacks because they work in session logic and can not decide spoof udp so we will bring a transparent bridge between router and firewall which will clean up the dirty traffic
and we are always under attack %98 of this attacks should be catched by this filter
If this attack hits and goes in to firewall it locks all the system
If we limit the firewall udp threshold it blocks real connections this is a type of clean up work of traffic
Which of http://filedb.experts-exchange.com/incoming/2015/06_w23/800_916895/Screenshot-18.png would become criteria, and why?
ASKER
If a server sends packets to a destination TTL never changes.
because TTL was count depending on the router count on the road.
so If attack comes from a server TTL never changes , If attack comes from 2 servers there should be 2 different TTL value in the density of the logs depending on normal traffic vs attack traffick
so TTL & destination grouped ip log count shows us the attacker's traffic probably.
Of course there should be real customers from the same ttl other criteria's allow us to block attack specificly and less customer to be effected
because TTL was count depending on the router count on the road.
so If attack comes from a server TTL never changes , If attack comes from 2 servers there should be 2 different TTL value in the density of the logs depending on normal traffic vs attack traffick
so TTL & destination grouped ip log count shows us the attacker's traffic probably.
Of course there should be real customers from the same ttl other criteria's allow us to block attack specificly and less customer to be effected
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
thank you for this good response ;
print("$ms"."\t"."$ip_obj-
"$ip_obj->{dest_ip}"."\t".
we got the values as this not from the screen your code seems ok but i do not know how to compare them :)
print("$ms"."\t"."$ip_obj-
"$ip_obj->{dest_ip}"."\t".
we got the values as this not from the screen your code seems ok but i do not know how to compare them :)
ASKER
We have made some changes and resolved some bugs.
This is working better but could not decide how to solve the flow descisions
This is working better but could not decide how to solve the flow descisions
1433361008586 128 178.20.225.235 54513 255.255.255.255 48887 55 2f0000000500000000000000f8be0000
1433361035972 128 178.20.225.235 54513 255.255.255.255 48887 55 2f0000000500000000000000f8be0000
1433361059760 128 178.20.225.235 54513 255.255.255.255 48887 55 2f0000000500000000000000f8be0000
1433361065737 128 37.123.102.245 137 37.123.102.255 137 58 bc1b0110000100000000000020464944
1433361083986 128 178.20.225.235 54513 255.255.255.255 48887 55 2f0000000500000000000000f8be0000
1433361092581 1 37.123.103.30 61087 224.0.0.252 5355 30 326d00000001000000000000047770
1433361108382 128 178.20.225.235 54513 255.255.255.255 48887 55 2f0000000500000000000000f8be0000
1433361131553 128 178.20.225.235 54513 255.255.255.255 48887 55 2f0000000500000000000000f8be0000
1433361155009 128 178.20.225.235 54513 255.255.255.255 48887 55 2f0000000500000000000000f8be0000
1433361175924 128 37.123.101.67 137 37.123.101.255 137 58 e83d0110000100000000000020464846
1433361179300 128 178.20.225.235 54513 255.255.255.255 48887 55 2f0000000500000000000000f8be0000
1433361193085 1 37.123.103.30 61087 224.0.0.252 5355 30 326d00000001000000000000047770
1433361203707 128 178.20.225.235 54513 255.255.255.255 48887 55 2f0000000500000000000000f8be0000
1433361228626 128 178.20.225.235 54513 255.255.255.255 48887 55 2f0000000500000000000000f8be0000
1433361251772 128 178.20.225.235 54513 255.255.255.255 48887 55 2f0000000500000000000000f8be0000
1433361276255 128 178.20.225.235 54513 255.255.255.255 48887 55 2f0000000500000000000000f8be0000
1433361300329 128 178.20.225.235 54513 255.255.255.255 48887 55 2f0000000500000000000000f8be0000
1433361323604 128 178.20.225.235 54513 255.255.255.255 48887 55 2f0000000500000000000000f8be0000
1433361348029 128 178.20.225.235 54513 255.255.255.255 48887 55 2f0000000500000000000000f8be0000
1433361371867 128 178.20.225.235 54513 255.255.255.255 48887 55 2f0000000500000000000000f8be0000
1433361396217 128 178.20.225.235 54513 255.255.255.255 48887 55 2f0000000500000000000000f8be0000
1433361396326 128 37.123.103.30 137 37.123.103.31 137 58 de690110000100000000000020464846ASKER
also we have faced with another problem - sampling rate-
tcpdump or net::pcap packages
returns the results in rates not a real time flow.
So it can not catch the 1 second easily. some times if there is no heavy traffic it returns the lines in 3 seconds that should makes every thing complicated :(
maybe we need to use mysql for operations with recording datas
tcpdump or net::pcap packages
returns the results in rates not a real time flow.
So it can not catch the 1 second easily. some times if there is no heavy traffic it returns the lines in 3 seconds that should makes every thing complicated :(
maybe we need to use mysql for operations with recording datas
ASKER
should you please also check this question
https://www.experts-exchange.com/questions/28685236/Perl-strange-behaviour.html
https://www.experts-exchange.com/questions/28685236/Perl-strange-behaviour.html
What is a "destination"? What is a "side of 255.255.255.255"?
What should the software decide?
What is a criteria ?