suricata drop unknown user agent

how should i be sure that suricata dropping that traffic or not useragent unknown


10/08/2015-17:08:55.167477 www.xxx.com [**] / [**] <useragent unknown> [**] 89.184.73.241:58668 -> 37.123.101.43:80
10/08/2015-17:08:55.212255 www.xxx.com [**] / [**] <useragent unknown> [**] 89.184.75.24:43520 -> 37.123.101.43:80
10/08/2015-17:08:55.386484 www.xxx.com [**] / [**] <useragent unknown> [**] 89.184.73.241:58770 -> 37.123.101.43:80
10/08/2015-17:08:55.652528 www.xxx.com [**] / [**] <useragent unknown> [**] 89.184.73.241:58830 -> 37.123.101.43:80
10/08/2015-17:08:56.105059 www.xxx.com [**] / [**] <useragent unknown> [**] 89.184.75.24:43825 -> 37.123.101.43:80
10/08/2015-17:08:56.145159 www.xxx.com [**] / [**] <useragent unknown> [**] 77.87.197.102:42975 -> 37.123.101.43:80
10/08/2015-17:08:56.477456 www.xxx.com [**] / [**] <useragent unknown> [**] 77.87.197.102:43199 -> 37.123.101.43:80
10/08/2015-17:08:56.544917 www.xxx.com [**] / [**] <useragent unknown> [**] 89.184.75.24:43983 -> 37.123.101.43:80

Open in new window

FireBallITAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

btanExec ConsultantCommented:
Overall, packets that are being dropped are saved in the drop.log file, a Netfilter log format. those are all in the fast.log which is the name of the file in the default logging directory (e.g. default-log-dir: /var/log/suricata). But specific to HTTP traffic, you should look at http.log as it keeps track of all HTTP-traffic events. It contains the HTTP request, hostname, URI and the User-Agent.

Good to check inside your Suricata.yaml, which contains the configuration of Suricata that signature such as "useragent unknown" (or equivalent) should have the  Action property that stated to have it "Drop".

See the action for information

>For Drop, it is only concern about  the IPS/inline mode. If the program finds a signature that matches, containing drop, it stops immediately. So what we should observe in "Drop" is the receiver does not receive a message of what is going on, resulting in a time-out (certainly with TCP). Suricata generates an alert for this packet.

>For Alert, it is when a signature matches and contains alert. Only the system administrator can notice this alert.

>For Reject, Both receiver and sender receive a reject packet. Slightly different from the "Drop" is that if the offending packet concerns TCP, it will be a Reset-packet. For all other protocols it will be an ICMP-error packet. But when it is in Inline/IPS mode, the offending packet will also be dropped like with the 'Drop' action.

Ref - https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Suricatayaml
FireBallITAuthor Commented:
should i use suricata on a machine which is configured to bridge interfaces with no apache for to clean this type of traffic  ? have you ever tried sth. like that ?

i want to do sth like that


Switch ---- SURICATA SERVER ----- Web hosting server

should it clean the traffic as given above ?
btanExec ConsultantCommented:
as drawn it is inline, I do suggest you consider this placement which you can still function as bridge. The example uses br0, as the bridge interface and the need to add iptables rules for the FORWARD chain, for traffic traversing the bridge.
http://taosecurity.blogspot.sg/2014/01/suricata-20beta2-as-ips-on-ubuntu-1204.html
Get Cisco Certified in IT Security

There’s a high demand for IT security experts and network administrators who can safeguard the data that individuals, corporations, and governments rely on every day. Pursue your B.S. in Network Operations and Security and gain the credentials you need for this high-growth field.

FireBallITAuthor Commented:
so we should succedd with a topology like added below.

s
FireBallITAuthor Commented:
Waf is suricata

First link on the left for out bound
Second link on the left is udp inbound
third link on the left is tcp inbound


under the srx 80 port is redirecting to the WAF

end ex 4500 keeping all gateways on it mx is just using for static routes.

So i think that should success to clean up illegal http traffic
btanExec ConsultantCommented:
suricata is not waf per se, you should at least go for mod_security. Ips vs waf is very different. nonetheless, i see your intent is to inspect the inbound traffic only. the bridge is supposed to alright - I assuming your other traffic routing is fine.  

you probably can run tcpdump for the suricata interfaces to see the traffic that is expected incoming but I am thinking does the tcp 3 way handshake actually is established with traffic of NEW, ESTABLISHED minimally...also note the example link is using NFQUEUE in iptables rules so that it will send packets to Suricata for processing. https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Setting_up_IPSinline_for_Linux
FireBallITAuthor Commented:
Yes but our main aim point is to protect port 80 from botnet attacks. We have no other problem on our network .
Do you have a waf advice that works like squid proxy. I mean which does not require a config for each web site and apply owasp rules on this mode
btanExec ConsultantCommented:
waf should be able to handle transparent bridge like layer 2 inspection too.
https://techlib.barracuda.com/waf/deploymentmode
FireBallITAuthor Commented:
yes we have talked the them but their software needs to create each site's config on the waf
btanExec ConsultantCommented:
that is what it has to as t ogo into that L7 layer you need that specific handling. However, that there may means of lesser hassle though it is not a straight off like placing an ips. see F5 transparent mode, better to check with their technical team to drill into your use case...
https://devcentral.f5.com/questions/asm-transparent-bridge-mode-detect-capabilities
FireBallITAuthor Commented:
yes but squid + nginx is a good partner to worki without writing config on a server . Which can handle the traffic from wan and get responses from lan.

But i could not find 2 things

1. mod_security and nginx integration
2. showing the default proxy squid and redirecting all connections to the squid
btanExec ConsultantCommented:
yes nginx is reverse proxy and suite the use case. Squid can be good for either forward or reverse proxy. but having to get modsec into the picture seems more complicated as to having a single box to do those inspection - of course I am not into those solutioning .. In the past, I did chanced on cacheguard but did not drill deeper http://sourceforge.net/projects/webgateway/, sharing for your info if it does make sense to your use case or through ICAP with squid still being transparent  http://xmodulo.com/transparent-https-filtering-proxy-centos.html

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
FireBallITAuthor Commented:
Thank you so much for this good conversation and for your time i will check deeper with this links
btanExec ConsultantCommented:
thanks for sharing
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Linux Security

From novice to tech pro — start learning today.