georgopanos
asked on
Capture HTTP/HTTPS and IP address orgin
I am looking for a simple command line linux utility to have it run on automatically and capture HTTP/HTTPS and the computer IP it originated from.
Do you want to capture the HTTP and HTTPS payload as well? If so, then use tcpdump, wireshark, ngrep or similar to do a packet capture. Note that it's a little more involved to capture HTTPS in plain text.
ASKER
I think wireshark is overkill for what I want.
I basically want a simple log file that will list
HTTP/HTTPS - orgin IP address or (computer name)
I was looking for a simple command line utility I can setup on my Rasberry Pi to autorun on boot with a Network tap and capture the above INFORMATION from a network. Just to monitor what is being browsed.
I basically want a simple log file that will list
HTTP/HTTPS - orgin IP address or (computer name)
I was looking for a simple command line utility I can setup on my Rasberry Pi to autorun on boot with a Network tap and capture the above INFORMATION from a network. Just to monitor what is being browsed.
Why not just do
tail -f /var/log/apache2/access.lo g
tail -f /var/log/apache2/access.lo
ASKER
I am not running a website, I want to see the websites the computers on my network are going to.
Ah, makes sense now.
Assuming the traffic goes through your pi, or you have a promiscuous network, then you can do:
tcpdump port 80 or port 443
if that's a little too verbose, you could do
tcpdump port 80 or port 443 | sed "s/\.http.*//"
Assuming the traffic goes through your pi, or you have a promiscuous network, then you can do:
tcpdump port 80 or port 443
if that's a little too verbose, you could do
tcpdump port 80 or port 443 | sed "s/\.http.*//"
in single line
#tcpdump port 80
#tcpdump port 80
ASKER
tintin: this works for my needs but one other question,
I see for example when it runs the computer will hit an identical address 10 times in a row
for example:
11:51 10.1.10.1 > compute.amazonaws.com
11:51 10.1.10.1 > compute.amazonaws.com
11:51 10.1.10.1 > compute.amazonaws.com
11:51 10.1.10.1 > compute.amazonaws.com
11:51 10.1.10.1 > compute.amazonaws.com
11:51 10.1.10.1 > compute.amazonaws.com
11:51 10.1.10.1 > compute.amazonaws.com
11:51 10.1.10.1 > compute.amazonaws.com
11:51 10.1.10.1 > compute.amazonaws.com
11:52 10.1.10.1> www.google.com
11:52 10.1.10.1> www.google.com
11:53 10.1.10.1 > compute.amazonaws.com
11:53 10.1.10.1 > compute.amazonaws.com
11:53 10.1.10.1 > compute.amazonaws.com
11:53 10.1.10.1 > compute.amazonaws.com
how can I run the tcpdump and do an IF THEN comparison
to cut the above down to look like this 3 lines instead of 15 just to be able to read the log
easier.
11:51 10.1.10.1 > compute.amazonaws.com
11:52 10.1.10.1> www.google.com
11:53 10.1.10.1 > compute.amazonaws.com
Thanks
I see for example when it runs the computer will hit an identical address 10 times in a row
for example:
11:51 10.1.10.1 > compute.amazonaws.com
11:51 10.1.10.1 > compute.amazonaws.com
11:51 10.1.10.1 > compute.amazonaws.com
11:51 10.1.10.1 > compute.amazonaws.com
11:51 10.1.10.1 > compute.amazonaws.com
11:51 10.1.10.1 > compute.amazonaws.com
11:51 10.1.10.1 > compute.amazonaws.com
11:51 10.1.10.1 > compute.amazonaws.com
11:51 10.1.10.1 > compute.amazonaws.com
11:52 10.1.10.1> www.google.com
11:52 10.1.10.1> www.google.com
11:53 10.1.10.1 > compute.amazonaws.com
11:53 10.1.10.1 > compute.amazonaws.com
11:53 10.1.10.1 > compute.amazonaws.com
11:53 10.1.10.1 > compute.amazonaws.com
how can I run the tcpdump and do an IF THEN comparison
to cut the above down to look like this 3 lines instead of 15 just to be able to read the log
easier.
11:51 10.1.10.1 > compute.amazonaws.com
11:52 10.1.10.1> www.google.com
11:53 10.1.10.1 > compute.amazonaws.com
Thanks
Depends if you want to read it in realtime or not.
If you capture to a log, you can simply do:
uniq -d logfile
If you capture to a log, you can simply do:
uniq -d logfile
otherwise
#sort logfile | uniq -d
#sort logfile | uniq -d
ASKER
I would like to sort it in real time if possible so as to display it to a screen
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Great! Thank you!!!!
if you are running apache, it will appear in the access log
by default it will include the source IP address, date, time, page requested, browser used and result code