Want to protect your cyber security and still get fast solutions? Ask a secure question today.Go Premium

x
?
Solved

Capture HTTP/HTTPS and IP address orgin

Posted on 2014-01-14
13
Medium Priority
?
349 Views
Last Modified: 2014-01-21
I am looking for a simple command line linux utility to have it run on automatically and capture HTTP/HTTPS and the computer IP it originated from.
0
Comment
Question by:georgopanos
  • 5
  • 5
  • 2
  • +1
13 Comments
 
LVL 36

Expert Comment

by:Seth Simmons
ID: 39781097
if you are talking about seeing where traffic is coming from to access your web site, it should be in the logs

if you are running apache, it will appear in the access log
by default it will include the source IP address, date, time, page requested, browser used and result code
0
 
LVL 48

Expert Comment

by:Tintin
ID: 39781103
Do you want to capture the HTTP and HTTPS payload as well?  If so, then use tcpdump, wireshark, ngrep or similar to do a packet capture.   Note that it's a little more involved to capture HTTPS in plain text.
0
 

Author Comment

by:georgopanos
ID: 39781116
I think wireshark is overkill for what I want.

I basically want a simple log file that will list

HTTP/HTTPS - orgin IP address or (computer name)

I was looking for a simple command line utility I can setup on my Rasberry Pi to autorun on boot with a Network tap and capture the above INFORMATION from a network. Just to monitor what is being browsed.
0
Evaluating UTMs? Here's what you need to know!

Evaluating a UTM appliance and vendor can prove to be an overwhelming exercise.  How can you make sure that you're getting the security that your organization needs without breaking the bank? Check out our UTM Buyer's Guide for more information on what you should be looking for!

 
LVL 48

Expert Comment

by:Tintin
ID: 39781229
Why not just do

tail -f /var/log/apache2/access.log
0
 

Author Comment

by:georgopanos
ID: 39781856
I am not running a website, I want to see the websites the computers on my network are going to.
0
 
LVL 48

Expert Comment

by:Tintin
ID: 39783962
Ah, makes sense now.

Assuming the traffic goes through your pi, or you have a promiscuous network, then you can do:

tcpdump port 80 or port 443

if that's a little too verbose, you could do

tcpdump port 80 or port 443 | sed "s/\.http.*//"
0
 
LVL 13

Expert Comment

by:Sandy
ID: 39788112
in single line

#tcpdump port 80
0
 

Author Comment

by:georgopanos
ID: 39795131
tintin: this works for my needs but one other question,

I see for example when it runs the computer will hit an identical address 10 times in a row
for example:

11:51 10.1.10.1 > compute.amazonaws.com
11:51 10.1.10.1 > compute.amazonaws.com
11:51 10.1.10.1 > compute.amazonaws.com
11:51 10.1.10.1 > compute.amazonaws.com
11:51 10.1.10.1 > compute.amazonaws.com
11:51 10.1.10.1 > compute.amazonaws.com
11:51 10.1.10.1 > compute.amazonaws.com
11:51 10.1.10.1 > compute.amazonaws.com
11:51 10.1.10.1 > compute.amazonaws.com
11:52 10.1.10.1> www.google.com
11:52 10.1.10.1> www.google.com
11:53 10.1.10.1 > compute.amazonaws.com
11:53 10.1.10.1 > compute.amazonaws.com
11:53 10.1.10.1 > compute.amazonaws.com
11:53 10.1.10.1 > compute.amazonaws.com

how can I run the tcpdump and do an IF THEN comparison
to cut the above down to look like this 3 lines instead of 15 just to be able to read the log
easier.

11:51 10.1.10.1 > compute.amazonaws.com
11:52 10.1.10.1> www.google.com
11:53 10.1.10.1 > compute.amazonaws.com

Thanks
0
 
LVL 48

Expert Comment

by:Tintin
ID: 39795946
Depends if you want to read it in realtime or not.

If you capture to a log, you can simply do:

uniq -d logfile
0
 
LVL 13

Expert Comment

by:Sandy
ID: 39795981
otherwise

#sort logfile | uniq -d
0
 

Author Comment

by:georgopanos
ID: 39796882
I would like to sort it in real time if possible so as to display it to a screen
0
 
LVL 48

Accepted Solution

by:
Tintin earned 2000 total points
ID: 39798459
Try this

tcpdump -q dst port http or https|awk '{print $1,$5}'|sed "s/\.http.*//"|sed "s/\.[0-9]* / /"|uniq -d


Note that you may need to wait a little while to see any output as it will be buffered.
0
 

Author Closing Comment

by:georgopanos
ID: 39798504
Great! Thank you!!!!
0

Featured Post

Who's Defending Your Organization from Threats?

Protecting against advanced threats requires an IT dream team – a well-oiled machine of people and solutions working together to defend your organization. Download our resource kit today to learn more about the tools you need to build you IT Dream Team!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

SSH (Secure Shell) - Tips and Tricks As you all know SSH(Secure Shell) is a network protocol, which we use to access/transfer files securely between two networked devices. SSH was actually designed as a replacement for insecure protocols that sen…
I. Introduction There's an interesting discussion going on now in an Experts Exchange Group — Attachments with no extension . This reminded me of questions that come up here at EE along the lines of, "How can I tell the type of file from its cont…
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
Connecting to an Amazon Linux EC2 Instance from Windows Using PuTTY.
Suggested Courses
Course of the Month10 days, 22 hours left to enroll

572 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question