We help IT Professionals succeed at work.
Get Started

Regex for parsing out Apache common log (plus one extra field)

Geoff Millikan
on
730 Views
Last Modified: 2012-05-10
The below pattern parses out the fields from $str1 just fine but it doesn't work on to other strings.  Can you fix the regex pattern so it works on all strings?

Thanks, http://www.t1shopper.com/

$str1='67.195.37.124 - - [20/Apr/2010:05:32:22 +0000] "GET /us/ga/b.html HTTP/1.0" 200 1761 "-" "Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; http://help.yahoo.com/help/us/ysearch/slurp)" "-"';

$str2='67.225.164.12 - - [03/Jan/2011:21:15:39 +0000] "GET / HTTP/1.1" 200 8973 "" "\"Mozilla/4.0 (compatible; MSIE 6.0; Windows 98; Win 9x 4.90)\"" "-"';

$str3='77.238.196.184 - - [01/Jan/2011:20:54:07 +0000] "GET /tools/port-scan/result/?scan_host=cairosat.zapto.org&ports=2000&portscansubmit=Scan&port_start=&port_end= HTTP/1.1" 200 9640 "http://www.t1shopper.com/tools/port-scan/" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.2.8) Gecko/20100722 Firefox/2.0.0.3 \"MEGAUPLOAD 1.0\"" "-"';

$str4='91.55.106.90 - - [27/Dec/2010:17:16:07 +0000] "GET /ssi/t1shopper.js HTTP/1.1" 200 2012 "http://www.t1shopper.com/tools/port-scan/" "\"Bundestrojaner 2.0 - www.rettedeinefreiheit.de\"" "-"';

$str5='72.37.171.76 - - [22/Dec/2010:17:55:51 +0000] "GET /tools/calculate/ HTTP/1.0" 200 37778 "http://www.bing.com/search?q=\"kilobyte+to+megabyte\"+\"converter\"&src=IE-Address" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30; InfoPath.2; MS-RTC LM 8)" "-"';

$log_pattern = '#^([^ ]+) ([^ ]+) ([^ ]+) \[([^\]]+)\] "([^ ]+) ([^ ]+) ([^/]+)/([^"]+)" ([^ ]+) ([^ ]+) "([^"]*)" "([^"]+)" "([^"]+)"#';

preg_match($log_pattern, $str2, $matches);

print_r($matches);

Open in new window


PS: We've been working to get this Regex right for a few years but we keep having exceptions come up.  Here's the thread of past answers:
https://www.experts-exchange.com/Programming/Languages/Regular_Expressions/Q_26028925.html
https://www.experts-exchange.com/Web_Development/Web_Languages-Standards/PHP/Q_26184204.html
https://www.experts-exchange.com/Programming/Languages/Regular_Expressions/Q_25968268.html
https://www.experts-exchange.com/Programming/Languages/Regular_Expressions/Q_23544714.html
Comment
Watch Question
Most Valuable Expert 2011
Author of the Year 2014
Commented:
This problem has been solved!
Unlock 2 Answers and 7 Comments.
See Answers
Why Experts Exchange?

Experts Exchange always has the answer, or at the least points me in the correct direction! It is like having another employee that is extremely experienced.

Jim Murphy
Programmer at Smart IT Solutions

When asked, what has been your best career decision?

Deciding to stick with EE.

Mohamed Asif
Technical Department Head

Being involved with EE helped me to grow personally and professionally.

Carl Webster
CTP, Sr Infrastructure Consultant
Ask ANY Question

Connect with Certified Experts to gain insight and support on specific technology challenges including:

  • Troubleshooting
  • Research
  • Professional Opinions
Did You Know?

We've partnered with two important charities to provide clean water and computer science education to those who need it most. READ MORE