We help IT Professionals succeed at work.
Get Started

how to grep urls from apache access logs and again run them using wget - for testing purpose

jayatallen
jayatallen asked
on
8 Comments
3 Solutions
858 Views
Last Modified: 2012-05-11
hi Folks,

i want to run few urls against my application server for testing purpose.Basically these are urls from access logs and we want to test if these urls causing application slowness.
Is there any way i can grab the hits from access log and again invoke those urls using any kind of program or script.

Below is snippet from access log:

Open in new window




155.180.105.36 - - [09/Jul/2011:11:46:50 -0400] "GET /cm/Satellite?pagename=CYQ/Href&urlname=cyqorgan/am/wateress/aboutus HTTP/1.1" 200 29811
155.180.105.36 - - [09/Jul/2011:11:46:52 -0400] "GET /cm/Satellite?pagename=CYQ/Href&urlname=cyqorgan/am/wateress/aboutus HTTP/1.1" 200 29732
155.180.105.36 - - [09/Jul/2011:11:46:52 -0400] "GET /cm/Satellite?pagename=CYQ/Href&urlname=cyqorgan/am/wateress/aboutus HTTP/1.1" 200 29732
155.180.105.36 - - [09/Jul/2011:11:46:52 -0400] "GET /cm/Satellite?pagename=CYQ/Href&urlname=cyqorgan/am/wateress/aboutus HTTP/1.1" 200 29732
155.180.105.36 - - [09/Jul/2011:11:46:52 -0400] "GET /cm/Satellite?pagename=CYQ/Href&urlname=cyqorgan/am/wateress/aboutus HTTP/1.1" 200 29732

Open in new window


I was thinking to do awk and grep the url but it missing the hostname.
so, is there any way to append hostname and wget before the url and then put it in a shell script and run  it .
something like

Open in new window

wget hostname /cm/Satellite?pagename=CYQ/Href&urlname=cyqorgan/am/wateress/aboutus HTTP/1.1" 200 29732
wget hostname /cm/Satellite?pagename=CYQ/Href&urlname=cyqorgan/am/wateress/aboutus HTTP/1.1" 200 29732
wget hostname /cm/Satellite?pagename=CYQ/Href&urlname=cyqorgan/am/wateress/aboutus HTTP/1.1" 200 29732
wget hostname /cm/Satellite?pagename=CYQ/Href&urlname=cyqorgan/am/wateress/aboutus HTTP/1.1" 200 29732
wget hostname  /cm/Satellite?pagename=CYQ/Href&urlname=cyqorgan/am/wateress/aboutus HTTP/1.1" 200 29732
wget hostname  /cm/Satellite?pagename=CYQ/Href&urlname=cyqorgan/am/wateress/aboutus HTTP/1.1" 200 29732

Open in new window

Comment
Watch Question
Application Support Analyst
Commented:
This problem has been solved!
Unlock 3 Answers and 8 Comments.
See Answers
Why Experts Exchange?

Experts Exchange always has the answer, or at the least points me in the correct direction! It is like having another employee that is extremely experienced.

Jim Murphy
Programmer at Smart IT Solutions

When asked, what has been your best career decision?

Deciding to stick with EE.

Mohamed Asif
Technical Department Head

Being involved with EE helped me to grow personally and professionally.

Carl Webster
CTP, Sr Infrastructure Consultant
Ask ANY Question

Connect with Certified Experts to gain insight and support on specific technology challenges including:

  • Troubleshooting
  • Research
  • Professional Opinions
Did You Know?

We've partnered with two important charities to provide clean water and computer science education to those who need it most. READ MORE