troubleshooting Question

how to grep urls from apache access logs and again run them using wget - for testing purpose

Avatar of jayatallen
jayatallen asked on
Apache Web ServerShell Scripting
8 Comments3 Solutions860 ViewsLast Modified:
hi Folks,

i want to run few urls against my application server for testing purpose.Basically these are urls from access logs and we want to test if these urls causing application slowness.
Is there any way i can grab the hits from access log and again invoke those urls using any kind of program or script.

Below is snippet from access log:

Open in new window




155.180.105.36 - - [09/Jul/2011:11:46:50 -0400] "GET /cm/Satellite?pagename=CYQ/Href&urlname=cyqorgan/am/wateress/aboutus HTTP/1.1" 200 29811
155.180.105.36 - - [09/Jul/2011:11:46:52 -0400] "GET /cm/Satellite?pagename=CYQ/Href&urlname=cyqorgan/am/wateress/aboutus HTTP/1.1" 200 29732
155.180.105.36 - - [09/Jul/2011:11:46:52 -0400] "GET /cm/Satellite?pagename=CYQ/Href&urlname=cyqorgan/am/wateress/aboutus HTTP/1.1" 200 29732
155.180.105.36 - - [09/Jul/2011:11:46:52 -0400] "GET /cm/Satellite?pagename=CYQ/Href&urlname=cyqorgan/am/wateress/aboutus HTTP/1.1" 200 29732
155.180.105.36 - - [09/Jul/2011:11:46:52 -0400] "GET /cm/Satellite?pagename=CYQ/Href&urlname=cyqorgan/am/wateress/aboutus HTTP/1.1" 200 29732

Open in new window


I was thinking to do awk and grep the url but it missing the hostname.
so, is there any way to append hostname and wget before the url and then put it in a shell script and run  it .
something like

Open in new window

wget hostname /cm/Satellite?pagename=CYQ/Href&urlname=cyqorgan/am/wateress/aboutus HTTP/1.1" 200 29732
wget hostname /cm/Satellite?pagename=CYQ/Href&urlname=cyqorgan/am/wateress/aboutus HTTP/1.1" 200 29732
wget hostname /cm/Satellite?pagename=CYQ/Href&urlname=cyqorgan/am/wateress/aboutus HTTP/1.1" 200 29732
wget hostname /cm/Satellite?pagename=CYQ/Href&urlname=cyqorgan/am/wateress/aboutus HTTP/1.1" 200 29732
wget hostname  /cm/Satellite?pagename=CYQ/Href&urlname=cyqorgan/am/wateress/aboutus HTTP/1.1" 200 29732
wget hostname  /cm/Satellite?pagename=CYQ/Href&urlname=cyqorgan/am/wateress/aboutus HTTP/1.1" 200 29732

Open in new window

ASKER CERTIFIED SOLUTION
Kamaraj Subramanian
Application Support Analyst

Our community of experts have been thoroughly vetted for their expertise and industry experience.

Log in to continue reading
Become an EE member today7-DAY FREE TRIAL
Members can start a 7-Day Free trial then enjoy unlimited access to the platform for $9.99/mo
View membership options
Unlock 3 Answers and 8 Comments.
or
Learn why we charge membership fees
We get it - no one likes a content blocker. Take one extra minute and find out why we block content.
See how we're fighting big data
The Value of Experts Exchange in My Daily IT Life

Experts Exchange (EE) has become my company's go-to resource to get answers. I've used EE to make decisions, solve problems and even save customers. OutagesIO has been a challenging project and... Keep reading >>

Mike

Owner of Outages.IO
Phoenix, Arizona, United States
Member Since 2016
Join a full scale community that combines the best parts of other tools into one platform.
Unlock 3 Answers and 8 Comments.
View membership options
“All of life is about relationships, and EE has made a virtual community a real community. It lifts everyone's boat.”
William Peck

Member since 2004