jayatallen
asked on
how to grep urls from apache access logs and again run them using wget - for testing purpose
hi Folks,
i want to run few urls against my application server for testing purpose.Basically these are urls from access logs and we want to test if these urls causing application slowness.
Is there any way i can grab the hits from access log and again invoke those urls using any kind of program or script.
Below is snippet from access log:
155.180.105.36 - - [09/Jul/2011:11:46:50 -0400] "GET /cm/Satellite?pagename=CYQ
155.180.105.36 - - [09/Jul/2011:11:46:52 -0400] "GET /cm/Satellite?pagename=CYQ
155.180.105.36 - - [09/Jul/2011:11:46:52 -0400] "GET /cm/Satellite?pagename=CYQ
155.180.105.36 - - [09/Jul/2011:11:46:52 -0400] "GET /cm/Satellite?pagename=CYQ
155.180.105.36 - - [09/Jul/2011:11:46:52 -0400] "GET /cm/Satellite?pagename=CYQ
I was thinking to do awk and grep the url but it missing the hostname.
so, is there any way to append hostname and wget before the url and then put it in a shell script and run it .
something like
wget hostname /cm/Satellite?pagename=CYQ
wget hostname /cm/Satellite?pagename=CYQ
wget hostname /cm/Satellite?pagename=CYQ
wget hostname /cm/Satellite?pagename=CYQ
wget hostname /cm/Satellite?pagename=CYQ
i want to run few urls against my application server for testing purpose.Basically these are urls from access logs and we want to test if these urls causing application slowness.
Is there any way i can grab the hits from access log and again invoke those urls using any kind of program or script.
Below is snippet from access log:
155.180.105.36 - - [09/Jul/2011:11:46:50 -0400] "GET /cm/Satellite?pagename=CYQ
155.180.105.36 - - [09/Jul/2011:11:46:52 -0400] "GET /cm/Satellite?pagename=CYQ
155.180.105.36 - - [09/Jul/2011:11:46:52 -0400] "GET /cm/Satellite?pagename=CYQ
155.180.105.36 - - [09/Jul/2011:11:46:52 -0400] "GET /cm/Satellite?pagename=CYQ
155.180.105.36 - - [09/Jul/2011:11:46:52 -0400] "GET /cm/Satellite?pagename=CYQ
I was thinking to do awk and grep the url but it missing the hostname.
so, is there any way to append hostname and wget before the url and then put it in a shell script and run it .
something like
wget hostname /cm/Satellite?pagename=CYQ
wget hostname /cm/Satellite?pagename=CYQ
wget hostname /cm/Satellite?pagename=CYQ
wget hostname /cm/Satellite?pagename=CYQ
wget hostname /cm/Satellite?pagename=CYQ
Last Comment
ASKER
thank you your reply.
copied few lines from access log to a file named it test and then used given command
bash-3.00$ awk -F"\"" {printf("wget hostname %s %s\n",$2,$3)}' test
bash: syntax error near unexpected token `('
bash-3.00$
Please help.
copied few lines from access log to a file named it test and then used given command
bash-3.00$ awk -F"\"" {printf("wget hostname %s %s\n",$2,$3)}' test
bash: syntax error near unexpected token `('
bash-3.00$
Please help.
ASKER CERTIFIED SOLUTION
THIS SOLUTION IS ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
ASKER
tried ...very close..
after runing given command, one of line from output pasted below:
wget hostname GET /cm/Satellite?blobcol=urld
any way we can change it to like:
wget hostname/cm/Satellite?blob
I mean how i can remove GET and no spaces between hostname/cm.
thank you very much.
after runing given command, one of line from output pasted below:
wget hostname GET /cm/Satellite?blobcol=urld
any way we can change it to like:
wget hostname/cm/Satellite?blob
I mean how i can remove GET and no spaces between hostname/cm.
thank you very much.
SOLUTION
THIS SOLUTION IS ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
Good, if you got the answer, then please close this question :)
SOLUTION
THIS SOLUTION IS ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
ASKER
answer provided by Guru helped..i just copied it.
Apache Web Server
The Apache HTTP Server is a secure, efficient and extensible server that provides HTTP services in sync with the current HTTP standards. Typically Apache is run on a Unix-like operating system, but it is available for a wide variety of operating systems, including Linux, Novell NetWare, Mac OS-X and Windows. Released under the Apache License, Apache is open-source software.
21K
Questions
--
Followers
--
Top Experts
Get a personalized solution from industry experts
TRUSTED BY
Open in new window