asked on Execute multiple curl cmds with sleep and send output to file
First off, I am, in no way a Unix/Linux expert, so please bear with me. Also, I sure hope I've selected the proper topics for this question...
I need to run a few thousand curl cmds but I'm having difficulties figuring out how to go about doing this.
1. The curl is an XPOST (e.g. JSON)
For example:
curl -XPOST -H 'name1: value1' -H 'name2: value2' -H "Content-type: application/json" -d '{ "name3": [{"blah1": "blah1","aaa1": "blah2"}]}' 'http://domain.com'
curl -XPOST -H 'name1: value1' -H 'name2: value2' -H "Content-type: application/json" -d '{ "name3": [{"blah2": "blah2","aaa2": "blah2"}]}' 'http://domain.com'
curl -XPOST -H 'name1: value1' -H 'name2: value2' -H "Content-type: application/json" -d '{ "name3": [{"blah3": "blah3","aaa3": "blah2"}]}' 'http://domain.com'
The thing is I need to run this in a script so that there is a delay between each one. Also, once 1 curl is run, I get back a result which I'd like to save.
I thought I could just load all of the curl cmds in a text file and run something like:
while read line; do $line; sleep 5; done < input_curl.txt >> output_curl.txt
The resulting curl response would be something like:
{"name2":"value2","resultn
My thought was that I'd be able to run each curl, line by line, then have it sleep for 5 seconds, and then run the next one. And, the curl result would be writen to output_curl.txt. Of course, that doesn't work.
Any help would be greatly appreciated.
Thanks,
Larry
I need to run a few thousand curl cmds but I'm having difficulties figuring out how to go about doing this.
1. The curl is an XPOST (e.g. JSON)
For example:
curl -XPOST -H 'name1: value1' -H 'name2: value2' -H "Content-type: application/json" -d '{ "name3": [{"blah1": "blah1","aaa1": "blah2"}]}' 'http://domain.com'
curl -XPOST -H 'name1: value1' -H 'name2: value2' -H "Content-type: application/json" -d '{ "name3": [{"blah2": "blah2","aaa2": "blah2"}]}' 'http://domain.com'
curl -XPOST -H 'name1: value1' -H 'name2: value2' -H "Content-type: application/json" -d '{ "name3": [{"blah3": "blah3","aaa3": "blah2"}]}' 'http://domain.com'
The thing is I need to run this in a script so that there is a delay between each one. Also, once 1 curl is run, I get back a result which I'd like to save.
I thought I could just load all of the curl cmds in a text file and run something like:
while read line; do $line; sleep 5; done < input_curl.txt >> output_curl.txt
The resulting curl response would be something like:
{"name2":"value2","resultn
My thought was that I'd be able to run each curl, line by line, then have it sleep for 5 seconds, and then run the next one. And, the curl result would be writen to output_curl.txt. Of course, that doesn't work.
Any help would be greatly appreciated.
Thanks,
Larry
JSONLinuxUnix OSSystem Programming
Last Comment
omarfarid
where the curl lines will be taken from?
ASKER
Hi omarfarid,
As I wrote, there are thousands of them, so I thought it would be best to get all of the curls into a file, 1 line for each, and execute that file from the cmd line. I'm not saying that's the correct way going about doing this though.
Thanks,
Larry
As I wrote, there are thousands of them, so I thought it would be best to get all of the curls into a file, 1 line for each, and execute that file from the cmd line. I'm not saying that's the correct way going about doing this though.
Thanks,
Larry
the while loop should work
while read line
do
sh $line
sleep 5
done < input_curl.txt >> output_curl.txt
but why you want to send all output to one file?
while read line
do
sh $line
sleep 5
done < input_curl.txt >> output_curl.txt
but why you want to send all output to one file?
ASKER
Not sure about the "sh". The input file is a text file. The reason why I need the curl response writen to a file is because I need to make sure there are no errors being returned. And, since I'm not running these curl cmds one at a time, I'd never see the results unless it's written to a file. That's why I used >> so that each curl response would get appended.
what is your question for sh ?
ASKER
When I run that with the sh I get:
-bash-4.1$ while read line; do sh $line; sleep 5; done < input_curl.txt >> output_curl.txt
/usr/bin/curl: /usr/bin/curl: cannot execute binary file
-bash-4.1$ while read line; do sh $line; sleep 5; done < input_curl.txt >> output_curl.txt
/usr/bin/curl: /usr/bin/curl: cannot execute binary file
ASKER CERTIFIED SOLUTION
THIS SOLUTION ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
ASKER
That did it! Thank you so much omarfarid!
ASKER
Thank you!
Welcome :)