Execute multiple curl cmds with sleep and send output to file

First off, I am, in no way a Unix/Linux expert, so please bear with me. Also, I sure hope I've selected the proper topics for this question...

I need to run a few thousand curl cmds but I'm having difficulties figuring out how to go about doing this.

1. The curl is an XPOST (e.g. JSON)
For example:
curl -XPOST -H 'name1: value1' -H 'name2: value2' -H "Content-type: application/json" -d '{ "name3": [{"blah1": "blah1","aaa1": "blah2"}]}' 'http://domain.com'
curl -XPOST -H 'name1: value1' -H 'name2: value2' -H "Content-type: application/json" -d '{ "name3": [{"blah2": "blah2","aaa2": "blah2"}]}' 'http://domain.com'
curl -XPOST -H 'name1: value1' -H 'name2: value2' -H "Content-type: application/json" -d '{ "name3": [{"blah3": "blah3","aaa3": "blah2"}]}' 'http://domain.com'

The thing is I need to run this in a script so that there is a delay between each one. Also, once 1 curl is run, I get back a result which I'd like to save.

I thought I could just load all of the curl cmds in a text file and run something like:

while read line; do $line; sleep 5; done < input_curl.txt >> output_curl.txt

The resulting curl response would be something like:
{"name2":"value2","resultname1":resultvalue1,"resultvalue2":"resultname2","resultname3": etc etc...

My thought was that I'd be able to run each curl, line by line, then have it sleep for 5 seconds, and then run the next one. And, the curl result would be writen to output_curl.txt. Of course, that doesn't work.

Any help would be greatly appreciated.

Thanks,
Larry
biglarrrrAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

omarfaridCommented:
where the curl lines will be taken from?
0
biglarrrrAuthor Commented:
Hi omarfarid,

As I wrote, there are thousands of them, so I thought it would be best to get all of the curls into a file, 1 line for each, and execute that file from the cmd line. I'm not saying that's the correct way going about doing this though.

Thanks,
Larry
0
omarfaridCommented:
the while loop should work

while read line
do
sh $line
sleep 5
done < input_curl.txt >> output_curl.txt

but why you want to send all output to one file?
0
Ultimate Tool Kit for Technology Solution Provider

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.

biglarrrrAuthor Commented:
Not sure about the "sh". The input file is a text file. The reason why I need the curl response writen to a file is because I need to make sure there are no errors being returned. And, since I'm not running these curl cmds one at a time, I'd never see the results unless it's written to a file. That's why I used >> so that each curl response would get appended.
0
omarfaridCommented:
what is your question for sh ?
0
biglarrrrAuthor Commented:
When I run that with the sh I get:

-bash-4.1$ while read line; do sh $line; sleep 5; done < input_curl.txt >> output_curl.txt
/usr/bin/curl: /usr/bin/curl: cannot execute binary file
0
omarfaridCommented:
sorry, you may remove sh

the while loop should do what you want, but if you want to capture errors the you need to redirect stderr as well:

while read line
do
$line
sleep 5
done < input_curl.txt >> output_curl.txt 2>&1
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
biglarrrrAuthor Commented:
That did it! Thank you so much omarfarid!
0
biglarrrrAuthor Commented:
Thank you!
0
omarfaridCommented:
Welcome :)
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
JSON

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.