Create well-organized and polished visualizations of your virtual and backup environments when planning VMware vSphere, Microsoft Hyper-V or Veeam deployments. It helps you to gain better visibility and valuable business insights.
Course Of The Month
8 days, 14 hours left to enrollBecome a Premium Member and unlock a new, free course in leading technologies each month.
Solved
Execute multiple curl cmds with sleep and send output to file
Posted on 2016-08-16
First off, I am, in no way a Unix/Linux expert, so please bear with me. Also, I sure hope I've selected the proper topics for this question...
I need to run a few thousand curl cmds but I'm having difficulties figuring out how to go about doing this.
1. The curl is an XPOST (e.g. JSON)
For example:
curl -XPOST -H 'name1: value1' -H 'name2: value2' -H "Content-type: application/json" -d '{ "name3": [{"blah1": "blah1","aaa1": "blah2"}]}' 'http://domain.com'
curl -XPOST -H 'name1: value1' -H 'name2: value2' -H "Content-type: application/json" -d '{ "name3": [{"blah2": "blah2","aaa2": "blah2"}]}' 'http://domain.com'
curl -XPOST -H 'name1: value1' -H 'name2: value2' -H "Content-type: application/json" -d '{ "name3": [{"blah3": "blah3","aaa3": "blah2"}]}' 'http://domain.com'
The thing is I need to run this in a script so that there is a delay between each one. Also, once 1 curl is run, I get back a result which I'd like to save.
I thought I could just load all of the curl cmds in a text file and run something like:
while read line; do $line; sleep 5; done < input_curl.txt >> output_curl.txt
The resulting curl response would be something like:
{"name2":"value2","resultn
My thought was that I'd be able to run each curl, line by line, then have it sleep for 5 seconds, and then run the next one. And, the curl result would be writen to output_curl.txt. Of course, that doesn't work.
Any help would be greatly appreciated.
Thanks,
Larry
I need to run a few thousand curl cmds but I'm having difficulties figuring out how to go about doing this.
1. The curl is an XPOST (e.g. JSON)
For example:
curl -XPOST -H 'name1: value1' -H 'name2: value2' -H "Content-type: application/json" -d '{ "name3": [{"blah1": "blah1","aaa1": "blah2"}]}' 'http://domain.com'
curl -XPOST -H 'name1: value1' -H 'name2: value2' -H "Content-type: application/json" -d '{ "name3": [{"blah2": "blah2","aaa2": "blah2"}]}' 'http://domain.com'
curl -XPOST -H 'name1: value1' -H 'name2: value2' -H "Content-type: application/json" -d '{ "name3": [{"blah3": "blah3","aaa3": "blah2"}]}' 'http://domain.com'
The thing is I need to run this in a script so that there is a delay between each one. Also, once 1 curl is run, I get back a result which I'd like to save.
I thought I could just load all of the curl cmds in a text file and run something like:
while read line; do $line; sleep 5; done < input_curl.txt >> output_curl.txt
The resulting curl response would be something like:
{"name2":"value2","resultn
My thought was that I'd be able to run each curl, line by line, then have it sleep for 5 seconds, and then run the next one. And, the curl result would be writen to output_curl.txt. Of course, that doesn't work.
Any help would be greatly appreciated.
Thanks,
Larry
[X]
Welcome to Experts Exchange
Add your voice to the tech community where 5M+ people just like you are talking about what matters.
- Help others & share knowledge
- Earn cash & points
- Learn & ask questions
5-
5
10 Comments
Hi omarfarid,
As I wrote, there are thousands of them, so I thought it would be best to get all of the curls into a file, 1 line for each, and execute that file from the cmd line. I'm not saying that's the correct way going about doing this though.
Thanks,
Larry
As I wrote, there are thousands of them, so I thought it would be best to get all of the curls into a file, 1 line for each, and execute that file from the cmd line. I'm not saying that's the correct way going about doing this though.
Thanks,
Larry
the while loop should work
while read line
do
sh $line
sleep 5
done < input_curl.txt >> output_curl.txt
but why you want to send all output to one file?
while read line
do
sh $line
sleep 5
done < input_curl.txt >> output_curl.txt
but why you want to send all output to one file?
Not sure about the "sh". The input file is a text file. The reason why I need the curl response writen to a file is because I need to make sure there are no errors being returned. And, since I'm not running these curl cmds one at a time, I'd never see the results unless it's written to a file. That's why I used >> so that each curl response would get appended.
When I run that with the sh I get:
-bash-4.1$ while read line; do sh $line; sleep 5; done < input_curl.txt >> output_curl.txt
/usr/bin/curl: /usr/bin/curl: cannot execute binary file
-bash-4.1$ while read line; do sh $line; sleep 5; done < input_curl.txt >> output_curl.txt
/usr/bin/curl: /usr/bin/curl: cannot execute binary file
sorry, you may remove sh
the while loop should do what you want, but if you want to capture errors the you need to redirect stderr as well:
while read line
do
$line
sleep 5
done < input_curl.txt >> output_curl.txt 2>&1
the while loop should do what you want, but if you want to capture errors the you need to redirect stderr as well:
while read line
do
$line
sleep 5
done < input_curl.txt >> output_curl.txt 2>&1
Featured Post
Back up all your Microsoft Windows Server – on-premises, in remote locations, in private and hybrid clouds. Your entire Windows Server will be backed up in one easy step with patented, block-level disk imaging. We achieve RTOs (recovery time objectives) as low as 15 seconds.
Question has a verified solution.
If you are experiencing a similar issue, please ask a related question
Introduction
JSON is an acronym for JavaScript Object Notation. It is a text-string data transport mechanism, capable of representing simple or complex data structures in a consistent and easy-to-read manner. Similar in concept to XML, but more e…
It’s 2016. Password authentication should be dead — or at least close to dying. But, unfortunately, it has not traversed Quagga stage yet. Using password authentication is like laundering hotel guest linens with a washboard — it’s Passé.
Learn how to find files with the shell using the find and locate commands.
Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
This video will show you how to get GIT to work in Eclipse.
It will walk you through how to install the EGit plugin in eclipse and how to checkout an existing repository.
Suggested Courses
Course of the Month8 days, 14 hours left to enroll
734 members asked questions and received personalized solutions in the past 7 days.
Join the community of 500,000 technology professionals and ask your questions.