Solved

check unix curl command return value

Posted on 2016-09-27
7
61 Views
Last Modified: 2016-09-27
Hi Experts,
I am calling a webservice via curl on AIX:
    curl -k http://www.mocky.io/v2/57dd391e0f00007918a2d520 --user MyUser:MyUserPW-H "Content-Type: application/csv" -o myfile.csv

Now I'd like a way to check the return value of the curl command
I've tried:
retval=$(curl -k http://www.mocky.io/v2/57dd391e0f00007918a2d520 --user MyUser:MyUserPW-H "Content-Type: application/csv" -o myfile.csv)

but retval is always empty
Thank you for your support
0
Comment
Question by:mruff
  • 4
  • 3
7 Comments
 
LVL 68

Expert Comment

by:woolmilkporc
Comment Utility
Check the $? variable after completion.

curl -k http://www.mocky.io/v2/57dd391e0f00007918a2d520 --user MyUser:MyUserPW-H "Content-Type: application/csv" -o myfile.csv

RC=$?

if [[ $RC -ne 0 ]]; then
  echo "curl failed with returncode $RC"
    else
      echo "curl succeeded"
fi

If you want curl to exit with code 22 in case of a 404 response instead of outputting the document containing this 404 message provide the -f flag to curl:

curl -f -k http://www.mocky.io/v2/57dd391e0f00007918a2d520 --user MyUser:MyUserPW-H "Content-Type: application/csv" -o myfile.csv
...
...

The curl man page contains an exhaustive list of possible return values:
https://linux.die.net/man/1/curl :

1
Unsupported protocol. This build of curl has no support for this protocol.
2
Failed to initialize.
3
URL malformed. The syntax was not correct.
5
Couldn't resolve proxy. The given proxy host could not be resolved.
6
Couldn't resolve host. The given remote host was not resolved.
7
Failed to connect to host.
8
FTP weird server reply. The server sent data curl couldn't parse.
9
FTP access denied. The server denied login or denied access to the particular resource or directory you wanted to reach. Most often you tried to change to a directory that doesn't exist on the server.
11
FTP weird PASS reply. Curl couldn't parse the reply sent to the PASS request.
13
FTP weird PASV reply, Curl couldn't parse the reply sent to the PASV request.
14
FTP weird 227 format. Curl couldn't parse the 227-line the server sent.
15
FTP can't get host. Couldn't resolve the host IP we got in the 227-line.
17
FTP couldn't set binary. Couldn't change transfer method to binary.
18
Partial file. Only a part of the file was transferred.
19
FTP couldn't download/access the given file, the RETR (or similar) command failed.
21
FTP quote error. A quote command returned error from the server.
22
HTTP page not retrieved. The requested url was not found or returned another error with the HTTP error code being 400 or above. This return code only appears if -f/--fail is used.
23
Write error. Curl couldn't write data to a local filesystem or similar.
25
FTP couldn't STOR file. The server denied the STOR operation, used for FTP uploading.
26
Read error. Various reading problems.
27
Out of memory. A memory allocation request failed.
28
Operation timeout. The specified time-out period was reached according to the conditions.
30
FTP PORT failed. The PORT command failed. Not all FTP servers support the PORT command, try doing a transfer using PASV instead!
31
FTP couldn't use REST. The REST command failed. This command is used for resumed FTP transfers.
33
HTTP range error. The range "command" didn't work.
34
HTTP post error. Internal post-request generation error.
35
SSL connect error. The SSL handshaking failed.
36
FTP bad download resume. Couldn't continue an earlier aborted download.
37
FILE couldn't read file. Failed to open the file. Permissions?
38
LDAP cannot bind. LDAP bind operation failed.
39
LDAP search failed.
41
Function not found. A required LDAP function was not found.
42
Aborted by callback. An application told curl to abort the operation.
43
Internal error. A function was called with a bad parameter.
45
Interface error. A specified outgoing interface could not be used.
47
Too many redirects. When following redirects, curl hit the maximum amount.
48
Unknown TELNET option specified.
49
Malformed telnet option.
51
The peer's SSL certificate or SSH MD5 fingerprint was not ok.
52
The server didn't reply anything, which here is considered an error.
53
SSL crypto engine not found.
54
Cannot set SSL crypto engine as default.
55
Failed sending network data.
56
Failure in receiving network data.
58
Problem with the local certificate.
59
Couldn't use specified SSL cipher.
60
Peer certificate cannot be authenticated with known CA certificates.
61
Unrecognized transfer encoding.
62
Invalid LDAP URL.
63
Maximum file size exceeded.
64
Requested FTP SSL level failed.
65
Sending the data requires a rewind that failed.
66
Failed to initialise SSL Engine.
67
The user name, password, or similar was not accepted and curl failed to log in.
68
File not found on TFTP server.
69
Permission problem on TFTP server.
70
Out of disk space on TFTP server.
71
Illegal TFTP operation.
72
Unknown TFTP transfer ID.
73
File already exists (TFTP).
74
No such user (TFTP).
75
Character conversion failed.
76
Character conversion functions required.
77
Problem with reading the SSL CA cert (path? access rights?).
78
The resource referenced in the URL does not exist.
79
An unspecified error occurred during the SSH session.
80
Failed to shut down the SSL connection.
82
Could not load CRL file, missing or wrong format (added in 7.19.0).
83
Issuer check failed (added in 7.19.0).
0
 

Author Comment

by:mruff
Comment Utility
Perfect, thank you, I've tested with linux (opesuse) and it worked, have to test it tomorrow at the office with AIX, if it works with AIX as well, which I expect, then I assign you the points
0
 
LVL 68

Expert Comment

by:woolmilkporc
Comment Utility
AIX has ksh by default, and this shell will (in our case) show exactly the same behavior as bash under Linux.
Actually, I use to work with AIX/ksh way more often than with Linux/bash and I always apply a scheme like the above for testing return values.

You'll be fine, I'm sure!

wmp
0
6 Surprising Benefits of Threat Intelligence

All sorts of threat intelligence is available on the web. Intelligence you can learn from, and use to anticipate and prepare for future attacks.

 

Author Comment

by:mruff
Comment Utility
I've got another small question:
How can I evaluate the return value AND log the output of the curl command to a file

    curl -f -H "Content-Type: application/csv" $curURL -o ${fileDestintionPath}${extractDate}_${resultFileName}  2>&1 | tee -a ${extractDate}_logile.txt
    RC=$?
    echo 'curlretval:'$RC
    curl -f -H "Content-Type: application/csv" bla -o ${fileDestintionPath}${extractDate}_${resultFileName}  | tee -a ${extractDate}_logile.txt
    RC=$?
    echo 'curlretval:'$RC
 
both solution do not work


Many thanks for this solution
0
 
LVL 68

Accepted Solution

by:
woolmilkporc earned 500 total points
Comment Utility
In this case you can't use "-f" because with this flag curl won't write anything to stdout/stderr, it will just set the return value.

This should work:

curl -H "Content-Type: application/csv" $curURL -o ${fileDestintionPath}${extractDate}_${resultFileName}  2>&1 | tee -a ${extractDate}_logile.txt

With the above the error messages will go to the log file and additionally to your terminal.

To populate just the log file and keep the terminal silent use

curl -H "Content-Type: application/csv" $curURL -o ${fileDestintionPath}${extractDate}_${resultFileName}  > ${extractDate}_logile.txt 2>&1

Checking for a 404 result must be done by examining the output file ${fileDestintionPath}${extractDate}_${resultFileName}, e. g. with grep:

grep -q "404" ${fileDestintionPath}${extractDate}_${resultFileName} && echo 404 found!

"grep 404" is just a simple example, better use a more meaningful string, depending on what the URLs you're going to extract would provide.
0
 

Author Comment

by:mruff
Comment Utility
Many thanks, found a solution with evaluating return value and write to log file


this works very well

checkRetVal () {
      #echo 'checkRetVal:curlretval:'$1
        if [[ $1 -ne 0 ]]; then
          echo 'failed with returncode:' $1 'please check the log file: ' ${currentpath}'/'${extractDate}_logfile.txt
        echo 'curlcmd failed with returncode:' $1 >>  ${extractDate}_logile.txt
              exit 255
        else
      echo 'curlcmd succeeded' >> ${extractDate}_logile.txt
      fi
      return 0      
}

    RC=$?
    #echo 'curlretval:'$RC
    checkRetVal $RC
    curl -f -H "Content-Type: application/csv" bla -o ${fileDestintionPath}${extractDate}_${resultFileName}  >> ${extractDate}_logfile.txt 2>&1
    RC=$?
    #echo 'curlretval:'$RC
    checkRetVal $RC
    curl -f -H "Content-Type: application/csv" $curURL -o ${fileDestintionPath}${extractDate}_${resultFileName}  >> ${extractDate}_logfile.txt 2>&1
    RC=$?
    checkRetVal $RC

the result in the console:
failed with returncode: 6 please check the log file:  /home/wbk/pgm/2016-09-27-21-47-18_logile.txt

and the log file:

ore 2016-09-27-21-51-50_logfile.txt
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    34    0    34    0     0    197      0 --:--:-- --:--:-- --:--:--   198
curlcmd succeeded
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (6) Could not resolve host: bla
curlcmd failed with returncode: 6
0
 

Author Closing Comment

by:mruff
Comment Utility
Perfect solution, thx
0

Featured Post

Maximize Your Threat Intelligence Reporting

Reporting is one of the most important and least talked about aspects of a world-class threat intelligence program. Here’s how to do it right.

Join & Write a Comment

If you have a server on collocation with the super-fast CPU, that doesn't mean that you get it running at full power. Here is a preamble. When doing inventory of Linux servers, that I'm administering, I've found that some of them are running on l…
Join Greg Farro and Ethan Banks from Packet Pushers (http://packetpushers.net/podcast/podcasts/pq-show-93-smart-network-monitoring-paessler-sponsored/) and Greg Ross from Paessler (https://www.paessler.com/prtg) for a discussion about smart network …
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…
This video shows how to set up a shell script to accept a positional parameter when called, pass that to a SQL script, accept the output from the statement back and then manipulate it in the Shell.

743 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now