Solved

check unix curl command return value

Posted on 2016-09-27
7
152 Views
Last Modified: 2016-09-27
Hi Experts,
I am calling a webservice via curl on AIX:
    curl -k http://www.mocky.io/v2/57dd391e0f00007918a2d520 --user MyUser:MyUserPW-H "Content-Type: application/csv" -o myfile.csv

Now I'd like a way to check the return value of the curl command
I've tried:
retval=$(curl -k http://www.mocky.io/v2/57dd391e0f00007918a2d520 --user MyUser:MyUserPW-H "Content-Type: application/csv" -o myfile.csv)

but retval is always empty
Thank you for your support
0
Comment
Question by:mruff
  • 4
  • 3
7 Comments
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 41818250
Check the $? variable after completion.

curl -k http://www.mocky.io/v2/57dd391e0f00007918a2d520 --user MyUser:MyUserPW-H "Content-Type: application/csv" -o myfile.csv

RC=$?

if [[ $RC -ne 0 ]]; then
  echo "curl failed with returncode $RC"
    else
      echo "curl succeeded"
fi

If you want curl to exit with code 22 in case of a 404 response instead of outputting the document containing this 404 message provide the -f flag to curl:

curl -f -k http://www.mocky.io/v2/57dd391e0f00007918a2d520 --user MyUser:MyUserPW-H "Content-Type: application/csv" -o myfile.csv
...
...

The curl man page contains an exhaustive list of possible return values:
https://linux.die.net/man/1/curl :

1
Unsupported protocol. This build of curl has no support for this protocol.
2
Failed to initialize.
3
URL malformed. The syntax was not correct.
5
Couldn't resolve proxy. The given proxy host could not be resolved.
6
Couldn't resolve host. The given remote host was not resolved.
7
Failed to connect to host.
8
FTP weird server reply. The server sent data curl couldn't parse.
9
FTP access denied. The server denied login or denied access to the particular resource or directory you wanted to reach. Most often you tried to change to a directory that doesn't exist on the server.
11
FTP weird PASS reply. Curl couldn't parse the reply sent to the PASS request.
13
FTP weird PASV reply, Curl couldn't parse the reply sent to the PASV request.
14
FTP weird 227 format. Curl couldn't parse the 227-line the server sent.
15
FTP can't get host. Couldn't resolve the host IP we got in the 227-line.
17
FTP couldn't set binary. Couldn't change transfer method to binary.
18
Partial file. Only a part of the file was transferred.
19
FTP couldn't download/access the given file, the RETR (or similar) command failed.
21
FTP quote error. A quote command returned error from the server.
22
HTTP page not retrieved. The requested url was not found or returned another error with the HTTP error code being 400 or above. This return code only appears if -f/--fail is used.
23
Write error. Curl couldn't write data to a local filesystem or similar.
25
FTP couldn't STOR file. The server denied the STOR operation, used for FTP uploading.
26
Read error. Various reading problems.
27
Out of memory. A memory allocation request failed.
28
Operation timeout. The specified time-out period was reached according to the conditions.
30
FTP PORT failed. The PORT command failed. Not all FTP servers support the PORT command, try doing a transfer using PASV instead!
31
FTP couldn't use REST. The REST command failed. This command is used for resumed FTP transfers.
33
HTTP range error. The range "command" didn't work.
34
HTTP post error. Internal post-request generation error.
35
SSL connect error. The SSL handshaking failed.
36
FTP bad download resume. Couldn't continue an earlier aborted download.
37
FILE couldn't read file. Failed to open the file. Permissions?
38
LDAP cannot bind. LDAP bind operation failed.
39
LDAP search failed.
41
Function not found. A required LDAP function was not found.
42
Aborted by callback. An application told curl to abort the operation.
43
Internal error. A function was called with a bad parameter.
45
Interface error. A specified outgoing interface could not be used.
47
Too many redirects. When following redirects, curl hit the maximum amount.
48
Unknown TELNET option specified.
49
Malformed telnet option.
51
The peer's SSL certificate or SSH MD5 fingerprint was not ok.
52
The server didn't reply anything, which here is considered an error.
53
SSL crypto engine not found.
54
Cannot set SSL crypto engine as default.
55
Failed sending network data.
56
Failure in receiving network data.
58
Problem with the local certificate.
59
Couldn't use specified SSL cipher.
60
Peer certificate cannot be authenticated with known CA certificates.
61
Unrecognized transfer encoding.
62
Invalid LDAP URL.
63
Maximum file size exceeded.
64
Requested FTP SSL level failed.
65
Sending the data requires a rewind that failed.
66
Failed to initialise SSL Engine.
67
The user name, password, or similar was not accepted and curl failed to log in.
68
File not found on TFTP server.
69
Permission problem on TFTP server.
70
Out of disk space on TFTP server.
71
Illegal TFTP operation.
72
Unknown TFTP transfer ID.
73
File already exists (TFTP).
74
No such user (TFTP).
75
Character conversion failed.
76
Character conversion functions required.
77
Problem with reading the SSL CA cert (path? access rights?).
78
The resource referenced in the URL does not exist.
79
An unspecified error occurred during the SSH session.
80
Failed to shut down the SSL connection.
82
Could not load CRL file, missing or wrong format (added in 7.19.0).
83
Issuer check failed (added in 7.19.0).
0
 

Author Comment

by:mruff
ID: 41818305
Perfect, thank you, I've tested with linux (opesuse) and it worked, have to test it tomorrow at the office with AIX, if it works with AIX as well, which I expect, then I assign you the points
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 41818333
AIX has ksh by default, and this shell will (in our case) show exactly the same behavior as bash under Linux.
Actually, I use to work with AIX/ksh way more often than with Linux/bash and I always apply a scheme like the above for testing return values.

You'll be fine, I'm sure!

wmp
0
Master Your Team's Linux and Cloud Stack

Come see why top tech companies like Mailchimp and Media Temple use Linux Academy to build their employee training programs.

 

Author Comment

by:mruff
ID: 41818437
I've got another small question:
How can I evaluate the return value AND log the output of the curl command to a file

    curl -f -H "Content-Type: application/csv" $curURL -o ${fileDestintionPath}${extractDate}_${resultFileName}  2>&1 | tee -a ${extractDate}_logile.txt
    RC=$?
    echo 'curlretval:'$RC
    curl -f -H "Content-Type: application/csv" bla -o ${fileDestintionPath}${extractDate}_${resultFileName}  | tee -a ${extractDate}_logile.txt
    RC=$?
    echo 'curlretval:'$RC
 
both solution do not work


Many thanks for this solution
0
 
LVL 68

Accepted Solution

by:
woolmilkporc earned 500 total points
ID: 41818457
In this case you can't use "-f" because with this flag curl won't write anything to stdout/stderr, it will just set the return value.

This should work:

curl -H "Content-Type: application/csv" $curURL -o ${fileDestintionPath}${extractDate}_${resultFileName}  2>&1 | tee -a ${extractDate}_logile.txt

With the above the error messages will go to the log file and additionally to your terminal.

To populate just the log file and keep the terminal silent use

curl -H "Content-Type: application/csv" $curURL -o ${fileDestintionPath}${extractDate}_${resultFileName}  > ${extractDate}_logile.txt 2>&1

Checking for a 404 result must be done by examining the output file ${fileDestintionPath}${extractDate}_${resultFileName}, e. g. with grep:

grep -q "404" ${fileDestintionPath}${extractDate}_${resultFileName} && echo 404 found!

"grep 404" is just a simple example, better use a more meaningful string, depending on what the URLs you're going to extract would provide.
0
 

Author Comment

by:mruff
ID: 41818605
Many thanks, found a solution with evaluating return value and write to log file


this works very well

checkRetVal () {
      #echo 'checkRetVal:curlretval:'$1
        if [[ $1 -ne 0 ]]; then
          echo 'failed with returncode:' $1 'please check the log file: ' ${currentpath}'/'${extractDate}_logfile.txt
        echo 'curlcmd failed with returncode:' $1 >>  ${extractDate}_logile.txt
              exit 255
        else
      echo 'curlcmd succeeded' >> ${extractDate}_logile.txt
      fi
      return 0      
}

    RC=$?
    #echo 'curlretval:'$RC
    checkRetVal $RC
    curl -f -H "Content-Type: application/csv" bla -o ${fileDestintionPath}${extractDate}_${resultFileName}  >> ${extractDate}_logfile.txt 2>&1
    RC=$?
    #echo 'curlretval:'$RC
    checkRetVal $RC
    curl -f -H "Content-Type: application/csv" $curURL -o ${fileDestintionPath}${extractDate}_${resultFileName}  >> ${extractDate}_logfile.txt 2>&1
    RC=$?
    checkRetVal $RC

the result in the console:
failed with returncode: 6 please check the log file:  /home/wbk/pgm/2016-09-27-21-47-18_logile.txt

and the log file:

ore 2016-09-27-21-51-50_logfile.txt
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    34    0    34    0     0    197      0 --:--:-- --:--:-- --:--:--   198
curlcmd succeeded
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (6) Could not resolve host: bla
curlcmd failed with returncode: 6
0
 

Author Closing Comment

by:mruff
ID: 41818606
Perfect solution, thx
0

Featured Post

Ransomware: The New Cyber Threat & How to Stop It

This infographic explains ransomware, type of malware that blocks access to your files or your systems and holds them hostage until a ransom is paid. It also examines the different types of ransomware and explains what you can do to thwart this sinister online threat.  

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
ftp to port 21 4 54
CENTOS DHCP Server / PXE/TFTP 14 156
PHP error function not working on AWS 10 74
Upgraded from Debian 7 to 8.7 and got black screen 20 69
Using 'screen' for session sharing, The Simple Edition Step 1: user starts session with command: screen Step 2: other user (logged in with same user account) connects with command: screen -x Done. Both users are connected to the same CLI sessio…
Java performance on Solaris - Managing CPUs There are various resource controls in operating system which directly/indirectly influence the performance of application. one of the most important resource controls is "CPU".   In a multithreaded‚Ķ
Connecting to an Amazon Linux EC2 Instance from Windows Using PuTTY.
This demo shows you how to set up the containerized NetScaler CPX with NetScaler Management and Analytics System in a non-routable Mesos/Marathon environment for use with Micro-Services applications.

830 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question