Why does curl not return a value in bash script?

February 7, 2023

My task is very simple – nevertheless I have been sitting already for hours and have no idea why it doesn’t work.

In linux bash script I want to get the result of a webservice call with curl. I’m not interested in the content, only the status code:

#!/bin/bash
set -euo pipefail # put bash in "strict mode"

echo "Before"
response2=$(curl  -o /dev/null -s  -w '%{http_code}'  -u username:password-X POST https://xxxxx.yyyyy.at:8081/MyPath/service/rest/crypto/encrypt -H 'Content-Type: application/json' -d '{"deviceId": "ABCD","payload": "ABCD"}')
echo "after"

It works when there is a valid request

Before...
200

Also, when the path of the service is wrong, it gives http error code

Before...
500

But when the host is wrong (not existent hostname) I get

Before...

and the script terminates (although the call is from a looping menue).

Why is this the case?

The manual call of curl with same parameters gives

as output, so why this output is not displayed in my script?

A reproducable example is (server name not existing):

#!/bin/bash
set -euo pipefail


#- Check kms
f_check_kms(){

    echo "Before..."
    response2=$(curl  -o /dev/null -s  -w '%{http_code}'  -u user:xxxx -X POST https://xxxx.xxx.intra.graz.at:8081/ATM-KeyManagement-REST-Service/service/rest/crypto/encryptUCast -H 'Content-Type: application/json' -d '{"deviceId": "SAG0530000016261", "encryptionSuite": "DLMS_SUITE_0", "securityMode": "AUTHENT_AND_ENCRYPT", "roleId": "001","initialVector": "4D4D4D0000BC614E01234567","payload": "ABCD","usedGuek":"NO","usedGak":"NO"}')
    
    echo "$response2"

}



f_check_kms

>Solution :

You’re running your script with set -e to make the shell interpreter exit when any¹ unchecked² command exits with a nonzero status, and when you provide an invalid hostname, curl exits with a nonzero exit status.

Because you’re passing -s for silent mode, it doesn’t print any error messages about this (you asked it not to!). It does still print the http_code you asked for, but because the script exits, the echo "after" is never reached, and whatever other code you’re relying on to print the contents of the response2 variable is likewise never reached.

Suppressing this is as simple as adding a conditional to the end, like the || : sequence below:

response2=$(curl -o /dev/null -s  -w '%{http_code}' -u username:password \
  -X POST https://xxxxx.yyyyy.at:8081/MyPath/service/rest/crypto/encrypt \
  -H 'Content-Type: application/json' \
  -d '{"deviceId": "ABCD","payload": "ABCD"}' \
) || : "Ignoring exit status of $?"

You’ll be able to see that message when running your script in trace mode (set -x / bash -x yourscript), but it’ll be otherwise invisible, and because || is branching on curl’s exit status, this marks curl as "checked" so set -e won’t decide to exit based on its exit status.

_{¹ Not really true: set -e has a bunch of exceptions it doesn’t exit over, and those exceptions change between individual shell releases.}

_{² This is a very unintuitively-defined word: For example, when you check the exit status of a function, everything that function calls can become "checked", so set -e‘s behavior is extremely context-sensitive and hard to predict; what’s checked when a function is called one time might not be checked when it’s called again later.}