Amazon S3 authentication trouble (auth Version 4)

Hi,

I'm struggling to get an AWS S3 "PUT" request to authenticate (it's the first thing I've tried with S3) using a Perl script.

The authentication works (but the request doesn't) if I do this (access key and bucket name substituted):

Canonical Request:
PUT
/ourbucketname/winklydoo.txt

content-length:11
date:Thu, 09 Nov 2017 17:18:15 GMT
host:s3.amazonaws.com
x-amz-content-sha256:b94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9
x-amz-date:20171109T171815Z

content-length;date;host;x-amz-content-sha256;x-amz-date
b94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9

Open in new window



String to Sign:
AWS4-HMAC-SHA256
20171109T171815Z
20171109/eu-central-1/s3/aws4_request
f2163f181086c0a1a30b50efa3c374a33822e82b4048464bca22ecbd0d8938cc

Open in new window



HTTP headers:
Date: Thu, 09 Nov 2017 17:18:15 GMT
Authorization: AWS4-HMAC-SHA256 Credential=OURAWSACCESSKEY/20171109/eu-central-1/s3/aws4_request,SignedHeaders=content-length;date;host;x-amz-content-sha256;x-amz-date,Signature=d6197e451cb634c304dd02b5e101cc7517644e2361ab6cfa7b2f9e97fb92073f
Host: s3.amazonaws.com
Content-Length: 11
x-amz-content-sha256: b94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9
x-amz-date: 20171109T171815Z

Open in new window



DATA:
hello world

Open in new window



This suggests to me that the authentication worked for this request, but the (not surprising) error that comes back is:
<Code>PermanentRedirect</Code>
<Message>The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.</Message>
<Bucket>ourbucketname</Bucket>
<Endpoint>ourbucketname.s3.amazonaws.com</Endpoint>

Open in new window




However, if I edit my code to use the endpoint it gave as the HTTP host name, it fails with an authentication error. This is what I'm now sending:

Canonical Request:
PUT
/ourbucketname/winklydoo.txt

content-length:11
date:Thu, 09 Nov 2017 17:22:09 GMT
host:ourbucketname.s3.amazonaws.com
x-amz-content-sha256:b94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9
x-amz-date:20171109T172209Z

content-length;date;host;x-amz-content-sha256;x-amz-date
b94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9

Open in new window



String to Sign:
AWS4-HMAC-SHA256
20171109T172209Z
20171109/eu-central-1/s3/aws4_request
6d4922dc0ccd0a3ece812a4d7dd81a38bac866e16ae0cf758bb3a14b15bf1f0e

Open in new window



HTTP headers:
Date: Thu, 09 Nov 2017 17:22:09 GMT
Authorization: AWS4-HMAC-SHA256 Credential=OURAWSACCESSKEY/20171109/eu-central-1/s3/aws4_request,SignedHeaders=content-length;date;host;x-amz-content-sha256;x-amz-date,Signature=65b7f9a8cf9d68964d754282c4513ca71528b804cafb6e6e9e32be86f0ace7df
Host: ourbucketname.s3.amazonaws.com
Content-Length: 11
x-amz-content-sha256: b94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9
x-amz-date: 20171109T172209Z

Open in new window



(same data sent).


This time, the error that comes back contains this:
<Code>SignatureDoesNotMatch</Code>
<Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message>
<AWSAccessKeyId>OURAWSACCESSKEY</AWSAccessKeyId>
<StringToSign>AWS4-HMAC-SHA256
20171109T172209Z
20171109/eu-central-1/s3/aws4_request
6d4922dc0ccd0a3ece812a4d7dd81a38bac866e16ae0cf758bb3a14b15bf1f0e</StringToSign>
<SignatureProvided>65b7f9a8cf9d68964d754282c4513ca71528b804cafb6e6e9e32be86f0ace7df</SignatureProvided>

Open in new window



Given that the returned StringToSign exactly matches what I am sending, I presume this bit must be right.

This is how I'm generating the signature (pasted directly from Amazon's docs and then edited to turn it into valid Perl code):

my $DateKey              = HMAC_SHA256("AWS4".$self->{secret}, $dstamp);
my $DateRegionKey        = HMAC_SHA256($DateKey, $region);
my $DateRegionServiceKey = HMAC_SHA256($DateRegionKey, "s3");
my $SigningKey           = HMAC_SHA256($DateRegionServiceKey, "aws4_request");

return "AWS4-HMAC-SHA256 ".
       "Credential=".$self->{access_key}."/$dstamp/$region/s3/aws4_request,".
       "SignedHeaders=$signed_headers,".
       "Signature=".HMAC_SHA256($SigningKey, $string_to_sign);

Open in new window



I've been trying to get this working for 2 days solid and for the life of me can't see what I'm doing wrong. Can anyone help?
Kingsley TartAsked:
Who is Participating?
 
Kingsley TartConnect With a Mentor Author Commented:
Hi Phil. It turns out that I was calculating the signature incorrectly (accidentally using hmac_sha256_hex instead of just hmac_sha256 in the interim steps).

BTW, when I sent you my original message, I was expecting to pay $1.25 per minute; I didn't expect free help from you, but thanks for answering.

Cheers,
Kingsley.
1
 
Phil PhillipsDirector of DevOps & Quality AssuranceCommented:
I've usually seen the first error when trying to use an endpoint that is in a different region.  Looks like you're using eu-central-1, so maybe try with this endpoint instead: s3.eu-central-1.amazonaws.com

(If that doesn't help, then we can dive deeper into the code just to make sure you're signing things correctly).
0
 
Kingsley TartAuthor Commented:
I realised what I was doing wrong.
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.