Amazon S3 authentication trouble (auth Version 4)

Hi,

I'm struggling to get an AWS S3 "PUT" request to authenticate (it's the first thing I've tried with S3) using a Perl script.

The authentication works (but the request doesn't) if I do this (access key and bucket name substituted):

Canonical Request:
PUT
/ourbucketname/winklydoo.txt

content-length:11
date:Thu, 09 Nov 2017 17:18:15 GMT
host:s3.amazonaws.com
x-amz-content-sha256:b94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9
x-amz-date:20171109T171815Z

content-length;date;host;x-amz-content-sha256;x-amz-date
b94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9

Open in new window



String to Sign:
AWS4-HMAC-SHA256
20171109T171815Z
20171109/eu-central-1/s3/aws4_request
f2163f181086c0a1a30b50efa3c374a33822e82b4048464bca22ecbd0d8938cc

Open in new window



HTTP headers:
Date: Thu, 09 Nov 2017 17:18:15 GMT
Authorization: AWS4-HMAC-SHA256 Credential=OURAWSACCESSKEY/20171109/eu-central-1/s3/aws4_request,SignedHeaders=content-length;date;host;x-amz-content-sha256;x-amz-date,Signature=d6197e451cb634c304dd02b5e101cc7517644e2361ab6cfa7b2f9e97fb92073f
Host: s3.amazonaws.com
Content-Length: 11
x-amz-content-sha256: b94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9
x-amz-date: 20171109T171815Z

Open in new window



DATA:
hello world

Open in new window



This suggests to me that the authentication worked for this request, but the (not surprising) error that comes back is:
<Code>PermanentRedirect</Code>
<Message>The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.</Message>
<Bucket>ourbucketname</Bucket>
<Endpoint>ourbucketname.s3.amazonaws.com</Endpoint>

Open in new window




However, if I edit my code to use the endpoint it gave as the HTTP host name, it fails with an authentication error. This is what I'm now sending:

Canonical Request:
PUT
/ourbucketname/winklydoo.txt

content-length:11
date:Thu, 09 Nov 2017 17:22:09 GMT
host:ourbucketname.s3.amazonaws.com
x-amz-content-sha256:b94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9
x-amz-date:20171109T172209Z

content-length;date;host;x-amz-content-sha256;x-amz-date
b94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9

Open in new window



String to Sign:
AWS4-HMAC-SHA256
20171109T172209Z
20171109/eu-central-1/s3/aws4_request
6d4922dc0ccd0a3ece812a4d7dd81a38bac866e16ae0cf758bb3a14b15bf1f0e

Open in new window



HTTP headers:
Date: Thu, 09 Nov 2017 17:22:09 GMT
Authorization: AWS4-HMAC-SHA256 Credential=OURAWSACCESSKEY/20171109/eu-central-1/s3/aws4_request,SignedHeaders=content-length;date;host;x-amz-content-sha256;x-amz-date,Signature=65b7f9a8cf9d68964d754282c4513ca71528b804cafb6e6e9e32be86f0ace7df
Host: ourbucketname.s3.amazonaws.com
Content-Length: 11
x-amz-content-sha256: b94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9
x-amz-date: 20171109T172209Z

Open in new window



(same data sent).


This time, the error that comes back contains this:
<Code>SignatureDoesNotMatch</Code>
<Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message>
<AWSAccessKeyId>OURAWSACCESSKEY</AWSAccessKeyId>
<StringToSign>AWS4-HMAC-SHA256
20171109T172209Z
20171109/eu-central-1/s3/aws4_request
6d4922dc0ccd0a3ece812a4d7dd81a38bac866e16ae0cf758bb3a14b15bf1f0e</StringToSign>
<SignatureProvided>65b7f9a8cf9d68964d754282c4513ca71528b804cafb6e6e9e32be86f0ace7df</SignatureProvided>

Open in new window



Given that the returned StringToSign exactly matches what I am sending, I presume this bit must be right.

This is how I'm generating the signature (pasted directly from Amazon's docs and then edited to turn it into valid Perl code):

my $DateKey              = HMAC_SHA256("AWS4".$self->{secret}, $dstamp);
my $DateRegionKey        = HMAC_SHA256($DateKey, $region);
my $DateRegionServiceKey = HMAC_SHA256($DateRegionKey, "s3");
my $SigningKey           = HMAC_SHA256($DateRegionServiceKey, "aws4_request");

return "AWS4-HMAC-SHA256 ".
       "Credential=".$self->{access_key}."/$dstamp/$region/s3/aws4_request,".
       "SignedHeaders=$signed_headers,".
       "Signature=".HMAC_SHA256($SigningKey, $string_to_sign);

Open in new window



I've been trying to get this working for 2 days solid and for the life of me can't see what I'm doing wrong. Can anyone help?
Kingsley TartAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Phil PhillipsDevOps ArchitectCommented:
I've usually seen the first error when trying to use an endpoint that is in a different region.  Looks like you're using eu-central-1, so maybe try with this endpoint instead: s3.eu-central-1.amazonaws.com

(If that doesn't help, then we can dive deeper into the code just to make sure you're signing things correctly).
0
Kingsley TartAuthor Commented:
Hi Phil. It turns out that I was calculating the signature incorrectly (accidentally using hmac_sha256_hex instead of just hmac_sha256 in the interim steps).

BTW, when I sent you my original message, I was expecting to pay $1.25 per minute; I didn't expect free help from you, but thanks for answering.

Cheers,
Kingsley.
1

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Kingsley TartAuthor Commented:
I realised what I was doing wrong.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
AWS

From novice to tech pro — start learning today.