How to use IAM role to access S3 bucket

I want to upload a file to S3 bucket, but my company want to use IAM role as opposed to access keys. This is AWS documentation on how to upload to S3 asynchronously:

S3AsyncClient client = S3AsyncClient.create();
		CompletableFuture<PutObjectResponse> future = client.putObject(
				PutObjectRequest.builder().bucket(BUCKET)
						.key(fileName)
						.build(),
				AsyncRequestBody
						.fromFile(fromFile.toPath()));
		future.whenComplete((resp, err) -> {
			try {
				if (resp != null) {
					System.out.println("my response: " + resp);
				} else {
					// Handle error
					err.printStackTrace();
				}
			} finally {
				// Lets the application shut down. Only close the client when
				// you are completely done with it.
				client.close();
				
			}
		});

		future.join();
	}

Open in new window

I don't see anywhere to put in IAM role info. I tried to put it in ~/.aws/credentials in this form:

[useraccount]
aws_access_key_id=<key>
aws_secret_access_key=<secret>

[somerole]
role_arn=<the ARN of the role you want to assume>
source_profile=useraccount

but so far haven't gotten it to work. I read somewhere you need to use STSAssumeRoleSessionCredentialsProvider but didn't see any good examples. My main question is do I even need to do anything if I already assigned the IAM role to an ECS instance. Can someone help me? Thanks.

JavaAWS

Last Comment

Michael Lam

8/22/2022 - Mon

Phil Phillips

1/22/2020

If you've assigned an IAM role to the instance, you shouldn't need to configure anything additional for AWS credentials. If AWS credentials are missing, the client will default to attempting to use IAM role.

Some debugging steps you can try:

Remove AWS credential files
Use AWS CLI to see if you're able to do simple s3 commands (ie: aws s3 ls)

Michael Lam

1/23/2020

ASKER

Yes you are right, I found that after I put in the config and credentials files, it works, except for the first time. For the first time my code tries to upload to S3, it will get:

Failed to upload to S3: software.amazon.awssdk.core.exception.SdkClientException: Unable to load credentials from any of the providers in the chain AwsCredentialsProviderChain(credentialsProviders=[SystemPropertyCredentialsProvider(), EnvironmentVariableCredentialsProvider(), ProfileCredentialsProvider(), WebIdentityTokenCredentialsProvider(), ContainerCredentialsProvider(), InstanceProfileCredentialsProvider()]) : [SystemPropertyCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., EnvironmentVariableCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., ProfileCredentialsProvider(): java.nio.channels.ClosedByInterruptException, WebIdentityTokenCredentialsProvider(): Either the environment variable AWS_WEB_IDENTITY_TOKEN_FILE or the javaproperty aws.webIdentityTokenFile must be set., ContainerCredentialsProvider(): Interrupted waiting to refresh the value., InstanceProfileCredentialsProvider(): Interrupted waiting to refresh the value.]

but subsequent attempts would work, until I log in the next time, then the same issue occurs.

Michael Lam

1/23/2020

ASKER

This is how I generate the credential files in ECS

mkdir -p $ORACLE_HOME/.aws
echo "[default]" > $ORACLE_HOME/.aws/config
echo "region=${AWS_REGION}" >> $ORACLE_HOME/.aws/config
echo "output=json" >> $ORACLE_HOME/.aws/config

echo "[${AWS_PROFILE}]" > $ORACLE_HOME/.aws/credentials
echo "role_arn=${AWS_ROLE_ARN}" >> $ORACLE_HOME/.aws/credentials
echo "source_profile=[default]" >> $ORACLE_HOME/.aws/credentials

Experts Exchange is like having an extremely knowledgeable team sitting and waiting for your call. Couldn't do my job half as well as I do without it!

James Murphy

Phil Phillips

1/23/2020

Hmm, this part is interesting:

InstanceProfileCredentialsProvider(): Interrupted waiting to refresh the value

Open in new window

I wonder if it helps at all to set up the credentials provider to get the IAM credentials asynchronously. Maybe try replacing the first line of your code with something like:

AwsCredentialsProvider awsCredentialsProvider = InstanceProfileCredentialsProvider.builder().asyncCredentialUpdateEnabled(Boolean.TRUE).build();
S3AsyncClient client = S3AsyncClient.builder().credentialsProvider(awsCredentialsProvider).build();

Open in new window

Other than that, I'm not too sure - haven't worked too much with Java + AWS/S3 recently.

Michael Lam

1/24/2020

ASKER

I implemented you code, and got this:

Jan 24, 2020 02:49 (UTC-8): Jan 24 10:49:42:256:t@1997049922:ExportPPTSingleBudgetBean:Failed to upload to S3: java.lang.IllegalStateException: Interrupted waiting to refresh the value.
Jan 24, 2020 02:49 (UTC-8): Jan 24 10:49:42:256:t@1997049922:ExportPPTSingleBudgetBean:com.fasttrack.utilities.FTException: Failed to upload to S3: java.lang.IllegalStateException: Interrupted waiting to refresh the value.
Jan 24, 2020 02:49 (UTC-8):       at com.fasttrack.gm.servlet.ExportBudgetBase.lambda$uploadS3Async$1(ExportBudgetBase.java:1570)
Jan 24, 2020 02:49 (UTC-8):       at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760)
Jan 24, 2020 02:49 (UTC-8):       at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:778)
Jan 24, 2020 02:49 (UTC-8):       at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2140)
Jan 24, 2020 02:49 (UTC-8):       at com.fasttrack.gm.servlet.ExportBudgetBase.uploadS3Async(ExportBudgetBase.java:1562)
Jan 24, 2020 02:49 (UTC-8):       at com.fasttrack.gm.servlet.ExportBudgetBase.saveWorkBook(ExportBudgetBase.java:1520)
Jan 24, 2020 02:49 (UTC-8):       at com.fasttrack.gm.servlet.BudgetBean.processExportRequest(BudgetBean.java:7748)
Jan 24, 2020 02:49 (UTC-8):       at com.fasttrack.gm.servlet.BudgetBean.access$300(BudgetBean.java:92)
Jan 24, 2020 02:49 (UTC-8):       at com.fasttrack.gm.servlet.BudgetBean$AsyncExporter$1.call(BudgetBean.java:122)
Jan 24, 2020 02:49 (UTC-8):       at com.fasttrack.gm.servlet.BudgetBean$AsyncExporter$1.call(BudgetBean.java:118)
Jan 24, 2020 02:49 (UTC-8):       at java.util.concurrent.FutureTask.run(FutureTask.java:266)
Jan 24, 2020 02:49 (UTC-8):       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
Jan 24, 2020 02:49 (UTC-8):       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
Jan 24, 2020 02:49 (UTC-8):       at java.lang.Thread.run(Thread.java:748)
Jan 24, 2020 02:49 (UTC-8): Jan 24 10:49:42:258:t@1997049922:ExportPPTSingleBudgetBean:Failed to upload to S3: java.lang.IllegalStateException: Interrupted waiting to refresh the value.
Jan 24, 2020 02:49 (UTC-8): Jan 24 10:49:42:258:t@1997049922:ExportPPTSingleBudgetBean:com.fasttrack.utilities.FTException: Failed to upload to S3: java.lang.IllegalStateException: Interrupted waiting to refresh the value.
Jan 24, 2020 02:49 (UTC-8):       at com.fasttrack.gm.servlet.ExportBudgetBase.uploadS3Async(ExportBudgetBase.java:1586)
Jan 24, 2020 02:49 (UTC-8):       at com.fasttrack.gm.servlet.ExportBudgetBase.saveWorkBook(ExportBudgetBase.java:1520)
Jan 24, 2020 02:49 (UTC-8):       at com.fasttrack.gm.servlet.BudgetBean.processExportRequest(BudgetBean.java:7748)
Jan 24, 2020 02:49 (UTC-8):       at com.fasttrack.gm.servlet.BudgetBean.access$300(BudgetBean.java:92)
Jan 24, 2020 02:49 (UTC-8):       at com.fasttrack.gm.servlet.BudgetBean$AsyncExporter$1.call(BudgetBean.java:122)
Jan 24, 2020 02:49 (UTC-8):       at com.fasttrack.gm.servlet.BudgetBean$AsyncExporter$1.call(BudgetBean.java:118)
Jan 24, 2020 02:49 (UTC-8):       at java.util.concurrent.FutureTask.run(FutureTask.java:266)
Jan 24, 2020 02:49 (UTC-8):       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
Jan 24, 2020 02:49 (UTC-8):       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
Jan 24, 2020 02:49 (UTC-8):       at java.lang.Thread.run(Thread.java:748)

Michael Lam

1/24/2020

ASKER

Jan 24, 2020 02:52 (UTC-8): Jan 24 10:52:02:585:t@1738448475:ExportPPTSingleBudgetBean:Failed to upload to S3: software.amazon.awssdk.core.exception.SdkClientException: Unable to load credentials from service endpoint.

⚡ FREE TRIAL OFFER

Try out a week of full access for free.

Find out why thousands trust the EE community with their toughest problems.

Michael Lam

1/26/2020

ASKER

I think the issue is not about credentials but why the S3Client is getting interrupted. I will post another question.

ASKER CERTIFIED SOLUTION

Michael Lam

1/26/2020

THIS SOLUTION ONLY AVAILABLE TO MEMBERS.

View this solution by signing up for a free trial.