asked on 8/31/2018

How to read s3 files from DOcker container

how we can read s3 bucket files form docker container
what are the package needed and what is the command

how we can put data to maria DB form docker container
what are the package needed and what is the command

Java Docker AWS Linux

Last Comment

CEHJ

9/5/2018

ASKER CERTIFIED SOLUTION

CEHJ

8/31/2018

THIS SOLUTION IS ONLY AVAILABLE TO MEMBERS.

View this solution by signing up for a free trial.

Members can start a 7-Day free trial and enjoy unlimited access to the platform.

See Pricing Options

Start Free Trial

Prabhin MP

9/2/2018

step1: make sure that aws cli installed in docker container.
if not installed include aws cli in docker file. and Create Environment variables for Access and secret key or move manually from the host machine to Docker container.

aws s3 cp s3://my-bucket/tt.txt . to read the file.
use pipe to pass the output to mariadb or mysql

Hope these info is enough

CEHJ

9/2/2018

aws s3 cp s3://my-bucket/tt.txt . to read the file.
use pipe to pass the output to mariadb or mysql

Again please don't repeat information already stated

Prabhin MP

9/2/2018

I didn't find an alternative for AWS cp command. I also stated the installation of AWS cli . Then you also make sure that while providing answer all those info are added

CEHJ

9/2/2018

I also stated the installation of AWS cli .

Not necessary really - it's obvious that those are command line commands

Sathish David Kumar N

9/2/2018

ASKER

Thank you both .

But my docker script will be automatically execute every day
is this possible ?

CEHJ

9/2/2018

Yes. You could use a cron job

Sathish David Kumar N

9/2/2018

ASKER

any ref link ?

CEHJ

9/2/2018

Hang on - from your new question, i see you're using Lambda. I suggest you confine this issue to that question

Sathish David Kumar N

9/2/2018

ASKER

I have different idea. But my target is I need to read file content from s3 and put it in Maria DB

I need to know which one is best
Lambda or directly run cron Job .

For All this i have only Idea .. I dont know how to do .

If any link related to this is very helpful

CEHJ

9/2/2018

I need to know which one is best
Lambda or directly run cron Job .

That's really a quite different question and should be addressed within the new question you just posted. The current one has been answered

Sathish David Kumar N

9/3/2018

ASKER

aws s3 cp s3://my-bucket/tt.txt . to read the file. use pipe to pass the output to mariadb

any sample reference for this ?

CEHJ

9/3/2018

You don't need one ;) I've given you the actual code (naturally you'll have to change the names) that was just described by someone else

Sathish David Kumar N

9/3/2018

ASKER

yes its direct one . But my script will call private s3 bucket , So I need to pass key to access that

So How to do that ?

CEHJ

9/3/2018

Not sure i'm afraid - you might need to check the documentation on that issue

Sathish David Kumar N

9/3/2018

ASKER

Hi CEHJ... i needto read the file..

cp will copy the file not content!!
is there any cmd available or any docker image for reading the csv file??

CEHJ

9/3/2018

The aws s3 command set is extremely limited. The following will show you the contents if the rest of the sql supports it

aws s3 cp s3://my-bucket/test.csv  >(cat)

Open in new window

Sathish David Kumar N

9/4/2018

ASKER

aws s3 cp s3://my-bucket/tt.txt . to read the file. use pipe to pass the output to mariadb

Can you give me sample code for pipe to pass mariaDB

CEHJ

9/4/2018

My very first comment gave you sample code

David Favor

9/4/2018

Looks like this question is bouncing back + forth between multiple questions.

CEHJ looks to have answered your original question in his first comment.

Please close this question as answered by CEHJ + then open other questions... one question/post for your other questions.

Likely this will provide more useful info for you.

Sathish David Kumar N

9/5/2018

ASKER

Thanks

CEHJ

9/5/2018