Copy everything from an S3 Folder to a local drive on an EC2 Instance using Powershell

I have a folder in an S3 Bucket ( ppq-meta/sql ). Using powershell I want to copy everything in that S3 Bucket "ppq-meta" folder "sql" to a local drive z:\test\. This will be run on an EC2 instance, it already has AWS Powershell installed.  Names for S3 bucket and for folder are in lowercase, I remember it being case sensitive.

I know I can do this with aws-cli but don't want to install aws-cli on instance:
aws s3 cp s3://ppq-meta/sql z:\test --recursive

Open in new window


What I have tried in Powershell is below, but it keeps giving me errors.
Copy-S3Object -AccessKey Enter-Access-Key-Here -SecretKey Enter-Secret-Key-Here -BucketName ppq-meta -KeyPrefix sql -key "*.*" -LocalFolder z:\test

Open in new window


Error:

Copy-S3Object : A parameter cannot be found that matches parameter name 'KeyPrefix'.
At line:1 char:120
+ ... tName ppq-meta -KeyPrefix sql -key "*.*" -LocalFolder z:\test
+                    ~~~~~~~~~~
    + CategoryInfo          : InvalidArgument: (:) [Copy-S3Object], ParameterBindingException
    + FullyQualifiedErrorId : NamedParameterNotFound,Amazon.PowerShell.Cmdlets.S3.CopyS3ObjectCmdlet

Open in new window


What am I doing wrong?
Ed WalshSenior Director of Applications and Cloud ServicesAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

David Johnson, CD, MVPOwnerCommented:
try enclosing the <string> in -KeyPrefix in quotes i.e.
-KeyPrefix "sql"

Copy-S3Object
-BucketName <String>
-LocalFolder <String>
-KeyPrefix <String>
-ModifiedSinceDate <DateTime>
-UnmodifiedSinceDate <DateTime>
-Force <SwitchParameter>
-UseAccelerateEndpoint <SwitchParameter>
-UseDualstackEndpoint <SwitchParameter>
https://docs.aws.amazon.com/powershell/latest/reference/items/Copy-S3Object.html
0
Ed WalshSenior Director of Applications and Cloud ServicesAuthor Commented:
@David Johnson, that didn't help same error. I also tried with single quotes ' and no quotes same results.

Copy-S3Object : A parameter cannot be found that matches parameter name 'KeyPrefix'.
At line:1 char:120
+ ... tName ppq-meta -KeyPrefix "sql" -key "*.*" -LocalFolder z:\test
+                    ~~~~~~~~~~
    + CategoryInfo          : InvalidArgument: (:) [Copy-S3Object], ParameterBindingException
    + FullyQualifiedErrorId : NamedParameterNotFound,Amazon.PowerShell.Cmdlets.S3.CopyS3ObjectCmdlet

Open in new window

0
David Johnson, CD, MVPOwnerCommented:
Are you using AWS Tools for Windows PowerShell or AWS Tools for PowerShell Core?

command works for AWS tools for Windows Powershell AFAIK
0
Making Bulk Changes to Active Directory

Watch this video to see how easy it is to make mass changes to Active Directory from an external text file without using complicated scripts.

Ed WalshSenior Director of Applications and Cloud ServicesAuthor Commented:
I have the AWS Powershell Module that comes pre-installed with EC2 instances.

ModuleType Version    Name                                ExportedCommands                                                                                                                                  
---------- -------    ----                                ----------------                                                                                                                                  
Binary     3.1.20.0   AWSPowerShell                       {Add-ASAAttachmentsToSet, Add-ASACommunicationToCase, Add-ASInstances, Add-ASLoadBalancer...}                                                     

Open in new window

0
Ed WalshSenior Director of Applications and Cloud ServicesAuthor Commented:
Ok I figured it out

What I originally tried and didnt work:
Copy-S3Object -AccessKey Enter-Access-Key-Here -SecretKey Enter-Secret-Key-Here -BucketName ppq-meta -KeyPrefix sql -key "*.*" -LocalFolder z:\test

Open in new window


What does work:
Copy-S3Object -AccessKey Enter-Access-Key-Here -SecretKey Enter-Secret-Key-Here -Region us-east-1 -BucketName ppq-meta -KeyPrefix sql -Folder z:\test

Open in new window


This will copy all files and folders in the AWS S3 bucket to local folder z:\test .
No need to enter in wild card for this to work or even a recursive flag. It makes all subfolders and files within on the S3 to the Local folder.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Ed WalshSenior Director of Applications and Cloud ServicesAuthor Commented:
Answers provided did not work, I continued to research issue and found the correct solution. Posting solution for any user who comes across the same issue that I had.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Powershell

From novice to tech pro — start learning today.