Copy everything from an S3 Folder to a local drive on an EC2 Instance using Powershell

I have a folder in an S3 Bucket ( ppq-meta/sql ). Using powershell I want to copy everything in that S3 Bucket "ppq-meta" folder "sql" to a local drive z:\test\. This will be run on an EC2 instance, it already has AWS Powershell installed.  Names for S3 bucket and for folder are in lowercase, I remember it being case sensitive.

I know I can do this with aws-cli but don't want to install aws-cli on instance:
aws s3 cp s3://ppq-meta/sql z:\test --recursive

Open in new window


What I have tried in Powershell is below, but it keeps giving me errors.
Copy-S3Object -AccessKey Enter-Access-Key-Here -SecretKey Enter-Secret-Key-Here -BucketName ppq-meta -KeyPrefix sql -key "*.*" -LocalFolder z:\test

Open in new window


Error:

Copy-S3Object : A parameter cannot be found that matches parameter name 'KeyPrefix'.
At line:1 char:120
+ ... tName ppq-meta -KeyPrefix sql -key "*.*" -LocalFolder z:\test
+                    ~~~~~~~~~~
    + CategoryInfo          : InvalidArgument: (:) [Copy-S3Object], ParameterBindingException
    + FullyQualifiedErrorId : NamedParameterNotFound,Amazon.PowerShell.Cmdlets.S3.CopyS3ObjectCmdlet

Open in new window


What am I doing wrong?
Ed WalshSenior Director of Applications and Cloud ServicesAsked:
Who is Participating?
 
Ed WalshConnect With a Mentor Senior Director of Applications and Cloud ServicesAuthor Commented:
Ok I figured it out

What I originally tried and didnt work:
Copy-S3Object -AccessKey Enter-Access-Key-Here -SecretKey Enter-Secret-Key-Here -BucketName ppq-meta -KeyPrefix sql -key "*.*" -LocalFolder z:\test

Open in new window


What does work:
Copy-S3Object -AccessKey Enter-Access-Key-Here -SecretKey Enter-Secret-Key-Here -Region us-east-1 -BucketName ppq-meta -KeyPrefix sql -Folder z:\test

Open in new window


This will copy all files and folders in the AWS S3 bucket to local folder z:\test .
No need to enter in wild card for this to work or even a recursive flag. It makes all subfolders and files within on the S3 to the Local folder.
0
 
David Johnson, CD, MVPOwnerCommented:
try enclosing the <string> in -KeyPrefix in quotes i.e.
-KeyPrefix "sql"

Copy-S3Object
-BucketName <String>
-LocalFolder <String>
-KeyPrefix <String>
-ModifiedSinceDate <DateTime>
-UnmodifiedSinceDate <DateTime>
-Force <SwitchParameter>
-UseAccelerateEndpoint <SwitchParameter>
-UseDualstackEndpoint <SwitchParameter>
https://docs.aws.amazon.com/powershell/latest/reference/items/Copy-S3Object.html
0
 
Ed WalshSenior Director of Applications and Cloud ServicesAuthor Commented:
@David Johnson, that didn't help same error. I also tried with single quotes ' and no quotes same results.

Copy-S3Object : A parameter cannot be found that matches parameter name 'KeyPrefix'.
At line:1 char:120
+ ... tName ppq-meta -KeyPrefix "sql" -key "*.*" -LocalFolder z:\test
+                    ~~~~~~~~~~
    + CategoryInfo          : InvalidArgument: (:) [Copy-S3Object], ParameterBindingException
    + FullyQualifiedErrorId : NamedParameterNotFound,Amazon.PowerShell.Cmdlets.S3.CopyS3ObjectCmdlet

Open in new window

0
Simplify Active Directory Administration

Administration of Active Directory does not have to be hard.  Too often what should be a simple task is made more difficult than it needs to be.The solution?  Hyena from SystemTools Software.  With ease-of-use as well as powerful importing and bulk updating capabilities.

 
David Johnson, CD, MVPOwnerCommented:
Are you using AWS Tools for Windows PowerShell or AWS Tools for PowerShell Core?

command works for AWS tools for Windows Powershell AFAIK
0
 
Ed WalshSenior Director of Applications and Cloud ServicesAuthor Commented:
I have the AWS Powershell Module that comes pre-installed with EC2 instances.

ModuleType Version    Name                                ExportedCommands                                                                                                                                  
---------- -------    ----                                ----------------                                                                                                                                  
Binary     3.1.20.0   AWSPowerShell                       {Add-ASAAttachmentsToSet, Add-ASACommunicationToCase, Add-ASInstances, Add-ASLoadBalancer...}                                                     

Open in new window

0
 
Ed WalshSenior Director of Applications and Cloud ServicesAuthor Commented:
Answers provided did not work, I continued to research issue and found the correct solution. Posting solution for any user who comes across the same issue that I had.
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.