asked on

Copy everything from an S3 Folder to a local drive on an EC2 Instance using Powershell

I have a folder in an S3 Bucket ( ppq-meta/sql ). Using powershell I want to copy everything in that S3 Bucket "ppq-meta" folder "sql" to a local drive z:\test\. This will be run on an EC2 instance, it already has AWS Powershell installed. Names for S3 bucket and for folder are in lowercase, I remember it being case sensitive.

I know I can do this with aws-cli but don't want to install aws-cli on instance:

aws s3 cp s3://ppq-meta/sql z:\test --recursive

Open in new window

What I have tried in Powershell is below, but it keeps giving me errors.

Copy-S3Object -AccessKey Enter-Access-Key-Here -SecretKey Enter-Secret-Key-Here -BucketName ppq-meta -KeyPrefix sql -key "*.*" -LocalFolder z:\test

Open in new window

Error:

Copy-S3Object : A parameter cannot be found that matches parameter name 'KeyPrefix'.
At line:1 char:120
+ ... tName ppq-meta -KeyPrefix sql -key "*.*" -LocalFolder z:\test
+                    ~~~~~~~~~~
    + CategoryInfo          : InvalidArgument: (:) [Copy-S3Object], ParameterBindingException
    + FullyQualifiedErrorId : NamedParameterNotFound,Amazon.PowerShell.Cmdlets.S3.CopyS3ObjectCmdlet

Open in new window

What am I doing wrong?

David Johnson, CD

try enclosing the <string> in -KeyPrefix in quotes i.e.
-KeyPrefix "sql"

Copy-S3Object
-BucketName <String>
-LocalFolder <String>
-KeyPrefix <String>
-ModifiedSinceDate <DateTime>
-UnmodifiedSinceDate <DateTime>
-Force <SwitchParameter>
-UseAccelerateEndpoint <SwitchParameter>
-UseDualstackEndpoint <SwitchParameter>

https://docs.aws.amazon.com/powershell/latest/reference/items/Copy-S3Object.html

Ed Walsh

ASKER

@David Johnson, that didn't help same error. I also tried with single quotes ' and no quotes same results.

Copy-S3Object : A parameter cannot be found that matches parameter name 'KeyPrefix'.
At line:1 char:120
+ ... tName ppq-meta -KeyPrefix "sql" -key "*.*" -LocalFolder z:\test
+                    ~~~~~~~~~~
    + CategoryInfo          : InvalidArgument: (:) [Copy-S3Object], ParameterBindingException
    + FullyQualifiedErrorId : NamedParameterNotFound,Amazon.PowerShell.Cmdlets.S3.CopyS3ObjectCmdlet

Open in new window

David Johnson, CD

Are you using AWS Tools for Windows PowerShell or AWS Tools for PowerShell Core?

command works for AWS tools for Windows Powershell AFAIK

Ed Walsh

ASKER

I have the AWS Powershell Module that comes pre-installed with EC2 instances.

ModuleType Version    Name                                ExportedCommands                                                                                                                                  
---------- -------    ----                                ----------------                                                                                                                                  
Binary     3.1.20.0   AWSPowerShell                       {Add-ASAAttachmentsToSet, Add-ASACommunicationToCase, Add-ASInstances, Add-ASLoadBalancer...}

Open in new window

ASKER CERTIFIED SOLUTION

Ed Walsh

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

Ed Walsh

ASKER

Answers provided did not work, I continued to research issue and found the correct solution. Posting solution for any user who comes across the same issue that I had.