Solved

s3 PUT notification

Posted on 2012-04-05
4
665 Views
Last Modified: 2014-11-12
Hi experts,

We have been using Amazon's s3 service for imports and exports.

Currently, we simply use the s3.exe for putting / getting files to and from s3.

s3 auth <key> <secret>
s3 put <bucket>/<folder>/ C:\Batch\<folder>

We have been having some issues with files not making it s3 which causes issues downstream.

The question we have is:
How can we receive notification that the file was successfully received in s3?

Either via the cmd line script or via an amazon service. We have been looking at the SNS service from Amazon, but are not sure if this can be used for this purpose.

Thanks for your help!
0
Comment
Question by:JFranks22
  • 2
4 Comments
 
LVL 78

Assisted Solution

by:David Johnson, CD, MVP
David Johnson, CD, MVP earned 250 total points
Comment Utility
Without  modifying the code what you could do is

s3 put filename
cd temp
s3 get filename

Personally I use the Cloudberry S3 Explorer with the sync option
0
 

Author Comment

by:JFranks22
Comment Utility
Thanks ve3ofa, we have considered that solution.

We are keen to know if there is a way to receive notification that the file was successfully received in s3 either by:
- Amazon notification
- A "file exists" s3 command line function

Thanks for any further feedback experts!
0
 

Author Comment

by:JFranks22
Comment Utility
Any further thoughts?
0
 
LVL 40

Accepted Solution

by:
RQuadling earned 250 total points
Comment Utility
Using the Amazon's AWS SDK for PHP, the (reduced) code I use is ...

	if (!$o_S3->if_bucket_exists($a_S3BucketParts['host']))
		{
		$o_CLIP->displayError("<destination> {$a_S3BucketParts['host']} does not exist.");
		}

	$a_S3Options = array
		(
		'acl'        => AmazonS3::ACL_PUBLIC,
		'fileUpload' => $o_CommandLine->args['source'],
		'storage'    => AmazonS3::STORAGE_STANDARD
		);

	$o_Response = $o_S3->create_object($a_S3BucketParts['host'], trim($a_S3BucketParts['path'], '/'), $a_S3Options);
	if (!$o_Response->isOK())
		{
		$o_CLIP->displayError("Unable to upload {$o_CommandLine->args['source']} to {$o_CommandLine->args['destination']} with the error of {$o_Response}");
		}

Open in new window


So, when uploading a file using the SDK, you can also examine the response to determine if the upload was successful.

Now, based upon the last 4 or 5 months of use, with maybe half a million uploads, I've probably had 3 or 4 failures. Retrying the put worked. So I've not needed to spend any time handling the potential failures beyond recording things.
0

Featured Post

What Should I Do With This Threat Intelligence?

Are you wondering if you actually need threat intelligence? The answer is yes. We explain the basics for creating useful threat intelligence.

Join & Write a Comment

Monitoring systems evolution, cloud technology benefits and cloud cost calculators business utility.
Big data transfers via information superhighways require special attention and protection. Learn more about the IT-regulations of the country where your server is located. Analyze cloud providers and their encryption systems for safe data transit. S…
Connecting to an Amazon Linux EC2 Instance from Windows Using PuTTY.
This Micro Tutorial will explain how to export DynamoDB tables in Amazon Web Services.

744 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

18 Experts available now in Live!

Get 1:1 Help Now