asked on

Monitoring AWS S3 public buckets

Dear colleagues,
I'm looking to monitor buckets in Amazon S3 that have open access permissions.

I know there are enterprise solutions like Alienware, UpGuard, Acunetix, Cloudcheckr, Lightrail, etc and open source solutions like CloudCoreo, Threatresponse.cloud and Activity Aware IDS. I know there's a built-in AWS Trusted Advisor Security Check, available for free with a Business support plan. But I think it should be possible to build an out-of-the-box solution.

I'm sending CloudTrail events to a CloudWatch Logs log stream in the log group and then Create a Metric Filter and an alarm. I need help writing a Filter Pattern that triggers if a user changes the permissions of the existing bucket, which actually is the use case.

Shalom Carmel

You can always create a Lambda function to trigger on the cloudtrail audit bucket, and send alerts via SNS.

Here is a skeleton lambda in python to process cloudtrail logs. The last lines check for S3 events, and you have to add some logic to deal with the specific s3 events you need.

import json
import gzip
import boto3
import os


def lambda_handler(event, context):
	
	sns=boto3.client('sns')
		
	# Lambda allows storing temp files only in the tmp subfolder
	if not os.path.exists('/tmp'):
		os.makedirs('/tmp')


	downloaded_file_path = '/tmp/.dshjhga.gz'  # or some other bullshit file name


	print 'Lambda function starting'

	# defines a s3 boto client
	s3 = boto3.client('s3')

	print("Received event")
	# attribute bucket and file name/path to variables
	bucket = event['Records'][0]['s3']['bucket']['name']
	key = event['Records'][0]['s3']['object']['key']
	# where to save the downloaded file

	# downloads file to above path
	s3.download_file(bucket, key, downloaded_file_path)

	# opens gz file for reading
	gzfile = gzip.open(downloaded_file_path, "r")

	# loads contents of the Records key into variable (our actual cloudtrail log entries!)
	response = json.loads(gzfile.readlines()[0])["Records"]

	# loops over the events in the json
	for audit_event in response:
		if (  audit_event["eventSource"] == "s3.amazonaws.com" ):
			# do your stuff here

Open in new window

ASKER CERTIFIED SOLUTION

Laroy Shtotland

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

Shalom Carmel

Alternative solution