Tessando
asked on
Python Script Help for Traversing S3 Buckets
I've got a Python script that traverses S3 Buckets and prints out what folders and files have public permissions. This can be handy when auditing for security issues.
Right now, the script runs fine, but times-out by the time it hits the third bucket.
Can someone please help me find a way to "hard code" this on a per-bucket basis? In other words, if I have a bucket called "art-bucket", how could I get the script to traverse JUST that bucket and provide me the results.
BTW - I've installed both boto3 & Paginator
Thanks for your help.
Right now, the script runs fine, but times-out by the time it hits the third bucket.
Can someone please help me find a way to "hard code" this on a per-bucket basis? In other words, if I have a bucket called "art-bucket", how could I get the script to traverse JUST that bucket and provide me the results.
BTW - I've installed both boto3 & Paginator
Thanks for your help.
#This Script will use Paginator to print result for each bucket, executed in multiple threads
import boto3
import threading
import os.path
ACCESS_KEY = 'AKIAIXXXXXXXXXXX'
SECRET_ACCESS_KEY = 'XXUPJIsSXXxxXXxxXXo9Fl5TzSxXXxxXX3ly2XXlxjXXxxXX'
session = boto3.Session(aws_access_key_id = ACCESS_KEY, aws_secret_access_key = SECRET_ACCESS_KEY)
maxthreads = 5
sema = threading.Semaphore(value=maxthreads)
def list_object(bucket):
try:
s3 = session.client('s3')
flag1 = objcount = 0
paginator = s3.get_paginator('list_objects')
page_iterator = paginator.paginate(Bucket= bucket)
for page in page_iterator:
if 'Contents' in page:
for obj in page['Contents']:
uniobj = obj['Key'].encode('ascii', 'ignore').decode('ascii')
objAcl = s3.get_object_acl(Bucket=bucket, Key=obj['Key'])
flag2 = 0
for perm in objAcl['Grants']:
try:
if perm['Grantee']['Type'] == 'Group' and perm['Grantee']['URI'] == 'http://acs.amazonaws.com/groups/global/AllUsers':
if flag1 == 0:
f = open(bucket + '_list.txt', 'w')
print >> f, 'Bucket Name: ' + bucket + '\n####################'
flag1 = 1
if flag2 == 0:
print >> f, '\n' + uniobj + ':'
flag2 = 1
permdets = perm['Permission']
if permdets == 'READ':
print >> f, 'Public Permission for Reading Object'
elif permdets == 'READ_ACP':
print >> f, 'Public Permission for Reading Object ACL'
elif permdets == 'WRITE_ACP':
print >> f, 'Public Permission for Adding/Modifying Object ACL'
#f.close()
except Exception as e:
print (uniobj + ':')
print (e)
continue
objcount += 1
else:
pass
if os.path.isfile((bucket + '_list.txt')):
f.close()
except Exception as e:
print (bucket + ':')
print (e)
print("%s: DONE : Scanned %i Objects" % (bucket, objcount))
s3 = session.client('s3')
for bucket in s3.list_buckets()['Buckets']:
try:
thread = threading.Thread(target = list_object, args=(bucket['Name'],)).start()
except Exception as e:
continue
ASKER
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Replace it with the following to traverse ONLY some buckets:
Open in new window