Amazon EC2 Backup to S3 Bucket using Python Code

Not python, but a shell script that is scheduled on all of my servers.
The script gets the server's meta data and uses it to create a snapshot.

It still uses the old EC2 cli instead of the generic aws cli, and still works.

When using python, the concept will be identical:
* Get instance meta-data
* Create snapshots for the instance volumes
* Tag the snapshots correctly

#!/bin/sh

# You don't need these, assign the servers to a role instead
export AWS_ACCESS_KEY=<aws-access-key>
export AWS_SECRET_KEY=<aws-secret-key>


export PATH=$PATH:/opt/aws/bin
export JAVA_HOME=/usr/lib/jvm/jre
export EC2_HOME=/opt/aws/apitools/ec2

DATE=`date +%Y-%m-%d`
TIME=`date +'%R:%S'`

# find availability zone, for example eu-west-1b
EC2_AVAILABILITY_ZONE=`curl http://169.254.169.254/latest/meta-data/placement/availability-zone`

# remove last letter to get region
EC2_REGION="${EC2_AVAILABILITY_ZONE%?}"
# set EC2_URL environment variable. We ignore it by using the --region switch directly
# export EC2_URL=ec2.$EC2_REGION.amazonaws.com

# find this instance id
EC2_INSTANCE_ID=`curl http://169.254.169.254/latest/meta-data/instance-id/`

# find this volume. only works for single volume servers.
ec2string=`ec2-describe-instance-attribute  --region $EC2_REGION  $EC2_INSTANCE_ID -b`
ec2arr=($ec2string)
EC2_VOLUME=${ec2arr[2]}

# send backup and get snapshot id
echo ec2-create-snapshot  --region $EC2_REGION  $EC2_VOLUME -d "Weekly backup $DATE.$TIME"

ec2string=`ec2-create-snapshot  --region $EC2_REGION  $EC2_VOLUME -d "Weekly backup $DATE.$TIME"`
ec2arr=($ec2string)
EC2_SNAPSHOT=${ec2arr[1]}


# get Name attribute
echo ec2-describe-tags --region $EC2_REGION --filter "resource-id=$EC2_INSTANCE_ID" --filter "key=Name"

ec2string=`ec2-describe-tags --region $EC2_REGION --filter resource-id=$EC2_INSTANCE_ID --filter key=Name`
ec2arr=($ec2string)
EC2_INSTANCE_NAME=${ec2arr[4]}

# create Name tag for snapshot
echo ec2-create-tags  --region $EC2_REGION  $EC2_SNAPSHOT --tag "Name=$EC2_INSTANCE_NAME"
ec2-create-tags  --region $EC2_REGION  $EC2_SNAPSHOT --tag "Name=$EC2_INSTANCE_NAME"

Select allOpen in new window

Hi shalomc and thank you for getting back to me.

Just wanted to clarify, does this script run on the AWS Linux AMI and effect all Instances or does it need to run separately on each instance that needs backing up? If that is the case I don't believe I can use this because most of the instances I use are windows based.

This is a link to a python script that creates snapshots of volumes based on tags.
https://medium.com/chrisjerry/snapshot-creation-and-delete-snap-older-x-days-python-script-63e224a699ff

When I run the snapshot creation script it is throwing this error:

  File "backup.py", line 4
SyntaxError: Non-ASCII character '\xe2' in file backup.py on line 4, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details

Select allOpen in new window

it appears to be pointing at this line, I've looked at the code and tried to fix it but it does not appear to be making any difference

ec = boto3.client(‘ec2’,’ap-southeast-2') #mention your region to reboot

Select allOpen in new window

Try to add this line at the top of your code

# -*- coding: utf-8 -*-

Select allOpen in new window

Invalid Syntax Error

  File "backup.py", line 5
    ec = boto3.client(‘ec2’,’ap-southeast-2') #mention your region to reboot

Select allOpen in new window

I checked the original to see if anything was wrong but it all seems to check out

Looks like the code you posted has this character as single quotes: ‘

‘ec2’

Select allOpen in new window

Try to replace all single quotes with double quotes, lets see if it helps

Okay,

I made the changes but then I realized that most of the script was like that so I started making changes which appeared to be working but then I got to this.

  File "backup.py", line 12
    print "Backing up %s in %s %" (volume["VolumeId"], volume["AvailabilityZone"])
        ^
IndentationError: expected an indented block

Select allOpen in new window

Currently the code is:

# -*- coding: utf-8 -*-
import boto3
import collections
import datetime
ec = boto3.client("ec2","ap-southeast-2") #mention your region to reboot
to_tag = 0
def lambda_handler(event, context):
 print 'hi'
 reservations = ec.describe_volumes( Filters=[ {"Name": "tag-key", "Values": ["backup", "True"]}, ] )
 print(reservations)
 for volume in reservations["Volumes"]:
 print "Backing up %s in %s %" (volume["VolumeId"], volume["AvailabilityZone"])

 # Create snapshot
 reservations = ec.create_snapshot(VolumeId=volume["VolumeId"],Description="Lambda backup for ebs" + volume["VolumeId"])

 result = reservations["SnapshotId"]
 print(result)

 ec.create_tags(
 Resources=[result],Tags=[
 {‘Key’: ‘Name’, ‘Value’: ‘snapshot’ },
 ]
 )

Select allOpen in new window

Well, you gotta know python to mess with python :)

Line 11 is a for loop, so you have to indent the loop block.

# -*- coding: utf-8 -*-
import boto3
import collections
import datetime
ec = boto3.client("ec2","ap-southeast-2") #mention your region to reboot
to_tag = 0
def lambda_handler(event, context):
 print 'hi'
 reservations = ec.describe_volumes( Filters=[ {"Name": "tag-key", "Values": ["backup", "True"]}, ] )
 print(reservations)
 for volume in reservations["Volumes"]:
     print "Backing up %s in %s %" (volume["VolumeId"], volume["AvailabilityZone"])

     # Create snapshot
     reservations = ec.create_snapshot(VolumeId=volume["VolumeId"],Description="Lambda backup for ebs" + volume["VolumeId"])

     result = reservations["SnapshotId"]
     print(result)

     ec.create_tags(
     Resources=[result],Tags=[
         {‘Key’: ‘Name’, ‘Value’: ‘snapshot’ },
         ]
     )

Select allOpen in new window

Hi,

It may just be me but I setup the code described above and setup a instance on aws with the appropriate tags, ran the script but nothing happened and there was no errors.

Not sure what is happening, I setup a access key with the appropriate permissions, I have two instances with the tag Backup and value true.

Any thoughts?

Tags are case sensitive

I have confirmed the tags are correct. Same effect. Could it be in the User Permissions

• AmazonSSMFullAccess
• AmazonEC2FullAccess

Log the responses you receive from AWS.

See here how http://boto3.readthedocs.io/en/latest/reference/core/boto3.html

According the to the Security Dashboard the key has not been used yet. But I already have run "awsconfigure" on the instance.

try to run this from the command line.  Does it work? Do you see your volume?

aws ec2 describe-volumes --region ap-southeast-2

Select allOpen in new window

I have run the command and I do see a full list of volumes