Reducing & optimising your Amazon Web Services (AWS) Costs across common services

Reducing and optimising your costs with AWS

 
One of the many reasons that people choose to deploy their infrastructure with a Public cloud provider such as AWS is the incredible potential of cost savings it has to offer against hosting your own infrastructure on premise.  However if you do not maintain and keep a close eye on your Cloud infrastructure then you can start to fall into traps that will end up costing you more than your projected AWS costs. 
 
This article is a guide on how to keep your costs to a minimum when using some of the most common AWS services ensuring that your estate remains optimised and your infrastructure is cost efficient.
 
AWS draws upon three main streams of revenue for use of its Services – Compute, Storage and Data Transfer.  There are other more specific streams for example where you are charged for the number of hits a CloudFront distribution might get, but generally the largest costs fit into the categories mentioned.
 

AWS Free Tier

 
When first using AWS most people will have set up their account using the Free Tier.  This Free Tier allows you to use many of AWS services for free for a year; however you must be aware that there are limitations to this ‘free’ usage.
 
More information on the Free Tier and to see what these limitations are can be found here
 
Their featured Services include the following limitations
 As soon as you use services outside of these limitations then you will start to incur costs so be mindful of this when deploying your initial services.
 
To help you monitor and manage this it’s always a good idea when implementing your first deployment on AWS to set up a billing alert.  This billing alert can be configured to notify you by e-mail when your AWS bill reaches a certain point allowing you to take action if need be.  This can help to prevent a nasty surprise at the end of the month when your bill is unexpectedly and considerably more than you thought (or can afford).  I have seen a number of people experience this on the AWS Official forum as they plead with AWS to waiver their costs.
 
To set up a billing alarm follow the simple steps located here.
 

EC2 Instances

 
When designing your infrastructure and service within AWS you will more than likely be using EC2 instances as your Compute power, and as previously mentioned this is one of the main streams of income for AWS.  Therefore it’s essential you manage your EC2 instances correctly, especially if the number of EC2 instances you have run in the hundreds!
 
AWS offer a huge range of instance types to choose from which are optimised for different performance requirements allowing you to choose the most appropriate type for the service/application that you intend to run. 
 
There are four different families of Instances, and nested within each are a range of Instance types providing varied sized performance based on Compute CPU, Storage and Memory.  The different Instance family types are:
 

• General Purpose
• Compute Optimised
• Memory Optimised
• Storage Optimised

 
The smallest instance types can cost as little as $0.013 and hour for a General Purpose t2.micro up to $5.52 an hour for a Storage optimised d2.8xlarge.
 
The complete range of AWS instances types and additional detailed information on each can be found here.
 

Which instance type should you use?

 
When choosing your instance type you should first liaise with your internal App Dev/OS Support teams who will have a deeper understanding of what the minimum performance is required to run the Applications and operating systems that are going to be installed on the instance.  This will give you a good baseline to work from.  If you do not have the luxury of having someone giving you this information then you simply need to give an ‘educated guess’ on your CPU/Storage/Memory requirements. 
 

What if I over/under provision my instance type?

 
If you over provision or under provision your EC2 instance do not be worried, this isn’t a problem within AWS, in fact it allows you to optimise your EC2 instances ensuring you are only paying for what you need.  EC2 allows you to resize your instance types with ease giving you full customisation of your estate and ensures you have the flexibility ensuring its optimisation.   For further information on changing your instance type size click here.
 
To help you sanitise your chosen selection you should monitor your EC2 environment using the AWS CloudWatch metrics.  When your EC2 instances are up and running with your required apps installed etc then you must monitor your CloudWatch metrics to ascertain if you are over/under utilising your instance.
 
If for example your CPU utilisation is consistently high (80%+) , you may wish to look at a larger instance type with additional CPU power to prevent a bottleneck.  Alternatively if your CPU is consistently running below 5-10% at its peak, then you ‘maybe ‘able to decrease your instance size saving you money.  I say ‘maybe’ as it’s important to look at more than one metric when basing your decision on resizing instance type.  In this scenario, you may not be able to decrease your instance size type if your Network throughput is running high.  Decreasing your instance size also decreases network throughput and therefore would cause a bottleneck in this circumstance.
 
The key is to finding the optimal performance of your applications and service for the instance types available
 

EC2 On-Demand, Reserved or Spot Instances?

 
There are three different methods of acquiring your instances, all of which come with a different price metrics
 

• On Demand
  
  
  • These instance types are available as and when you need them for a set price set at an hourly rate.  There are no upfront costs and are allow you to add these instances without any prior planning of your infrastructure
• Reserved
  
  • Reserved instance types allow you to provision instances for either 1 or 3 years.  By selecting to reserve instances it gives a greater cost saving over On Demand instances.  As a result if you know you are going to be using an EC2 instance of a certain type for a set period of time it is highly recommended that you do so with a Reserved instance as this will provide you with an instant cost saving 
• Spot
  
  • AWS allows you to bid a price for unused Compute capacity that AWS has.  Based off demand the ‘Spot’ price changes and if the price goes above your ‘bid’ price then you will lose you instance.  You will not get any warning and you do not have time to safely shut down and migrate your servers, they will simply be taken from you and all data removed.  Spot instances are good if you need to perform processing that can be interrupted and resumed at a later stage

 

Cost options for Reserved Instances

 
Prior to February 2015 there were options to purchase your reserved instances based on Light, Medium or Heavy utilisation.  More information on these options can be found here: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/reserved-instances-offerings.html.  For this article I shall focus on the current payment options which are defined as:
 

• No Upfront
  
  
  • As the name implies, there are no upfront costs when selecting this options however due to the reservation you are provided with a discounted hourly rate for the whole duration of the team regardless of how many hours you actually use the instance for.  You can only select this option when reserving for 1 year
• Partial Upfront
  
  • Reserving your instances with a partial upfront cost provides an even greater reduction of your hourly rate, again regardless of usage of your EC2 instance
• All Upfront
  
  • A single payment is made of your reserved instance from the outset and no other payments are required for the duration of the reservation regardless of number of hours used by your EC2 instances

 
For more information on buying Reserved Instances please see here.
 
Further information on EC2 Price metrics covering all of the above can be found here.
 

Shutdown/Terminate unused EC2

 
As you are probably aware by now, AWS charges you only for what you use when you use it.  Therefore there are significant savings to be made on your Compute and Storage when they become redundant and are no longer required.
 
Many corporations utilise AWS for Development and Test areas before migrating them to production.  Its best practise to shut down your Dev and Test instances at the end of the day (providing no overnight processing is taking place) to significantly reduce your Compute usage (only pay for what you use when you use it).  Even though you EC2 instances are not being used through the night, as long as they are up and running then you are being billed for the Compute usage. 
 
I have seen examples of people running a script that shuts down all EC2 instances that contain a certain ‘Tag’ defining them as a ‘Dev server’ after 17:30.  This is a great idea to automate your infrastructure in optimising your EC2 costs for you.
 

EBS Optimisation

 
When creating your EBS volumes there is an option for Provisioned IOPS.  This allows you to have enhanced dedicated throughout on your I/O.  For this increased performance you will charged a premium depending on the size of your volume.  Before selecting PIOPS (Provisioned IOPS) be sure that you actually need it, does your application require this level of performance?  You can use CloudWatch on your EBS volumes to ascertain if your application is struggling with the I/O read and writes on your EBS volumes.  If so, then you could perhaps use PIOPS to help alleviate the problem.  Similarly, if you had previously selected PIOPS, monitor your CloudWatch metrics as you may not need that level of throughput as you initially thought and so you could remove the PIOPS from your volume and optimising your EBS costs.
 
Monitor your current EBS volumes to ascertain if you have any detached EBS volumes just sitting there with no EC2 associations.  If your EBS volumes are not in use and attached to an available EC2 instance then create a snapshot of that EBS volume and delete the Volume.  Having a snapshot of the volume is cheaper that keeping the EBS volume.  Should you need to recreate the volume at any point then you still have the snapshot available to do so which you can then reattach to a new EC2 instance. 
 
For more information on creating snapshots of unused EBS Volumes see here.
 

Elastic IP addresses

 
You are allowed 5 Elastic IP addresses per region per AWS account, you can request more if required (details on how to do this can be found here.  When you create an EIP and associate it an instance you not charged for the use of that EIP.  However, if you disassociate the EIP and do not use it on another instance then you will be charged for having that EIP unless you release it back into the pool. 
 
So, to reiterate – If you have an EIP associated to an instance it will NOT cost you.  If you have an EIP un-associated and that EIP is available within your AWS account, you WILL be charged for not using it unless you release it back to the pool.
 
Please note, that if you associate a 2^nd EIP to the same instance then you will be charged for the 2^nd and any additional EIPs on the same instance.
 
Further information on pricing for EIPs can be found here.
 

Elastic Load Balancers

 
When deploying ELBs ensure that you have sufficient EC2 instances sitting behind them that are receiving the data.  When using ELBs you are charged for each hour you are using it and for each GB of data that is passed through the ELB.
 
As and when your infrastructure scales up and down be sure you review you use of ELB’s, sometimes you may run into circumstances where you may only have 1 or even no instances sitting behind them.  Terminate your ELBs when they are not providing a sufficient service as the costs of these can mount up unnecessarily when underutilised.
 
For more information on ELB pricing please click here.
 

S3, RRS and Glacier

 
S3 is AWS answer to unlimited object based storage that boasts a 99.999999999% durability of data with 99.99% availability.  With this service your data is automatically replicated across many different availability zones preventing the loss of your data.  S3 is a fantastic storage solution where you need to ensure the durability of that data. 
 
If you need the flexibility and scalability of S3, but you do not necessarily need the same level of durability then you can choose to store you data on S3 but with the Reduced Redundancy Storage option (RRS).  This simply means that your data isn’t replicated across as many availability zones and the durability of your data is reduced to 99.99% whilst still maintaining the 99.99% availability.  As a result it suggested that data stored on RRS should only be data that can be reproduced.  By using the RRS option you can make significant savings. 
 
Glacier is another storage solution by AWS, however this is essentially used for cold storage.  It provides the same level of durability and availability as S3, however at a significant cost reduction.  As a trade off from this cost Glacier is purely meant for data retention as to retrieve data within Glacier a request needs to be submitted, which can then take between 3-5 hours to process. 
 
S3, RRS and Glacier Pricing information can be found here.
 
For more information on the different storage types and deeper analysis of storage services, please see my other article: Selecting the right AWS storage for your needs.
 
CloudFront – Content Delivery Network.
 
As mentioned at the beginning of this article, one of the revenue streams AWS maximises on is Data Transfer out of the AWS environment. 
 
If you are hosting websites that serves to customers globally or to specific regions then utilising AWS CloudFront service would be of great use to you both from a data latency perspective and also cost optimisation. 
 
CloudFront is a Content Delivery Network that allows you to utilise AWS Edge Locations to reduce latency for static web content delivered to your customers.  By utilising the Edge locations data is cached for a specific period of time thus reducing the data transfer costs.
 
For further pricing information relating to CloudFront please see here.
 
There are many ways to help achieve cost optimisation within AWS and I have merely scratched the surface with some of the basic services that many people use.   Cost optimisation is a continual process and development when utilising AWS due to the scalability and flexibility of what the Cloud provides.  Your architecture and services are constantly changing with the demands of your applications, services and customers.  With these changes try to analyse how you can use these changes to gain a cost advantage and stay on top of your estate. 
 
Thank you for taking the time to read my article, if you have any feedback please do leave a comment below.

If you liked this article or found it helpful please click the 'Good Article' button at the bottom of this article, it would be very much appreciated.
    
I look forward to your comments and suggestions.

Stuart Scott