Link to home
Start Free TrialLog in
Avatar of enthuguy
enthuguyFlag for Australia

asked on

Python to produce nice tabular output

HI,

I'm trying to modify below python script o produce nice tabular output. (Right now its not in a readable format)

Thanks in advance

Script source:
https://github.com/hjacobs/aws-cost-and-usage-report/blob/master/aws-cost-and-usage-report.py

Current output
./aws-cost-and-usage-report.py
TimePeriod	LinkedAccount	Service                                 	Amount	Unit	Estimated
2019-11-08 	 21212121212121	AWS CloudTrail 	 	 	                       0.153943 	 USD 	 False
2019-11-08 	 21212121212121	AWS Config 	 	 	                          9.213 	 USD 	 False
2019-11-08 	 21212121212121	AWS Direct Connect 	 	 	                   0.2797877163 	 USD 	 False
2019-11-08 	 21212121212121	AWS Key Management Service 	 	 	                   1.4141780112 	 USD 	 False
2019-11-08 	 21212121212121	AWS Lambda 	 	 	                   0.0804225759 	 USD 	 False
2019-11-08 	 21212121212121	Amazon DynamoDB 	 	 	                   0.3836161225 	 USD 	 False
2019-11-08 	 21212121212121	Amazon EC2 Container Registry (ECR) 	 	 	                   0.0783308328 	 USD 	 False
2019-11-08 	 21212121212121	Amazon EC2 Container Service 	 	 	                              0 	 USD 	 False
2019-11-08 	 21212121212121	EC2 - Other 	 	 	                   6.8639388761 	 USD 	 False
2019-11-08 	 21212121212121	Amazon Elastic Compute Cloud - Compute 	 	 	                  73.1890902202 	 USD 	 False
2019-11-08 	 21212121212121	Amazon Elastic File System 	 	 	                   2.2110942898 	 USD 	 False
2019-11-08 	 21212121212121	Amazon Elastic Load Balancing 	 	 	                   4.8388505022 	 USD 	 False
2019-11-08 	 21212121212121	Amazon GuardDuty 	 	 	                    0.761623977 	 USD 	 False
2019-11-08 	 21212121212121	Amazon Relational Database Service 	 	 	                  21.2797291955 	 USD 	 False

Open in new window

Avatar of noci
noci

How about using format...  (https://www.programiz.com/python-programming/methods/string/format )
Then use for heading & data:

...
template="{10} {14} {40} {16.10} {6} {10}"
...
print(template.format("TimePeriod", "LinkedAccount", "Service", "Amount", "Unit", "Estimated"))
...
   print(template.format(result_by_time['TimePeriod']['Start'], group['Keys']['LINKED_ACCOUNT'],  group['Keys']['SERVICE'], amount, unit,  result_by_time['Estimated']))
...

Open in new window

You still need the subfields for the keys to be corrected i guessed them from the source.

The format might need a bit of adjustment. I guessed the widths from the above example.
Avatar of enthuguy

ASKER

Thanks a lot Noci, will update you
Sorry, this is what I updated. since i'm not familiar with python


#!/usr/bin/env python3

import argparse
import boto3
import datetime

parser = argparse.ArgumentParser()
parser.add_argument('--days', type=int, default=30)
args = parser.parse_args()


now = datetime.datetime.utcnow()
start = (now - datetime.timedelta(days=args.days)).strftime('%Y-%m-%d')
end = now.strftime('%Y-%m-%d')
template="{10} {14} {40} {16.10} {6} {10}"

cd = boto3.client('ce', 'ap-southeast-2')

results = []

token = None
while True:
    if token:
        kwargs = {'NextPageToken': token}
    else:
        kwargs = {}
    data = cd.get_cost_and_usage(TimePeriod={'Start': start, 'End':  end}, Granularity='DAILY', Metrics=['UnblendedCost'], GroupBy=[{'Type': 'DIMENSION', 'Key': 'LINKED_ACCOUNT'}, {'Type': 'DIMENSION', 'Key': 'SERVICE'}], **kwargs)
    results += data['ResultsByTime']
    token = data.get('NextPageToken')
    if not token:
        break

# print('\t'.join(['TimePeriod', 'LinkedAccount', 'Service'.ljust(40, ' '), 'Amount', 'Unit', 'Estimated']))
print(template.format("TimePeriod", "LinkedAccount", "Service", "Amount", "Unit", "Estimated")

for result_by_time in results:
    for group in result_by_time['Groups']:
        amount = group['Metrics']['UnblendedCost']['Amount']
        unit = group['Metrics']['UnblendedCost']['Unit']
        # comp = join(group['Keys'])
        # rep_start_time = result_by_time['TimePeriod']['Start']
        # print(rep_start_time.ljust(10, ' '), '\t', '\t' .join(group['Keys']), '\t', '\t', '\t', amount.rjust(30, ' '), '\t', unit, '\t', result_by_time['Estimated'])
        print(template.format(result_by_time['TimePeriod']['Start'], group['Keys'],  amount, '\t', unit, '\t', result_by_time['Estimated'])

Open in new window



Error

./aws-cost-and-usage-report.py --days=7
Traceback (most recent call last):
  File "./aws-cost-and-usage-report.py", line 34, in <module>
    print(template.format("TimePeriod", "LinkedAccount", "Service", "Amount", "Unit", "Estimated"))
IndexError: tuple index out of range

Open in new window

there are missing ) with the print statements.  And i removed the '\t' from the 2nd string.
(I copied a wrong example earlier...)
(add : in the formats type field for float, and difference between header & data template)
#!/usr/bin/env python3

import argparse
import boto3
import datetime

parser = argparse.ArgumentParser()
parser.add_argument('--days', type=int, default=30)
args = parser.parse_args()


now = datetime.datetime.utcnow()
start = (now - datetime.timedelta(days=args.days)).strftime('%Y-%m-%d')
end = now.strftime('%Y-%m-%d')
dtemplate="{:10} {:14} {:40} {:16.10f} {:6} {:10}"
htemplate="{:10} {:14} {:40} {:16} {:6} {:10}"

cd = boto3.client('ce', 'ap-southeast-2')

results = []

token = None
while True:
    if token:
        kwargs = {'NextPageToken': token}
    else:
        kwargs = {}
    data = cd.get_cost_and_usage(TimePeriod={'Start': start, 'End':  end}, Granularity='DAILY', Metrics=['UnblendedCost'], GroupBy=[{'Type': 'DIMENSION', 'Key': 'LINKED_ACCOUNT'}, {'Type': 'DIMENSION', 'Key': 'SERVICE'}], **kwargs)
    results += data['ResultsByTime']
    token = data.get('NextPageToken')
    if not token:
        break

# print('\t'.join(['TimePeriod', 'LinkedAccount', 'Service'.ljust(40, ' '), 'Amount', 'Unit', 'Estimated']))
print(htemplate.format("TimePeriod", "LinkedAccount", "Service", "Amount", "Unit", "Estimated"))

for result_by_time in results:
    for group in result_by_time['Groups']:
        amount = group['Metrics']['UnblendedCost']['Amount']
        unit = group['Metrics']['UnblendedCost']['Unit']
        # comp = join(group['Keys'])
        # rep_start_time = result_by_time['TimePeriod']['Start']
        # print(rep_start_time.ljust(10, ' '), '\t', '\t' .join(group['Keys']), '\t', '\t', '\t', amount.rjust(30, ' '), '\t', unit, '\t', result_by_time['Estimated'])
        print(dtemplate.format(result_by_time['TimePeriod']['Start'], group['Keys']['LINKED_ACCOUNT'],  group['Keys']['SERVICE'], amount, unit,  result_by_time['Estimated']))

Open in new window

Hi,
You can using one of the modules:

https://pypi.org/project/tabulate/

or

https://pypi.org/project/tabulate/
ex: print(tabulate(table, headers, tablefmt="pipe"))

or

https://pypi.org/project/beautifultable/
Hi noci, sorry again.

kindly help pls

./aws-cost-and-usage-report.py
TimePeriod LinkedAccount  Service                                  Amount           Unit   Estimated
Traceback (most recent call last):
  File "./aws-cost-and-usage-report.py", line 44, in <module>
    print(dtemplate.format(result_by_time['TimePeriod']['Start'], group['Keys']['LINKED_ACCOUNT'],  group['Keys']['SERVICE'], amount, unit,  result_by_time['Estimated']))
TypeError: list indices must be integers or slices, not str
Thanks Juan and Suhas,
I tried these as a standalone scripts it works fine.

My challenge is how to incorporate this to my script :)
The group['Keys]['LINKED_ACCOUNT] & group['Keys']['SERVICE'] probably are not strings. This might need to be changed to:

 group['Keys][0] & group['Keys'][1]

print(dtemplate.format(result_by_time['TimePeriod']['Start'], group['Keys'][0],  group['Keys'][1], amount, unit,  result_by_time['Estimated']))

Open in new window


I have no access to AWS so i cannot verify the code.
its ok, I can understand....thx for your help

may be we are getting close :)


./aws-cost-and-usage-report.py
TimePeriod LinkedAccount  Service                                  Amount           Unit   Estimated
Traceback (most recent call last):
  File "./aws-cost-and-usage-report.py", line 45, in <module>
    print(dtemplate.format(result_by_time['TimePeriod']['Start'], group['Keys'][0],  group['Keys'][1], amount, unit,  result_by_time['Estimated']))
ValueError: Unknown format code 'f' for object of type 'str'

Open in new window

ASKER CERTIFIED SOLUTION
Avatar of noci
noci

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Thanks a lot Noci, you are simply great!

Even though you did not have the aws env...you helped me. :)