Python to produce nice tabular output

enthuguy
enthuguy used Ask the Experts™
on
HI,

I'm trying to modify below python script o produce nice tabular output. (Right now its not in a readable format)

Thanks in advance

Script source:
https://github.com/hjacobs/aws-cost-and-usage-report/blob/master/aws-cost-and-usage-report.py

Current output
./aws-cost-and-usage-report.py
TimePeriod	LinkedAccount	Service                                 	Amount	Unit	Estimated
2019-11-08 	 21212121212121	AWS CloudTrail 	 	 	                       0.153943 	 USD 	 False
2019-11-08 	 21212121212121	AWS Config 	 	 	                          9.213 	 USD 	 False
2019-11-08 	 21212121212121	AWS Direct Connect 	 	 	                   0.2797877163 	 USD 	 False
2019-11-08 	 21212121212121	AWS Key Management Service 	 	 	                   1.4141780112 	 USD 	 False
2019-11-08 	 21212121212121	AWS Lambda 	 	 	                   0.0804225759 	 USD 	 False
2019-11-08 	 21212121212121	Amazon DynamoDB 	 	 	                   0.3836161225 	 USD 	 False
2019-11-08 	 21212121212121	Amazon EC2 Container Registry (ECR) 	 	 	                   0.0783308328 	 USD 	 False
2019-11-08 	 21212121212121	Amazon EC2 Container Service 	 	 	                              0 	 USD 	 False
2019-11-08 	 21212121212121	EC2 - Other 	 	 	                   6.8639388761 	 USD 	 False
2019-11-08 	 21212121212121	Amazon Elastic Compute Cloud - Compute 	 	 	                  73.1890902202 	 USD 	 False
2019-11-08 	 21212121212121	Amazon Elastic File System 	 	 	                   2.2110942898 	 USD 	 False
2019-11-08 	 21212121212121	Amazon Elastic Load Balancing 	 	 	                   4.8388505022 	 USD 	 False
2019-11-08 	 21212121212121	Amazon GuardDuty 	 	 	                    0.761623977 	 USD 	 False
2019-11-08 	 21212121212121	Amazon Relational Database Service 	 	 	                  21.2797291955 	 USD 	 False

Open in new window

Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
nociSoftware Engineer
Distinguished Expert 2018

Commented:
How about using format...  (https://www.programiz.com/python-programming/methods/string/format )
Then use for heading & data:

...
template="{10} {14} {40} {16.10} {6} {10}"
...
print(template.format("TimePeriod", "LinkedAccount", "Service", "Amount", "Unit", "Estimated"))
...
   print(template.format(result_by_time['TimePeriod']['Start'], group['Keys']['LINKED_ACCOUNT'],  group['Keys']['SERVICE'], amount, unit,  result_by_time['Estimated']))
...

Open in new window

You still need the subfields for the keys to be corrected i guessed them from the source.

The format might need a bit of adjustment. I guessed the widths from the above example.

Author

Commented:
Thanks a lot Noci, will update you

Author

Commented:
Sorry, this is what I updated. since i'm not familiar with python


#!/usr/bin/env python3

import argparse
import boto3
import datetime

parser = argparse.ArgumentParser()
parser.add_argument('--days', type=int, default=30)
args = parser.parse_args()


now = datetime.datetime.utcnow()
start = (now - datetime.timedelta(days=args.days)).strftime('%Y-%m-%d')
end = now.strftime('%Y-%m-%d')
template="{10} {14} {40} {16.10} {6} {10}"

cd = boto3.client('ce', 'ap-southeast-2')

results = []

token = None
while True:
    if token:
        kwargs = {'NextPageToken': token}
    else:
        kwargs = {}
    data = cd.get_cost_and_usage(TimePeriod={'Start': start, 'End':  end}, Granularity='DAILY', Metrics=['UnblendedCost'], GroupBy=[{'Type': 'DIMENSION', 'Key': 'LINKED_ACCOUNT'}, {'Type': 'DIMENSION', 'Key': 'SERVICE'}], **kwargs)
    results += data['ResultsByTime']
    token = data.get('NextPageToken')
    if not token:
        break

# print('\t'.join(['TimePeriod', 'LinkedAccount', 'Service'.ljust(40, ' '), 'Amount', 'Unit', 'Estimated']))
print(template.format("TimePeriod", "LinkedAccount", "Service", "Amount", "Unit", "Estimated")

for result_by_time in results:
    for group in result_by_time['Groups']:
        amount = group['Metrics']['UnblendedCost']['Amount']
        unit = group['Metrics']['UnblendedCost']['Unit']
        # comp = join(group['Keys'])
        # rep_start_time = result_by_time['TimePeriod']['Start']
        # print(rep_start_time.ljust(10, ' '), '\t', '\t' .join(group['Keys']), '\t', '\t', '\t', amount.rjust(30, ' '), '\t', unit, '\t', result_by_time['Estimated'])
        print(template.format(result_by_time['TimePeriod']['Start'], group['Keys'],  amount, '\t', unit, '\t', result_by_time['Estimated'])

Open in new window



Error

./aws-cost-and-usage-report.py --days=7
Traceback (most recent call last):
  File "./aws-cost-and-usage-report.py", line 34, in <module>
    print(template.format("TimePeriod", "LinkedAccount", "Service", "Amount", "Unit", "Estimated"))
IndexError: tuple index out of range

Open in new window

Expert Spotlight: Joe Anderson (DatabaseMX)

We’ve posted a new Expert Spotlight!  Joe Anderson (DatabaseMX) has been on Experts Exchange since 2006. Learn more about this database architect, guitar aficionado, and Microsoft MVP.

nociSoftware Engineer
Distinguished Expert 2018

Commented:
there are missing ) with the print statements.  And i removed the '\t' from the 2nd string.
(I copied a wrong example earlier...)
(add : in the formats type field for float, and difference between header & data template)
#!/usr/bin/env python3

import argparse
import boto3
import datetime

parser = argparse.ArgumentParser()
parser.add_argument('--days', type=int, default=30)
args = parser.parse_args()


now = datetime.datetime.utcnow()
start = (now - datetime.timedelta(days=args.days)).strftime('%Y-%m-%d')
end = now.strftime('%Y-%m-%d')
dtemplate="{:10} {:14} {:40} {:16.10f} {:6} {:10}"
htemplate="{:10} {:14} {:40} {:16} {:6} {:10}"

cd = boto3.client('ce', 'ap-southeast-2')

results = []

token = None
while True:
    if token:
        kwargs = {'NextPageToken': token}
    else:
        kwargs = {}
    data = cd.get_cost_and_usage(TimePeriod={'Start': start, 'End':  end}, Granularity='DAILY', Metrics=['UnblendedCost'], GroupBy=[{'Type': 'DIMENSION', 'Key': 'LINKED_ACCOUNT'}, {'Type': 'DIMENSION', 'Key': 'SERVICE'}], **kwargs)
    results += data['ResultsByTime']
    token = data.get('NextPageToken')
    if not token:
        break

# print('\t'.join(['TimePeriod', 'LinkedAccount', 'Service'.ljust(40, ' '), 'Amount', 'Unit', 'Estimated']))
print(htemplate.format("TimePeriod", "LinkedAccount", "Service", "Amount", "Unit", "Estimated"))

for result_by_time in results:
    for group in result_by_time['Groups']:
        amount = group['Metrics']['UnblendedCost']['Amount']
        unit = group['Metrics']['UnblendedCost']['Unit']
        # comp = join(group['Keys'])
        # rep_start_time = result_by_time['TimePeriod']['Start']
        # print(rep_start_time.ljust(10, ' '), '\t', '\t' .join(group['Keys']), '\t', '\t', '\t', amount.rjust(30, ' '), '\t', unit, '\t', result_by_time['Estimated'])
        print(dtemplate.format(result_by_time['TimePeriod']['Start'], group['Keys']['LINKED_ACCOUNT'],  group['Keys']['SERVICE'], amount, unit,  result_by_time['Estimated']))

Open in new window

Suhas .Senior QA Manager

Commented:
Hi,
You can using one of the modules:

https://pypi.org/project/tabulate/

or

https://pypi.org/project/tabulate/
ex: print(tabulate(table, headers, tablefmt="pipe"))

or

https://pypi.org/project/beautifultable/

Author

Commented:
Hi noci, sorry again.

kindly help pls

./aws-cost-and-usage-report.py
TimePeriod LinkedAccount  Service                                  Amount           Unit   Estimated
Traceback (most recent call last):
  File "./aws-cost-and-usage-report.py", line 44, in <module>
    print(dtemplate.format(result_by_time['TimePeriod']['Start'], group['Keys']['LINKED_ACCOUNT'],  group['Keys']['SERVICE'], amount, unit,  result_by_time['Estimated']))
TypeError: list indices must be integers or slices, not str

Author

Commented:
Thanks Juan and Suhas,
I tried these as a standalone scripts it works fine.

My challenge is how to incorporate this to my script :)
nociSoftware Engineer
Distinguished Expert 2018

Commented:
The group['Keys]['LINKED_ACCOUNT] & group['Keys']['SERVICE'] probably are not strings. This might need to be changed to:

 group['Keys][0] & group['Keys'][1]

print(dtemplate.format(result_by_time['TimePeriod']['Start'], group['Keys'][0],  group['Keys'][1], amount, unit,  result_by_time['Estimated']))

Open in new window


I have no access to AWS so i cannot verify the code.

Author

Commented:
its ok, I can understand....thx for your help

may be we are getting close :)


./aws-cost-and-usage-report.py
TimePeriod LinkedAccount  Service                                  Amount           Unit   Estimated
Traceback (most recent call last):
  File "./aws-cost-and-usage-report.py", line 45, in <module>
    print(dtemplate.format(result_by_time['TimePeriod']['Start'], group['Keys'][0],  group['Keys'][1], amount, unit,  result_by_time['Estimated']))
ValueError: Unknown format code 'f' for object of type 'str'

Open in new window

Software Engineer
Distinguished Expert 2018
Commented:
Aparantly the amount field is a string not a float number.   You can use htemplate to quickly test it. The reason for two templates is the assumption the amount was a numeric value not a string. That may cause alignment issues around the decimal point though.

So the next might be better:
print(dtemplate.format(result_by_time['TimePeriod']['Start'], group['Keys'][0],  group['Keys'][1], float(amount), unit,  result_by_time['Estimated']))

Open in new window

Author

Commented:
Thanks a lot Noci, you are simply great!

Even though you did not have the aws env...you helped me. :)

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial