• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 90
  • Last Modified:

Python3 nested syntax while executing subprocess call.

HI Experts,
could you hep with this tricky syntax please? I'm trying to execute an aws cli within python3.

Simple aw cli works fine, but could you help me with nested aws cli

This one works fine.
p = Popen(['aws', 'ec2', 'describe-instances', '--instance-ids', instance , '--output', 'table'])

Open in new window


Pls help with this. Trying to execute in above subprocess syntax. This one describe an aws instance based on user friendly tag name.
aws ec2 describe-instances --instance-ids $(aws ec2 describe-instances --filters  "Name=tag:Name,Values=testjumpbox" --query "Reservations[].Instances[].[InstanceId]" --output text | tr '\n' ' ') --output table

Open in new window

0
enthuguy
Asked:
enthuguy
  • 8
  • 4
1 Solution
 
enthuguyAuthor Commented:
Tried below but still no luck

>>> p = Popen(['aws', 'ec2', 'describe-instances', '--instance-ids', '$(aws', 'ec2', 'describe-instances', '--filters', '"Name=tag:Name,Values=testjumpbox"', '--query', '"Reservations[].Instances[].[InstanceId]"', '--output', 'text', '|', 'tr', '\n' ' ', ')'])
>>>
Error parsing parameter '--filters': Expected: '=', received: '"' for input:
"Name=tag:Name,Values=testjumpbox"
^

Open in new window



>>> p = Popen(['aws', 'ec2', 'describe-instances', '--instance-ids', '$(aws', 'ec2', 'describe-instances', '--filters', 'Name=tag:Name,Values=testjumpbox', '--query', '"Reservations[].Instances[].[InstanceId]"', '--output', 'text', '|', 'tr', '\n' ' ', ')'])
>>>
Error parsing parameter '--filters': Expected: '=', received: '|' for input:
|
^

Open in new window

0
 
enthuguyAuthor Commented:
Since it didnt like the shell pipe "|"
was trying this..but still no luck.....I think, we are getting closer with you help...we can close it.

p = Popen(['aws', 'ec2', 'describe-instances', '--instance-ids', '$(aws', 'ec2', 'describe-instances', '--filters', 'Name=tag:Name,Values=testjumpbox', '--query', 'Reservations[].Instances[].[InstanceId]', '--output', 'text' ')'])
>>> usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:

  aws help
  aws <command> help
  aws <command> <subcommand> help
aws: error: argument --output: Invalid choice, valid choices are:

json                                     | text
table


Invalid choice: 'text)', maybe you meant:

  * text

Open in new window

0
 
NorieVBA ExpertCommented:
Couldn't you put the code you want to execute in a script file and then call/run that using Popen?

p = Popen(['myscript.bsh', 'shell=true'])

Open in new window

0
Ultimate Tool Kit for Technology Solution Provider

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.

 
enthuguyAuthor Commented:
Good idea nonie,

But trying to avoid shell :)
0
 
NorieVBA ExpertCommented:
In your first post you have this, is this exactly how you would run things from the command line?
aws ec2 describe-instances --instance-ids $(aws ec2 describe-instances --filters  "Name=tag:Name,Values=testjumpbox" --query "Reservations[].Instances[].[InstanceId]" --output text | tr '\n' ' ') --output table

Open in new window

PS Have you looked at the awscli package for Python?
0
 
NorieVBA ExpertCommented:
Does this work?
command = 'aws ec2 describe-instances --instance-ids $(aws ec2 describe-instances --filters  "Name=tag:Name,Values=testjumpbox" --query "Reservations[].Instances[].[InstanceId]" --output text | tr \'\\n\' \' \' --output table'

p = Popen(command, universal_newlines=True, shell=true)

Open in new window

1
 
enthuguyAuthor Commented:
Thx noire, will give a try
0
 
enthuguyAuthor Commented:
# Tried to debug using Python terminal by splitting the execution into two (1. get an instance id by name, then describe instance by an instance id)
# Step 1: Get Instance Id
p = Popen(['aws', 'ec2', 'describe-instances', '--filters', 'Name=tag:Name,Values=testjumpbox', '--query', 'Reservations[].Instances[].[InstanceId]', '--output', 'text'])
>>> i-03043e0d99c624abe

Open in new window


# Tried to store the instance id into a variable, then use it in the next aws Popen cmd, but didn't work. It printed the actual cmd :)
>>> gInstanceId = (['aws', 'ec2', 'describe-instances', '--filters', 'Name=tag:Name,Values=testjumpbox', '--query', 'Reservations[].Instances[].[InstanceId]', '--output', 'text'])
>>> print (gInstanceId)
['aws', 'ec2', 'describe-instances', '--filters', 'Name=tag:Name,Values=testjumpbox', '--query', 'Reservations[].Instances[].[InstanceId]', '--output', 'text']

Open in new window


>>> gInstanceId = ['aws', 'ec2', 'describe-instances', '--filters', 'Name=tag:Name,Values=testjumpbox', '--query', 'Reservations[].Instances[].[InstanceId]', '--output', 'text']
>>> print (gInstanceId)
['aws', 'ec2', 'describe-instances', '--filters', 'Name=tag:Name,Values=testjumpbox', '--query', 'Reservations[].Instances[].[InstanceId]', '--output', 'text']

Open in new window

0
 
enthuguyAuthor Commented:
Quick update, slighly better situation :)

>>> import configparser, argparse, subprocess
>>> from subprocess import Popen, PIPE, check_output
>>> out = check_output (['aws', 'ec2', 'describe-instances', '--filters', 'Name=tag:Name,Values=testjumpbox', '--query', 'Reservations[].Instances[].[InstanceId]', '--output', 'text'])
>>> print(out)
b'i-03043e0d99c624abe\n'

Open in new window


Could you help me how to...
  1. Remove leading 'b' and '\n'
  2. Validate if it is not blank/null
0
 
enthuguyAuthor Commented:
Current value
>>> print(out)
b'i-03043e0d99c624abe\n'

Strip removes trailing newline, but couldn't remove leading 'b' char   :)
>>> print(out.strip())
b'i-03043e0d99c624abe'

Open in new window


one more to go :)
0
 
NorieVBA ExpertCommented:
Perhaps.
print(out.strip()[1:])

Open in new window


PS Did this not work?
command = 'aws ec2 describe-instances --instance-ids $(aws ec2 describe-instances --filters  "Name=tag:Name,Values=testjumpbox" --query "Reservations[].Instances[].[InstanceId]" --output text | tr \'\\n\' \' \' --output table'

p = Popen(command, universal_newlines=True, shell=true)

Open in new window

0
 
enthuguyAuthor Commented:
Thanks a lot Norie, you are a start.
Your solution is actually much easier to code/read as well :)

Uppercase T in shell=True
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Featured Post

Managing Security & Risk at the Speed of Business

Gartner Research VP, Neil McDonald & AlgoSec CTO, Prof. Avishai Wool, discuss the business-driven approach to automated security policy management, its benefits and how to align security policy management with business processes to address today's security challenges.

  • 8
  • 4
Tackle projects and never again get stuck behind a technical roadblock.
Join Now