Python3 nested syntax while executing subprocess call.

HI Experts,
could you hep with this tricky syntax please? I'm trying to execute an aws cli within python3.

Simple aw cli works fine, but could you help me with nested aws cli

This one works fine.
p = Popen(['aws', 'ec2', 'describe-instances', '--instance-ids', instance , '--output', 'table'])

Open in new window


Pls help with this. Trying to execute in above subprocess syntax. This one describe an aws instance based on user friendly tag name.
aws ec2 describe-instances --instance-ids $(aws ec2 describe-instances --filters  "Name=tag:Name,Values=testjumpbox" --query "Reservations[].Instances[].[InstanceId]" --output text | tr '\n' ' ') --output table

Open in new window

enthuguyAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

enthuguyAuthor Commented:
Tried below but still no luck

>>> p = Popen(['aws', 'ec2', 'describe-instances', '--instance-ids', '$(aws', 'ec2', 'describe-instances', '--filters', '"Name=tag:Name,Values=testjumpbox"', '--query', '"Reservations[].Instances[].[InstanceId]"', '--output', 'text', '|', 'tr', '\n' ' ', ')'])
>>>
Error parsing parameter '--filters': Expected: '=', received: '"' for input:
"Name=tag:Name,Values=testjumpbox"
^

Open in new window



>>> p = Popen(['aws', 'ec2', 'describe-instances', '--instance-ids', '$(aws', 'ec2', 'describe-instances', '--filters', 'Name=tag:Name,Values=testjumpbox', '--query', '"Reservations[].Instances[].[InstanceId]"', '--output', 'text', '|', 'tr', '\n' ' ', ')'])
>>>
Error parsing parameter '--filters': Expected: '=', received: '|' for input:
|
^

Open in new window

0
enthuguyAuthor Commented:
Since it didnt like the shell pipe "|"
was trying this..but still no luck.....I think, we are getting closer with you help...we can close it.

p = Popen(['aws', 'ec2', 'describe-instances', '--instance-ids', '$(aws', 'ec2', 'describe-instances', '--filters', 'Name=tag:Name,Values=testjumpbox', '--query', 'Reservations[].Instances[].[InstanceId]', '--output', 'text' ')'])
>>> usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:

  aws help
  aws <command> help
  aws <command> <subcommand> help
aws: error: argument --output: Invalid choice, valid choices are:

json                                     | text
table


Invalid choice: 'text)', maybe you meant:

  * text

Open in new window

0
NorieAnalyst Assistant Commented:
Couldn't you put the code you want to execute in a script file and then call/run that using Popen?

p = Popen(['myscript.bsh', 'shell=true'])

Open in new window

0
Defend Against the Q2 Top Security Threats

Were you aware that overall malware worldwide was down a surprising 42% from Q1'18? Every quarter, the WatchGuard Threat Lab releases an Internet Security Report that analyzes the top threat trends impacting companies worldwide. Learn more by viewing our on-demand webinar today!

enthuguyAuthor Commented:
Good idea nonie,

But trying to avoid shell :)
0
NorieAnalyst Assistant Commented:
In your first post you have this, is this exactly how you would run things from the command line?
aws ec2 describe-instances --instance-ids $(aws ec2 describe-instances --filters  "Name=tag:Name,Values=testjumpbox" --query "Reservations[].Instances[].[InstanceId]" --output text | tr '\n' ' ') --output table

Open in new window

PS Have you looked at the awscli package for Python?
0
NorieAnalyst Assistant Commented:
Does this work?
command = 'aws ec2 describe-instances --instance-ids $(aws ec2 describe-instances --filters  "Name=tag:Name,Values=testjumpbox" --query "Reservations[].Instances[].[InstanceId]" --output text | tr \'\\n\' \' \' --output table'

p = Popen(command, universal_newlines=True, shell=true)

Open in new window

1

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
enthuguyAuthor Commented:
Thx noire, will give a try
0
enthuguyAuthor Commented:
# Tried to debug using Python terminal by splitting the execution into two (1. get an instance id by name, then describe instance by an instance id)
# Step 1: Get Instance Id
p = Popen(['aws', 'ec2', 'describe-instances', '--filters', 'Name=tag:Name,Values=testjumpbox', '--query', 'Reservations[].Instances[].[InstanceId]', '--output', 'text'])
>>> i-03043e0d99c624abe

Open in new window


# Tried to store the instance id into a variable, then use it in the next aws Popen cmd, but didn't work. It printed the actual cmd :)
>>> gInstanceId = (['aws', 'ec2', 'describe-instances', '--filters', 'Name=tag:Name,Values=testjumpbox', '--query', 'Reservations[].Instances[].[InstanceId]', '--output', 'text'])
>>> print (gInstanceId)
['aws', 'ec2', 'describe-instances', '--filters', 'Name=tag:Name,Values=testjumpbox', '--query', 'Reservations[].Instances[].[InstanceId]', '--output', 'text']

Open in new window


>>> gInstanceId = ['aws', 'ec2', 'describe-instances', '--filters', 'Name=tag:Name,Values=testjumpbox', '--query', 'Reservations[].Instances[].[InstanceId]', '--output', 'text']
>>> print (gInstanceId)
['aws', 'ec2', 'describe-instances', '--filters', 'Name=tag:Name,Values=testjumpbox', '--query', 'Reservations[].Instances[].[InstanceId]', '--output', 'text']

Open in new window

0
enthuguyAuthor Commented:
Quick update, slighly better situation :)

>>> import configparser, argparse, subprocess
>>> from subprocess import Popen, PIPE, check_output
>>> out = check_output (['aws', 'ec2', 'describe-instances', '--filters', 'Name=tag:Name,Values=testjumpbox', '--query', 'Reservations[].Instances[].[InstanceId]', '--output', 'text'])
>>> print(out)
b'i-03043e0d99c624abe\n'

Open in new window


Could you help me how to...
  1. Remove leading 'b' and '\n'
  2. Validate if it is not blank/null
0
enthuguyAuthor Commented:
Current value
>>> print(out)
b'i-03043e0d99c624abe\n'

Strip removes trailing newline, but couldn't remove leading 'b' char   :)
>>> print(out.strip())
b'i-03043e0d99c624abe'

Open in new window


one more to go :)
0
NorieAnalyst Assistant Commented:
Perhaps.
print(out.strip()[1:])

Open in new window


PS Did this not work?
command = 'aws ec2 describe-instances --instance-ids $(aws ec2 describe-instances --filters  "Name=tag:Name,Values=testjumpbox" --query "Reservations[].Instances[].[InstanceId]" --output text | tr \'\\n\' \' \' --output table'

p = Popen(command, universal_newlines=True, shell=true)

Open in new window

0
enthuguyAuthor Commented:
Thanks a lot Norie, you are a start.
Your solution is actually much easier to code/read as well :)

Uppercase T in shell=True
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
python3

From novice to tech pro — start learning today.