shell to python

Hello,

i started new for python, i would like know the equivalent python code for below shell script.

#!/bin/bash
data=?fields=attachment
while read n
do
        curl -u admin:admin@123 -X GET https://jira-dev.abc.com/rest/api/2/issue/$n$data | /usr/bin/jq '.id' >> isad.csv

done < filename

where filename contains
SEC-1
SEC-2
SEC-3

regards
Lingesh
lingesh bnAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

arnoldCommented:
Looks as though you are retrieving SEC fillings
and parsing them.

Have you attempted to create the logic to achieve the same ?

break down what it is doing.
there are three items it wants, it then uses jq to process it.

AS your script illustrates, it is a restricted site.
My suggestion break down the process.
1) retrieve all documents.
2) pass each document for data extraction
3) write out the information you are after
arnoldCommented:
the following should work:
#!/bin/bash


if [ $# -ne 2 ]
then
        echo " Wrong number of arguments. Usage: scriptname number1 number2"
        exit 1
fi


add()
{
        let "sum= $1 + $2 "
}

add $1 $2
echo " The Sum of $1 + $2 is : $sum"

Open in new window


https://www.linuxjournal.com/content/return-values-bash-functions
dfkeCommented:
Hi Arnold,

I think you posted an answer to another question :)

In python it could be something like:

#!/usr/bin/env python

import sys,os,subprocess,glob
class Getsec(object):
  __slots__ = ["val"]
  def __init__(self, value=''):
    self.val = value

def Glob(value):
  ret = glob.glob(value)
  if (len(ret) < 1):
    ret = [ value ]
  return ret

data=Getsec(Glob("?fields=attachment"))
< filenamewhile (n = Sec(raw_input())):
    _rcr1, _rcw1 = os.pipe()
    if os.fork():
        os.close(_rcw1)
        os.dup2(_rcr1, 0)
        subprocess.call("/usr/bin/jq" + " " + ".id",shell=True,stdout=file("isad.csv",'ab'))
        >> isad.csv
    else:
        os.close(_rcr1)
        os.dup2(_rcw1, 1)
        subprocess.call(["curl","-u","admin:admin@123","-X","GET","https://jira-dev.abc.com/rest/api/2/issue/"+str(n.val)+str(data.val)],shell=True)
        sys.exit(0)
 < filename

Open in new window

Cheers
5 Ways Acronis Skyrockets Your Data Protection

Risks to data security are risks to business continuity. Businesses need to know what these risks look like – and where they can turn for help.
Check our newest E-Book and learn how you can differentiate your data protection business with advanced cloud solutions Acronis delivers

nociSoftware EngineerCommented:
#!/usr/bin/env python

import sys,os,subprocess

data="?fields=attachment"   # note quotes are missing in the bash source...

# python-2: raw_input()
# python 3: input()

try:
    isad = open('isad.csv','ab')
    n = raw_input()
    while n:
        subprocess.call("curl -u admin:admin@123 -X GET https://jira-dev.abc.com/rest/api/2/issue/{}{} | /usr/bin/jq '.id'".format(n,data) ,shell=True,stdout=isad)
        n = raw_input()
except EOFError:
    isad.close()

Open in new window


You probably could split out the subprocess.call further input separate processes while joining their pipeline.  (Popen()  method).
arnoldCommented:
Yes, had two questions open, and did not check to which ....
lingesh bnAuthor Commented:
@noci where i can give input, here my input is filename which contains names like below
SEC-1
SEC-2
SEC-3

regards
Lingesh
nociSoftware EngineerCommented:
Input is read fron STDIN..

So running the python script:

python above-script.py <filename
arnoldCommented:
You could use it the same way you had, open a file and read in a line at a time
n instead of raw will read from file.

Not sure whether the change Noci provided is what was asked for implied.

My guess is the request was to achieve the same thing the bash script was doing in native Python.
I.e. Open file of data, open file for writing
Use Python native outbound connection to authenticate/request the data, process the data, while writing out the results.
Go through the requested files.....

I.e. If you run
date && bash bashscript.sh && date
Then rerun the Noci provided Python alternative in the same way
See which performs the task faster.

The difference in timing will reflect overhead and possible change in transport of data ....

If needed, averaging by running the same set a few times.

Do not know Python, but if the request was to change to perl, using socket, net::http etc. the script would initiate and establish a connection to the remote server, request data, iterate through the data extracting what is of interest. To you while writing it out.
lingesh bnAuthor Commented:
@noci getting below error.
0parse error: Invalid numeric literal at line 12, column 0
curl: (23) Failed writing body (4096 != 7192)

regards
Lingesh
nociSoftware EngineerCommented:
Looks like jq stopped reading after the first 4K of data has been sent to the ...
Not sure were the parse error comes from, is it possible that it is from the jq command?
can you provide example data?

jq could be this one:  https://stedolan.github.com/jq/
in that case python might have a library onboard to handle this even better.
Python also has bindings to access libcurll directly.   but it all depends on what you need to do.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Linux

From novice to tech pro — start learning today.