shell to python

lingesh bn
lingesh bn used Ask the Experts™
on
Hello,

i started new for python, i would like know the equivalent python code for below shell script.

#!/bin/bash
data=?fields=attachment
while read n
do
        curl -u admin:admin@123 -X GET https://jira-dev.abc.com/rest/api/2/issue/$n$data | /usr/bin/jq '.id' >> isad.csv

done < filename

where filename contains
SEC-1
SEC-2
SEC-3

regards
Lingesh
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Distinguished Expert 2017

Commented:
Looks as though you are retrieving SEC fillings
and parsing them.

Have you attempted to create the logic to achieve the same ?

break down what it is doing.
there are three items it wants, it then uses jq to process it.

AS your script illustrates, it is a restricted site.
My suggestion break down the process.
1) retrieve all documents.
2) pass each document for data extraction
3) write out the information you are after
Distinguished Expert 2017

Commented:
the following should work:
#!/bin/bash


if [ $# -ne 2 ]
then
        echo " Wrong number of arguments. Usage: scriptname number1 number2"
        exit 1
fi


add()
{
        let "sum= $1 + $2 "
}

add $1 $2
echo " The Sum of $1 + $2 is : $sum"

Open in new window


https://www.linuxjournal.com/content/return-values-bash-functions
Hi Arnold,

I think you posted an answer to another question :)

In python it could be something like:

#!/usr/bin/env python

import sys,os,subprocess,glob
class Getsec(object):
  __slots__ = ["val"]
  def __init__(self, value=''):
    self.val = value

def Glob(value):
  ret = glob.glob(value)
  if (len(ret) < 1):
    ret = [ value ]
  return ret

data=Getsec(Glob("?fields=attachment"))
< filenamewhile (n = Sec(raw_input())):
    _rcr1, _rcw1 = os.pipe()
    if os.fork():
        os.close(_rcw1)
        os.dup2(_rcr1, 0)
        subprocess.call("/usr/bin/jq" + " " + ".id",shell=True,stdout=file("isad.csv",'ab'))
        >> isad.csv
    else:
        os.close(_rcr1)
        os.dup2(_rcw1, 1)
        subprocess.call(["curl","-u","admin:admin@123","-X","GET","https://jira-dev.abc.com/rest/api/2/issue/"+str(n.val)+str(data.val)],shell=True)
        sys.exit(0)
 < filename

Open in new window

Cheers
Ensure you’re charging the right price for your IT

Do you wonder if your IT business is truly profitable or if you should raise your prices? Learn how to calculate your overhead burden using our free interactive tool and use it to determine the right price for your IT services. Start calculating Now!

nociSoftware Engineer
Distinguished Expert 2018

Commented:
#!/usr/bin/env python

import sys,os,subprocess

data="?fields=attachment"   # note quotes are missing in the bash source...

# python-2: raw_input()
# python 3: input()

try:
    isad = open('isad.csv','ab')
    n = raw_input()
    while n:
        subprocess.call("curl -u admin:admin@123 -X GET https://jira-dev.abc.com/rest/api/2/issue/{}{} | /usr/bin/jq '.id'".format(n,data) ,shell=True,stdout=isad)
        n = raw_input()
except EOFError:
    isad.close()

Open in new window


You probably could split out the subprocess.call further input separate processes while joining their pipeline.  (Popen()  method).
Distinguished Expert 2017

Commented:
Yes, had two questions open, and did not check to which ....

Author

Commented:
@noci where i can give input, here my input is filename which contains names like below
SEC-1
SEC-2
SEC-3

regards
Lingesh
nociSoftware Engineer
Distinguished Expert 2018

Commented:
Input is read fron STDIN..

So running the python script:

python above-script.py <filename
Distinguished Expert 2017

Commented:
You could use it the same way you had, open a file and read in a line at a time
n instead of raw will read from file.

Not sure whether the change Noci provided is what was asked for implied.

My guess is the request was to achieve the same thing the bash script was doing in native Python.
I.e. Open file of data, open file for writing
Use Python native outbound connection to authenticate/request the data, process the data, while writing out the results.
Go through the requested files.....

I.e. If you run
date && bash bashscript.sh && date
Then rerun the Noci provided Python alternative in the same way
See which performs the task faster.

The difference in timing will reflect overhead and possible change in transport of data ....

If needed, averaging by running the same set a few times.

Do not know Python, but if the request was to change to perl, using socket, net::http etc. the script would initiate and establish a connection to the remote server, request data, iterate through the data extracting what is of interest. To you while writing it out.

Author

Commented:
@noci getting below error.
0parse error: Invalid numeric literal at line 12, column 0
curl: (23) Failed writing body (4096 != 7192)

regards
Lingesh
nociSoftware Engineer
Distinguished Expert 2018

Commented:
Looks like jq stopped reading after the first 4K of data has been sent to the ...
Not sure were the parse error comes from, is it possible that it is from the jq command?
can you provide example data?

jq could be this one:  https://stedolan.github.com/jq/
in that case python might have a library onboard to handle this even better.
Python also has bindings to access libcurll directly.   but it all depends on what you need to do.

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial