Korn shell how to read external file and pass parameters to another file

@tancat,

1. Replaced:
/bin/ksh -x
With:
/bin/ksh
you can replace if you need debug options.

2. Before using a file verify the nonsexist of that file.
3. Use related path or full path for using any file inside a script or any command( unless related alias or related function is available)
4. Replaced:
start_date='20170703'
stop_date='20170704'
With:
start_date=dd/mmm/yyyy (today date)
stop_date==dd/mmm/yyyy (tomorrow date)

test_main_script.ksh

#!/bin/ksh
#main script
if test ! -f ./test.txt
then
	echo "./test.txt No such file"
	echo "/bin/ls -ld ./test.txt"
	/bin/ls -ld "./test.txt"
else
	while read -r a b
	do
		if test ! -f ./env.vars
		then
			echo "./env.vars No such file"
			echo "/bin/ls -ld ./env.vars"
			/bin/ls -ld "./env.vars"
			break
		fi
		. ./env.vars $a $b
		echo start_date=$start_date
		echo stop_date=$stop_date
		echo row_id=$row_id
		echo row_name=$row_name
		echo 'This is the main script'
	done <./test.txt
fi

Select allOpen in new window

env.vars

#!/bin/ksh
start_date=''`/bin/date "+%d/%b/%Y" | /usr/bin/tr -d "\n"`''
stop_date=''`/bin/date "+%d/%b/%Y" --date="-1day ago"| /usr/bin/tr -d "\n"`''
row_id=$1
row_name=$2

Select allOpen in new window

Like the same you can handle further exceptions also if script need more requirement / handling exceptions at all OS.

I like following default option from /bin/ksh
set -o vi
which used to help me a lot while searching history of previously typed commands.
instead of default option from /bin/bash
set -o emacs
(Of course we change that in /bin/bash if we need that).

Hi Murugesan,

Although using this:
    . ./env.vars $a $b
instead of my (slightly shorter but potentially dangerous) suggestion of this:
    . env.vars $a $b
is safer, because it will avoid any problems of any other copies of env.vars being picked up from other directories in the user's $PATH, what is the point of prefixing filenames with "./" in cases like these?:
    if test ! -f ./test.txt
    echo "./test.txt No such file"
    /bin/ls -ld "./test.txt"
    if test ! -f ./env.vars
    echo "./env.vars No such file"
    echo "/bin/ls -ld ./env.vars"
    /bin/ls -ld "./env.vars"

And using lines like this:
    echo "/bin/ls -ld ./env.vars"
are presumably your way of showing what's going on for debugging purposes.  Adding such debug lines can be avoided by using the '-x' switch in the first line, which you have suggested be removed.  I agree it should be removed, but maybe tancat was going to do that after debugging was complete.

And what is the point in the pairs of single quotes at each end of this line?:
    start_date=''`/bin/date "+%d/%b/%Y" | /usr/bin/tr -d "\n"`''
Why don't you remove those single quotes completely:
    start_date=`/bin/date "+%d/%b/%Y" | /usr/bin/tr -d "\n"`
And what's the point of using tr to remove the newline char?  This gives the same result:
    start_date=`/bin/date "+%d/%b/%Y"`
And you could even remove the double quotes:
    start_date=`/bin/date +%d/%b/%Y`
and since %d/%b/%Y produces dates like this:
    03/Jul/2017
but tancat seems to want dates like this:
    20170703
why don't you have something like this:
    start_date=`/bin/date +%Y%m%d`
but all this may be inappropriate if the dates tancat requires are not today and yesterday.  That was a significant assumption, especially since the dates tancat's provided are from almost month ago.

And the '--date' switch you used here:
    stop_date=''`/bin/date "+%d/%b/%Y" --date="-1day ago"| /usr/bin/tr -d "\n"`''
could be useful, but it won't work with some versions of the date command which don't have that switch.

Hi tancat,

Exceptions like:
1. hacker (Example /usr/bin/sudo /usr/sbin/lsof ....) can do any action
2. Sample exceptions:
          executing script . fullpath/script and not allowing that script when executed inside other script.....
          allowing number of times that script being executed by particular user or by group of users....or by all
          allowing script execution last modified date
          allowing script executing using RSA keys /usr/bin/ssh-keygen (all exceptions inside that)
          not executing that script when same script running at remote host particular number of hosts....
          Exceptions given by tel2 comment
          ...like that number of exceptions occurs,

Hence mentioned the following
>> Like the same you can handle further exceptions also if script need more requirement / handling exceptions at all OS (
AIX
HP-UX
Linux
SunOS
Windows CYGWIN_NT or mingw
...all sub versions all OS.)

$ $ echo "AIX HP-UX Linux SunOS Windows" | /usr/bin/tr " " "\n" | /bin/sort -u
AIX
HP-UX
Linux
SunOS
Windows

Hi Murugesan,

I see your latest post, but what are your answers to my questions?  I'd like to know why you wrote your code the way you did, in the ways I mentioned.  This might be helpful to tancat, too.

Thanks.
tel2

Hi tancat/tel2,

Reason for writing the script:
1. stabilize the script so that the development needs to be finalized => means => that needs to be written or updated only in case of new requirement.
Hence helping to stabilized release of any script.

Even though used the word "script" that is pointing to any source code( COBOL/java/C++/C/python/script/.cmd/.vbs/...)

$ echo "tancat/tel2" | /usr/bin/tr "/" "\n" | /bin/sort -u
tancat
tel2

Select allOpen in new window

$ echo  -n ".cmd/.vbs/C/C++/COBOL/java/python/script/..." | /usr/bin/tr "/" "\n" | /bin/sort -u
...
.cmd
.vbs
C
C++
COBOL
java
python
script

Select allOpen in new window

Also write the code so that when you are not in that project your name will be there forever until that binary(internally your code) is there.
This does not mean

$ /bin/cat binaryname 2>/dev/null | /bin/egrep YourName
$ echo $?
1
$

Select allOpen in new window

Always use full path when executing any commands either at terminal or inside scripts(/code).

Hi Murugesan,

Thanks for your response, but I don't see how they answer the specific questions I asked.  In my post, I wrote:
   "I see your latest post, but what are your answers to my questions?  I'd like to know why you wrote your code the way you did, in the ways [or areas] I mentioned."
If you scroll up to my post #42276390, you'll see my specific questions.  For example, the first one is:
   what is the point of prefixing filenames with "./" in cases like these?:
        if test ! -f ./test.txt
        echo "./test.txt No such file"
        /bin/ls -ld "./test.txt"
        if test ! -f ./env.vars
        echo "./env.vars No such file"
        echo "/bin/ls -ld ./env.vars"
        /bin/ls -ld "./env.vars"
Your response of:
    "Always use full path when executing any commands either at terminal or inside scripts(/code)."
is not relevant to this question.  My question relates to 'prefixing filenames with "./" ' (i.e. dot slash).  None of the above cases involve executing the filenames (or scripts) which are prefixed with dot slash (see the underlined filenames, above).

And there are several other questions in that post for you.

Thanks.
tel2

Hi tancat/tel2,

Reason for using:
      . ./env.vars
      This does not mean use ./
      Use related path instead of env.vars
      Since that is dependent on PATH environment variable at all OS.
      PATH environment variable itself can be changed using multiple ways.
      One sample way:
            C:\Windows\System32\rundll32.exe sysdm.cpl,EditEnvironmentVariables
      Hence take care of using related path inside or outside the script.
Added echo "/bin/ls -ld ./env.vars"
      does not mean that it is required in that script.
      It was a sample to display the client/developer related output when the script is executing
      You can take the time taken by the script and you can inform the client the reason for the same.
      Not only the time you need to take care of required output using required commands.
      -x displays everything(this is required when we are not able to find where the error condition occurs)
For single quote usage I cannot remember the exception happened during 2008 (for HP-UX, Linux and SunOS related Makefile issues).
This does not mean Makefile alone, that is applicable for
      1. script
      2. script using other script
      3. ssh to remote system
      4. ssh required commands to remote system
      5. make using remote system using ssh
      quote descriptions here:
      https://superuser.com/questions/874562/difference-when-using-backticks-in-double-quotes-and-single-quotes-in-bash
      6. /usr/bin/wget RSA public file from apache url and that public file having regular expression characters(including special characters like back tick)
      , faced issue using "`command`"
>> some versions of the date command which don't have that switch
      Hence mentioned handle all exceptions at all OS
      OS=''`/bin/uname -s | /bin/sed "s/\-.*//;" | /usr/bin/tr -d "\n"`''
      if test "relatedOS" = "$OS"
      then
            related command(example /bin/date.... and related parameter including nano seconds at required OS)
      elif test "relatedOS" = "$OS"
      then
            related command(example /bin/date....)
      else
            echo related error and handle $? here too.
      fi

Thanks for your answers, Murugesan.

Some things I still don't understand include:

Looking at this code of yours:
   start_date=''`/bin/date "+%d/%b/%Y" | /usr/bin/tr -d "\n"`''
Why did you include '| /usr/bin/tr -d "\n"' to remove the newline char?
Why didn't you just put:
    start_date=''`/bin/date "+%d/%b/%Y"`''
I get the same results.
   
Also, in that same line, why did you use the date format of:
   +%d/%b/%Y
which returns dates like:
    03/Jul/2017
but tancat seems to want dates like this:
    20170703
so why don't you have something like this use the format:
    +%Y%m%d

Thanks.
tel2

Hi tancat,

That does not mean remove new line for that /bin/date command.
When you assign any command output to a variable handle that exception also.
Hence mentioned handle exceptions.

Use +%d/%b/%Y or +.... as per the requirement including nano seconds or seconds as per OS.

Hi Murugesan,

> That does not mean remove new line for that /bin/date command.
> When you assign any command output to a variable handle that exception also.
> Hence mentioned handle exceptions.
I don't understand how that answers my question.  Here's your command again:
   start_date=''`/bin/date "+%d/%b/%Y" | /usr/bin/tr -d "\n"`''
What exceptions is the '| /usr/bin/tr -d "\n"' part handling?

> Use +%d/%b/%Y or +.... as per the requirement including nano seconds or seconds as per OS.
The requirement that was shown in the original post was:
    start_date='20170703'
    stop_date='20170704'
So my point is, +%d/%b/%Y will not do that, but +%Y%m%d will do that, right?

Thanks.
tel2

Need comment/query from tancat if you cannot understand any commands or statements mentioned here.

Why?

...

Hi Murugesan,

In response to the response you just posted but then cleared.

>>> "That does not mean remove new line for that /bin/date command."
>> What exceptions is the '| /usr/bin/tr -d "\n"' part handling?
> When you assign any command output to a variable handle that exception also by removing new lines, regular expression characters or special characters, ...(example SAP)

First you said "That does not mean remove new line for that /bin/date command".
Now you seem to be saying the reason for it is to "handle that exception also by removing new lines...".
Q1. So, did you add the '| /usr/bin/tr -d "\n"' part to remove a newline or not?

Q2. How could the output of the /bin/date "+%d/%b/%Y" command contain any of those types of characters (newlines, other special characters, etc)?

Q3. How could the '| /usr/bin/tr -d "\n"' remove "regular expression characters or special characters, ...(example SAP)"?

Q4. What has the SAP example, got to do with the output of the date command?

> round robin algorithm
Q5. What has the round robin algorithm got to do with any of this?

Q6. Why do you write this:
      if test ! -f ./test.txt
instead of this?:
      if test ! -f test.txt

Please answer questions Q1 - Q6.

Thanks.
tel2

Q1
      Yes
Q2
      It was not related to this query and not to /bin/date
      Just KT while assiging output of a command to variable.
      Better handle that exceptions while using commands providing new line errors
      Not related to 29053617, only KT
Q3
      Like /usr/bin/tr -d "\n"
      handle the same thing for handing special characters also instead of "\n"
Q4
      out of scope for this query 29053617. Remove all of my comments having the string SAP.
Q5

@ECHO OFF
:TENCAT
echo TENCAT query
echo TEL2 comment
echo MY comment
:ROUND_ROBIN_ALGORITHM
:TEL2
echo TEL2 query
echo MY answer
echo Do not go to TENCAT
GOTO ROUND_ROBIN_ALGORITHM

Select allOpen in new window

This does not mean you alone me too :)
Q6
It was posted
>> Use related path instead of env.vars
This does not mean only for env.vars, but for all files being used for any commands.
>> but for all files
each of them(socket/pipe/directory,..) in all OS are treated as files.

myrequest not to use please/kindly :)

Hey guys - sorry, I had the flu yesterday, and nothing I tried to do made any sense (fever, dehydrated), so I gave up and will try again this morning.

Obviously this is not real code, this is simplified to focus just on the part that I'm trying to figure out.

We have been strongly encouraged by our admin to use ksh, and he seems adverse to using modern tools, so bash being more modern makes perfect sense.  :)

I will let you know how things progress.

Thanks!

@tancat,
Yes, I expected it was probably a cut-down version of your code.
I wouldn't exactly call bash modern.  It's "more modern" than ksh, but bash has been around for over 10 years and from what I've seen is the most popular UNIX/Linux scripting shell these days.  It's hardly the leading-bleading edge of technology.  It's very stable from what I've seen.  And the fact (?) that you can run most ksh scripts under bash (I think) means transition is easy, and then you've got the extra features, so you can do things like this:
    sub=${orig:2}     # Return chars 3 to end (i.e. remove 1st 2 chars)
    sub=${orig:2:3}   # Return chars 3-5 (char 2 is the 3rd char since this is 0 based, and we're taking 3 chars)
for putting a substrings of $orig into $sub, instead having to do messy looking calls to commands like 'echo' and 'cut', e.g.:
    sub=`echo $orig | cut -c3-`
    sub=`echo $orig | cut -c3-5`
But I think ksh has its advantages, too.
Did your admin explain why (s)he doesn't want people using bash?  If it's for consistency with existing scripts, then I guess that has merit.

By the way, are you aware that instead of editing the script to add '-x' to the 'shebang' line (i.e. the first line of most scripts, which starts with '#!') while debugging, then taking it out again later, you can just execute the script like this:
    ksh -x test_main_script.sh
But note that:
- If that script calls another script, it will not be run in debug mode (unless you change the call syntax, I guess),
- The above method will run the script even if it doesn't have execute permission or a shebang line.

Re this:
    echo start_date=$start_date
I wasn't thinking - I would usually put that in quotes:
    echo "start_date=$start_date"
because there are various characters that you might add to such a line which would either cause a syntax error or make it not work as expected (e.g. multiple spaces in a row become one).

And the reason I prefer this:
    echo start_date=$start_date
over this:
    echo start_date=${start_date}
is, it's shorter, easier to read (in my view), and it's not often that you need the braces.  You need them where you need to clearly delimit the variable name, so the shell knows where it ends, e.g.:
    echo "ABC$start_date123"
will not print $start_date because it will assume you are referring to the variable $start_date123.  But if you want to be totally consistent, then you could use them all the time.  But I don't even see consistent people writing things like:
    row_id=${1}
but that might be because such parameters are automatically limited to a single digit (i.e. $10 is not the 10th parameter, in ksh at least).

Are we far enough off topic yet?

In case you missed it above, if you decide to use my code, or any part of it, I suggest you change this line:
    . env.vars $a $b    # Note the dot space prefix.
to this:
    . ./env.vars $a $b  # Note the dot space dot slash prefix.

>> We have been strongly encouraged by our admin to use ksh
agreed, I like ksh for 14 years, but using mksh.exe at windows => this used to change based on requirement.....
>> makes perfect sense.
yes :))
2. Take care of your health
1. and give task(script $!)
helping your career $?

echo "please/kindly/sorry" | /bin/sed "s/please//;"

$ echo -n "Please Kindly Sorry" | /bin/sed "s/[P|p][l|L][e|E][a|A][s|S][e|E]//;s/[K|k][I|i][N|n][D|d][L|l][Y|y]//;s/Sor
ry/:)/;";echo " $?"
  :) 0
$

Select allOpen in new window

Hi Murugesan,

I doubt that tancat, needs any of this date stuff at this stage, but I've been trying to get you to explain why it is necessary to have this:
    start_date=''`/bin/date "+%d/%b/%Y" | /usr/bin/tr -d "\n"`''
instead of something like this:
    start_date=''`/bin/date "+%d/%b/%Y"`''
and you've spoken of removing the newline, etc, but none of your explanations about this sound reasonable to me.  So, I think the ultimate test is, comparing the results of both commands.
From ksh I run these commands:
    $ start_date1=''`/bin/date "+%d/%b/%Y" | /usr/bin/tr -d "\n"`''
    $ start_date2=''`/bin/date "+%d/%b/%Y"`''
    $ start_date3=`/bin/date +%d/%b/%Y`
    $ echo "-$start_date1-$start_date2-$start_date3-"
    -01/Sep/2017-01/Sep/2017-01/Sep/2017-
    $ echo -n "-$start_date1-$start_date2-$start_date3-" | od -bc
    0000000  055 060 061 057 123 145 160 057 062 060 061 067 055 060 061 057
               -   0   1   /   S   e   p   /   2   0   1   7   -   0   1   /
    0000020  123 145 160 057 062 060 061 067 055 060 061 057 123 145 160 057
               S   e   p   /   2   0   1   7   -   0   1   /   S   e   p   /
    0000040  062 060 061 067 055
               2   0   1   7   -
    0000045
And if that's not enough, let's test for equality:
    $ if test $start_date1 = $start_date2;then echo Same;fi
    Same
    $ if test $start_date1 = $start_date3;then echo Same;fi
    Same

Q7. So, do you now agree that the '| /usr/bin/tr -d "\n"' is unnecessary in this situation?
Q8. If not, what is your evidence for not agreeing?

Q9. What do you mean by "KT"?

Thanks.
tel2

Q7. So, do you now agree that the '| /usr/bin/tr -d "\n"' is unnecessary in this situation?
Q2. How could the output of the /bin/date "+%d/%b/%Y" command contain any of those types of characters (newlines, other special characters, etc)?
Hence Q8 (else case not executed)
Q9 Knowledge transfer
Sample http://www.google.com/ncr:
https://www.google.com/search?source=hp&q=KT+stands+for&oq=KT+stands+for&gs_l=psy-ab.3..0j0i22i30k1l3.1448.3231.0.3503.14.11.0.0.0.0.211.1151.0j4j2.6.0....0...1.1.64.psy-ab..8.6.1149.6..35i39k1j0i20k1j0i131i46k1j46i131k1j0i67k1j0i22i10i30k1.SD7KOKUz4Ng
KT - What does KT stand for? The Free Dictionary
acronyms.thefreedictionary.com/KT
Knowledge Transfer. KT. Carat. KT. Katie (girl's name)

@tel2 - it turns out that I did just need to replace the / with a space - so this works:

. env.vars $a $b

Select allOpen in new window

I thought I had tried that, but I was probably already getting sick and wasn't thinking straight.  

As a monkey in training, as far as the shell scripts go, I do a lot of copy and paste.  I don't know why we use 'export' except that the shell scripts are often called by Active Batch, so I assumed that the 'export parameter' statement was a variation of $1, $2 as far as accepting values.  I prefer simpler code, so I may try to remove those to see what happens with Active Batch.  

I also sometimes see ${variable_name} and other times $variable_name but I haven't figured out why, although it's likely that I've copied code that was copied by someone else, that was copied by someone else, and none of us really know exactly what we are doing so there's no consistency at all.  

My admin is an older gentleman whose answers to my questions are usually technically correct but many times useless.  "Don't do that because xyz."  But he doesn't often share with me what I should do instead.  So when he says, "You should use korn shell" I don't argue.  

I very much appreciate the additional information about the shell commands.  

Thank you!

Thank you!!

@tancat, thanks for the points.
Simply having:
  . env.vars $a $b
instead of:
  . ./env.vars $a $b
will work as long as the env.vars script is in your path (or current directory, I think).  But if you specify the full path to it, then that will make sure you pick up the right version of it, in case there could be more than one.

I trust you read my reason for (sometimes) needing ${variable_name} instead of just $variable_name.  It may not be the only reason.


@Murugesan,
Thank you for finally admitting (?) that '| /usr/bin/tr -d "\n"' is unnecessary in this situation.  It took a lot of work to get to that point, and I'm not sure why.  At least I think you've admitted to it, but as with many of your posts, I found it hard to be sure of what you were trying to say, because of your cryptic way of saying things.  Communications are restricted enough by the fact that we are doing everything in writing, and have different ways of speaking English, without adding cryptic comments like many of those I've asked you to clarify.  I guess this may take some of the fun out of it for you, but on the other side, more people may understand you, and if the asker understands you then you may get more points.  So straight forward answers like "Yes" or "Yes, sorry, I was mistaken" in response to Q7 would be clearest, because not all of us think like you do.

On the other hand, an answer like "Yes" to questions like:
    "Q1. So, did you add the '| /usr/bin/tr -d "\n"' part to remove a newline or not?"
are not very clear, because the answer should be one or the other, and "Yes" doesn't state which one it should be.

I am aware of how to use Google to search for things like KT, and I did that before asking you, but none of the results on the first page included "Knowledge Transfer", so I asked you what it meant.  But it's best to make such things clear the first time you use such acronyms, so readers don't need to waste time Googling.

tel2

Hi tancat/tel2,

Thank you for providing my expectation and related comments.

>> because of your cryptic way of saying things
since I like RSA :) /usr/bin/ssh-keygen

>> you may get more points
just writing here =>
for my fun,
using/providing
KT/my time pass or /bin/sleep 18 or #Comments :)

Regards,
Murugesandins