Optimize bash script

We have a config file using a standard INI-style format:
[globals]
publicint.ssh.host = myhost
publicint.ssh.user = myuser
# ... etc

Open in new window

We have a bash script named readConfig.sh that knows how to read that file:
[me@MJZFCAP:/cygdrive/c/bb/www/bluebird/scripts]
$> ./readConfig.sh --group globals publicint.ssh.host
myserv

Open in new window

I am making a script which will take a series of parameters (e.g., ./mynewscript.sh --ssh_host=myotherserv --ssh_user=myotheruser).  Note that the parameter names in the config file are equivalent to ${bash_parameter_name//_/.}, with a prefix of "publicint."  These parameters all have default values defined in the script.  If a parameter has a value assigned in the config file, the config file takes precedence.  If a parameter is passed on the command line, it will take precedence over the default and config file.

In my bash script, I created this function to implement that concept.  The function takes a single parameter - the name of the variable to populate.  Is there a better or more efficient way to do this?

script_dir=`dirname $0`
script_dir=`cd $script_dir; echo $PWD`
readConfig=$script_dir/readConfig.sh
readAlias=$readConfig" --group globals publicint."

# set all script defaults
default_ssh_host=myserver
default_ssh_user=myuser
default_tunnel_port=7777
default_socket_file=bbintegrator

# function to set a parameter variable to the proper cascaded value
# cascade, in order of greatest precedence, is (command line)->(config file)->(script default)
set_param() {
  # expects one parameter - the name of the variable to populate
  # if the variable is empty, try reading it from the config file
  if [[ -z "${!1}" ]]; then
    tmpvar=${i//_/.}
    #echo $1" is blank, reading config with "$tmpvar
    eval "$1=`${readAlias}${tmpvar}`"
    #echo "i = "$1", val i = "${!1}
  fi
  # if the variable is STILL empty, set to default value
  if [[ -z "${!1}" ]]; then
    #echo "value still blank, using default"
    eval "$1=\$default_"$1
  fi
  #echo "final value i = "$1", val i = "${!1}
}

# set up the variables to use command line, OR config values, OR default values
for i in ssh_host ssh_user tunnel_post socket_file; do
  set_param $i
  echo "final value i = "$i", val i = "${!i}
done

Open in new window

LVL 52
Steve BinkAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

ozoCommented:
does ./readConfig.sh have any other options?
0
Steve BinkAuthor Commented:
Several, but they aren't terribly relevant to what I'm trying to do here...  In the context of the other available options, the script I am making will be a "global" script covering all instances found in the config file.  The key/value pairs I am searching for will always be in the globals section of the config.

https://github.com/nysenate/Bluebird-CRM/blob/dev/scripts/readConfig.sh
0
wilcoxonCommented:
Does it have to be a bash script?  This sort of script would be trivial in perl.  It likely would not be more efficient since it would have to invoke the perl interpreter in order to run but, if in perl, you could easily avoid shelling out to call readConfig.sh (or replace readConfig.sh with a perl module that could be called from small wrapper readConfig.pl or called directly from things like your script).
0
The 7 Worst Nightmares of a Sysadmin

Fear not! To defend your business’ IT systems we’re going to shine a light on the seven most sinister terrors that haunt sysadmins. That way you can be sure there’s nothing in your stack waiting to go bump in the night.

Steve BinkAuthor Commented:
It does have to be a bash script to fit in with our current platform (nothing else uses perl, and we're working toward homogenization).  

This script works as it is...I just want to know if there is a better way to do it.  My only reason for posting the script here is that I'm trying to learn more about bash scripting.
0
wilcoxonCommented:
I'm certainly no bash expert but I don't know of a more efficient (or more readable way) to do what you want in bash.
0
Steve BinkAuthor Commented:
"Good enough" is acceptable.  I'll wait a bit to see if anyone else has some input.
0
simon3270Commented:
You don't need to use $i in the line which changes "_" to "." - use $1 instead:
    tmpvar=${1//_/.}

That way the function doesn't depend on the loop variable you use.

The code is quite complex (variable redirection like this is not common), so a few comments might save you some head-scratching later on!

Otherwise fine.
0
Steve BinkAuthor Commented:
>>> You don't need to use $i in the line which changes "_" to "." - use $1 instead

That is actually an error in the code.  :)  I originally wrote the function inline, and moved it to be a function just before I posted this question.  I had changed the rest of the references, but guess I missed that one.  Thanks for pointing it out.
0
gheistCommented:
No need to reinvent the wheel:
https://github.com/rudimeier/bash_ini_parser
0
Steve BinkAuthor Commented:
The INI parser we have is already in place, and probably pre-dates that project.  

Also, the INI parser isn't really the question here...this is more about the process I created for resolving a hierarchy of possible configuration assignments.  As I said, I'm still learning bash.  While the script I came up with works, it seems a bit clumsy to me.  That same process in PHP or Python looks much more straight-forward.  

Is there anything I can do to improve the algorithm?
0
gheistCommented:
Use a library. It must be optimal already.
0
Steve BinkAuthor Commented:
That kind of misses the point of my question.
0
simon3270Commented:
I think your solution is pretty minimal, if you want to check variables independently - I'd be happy to have written it!

The only thing I can think to improve it is to pre-read the entire config file and set up an associative array of variables and values
  declare -A valarr

then loop through the ini file, and for each variable $var with value $val do
   valarr["$var"] = "$val"

Note thet "$var" cannot contain dots, so you would have to change all of those to underscores.

Then read from that array in your function rather than call readConfig.sh each time.  That only works if wverything is in  a single ini section, and the variable names are unique.  

So your function could have

    eval $i=\${valarr["publicint_$i"]}

(and you'd miss out the step whcih changes "_" to "." because you actually want the "_"s)
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Steve BinkAuthor Commented:
>>> pre-read the entire config file

I like that idea, mostly because there is a noticeable delay as the script spins up and checks all the variables.  The readConfig.sh script does have a flag to read the entire section, though I only need to check 5 or 6 of the ~70 values available.  Do you still think it is worthwhile?

>>> Note thet "$var" cannot contain dots,

Can you explain more about this requirement?
0
simon3270Commented:
Reading all of the values may make more sense - the time taken to start a shell to load up a script may be quite large, while simply reading a text file can be quick (it is probably all read in a couple of i/o calls), so you are swapping 5 or 6 slow proceses for one possibly less slow one.

As for the dots, that was me confusing myself - variable names cannot contain dots in current bash versions (I understand that they may have been allowed in older versions), so $a.b.c is not a valid variable usage (the shell would replace this with the contents of $a followed by ".b.c") and a.b.c=1 is just not accepted.  However, you can have dots in the array index values, so valarr["a.b.c"] is valid, and if $var contains "a.b.c", valarr["$var"] is allowed.

Assuming that the .ini file entries have dots in them and the parameter to your function has underscores, it is up to you whether you leave the dots in the array indices and convert the underscores to dots as you do now, or convert the dots that youread from the file into underscores before you use them as array indices and so don't ned to convert underscores to dots.
i.e if the .ini file has publicint.ssh.host, you can either:
- store this as valarr["publicint.ssh.host"], then when you pass "ssh_host" into the function, you convert "_" to "." as tmpvar then read valarr["publicint.${tmpvar}"]
- or convert "publicint.ssh.host" to "publicint_ssh_host" when you read it from the .ini file and store the value in valarr["publicint_ssh_host"], then when you pass "ssh_host" as $1 to your function, you retrieve it with valarr["publicint_$1"].
0
Steve BinkAuthor Commented:
You make a lot of sense.  :)  I'll try adapting that tomorrow and let you know how it goes.
0
Steve BinkAuthor Commented:
The idea was shot down during code review as unnecessary.  The sentiment is that the delay introduced by reading ~15 data points individually was not significant enough to justify reading the entire [globals] section.

I did have a PoC worked up, and it did feel quicker (just based on my observation...no benchmarks), but alas...

Anyways, thanks for your input.  We are moving on, so I'll close this out.
0
Steve BinkAuthor Commented:
Thanks much to all participants!
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Shell Scripting

From novice to tech pro — start learning today.