Suggestions for shell script optimization articles and/or software

Posted on 2010-04-05
Medium Priority
Last Modified: 2013-12-26
I have a 2000+ bash script that performs a maintenance activity and enumerates storage devices & reports state.   It needs to run every 10 seconds, and after a few rounds of optimization, I've got it down to 4-8 seconds, depending on what changes between the last and current invocation.  This script runs in an appliance, and compiling to C is not an option (because the company that contracted me wants it as a shell script for readability, and ease-of-maintenance).  This runs on a multi-core system, and CPU overhead is not much of a bottleneck, but as we all know, shell scripts are basically pigs, so I want to be frugal with CPU resources but not anal about using them

So rather than post a section and ask for suggestions, I thought I would open this up for something more useful to the community as a whole.

Q1:  Are there any freeware or open source shell script profiling tools that could assist in finding areas of the code that are greatest problems?  Ideally is there a low-overhead way to create a precision timer and start/stop it as necessary to show elapsed time, and what the holdup was? I.e, for a given chunk of code, can I see if the process performed disk I/O, had any wait states, or context switching, which would then be a candidate for some tweaking?  

Q2: Any good articles, tips, whatever in general on minimizing execution time of shell script?   This particular environment is latest LINUX kernel and I am using bash, so while it will be useful in general, please keep in mind that if there is a product that can help, it has to work on bash/LINUX.

For example, a tip might be, "If you have to perform frequent lookups that involve reading data files, make use of /dev/shm to store those results, and just see if whatever you are looking up has changed since last polling time.  If it has not changed, then just read results from a tmp file stored in the /dev/shm ramdisk. If it has changed, or no cross reference exists, then enumerate as before, and save results in /dev/shm.

Or another, use the native (( )) feature to perform numerical calculations rather than call the expr utility.  I.e,  (( x = x + 10 )) is more efficient then x=`expr $x + 10`

Thank you.  (yes, expect a lot of split points over next day or two)
Question by:David
  • 2
  • 2
LVL 47

Author Comment

ID: 29800963
I did find a great one after I posted this, that resolves Q1, a kernel timer. Here are the details for others .. It works quite nicely BTW

1      timer_stats - timer usage statistics
2      ------------------------------------
4      timer_stats is a debugging facility to make the timer (ab)usage in a Linux
5      system visible to kernel and userspace developers. If enabled in the config
6      but not used it has almost zero runtime overhead, and a relatively small
7      data structure overhead. Even if collection is enabled runtime all the
8      locking is per-CPU and lookup is hashed.
10      timer_stats should be used by kernel and userspace developers to verify that
11      their code does not make unduly use of timers. This helps to avoid unnecessary
12      wakeups, which should be avoided to optimize power consumption.
14      It can be enabled by CONFIG_TIMER_STATS in the "Kernel hacking" configuration
15      section.
17      timer_stats collects information about the timer events which are fired in a
18      Linux system over a sample period:
20      - the pid of the task(process) which initialized the timer
21      - the name of the process which initialized the timer
22      - the function where the timer was intialized
23      - the callback function which is associated to the timer
24      - the number of events (callbacks)
26      timer_stats adds an entry to /proc: /proc/timer_stats
28      This entry is used to control the statistics functionality and to read out the
29      sampled information.
31      The timer_stats functionality is inactive on bootup.
33      To activate a sample period issue:
34      # echo 1 >/proc/timer_stats
36      To stop a sample period issue:
37      # echo 0 >/proc/timer_stats
39      The statistics can be retrieved by:
40      # cat /proc/timer_stats
42      The readout of /proc/timer_stats automatically disables sampling. The sampled
43      information is kept until a new sample period is started. This allows multiple
44      readouts.
46      Sample output of /proc/timer_stats:
48      Timerstats sample period: 3.888770 s
49        12,     0 swapper          hrtimer_stop_sched_tick (hrtimer_sched_tick)
50        15,     1 swapper          hcd_submit_urb (rh_timer_func)
51         4,   959 kedac            schedule_timeout (process_timeout)
52         1,     0 swapper          page_writeback_init (wb_timer_fn)
53        28,     0 swapper          hrtimer_stop_sched_tick (hrtimer_sched_tick)
54        22,  2948 IRQ 4            tty_flip_buffer_push (delayed_work_timer_fn)
55         3,  3100 bash             schedule_timeout (process_timeout)
56         1,     1 swapper          queue_delayed_work_on (delayed_work_timer_fn)
57         1,     1 swapper          queue_delayed_work_on (delayed_work_timer_fn)
58         1,     1 swapper          neigh_table_init_no_netlink (neigh_periodic_timer)
59         1,  2292 ip               __netdev_watchdog_up (dev_watchdog)
60         1,    23 events/1         do_cache_clean (delayed_work_timer_fn)
61      90 total events, 30.0 events/sec
63      The first column is the number of events, the second column the pid, the third
64      column is the name of the process. The forth column shows the function which
65      initialized the timer and in parenthesis the callback function which was
66      executed on expiry.
68          Thomas, Ingo
70      Added flag to indicate 'deferrable timer' in /proc/timer_stats. A deferrable
71      timer will appear as follows
72        10D,     1 swapper          queue_delayed_work_on (delayed_work_timer_fn)
LVL 48

Accepted Solution

Tintin earned 1336 total points
ID: 29825698
I've been writing shell scripts for over 20 years and the single biggest thing that really slows down shell scripts is processing a file line by line, eg:

while read line

This construct is OK for smallish files, but when you start processing larger files, the slow  down is greater and greater.  In a lot of cases, a simple awk or possibly sed/awk can be used to speed up things.

For example a loop with

while read f1 f2
   echo "f1" >>output
done <file

is going to be many orders of magnitudes slower than

awk '{print $1}' file >output

The other area which can slow things down a lot is unnecessary calls to external commands when you can use the shell builtin functions.  bash is quite powerful (for a shell) for things like string manipulation, so quite often, bash scripts that use cut and sed can be rewritten and sped up using the bash builtins.

LVL 48

Assisted Solution

Tintin earned 1336 total points
ID: 29826673
Regarding Q1, the bash debugger may be useful.  I know it's not a profiler, but it can be useful for stepping through code to see what steps it is taking.


For the type of detailed info that you're after, I think DTrace would give you everything you need,  If you're not familar with DTrace, it was originally developed for Solaris kernels, but has since been ported for Linux, FreeBSD and OS/X.

See http://freshmeat.net/projects/dtrace-for-linux

It is a very powerful and flexible tool, although it can take a while to learn how to use it.  There are a bunch of existing DTrace scripts you should be able to find that will help you.


Assisted Solution

JIEXA earned 664 total points
ID: 29894804
For Q1, "strace -f -T ...", you can see what actual system calls took time. Also check the calls that return errors ("grep ' E'" for the strace command).
For Q2, please check for correct PATH and LD_LIBRARY_PATH values: minimalistic and with correct/optimized order.
LVL 47

Author Closing Comment

ID: 31710944
All wonderful tips, every one of them allowed me to squeeze a few seconds of my script so now it is much more efficient.  

Another trick I used in addition to the timer was - Pre-processing configuration and some metadata information that I will refer to in subsequent calls and placing it in the /dev/shm directory.

Featured Post

Free Tool: Path Explorer

An intuitive utility to help find the CSS path to UI elements on a webpage. These paths are used frequently in a variety of front-end development and QA automation tasks.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Recently, an awarded photographer, Selina De Maeyer (http://www.selinademaeyer.com/), completed a photo shoot of a beautiful event (http://www.sintjacobantwerpen.be/verslag-en-fotoreportage-van-de-sacramentsprocessie-door-antwerpen#thumbnails) in An…
In this post we will learn different types of Android Layout and some basics of an Android App.
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
This video will show you how to get GIT to work in Eclipse.   It will walk you through how to install the EGit plugin in eclipse and how to checkout an existing repository.
Suggested Courses
Course of the Month6 days, 4 hours left to enroll

589 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question