Solved

grub2 scripting. looking for complete documentation in order to sort items automatically

Posted on 2013-12-18
9
1,050 Views
Last Modified: 2013-12-29
hello all

i'm trying to do various stuff in grub2 shell but i have a hard time finding proper documentation.

---
Q1

anybody knows where to find a proper exhausive documentation containing namely the supported syntaxes for commands such as regexp and test ? ( which are not available in any of the pages i visited at gnu.org and namely not in http://www.gnu.org/software/grub/manual/html_node/Shell_002dlike-scripting.html )

---
Q2

i have a script that detects available kernels and provides a boot entry for each of them with the corresponding initrd.

i have a hard time to sort the kernel by reverse version order :
- i have no idea if a sort command is available and anyway it does not seem like pipes are supported so it probably would be useless
- i tried various shell syntaxes for the test command but i'm only able to sort numerically using "test NUMBER -gt NUMBER" which work for decimal numbers but will not allow version comparison (i'd need to compare stuff such as 3.8.1-11)
- i have no idea what kind of string manipulation functions are available other than "regexp" and i only know the basic usage since i have yet to find where it is documented
- there migh be a way to list files matching a pattern and output them sorted somehow but i have not found it yet

obviously i can parse the major, minor, sub-minor and whatever you would call the last number, store them into as many variables and use a bunch of if-else comparisons to reach this goal

but it is a major pita and it is not generic enough since it will not allow to compare versions such as 11a with 11b (quite useless for linux kernels but i'd like it more generic)

i'd also be interested in a sort based on mtime or ctime for filesystems that handle it but i have no idea if/how that would be supported in grub shell

i am NOT interested by solutions that involve using a shell script to write grub.cfg : this is both trivial and not what i'm looking for

---

thanks for any hints you can provide
0
Comment
Question by:skullnobrains
  • 6
  • 2
9 Comments
 
LVL 34

Expert Comment

by:Duncan Roe
Comment Utility
The most exhaustive documentation for Grub is grub.info, which you access by typing info grub on your keyboard.
You will not be pleased to hear that a search for regexp found only the following
20 Regexp
*********

Regexps work on unicode characters, however no attempt at checking
cannonical equivalence has been made. Moreover the classes like
[:alpha:] match only ASCII subset.
Unfortunately, this means that Gnu have not seen fit to document grub regexp further.
0
 
LVL 26

Author Comment

by:skullnobrains
Comment Utility
unfortunately, I already looked at most info pages to no avail, but thanks nevertheless

currently i'm using regexp like this
regexp -s varname pattern subject

Open in new window

i found out this undocumented syntax in somebody else's code by accident
when the pattern captures something, it gets stored into the specified variable
i still did not figure out how to capture several strings at the same time (if feasible)
and i yet have to try various syntaxes in order to do a replacement

any ideas or links still welcome
0
 
LVL 34

Assisted Solution

by:Gary Patterson
Gary Patterson earned 500 total points
Comment Utility
Disclaimer: not a grub expert, but I thought you might find some of this useful.

Re: regexp.  Not exactly "undocumented syntax", but you need to look at the source code to find it:

https://chromium.googlesource.com/chromiumos/third_party/grub2/+/refs/heads/stabilize-4636.B/grub-core/commands/regexp.c

static const struct grub_arg_option options[] =
  {
    { "set", 's', GRUB_ARG_OPTION_REPEATABLE,
      /* TRANSLATORS: in regexp you can mark some
         groups with parentheses. These groups are
         then numbered and you can save some of
         them in variables. In other programs
         those components aree often referenced with
         back slash, e.g. \1. Compare
         sed -e 's,\([a-z][a-z]*\),lowercase=\1,g'
         The whole matching component is saved in VARNAME, not its number.
       */
      N_("Store matched component NUMBER in VARNAME."),
      N_("[NUMBER:]VARNAME"), ARG_TYPE_STRING },
    { 0, 0, 0, 0, 0, 0 }
  };

So it follows that these are syntax examples:

regexp -s variable re source
or
regexp -s 1:variable1 -s 2: variable2 ... -s N:variableN REwith()segments source

Looks like you can also just specify the sequence number you want:

regexp -s 2:variable REwith()segments source

Found this syntax example at http://www.mail-archive.com/grub-devel@gnu.org/msg16403/autoiso.cfg:

regexp -s 1:v1 -s 2:v2 '/boot/vmlinuz-(.*)-sidux-(.*)' "$kernel"

Looks like v1 would be assigned to the substring matching the first (.*) segment and v2 the second (.*) segment.

Also found lots of regexp examples with these Google searches:

grub2 "regexp -s"
grub2 "regexp set"

You might try a similar approach with "test" command: start with the grub2 test.c source code, and then use what you find to compose search strings to find examples.

Hope some of that is useful.

- Gary
0
 
LVL 26

Author Comment

by:skullnobrains
Comment Utility
pretty useful yeah : though i actually had figured out those syntaxes empirically, i had yet to realize the -s option could be repeated, and no other options were available

as far as regexp is concerned, this is what i understood so far
- regexp will return true if the regexp matches, false otherwise with or without -s switch
- regular expressions are not "docked" (there is no implicit ^ or $ at the beginning/end)
- no modificators are available (unsure about that one)
- "-s VARNAME" will store whatever is captured by parenthesis in VARNAME
- nested parenthesis work as expected
- using -s without parenthesis is an error (regexp returns false even if there is a match)
- when regexp does not match, VARNAME is left unchanged
- variables can be assigned to a specific captured string using "-s NUM:VARNAME"
- "-s" can be repeated
- "-s" is the only available option (not sure about than one either)
i have not needed or tried anything like (?:...), backreferences or the likes. apparently the engine is using extended-regexp-like syntax

as far as test is concerned, there seem to be a whole wealth of available options including -nt and -ot (newer/older than)... which are unfortunately unavailable in the version i've been playing with (2.00-19ubunt). i yet have to check if this is a google-hacked version or if newer versions feature the same.

it also seems easier to patch the source code rather than actually implement version comparison in the grub shell. unfortunately, i'm unsure I can do that given my requirements but i can definitely impose a minimal grub version

thanks a lot for your help so far. any other thoughts are more than welcome. i'll keep this thread updated
0
Top 6 Sources for Identifying Threat Actor TTPs

Understanding your enemy is essential. These six sources will help you identify the most popular threat actor tactics, techniques, and procedures (TTPs).

 
LVL 34

Expert Comment

by:Gary Patterson
Comment Utility
You've done a lot of work.  Maybe you can update the Grub2 documentation with it so others don't have to go through the same thing.

Looking at the regexp.c source code link above, the call to regcomp (Compile Regular Expression) specifies REG_EXTENDED.  Assuming the version you are using does the same, then the POSIX standard should apply (any exceptions are supposed to be explicitly documented in utilities that make exceptions to the standard):

http://pubs.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap09.html

I don't think I have a lot more to contribute here.  This is a little out of my area of expertise, but I thought I'd chip in what I could.  It has been interesting, and I learned a little about grub myself.

- Gary Patterson
0
 
LVL 26

Author Comment

by:skullnobrains
Comment Utility
gary, you were actually very helpful in making me realize that

- although i was testing with 2.00_19ubunt..., the version this is based on is actually most likely somewhere along the 1.98.x or 1.99.x versions, (and the version numbering is so messed up i'm not even sure about that)

- looking at the source myself was probably much better use of my time than empirically toying with various options or other dumb reverse engineering techniques.

i'll accept your answer at some point, but keep this thread open until i have time to work on this again and post some useful code. i spent quite a long time expecting grub2 to compile on ubuntu and ended up compiling it from ports in a freebsd vm... which took quite some time and i had to switch to more urgent matters ( like Xmas ;)

feel free to post any relevant thoughts in the meantime
0
 
LVL 26

Author Comment

by:skullnobrains
Comment Utility
as far as the test command is concerned :

i was wrong, there is some kind of unusable explanation in the "other" section of the documentation


version comparisons using '-plt' and '-pgt' work pretty much as expected for number-based version comparison. it cannot be fooled too easily by different prefix or wrong separators. strange comparisons produce a negative answer. (aka comparing weird stuff such as "bar-1" with "baz-1" will produce a negative answer both ways. same applies to "11a" and "11b")

newer and older than work using -ot and -lt. i have not digged into bias yet. if either file does not exist, the comparison returns false

lexicographical comparison work for ascii in what seems to be a reliable way. the comparison is case insensitive says the manual. i have not checked. some comparators starting with < or > need to be double-quoted. quoting other comparators produces an error

-gt -eq -le and the likes apparently work as expected

i have not checked these yet, but -z and -n should work as expected

-f -d -e should be ok. i only used -f for now

-a -o and parenthesis seem to be implemented somehow, but i never tried using them. parenthesis are likely to require quoting

the understanding of how the shell actually works, of variable scopes, and how to use functions is out of the scope of this question, but i'm currently trying to figure those out
0
 
LVL 26

Accepted Solution

by:
skullnobrains earned 0 total points
Comment Utility
seems like i need to close this post with a piece of working code. sorry i have no time to document all that i found out in this thread

Q1 : basically i'm not expecting an answer anymore. seems like this documentation just does not exist. additionally, there are huge differences between minor versions along the 2.00.xx lines

Q2 : here is a very dumb and very unefficient but working sort implementation

# usage : sort SORT_ORDER word1 [ word2 ... ]
# elements are expected to be properly sortable given the SORT_ORDER
# SORT_ORDER is any comparison understood by grub's builtin test function
# the resulting sorted string will be stored in $sort_ret
# return codes are not meaningfull (yet)
# SORT_ORDERS (version-dependant) : < > = != -ot -nt -pgt -plt -lt -le -gt -ge
# example 
function sort {
	sort_op=$1
	shift
	sort_ret=$1
	shift
	while	test -n "$1"
	do	sort_needle=$1
		shift
		sort_ret_this="$sort_ret"
		sort_ret=""
		sort_append=""
		#echo sort_needle=$sort_needle
		#echo into=$sort_ret_this
		for	sort_word in $sort_ret_this
		do	if	test -n "$sort_append" -o $sort_word "$sort_op" $sort_needle
			then	#echo _ A $sort_word
				sort_ret="$sort_ret $sort_word"
			else	#echo _ N $sort_needle $sort_word
				sort_ret="$sort_ret $sort_needle $sort_word"
				sort_append=yes
			fi
		done
		if	test -z "$sort_append"
		then	#echo _ L $sort_needle
			sort_ret="$sort_ret $sort_needle"
		fi
	done
}

Open in new window


results are stored in the $sort_ret variable

-plt and -pgt are suitable to sort linux kernel versions

sorting stuff like 11a 11b still requires more writing
0
 
LVL 26

Author Closing Comment

by:skullnobrains
Comment Utility
best answer
0

Featured Post

Do You Know the 4 Main Threat Actor Types?

Do you know the main threat actor types? Most attackers fall into one of four categories, each with their own favored tactics, techniques, and procedures.

Join & Write a Comment

How to remove superseded packages in windows w60 or w61 installation media (.wim) or online system to prevent unnecessary space. w60 means Windows Vista or Windows Server 2008. w61 means Windows 7 or Windows Server 2008 R2. There are various …
I use more than 1 computer in my office for various reasons. Multiple keyboards and mice take up more than just extra space, they make working a little more complicated. Using one mouse and keyboard for all of my computers makes life easier. This co…
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
You have products, that come in variants and want to set different prices for them? Watch this micro tutorial that describes how to configure prices for Magento super attributes. Assigning simple products to configurable: We assigned simple products…

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now