Solved

How to run git alias with shell other than sh

Posted on 2016-10-14
17
45 Views
Last Modified: 2016-10-21
I made aliases to edit and retrieve branch descriptions:
#> cat .gitconfig
[alias]
        about = "!f(){ git config branch.\"${1-$(git rev-parse --abbrev-ref HEAD)}\".description; }; f"
        add-about = "!f(){ git branch --edit-description \"$1\"; }; f"

Open in new window

Those are nifty, but I also want to check the description for all branches at once.  So I have this script:
#> cat /home/user/bin/git-all-about
#!/bin/bash
git branch | while read line; do lline=${line/\*/}; echo -e $lline;git about $lline;echo -e "\n-----------------------------\n"; done;

Open in new window

That works as I want, but I would prefer to keep the command in my .gitconfig instead of a separate script file.  I had to create this separately because trying to create an alias with the "!f(){}; f" strategy would always fail... I'd receive an error about "bad substitution".  

After some research, I discovered this is *probably* because git's bang syntax spawns the sh shell, and sh does not handle complicated substitutions (like ${line/\*/}), or almost any other string manipulation.  Note that the git-about alias also uses a substitution, but that one has no issues.

Question: Is there a way to have git spawn an arbitrary shell when running an external command as an alias?

Alternatively, is there a way to rewrite the script without using non-sh substitutions?
0
Comment
Question by:Steve Bink
  • 9
  • 4
  • 4
17 Comments
 
LVL 34

Expert Comment

by:Duncan Roe
ID: 41844989
git *may* respect environment variable $SHELL. So if you don't have this in your environment already, try export SHELL=/bin/bash
0
 
LVL 26

Expert Comment

by:skullnobrains
ID: 41845100
you can replace
lline=${line/\*/};

with
lline=`echo $line | tr -d '*'`

or
lline=`echo $line | sed s/\*//g`

and probably a few other options. pick yours
0
 
LVL 26

Accepted Solution

by:
skullnobrains earned 300 total points
ID: 41845103
or possibly replace the whole script with this untested one

git branch  | tr -d '*' | xargs -n 1 git about | sed 's/PATTERN$/\n----------\n'

you'd need to find a pattern in the first or last line of each git about command if you want the last sed to properly add the '-----' in between

or use

git branch  | tr -d '*' | while read line ; do git about $line ; echo -e "\n----------\n" ; done
0
 
LVL 50

Author Comment

by:Steve Bink
ID: 41845839
My $SHELL is already bash.  I found this in the git source:
#ifndef GIT_WINDOWS_NATIVE
		argv_array_push(out, SHELL_PATH);
#else
		argv_array_push(out, "sh");
#endif
		argv_array_push(out, "-c");

Open in new window


The makefile says SHELL_PATH=$SHELL, if it is defined.  Then I tried:
# git config --add alias.mysh '!echo $SHELL'
# git mysh
/bin/bash

Open in new window

So, still not sure what it is using.  I can't find the definition for GIT_WINDOWS_NATIVE, though I did see a comment claiming it was not defined for Cygwin (not applicable).  If it is using $SHELL, then maybe some changes to the quoting can get the original line to work.  Bonus points for figuring that out.  :)

The working line I have so far:
git config --add alias.all-about '!f(){ git branch  | tr -d '"'"'*'"'"' | while read line; do echo $line; git about $line ; echo "\\n----------\\n" ; done; };f'

Open in new window

0
 
LVL 34

Expert Comment

by:Duncan Roe
ID: 41845957
Are you running on a Linux system? (just  checking, since you mention Cygwin)
GIT_WINDOWS_NATIVE is defined for builds that use the (Microsoft) Windows API, typically mingw and mingw64. Cygwin also runs under Windows but provides a comprehensive emulation of the Linux API, so does not need GIT_WINDOWS_NATIVE. All the above are ports, rather than being supported by the release git Makefile.
0
 
LVL 34

Expert Comment

by:Duncan Roe
ID: 41845973
Sorry I just noticed - you know C. I hesitate to say, but I think you may have found a bug in git. Look at the entire function containing the line you highlighted
static const char **prepare_shell_cmd(struct argv_array *out, const char **argv)
{
	if (!argv[0])
		die("BUG: shell command is empty");

	if (strcspn(argv[0], "|&;<>()$`\\\"' \t\n*?[#~=%") != strlen(argv[0])) {
#ifndef GIT_WINDOWS_NATIVE
		argv_array_push(out, SHELL_PATH);
#else
		argv_array_push(out, "sh");
#endif
		argv_array_push(out, "-c");

		/*
		 * If we have no extra arguments, we do not even need to
		 * bother with the "$@" magic.
		 */
		if (!argv[1])
			argv_array_push(out, argv[0]);
		else
			argv_array_pushf(out, "%s \"$@\"", argv[0]);
	}

	argv_array_pushv(out, argv);
	return out->argv;
}

Open in new window

Do you agree with me that this function only ever pushes a shell invocation if argv[0] contains a special character?
Although that might not matter, since newline is a special character.
If it does push a shell invocation, it then pushes all of argv to the output array again.
I've cloned git, plan to debug this when I get time.
0
 
LVL 34

Expert Comment

by:Duncan Roe
ID: 41846024
Started debugging although I should really be doing something else.
[alias]
        tryme = !echo $PWD
---
(gdb) b prepare_shell_cmd
Breakpoint 2 at 0x816bb25: file run-command.c, line 165.
(gdb) c
Continuing.
/usr/src/git/.git
[Inferior 1 (process 32276) exited normally]

Open in new window

git does not go through this function. Needs more debugging (later)
0
 
LVL 50

Author Comment

by:Steve Bink
ID: 41846176
My C skills aren't very strong, but I agree with your assessment.  That list of characters includes space, too.  That implies $SHELL is the "standard", with git falling back to sh if it detects an extremely simple command.  The fallback is an odd construct, though... if I set a $SHELL, there may be a reason, and command simplicity is not necessarily relevant to that choice.  So, not necessarily a bug, but certainly a questionable design choice, IMHO.

Interesting that prepare_shell_cmd() wasn't in the flow.  Can you verify that handle_alias() is in the stack at some point?  Assuming it is, L248 looks for the initial bang, L254 sets child.use_shell, and L258 calls run_command().  From there, run_command() calls start_command(), which (assuming GIT_WINDOWS_NATIVE is not defined) *should* call execv_shell_cmd(), which immediately calls prepare_shell_cmd().  If I'm reading it right, that execution path should be handling any alias using an external command.

Also, this is all being done on Ubuntu, running inside VirtualBox, hosted off Win10 or Win7.  Not sure if it is important, but my repos are typically in Windows directories exposed through VB shared folders.  The underlying file system is NTFS, which caused me a particular headache recently.
0
Do You Know the 4 Main Threat Actor Types?

Do you know the main threat actor types? Most attackers fall into one of four categories, each with their own favored tactics, techniques, and procedures.

 
LVL 50

Author Comment

by:Steve Bink
ID: 41846184
One other thought occurred to me: my git install was from an apt package.  I'm pretty clueless about the internals of apt-get and family, so I'm wondering if it detects the current $SHELL setting during installation.  If I had compiled myself, it would be easy to say SHELL_PATH would be /bin/bash, since that is Ubuntu's default and what I always use.  Can anyone verify that my current environment propagates into apt-get's runtime environment?
0
 
LVL 34

Assisted Solution

by:Duncan Roe
Duncan Roe earned 200 total points
ID: 41846376
170                     argv_array_push(out, SHELL_PATH);
(gdb) p SHELL_PATH
$5 = "/bin/sh"

Open in new window

SHELL_PATH is set in the Makefile. If you had an environment variable $SHELL_PATH at build time, make would use that. apt-get does not build: it fetches a built package.
0
 
LVL 34

Expert Comment

by:Duncan Roe
ID: 41846381
The reason I didn't hit prepare_shell_cmd() before is that you have to tell gdb to debug the child process after the fork() call.
0
 
LVL 34

Assisted Solution

by:Duncan Roe
Duncan Roe earned 200 total points
ID: 41846569
This is a workaround to force bash to run
[alias]
        tryme = !bash -c \"echo $PWD\"

Open in new window

You have to escape the double quotes.
This is the tail end of a debug session (I am printing the file and argv[] arguments to execvp())
[Switching to Thread 0xb7bd6940 (LWP 1818)]
$1 = 0x8269fd8 "/bin/sh"
$2 = {0x8269fd8 "/bin/sh", 0x826ad30 "-c", 0x826ad40 "bash -c \"echo $PWD\"", 0x826ad58 "bash -c \"echo $PWD\"", 0x0}
process 1818 is executing new program: /bin/bash
process 1818 is executing new program: /bin/bash
/usr/src/git/.git

Open in new window

On my system, /bin/sh is a symbolic link to /bin/bash. But bash behaves differently according to whether it is invoked as sh or bash: I think you believe that is your problem.
As per my conjecture in https:#a41845973, the code does push a second copy of argv. There should be an else at line 23. This might be a problem in some cases, I cannot say. Certainly it's untidy and I should submit a bug report with patch, but not tonight.
1
 
LVL 34

Expert Comment

by:Duncan Roe
ID: 41846575
With a really simple command
[alias]
        tryme = !pwd

Open in new window

you don't get the duplicate argument
[Switching to Thread 0xb7bd6940 (LWP 1861)]
$1 = 0x8269ff8 "pwd"
$2 = {0x8269ff8 "pwd", 0x0}
process 1861 is executing new program: /bin/pwd
/usr/src/git/.git

Open in new window

0
 
LVL 26

Expert Comment

by:skullnobrains
ID: 41846851
given the complexity of the rewrite, it seems much simpler than trying to run bash when it just wants to run dash. i'm unsure it is even feasible without recompiling git but then i'm not a git expert and actually quite reluctant at using it
1
 
LVL 50

Author Comment

by:Steve Bink
ID: 41847174
OK, so I found this explanation of using dash instead of sh, which sheds a bit of light.  Given that apt-get is downloading a built package, and dash is meant for non-interactive shells, I think I can safely assume dash is the shell being used to execute aliases using the bang-function strategy.  Also:
# ls /bin/sh
lrwxrwxrwx 1 root root 4 Jun 12 14:34 /bin/sh -> dash

Open in new window


The clarification of using regex replacement explicitly states that it is not supported in dash, and other tools (awk/sed/etc) are recommended alternatives.  That is exactly what I've done by implementing the solution proposed in #41845103, leveraging the tr utility to remove the pesky asterisk.  I'm satisfied with this solution, since it provides portability with minimum hassle.  Quoting is an ugly business.  

Other possible solutions include:
  • Custom compiling git to specify the desired shell
  • Using '!bash -c "command"' to force the use of bash

I don't like the first because manual package management is unpleasant.  I think I'd rather slice off my eyelids than deal with the excessive quoting issues inherent in the second.

@Duncan: please do post back here if you submit a report/patch.  I'm interested in knowing about the second copy, as well.  I think you're on target, and that's just the kind of subtle bug which escapes notice.

Thanks for an excellent discussion guys.  Major points to skullnobrains for the actual solution, runner-up to Duncan for entertaining my curiosity and following bread crumbs.  :)
0
 
LVL 26

Expert Comment

by:skullnobrains
ID: 41847409
I think I'd rather slice off my eyelids than deal with the excessive quoting issues inherent in the second

this can be dealt with

bash -s <<'EOSCRIPT'

[ ... your bash script here ...]

EOSCRIPT
0
 
LVL 34

Expert Comment

by:Duncan Roe
ID: 41853590
I submitted a bug report but it turns out not to be a bug, as patiently explained to me
Date: Fri, 21 Oct 2016 05:00:29 -0400
From: Jeff King <xxxx@xxxx>
To: git <git@vger.kernel.org>
Subject: Re: [BUG] [PATCH]: run-command.c

On Fri, Oct 21, 2016 at 04:50:13PM +1100, Duncan Roe wrote:

> For example, if .git/config has this alias (the sleep is to leave time to
> examine output from ps, &c.):
>
> [alias]
>       tryme = "!echo $PWD;sleep 600"
>
> [...]
> 16:42:06$ ps axf|grep -A2 trym[e]
>  2599 pts/4    S+     0:00      \_ git tryme
>  2601 pts/4    S+     0:00          \_ /bin/sh -c echo $PWD;sleep 600 echo $PWD;sleep
+600
>  2602 pts/4    S+     0:00              \_ sleep 600
> 16:42:45$ cat /proc/2601/cmdline | xargs -0 -n1 echo
> /bin/sh
> -c
> echo $PWD;sleep 600
> echo $PWD;sleep 600

This duplicated argument is expected and normal. The arguments after "-c
whatever" become positional parameters $0, $1, etc. The actual script
arguments start at "$1", and "$0" is typically the "script name".
So you have to stick some placeholder value in the "$0" slot, so that
the sub-script can find the actual arguments. E.g., try:

  sh -c '
    for i in "$@"; do
      echo "got $i"
    done
  ' one two three

it will print only:

  got two
  got three

But if you stick a placeholder there, it works:

  sh -c '
    for i in "$@"; do
      echo "got $i"
    done
  ' placeholder one two three

The value of the placeholder does not matter to the shell. But it is
accessible to the script inside via $0:

  sh -c '
    echo "\$0 = $0"
    echo "\$1 = $1"
    echo "\$2 = $2"
    echo "\$3 = $3"
  ' placeholder one two three

Since our script does not have a filename, we just stick the script
contents there (which is really just a convention, and one I doubt
anybody is really relying on, but there's no point in breaking it now).

> --- a/run-command.c
> +++ b/run-command.c
> @@ -182,8 +182,8 @@ static const char **prepare_shell_cmd(struct argv_array *out,
+const char **argv)
>               else
>                       argv_array_pushf(out, "%s \"$@\"", argv[0]);
>       }
> -
> -     argv_array_pushv(out, argv);
> +     else
> +             argv_array_pushv(out, argv);
>       return out->argv;
>  }

Try running "make test" with this. Lots of things break, because we are
not sending the positional parameters to the shell script at all.

If we just cared about the positional parmeters, we _could_ do something
like:

  if (argv[0]) {
        argv_array_push(out, "sh");
        argv_array_pushv(out, argv + 1);
  }

That would omit "$0" entirely when we have no positional parameters (and
the shell generally fills in "sh" there itself), and provide a dummy
"sh" value when we need to use it as a placeholder.

But again, there's no real value in breaking the existing convention.

-Peff

Open in new window

I responded
Agreed - tests 110 and 111 in t1300-repo-config.sh fail. After that, "make test"
gives up, losing about 14000 lines of output.

Sorry for the noise,

Cheers ... Duncan.

Open in new window

0

Featured Post

Do You Know the 4 Main Threat Actor Types?

Do you know the main threat actor types? Most attackers fall into one of four categories, each with their own favored tactics, techniques, and procedures.

Join & Write a Comment

A publishing tool, a Version Control System, or a Collaboration Platform! These can be some of the defining words for the two very famous web-hosting Git repositories: Bitbucket and Github. Git is widely used amongst the programmers and developers f…
Utilizing an array to gracefully append to a list of EmailAddresses
How to create a branch, fetch changes, and merge them into another branch using the EGit plugin for Eclipse.
How to create a Git repository using GitHub, and how to clone and checkout the repository using the EGit plugin for Eclipse.

757 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

18 Experts available now in Live!

Get 1:1 Help Now