We help IT Professionals succeed at work.
Troubleshooting Question

-bash: [: too many arguments. I get this error when ssh to a new Azure VM

msidnam
msidnam asked
on
58 Views
Last Modified: 2020-09-13
I created a new server from the Azure Marketplace (CIS Hardened Ubuntu 18.04) and added a pub/private key that I've been using for other servers using the same image.

When I ssh to that new server I'm getting the error: "-bash: [: too many arguments"

I've tried to search that error but most of the results are for people making scripts in bash. not SSH to servers.
Comment
Watch Question

CERTIFIED EXPERT
Top Expert 2007

Commented:
Check the .profile or .bash_profile or similar files in /etc for wrong command or * character. This could be the issue.

Author

Commented:
this is what I have in the .profile. I dont see anything out of the ordinary:

# ~/.profile: executed by the command interpreter for login shells.
# This file is not read by bash(1), if ~/.bash_profile or ~/.bash_login
# exists.
# see /usr/share/doc/bash/examples/startup-files for examples.
# the files are located in the bash-doc package.

# the default umask is set in /etc/profile; for setting the umask
# for ssh logins, install and configure the libpam-umask package.
#umask 022

# if running bash
if [ -n "$BASH_VERSION" ]; then
    # include .bashrc if it exists
    if [ -f "$HOME/.bashrc" ]; then
        . "$HOME/.bashrc"
    fi
fi

# set PATH so it includes user's private bin if it exists

if [ -d "$HOME/bin" ] ; then
    PATH="$HOME/bin:$PATH"
fi

# set PATH so it includes user's private bin if it exists
if [ -d "$HOME/.local/bin" ] ; then
    PATH="$HOME/.local/bin:$PATH"
fi



I don't see any files with * or commands in the /etc directory.

I did setup another VM from Azure thats just ubuntu 18.04 (Not the CIS Hardened version) and that one does not give me the error when using ssh to access the server. Im wondering if Azure has a bad image?

CERTIFIED EXPERT
Top Expert 2007

Commented:

What about the other files
~/.bash_profile or ~/.bash_login
How  do you ssh? Can you show the command?

Author

Commented:
no bash_profile or .bash_login. Only a .bash_logout

We ssh in using local linux machines from home as well as putty for a windows workstation.

We also ssh from within other Azure VM's. The command we use is: ssh -i .ssh/privkey.pem username@ipaddress

We get the error using all three of these ways. But only on the new VM we created. the others (which have been up and running for over a year) don't give us those errors. If i do a uname -a, I do notice a different version.

VM not working : Linux <computer name> 5.4.0-1025-azure #25~18.04.1-Ubuntu SMP Sat Sep 5 15:28:57 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

VM working: Linux <computer name> 5.3.0-1022-azure #23~18.04.1-Ubuntu SMP Mon May 11 11:55:56 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

The non working one seems to be a higher version. perhaps that image is having an issue?

nociSoftware Engineer
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
Filenames starting with a . are hidden from view.. (like the DOS/Windows HIDDEN file attribute).
ls -a make them visible.

file used in login:
/etc/profile
~/.profile
~/.bash_profile
~/.bash_login

Normal script running:
~/.bashrc
/etc/bash/bashrc   (+ /etc/bash/bashrc.d)

Then any command can be a bash script containing a problem...
Often /etc/profile uses a bunch of scipts in /etc/profile.d

Author

Commented:
I went ahead and created two more VMs. One running Ubuntu 18.04 (Not CIS Hardened) and another one, CIS Hardened 16.04 (Not 18.04). Both of those work with no issues.

The error comes straight out of the box right after creating it. I have to think its something with that image, kernel, etc. We have not added anything to the server at all (scripts, updates, apps, etc) when this happens. 
David FavorFractional CTO
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
This type of annoyance is dirt simple to debug.

Go through the files in noci's comment.

Place the following at top of each file...

echo ".bashrc" (or related file name)

set -x

Open in new window


Then login.

You'll have the filename output, then output will die in the file + line number where the problem occurs...

You'll have to do the same thing with any /etc/profile.d/* files, if you've modified these...

Author

Commented:
Just to be clear. This happens right out of the box after I create the Azure VM. We do not modify or add anything to the server, unless the system does it itself during initial boot up since its a CIS version of Ubuntu. But that I don't know since i dont see the machine booting since its in Azure.

i did set -x on .bashrc and .profile but I dont see anywhere where it stops and tells me a line number.

This is what I get when logging on:

# Message of the DAY
Last login: Sat Sep 12 00:11:47 2020 from x.x.x.x
-bash: [: too many arguments


But its said that even before putting in set -x at the top of .bashrc and .profile.

CERTIFIED EXPERT
Top Expert 2007

Commented:
Who is the root user of this system?
root may need to check the file /etc/profile which login bash will read and execute.

Also, can you use -vv with ssh command? This should give some debugging info.
David FavorFractional CTO
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
Now you've provided enough information to debug the problem.

Context is everything.

Remove all the echo statements + set -x statements as the problem seems related to PAM, which is where MOTD (message of the day) output generation occurs...

Most likely somehow an error has crept into your MOTD generation, which is wildly different across Distros + also modified by various PAM modules installed.

One of the first actions I take on every new install is just to disable PAM, as it's rarely ever required.

If this is possible, add the following to your /etc/ssh/ssd_config file, then do a service sshd restart.

usePAM no
useDNS no

Open in new window


This makes ssh logins much faster also.
David FavorFractional CTO
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
If you must debug PAM, then you'll do this...

# in one window
inotifywait -e OPEN -mrq /etc

# in another window login

Open in new window


This will show you an ordered trace of each file opened related to your login, likely /etc/pam.d/* or something similar.

Likely you'll instantly see what file name is involved.

If not, you'll just have to open each file + fix the problem.
David FavorFractional CTO
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
Most common problem is a file missing #!/bin/bash in top line or having something else in top line, then BASH code inside the file.

Author

Commented:
I went ahead and removed the MOTD for my login but as soon as I ssh I still get:

login as: user
Authorized uses only. All activity may be monitored and reported.
user@x.x.x.x's password:
-bash: [: too many arguments
user@dev-test:~$


I used this to disable the MOTD: 
 touch $HOME/.hushlogin
When I out usePAM no and useDNS no in the sshd_config file it didnt take it away.

I'm using a pub/priv key to login to ssh. Not sure if that has anything to do with it.
CERTIFIED EXPERT
Top Expert 2007

Commented:
Can you try login with username and password and see?

Author

Commented:
omarfarid,
Attached is what i get if i ssh using -vv
ssh_login.txt

This happens if i use a password as well. I created a new user with just a password and same thing. 
CERTIFIED EXPERT
Top Expert 2007

Commented:
Do you have this dir /etc/init.d  ?

If yes, what you get if you run

ls -l /etc/update-motd.d

Also, are you setting your shell prompt (PS1) or any env variable while login?
CERTIFIED EXPERT
Top Expert 2007

Commented:
Can you change the login shell for user in /etc/passwd to /usr/bin/bash -x ?

I never tried that, but you may try it for specific user for testing.
nociSoftware Engineer
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
You cant add parameter in /etc/passwd.  you just try to use the  file  "/usr/bin/bash -x"    which doesn't exist.  So logon will fail.
nociSoftware Engineer
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
Did you verify the /etc/bash_completion.d and ~/.bash_completion directories/scripts, it is also scanned for scripts.
You can run the following: strace bash -l to see what happens during logon... bash -l (el)  is how the login is run.
David FavorFractional CTO
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
You said, "When I out usePAM no and useDNS no in the sshd_config file it didn't take it away."

Clarify what this means.

Maybe you got some error when you restarted sshd or maybe this seems to have no effect...

At least for sshd versions released over the past decade, sshd accepts these settings...

net17 # cat /etc/ssh/sshd_config | tail 
# http://linuxaria.com/howto/4-tricks-to-speed-up-ssh-connections
# AddressFamily inet
# http://www.azoft.com/spotlight/2011/04/18/ssh-ubuntu-100.html
# http://blog.tcs.de/speed-up-ssh-connections-with-compression-on-os-x/
# http://forums.gentoo.org/viewtopic-t-47746-view-next.html?sid=d4b05b81dcfe95b8e6f5313b06cd7b26
# touch ~/.hushlogin
PrintMotd yes
UsePAM no
UseDNS no
#GatewayPorts yes

net17 # service sshd restart

Open in new window


Notice the settings in the sshd config file, then restarting sshd works correctly.

Tip: You may have to resort to using inotifywait to trace file execution (explained above).

Sometimes the simple solution is seeing what files are executed.
David FavorFractional CTO
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
Another debug trick...

# On your server open an ssh connection, then run a copy of sshd with maximum debug spew...
/usr/sbin/sshd -p 55555 -D -ddd -e

# Then initiate an ssh session with maximum debug spew...
ssh -vvvv -i /path-to-key $user@$host

Open in new window


I doubt this will help + work trying, if all else fails.

More than likely the inotifywait trick will work... and... likely also good to change the inotifywait command to also trace all user login files hit, which will be the following for the root user...

inotifywait -e OPEN -mrq /etc ~root

Open in new window


This will expand the inotifywait command to show how /etc system files + user login files interact... or run in sequence... related to each other...

The other trick is just to comment out everything in all your startup files, as the BASH error might be caused by some external program being run in some way.

Bottom line is you rule out everything, until the solution is all that's left.

Author

Commented:
A lot of suggestions, thank you. I'll try to answer in order without sounding confusing.

omarfarid,
When I do ls -l /etc/update-motd.d  i get, total 0

noci,
I did look at directory  /etc/bash_completion.d  and it has a few files in the directory but when i view them I dont see anything odd, although I'm not sure what to look for.

When i do the strace command it gives me a lot of info that makes no sense to me and it wouldn't let me cntrl-c out of it.

David,
For UsePam no and UseDNs no, I added them to the end of the sshd_config file (note, when I dded the entries i had usePAM and useDNS with a lowercase U. The entries in the file had uppercase U), restarted sshd service and logged back in. I did not notice any change as I still got the error. Looking at your latest comment, I went back in the file and noticed that the file already has entries for them (based on your comment) UsePAM was set to yes I changed it to no. UseDNS was commented out I uncommented it, saved the file, restarted sshd, logged out and then back in and still the same thing (Note, I removed my entries i put in at the end of the file). I get the too many arguments error and thats it.

I tried doing the sshd command with a different port but but since this is a hardened CIS image i think i need to open the firewall or iptables in ubuntu and I am not familiar with that.

I did do the inotifywait on one session and then logged on to another session and I'll attach watch it shows when logging in. I see it hitting the directories/scripts that you all mention. But I still cant find out where the issue is.I'll attach watch displays when i login. I'm only attaching what i see immediately after putting in my password. It has a lot more stuff before logging in that i can attach as well.

I'm doing my best to give you all the info. I don't work on Linux much. I know enough to get around the OS and do some basic troubleshooting. apologies if my answers are not what you are expecting.

Also, remember/keep in mind that this happens immediately after I create the VM. I am not adding anything to the system, scripts, directories, programs, etc. I also do not know what the image changes during boot up. Since its a CIS Hardened Ubuntu image it very well could have scripts running when it first boots up to harden it but I dont know.All i know is right after Azure says that the resource is ready, i ssh to the newly created VM and get the too many arguments error. This VM is for our devlopers and the only thing they've told me (which alerted me to the issue) is this:

"Something to do with the initial shell loading seems to break variable substitution in a myriad of interesting ways. I think we should kill it and spawn another one."

"We moved past blockers with the /tmp directory being possibly mounted with `noexec` by explicitly setting the tmp dir for the Ruby orchestration framework capistrano. Then ssh keys. Now it's another root directory access being blocked and we were trying to workaround that"

Straight out of the gate the new VM has issues.


I appreciate your suggestions but I'm not sure if this is something that can be fixed. I really think the image from the Azure Marketplace is hosed.

inotifywait.txt

Software Engineer
CERTIFIED EXPERT
Distinguished Expert 2019
Commented:
This one is on us!
(Get your first solution completely free - no credit card required)
UNLOCK SOLUTION
David FavorFractional CTO
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
The inotifywait file you posted is binary, so can't be reviewed.

Author

Commented:
noci,
can i do strace bash -l 2>output.txt  from one session and then login using another session? will it still give me the data? or is strace bash -l 2>output.txt  only on that one session?

Author

Commented:
David,
Try this one. Not sure what happened.

inotifywait.txt

Author

Commented:
noci,
pushing it to a file was much easier to read and search. This is what i get:

write(2, "/etc/profile: line 22: [: too ma"..., 45/etc/profile: line 22: [: too many arguments
) = 45

when i view /etc/profile, line 22 is the one bolded:

if [ -d /etc/profile.d ]; then
  for i in /etc/profile.d/*.sh; do
    if [ -r $i ]; then
      . $i

I'm not a programmer but I'm guessing the line of code is iterating through each of those .sh files in /etc/profile.d? I also notice, =45 but i dont know if thats another line number in the same file or a different file.

So its looking like one of those files in profile.d you all mentioned earlier but how do we determine which one?

Author

Commented:
So i did a set -x on the /etc/profile (not the one inside my home dir) and got this:

+ for i in /etc/profile.d/*.sh
+ '[' -r /etc/profile.d/01-locale-fix.sh /etc/profile.d/apps-bin-path.sh /etc/profile.d/bash_completion.sh /etc/profile.d/cedilla-portuguese.sh '/etc/profile.d/*.sh' /etc/profile.d/tmout.sh /etc/profile.d/Z97-byobu.sh /etc/profile.d/Z99-cloudinit-warnings.sh /etc/profile.d/Z99-cloud-locale-test.sh ']'
-bash: [: too many arguments
+ for i in /etc/profile.d/*.sh
+ '[' -r /etc/profile.d/tmout.sh ']'
+ . /etc/profile.d/tmout.sh
++ TMOUT=900

I dont know if that means Z99-cloud-locale-test.sh  is the bad sh file or just happens to be the last one in the list. or maybe the whole line is a bad argument:

+ '[' -r /etc/profile.d/01-locale-fix.sh /etc/profile.d/apps-bin-path.sh /etc/profile.d/bash_completion.sh /etc/profile.d/cedilla-portuguese.sh '/etc/profile.d/*.sh' /etc/profile.d/tmout.sh /etc/profile.d/Z97-byobu.sh /etc/profile.d/Z99-cloudinit-warnings.sh /etc/profile.d/Z99-cloud-locale-test.sh ']'
CERTIFIED EXPERT
Top Expert 2007

Commented:
I think issue is in

for i in /etc/profile.d/*.sh

Author

Commented:
I commented out (in bold):

# /etc/profile: system-wide .profile file for the Bourne shell (sh(1))
# and Bourne compatible shells (bash(1), ksh(1), ash(1), ...).

if [ "${PS1-}" ]; then
  if [ "${BASH-}" ] && [ "$BASH" != "/bin/sh" ]; then
    # The file bash.bashrc already sets the default PS1.
    # PS1='\h:\w\$ '
    if [ -f /etc/bash.bashrc ]; then
      . /etc/bash.bashrc
    fi
  else
    if [ "`id -u`" -eq 0 ]; then
      PS1='# '
    else
      PS1='$ '
    fi
  fi
fi

#if [ -d /etc/profile.d ]; then
#  for i in /etc/profile.d/*.sh; do
#    if [ -r $i ]; then
#      . $i
#    fi

  done
  unset i
fi
umask 027

and got this:

-bash: /etc/profile: line 25: syntax error near unexpected token `done'
-bash: /etc/profile: line 25: `  done'



at least its a new error.

Author

Commented:
So now i commented out the whole thing (sorry not a programmer didnt realize it was all the same line of code:

#if [ -d /etc/profile.d ]; then
#  for i in /etc/profile.d/*.sh; do
#    if [ -r $i ]; then
#      . $i
#    fi

# done
#  unset i
#fi

 and i no longer get the too many arguments.
CERTIFIED EXPERT
Top Expert 2007

Commented:
You should comment done as well
CERTIFIED EXPERT
Top Expert 2007

Commented:
You may check what you have for
etc/profile.d/*.sh
And if you have time and knowledge inspect each script

Author

Commented:
omarfarid,
Thank you. I may have the developers who use this version of Ubuntu take a look since they may understand a little more the scripts and whats happening.

Thank you all for the suggestions and patience. I'm still getting used to this new answer system (i know its been around for a bit but I have no idea what happened to the points system) so please bear with me as I do my best to mark your answers and give you all credit.

Author

Commented:
Thank you all again for the help. Much appreciated. 
nociSoftware Engineer
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
The problem is probably a file with a space in the name in /etc/profile.d .. (that should not be there.... btw).

if [ -d /etc/profile.d ]; then
  for i in /etc/profile.d/*.sh; do
    if [ -r "$i" ]; then
      . "$i"
    fi
 done
  unset i
fi

The code is expected to be present as software that needs some stuff "preset" for a shell will drop a file in /etc/profile.d
(f.e. to setup aliases, of environment variables.., like EDITOR etc.).
Do your self a future favour and reinstall to the original code.

Then verify /etc/profile.d doesn't contain a file with a blank or '*' in the name!
That '*' or blank needs to go.

Be careful   rm -f * removes ALL files...   use rm -fi '*.sh'   the QUOTES are important  the -i also, only say Y on a file with '*' as name.

Author

Commented:
noci,
You mentioned, "Do your self a future favour and reinstall to the original code."

We haven't changed any code in the VM. This happens right after we create it from the Azure marketplace.  
nociSoftware Engineer
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
You said you modified /etc/profile... so undo those changes..., and add " aound $i  which will help here.
There is a reason for these files....
/etc/profile.d/01-locale-fix.sh /etc/profile.d/apps-bin-path.sh /etc/profile.d/bash_completion.sh /etc/profile.d/cedilla-portuguese.sh '/etc/profile.d/*.sh' /etc/profile.d/tmout.sh /etc/profile.d/Z97-byobu.sh /etc/profile.d/Z99-cloudinit-warnings.sh /etc/profile.d/Z99-cloud-locale-test.sh
to be present in that directory /etc/profile.d
Disabling them will not help with a next logon.

And THIS is the problematic one: '/etc/profile.d/*.sh'


Author

Commented:
noci,
Gotcha, I understand what you are saying now.

This is what i show in the /etc/profile.d:

'*.sh'              Z99-cloud-locale-test.sh    bash_completion.sh
 01-locale-fix.sh   Z99-cloudinit-warnings.sh   cedilla-portuguese.sh
 Z97-byobu.sh       apps-bin-path.sh            tmout.sh

The two Z99 files are highlighted in green (not sure what that means). I don't see any spaces in the files, and one does have * in it, '*.sh', but in the /etc/profile file it does reference that *.sh file:

for i in /etc/profile.d/*.sh; do

I'm guessing its saying that to run all those scripts without having to reference each one?


nociSoftware Engineer
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
No in the for statement it want a list of all file files ending in *.sh (* is wildcard for ANY character).
So it is not a needed file..
All the other files ARE needed though, and there is a reason for them to be run.
They are green because they are executable. which is not relevant.

Author

Commented:
Apologies for the confusion. I am understanding it better now. I did what you mentioned and after logging in i do not get the error anymore.

I'll check with my developers to see if this also fixes the other issues they were having. Thank you.
David FavorFractional CTO
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
Glad you got this fixed!

This type of problem can be a serious annoyance + time drain to fix.

Gain unlimited access to on-demand training courses with an Experts Exchange subscription.

Get Access
Why Experts Exchange?

Experts Exchange always has the answer, or at the least points me in the correct direction! It is like having another employee that is extremely experienced.

Jim Murphy
Programmer at Smart IT Solutions

When asked, what has been your best career decision?

Deciding to stick with EE.

Mohamed Asif
Technical Department Head

Being involved with EE helped me to grow personally and professionally.

Carl Webster
CTP, Sr Infrastructure Consultant
Empower Your Career
Did You Know?

We've partnered with two important charities to provide clean water and computer science education to those who need it most. READ MORE

Ask ANY Question

Connect with Certified Experts to gain insight and support on specific technology challenges including:

  • Troubleshooting
  • Research
  • Professional Opinions
Unlock the solution to this question.
Join our community and discover your potential

Experts Exchange is the only place where you can interact directly with leading experts in the technology field. Become a member today and access the collective knowledge of thousands of technology experts.

*This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

OR

Please enter a first name

Please enter a last name

8+ characters (letters, numbers, and a symbol)

By clicking, you agree to the Terms of Use and Privacy Policy.