Solved

Linux 'sort' order: cron vs command line

Posted on 2014-11-17
6
457 Views
Last Modified: 2014-11-20
Hi Experts,

If I run this Bash script:
#!/bin/bash

echo "Unsorted:"
cat sort_test11
od -bc sort_test11
LC_ALL=""
echo "LC_All=$LC_ALL"
/bin/sort sort_test11
LC_ALL=C
echo "LC_All=$LC_ALL"
/bin/sort sort_test11
LC_ALL=en_US.utf8
echo "LC_All=$LC_ALL"
/bin/sort sort_test11

Open in new window


From the command line I get this output:

Unsorted
z
a
_
A
Z
0000000 172 012 141 012 137 012 101 012 132 012
          z  \n   a  \n   _  \n   A  \n   Z  \n
0000012
LC_ALL=
A
Z
_
a
z
LC_ALL=C
A
Z
_
a
z
LC_ALL=en_US.utf8
_
a
A
z
Z

But from cron I get this output:

Unsorted
z
a
_
A
Z
0000000 172 012 141 012 137 012 101 012 132 012
          z  \n   a  \n   _  \n   A  \n   Z  \n
0000012
LC_ALL=
_
a
A
z
Z
LC_ALL=C
_
a
A
z
Z
LC_ALL=en_US.utf8
_
a
A
z
Z

As you can see, the differences between command line and cron sort output occur when LC_ALL contains nothing or 'C'.  All cron job sorting seems to be case insensitive, but the only case insensitive output from the command line run occurs when LC_ALL=en_US.utf8.

Questions:
Q1. Why these differences between sorting when running the same script from cron and the command line?

Q2. How should I get case sensitive sorting from the cron job?

I haven't found anything useful in "man sort" on this platform yet.  Nor Google.

Here's the version of sort:
$ /bin/sort --version
sort (GNU coreutils) 8.4
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Written by Mike Haertel and Paul Eggert.

And the version of CentOS:
$ uname -a
Linux <servername>.<domainname>.com 2.6.32-358.11.1.el6.x86_64 #1 SMP Wed Jun 12 03:34:52 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

Thanks.
tel2
0
Comment
Question by:tel2
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 3
6 Comments
 
LVL 4

Accepted Solution

by:
popesy earned 500 total points
ID: 40449937
Hi

Have you tried setting the LC_ALL var in the /etc/crontab?

Perhaps there's some other environment setting that's affecting running from the shell vs. running from cron.

Also, shouldn't you be 'export'ing the LC_ALL var to set it?

Cheers, JP
0
 
LVL 12

Author Comment

by:tel2
ID: 40450660
Thanks for that, popesy.

> "Have you tried setting the LC_ALL var in the /etc/crontab?"
No, but I can't imagine how that could help if it doesn't work in the script.  But see below.

> "Also, shouldn't you be 'export'ing the LC_ALL var to set it?"
Good point!  I had tried this in some of my tests, but obviously not the right tests.  Here's the new cron output:
sort_test11
z
a
_
A
Z
0000000 172 012 141 012 137 012 101 012 132 012
          z  \n   a  \n   _  \n   A  \n   Z  \n
0000012
export LC_ALL=
_
a
A
z
Z
export LC_ALL=C
A
Z
_
a
z
export LC_ALL=en_US.utf8
_
a
A
z
Z
As you can see, we have success.  The LC_ALL=C is working case sensitively!

Command line script output has not been changed by exporting LC_ALL.

But that raises a (possibly just academic) question:
Q3. Why is exporting LC_ALL not required in the script when it's run from the command line?  Could it be that LC_ALL is somehow already exported in my command line environment?  How can I tell?  The command "export -p | grep LC" gives me no output.
If I run "locale" from cron I get this output:
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=
If I run it from the command line I get this:
LANG=
LC_CTYPE="POSIX"
LC_NUMERIC="POSIX"
LC_TIME="POSIX"
LC_COLLATE="POSIX"
LC_MONETARY="POSIX"
LC_MESSAGES="POSIX"
LC_PAPER="POSIX"
LC_NAME="POSIX"
LC_ADDRESS="POSIX"
LC_TELEPHONE="POSIX"
LC_MEASUREMENT="POSIX"
LC_IDENTIFICATION="POSIX"
LC_ALL=
0
 
LVL 4

Expert Comment

by:popesy
ID: 40451869
Hi tel2

For your shell environment, try;

env | grep LC

Open in new window


Cheers, JP
0
Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 12

Author Comment

by:tel2
ID: 40453529
Hi popesy,

Chech this out:

$ env | grep LC      -> Nothin'
$ LC_ALL=C
$ env | grep LC      -> Nothin'
$ export LC_ALL=C
$ env | grep LC
LC_ALL=C

As you can see, the only time anything was returned by the grep was when I first manually exported LC_ALL=C.  So, I still don't know the answer to Q3, and unless you or someone else has any other ideas soon, I might have to just leave that acedemic question as one of those unsolved mysteries of the universe, and award points.
0
 
LVL 4

Expert Comment

by:popesy
ID: 40454821
Hi tel2

It seems that distros differ in this regard.  I've got RHEL and SLES and 'locale' output is slightly different on both.  LC_ALL is not set at the command line without explicitly exporting it for me.  This is for both distros I have.

There are a few options to check the default settings;

/etc/sysconfig/language (SLES)

or

/etc/sysconfig/i18n (RH and other, CentOS perhaps?)

The LC* variables are for specific language settings (like what you want!) such that these can be overridden as required.  I know this doesn't answer your Q3 exactly, but may give some additional understanding.

Cheers, JP.
0
 
LVL 12

Author Closing Comment

by:tel2
ID: 40456113
Thanks for your efforts, JP
0

Featured Post

Free Tool: IP Lookup

Get more info about an IP address or domain name, such as organization, abuse contacts and geolocation.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Utilizing an array to gracefully append to a list of EmailAddresses
Fine Tune your automatic Updates for Ubuntu / Debian
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:

752 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question