Concatenation of strings and variables issue

I have a shell script that moves files from a source directory to a working directory where the files are processed (used to populate an Oracle database), then moves them to an archive directory. All of the files end with a file extension that's equal to the year and week (e.g.this week would be 201524) that we refer to as the cycle. Occasionally, duplicate files are transferred to the source directory after they have already been processed before. This is all done through an automated process in production over which we have no control, other than modifying the shell script. My current attempt at the script is something like this -
mv /working/*.* /archive
mv /source/newfiles.* /working
cycle=`sqlplus -S user@prod/password << EOF
select substr(max(filename,instr(max(filename,'.',-1) + 1) ==> this gets the extension
from filetable
where status = 'Completed'; ==> means file has already been processed
EOF`
grep -c $cycle /working/*.* ==> searched for files with extensions that were already processed
exit_code=$?
if [ $exit_code -eq 0 ] ==> means file that files were found
then
mv -f /working/*.$cycle /archive ==> this is where the problem is
fi
...
If I echo "mv -f /working/*.$cycle /archive", it comes out as
mv -f /working/*.
201523 /archive
It creates a new line
I have also tried creating a variable for each part of the string with the $cycle and concatenating the variables into another variable, but I then get a space instead of the linefeed between the mv -f /working/*. and the $cycle
Can anyone show me how to concatenate strings with variables without breaks of any kind?
LVL 32
awking00Asked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

flow01Commented:
The break is probably in the sqlplus output and thus included in the cycle  variable.
add
echo start showing cycle
echo $cycle
echo end  showing cycle
-- to see if that assumption is true and which part of the output you need
-- if its the first line you need maybe
cycle2=`echo $cycle | head -n1`
echo start showing cycle2
echo $cycle2
echo end  showing cycle2
-- will help
0
Murugesan NagarajanCommented:
Try using:

Replace:
cycle=`sqlplus -S user@prod/password << EOF
 select substr(max(filename,instr(max(filename,'.',-1) + 1) ==> this gets the extension
 from filetable
 where status = 'Completed'; ==> means file has already been processed
 EOF`

Open in new window

With:
cycle=''`sqlplus -S user@prod/password << EOF
 select substr(max(filename,instr(max(filename,'.',-1) + 1) ==> this gets the extension
 from filetable
 where status = 'Completed'; ==> means file has already been processed
EOF`''
export cycle=''`/bin/echo -ne $cycle | /bin/egrep -v "^$" | /bin/sed -e "N;s/\n//;s/^ //;"`''
# Following line written to check whether that is working or not
/bin/echo "Current value of cycle [ $cycle ]"

Open in new window

0
awking00Author Commented:
In reality, the actual max(filename) in the filetable on the database where it contains 'F001' and status = 'Completed' is PDBMW.BMTRB0.F001AA.T201507. The filetable will also have files that contain 'F000' and 'F002', which are control and header files, with the same extension. The 'F001' file contains the data and, if it has a status of 'Completed', the control and header file will also.
The "cycle= ..." script actually returns 'T201507' without any spaces or linefeeds when I issue an echo $cycle command. So the problem doesn't seem to exist in that part. It's when I do something like -
movecommand="mv /working/PDBMW."$cycle" /archive" and echo movecomand, it returns 'mv /working/PDBMW.[WITH A SPACE HERE]T201507 /archive'
I have tried with and without quotes, with and without curly braces, and I've tried creating variables for the first and last parts of the string surrounding the $cycle variable and I still keep getting that space (and sometimes even a linefeed) at that point.
0
Cloud Class® Course: C++ 11 Fundamentals

This course will introduce you to C++ 11 and teach you about syntax fundamentals.

flow01Commented:
strange behaviour of  echo prevents detection
cycle= T201507
echo start showing cycle
echo $cycle
echo end  showing cycle
echo "."$cycle # returns a space ?
echo " "$cycle
echo "            "$cycle
echo "mv /working/PDBMW."$cycle" /archive"
# but you can use that behaviour to remove a leading space
cycle=`echo $cycle`
echo "mv /working/PDBMW."$cycle" /archive"

learned something myself today
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Murugesan NagarajanCommented:
FOR YOUR INFORMATION:
When a variable contains value prefixed with space,
echo $cycle does not display space in that result

Example cycle prefixed using space but it does not display space using echo:
$ cycle="      testing"
$ echo $cycle | od -bc
0000000 164 145 163 164 151 156 147 012
          t   e   s   t   i   n   g  \n
0000010
$ echo "[$cycle"
[      testing

Open in new window


Example cycle prefixed using space and displaying space using echo:
$ cycle="      testing"
$ echo "[$cycle" | od -bc
0000000 133 040 040 040 040 040 040 164 145 163 164 151 156 147 012
          [                           t   e   s   t   i   n   g  \n
0000017
$ echo "[$cycle"
[      testing

Open in new window


Example code to update variable for removing space in cycle variable:
#!/bin/ksh
#The following line assign the word T201507 without space.
CycleWithoutSpace="T201507"
/bin/echo "$CycleWithoutSpace" | /bin/egrep "^ " >/dev/null 2>&1
Ret=$?
if [ 0 -eq $Ret ]
then
	/bin/echo "RESULT1 variable CycleWithoutSpace [$CycleWithoutSpace] contains         space"
	/bin/echo "RESULT1 Following line removing space in CycleWithoutSpace [$CycleWithoutSpace]"
	CycleWithoutSpace=''`/bin/echo -ne "$CycleWithoutSpace" | /bin/sed "s/ *//;"`''
	/bin/echo "RESULT1 After removing space in variable CycleWithoutSpace [$CycleWithoutSpace]"
else
	/bin/echo "RESULT1 variable CycleWithoutSpace  does not contain space [$CycleWithoutSpace]"
fi
#echo multiple new lines
/bin/echo -ne "\n\n\n\n\n\n"


#The following line assign the word T201507 prefixed with six spaces.
CycleHaving_Space="      T201507"
/bin/echo "$CycleHaving_Space" | /bin/egrep "^ " >/dev/null 2>&1
Ret=$?
if [ 0 -eq $Ret ]
then
	/bin/echo "RESULT2 variable CycleHaving_Space contains          space [$CycleHaving_Space] "
	/bin/echo "RESULT2 Hence removing space in variable CycleHaving_Space [$CycleHaving_Space]"
	CycleHaving_Space=''`/bin/echo -ne "$CycleHaving_Space" | /bin/sed "s/ *//;"`''
	/bin/echo "RESULT2 After removing space in variable CycleHaving_Space [$CycleHaving_Space]"
else
	/bin/echo "RESULT2 variable CycleHaving_Space [$CycleHaving_Space] does not contain space"
fi

Open in new window


OUTPUT:
[out]
$ ./echoSample.sh
RESULT1 variable CycleWithoutSpace  does not contain space [T201507]






RESULT2 variable CycleHaving_Space contains          space [      T201507]
RESULT2 Hence removing space in variable CycleHaving_Space [      T201507]
RESULT2 After removing space in variable CycleHaving_Space [T201507]
[/out]
0
Murugesan NagarajanCommented:
Informations using echo and same on using /bin/echo
Example:
$ cycle="     T201507"
$ echo $cycle
T201507
$ echo "$cycle"
     T201507
$ echo '$cycle'
$cycle
$ echo Murugesan$cycle
Murugesan T201507

Open in new window

0
Murugesan NagarajanCommented:
Sorry for long discussion,
I forgot to mention the following related to shell program:
After logon it create the shell program(example /bin/ksh) for logged on user
                  /bin/ksh
                         |
                        \|/
                 read input from user
                         /|\
User input taken character by character.
Example:
                         /bin/echo $cycle
Assume [cycle] is having the value [      T201507]
Here shell program reads the character from user like the following:
                         /bin/echo $cycle
This statement taken as echo followed by 7 spaces and T201507:
                         /bin/echo      T201507
However shell program pass that parameter T201507 as first parameter since space is taken as parameter seperator by default when not using double quotes by the shell.
This is applicable to
1. shell
2. Windows key R => windows run
3. command prompt(cmd.exe)
4. batch script
5. parameter passed to start any programs

You can understand this using following C program:
#include <stdio.h>
int main( int argc, char *argv[])
{
	if ( 1 == argc)
	{
		printf( "No parameter passed\n");
	}
	else
	{
		printf( "Value of first parameter is [%s]\n", argv[1]);
	}
	printf( "Number of parameters [%d]\n", argc-1);
	return 0;
}

Open in new window



COMPILATION:
$ gcc -g -Wall parse.c -o parse


OUTPUT without parameters:
$ ./parse
No parameter passed
Number of parameters [0]


OUTPUT using one parameter "$cycle" without double quotes:
$ cycle="      T201507"
$ ./parse $cycle
Value of first parameter is [T201507]
Number of parameters [1]


OUTPUT using one parameter "$cycle" using double quotes:
$ ./parse "$cycle"
Value of first parameter is [      T201507]
Number of parameters [1]
0
awking00Author Commented:
Thanks to you both. Apparently the problem was the sql returning white space of some sort to the cycle variable combined with the strange behavior of echo.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Shell Scripting

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.