asked on

Concatenation of strings and variables issue

I have a shell script that moves files from a source directory to a working directory where the files are processed (used to populate an Oracle database), then moves them to an archive directory. All of the files end with a file extension that's equal to the year and week (e.g.this week would be 201524) that we refer to as the cycle. Occasionally, duplicate files are transferred to the source directory after they have already been processed before. This is all done through an automated process in production over which we have no control, other than modifying the shell script. My current attempt at the script is something like this -
mv /working/*.* /archive
mv /source/newfiles.* /working
cycle=`sqlplus -S user@prod/password << EOF
select substr(max(filename,instr(max(filename,'.',-1) + 1) ==> this gets the extension
from filetable
where status = 'Completed'; ==> means file has already been processed
EOF`
grep -c $cycle /working/*.* ==> searched for files with extensions that were already processed
exit_code=$?
if [ $exit_code -eq 0 ] ==> means file that files were found
then
mv -f /working/*.$cycle /archive ==> this is where the problem is
fi
...
If I echo "mv -f /working/*.$cycle /archive", it comes out as
mv -f /working/*.
201523 /archive
It creates a new line
I have also tried creating a variable for each part of the string with the $cycle and concatenating the variables into another variable, but I then get a space instead of the linefeed between the mv -f /working/*. and the $cycle
Can anyone show me how to concatenate strings with variables without breaks of any kind?

flow01

The break is probably in the sqlplus output and thus included in the cycle variable.
add
echo start showing cycle
echo $cycle
echo end showing cycle
-- to see if that assumption is true and which part of the output you need
-- if its the first line you need maybe
cycle2=`echo $cycle | head -n1`
echo start showing cycle2
echo $cycle2
echo end showing cycle2
-- will help

Murugesan Nagarajan

Try using:

Replace:

cycle=`sqlplus -S user@prod/password << EOF
 select substr(max(filename,instr(max(filename,'.',-1) + 1) ==> this gets the extension
 from filetable
 where status = 'Completed'; ==> means file has already been processed
 EOF`

Open in new window

With:

cycle=''`sqlplus -S user@prod/password << EOF
 select substr(max(filename,instr(max(filename,'.',-1) + 1) ==> this gets the extension
 from filetable
 where status = 'Completed'; ==> means file has already been processed
EOF`''
export cycle=''`/bin/echo -ne $cycle | /bin/egrep -v "^$" | /bin/sed -e "N;s/\n//;s/^ //;"`''
# Following line written to check whether that is working or not
/bin/echo "Current value of cycle [ $cycle ]"

Open in new window

awking00

ASKER

In reality, the actual max(filename) in the filetable on the database where it contains 'F001' and status = 'Completed' is PDBMW.BMTRB0.F001AA.T201507. The filetable will also have files that contain 'F000' and 'F002', which are control and header files, with the same extension. The 'F001' file contains the data and, if it has a status of 'Completed', the control and header file will also.
The "cycle= ..." script actually returns 'T201507' without any spaces or linefeeds when I issue an echo $cycle command. So the problem doesn't seem to exist in that part. It's when I do something like -
movecommand="mv /working/PDBMW."$cycle" /archive" and echo movecomand, it returns 'mv /working/PDBMW.[WITH A SPACE HERE]T201507 /archive'
I have tried with and without quotes, with and without curly braces, and I've tried creating variables for the first and last parts of the string surrounding the $cycle variable and I still keep getting that space (and sometimes even a linefeed) at that point.

ASKER CERTIFIED SOLUTION

flow01

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

SOLUTION

Murugesan Nagarajan

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

Murugesan Nagarajan

Informations using echo and same on using /bin/echo
Example:

$ cycle="     T201507"
$ echo $cycle
T201507
$ echo "$cycle"
     T201507
$ echo '$cycle'
$cycle
$ echo Murugesan$cycle
Murugesan T201507

Open in new window

Murugesan Nagarajan

Sorry for long discussion,
I forgot to mention the following related to shell program:
After logon it create the shell program(example /bin/ksh) for logged on user
/bin/ksh
|
\|/
read input from user
/|\
User input taken character by character.
Example:
/bin/echo $cycle
Assume [cycle] is having the value [ T201507]
Here shell program reads the character from user like the following:
/bin/echo $cycle
This statement taken as echo followed by 7 spaces and T201507:
/bin/echo T201507
However shell program pass that parameter T201507 as first parameter since space is taken as parameter seperator by default when not using double quotes by the shell.
This is applicable to
1. shell
2. Windows key R => windows run
3. command prompt(cmd.exe)
4. batch script
5. parameter passed to start any programs

You can understand this using following C program:

#include <stdio.h>
int main( int argc, char *argv[])
{
	if ( 1 == argc)
	{
		printf( "No parameter passed\n");
	}
	else
	{
		printf( "Value of first parameter is [%s]\n", argv[1]);
	}
	printf( "Number of parameters [%d]\n", argc-1);
	return 0;
}

Open in new window

COMPILATION:
$ gcc -g -Wall parse.c -o parse

OUTPUT without parameters:
$ ./parse
No parameter passed
Number of parameters [0]

OUTPUT using one parameter "$cycle" without double quotes:
$ cycle=" T201507"
$ ./parse $cycle
Value of first parameter is [T201507]
Number of parameters [1]

OUTPUT using one parameter "$cycle" using double quotes:
$ ./parse "$cycle"
Value of first parameter is [ T201507]
Number of parameters [1]

awking00

ASKER

Thanks to you both. Apparently the problem was the sql returning white space of some sort to the cycle variable combined with the strange behavior of echo.