Link to home
Create AccountLog in
Avatar of yokai
yokai

asked on

Smoothing Algorithm

I have been tasked with creating a smoothing algorithm for data contained in a pipe-delimited flat file.  I need to:

1.  Open the flat file and read it line by line
2.  Ignore the first 2 columns (they are just a site & product ID)
3.  Read in five columns at a time, do some math, write the value out to another flat file (that will be formatted the same way as the original).

The format of the incoming file is:
1|2|3|4|5|6|7|8|9
10|11|12|13|14|15
etc, etc

Any suggestions?  I'm figuring I'll use pointers and strtok...?
ASKER CERTIFIED SOLUTION
Avatar of Infinity08
Infinity08
Flag of Belgium image

Link to home
membership
Create an account to see this answer
Signing up is free. No credit card required.
Create Account
Avatar of yokai
yokai

ASKER

How can I get one line from the file, and put it into a pointer for processing?

For instance, if the first line of file name.txt is:
1|2|3|4|5|6|7|8

I want to use strtok and get each number into the pointer for processing...  do the math until the entire line is complete and move on to the next number.  

Does that make sense?
>> How can I get one line from the file, and put it into a pointer for processing?

Take a look at the fgets function I posted earlier. It reads one line of input into a char array (string).

You can then use strtok on that string, and split it up in tokens (using the | as separator). And finally, you can convert each token to its numerical value using atoi.


>> Does that make sense?

Yes it does. To repeat the same thing for every line, you just put the code in a loop, and repeat the loop until the end of the file is reached.
Avatar of yokai

ASKER

What the best way to put this into a loop?  When I run this, I basically get a "1" and then the program ends ("1" is the first column of the input file).

I tried while (pFile != EOF) (which blows up) and while (pFile != NULL) (which goes into an infinite loop).  So I tried while(outBound != NULL) and that gives me an infinite loop as well.

Yeah, I'm not so good at this.... :)
#include <stdio.h>
#include <cstring>
#include <string.h>
 
int 
main(int argc, char *argv[])
{
  FILE *pFile;
  char readIn[1000];
  char *outBound;
 
  pFile = fopen("193_pos.txt", "r");
  
  fgets(readIn, 1000, pFile);
  outBound = strtok(readIn, "|");
  puts(outBound);
 
  fclose(pFile);
  return 0;
}

Open in new window

>> What the best way to put this into a loop?

There's a nice feof function for that :

        http://www.cplusplus.com/reference/clibrary/cstdio/feof.html

But I prefer it like this :

        while (fgets(readIn, 1000, pFile)) {
            /* process the line */
        }

It works, since fgets will return NULL when the end of the file is reached.
Avatar of yokai

ASKER

If I try:
while(fgets(readIn, 1000, pFile))
  {
        outBound = strtok(readIn, "|");
        puts(outBound);
  }
It will give me the first column of each line...  hmmm, getting there :)
How can I get it to go through each column in each line?  I'm thinking I need another loop in there.
>> It will give me the first column of each line...  hmmm, getting there :)

Well, the first call to strtok for a line is different from the subsequent calls (look at the example code on the reference page for strtok). The subsequent calls to strtok take NULL as the first parameter.


>> I'm thinking I need another loop in there.

Correct.
Avatar of yokai

ASKER

Better?

while(fgets(readIn, 1000, pFile))
  {
        outBound = strtok(readIn, " |");

        while (outBound != NULL)
        {
              puts(outBound);
              outBound = strtok (NULL, " |");
        }
  }
Looks good. Do you get the output you want ?
Avatar of yokai

ASKER

Yeah... now I just need to figure out how to process the columns!  Any suggestions?
>> now I just need to figure out how to process the columns!  Any suggestions?

As I said, you can convert the token to an int using atoi. And then you can process the values the way you need to (you didn't specify that).
Avatar of yokai

ASKER

Ok, got it converting to integers.  How can I put each value into another pointer / array?  Would I still use puts?
>> How can I put each value into another pointer / array?

What do you mean ? You want to put the ints in an array ? If so, create an int array :

        int array[SIZE];

and place them in there ... What kind of calculations do you need to do with the values ?


>> Would I still use puts?

puts outputs a string to the screen, so that's not what you want.
Avatar of yokai

ASKER

I'll try my best to describe the calc...

Basically I want to take the first five values after the first two columns in the line (so the 3rd value through the 7th).  I want to:

Multiply value 1 x 1
Multiply value 2 x 2
Multiply value 3 x 4
Multiply value 4 x 2
Multiply value 5 x 1

Add those values up and divide it by 10.  The remaining value will be place in column 3 of the second array (columns 1 & 2 will be the exact same as they were in the original file).

Then, take the next 5 numbers and do the same.  I keep doing this up until I reach the 2nd to last number in the row, at which point I stop.

Then, move on to the next row.  Lather, rinse, repeat.
Ok.

Easiest would be to already open the output file, and write the data to the file as you encounter it.

For each line :

1) read two tokens, and write them to the output file
2) read 5 tokens and store them in an int array (of size 5)
3) calculate the value based on the 5 ints in the array and write it to the output file
4) go to 2) if there is more data
Avatar of yokai

ASKER

What syntax would I use for calculating the value of the ints?  

Sorry, I know I'm asking a million questions LOL.  I really do appreciate the help....  thank you.
>> What syntax would I use for calculating the value of the ints?  

What do you mean ? As I said earlier, you can use atoi (refer to the reference page) to convert a string to an int.
>> Sorry, I know I'm asking a million questions LOL.  I really do appreciate the help....  thank you.

No problem. Ask away ;)
Avatar of yokai

ASKER

As an example, to do the multiplication:

a = int_a[1]*1;
b = int_a[2]*2;
c = int_a[3]*4;
d = int_a[4]*2;
e = int_a[5]*1;
f = a+b+c+d+e / 10;

Is this how I would do the actual calculations once I store the 5 calcs into an int array?
>> Is this how I would do the actual calculations once I store the 5 calcs into an int array?

You need extra ()'s in the last calculation :

        f = (a+b+c+d+e) / 10;

because / has higher precedence than +

Other than that, it looks ok.)
Avatar of yokai

ASKER

Darn!  I failed to mention a key point...  sorry :\

When I write the output file, it must be in the same pipe delimited format as the original!
>> When I write the output file, it must be in the same pipe delimited format as the original!

That's no problem. Simply add the |'s where needed :)
Avatar of yokai

ASKER

When running the program, I get the error:

Run-Time Check Failure #2 - Stack around the variable 'calc_arr' was corrupted.

"calc_arr" is my int array for the 5 values to run math on..
Can you show your code ? (all of it preferably)
Avatar of yokai

ASKER

Sure!  Just don't laugh at it LOL...
#include <stdio.h>
#include <cstring>
#include <string.h>
#include <stdlib.h>
 
int 
main(int argc, char *argv[])
{
  FILE *pFile;
  char readIn[1000];
  char *outBound;
  int outBound_i;
  int calc_arr[5];
  int i=0;
  int a,b,c,d,e,f;
 
  // Open POS file
  pFile = fopen("193.txt", "r");
    
  // While the file is open...
  while(fgets(readIn, 1000, pFile))
  {
	  // Read in each column of each line up to "|".
	  // Assign the value to "outBound".
	  outBound = strtok(readIn, "|");
 
	  while (i <= 5 && outBound != NULL)
	  {
		  // Convert current character to integer.
		  outBound_i = atoi(outBound);
		  calc_arr[i] = outBound_i;
		  i++;
 
		  // Set VAR to NULL to break from loop.
	      outBound = strtok (NULL, "|");
	  }
		a = calc_arr[0]*1;
		b = calc_arr[1]*2;
		c = calc_arr[2]*4;
		d = calc_arr[3]*2;
		e = calc_arr[4]*1;
		f = (a+b+c+d+e) / 10;
		printf("%d", f);
 
		i=0;
  }
 
  fclose(pFile);
  return 0;
}

Open in new window

>> while (i <= 5

This means that you'll loop 6 times (not 5) : for i = 0, 1, 2, 3, 4 and 5.
Your array has only room for 5 ints though. So, you have a buffer overflow (writing past the end of the array).


Also :

>>                 printf("%d", f);

You might want to either add a space or a newline in there to separate the different values (otherwise they'll be pasted together). For example :

        printf("%d ", f);
Avatar of yokai

ASKER

Ok, got it!  Now, I'm trying on a file that only has one row of numbers...  about 50 columns or so.  What's wrong with my loop?  I'm only getting the first output column, but it won't move on to the next...
#include <stdio.h>
#include <cstring>
#include <string.h>
#include <stdlib.h>
 
int 
main(int argc, char *argv[])
{
  FILE *pFile;
  char readIn[1000];
  char *outBound;
  int outBound_i;
  int calc_arr[5];
  int i=0;
  int a,b,c,d,e,f;
 
  // Open POS file
  pFile = fopen("193.txt", "r");
    
  // While the file is open...
  while(fgets(readIn, 1000, pFile))
  {
	  // Read in each column of each line up to "|".
	  // Assign the value to "outBound".
	  outBound = strtok(readIn, "|");
 
	  while (i <= 4 && outBound != NULL)
	  {
		  // Convert current character to integer.
		  outBound_i = atoi(outBound);
		  calc_arr[i] = outBound_i;
		  i++;
 
		  // Set VAR to NULL to break from loop.
	      outBound = strtok (NULL, "|");
	  }
		a = calc_arr[0]*1;
		b = calc_arr[1]*2;
		c = calc_arr[2]*4;
		d = calc_arr[3]*2;
		e = calc_arr[4]*1;
		f = (a+b+c+d+e) / 10;
		printf("%d|\n", f);
		i=0;
  }
 
  fclose(pFile);
  return 0;
}

Open in new window

Avatar of yokai

ASKER

Also, when trying to write to outbound file:
fwrite(f, 1, sizeof(f), oFile);

I get the following error:
error C2664: 'fwrite' : cannot convert parameter 1 from 'int' to const void *'
>> I'm only getting the first output column, but it won't move on to the next...

You'll need another loop.

The outer loop performs one iteration for each line
The inner loop takes 5 values from the line, and stores them in the array

You need a loop in between so you get something like this (pseudo code)

        for every line
            while not end of line
                read the next 5 values
                calculate f and write it to the output file
            end while
        end for


>> error C2664: 'fwrite' : cannot convert parameter 1 from 'int' to const void *'

fwrite is meant for binary data. You want to write plaintext I assume ? Take a look at fprintf :

        http://www.cplusplus.com/reference/clibrary/cstdio/fprintf.html
Avatar of yokai

ASKER

Oooook....  here's what I've got so far (again) :)

Everything compiles and runs...  but nothing happens.  Nothing gets written to the output.txt...


#include <stdio.h>
#include <cstring>
#include <string.h>
#include <stdlib.h>
 
int 
main(int argc, char *argv[])
{
  FILE *pFile;
  FILE *oFile;
  char readIn[1000];
  char *outBound;
  int outBound_i;
  int calc_arr[5];
  int i=0;
  int a,b,c,d,e,f;
 
  // Open POS file
  pFile = fopen("193.txt", "r");
  oFile = fopen("output.txt", "wb");
    
  // While the file is open...
  while(fgets(readIn, 10000, pFile))
  {
	  // Read in each column of each line up to "|".
	  // Assign the value to "outBound".
	  outBound = strtok(readIn, "|");
 
	  while (!feof(pFile))
	  {
		  while (i <= 4 && outBound != NULL)
		  {
			  // Convert current character to integer.
			  outBound_i = atoi(outBound);
			  calc_arr[i] = outBound_i;
			  i++;
 
			  // Set VAR to NULL to break from loop.
			  outBound = strtok (NULL, "|");
		  }
			a = calc_arr[0]*1;
			b = calc_arr[1]*2;
			c = calc_arr[2]*4;
			d = calc_arr[3]*2;
			e = calc_arr[4]*1;
			f = (a+b+c+d+e) / 10;
			fprintf (oFile, "%d|\n", f);
			printf("%d|\n", f);
			i=0;
	  }
  }
 
  // Close input file.
  fclose(pFile);
 
  // Close output file.
  fclose(oFile);
  return 0;
}

Open in new window

>>   oFile = fopen("output.txt", "wb");

"wb" means that you open it for writing in binary mode. Just "w" is for plaintext mode.


>>           while (!feof(pFile))

You want to check for the end of the line, not the end of the file ;)


>>                         fprintf (oFile, "%d|\n", f);

Do you want the newline there ?
Avatar of yokai

ASKER

Like this:
while (*outBound != '\n')
        {
              while (i <= 4 && outBound != NULL)
              { .......

>> Do you want the newline there ?
Nope, thanks!
>> while (*outBound != '\n')

You can actually use :

        while (outBound != NULL)
So, does it work better now ?
Avatar of yokai

ASKER

Well, I'm trying to debug...  and I keep getting an error....

File: fgets.c
Line: 57

Expression: ( str != NULL )

Then it asks me to Abort, Retry, Ignore.

The app compiles fine... but I can't debug :\
Are you still talking about the same code you posted earlier ?
Avatar of yokai

ASKER

Yes, same code.  Again, this only happens when I try to run the app from the IDE.  If I build it and run it from console, it doesn't give me this error.  I'd like to put some variables in a watch window.
What IDE are you using ?
Avatar of yokai

ASKER

Visual C++ 2005 Express Edition
No experience with that IDE. I assume you are building this as a C program (check your project settings to make sure)

Btw, this header does not belong in a C program :

        #include <cstring>

It's a C++ wrapper around the <string.h> header you already have. You can just remove it.
Avatar of yokai

ASKER

Ok, I removed that header.  I am compiling as C code, just double checked.  Not sure why it's giving me that error...

I think it has something to do with using the while(fgets(readIn, 10000, pFile)), since the error is coming from fget.c...

I have no clue how to get around that LOL
The only problem I can spot with it is that you declared readIn as an array of 1000 characters :

  char readIn[1000];

while you try to read 10000 characters into it :

  while(fgets(readIn, 10000, pFile))

(ten thousand versus one thousand)
Avatar of yokai

ASKER

I just tried changing each to match the other...  both at 1000 then both at 10000...  same error :\

Is there another line I can use rather than the while(fgets....?
>> Is there another line I can use rather than the while(fgets....?

It would be pretty stupid to have to change your (correct and standard) code to get around some obscure compiler error.

One more thing to try is to check pFile for NULL just before the while loop (to see if the file was correctly opened).
Avatar of yokai

ASKER

Yeah, I have to agree with you.  I will try adding the check on pFile and let you know what happens.
Avatar of yokai

ASKER

Hey, I figured it out.  I added error handling...

if (( pFile = fopen("C:\\App\\Debug\\193.txt", "r")) == (FILE *) 0) {
            printf ("Error!");
            return 1;
      }

I can now debug.  I'm going to close this "case" and I'll post more specific questions later.  Thank you for all your help!!