[Webinar] Streamline your web hosting managementRegister Today

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 2232
  • Last Modified:

Complex C program Design questions regarding arrays and file manipulation

I have a Korn shell script that needs to call a Pro C program that passes in a filename.  This file is created in the Korn shell script and it containes lines with pipe delimited data:

Each line is formatted to look like this:

1234|Pat|Smith|567 Main St Chicago, IL 60601| <blank space> |psmith@yahoo.com

This line was populated using PL/SQL.  Every time the Korn shell script is called, the file is populated with a variable number of lines each time so I'll number of lines will always change but have the same format as above.

Within the Pro C program, I need to query a table "customer" to get the phone number based on the first field in the pipe delimited line which is custID.  The phone number is stored like this 2125551234 in the table.  THEN, I need to call another Pro C program (which I have) to format the phone number to look like this (212) 555-1234.  

After I get the phone number with the correct format, I need to replace it with the <blank space> in each line of the file and then save that file.

There's alot going on here.  I need help with the design of this C program.  I have written some of it but I am concerned with my approach.  I am storing the information from the file in an array of custIDs.  The problem is I dont know how big this custID array will get since I never know what the number of lines in a file will be.  Also, my other concern is figuring out how to write to the same file I'm reading.  I dont think this is possible so I'm thinking of creating a temporary file to store this information.

Any design input would be GREATLY appreciated.
0
farekat
Asked:
farekat
  • 11
  • 8
  • 6
  • +1
5 Solutions
 
evilrixSenior Software Engineer (Avast)Commented:
Ok, firstly, since you are using C++ you can use std::vector rather than fixed size arrays. This is a dynamically allocated array, which is self managing.

http://www.cplusplus.com/reference/stl/vector/

Regarding reading/writing to a file at the same time. Yes, this is perfectly possible providing you accept that once data is over-written it can be re-read. To be honest though, it is otfen simpler to write out a new file. I guess that all depends upon the use case for your requirements. To access files you can use file streams.

http://www.cplusplus.com/reference/iostream/fstream/

If you did want to write a new file and then use it to overwrite the old file you could make use of the code below.

If you have further Qs I'd suggest you consider posting the code you have written so far along with specific Qs about it.

I hope this helps.

-Rx.


#include <io.h>
#include <cstdlib>
#include <iostream>
#include <fstream>
#include <string>
#include <stdexcept>
 
 
void copy_file(std::string const & sOldPath, std::string const & sNewPath, bool bOverWrite = false)
{
	if(!bOverWrite && (0 == access(sNewPath.c_str(), 0x00))) { throw std::runtime_error("Output file already exists"); }
 
	std::ifstream ifs(sOldPath.c_str(), std::ios::binary);
	if(!ifs.is_open()) { throw std::runtime_error("Unable to open input file"); }
 
	std::ofstream ofs(sNewPath.c_str(), std::ios::trunc | std::ios::binary);
	if(!ofs.is_open()) { throw std::runtime_error("Unable to open output file"); }
 
	ifs.seekg(0, std::ios::end);
	std::ifstream::pos_type size = ifs.tellg();
	ifs.seekg(0);
 
	std::string sData(size, std::string::value_type());
 
	ifs.read(&sData[0], size);
	if(!ifs.good()) { throw std::runtime_error("Unable to read input file"); }
 
	ofs.write(sData.data(), size);
	if(!ifs.good()) { throw std::runtime_error("Unable to write output file"); }
}
 
int main()
{
	int nRetval = 0;
 
	try
	{
		std::string sOldPath = "c:\\temp\\test.txt";
		std::string sNewPath = "c:\\temp\\newtest.txt";
 
		copy_file(sOldPath, sNewPath);
 
		std::cout << "Copied " << sOldPath << " to " << sNewPath << " successfully." << std::endl;
	}
	catch(std::exception const & ex)
	{
		std::cerr <<"Error. " << ex.what() << std::endl;
		nRetval = -1;
	}
 
#ifdef WIN32
	system("pause"); // Only works on Windows
#endif
 
	return nRetval;
}

Open in new window

0
 
jkrCommented:
Well, the 1st step is to extract the ID field. Since you want it to be C, that is quite easy, just use

char line[] = "1234|Pat|Smith|567 Main St Chicago, IL 60601| <blank space> |psmith@yahoo.com"
char custID[16];

char* line_end = strchr(line,'|');

*line_end = '\0'; // null terminate it

// copy to 'custID'
strcpy(custID,line);
char line[] = "1234|Pat|Smith|567 Main St Chicago, IL 60601| <blank space> |psmith@yahoo.com"
char custID[16];
 
char* line_end = strchr(line,'|');
 
*line_end = '\0'; // null terminate it
 
// copy to 'custID'
strcpy(custID,line);
 
// now, to actually read that from a file:
 
FILE* p = fopen("customers.txt","r");
 
char line[255];
 
while(fgets(line,255,p) {
 
  char custID[16];
 
  char* line_end = strchr(line,'|');
 
  *line_end = '\0'; // null terminate it
 
  // copy to 'custID'
  strcpy(custID,line);
 
  // now, make DB query based on data in 'custID'
}
 
fclose(p);

Open in new window

0
 
farekatAuthor Commented:
Regarding the file writing, I'm doing it in C.  I'm going to check out your suggestion jkr...thanks so much
0
Never miss a deadline with monday.com

The revolutionary project management tool is here!   Plan visually with a single glance and make sure your projects get done.

 
evilrixSenior Software Engineer (Avast)Commented:
>> Regarding the file writing, I'm doing it in C
You posted your Q to the C++ Zone, hence my assumption.
0
 
farekatAuthor Commented:
Sorry evilrix!  

I have a question:  in the while loop for the read, is it  possible to open another file and write to it the data from the file that I am looping through.  I want to in the while loop read each line, change that line, and then write it to a new file until I reach the end of the file I am looping through?
0
 
farekatAuthor Commented:
heres a little more explanation:

I want to read the line from the file which looks like ""1234|Pat|Smith|567 Main St Chicago, IL 60601| <blank space> |psmith@yahoo.com"

do the query of the database to retrieve the phone number (call the formatting C program), take that formatted phone number put it in place of the <blank space>, and then write that line to the new file...loop through to the next line in the file I'm reading and do it all over again...yikes..

0
 
evilrixSenior Software Engineer (Avast)Commented:
>> it  possible to open another file and write to it the data from the file that I am looping through
Yes. There is not reason why you can't have two file handles to two different files at the same time, reading from one and writing to another. That is what my C++ code above does, although obviously using streams and not handles. Do you need an example maybe?
0
 
jkrCommented:
Well, then, the code needs some more modifications:
#include <stdio.h>
 
int main() {
 
    FILE* p = fopen("customers.txt","r");
    FILE* out = fopen("output.txt","w"); 
 
    char line[255];
 
    while(fgets(line,255,p) {
 
      char custID[16];
      char phone[16];
      cher* delim;
 
      char* ID_end = strchr(line,'|');
 
      *ID_end = '\0'; // null terminate it
 
      // copy to 'custID'
      strcpy(custID,line);
 
      // restore line
      *ID_end = '|';
 
      // now, make DB query based on data in 'custID'
 
      //... get_phone_nr_from_db(custID,phone);
 
      // find the <blank space> (4th delimiter)
      delim = line;
      for (int i = 0; i < 4) {
 
        delim = strchr(delim,'|');
        delim++;
      }
 
      memcpy(delim,phone,strlen(phone));
 
      // write new line
      fputs(line,out);
 
    }
 
    fclose(p);
    fclose(out);
 
return 0;
}

Open in new window

0
 
farekatAuthor Commented:
I'm geting this weird error message on this line when running the application at this line:
*ID_end = '\0';

The message says:  "11416 Memory fault(coredump)"  I have no clue what this means or what could be causing it.
0
 
evilrixSenior Software Engineer (Avast)Commented:
strchr can return a NULL if it doesn't find what it's looking for and because this is never tested ID_end can, potentially, be a NULL pointer, which you will dereference. I think you need to add a check to make sure ID_end isn't NULL.

http://www.cplusplus.com/reference/clibrary/cstring/strchr.html
0
 
farekatAuthor Commented:
I took out the * before the *ID_end = '\0' and it passed that step...is that acceptable?  why do I need that star there to reference the pointer?  Will this still be accurate?
0
 
evilrixSenior Software Engineer (Avast)Commented:
No, all that will do is assign ID_end to be effectively a NULL pointer. Maybe something like the following...

char* ID_end = strchr(line,'|');
if(NULL == ID_end) { continue; }

Which will make it skip that line and return back to line 10 to read and process the next one. Is skipping the line acceptable?

-Rx.
0
 
evilrixSenior Software Engineer (Avast)Commented:
You have the potential for the same defect on line 34 (delim = strchr(delim,'|';), where it looks like it is skipping through fields and then at line 38 a field is replace using delim (memcpy(delim,phone,strlen(phone))). Again, if delim is NULL it'll crash, so you should consider also testing that variable and implementing something like a continue to skip the line.
0
 
farekatAuthor Commented:
why do i have to use a pointer anyway, can't i extract the custID from the line into a char variable and be done with it..do I have to add a null at the end if I do that?
0
 
evilrixSenior Software Engineer (Avast)Commented:
The customer ID is a string, right? Therefore you need a char * type variable to point to it (that's how C works). The strchr is searching the string for the '|' character, which I believe is your field delimeter, so that it can pull out the custID for you. This is done by temporarily converting the '|' found to a NULL char so that the strcpy below it can copy the custID into a new variable (called custID), the '|' is then put back.

If strchr fails to find the '|' it will return a NULL pointer rather than a pointer to where the '|' would have been found. This is the signal that the strchr failed to find a match. Given that you are processing records with each field delimetered by a '|' this would suggest you have corrupt data; however, your code should still be able to cope with this. I suggest the simplest way is to skip the line as I suggsted above.

None of this can be done just using a char variable as this only represents one 8 bit type, and is used to represent a single character.

Does that make sense?
0
 
jkrCommented:
>>I have no clue what this means or what could be causing it.

It basically means that the delimiter could not be found, which in turn means that your input data is not what it should be. Are there other lines that do not match the format

1234|Pat|Smith|567 Main St Chicago, IL 60601| <blank space> |psmith@yahoo.com

?
0
 
farekatAuthor Commented:
for (int i = 0; i < 4) {
 
        delim = strchr(delim,'|');
        delim++;
      }

this is supposed to be i++, right?
0
 
evilrixSenior Software Engineer (Avast)Commented:
Nope.

It is incrementing the delim pointer by one so that the strchr can search for the next field delimeter. If this wasn't done it would keep matching the same delimiter since delim , prior to incrementing, points to a delimiter.

http://www.cplusplus.com/reference/clibrary/cstring/strchr.html

const char * strchr ( const char * str, int character );
char * strchr (       char * str, int character );

Return Value
A pointer to the first occurrence of character in str.
If the value is not found, the function returns a null pointer.

-Rx.
0
 
jkrCommented:
No, actually

for (int i = 0; i < 4; ++i) {
 
        delim = strchr(delim,'|');
        delim++;
      }

It increments 'i' and the pointer, thus taking care that subsequent searches won't always find the same delimitoer again.
0
 
farekatAuthor Commented:
Do I have to free any variables from memory?
0
 
Infinity08Commented:
>> Do I have to free any variables from memory?

All variables that are allocated on the heap (using new and the like) need to be delete'd.
0
 
Infinity08Commented:
Here's a nice tutorial :

        http://www.cplusplus.com/doc/tutorial/dynamic.html
0
 
jkrCommented:
>>Do I have to free any variables from memory?

Not in the above code, it does not use any dynamic allocation.
0
 
farekatAuthor Commented:
I'm trying to add a prefix "PH|" to the line before I insert it into the second file so that the line in the second file looks like this:

PH|1234|Pat|Smith|567 Main St Chicago, IL 60601| 800-555-1212 |psmith@yahoo.com

I'm trying this with the code below with no luck of course...sigh


char *str1 = "PH|";
char *str3;
 
str3 = (char *)malloc((strlen(str1) + strlen(line) + 1)
                              *sizeof(char));
 
strcat(str3, str1);
strcat(str3, line);
 
fputs(str3, fp2);
free(str3);

Open in new window

0
 
Infinity08Commented:
char *str1 = "PH|";
char *str3;
 
str3 = (char *)malloc((strlen(str1) + strlen(line) + 1)
                              *sizeof(char));
 
strcpy(str3, str1);
strcat(str3, line);
 
fputs(str3, fp2);
free(str3);
0
 
Infinity08Commented:
Notice that I changed the first strcat to strcpy.
0
 
jkrCommented:
Why would you want to 'malloc()' anything? That's not necessary. If you have another field prefixing the ID, the parsing is slightly different, i.e.
#include <stdio.h>
 
int main() {
 
    FILE* p = fopen("customers.txt","r");
    FILE* out = fopen("output.txt","w"); 
 
    char line[255];
 
    while(fgets(line,255,p) {
 
      char custID[16];
      char phone[16];
      char* delim;
 
      char* ID_start;
      char* ID_end;
 
      delim = strchr(line,'|'); // find 1st delimiter
      ID_start = delim + 1;
      ID_end = strchr(ID_start,'|'); 
 
      *ID_end = '\0'; // null terminate it
 
      // copy to 'custID'
      strcpy(custID,ID_start);
 
      // restore line
      *ID_end = '|';
 
      // now, make DB query based on data in 'custID'
 
      //... get_phone_nr_from_db(custID,phone);
 
      // find the <blank space> (*now* 5th delimiter)
      delim = line;
      for (int i = 0; i < 5; ++i) {
 
        delim = strchr(delim,'|');
        delim++;
      }
 
      memcpy(delim,phone,strlen(phone));
 
      // write new line
      fputs(line,out);
 
    }
 
    fclose(p);
    fclose(out);
 
return 0;
}

Open in new window

0
 
farekatAuthor Commented:
infinity08..i tried it it worked but jkr, youre right...ill just initially create the file with that prefix and then parse it instead of adding the info later...
0
 
Infinity08Commented:
>> infinity08..i tried it it worked

The reason is that when you allocate memory using malloc, the memory is NOT initialized - ie. it contains whatever happend to be there before the malloc call.

So, when you then use strcat to add something to the end of the string, it will look for the first NULL character, and append it there, keeping all the garbage before that NULL character.

Note that calloc does the same as malloc, except that it also initializes the memory to all 0.
0
 
farekatAuthor Commented:
Thank you for your help, I just got a message stating that I need to close this question...this has been wonderful..i learned so much!  Thanks!
0

Featured Post

The new generation of project management tools

With monday.com’s project management tool, you can see what everyone on your team is working in a single glance. Its intuitive dashboards are customizable, so you can create systems that work for you.

  • 11
  • 8
  • 6
  • +1
Tackle projects and never again get stuck behind a technical roadblock.
Join Now