Solved

Read file from buffer into array

Posted on 2009-07-16
8
1,249 Views
Last Modified: 2012-05-07
Hi
I have written a code that reads a file in two passes
In the first pass it counts the number of rows and colum to allocate memory for a 2d array.
After rewind, in the second pass it reads each char of the file and puts them in the array.

The size of the array can varry but is approx( 1000 plus x 360). While the size of each file should be well less than 1MB.

As i need to process several files i was told i could use the buffer  to put it in the array instead of  pasing through the file the second time  as this could possibly reduce computing time.

However, i have no idea on how to modify the code to read form buffer and copy into the dynamically created array.  Any help would me much appreciated. Thx

Ps. i have attached a samlpe text file of small dimensions.
#include <stdio.h>

#include <stdlib.h>

#include <string.h>

 

 

 

int main(int argc, char *argv[])

	{

	FILE *fin, *fout ; 

	int colCnt,rowCnt, col=0,row=0;

	int i,j;

	int**matrix;

	
 

	if(argc!=2)

		{

		printf("Check Arguments\n");

		exit(0);

		}

	if(!(fin=fopen(argv[1],"r")))

		{

		printf("Unable to open file\n");

		exit(1);

		}

	

	while((colCnt=fgetc(fin))!='\n')

		{

		if(colCnt=='\t')

			{

			++col;

			}

		}

	

	row++;

	while((rowCnt=fgetc(fin))!=EOF)

		{

		if(rowCnt=='\n')

			{

			++row;

			}

		}
 

	rewind(fin);

	

	printf("Number of cols = %d ",col);

	printf("Number of rows = %d\n ",row);
 

	matrix =(int**)malloc(row*sizeof(int*));

	if (matrix == NULL)  

    {

        fprintf(stderr, "ERROR - malloc failed: Exiting Program!\n\n");

        exit(EXIT_FAILURE);

    }
 

	for(i=0;i<row;i++)

		{

		matrix[i]=(int*)malloc(col*sizeof(int));

		}
 

	for(i=0;i<row;i++)

		{

		for(j=0;j<col;j++)

			{

			fscanf(fin,"%d",&matrix[i][j]);

			printf("%d ",matrix[i][j]);

			}

		printf("\n");

		}

	fclose(fin);

	free(matrix);

	return 0;		

	}

Open in new window

hist.txt
0
Comment
Question by:Naverick
  • 4
  • 3
8 Comments
 
LVL 40

Expert Comment

by:evilrix
ID: 24873716
Sorry... just a slightly off topic response but I thought it might help... a good general purpose allocation strategy is to allocate memory using a power of 2 (2, 4, 8 16, 32 etc). This is a strategy that is very simple to implement and good for general purpose use. You could, rather than implement a buffer, just use realloc to resize your matrix using this strategy. If the matrix is a jagged array (not all rows need to be the same length) even better since you can grow each row as and when needed.

Use this suggestion as you will.. or not :)
0
 

Assisted Solution

by:ahao
ahao earned 50 total points
ID: 24874852
Firstly, you need reading file content to buffer, then process the buffer instead of file, I've change the code for you.


#include <stdio.h>

#include <stdlib.h>

#include <string.h>
 

int main(int argc, char *argv[])

{

	FILE *fin, *fout ; 

	int colCnt,rowCnt, col=0,row=0;

	int i,j;

	int**matrix;
 

	/// File Size

	int file_size = 0;

	/// Buffer

	char * p_buffer = NULL;

	char * p_begin = NULL;

	char buf[10];
 

	if(argc!=2)

	{

		printf("Check Arguments\n");

		exit(0);

	}

	if(!(fin=fopen(argv[1],"r")))

	{

		printf("Unable to open file\n");

		exit(1);

	}
 

	// Get File Size

	fseek(fin,0,SEEK_END);

	file_size = ftell(fin);

	// alloc buffer

	p_buffer = malloc(file_size+1);

	p_begin = p_buffer;

	// Read in file content

	fseek(fin,0,SEEK_SET);

	fread( p_buffer, 1, file_size, fin );

	*( p_buffer + file_size ) = EOF;

	// Close

	fclose(fin);
 
 

	//while((colCnt=fgetc(fin))!='\n')

	while((colCnt=*p_begin++)!='\n')

	{

		if(colCnt=='\t')

		{

			++col;

		}

	}
 

	row++;

	//while((rowCnt=fgetc(fin))!=EOF)

	while((rowCnt=*p_begin++)!=EOF)

	{

		if(rowCnt=='\n')

		{

			++row;

		}

	}
 

//	rewind(fin);

	// re-point to begining of the buffer

	p_begin = p_buffer;
 

	printf("Number of cols = %d ",col);

	printf("Number of rows = %d\n ",row);
 

	matrix =(int**)malloc(row*sizeof(int*));

	if (matrix == NULL)  

	{

		fprintf(stderr, "ERROR - malloc failed: Exiting Program!\n\n");

		exit(EXIT_FAILURE);

	}
 

	for(i=0;i<row;i++)

	{

		matrix[i]=(int*)malloc(col*sizeof(int));

	}
 

	for(i=0;i<row;i++)

	{

		for(j=0;j<col;j++)

		{

			//fscanf(fin,"%d",&matrix[i][j]);

			matrix[i][j] = strtoul(p_begin,&p_begin,10);
 

			printf("%d ",matrix[i][j]);

		}

		printf("\n");

	}
 

//	fclose(fin);

	/// Free Buffer

	free( p_buffer );
 

	free(matrix);

	return 0;               

}

Open in new window

0
 

Author Comment

by:Naverick
ID: 24874874
with respect to your comment...im not sure of the exact method, format annd syntax to allocate a large enough array and then resize it after  all data has been entered into it. All i know is malloc and realloc has to be used.
0
 

Author Comment

by:Naverick
ID: 24874892
just seen it...will try it out....but please can u explain to me what is iit that u have done..cos im not too good at this and i would like to understand the code. Thx a lot.
0
Do You Know the 4 Main Threat Actor Types?

Do you know the main threat actor types? Most attackers fall into one of four categories, each with their own favored tactics, techniques, and procedures.

 

Accepted Solution

by:
ahao earned 50 total points
ID: 24874937
OK..

 // Seek to file end and get File Size
        fseek(fin,0,SEEK_END);
        file_size = ftell(fin);
 // alloc a buffer which can contain whole file
        p_buffer = malloc(file_size+1);
// use another pointer, point to the beginning of the file buffer
        p_begin = p_buffer;
// seek to the beginning of the file and read whole file content into the buffer
        fseek(fin,0,SEEK_SET);
        fread( p_buffer, 1, file_size, fin );
        *( p_buffer + file_size ) = EOF;
// close file
        fclose(fin);

And..

// Read from the buffer instead of reading from file
       //while((rowCnt=fgetc(fin))!=EOF)
        while((rowCnt=*p_begin++)!=EOF)

// After first pass,  re-point to begining of the buffer
        p_begin = p_buffer;

// use strtoul to read each integer into the matrix
       //fscanf(fin,"%d",&matrix[i][j]);
       matrix[i][j] = strtoul(p_begin,&p_begin,10);
0
 

Author Comment

by:Naverick
ID: 24874958
Ive tried it and offcourse u know it works better than i know. A very frief algorithim for understanding would be great. Thanks a lot. While u r here can i ask you another question.

How can i dynamically create an array and then keep adding rows to it in a loop till required.
Or/and
How can i create a large array an someone mentioned above and later resize it to a smaller one or in other words remove rows.
0
 

Expert Comment

by:ahao
ID: 24875011
You can use realloc to resize the buffer.
For simplicity, I use one-dimensional array.


	int size = 5;
 

	int * p = (int*)malloc( size * sizeof(int) );

	// now the array size is 5
 

	// add one

	p = (int*)realloc( p, 6 * sizeof(int) );

	// now the array size is 6
 

	// remove one

	p = (int*)realloc( p, 4 * sizeof(int) );

	// now the array size is 4
 

	free( p );

Open in new window

0
 

Author Comment

by:Naverick
ID: 24875039
thatx a lot....will get back  to u when i get stuck tryin to implement this
0

Featured Post

Find Ransomware Secrets With All-Source Analysis

Ransomware has become a major concern for organizations; its prevalence has grown due to past successes achieved by threat actors. While each ransomware variant is different, we’ve seen some common tactics and trends used among the authors of the malware.

Join & Write a Comment

Have you thought about creating an iPhone application (app), but didn't even know where to get started? Here's how: ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ Important pre-programming comments: I’ve never tri…
Summary: This tutorial covers some basics of pointer, pointer arithmetic and function pointer. What is a pointer: A pointer is a variable which holds an address. This address might be address of another variable/address of devices/address of fu…
The goal of this video is to provide viewers with basic examples to understand and use structures in the C programming language.
The goal of this video is to provide viewers with basic examples to understand how to create, access, and change arrays in the C programming language.

743 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

9 Experts available now in Live!

Get 1:1 Help Now