• C

Read file from buffer into array

Hi
I have written a code that reads a file in two passes
In the first pass it counts the number of rows and colum to allocate memory for a 2d array.
After rewind, in the second pass it reads each char of the file and puts them in the array.

The size of the array can varry but is approx( 1000 plus x 360). While the size of each file should be well less than 1MB.

As i need to process several files i was told i could use the buffer  to put it in the array instead of  pasing through the file the second time  as this could possibly reduce computing time.

However, i have no idea on how to modify the code to read form buffer and copy into the dynamically created array.  Any help would me much appreciated. Thx

Ps. i have attached a samlpe text file of small dimensions.
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
 
 
 
int main(int argc, char *argv[])
	{
	FILE *fin, *fout ; 
	int colCnt,rowCnt, col=0,row=0;
	int i,j;
	int**matrix;
	
 
	if(argc!=2)
		{
		printf("Check Arguments\n");
		exit(0);
		}
	if(!(fin=fopen(argv[1],"r")))
		{
		printf("Unable to open file\n");
		exit(1);
		}
	
	while((colCnt=fgetc(fin))!='\n')
		{
		if(colCnt=='\t')
			{
			++col;
			}
		}
	
	row++;
	while((rowCnt=fgetc(fin))!=EOF)
		{
		if(rowCnt=='\n')
			{
			++row;
			}
		}
 
	rewind(fin);
	
	printf("Number of cols = %d ",col);
	printf("Number of rows = %d\n ",row);
 
	matrix =(int**)malloc(row*sizeof(int*));
	if (matrix == NULL)  
    {
        fprintf(stderr, "ERROR - malloc failed: Exiting Program!\n\n");
        exit(EXIT_FAILURE);
    }
 
	for(i=0;i<row;i++)
		{
		matrix[i]=(int*)malloc(col*sizeof(int));
		}
 
	for(i=0;i<row;i++)
		{
		for(j=0;j<col;j++)
			{
			fscanf(fin,"%d",&matrix[i][j]);
			printf("%d ",matrix[i][j]);
			}
		printf("\n");
		}
	fclose(fin);
	free(matrix);
	return 0;		
	}

Open in new window

hist.txt
NaverickAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

evilrixSenior Software Engineer (Avast)Commented:
Sorry... just a slightly off topic response but I thought it might help... a good general purpose allocation strategy is to allocate memory using a power of 2 (2, 4, 8 16, 32 etc). This is a strategy that is very simple to implement and good for general purpose use. You could, rather than implement a buffer, just use realloc to resize your matrix using this strategy. If the matrix is a jagged array (not all rows need to be the same length) even better since you can grow each row as and when needed.

Use this suggestion as you will.. or not :)
0
ahaoCommented:
Firstly, you need reading file content to buffer, then process the buffer instead of file, I've change the code for you.


#include <stdio.h>
#include <stdlib.h>
#include <string.h>
 
int main(int argc, char *argv[])
{
	FILE *fin, *fout ; 
	int colCnt,rowCnt, col=0,row=0;
	int i,j;
	int**matrix;
 
	/// File Size
	int file_size = 0;
	/// Buffer
	char * p_buffer = NULL;
	char * p_begin = NULL;
	char buf[10];
 
	if(argc!=2)
	{
		printf("Check Arguments\n");
		exit(0);
	}
	if(!(fin=fopen(argv[1],"r")))
	{
		printf("Unable to open file\n");
		exit(1);
	}
 
	// Get File Size
	fseek(fin,0,SEEK_END);
	file_size = ftell(fin);
	// alloc buffer
	p_buffer = malloc(file_size+1);
	p_begin = p_buffer;
	// Read in file content
	fseek(fin,0,SEEK_SET);
	fread( p_buffer, 1, file_size, fin );
	*( p_buffer + file_size ) = EOF;
	// Close
	fclose(fin);
 
 
	//while((colCnt=fgetc(fin))!='\n')
	while((colCnt=*p_begin++)!='\n')
	{
		if(colCnt=='\t')
		{
			++col;
		}
	}
 
	row++;
	//while((rowCnt=fgetc(fin))!=EOF)
	while((rowCnt=*p_begin++)!=EOF)
	{
		if(rowCnt=='\n')
		{
			++row;
		}
	}
 
//	rewind(fin);
	// re-point to begining of the buffer
	p_begin = p_buffer;
 
	printf("Number of cols = %d ",col);
	printf("Number of rows = %d\n ",row);
 
	matrix =(int**)malloc(row*sizeof(int*));
	if (matrix == NULL)  
	{
		fprintf(stderr, "ERROR - malloc failed: Exiting Program!\n\n");
		exit(EXIT_FAILURE);
	}
 
	for(i=0;i<row;i++)
	{
		matrix[i]=(int*)malloc(col*sizeof(int));
	}
 
	for(i=0;i<row;i++)
	{
		for(j=0;j<col;j++)
		{
			//fscanf(fin,"%d",&matrix[i][j]);
			matrix[i][j] = strtoul(p_begin,&p_begin,10);
 
			printf("%d ",matrix[i][j]);
		}
		printf("\n");
	}
 
//	fclose(fin);
	/// Free Buffer
	free( p_buffer );
 
	free(matrix);
	return 0;               
}

Open in new window

0
NaverickAuthor Commented:
with respect to your comment...im not sure of the exact method, format annd syntax to allocate a large enough array and then resize it after  all data has been entered into it. All i know is malloc and realloc has to be used.
0
10 Tips to Protect Your Business from Ransomware

Did you know that ransomware is the most widespread, destructive malware in the world today? It accounts for 39% of all security breaches, with ransomware gangsters projected to make $11.5B in profits from online extortion by 2019.

NaverickAuthor Commented:
just seen it...will try it out....but please can u explain to me what is iit that u have done..cos im not too good at this and i would like to understand the code. Thx a lot.
0
ahaoCommented:
OK..

 // Seek to file end and get File Size
        fseek(fin,0,SEEK_END);
        file_size = ftell(fin);
 // alloc a buffer which can contain whole file
        p_buffer = malloc(file_size+1);
// use another pointer, point to the beginning of the file buffer
        p_begin = p_buffer;
// seek to the beginning of the file and read whole file content into the buffer
        fseek(fin,0,SEEK_SET);
        fread( p_buffer, 1, file_size, fin );
        *( p_buffer + file_size ) = EOF;
// close file
        fclose(fin);

And..

// Read from the buffer instead of reading from file
       //while((rowCnt=fgetc(fin))!=EOF)
        while((rowCnt=*p_begin++)!=EOF)

// After first pass,  re-point to begining of the buffer
        p_begin = p_buffer;

// use strtoul to read each integer into the matrix
       //fscanf(fin,"%d",&matrix[i][j]);
       matrix[i][j] = strtoul(p_begin,&p_begin,10);
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
NaverickAuthor Commented:
Ive tried it and offcourse u know it works better than i know. A very frief algorithim for understanding would be great. Thanks a lot. While u r here can i ask you another question.

How can i dynamically create an array and then keep adding rows to it in a loop till required.
Or/and
How can i create a large array an someone mentioned above and later resize it to a smaller one or in other words remove rows.
0
ahaoCommented:
You can use realloc to resize the buffer.
For simplicity, I use one-dimensional array.


	int size = 5;
 
	int * p = (int*)malloc( size * sizeof(int) );
	// now the array size is 5
 
	// add one
	p = (int*)realloc( p, 6 * sizeof(int) );
	// now the array size is 6
 
	// remove one
	p = (int*)realloc( p, 4 * sizeof(int) );
	// now the array size is 4
 
	free( p );

Open in new window

0
NaverickAuthor Commented:
thatx a lot....will get back  to u when i get stuck tryin to implement this
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
C

From novice to tech pro — start learning today.