Want to win a PS4? Go Premium and enter to win our High-Tech Treats giveaway. Enter to Win

x
?
Solved

Read file from buffer into array

Posted on 2009-07-16
8
Medium Priority
?
1,276 Views
Last Modified: 2012-05-07
Hi
I have written a code that reads a file in two passes
In the first pass it counts the number of rows and colum to allocate memory for a 2d array.
After rewind, in the second pass it reads each char of the file and puts them in the array.

The size of the array can varry but is approx( 1000 plus x 360). While the size of each file should be well less than 1MB.

As i need to process several files i was told i could use the buffer  to put it in the array instead of  pasing through the file the second time  as this could possibly reduce computing time.

However, i have no idea on how to modify the code to read form buffer and copy into the dynamically created array.  Any help would me much appreciated. Thx

Ps. i have attached a samlpe text file of small dimensions.
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
 
 
 
int main(int argc, char *argv[])
	{
	FILE *fin, *fout ; 
	int colCnt,rowCnt, col=0,row=0;
	int i,j;
	int**matrix;
	
 
	if(argc!=2)
		{
		printf("Check Arguments\n");
		exit(0);
		}
	if(!(fin=fopen(argv[1],"r")))
		{
		printf("Unable to open file\n");
		exit(1);
		}
	
	while((colCnt=fgetc(fin))!='\n')
		{
		if(colCnt=='\t')
			{
			++col;
			}
		}
	
	row++;
	while((rowCnt=fgetc(fin))!=EOF)
		{
		if(rowCnt=='\n')
			{
			++row;
			}
		}
 
	rewind(fin);
	
	printf("Number of cols = %d ",col);
	printf("Number of rows = %d\n ",row);
 
	matrix =(int**)malloc(row*sizeof(int*));
	if (matrix == NULL)  
    {
        fprintf(stderr, "ERROR - malloc failed: Exiting Program!\n\n");
        exit(EXIT_FAILURE);
    }
 
	for(i=0;i<row;i++)
		{
		matrix[i]=(int*)malloc(col*sizeof(int));
		}
 
	for(i=0;i<row;i++)
		{
		for(j=0;j<col;j++)
			{
			fscanf(fin,"%d",&matrix[i][j]);
			printf("%d ",matrix[i][j]);
			}
		printf("\n");
		}
	fclose(fin);
	free(matrix);
	return 0;		
	}

Open in new window

hist.txt
0
Comment
Question by:Naverick
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 4
  • 3
8 Comments
 
LVL 40

Expert Comment

by:evilrix
ID: 24873716
Sorry... just a slightly off topic response but I thought it might help... a good general purpose allocation strategy is to allocate memory using a power of 2 (2, 4, 8 16, 32 etc). This is a strategy that is very simple to implement and good for general purpose use. You could, rather than implement a buffer, just use realloc to resize your matrix using this strategy. If the matrix is a jagged array (not all rows need to be the same length) even better since you can grow each row as and when needed.

Use this suggestion as you will.. or not :)
0
 

Assisted Solution

by:ahao
ahao earned 200 total points
ID: 24874852
Firstly, you need reading file content to buffer, then process the buffer instead of file, I've change the code for you.


#include <stdio.h>
#include <stdlib.h>
#include <string.h>
 
int main(int argc, char *argv[])
{
	FILE *fin, *fout ; 
	int colCnt,rowCnt, col=0,row=0;
	int i,j;
	int**matrix;
 
	/// File Size
	int file_size = 0;
	/// Buffer
	char * p_buffer = NULL;
	char * p_begin = NULL;
	char buf[10];
 
	if(argc!=2)
	{
		printf("Check Arguments\n");
		exit(0);
	}
	if(!(fin=fopen(argv[1],"r")))
	{
		printf("Unable to open file\n");
		exit(1);
	}
 
	// Get File Size
	fseek(fin,0,SEEK_END);
	file_size = ftell(fin);
	// alloc buffer
	p_buffer = malloc(file_size+1);
	p_begin = p_buffer;
	// Read in file content
	fseek(fin,0,SEEK_SET);
	fread( p_buffer, 1, file_size, fin );
	*( p_buffer + file_size ) = EOF;
	// Close
	fclose(fin);
 
 
	//while((colCnt=fgetc(fin))!='\n')
	while((colCnt=*p_begin++)!='\n')
	{
		if(colCnt=='\t')
		{
			++col;
		}
	}
 
	row++;
	//while((rowCnt=fgetc(fin))!=EOF)
	while((rowCnt=*p_begin++)!=EOF)
	{
		if(rowCnt=='\n')
		{
			++row;
		}
	}
 
//	rewind(fin);
	// re-point to begining of the buffer
	p_begin = p_buffer;
 
	printf("Number of cols = %d ",col);
	printf("Number of rows = %d\n ",row);
 
	matrix =(int**)malloc(row*sizeof(int*));
	if (matrix == NULL)  
	{
		fprintf(stderr, "ERROR - malloc failed: Exiting Program!\n\n");
		exit(EXIT_FAILURE);
	}
 
	for(i=0;i<row;i++)
	{
		matrix[i]=(int*)malloc(col*sizeof(int));
	}
 
	for(i=0;i<row;i++)
	{
		for(j=0;j<col;j++)
		{
			//fscanf(fin,"%d",&matrix[i][j]);
			matrix[i][j] = strtoul(p_begin,&p_begin,10);
 
			printf("%d ",matrix[i][j]);
		}
		printf("\n");
	}
 
//	fclose(fin);
	/// Free Buffer
	free( p_buffer );
 
	free(matrix);
	return 0;               
}

Open in new window

0
 

Author Comment

by:Naverick
ID: 24874874
with respect to your comment...im not sure of the exact method, format annd syntax to allocate a large enough array and then resize it after  all data has been entered into it. All i know is malloc and realloc has to be used.
0
Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

 

Author Comment

by:Naverick
ID: 24874892
just seen it...will try it out....but please can u explain to me what is iit that u have done..cos im not too good at this and i would like to understand the code. Thx a lot.
0
 

Accepted Solution

by:
ahao earned 200 total points
ID: 24874937
OK..

 // Seek to file end and get File Size
        fseek(fin,0,SEEK_END);
        file_size = ftell(fin);
 // alloc a buffer which can contain whole file
        p_buffer = malloc(file_size+1);
// use another pointer, point to the beginning of the file buffer
        p_begin = p_buffer;
// seek to the beginning of the file and read whole file content into the buffer
        fseek(fin,0,SEEK_SET);
        fread( p_buffer, 1, file_size, fin );
        *( p_buffer + file_size ) = EOF;
// close file
        fclose(fin);

And..

// Read from the buffer instead of reading from file
       //while((rowCnt=fgetc(fin))!=EOF)
        while((rowCnt=*p_begin++)!=EOF)

// After first pass,  re-point to begining of the buffer
        p_begin = p_buffer;

// use strtoul to read each integer into the matrix
       //fscanf(fin,"%d",&matrix[i][j]);
       matrix[i][j] = strtoul(p_begin,&p_begin,10);
0
 

Author Comment

by:Naverick
ID: 24874958
Ive tried it and offcourse u know it works better than i know. A very frief algorithim for understanding would be great. Thanks a lot. While u r here can i ask you another question.

How can i dynamically create an array and then keep adding rows to it in a loop till required.
Or/and
How can i create a large array an someone mentioned above and later resize it to a smaller one or in other words remove rows.
0
 

Expert Comment

by:ahao
ID: 24875011
You can use realloc to resize the buffer.
For simplicity, I use one-dimensional array.


	int size = 5;
 
	int * p = (int*)malloc( size * sizeof(int) );
	// now the array size is 5
 
	// add one
	p = (int*)realloc( p, 6 * sizeof(int) );
	// now the array size is 6
 
	// remove one
	p = (int*)realloc( p, 4 * sizeof(int) );
	// now the array size is 4
 
	free( p );

Open in new window

0
 

Author Comment

by:Naverick
ID: 24875039
thatx a lot....will get back  to u when i get stuck tryin to implement this
0

Featured Post

Concerto's Cloud Advisory Services

Want to avoid the missteps to gaining all the benefits of the cloud? Learn more about the different assessment options from our Cloud Advisory team.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

An Outlet in Cocoa is a persistent reference to a GUI control; it connects a property (a variable) to a control.  For example, it is common to create an Outlet for the text field GUI control and change the text that appears in this field via that Ou…
Windows programmers of the C/C++ variety, how many of you realise that since Window 9x Microsoft has been lying to you about what constitutes Unicode (http://en.wikipedia.org/wiki/Unicode)? They will have you believe that Unicode requires you to use…
Video by: Grant
The goal of this video is to provide viewers with basic examples to understand and use nested-loops in the C programming language.
The goal of this video is to provide viewers with basic examples to understand and use conditional statements in the C programming language.

610 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question