Writing out a structure to file and using mmap to map it... (confusion with pointers too :()

Maarvaadi
Maarvaadi used Ask the Experts™
on
I have a structure defined as :

struct MyStruct {
    int myVal[10000];
}

In my code segment I have defined and initialized it using :

struct MyStruct *mStruct;
mStruct = (struct MyStruct *) malloc(sizeof(struct MyStruct)); // I have if statements that it does gets malloced.

Once the above is malloced and good to go, I then write it out to a file :

    write(fd, (const void *) & MyStruct, sizeof(struct MyStruct));

I close the fd and check the file. The file is there with the right size.

In a separate process I need to map this file contents to memory so I have :

struct MyStruct *mStruct;

mStruct = (struct MyStruct *) mmap(NULL, (size_t) correctSize, PROT_READ, MAP_FILE | MAP_PRIVATE, fileFD, (off_t)offset); // fileFD is already opened and verified, correct size is the size of this structure, offset is correctly set.

And once I do this and do the following:

printf("Read value = %i\n",mStruct-> myVal[1]);

I get a bogus value and not my actual value I stored.

What am I doing wrong here?
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Top Expert 2009

Commented:
>>     write(fd, (const void *) & MyStruct, sizeof(struct MyStruct));

I assume you meant :

         write(fd, (const void *) mStruct, sizeof(struct MyStruct));


>> In a separate process I need to map this file contents to memory

Is this separate process located on the same machine, and compiled with the same compiler and compiler flags ? Does it also use the same definition of MyStruct ? If not, make sure that the layout of MyStruct is exactly the same in both processes (specifically with regard to the size of an int and the endianness of an int).


>> mStruct = (struct MyStruct *) mmap(NULL, (size_t) correctSize, PROT_READ, MAP_FILE | MAP_PRIVATE, fileFD, (off_t)offset); // fileFD is already

I assume offset is 0 ?
I assume that the return value of mmap is a valid pointer ? ie. mmap didn't fail and return MAP_FAILED ?


>> I get a bogus value and not my actual value I stored.

What was the value you stored ?
What was the value found in the file ? (the bytes that correspond to the value)
What was the value read in the second process ?
Hi Maarvaadi,

For your application, offset should probably be 0.

Just checking, but you're examining myVal[1], not the first element myVal[0].  

Kent

Author

Commented:
Here is a code segment which I have which shows the error I am talking about. Note that I am using a multi dimension array.

In the code below if the array is 36000 x 6000 I can successfully write and read correctly. But what I really need is is a 36000 x 72000 array, when I set the values to that it fails with a EXC_BAD_ACCESS.  
#include <stdio.h>
#include <sys/stat.h>
#include <sys/mman.h>
#include <stdlib.h>

//#define ROWS 36000
//#define COLS 72000
#define ROWS 36000
#define COLS 6000

struct MyStruct {
    int myVal[(int)(ROWS)][(int)(COLS)];
};

void writeFile(void) {
    struct MyStruct *mStruct;
    if ((mStruct = (struct MyStruct *) malloc(sizeof(struct MyStruct))) == NULL) {
        printf("unable to malloc");
        return;
    }

    
    // Initiliaze
    for (int i = 0; i < ROWS; i++)
        for(int j = 0; j < COLS; j++)
            mStruct->myVal[i][j] = i+j;
    
    // Lets write it out
    int fd;
    if ((fd = open("/tmp/myStruct", O_WRONLY | O_APPEND | O_CREAT | O_TRUNC, S_IRUSR | S_IWUSR)) == -1) {
        printf("Unable to open\n");
        return;
    }
    
    //write
    write(fd, (const void*)mStruct, sizeof(struct MyStruct));
    
    // close
    close(fd);

    printf("Value at 200 = %i\n",mStruct->myVal[200][200]);
}

void readFile(void) {
    struct MyStruct *mStruct;
    int fd;
    
    // open the fd
    if ((fd = open("/tmp/myStruct", O_RDONLY)) == -1) {
		perror("Unable to open file");
		exit(-1);
	}
    
    // map
    if ((mStruct = (struct MyStruct *) mmap(NULL, sizeof(struct MyStruct), PROT_READ, MAP_FILE | MAP_PRIVATE, fd, (off_t) 0)) == (struct MyStruct *) -1) {
		perror("Unable to mmap file");
		exit(-1);
	}
    
    printf("Value at 200 = %i\n",mStruct->myVal[200][200]);
    
    close(fd);
}

int main (int argc, const char * argv[]) {
    writeFile();
    
    readFile();
    
    return 0;
}

Open in new window

Success in ‘20 With a Profitable Pricing Strategy

Do you wonder if your IT business is truly profitable or if you should raise your prices? Learn how to calculate your overhead burden using our free interactive tool and use it to determine the right price for your IT services. Start calculating Now!

Author

Commented:
infinity08, I have attached the actual code segment with what I want to do. There seems to be a cap on how much my multidimension array can hold.. or I am missing something above.
Top Expert 2009
Commented:
That's a different error than what you explained earlier :)

The code you just posted looks good (except that you should check the return value of mmap against MAP_FAILED instead of -1)


>> But what I really need is is a 36000 x 72000 array

That would be more than 10GB of data on a 32bit machine. That seems a bit excessive ...


>> it fails with a EXC_BAD_ACCESS.

That refers to an invalid memory access.
Top Expert 2009

Commented:
>> That would be more than 10GB of data on a 32bit machine. That seems a bit excessive ...

Or to make it more explicitly clear : this will not work on a 32bit system. A 32bit value simply cannot store that size.

Author

Commented:
I didn't realize that until later what was going on. Thanks for confirming it.

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial