i'm trying to modify an existing application in such a way that a line appears to be prepended to all the files opened through a function that either loads the file into a buffer or mmaps it depending on it's size.
the function returns a struct that contains the memory adress where the file is mapped and it's length.
only reads are expected, so there is no need to bother about changing file lengths
obviously the buffer case is trivial and is already solved, but i have concerns regarding the mmap case.
the solution i came up with consists in allocating more space than needed using malloc/valloc, perform an mmap fixed to the first page boundary inside the allocated space, write the string before the mmaped first address and return a pointer to the beginning of the string.
i also tried to use an mmap anonymous bigger than needed instead of a valloc but ended up with segfaults.
i'm concerned with performance issues. i'm not a C developper ( this is one of the first C programs i toy with ) but i do program in other languages and can read C and understand what the code does. unfortunately i'm not really confident with myself espetially when it comes to memory management so i'm in need of a deeper understanding of what mmap and malloc do and determine if my solution is workable and/or find a better one.
the program is a custom imap server that i want to toy with in order to migrate away from it. i need to move about 100Tb of data in an unwknown number of files ( i guess around 10^8 ). it compiles on old 32bits debian versions and i'd rather not even attempt to port it to anything newer, or toy with other parts of the code.
here come my not really exhaustive questions
do i need to malloc the whole file's length before performing the mmap, or can i just map a few pages ?
what would be the difference in terms of memory consumption between a regular mmap and my current solution ?
can anybody confirm that the malloc will not actually write anything nor reserve ram space ? ( i'm obviously expecting virtual space to be reserved )
is there a better solution ? maybe something to produce a "glue" buffer that would actually be a buffer continuated in an mmap without that much toying ?
any other comments ? ideas ?
thanks for your time