Pre-loading files into main memory (Platform: Linux)

I am trying to pre-load a set of files (images to be displayed as a short movie) into main memory before they need to be displayed, so that once loaded disk accesses won't be necessary.  I could simply read each file, but it seems like I should just be able to request one byte from each page the file covers in order to have the whole file paged in.  Does this make sense?  

I'm not sure of the best method of going about this (or if there is already a good implementation of this?), so comments or suggestions are welcome.

AJ
LVL 4
arataniAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

grg99Commented:
It depends on the subtle algorithms in your OS's virtual memory system.   Actually, seeking to a new page probablly takes longer than reading the whole page, so there may be little or no or negative benefit from all that "optimization".
0
AxterCommented:
You can use POSIX functions to map a file to memory.
0
AxterCommented:
By mapping a file to memory, you let the OS handle the work of reading in the file.
When you map it to memory, you end up with a pointer that points to the contents of the file.

For more info, do a man on the following POSIX functions:
mmap
munmap

Check out the following link:
http://code.axter.com/mapfile2mem.cpp

The above link has cross platform code for Win32 and POSIX.  Ignore code in Win32 section.

Here's some example code :
int m_MapHandle = open(m_NameOfMyMapView, O_RDWR, 0);
struct stat sbuf;
fstat(m_MapHandle, &sbuf);
char* m_Buffer = (char*)mmap(NULL, sbuf.st_size, PROT_READ|PROT_WRITE, MAP_SHARED, m_MapHandle , 0);
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Cloud Class® Course: CompTIA Healthcare IT Tech

This course will help prep you to earn the CompTIA Healthcare IT Technician certification showing that you have the knowledge and skills needed to succeed in installing, managing, and troubleshooting IT systems in medical and clinical settings.

AxterCommented:
FYI:
The above code needs the following headers:
#include <unistd.h>
#include <sys/types.h>
#include <sys/mman.h>
#include <sys/fcntl.h>
#include <sys/stat.h>
0
grg99Commented:
> By mapping a file to memory, you let the OS handle the work of reading in the file.


>When you map it to memory, you end up with a pointer that points to the contents of the file.

Well, it's usually a "lazy" pointer.  The file isnt actually read until you touch an address in the mapped area.

Which still leaves his main question unanswered, is it better to touch every byte, or is it enough to touch one byte every page (whatever a page is).

0
AxterCommented:
>>Which still leaves his main question unanswered, is it better to touch every byte, or is it enough to touch one byte every page (whatever a page is).

If the file is mapped to memory, it will greatly increase performance compared to reading the file using standard C++ file methods.

I would recommend reading the first and last byte.
0
manav_mathurCommented:
>>Which still leaves his main question unanswered, is it better to touch every byte, or is it enough to touch one byte every page

wont page-thrashing still occur??
0
AxterCommented:
recommend split between grg99 and Axter
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
C++

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.