Learn how to a build a cloud-first strategyRegister Now

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 182
  • Last Modified:

Pre-loading files into main memory (Platform: Linux)

I am trying to pre-load a set of files (images to be displayed as a short movie) into main memory before they need to be displayed, so that once loaded disk accesses won't be necessary.  I could simply read each file, but it seems like I should just be able to request one byte from each page the file covers in order to have the whole file paged in.  Does this make sense?  

I'm not sure of the best method of going about this (or if there is already a good implementation of this?), so comments or suggestions are welcome.

AJ
0
aratani
Asked:
aratani
  • 5
  • 2
1 Solution
 
grg99Commented:
It depends on the subtle algorithms in your OS's virtual memory system.   Actually, seeking to a new page probablly takes longer than reading the whole page, so there may be little or no or negative benefit from all that "optimization".
0
 
AxterCommented:
You can use POSIX functions to map a file to memory.
0
 
AxterCommented:
By mapping a file to memory, you let the OS handle the work of reading in the file.
When you map it to memory, you end up with a pointer that points to the contents of the file.

For more info, do a man on the following POSIX functions:
mmap
munmap

Check out the following link:
http://code.axter.com/mapfile2mem.cpp

The above link has cross platform code for Win32 and POSIX.  Ignore code in Win32 section.

Here's some example code :
int m_MapHandle = open(m_NameOfMyMapView, O_RDWR, 0);
struct stat sbuf;
fstat(m_MapHandle, &sbuf);
char* m_Buffer = (char*)mmap(NULL, sbuf.st_size, PROT_READ|PROT_WRITE, MAP_SHARED, m_MapHandle , 0);
0
What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

 
AxterCommented:
FYI:
The above code needs the following headers:
#include <unistd.h>
#include <sys/types.h>
#include <sys/mman.h>
#include <sys/fcntl.h>
#include <sys/stat.h>
0
 
grg99Commented:
> By mapping a file to memory, you let the OS handle the work of reading in the file.


>When you map it to memory, you end up with a pointer that points to the contents of the file.

Well, it's usually a "lazy" pointer.  The file isnt actually read until you touch an address in the mapped area.

Which still leaves his main question unanswered, is it better to touch every byte, or is it enough to touch one byte every page (whatever a page is).

0
 
AxterCommented:
>>Which still leaves his main question unanswered, is it better to touch every byte, or is it enough to touch one byte every page (whatever a page is).

If the file is mapped to memory, it will greatly increase performance compared to reading the file using standard C++ file methods.

I would recommend reading the first and last byte.
0
 
manav_mathurCommented:
>>Which still leaves his main question unanswered, is it better to touch every byte, or is it enough to touch one byte every page

wont page-thrashing still occur??
0
 
AxterCommented:
recommend split between grg99 and Axter
0

Featured Post

Free Tool: Port Scanner

Check which ports are open to the outside world. Helps make sure that your firewall rules are working as intended.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

  • 5
  • 2
Tackle projects and never again get stuck behind a technical roadblock.
Join Now