• C

Copy memory locations (example: struct1(4096b) = 4xstruct4(1024b))

This question is a little bit hard to understand I think. So I've to explain it a bit more :-)

Currently I'm writing (yes I do) an operating system. In my mem.c there is the folllowing definition:

#define PAGE_SIZE 4096
unsigned int buffer[256]; // around (1MB / PAGE_SIZE)

I'm using pages for the first 1MB and it works. But the problem is that with every allocation 4096bytes (4k) were allocated. That's too much. So I've the following idea:

typedef struct MEM_LIST
{
      char status;      // free, used, kernel (driver, etc.), moveable, changed, swap
      struct MEM_LIST *start;
      struct MEM_LIST *prev;
      struct MEM_LIST *next;
} MEM_LIST;                  // structure is always 16byte large, even with "__attribute__ ((packed))"  -> align 16

MEM_LIST *ml, *start;

int mem_create_user(int entries)
{
      unsigned int i;
      char * ptr;
      MEM_LIST *tmp, *next;
      
      if (!mm_initialized) return MM_ERROR_NOT_INITIALIZED;
      // reserve kernel space, max. 320kb for allocating 32768 pages à 4096bytes (4GB)
      // checking if space is there, else )!/$&§/§§" :-)
    ptr = (char *)kmalloc(entries*sizeof(MEM_LIST));
    if (!ptr)
    {
          printf("memory: kernel panic: couldn't allocate user space...\n");
          printf("        running in bogus mode...");
          for (;;);
          return -1;      // shouldn't come here
    }
    kfree(ptr);
    printf("memory: Allocating user memory space...");
    ml = (MEM_LIST *)kmalloc(sizeof(MEM_LIST));
    if (ml)
    {
          ml->status = MM_FREE;
          ml->start = ml;
          ml->prev = ml;
          ml->next = ml;
          start = ml;
          for (i=0; i<entries-1; i ++)
          {
                tmp = (MEM_LIST *)kmalloc(sizeof(MEM_LIST));
                if (!tmp)
                {
                      mm_write_last_error(0);                     
                      return -1;
                  }
                     tmp->status = MM_FREE;
                next = ml->next;
                     ml->next = tmp;
                     next->prev =tmp;
                     tmp->next = next;
                     tmp->prev = ml;
            }
            printf("OK");
      }
      return 0;
}

It works quite well for max. 512entries, 'cos kmalloc used 4096bytes each page... Now my question:

??? IS IT POSSIBLE TO ALLOCATE A MEMORY AREA AND COPYING THE MEM_LIST STRUCTURE (13bytes per entry) INTO THE ALLOCATED SPACE ???

Example:
*ptr = MEMLIST;
ptr = ptr + sizeof(MEM_LIST);

COULD IT BE SOLVED?
LVL 3
MathiasIT SpecialistAsked:
Who is Participating?
 
Kent OlsenConnect With a Mentor Data Warehouse Architect / DBACommented:
Hi TDS,

void * pages;

while (entries--)
  *pages++ = new_entry;


In this scheme, 'pages' should probably be defined as a pointer to an object that is a 4K page.  This will make the code a lot easier to manage.

So Yes, you can do it this way.  It's actually quite effective.  Note that you will have effectively taken a contiguous block of memory large enough to store NUM_PAGES pointers.



Kent
0
 
Kent OlsenData Warehouse Architect / DBACommented:
Hi TDS,

I like writing operating systems.  :)  A bit of testosterone laden programming in a Microsoft world, huh?

The short answer to your question is "yes".  If it were me, I'd "lock" some number of pages into memory where you can keep things like your mem_list structure.  (Allowing the "page table" to be paged out makes it pretty tough to access it and do paging!)


Kent
0
 
Kent OlsenData Warehouse Architect / DBACommented:
Hi TDS,

Looking at your code more deeply, I see that you may be attempting something that's not easily workable.  You can't reserved the first 16 bytes of each page for system usage and trust the application to behave.  It will be tough to implement an addressing scheme that skips the first 16 bytes from the application but not from non-privileged mode kernel.

Your "page table" needs to be an array that you can index.  When you allocate pages, the memory within the page belongs to the the application/process to which it is assigned.  The length of the page table is defined at boot time by (MEMORY_SIZE / PAGE SIZE).


Kent
0
The Firewall Audit Checklist

Preparing for a firewall audit today is almost impossible.
AlgoSec, together with some of the largest global organizations and auditors, has created a checklist to follow when preparing for your firewall audit. Simplify risk mitigation while staying compliant all of the time!

 
MathiasIT SpecialistAuthor Commented:
Hi Kdo,

You said the the saize of the page taable is defined @ boot time, right? Okay, but how define an array with dynamic creation? I want to be able to have a kernel and a user memory space. The kernel memory is from 0-1 MB. In this area it should be a table, created with a linked list, which manage the pages in the user memory space.
Can you explain your way of thinking ?
0
 
Kent OlsenData Warehouse Architect / DBACommented:
Hi TDS,

Initializing your environment can be one of the trickiest parts of loading an operating system.  As the system gets bigger and more complex, you often have to do more to keep the boot process "out of the way".  Also, once the system is booted, you no longer need the code that was used to boot the system.

A pretty easy "first cut" is to define your static memory like this:

Pointer Region
Kernel Code
Tables

The Pointer Region is just that.  It contains the address of all of your static structures, like page tables, memory chain pointer, device pointers, etc.



#define PointTableStart 1  /* Start pointers at word 1.  Word 0 equalling zero has a lot of advantages later on  */

typdef struct
{
  UInt32 PageTableOffset;            // Offset from byte 0 where the page table starts
  UInt32 StackOffset;                // Address of the kernel's stack

/*  Other pointers to system tables  */

} StaticPointers_t;


Now once you've loaded the kernel, the kernel startup code will populate the pointer region based on the free memory after the last word of the kernel.


Kent
0
 
MathiasIT SpecialistAuthor Commented:
/*
|-------------------------------|
| 4 MB - max. MB user memory      |
|-------------------------------|
| 3MB - 4MB kernel stack        |
|-------------------------------|
| 2MB - 3MB kernel heap                  |
|-------------------------------|
| 1MB - 2MB kernel code                  |
|                                                |
|- - - - - - - - - - - - - - - -|
| 1MB - 1MB + 0x1000 reserved      |
|-------------------------------|
| 0MB - 1MB real mode                  |
|-------------------------------|
*/

You'll see that all regions needed are defined. The GDT is set up with this values. All things are working. The only thing I want to do i that in the kernel memory the memory map for the whole system is stored. Each entry should point to a 4K page. In my mind I had an idea:

void * pages;

while (entries--)
  *pages++ = new_entry;

Is that the right way or is there something mysterious?
0
 
MathiasIT SpecialistAuthor Commented:
Another chance to solve this problem is maybe the following code...

unsigned int buffer[] = { 0, };

But how many bytes are reserved or is it a pointer?
0
 
MathiasIT SpecialistAuthor Commented:
Can someone post me a valid example?
0
 
Kent OlsenData Warehouse Architect / DBACommented:

#define PAGE_SIZE 4096
#define NUM_PAGES 256

typdef struct
{
  char Contents[PAGE_SIZE];
}  Page_t;


In your low-memory pointers, reserve a word for the table address:

Page_t **PageTablePtr;


When you're building the tables, populate this field.

void *FreeWord;  /*  Table to the first free word in memory.  Must be higher address the the executable kernel  */

  PageTablePtr = (Page_t **)FreeWord;
  FreeWord += (NUM_PAGES * sizeof (PageTablePtr *));
  memset (*PageTablePtr, 0, NUM_PAGES * sizeof (PageTablePtr *));



Kent

0
 
Avik DasguptaConnect With a Mentor Commented:
>> Example:
>> *ptr = MEMLIST;
>> ptr = ptr + sizeof(MEM_LIST);

>>COULD IT BE SOLVED?

I am not sure but u can try the memcpy function for this.

Avik.
0
 
MathiasIT SpecialistAuthor Commented:
Kdo: That isn't really what I want, cause there are many structures in the allocated memory, but no structure has the ability like a double linked list (referencing to prev and next). That's my problem... To allocate a memory and define some structures in it is no problem, the problem is the linked list...
Avik77: Sorry, couldn't try it yet. Will probe it later :-)
0
 
Kent OlsenData Warehouse Architect / DBACommented:
Hi TDS,

It might not be what you want now, but I believe that it is in line with what you need.

The issue that you've raised isn't about managing memory, per se.  It is about managing memory pages.  (You seem to be wavering between the techniques needed to manage memory within an application and the techniques used to manage pages with the operating system.)  At the operating system level, the smallest amount of memory that can be assigned is a page.  The application is oblivious to the page structure.  All it knows is that there is some quantity of memory that it will distribute to itself as it needs it.

In a virtual memory operating system (one where memory is paged) each page of memory is shadowed by a disk location.  (The shadow can be in the page file or in a memory mapped file.)  For each page of "real" memory, control information must be kept.  You've begun this process with the 'Status' variable in your structure.  You'll also need to keep track of the disk address that is the "permanent" copy of the contents of the page.  This "permanent" copy is usually temporary as the location is in the page file.  (I've mentioned this here to help you visualize some of the management issues.)

Your operating system knows how much memory it has to manage.  One step of the boot sequence is to determine how much memory is available.  Knowing how much memory there is and how large a page is means that the O/S knows how many pages it has.

Given that you now exactly how much memory the page table require, why would you scatter the table throughout memory and manage it with a pointer chain?  You can always accomplish your goal of "linking" related pages with an index in the page table entry structure.

Kent
0
 
MathiasIT SpecialistAuthor Commented:
Okay, if I understand you know... the right way is to use 4k pages for the whole memory. For Example pages with a size of 1MB would address a 4GB address space. But the pages have to be static... isn't it silly to have 1MB for paging even you don't have 4GB RAM?
0
 
Kent OlsenData Warehouse Architect / DBACommented:
Hi TDS,

>> But the pages have to be static...

Some pages do need to be static.  An easy way to implement this is to "lock" the pages in low memory that are the system pointers, the kernel, and the system tables.  Pages at addresses higher than this are available for "normal" system/application use.


>> isn't it silly to have 1MB for paging even you don't have 4GB RAM?

If you're talking about the size of the page table, yes it is.  You know how many pages are in the system (MEMORY_SIZE / PAGE_SIZE) so you only need to make the table large enough to contain an entry for each page.

  (MEMORY_SIZE / PAGE_SIZE) * sizeof (PageTableEntry_t);


Kent
0
 
MathiasIT SpecialistAuthor Commented:
I accepted two answers because both of them pointed me the right way.

Kdo: Your comments were helpful to understand how I can make it. And I've made it :-)
My solution is a melting pott of the given comment and ma tries to solve the problem. No, I can address 4GB with about 500kb allocated memory (including special informations, etc.).

THANX A LOT !!!
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.