Solved

Copy memory locations (example: struct1(4096b) = 4xstruct4(1024b))

Posted on 2004-04-09
15
357 Views
Last Modified: 2011-09-20
This question is a little bit hard to understand I think. So I've to explain it a bit more :-)

Currently I'm writing (yes I do) an operating system. In my mem.c there is the folllowing definition:

#define PAGE_SIZE 4096
unsigned int buffer[256]; // around (1MB / PAGE_SIZE)

I'm using pages for the first 1MB and it works. But the problem is that with every allocation 4096bytes (4k) were allocated. That's too much. So I've the following idea:

typedef struct MEM_LIST
{
      char status;      // free, used, kernel (driver, etc.), moveable, changed, swap
      struct MEM_LIST *start;
      struct MEM_LIST *prev;
      struct MEM_LIST *next;
} MEM_LIST;                  // structure is always 16byte large, even with "__attribute__ ((packed))"  -> align 16

MEM_LIST *ml, *start;

int mem_create_user(int entries)
{
      unsigned int i;
      char * ptr;
      MEM_LIST *tmp, *next;
      
      if (!mm_initialized) return MM_ERROR_NOT_INITIALIZED;
      // reserve kernel space, max. 320kb for allocating 32768 pages à 4096bytes (4GB)
      // checking if space is there, else )!/$&§/§§" :-)
    ptr = (char *)kmalloc(entries*sizeof(MEM_LIST));
    if (!ptr)
    {
          printf("memory: kernel panic: couldn't allocate user space...\n");
          printf("        running in bogus mode...");
          for (;;);
          return -1;      // shouldn't come here
    }
    kfree(ptr);
    printf("memory: Allocating user memory space...");
    ml = (MEM_LIST *)kmalloc(sizeof(MEM_LIST));
    if (ml)
    {
          ml->status = MM_FREE;
          ml->start = ml;
          ml->prev = ml;
          ml->next = ml;
          start = ml;
          for (i=0; i<entries-1; i ++)
          {
                tmp = (MEM_LIST *)kmalloc(sizeof(MEM_LIST));
                if (!tmp)
                {
                      mm_write_last_error(0);                     
                      return -1;
                  }
                     tmp->status = MM_FREE;
                next = ml->next;
                     ml->next = tmp;
                     next->prev =tmp;
                     tmp->next = next;
                     tmp->prev = ml;
            }
            printf("OK");
      }
      return 0;
}

It works quite well for max. 512entries, 'cos kmalloc used 4096bytes each page... Now my question:

??? IS IT POSSIBLE TO ALLOCATE A MEMORY AREA AND COPYING THE MEM_LIST STRUCTURE (13bytes per entry) INTO THE ALLOCATED SPACE ???

Example:
*ptr = MEMLIST;
ptr = ptr + sizeof(MEM_LIST);

COULD IT BE SOLVED?
0
Comment
Question by:Mathias
  • 7
  • 7
15 Comments
 
LVL 45

Expert Comment

by:Kdo
Comment Utility
Hi TDS,

I like writing operating systems.  :)  A bit of testosterone laden programming in a Microsoft world, huh?

The short answer to your question is "yes".  If it were me, I'd "lock" some number of pages into memory where you can keep things like your mem_list structure.  (Allowing the "page table" to be paged out makes it pretty tough to access it and do paging!)


Kent
0
 
LVL 45

Expert Comment

by:Kdo
Comment Utility
Hi TDS,

Looking at your code more deeply, I see that you may be attempting something that's not easily workable.  You can't reserved the first 16 bytes of each page for system usage and trust the application to behave.  It will be tough to implement an addressing scheme that skips the first 16 bytes from the application but not from non-privileged mode kernel.

Your "page table" needs to be an array that you can index.  When you allocate pages, the memory within the page belongs to the the application/process to which it is assigned.  The length of the page table is defined at boot time by (MEMORY_SIZE / PAGE SIZE).


Kent
0
 
LVL 3

Author Comment

by:Mathias
Comment Utility
Hi Kdo,

You said the the saize of the page taable is defined @ boot time, right? Okay, but how define an array with dynamic creation? I want to be able to have a kernel and a user memory space. The kernel memory is from 0-1 MB. In this area it should be a table, created with a linked list, which manage the pages in the user memory space.
Can you explain your way of thinking ?
0
 
LVL 45

Expert Comment

by:Kdo
Comment Utility
Hi TDS,

Initializing your environment can be one of the trickiest parts of loading an operating system.  As the system gets bigger and more complex, you often have to do more to keep the boot process "out of the way".  Also, once the system is booted, you no longer need the code that was used to boot the system.

A pretty easy "first cut" is to define your static memory like this:

Pointer Region
Kernel Code
Tables

The Pointer Region is just that.  It contains the address of all of your static structures, like page tables, memory chain pointer, device pointers, etc.



#define PointTableStart 1  /* Start pointers at word 1.  Word 0 equalling zero has a lot of advantages later on  */

typdef struct
{
  UInt32 PageTableOffset;            // Offset from byte 0 where the page table starts
  UInt32 StackOffset;                // Address of the kernel's stack

/*  Other pointers to system tables  */

} StaticPointers_t;


Now once you've loaded the kernel, the kernel startup code will populate the pointer region based on the free memory after the last word of the kernel.


Kent
0
 
LVL 3

Author Comment

by:Mathias
Comment Utility
/*
|-------------------------------|
| 4 MB - max. MB user memory      |
|-------------------------------|
| 3MB - 4MB kernel stack        |
|-------------------------------|
| 2MB - 3MB kernel heap                  |
|-------------------------------|
| 1MB - 2MB kernel code                  |
|                                                |
|- - - - - - - - - - - - - - - -|
| 1MB - 1MB + 0x1000 reserved      |
|-------------------------------|
| 0MB - 1MB real mode                  |
|-------------------------------|
*/

You'll see that all regions needed are defined. The GDT is set up with this values. All things are working. The only thing I want to do i that in the kernel memory the memory map for the whole system is stored. Each entry should point to a 4K page. In my mind I had an idea:

void * pages;

while (entries--)
  *pages++ = new_entry;

Is that the right way or is there something mysterious?
0
 
LVL 3

Author Comment

by:Mathias
Comment Utility
Another chance to solve this problem is maybe the following code...

unsigned int buffer[] = { 0, };

But how many bytes are reserved or is it a pointer?
0
 
LVL 45

Accepted Solution

by:
Kdo earned 350 total points
Comment Utility
Hi TDS,

void * pages;

while (entries--)
  *pages++ = new_entry;


In this scheme, 'pages' should probably be defined as a pointer to an object that is a 4K page.  This will make the code a lot easier to manage.

So Yes, you can do it this way.  It's actually quite effective.  Note that you will have effectively taken a contiguous block of memory large enough to store NUM_PAGES pointers.



Kent
0
Threat Intelligence Starter Resources

Integrating threat intelligence can be challenging, and not all companies are ready. These resources can help you build awareness and prepare for defense.

 
LVL 3

Author Comment

by:Mathias
Comment Utility
Can someone post me a valid example?
0
 
LVL 45

Expert Comment

by:Kdo
Comment Utility

#define PAGE_SIZE 4096
#define NUM_PAGES 256

typdef struct
{
  char Contents[PAGE_SIZE];
}  Page_t;


In your low-memory pointers, reserve a word for the table address:

Page_t **PageTablePtr;


When you're building the tables, populate this field.

void *FreeWord;  /*  Table to the first free word in memory.  Must be higher address the the executable kernel  */

  PageTablePtr = (Page_t **)FreeWord;
  FreeWord += (NUM_PAGES * sizeof (PageTablePtr *));
  memset (*PageTablePtr, 0, NUM_PAGES * sizeof (PageTablePtr *));



Kent

0
 
LVL 2

Assisted Solution

by:Avik77
Avik77 earned 150 total points
Comment Utility
>> Example:
>> *ptr = MEMLIST;
>> ptr = ptr + sizeof(MEM_LIST);

>>COULD IT BE SOLVED?

I am not sure but u can try the memcpy function for this.

Avik.
0
 
LVL 3

Author Comment

by:Mathias
Comment Utility
Kdo: That isn't really what I want, cause there are many structures in the allocated memory, but no structure has the ability like a double linked list (referencing to prev and next). That's my problem... To allocate a memory and define some structures in it is no problem, the problem is the linked list...
Avik77: Sorry, couldn't try it yet. Will probe it later :-)
0
 
LVL 45

Expert Comment

by:Kdo
Comment Utility
Hi TDS,

It might not be what you want now, but I believe that it is in line with what you need.

The issue that you've raised isn't about managing memory, per se.  It is about managing memory pages.  (You seem to be wavering between the techniques needed to manage memory within an application and the techniques used to manage pages with the operating system.)  At the operating system level, the smallest amount of memory that can be assigned is a page.  The application is oblivious to the page structure.  All it knows is that there is some quantity of memory that it will distribute to itself as it needs it.

In a virtual memory operating system (one where memory is paged) each page of memory is shadowed by a disk location.  (The shadow can be in the page file or in a memory mapped file.)  For each page of "real" memory, control information must be kept.  You've begun this process with the 'Status' variable in your structure.  You'll also need to keep track of the disk address that is the "permanent" copy of the contents of the page.  This "permanent" copy is usually temporary as the location is in the page file.  (I've mentioned this here to help you visualize some of the management issues.)

Your operating system knows how much memory it has to manage.  One step of the boot sequence is to determine how much memory is available.  Knowing how much memory there is and how large a page is means that the O/S knows how many pages it has.

Given that you now exactly how much memory the page table require, why would you scatter the table throughout memory and manage it with a pointer chain?  You can always accomplish your goal of "linking" related pages with an index in the page table entry structure.

Kent
0
 
LVL 3

Author Comment

by:Mathias
Comment Utility
Okay, if I understand you know... the right way is to use 4k pages for the whole memory. For Example pages with a size of 1MB would address a 4GB address space. But the pages have to be static... isn't it silly to have 1MB for paging even you don't have 4GB RAM?
0
 
LVL 45

Expert Comment

by:Kdo
Comment Utility
Hi TDS,

>> But the pages have to be static...

Some pages do need to be static.  An easy way to implement this is to "lock" the pages in low memory that are the system pointers, the kernel, and the system tables.  Pages at addresses higher than this are available for "normal" system/application use.


>> isn't it silly to have 1MB for paging even you don't have 4GB RAM?

If you're talking about the size of the page table, yes it is.  You know how many pages are in the system (MEMORY_SIZE / PAGE_SIZE) so you only need to make the table large enough to contain an entry for each page.

  (MEMORY_SIZE / PAGE_SIZE) * sizeof (PageTableEntry_t);


Kent
0
 
LVL 3

Author Comment

by:Mathias
Comment Utility
I accepted two answers because both of them pointed me the right way.

Kdo: Your comments were helpful to understand how I can make it. And I've made it :-)
My solution is a melting pott of the given comment and ma tries to solve the problem. No, I can address 4GB with about 500kb allocated memory (including special informations, etc.).

THANX A LOT !!!
0

Featured Post

How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

Join & Write a Comment

Have you thought about creating an iPhone application (app), but didn't even know where to get started? Here's how: ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ Important pre-programming comments: I’ve never tri…
Preface I don't like visual development tools that are supposed to write a program for me. Even if it is Xcode and I can use Interface Builder. Yes, it is a perfect tool and has helped me a lot, mainly, in the beginning, when my programs were small…
Video by: Grant
The goal of this video is to provide viewers with basic examples to understand and use for-loops in the C programming language.
The goal of this video is to provide viewers with basic examples to understand how to create, access, and change arrays in the C programming language.

728 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now