• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 608
  • Last Modified:

RAM Memory addressing scheme of data

Hi fellows,

As a DBA, I would like to get a more precise idea of how is data (an array used for calculations for instance) represented in RAM at run time.   I know that RAM has an absolute physical adressing scheme based on linear hex indexes.  Should I consider that all values are referenced directly or are determined through offsets using a 2 dimension adressing scheme (RowAdresse/ColumnAdress) plugged onto memory bus.  I am not sure that makes sense.  How are ColumnAdresses, RowAddresses and Memory Blocks Adresses exactly used for in RAM?  Could anybody illustrate an explanation through an example using an array of values for instance....Thanks...
0
Racim BOUDJAKDJI
Asked:
Racim BOUDJAKDJI
  • 5
  • 5
1 Solution
 
jhanceCommented:
What you are asking about it one of the closestly held secrets of any daytabase application.  How well the database performs depends to a great extent on how well they organize things in and make use of memory.

You might review some open-source database system like MySQL or review some research papers on this topic.  But as far as commercial databases like MS SQL, Oracle, DB/2, etc. I doubt you'll find any mention of it in their publications.
0
 
Racim BOUDJAKDJIDatabase Architect - Dba - Data ScientistAuthor Commented:
What about physical level.  I am not interested with logical level.
0
 
Gary CaseRetiredCommented:
This should give you a good idea of the physical characteristics of a RAM array and how it is addressed:
http://www.corsairmicro.com/memory_basics/153707/index.html
0
Concerto's Cloud Advisory Services

Want to avoid the missteps to gaining all the benefits of the cloud? Learn more about the different assessment options from our Cloud Advisory team.

 
Gary CaseRetiredCommented:
... there are, however, two important elements related to using large arrays in databases that don't really depend on the physical RAM array (except, of course, the bandwidth of the memory interface is important):

(1)  with modern CPU's with large Level 2 caches, performance is greatly impacted by the percentage of cache hits during code execution => this is where databases are very tight-lipped regarding just how they structure their indices, etc. (as jhance noted); because if they are structured so typical access will result in high cache hit %'s they will perform better.   Note that external memory bandwidth makes virtually NO difference with in-cache accesses (although it does, of course, make a big difference when there's a cache miss and the cache is re-filling).

(2)  related to #1, but from a different viewpoint; the cache algorithm used by the CPU can make a big difference.   This is another "closely guarded secret" -- Intel & AMD certainly don't share this info.   Different algorithms fill the cache differently on cache misses -- and how well these algorithms result in cache hits can make a noticeable difference in performance.

The major database vendors are well aware of these characteristics -- and factor them in to their decisions about how to organize their database indices.   But as jhance noted, it's unlikely you'll be able to find the details ...  (although there are some good academic papers on the topic)

0
 
Racim BOUDJAKDJIDatabase Architect - Dba - Data ScientistAuthor Commented:
garycase,

Thank you very much for you valuable help and insight.  The whole point of this question for me is to prove to some idiot that a SQL table in RAM is necessarily accessed through a 2 axis (RowAddress/ColumnAddress) or 3 axis (BlockAddress/RowAddress/ColumnAddress) adressing scheme at physical level.  As a consequence all data constituing as SQL table in RAM is necessary dimension limited.  It would be bidimensional if a 2 axis coordinate is used to compute location in RAM and tridimensional if a 3 axis coordinate system is used.

The idiot maintains that adresses on RAM are exclusively linear and that SQL tables are multidimensionally represented in RAM.  I do not know whether you snow some SQL but consider the following query

select value1, value2 from table1
gives

1, 2
3, 6

if all datum (1, 2, 3,6)  are referenced through a 2 axis therefore the select physical representation is necessarily bidimensional.  Apparently, this is understandable only to the idiot that maintains its multidimensional.

Thank you for your insight on that.
0
 
Gary CaseRetiredCommented:
While conceptually memory is a large linear address space, physically it consists of blocks of RAM, each block of which is addressed via row and column addresses.   As I noted above, IF a database is structured to take advantage of the actual physical addresses, it can improve the performance characteristics, because memory addressing takes longer when the row has to change, and even longer if the block changes  (excluding cache hits of course).

0
 
Racim BOUDJAKDJIDatabase Architect - Dba - Data ScientistAuthor Commented:
I see...
According to what rules are commonly data accessed according via column and row adresses as opposed to linear access?  This of course on common RAMS (DDR, DDR2 etc...)

<<As I noted above, IF a database is structured to take advantage of the actual physical addresses, it can improve the performance characteristics>>  It is a complex issue... Are you talking about calculations performed between data ?
0
 
Gary CaseRetiredCommented:
No ... I'm referring to the distinction between the logical LINEAR address ==> i.e. memory is addressed with a 32-bit address (or 36-bit if PAE is enabled);  but those addresses are translated by the memory controller to a physical address that identifies which bank, row, and column the memory actually resides in.

So the CPU addresses a linear address space;  but the memory controller translates that.   In earlier X86 CPU's (remember the 286 paging structure??) you had to manage the bank switching in the code -- but all current CPU's implement a linear address mode for the CPU, and use the memory controller to translate the address to the physical structure of the RAM (bank/row/column).
0
 
Racim BOUDJAKDJIDatabase Architect - Dba - Data ScientistAuthor Commented:
<<No ... I'm referring to the distinction between the logical LINEAR address ==> i.e. memory is addressed with a 32-bit address (or 36-bit if PAE is enabled);  but those addresses are translated by the memory controller to a physical address that identifies which bank, row, and column the memory actually resides in.
>>
I see...Thank you very much for your help and valuable insight...

So if I understand this right, the relationship between the CPU and the RAM is made through a one dimensional reference to linear memory but the RAM memory controller translates this to a bidimensional physical adressing scheme?  Is that right?

<<So the CPU addresses a linear address space;  but the memory controller translates that.   In earlier X86 CPU's (remember the 286 paging structure??) you had to manage the bank switching in the code -- but all current CPU's implement a linear address mode for the CPU, and use the memory controller to translate the address to the physical structure of the RAM (bank/row/column).>>
Very interesting indeed...Is that why some CPU (Dual Core Opteron) have integrated the memory controller directly onto the chip?  To increase the effciency for directly referencing physical memory?
0
 
Gary CaseRetiredCommented:
Yes -- but if the memory modules are multi-bank modules there is also a bank-select (CS -- "chip select") signal that identifies what's essentially a 3rd dimension => which bank.
0
 
Racim BOUDJAKDJIDatabase Architect - Dba - Data ScientistAuthor Commented:
<<Yes -- but if the memory modules are multi-bank modules there is also a bank-select (CS -- "chip select") signal that identifies what's essentially a 3rd dimension => which bank.>>Indeed...You have perfectly answered this specific question...Thank you very much...Let me know if you need help on SQL Server ;)
0

Featured Post

Who's Defending Your Organization from Threats?

Protecting against advanced threats requires an IT dream team – a well-oiled machine of people and solutions working together to defend your organization. Download our resource kit today to learn more about the tools you need to build you IT Dream Team!

  • 5
  • 5
Tackle projects and never again get stuck behind a technical roadblock.
Join Now