Maarten Bruins
asked on
Why dentries are never saved to disc?
See: https://www.kernel.org/doc/Documentation/filesystems/vfs.txt
A directory is also a file (special file). The contents of a directory consists of:
Usually this data is stored in data block(s). I say "usually" because some systems may store this in the directory's inode (< 60 bytes). But usually the data is stored in data block(s). If you make an empty directory, then the contents are:
Usually you will see something like this:
8*512=4096 bytes. So 4096 bytes in the form of data blocks are allocated on disk. A dentry is just a directory entry, so:
So dentries are stored on disk. Then how I have to see this sentence?
If they would only live in RAM, then you will lose all your dentries if the computer restarts? And why they are not stored on disc/disk? A directory usually just shows 8 blocks, this means that it is stored on disk.
Do I make a mistake here or what's behind it?
Dentries live in RAM and are never saved to disc:
A directory is also a file (special file). The contents of a directory consists of:
. inode-number-current-directory
.. inode-number-parent-directory
filename_1 inode-number-A
filename_2 inode-number-B
et cetera...Usually this data is stored in data block(s). I say "usually" because some systems may store this in the directory's inode (< 60 bytes). But usually the data is stored in data block(s). If you make an empty directory, then the contents are:
. inode-number-current-directory
.. inode-number-parent-directoryUsually you will see something like this:
Blocks: 88*512=4096 bytes. So 4096 bytes in the form of data blocks are allocated on disk. A dentry is just a directory entry, so:
filename_1 inode-number-ASo dentries are stored on disk. Then how I have to see this sentence?
Dentries live in RAM and are never saved to disc:
If they would only live in RAM, then you will lose all your dentries if the computer restarts? And why they are not stored on disc/disk? A directory usually just shows 8 blocks, this means that it is stored on disk.
Do I make a mistake here or what's behind it?
David Johnson, CD
An inode in your context is a data structure that represents a file. A dentries is a data structure that represents a directory. These structures could be used to build a memory cache that represents the file structure on a disk. To get a directly listing, the OS could go to the dentries--if the directory is there--list its contents (a series of inodes). If not there, go to the disk and read it into memory so that it can be used again.https://stackoverflow.com/questions/29870068/what-are-pagecache-dentries-inodes
ASKER
So actually this is incorrect (what I already thought):
The dentries also live on disk and they are used to build a memory cache that represents the file structure on a disk. So in the end they are living on disk and in memory. Only the ones in memory are used, but in the first place they are stored on the disk.
Then my question still is, how they can say this:
Dentries live in RAM and are never saved to disc:
The dentries also live on disk and they are used to build a memory cache that represents the file structure on a disk. So in the end they are living on disk and in memory. Only the ones in memory are used, but in the first place they are stored on the disk.
Then my question still is, how they can say this:
Dentries live in RAM and are never saved to disc:
Ok so you are reading up on how Linux (OS?) handles files.....
In general (not only Linux...) an OS uses disks as a database, the lay out of that database varies wildly based on OS design. Mostly there is a common structure on Multiuser systems...
A space (might be one Lump, or Multiple Lumps) where file descriptions are stored (INDEXF.SYS on VMS, Inode tables on Linux, NTFS MFT ...)
And a list of names that reference file descriptions (Directories)
For some filesystems there is also a Log area where all changes are written before actualy executing them so a power failure will make the structure of the disks repairable.
FAT is slightly different as eash directory is atualy the "i-node" (loosly used) contents next to the name. The Separate FAT structure is just a chain of allocated diskblocks.
When actualy accessing the disk this needs to be searched through and to do that efficiently a lot of data is kept in memory.
Paths are not remember as strings but as a list of elements that are linked. That way a file named f.e.
/x/y/z and /x/y/a can share the x & y in memory search structure. Also to Search directory x the contents needs to be read, to check if y is present after that y is read and verifies if z and a are present. For the search on a the already read x & y data can be reused.
All data on disk is carefully kept in sync with data in memory all updates are send to the log (if log based filesystem) or immediately updated. If sent to the log a background task for the filesystem will realise all log entries into the acual On disk Structures.
Data in memory is based ON the on disk data, but is not the same. In memory a lot of runtime stuff is added, like locking entries to ensure consistency in multi processing systems. Buffering of data around the current positions, pointers to relevant parts of othet in memory parts of the disk structure.
In general (not only Linux...) an OS uses disks as a database, the lay out of that database varies wildly based on OS design. Mostly there is a common structure on Multiuser systems...
A space (might be one Lump, or Multiple Lumps) where file descriptions are stored (INDEXF.SYS on VMS, Inode tables on Linux, NTFS MFT ...)
And a list of names that reference file descriptions (Directories)
For some filesystems there is also a Log area where all changes are written before actualy executing them so a power failure will make the structure of the disks repairable.
FAT is slightly different as eash directory is atualy the "i-node" (loosly used) contents next to the name. The Separate FAT structure is just a chain of allocated diskblocks.
When actualy accessing the disk this needs to be searched through and to do that efficiently a lot of data is kept in memory.
Paths are not remember as strings but as a list of elements that are linked. That way a file named f.e.
/x/y/z and /x/y/a can share the x & y in memory search structure. Also to Search directory x the contents needs to be read, to check if y is present after that y is read and verifies if z and a are present. For the search on a the already read x & y data can be reused.
All data on disk is carefully kept in sync with data in memory all updates are send to the log (if log based filesystem) or immediately updated. If sent to the log a background task for the filesystem will realise all log entries into the acual On disk Structures.
Data in memory is based ON the on disk data, but is not the same. In memory a lot of runtime stuff is added, like locking entries to ensure consistency in multi processing systems. Buffering of data around the current positions, pointers to relevant parts of othet in memory parts of the disk structure.
This is only for the DCACHE this is where the context is
Directory Entry Cache (dcache)
------------------------------
The VFS implements the open(2), stat(2), chmod(2), and similar system calls. The pathname argument that is passed to them is used by the VFS to search through the directory entry cache (also known as the dentry cache or dcache). This provides a very fast look-up mechanism to translate a pathname (filename) into a specific dentry. Dentries live in RAM and are never saved to disc: they exist only for performance.
The dentry cache is meant to be a view into your entire filespace. As most computers cannot fit all dentries in the RAM at the same time, some bits of the cache are missing. In order to resolve your pathname into a dentry, the VFS may have to resort to creating dentries along the way, and then loading the inode. This is done by looking up the inode.
ASKER
@noci: You're just explaining general things, but you're not answering my question.
@David: But the dentries in RAM are based on the same dentries on disk, so:
This is not true, because what if a dentry is missing in cache? Then it goes to disk to get that dentry. And if you create a directory, then the dentries are not saved in RAM but they are saved to disk first. Actually they are never saved in RAM ... later on these saved dentries (on disk) are used to build a memory cache.
So actually I still have the same question.
@David: But the dentries in RAM are based on the same dentries on disk, so:
Dentries live in RAM and are never saved to disc
This is not true, because what if a dentry is missing in cache? Then it goes to disk to get that dentry. And if you create a directory, then the dentries are not saved in RAM but they are saved to disk first. Actually they are never saved in RAM ... later on these saved dentries (on disk) are used to build a memory cache.
So actually I still have the same question.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
@Maarten Bruins, yes and the general case is applicable here as well....
dentries in memory are just remnants of the things an OS needs from searching a directory while verifying the path.
The in memory stuff doesn't need to be save to disk.... the disk structures are in the lead... (The filesystems work under the assumtion the system may crash or have a power failure ANY time...) And they MUST survive that event.
Yep again a general issue that needs to be address by ALL OS'ses.
dentries in memory are just remnants of the things an OS needs from searching a directory while verifying the path.
The in memory stuff doesn't need to be save to disk.... the disk structures are in the lead... (The filesystems work under the assumtion the system may crash or have a power failure ANY time...) And they MUST survive that event.
Yep again a general issue that needs to be address by ALL OS'ses.
ASKER
Ah that's helpful! Actually they had to say it like that, because then I agree and understand. It's pretty confusing to use the word "dentry" in this context without saying they mean "dcache dentries", because what if a "dcache dentry" is not there? Then they will look at then non-dcache dentry, so somehow you have to distinguish these terms (even when it's about the dcache).
But thanks! I think that answered my question, because that's probably what they meant.
But thanks! I think that answered my question, because that's probably what they meant.
Everything works in the correct context, the dentries mentioned in that paragraph where in the heading of dcache
ASKER
@noci: Again that's not what my question is about. Just look at David's answer. That's what it is about. You're only mentioning general stuff.
That's right, but the dentries build up in memory are coming from the dentries on disk, so in the first place they are saved on disk. So it doesn't say anything about my question. But fortunately David understood what I meant.
Memory stuff doesn't need to be save to disk
That's right, but the dentries build up in memory are coming from the dentries on disk, so in the first place they are saved on disk. So it doesn't say anything about my question. But fortunately David understood what I meant.
Actualy there was one OS that had IN MEMORY prefered above anything else.
The Apollo Control computers. (AGC) that had a "ferrite-core" based memory so it could be restarted with a known memory state, and continue from where it left. (Actualy those had no disks...). There were also some RSTS-11 based system that worked in almost such a way (paper tape as backing store).
The Apollo Control computers. (AGC) that had a "ferrite-core" based memory so it could be restarted with a known memory state, and continue from where it left. (Actualy those had no disks...). There were also some RSTS-11 based system that worked in almost such a way (paper tape as backing store).
ASKER
@David:The problem is that they are not consistent. See:
So you are saying (in this context):
If the term "dentries" in this context would refer to the dentries in RAM, then of course they would fit, because they are already part of it. So in this case "dentries" must refer to the non-dcache dentries. They are using the word "dentry" for non-dcache dentries and for dcache dentries in the same context.
So you're wrong that all "dentries" in this context only refer to the dcache dentries. If that would be the case, then you would be right.
Now it's just bad written text in my opinion. Actually they had to distinguish "non-dcache dentries" and "dcache dentries". Documentation should be clear in my opinion. This costs people time to find out what exactly is meant. Maybe it's immediately clear for you, but probably anyway you don't need the documentation to know what is meant. People are reading the documentation, because they want to obtain some information about something. Evertime when I'm reading "dentry" I don't want to wonder/guess which one is meant. Probably (or maybe) you don't see the importance about it and it's just an unimportant detail for you, but you also have to think of people who are new to all this.
most computers cannot fit all dentries in the RAM at the same time
So you are saying (in this context):
dentries = dentries in RAMIf the term "dentries" in this context would refer to the dentries in RAM, then of course they would fit, because they are already part of it. So in this case "dentries" must refer to the non-dcache dentries. They are using the word "dentry" for non-dcache dentries and for dcache dentries in the same context.
So you're wrong that all "dentries" in this context only refer to the dcache dentries. If that would be the case, then you would be right.
Now it's just bad written text in my opinion. Actually they had to distinguish "non-dcache dentries" and "dcache dentries". Documentation should be clear in my opinion. This costs people time to find out what exactly is meant. Maybe it's immediately clear for you, but probably anyway you don't need the documentation to know what is meant. People are reading the documentation, because they want to obtain some information about something. Evertime when I'm reading "dentry" I don't want to wonder/guess which one is meant. Probably (or maybe) you don't see the importance about it and it's just an unimportant detail for you, but you also have to think of people who are new to all this.
ASKER
@noci: Still general information, but no answer to my question at all. Although I appriciate you want to help, try to focus on the question instead of adding more and more general information to this post.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Really interesting! That is indeed another way to look at it. So actually the directory entries (contents of the directory) described in my first post are not called dentries. At that moment, they are still directory entries. But when these directory entries are used to build a memory cache, then they are called dentries. Do I understand this correctly?
But then the following is indeed incorrect:
They can not fit all "directory entries" in RAM as dentries, but that's something different then they are saying.
But then I had a wrong idea what dentries are. Many websites say a dentry is a "directory entry", but then this is incorrect too. It's a directory entry used in memory as cache.
Now I know this, I'll read some thing back. I'll try to find some examples why I thought a dentry does refer to the contents of a directory-file. But I think many people do not realize that a dentry always specifically refers to the dcache. I'll come back to it. Thanks a lot already for now, because I've never realized that maybe I have the wrong definition of a dentry in my head.
But then the following is indeed incorrect:
most computers cannot fit all dentries in the RAM at the same time
They can not fit all "directory entries" in RAM as dentries, but that's something different then they are saying.
But then I had a wrong idea what dentries are. Many websites say a dentry is a "directory entry", but then this is incorrect too. It's a directory entry used in memory as cache.
Now I know this, I'll read some thing back. I'll try to find some examples why I thought a dentry does refer to the contents of a directory-file. But I think many people do not realize that a dentry always specifically refers to the dcache. I'll come back to it. Thanks a lot already for now, because I've never realized that maybe I have the wrong definition of a dentry in my head.