rstaveley
asked on
Missing INODEs in an ext3 file system
I have used up all my INODEs in a file system.
There are 19 million INODEs used up, but I reckon there are a few thousand directories and no more than 11 million files.
What might account for the phantom ~8 million INODEs?
[It doesn't look like deleted files with file handles held open by processed, account for them.]
There are 19 million INODEs used up, but I reckon there are a few thousand directories and no more than 11 million files.
What might account for the phantom ~8 million INODEs?
[It doesn't look like deleted files with file handles held open by processed, account for them.]
amg@rainbow-ec2:/mnt/amgdata$ df -i /mnt
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/sda2 19546112 19546112 0 100% /mnt
....
rainbow-ec2:~# lsof | grep deleted | wc -l
5
ASKER
Thanks for picking up the question, jools, but you didn't read it fully.
I know that I've used up the inodes, but I don't understand why 11 million files are using 19 million inodes, when there are only a few thousand directories.
11 million is a lot and yes, they are small. They are using 65% of the diskspace and 100% of inodes.I can mount another file system, but I'd like to understand what's happening with this one.
I know that I've used up the inodes, but I don't understand why 11 million files are using 19 million inodes, when there are only a few thousand directories.
11 million is a lot and yes, they are small. They are using 65% of the diskspace and 100% of inodes.I can mount another file system, but I'd like to understand what's happening with this one.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Would you say that there is a 10% difference in the numbers?
I'm wondering if they are reserved for the root user like the file system space, just checking the technotes to find out
I'm wondering if they are reserved for the root user like the file system space, just checking the technotes to find out
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
9 milion out of 19 milion is quit big number and I don't think it has to do with reservations.
I've just had a look at one of my systems
There are differences but mine is obviously smaller.
# find /home | wc -l
33783
# df -i /home
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/mapper/rootvg-home
5124480 33792 5090688 1% /home
still trying to find the technote on it (large database and all that) google just come up with the usual results, one inode per file. It may however have something to do with soft or hard linked files... still looking
There are differences but mine is obviously smaller.
# find /home | wc -l
33783
# df -i /home
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/mapper/rootvg-home
5124480 33792 5090688 1% /home
still trying to find the technote on it (large database and all that) google just come up with the usual results, one inode per file. It may however have something to do with soft or hard linked files... still looking
ASKER
Apologies, I've moved the goal posts slightly by purging some files (approx 2 million). There are still 17 million inodes used.
Here is the tune2fs:
Here is the tune2fs:
rainbow-ec2:~# df -i /dev/sda2
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/sda2 19546112 17190114 2355998 88% /mnt
rainbow-ec2:~# lsof | grep deleted | wc -l
5
rainbow-ec2:~# tune2fs -l /dev/sda2
tune2fs 1.40-WIP (14-Nov-2006)
Filesystem volume name: <none>
Last mounted on: <not available>
Filesystem UUID: 623de4ab-28e0-4f9f-8cb1-91f2651e2064
Filesystem magic number: 0xEF53
Filesystem revision #: 1 (dynamic)
Filesystem features: has_journal resize_inode dir_index filetype needs_recovery sparse_super large_file
Default mount options: (none)
Filesystem state: clean
Errors behavior: Continue
Filesystem OS type: Linux
Inode count: 19546112
Block count: 39088128
Reserved block count: 1954406
Free blocks: 38426742
Free inodes: 19546100
First block: 0
Block size: 4096
Fragment size: 4096
Reserved GDT blocks: 1014
Blocks per group: 32768
Fragments per group: 32768
Inodes per group: 16384
Inode blocks per group: 512
Filesystem created: Tue May 20 14:35:34 2008
Last mount time: Mon Jul 21 10:48:24 2008
Last write time: Mon Jul 21 10:48:24 2008
Mount count: 2
Maximum mount count: 27
Last checked: Tue May 20 14:35:34 2008
Check interval: 15552000 (6 months)
Next check after: Sun Nov 16 14:35:34 2008
Reserved blocks uid: 0 (user root)
Reserved blocks gid: 0 (group root)
First inode: 11
Inode size: 128
Journal inode: 8
Default directory hash: tea
Directory Hash Seed: c034e2cd-4cef-4e96-961e-f49561842e7e
Journal backup: inode blocks
ASKER
omarfarid, I'm just checking if there are any issues with unmounting the filesystem. I don't believe there should be, but this is an Amazon EC2 instance, so I'm not on familiar territory.
when I sow that it is mounted over /mnt , I thought your are testing with it, etc
ASKER
I need to apologise for wasting your time. I've just counted the files again and I must have been suffering from a fit of madness. I somehow got my count very wrong. It transpires that the INODE file count correlation is very close, as you'd expect. Thanks for sticking with me, guys. I need to drink more coffee.
rainbow-ec2:/mnt# find . | wc -l
17190105
rainbow-ec2:/mnt# df -i /mnt
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/sda2 19546112 17190114 2355998 88% /mnt
rainbow-ec2:/mnt# find . | wc -l
17190105
rainbow-ec2:/mnt# df -i /mnt
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/sda2 19546112 17190114 2355998 88% /mnt
ASKER
Apologies again and thanks for the company!
Crikey, so omarfarid must have had a vision as he posted saying 9 million files before we knew about it....
How about some lottery numbers for tonight omar?
Thx for the question btw, something to get the grey matter working...
How about some lottery numbers for tonight omar?
Thx for the question btw, something to get the grey matter working...
oh well, I thought we'd found a quirky ext3 file system for a minute....
I still want those lotto numbers omar :-)
I still want those lotto numbers omar :-)
I just read the question which says 19 millions used inodes and 11 millions files. I did a mistake in subtraction :)
Are they all small ish files?
You can create file systems with more inodes (at least you can on other *nix) if you need them