Today I got an escalation stating the following on one server:
We cannot execute *ANY* commands and looks like it will require a Linux server bounce.
ksh: top: /usr/bin/top: cannot execute [Too many open files in system]
dwilliams@mialsb02-t1$ ls -l
ksh: ls: /bin/ls: cannot execute [Too many open files in system]
After seeing this, I some how login to the server and increased the "file-max" parameter by executing the following command: echo "75536" > /proc/sys/fs/file-max
Earlier the file-max value was showing as 65536. After increasing the file-max parameter, am able to execute the commands and its stopped giving the "Too many files opened error".
However I want to know what is the current status of opened files. I did a count of 'lsof' output (lsof | wc -l) but it shows value less than 8000. I'm not sure if thats the right value, if that would have been the case, I shouldn't be getting "Too many files opened error" since I have so many left. Please let me what is the current way to check the number of files opened in a Linux system and how would I find those.