asked on

/dev/ram0 filling up (QNAP NAS) - help diagnose

Hello

Not sure this is the right place to ask but I'm rather desparate for a lead...

I'm maintaining a variety of QNAP NAS appliances with mostly good results. However recently one of them started to exhibit various problems which I managed to track to the /rev/ram0 somehow filling up after a few hours of runtime:

[/tmp] # df -h
Filesystem                Size      Used Available Use% Mounted on
/dev/ram0               139.5M    139.5M         0 100% /
tmpfs                    64.0M    512.0k     63.5M   1% /tmp
/dev/sda4               310.0M    246.1M     63.8M  79% /mnt/ext
/dev/md9                509.5M    101.2M    408.3M  20% /mnt/HDA_ROOT
/dev/md0                  5.4T      1.6T      3.8T  29% /share/MD0_DATA
tmpfs                    32.0M         0     32.0M   0% /.eaccelerator.tmp
tmpfs                     8.0M      4.0k      8.0M   0% /var/syslog_maildir
[/tmp] #

Open in new window

This is mostly an *nix question: how would I proceed to have a look into this ramdisk (?) to find out what's going on ? As you might have guess I am only moderately proficient in the matter.

FWIW it's a TS-459U with firmware 3.8.1 but i guess it's beyond the point for the time being.

Any pointer most welcome

arnold

look in /var/log /var/tmp /var/log/messages
cd /
ls | while read a; do
du -ks $a
done
The above will provide you with a directory and usage data
make sure the issue is not within /home

_OpenSys_

check the /var/log/messages and also do this cmd: dmesg

will show it all errors.

Regards.

Alexandre Takacs

ASKER

Thanks

Have looked into /var/log /var/tmp /var/log/messages - nothing obvious.

Here is the output of your script

[/var/tmp] # cd /
[/] # ls | while read a; do
> du -ks $a
> done
du: cannot access `Qmultimedia@': No such file or directory
2321    bin/
27      dev/
1346    etc/
27091   home/
7171    lib/
du: cannot access `linuxrc@': No such file or directory
12      lost+found/
322426  mnt/
du: cannot access `opt@': No such file or directory
du: cannot access `php.ini@': No such file or directory
du: cannot access `proc/22780/task/22780/fd/4': No such file or directory
du: cannot access `proc/22780/task/22780/fdinfo/4': No such file or directory
du: cannot access `proc/22780/fd/4': No such file or directory
du: cannot access `proc/22780/fdinfo/4': No such file or directory
0       proc/
43440   root/
1       rpc/
3899    sbin/
1734473514      share/
1       stunnel.pid
0       sys/
312     tmp/
54705   usr/
54      var/
[/] #

Open in new window

Again nothing obvious to my untrained eyes. The share folder contains the data and the reported size seems about right. The other folder seem withing acceptable sizes...

_OpenSys_

What are the "various problems" that you say?
Please explain.

arnold

look in /root /home /usr to see what

find /usr -mtime -90
this will look for files in /usr that are younger than 90 days.
See what has changed over the past 90 days. Usually unless you are doing updates, no changes should be happening in /usr /sbin /bin /lib /etc
You can repeat the same for /home and /root
Alternatively, you can search for large files
find /home -size +10240c
The above will look for files that are larger than 10240 characters ~ >10MB
do the same for /root and see whether you have large files that are building up.

Alexandre Takacs

ASKER

For instance can't save config data:

The system is unable to save your settings (file = [/etc/config/uLinux.conf], section = [TFTP Server], field = [PORT], value = [69]) due to insufficient ramdisk space. If restarting the server does not solve the problem please contact support for further assistance.

Open in new window

Scp file transfer will stop with "insufficient disk space" error although there is ample remaining place in /share

general instability of the web UI (some pages won't refresh at all)

arnold

What are you transferring via SCP between the QNAPs?

run the ls | while read
on another qnap and compare the amount of space used.
you are looking for /root /home /var/ /sbin /usr to see whether there is a difference in amount of space consumed.

This could point you to where the data might be accumulating.

Alexandre Takacs

ASKER

Thanks for your suggestions

The scp error can happens with small (100k) text files transferred to /share

Here is another snapshot just after such a failure:

[~] # df -h
Filesystem                Size      Used Available Use% Mounted on
/dev/ram0               139.5M    139.5M         0 100% /
tmpfs                    64.0M    316.0k     63.7M   0% /tmp
/dev/sda4               310.0M    246.1M     63.9M  79% /mnt/ext
/dev/md9                509.5M    101.4M    408.0M  20% /mnt/HDA_ROOT
/dev/md0                  5.4T      1.6T      3.8T  30% /share/MD0_DATA
tmpfs                    32.0M         0     32.0M   0% /.eaccelerator.tmp
tmpfs                     8.0M         0      8.0M   0% /var/syslog_maildir
[~] # cd /
[/] # ls | while read a; do
> du -ks $a
> done
du: cannot access `Qmultimedia@': No such file or directory
2321    bin/
27      dev/
1343    etc/
27091   home/
7171    lib/
du: cannot access `linuxrc@': No such file or directory
12      lost+found/
322502  mnt/
du: cannot access `opt@': No such file or directory
du: cannot access `php.ini@': No such file or directory
du: cannot access `proc/24169/task/24169/fd/4': No such file or directory
du: cannot access `proc/24169/task/24169/fdinfo/4': No such file or directory
du: cannot access `proc/24169/fd/4': No such file or directory
du: cannot access `proc/24169/fdinfo/4': No such file or directory
0       proc/
43440   root/
1       rpc/
3899    sbin/
1734533918      share/
1       stunnel.pid
0       sys/
316     tmp/
54705   usr/
57      var/
[/] #

Open in new window

Another QNAP (same model and firmware) not exhibiting this issue:

[~] # df -h
Filesystem                Size      Used Available Use% Mounted on
/dev/ram0               139.5M     97.1M     42.4M  70% /
tmpfs                    64.0M    340.0k     63.7M   1% /tmp
/dev/sda4               310.0M    246.1M     63.9M  79% /mnt/ext
/dev/md9                509.5M     92.3M    417.2M  18% /mnt/HDA_ROOT
/dev/md0                  3.6T    283.4G      3.3T   8% /share/MD0_DATA
tmpfs                    32.0M         0     32.0M   0% /.eaccelerator.tmp
[~] # cd /
[/] # ls | while read a; do
> du -ks $a
> done
du: cannot access `Qmultimedia@': No such file or directory
2321    bin/
27      dev/
1347    etc/
27091   home/
7171    lib/
du: cannot access `linuxrc@': No such file or directory
12      lost+found/
313134  mnt/
6       opt/
du: cannot access `php.ini@': No such file or directory
du: cannot access `proc/1207/task/1207/fd/4': No such file or directory
du: cannot access `proc/1207/task/1207/fdinfo/4': No such file or directory
du: cannot access `proc/1207/fd/4': No such file or directory
du: cannot access `proc/1207/fdinfo/4': No such file or directory
0       proc/
16      root/
1       rpc/
3899    sbin/
296977450       share/
1       stunnel.pid
0       sys/
340     tmp/
54703   usr/
64      var/
[/] #

Open in new window

Can you pinpoint anything (it definitely eludes me...)

ASKER CERTIFIED SOLUTION

arnold

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

Alexandre Takacs

ASKER

Indeed some unexpected material in there - managed to clean up and have thing running smoothly. Thanks.