• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 347
  • Last Modified:

high percentage on 'iowait'

I got the problem on the 'root' partition of the Linux RHEL3 server which is suddenly having the partition become read only. The case solve by rebooting the server.
On checking the 'top' status, it is found that an abnormal behavior is having the 'iowait' persist at a high rate and at the time of the incident, the 'iowait' always reaches 90-100 %
Will this be the cause of the partition becoming read only ?
If it is, what is the possible reason of the 'iowait' becoming high ?
Last but most important, how to solve this issue ?
Below showing that the iowait reaches 100% this morning and then the partition cannot write again.

03:44:02  up  7:02,  0 users,  load average: 0.00, 0.00, 0.00  
62 processes: 61 sleeping, 1 running, 0 zombie, 0 stopped
CPU states:  cpu    user    nice  system    irq  softirq  iowait    idle
           total    0.0%    0.0%    0.0%   0.0%     0.0%  100.0%    0.0%
Mem:  2056864k av,  175268k used, 1881596k free,       0k shrd,   67172k buff
       111756k active,              25436k inactive
Swap: 4192956k av,       0k used, 4192956k free                   66368k cached
0
francis_lai
Asked:
francis_lai
1 Solution
 
Seth SimmonsSr. Systems AdministratorCommented:
have you looked at /var/log/messages?
when this happens, try running iostat 3 to see what device is being accessed

could also be a device causing i/o wait...maybe a failing disk?
0
 
Daniel HelgenbergerCommented:
My best guess is the I/O operations are timing out. Thus you have high I/O waits and high CPU load (on a low percentage, check with top). These IO - waits are causing the 'read only' state of the root volume.

This is most likely a failing disk; which could be since I assume the installation is about five to seven years old (RHEL3).

Is the root volume on a RAID device?

You should, however, use rsync or any other backup tool to constantly mirror the contents of your root device and switch disks ASAP.
0
 
Duncan RoeSoftware DeveloperCommented:
I agree with Daniel. The only reason Linux flips a disk over to read-only is an I/O failure
0
 
TintinCommented:
What's the output of

df -h

and

iostat -x 5
0

Featured Post

Configuration Guide and Best Practices

Read the guide to learn how to orchestrate Data ONTAP, create application-consistent backups and enable fast recovery from NetApp storage snapshots. Version 9.5 also contains performance and scalability enhancements to meet the needs of the largest enterprise environments.

Tackle projects and never again get stuck behind a technical roadblock.
Join Now