It's really simple: this server has an SSD for the main drive, and a RAID array (mdadm) for storage.
Intermittently, it has a problem writing to the array: the write speeds slow down to the point where it's really not that usable.
I just finished testing the speed, and copying a 1G file from the SSD to the RAID array took almost an hour:
root@office:/tmp# rsync -vhr --progress test.img /mnt/raid/
sending incremental file list
1.07G 100% 301.99kB/s 0:57:52 (xfer#1, to-check=0/1)
sent 1.07G bytes received 31 bytes 309.25K bytes/sec
total size is 1.07G speedup is 1.0
The array seems to be fine:
root@office:/tmp# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 sdb sdd sdc
5860270080 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU]
unused devices: <none>
and syslog doesn't mention anything.
Where else should I be looking to troubleshoot this?
PS. About a month ago, we had to replace a switch after a thunderstorm. This problem started around that time as well. I can't tell if it is a failing motherboard or a controller, or something else that is broken. I would prefer to find a smoking gun before I advise the client to start replacing hardware...