DGS (General)takahiro.yamamoto - 16:01 Saturday 24 June 2023 (25698)
Print this reportNIC settings were modified for k1dc0This is related to the investigation of IPC glitch causes.
Because NIC latency for myrinet was 10us on k1dc0, I set as 1us. ethtool -C myri0 rx-usecs 1
I'm not sure when the latency was set as 10us. It might be changed in the trouble shooting of disk errors reported as klog#24818. In /etc/conf.d/local.start, this setting was commented out. So default value (10usec) was set when the computer booted up.
I will monitor the situation of IPC glitches in the next couple of days.
In this time, I set 1usec latency by a temporal command. In order to set it permanently, /etc/conf.d/local.start must be modified.
Comments to this report:
takahiro.yamamoto - 11:02 Thursday 29 June 2023 (25775)
Print this reportAfter changing NIC settings, glitch rate seems to be decreased drastically.
Timeline of glitch rate is as follows Before Apr 13: 2-8 events/day. On Apr 14: maintenance works and DAQ troubles as shown in klog#24818 Apr 15 - Jun 24: 20-70 events/day After Jun 25: 0-1 events/day
Maintenance works on Apr 14 probably made a mistake in NIC settings. Current situation is better than one before Apr 14. Maybe this comes from the reduction of CPU load (klog#25212) and so on.