Digital system was recovered from the planned power outage.
The recovery of the end stations proceeded largely as scheduled, and at the corner station, thanks to much assistance, the power supply system restoration was completed significantly ahead of schedule. Thanks a lot.
-----
Issues
VIS-TMSX is unavailable
Though all real-time models are now online, VIS-TMSX cannot be used due to LVDT LO issue reported in klog#36501. VIS-TMSX guardian must be kept in SAFE state (there haven't been any preparation of VIS-TMS control yet, so it's not serious for now)
hyades-1 is still offline
hyades-1 couldn't be launched due to errors on the system disk. Now hyades-0 and hyades-2 are the product servers and hyades-1 which was one of the product server in the past is now just a backup server for emergency cases. So it's not serious. We need more investigation and some additional work for recovering it. Purchasing new disk might be necessary.
Because of the trouble on hyades-1, some servers which mounted the data storage of hyades-1 cannot be launched. I launched them as rescue mode and modified a mount configuration in /etc/fstab. After then, they came back online. Let's ensure to add "nofail" option for NFS mount if the mount point is not essential on that system.
Malfunction of the power supply unit of servers
On k1ioo0 and k1script1, one of the redundant power supply units was beeping. They were connected to the UPS via the AC power strip in the server rack. Because other healthy servers were also connected to the same AC power strip and input power of UPS was shutout by the breaker at the power distribution panel, it may not be related to the power outage itself. Anyway, they were recovered by replacing the power supply units to spare ones.