Oshino-san, DanChen-san, Washimi-san, Ikeda

K-Log#37506 関連
https://klog.icrr.u-tokyo.ac.jp/osl/?r=37506

9/21の停電からの復旧対応

DC電源
・24V, 18V共に中央のDCはすべてトリップしていた。
・24V, 18V共に両エンド、２Fはトリップせず電源が供給されている状態まで復帰していた。
・V2 IOシャーシは前面ボタンがOFF状態で復帰していた。（停電復帰の通常動作）

FEPC
K1ix1, k1iy0,k1asc0, K1test0は再起動でk1bootがないのでマウントできず起動中で停止していた。
上記以外はすべて電源OFF状態にて停止していた。

リアルタイム系は、次の対応を行いました。
1. エアコン回復とk1bootの回復を待ち、18Vと24V系の電源側をOFF、ラック側もブレーカーOFFとシャーシの電源OFF、タップもOFFにした。
2. 制御盤を運転状態に変更
3. 24Vと18Vは、それぞれ電源をON後、ブレーカー電圧測定、ブレーカーON、タップ電圧測定、タップON、シャーシ電源ONを行った。
4. センターエリアの全ての電源を投入後、センター２階とYエンドとXエンドの１階と２階についても同様に対応を行い、ワークステーションもONにした。

OMC PZDドライバーやPOSのところの機器類、一部の機器の電源はOFFのままである。

センター2F、両エンドの1F、2FののDC電源はONのままであった。
また、リアルタイムPC、IOシャーシはすべて電源SWがすべてOFFであったのでこれをONにした。

5.センターのリアルタイム計算機の復旧をおこなった。
[復帰済み]
Dolphinなし: k1mcf0, k1ix1, k1iy1, k1ex1, k1ey1, k1ex0, k1ey0, k1test0, k1iy0
Dolphinあり: k1lsc0, k1asc0, k1als0, k1ioo, k1ioo1, k1imc0:
=> 一部IRIGB_TIMEのドリフトがあるが基本復帰を確認

[残り]
Dolphinなし：k1px1 (エレキ系電源未確認)
Dolphinあり: k1pr2, k1pr0,k1prm, k1bs,k1sr2, k1sr3, k1srm ,k1omc0, k1omc1
=> k1pr2は、カード認識していない（時間切れで詳細は明日確認)

---
Oshino-san, DanChen-san, Washimi-san, Ikeda

Related to K-Log#37506 
https://klog.icrr.u-tokyo.ac.jp/osl/?r=37506

Below is a summary of the recovery work following the power outage on September 21.

DC Power Supplies

* All central DC power supplies for both the 24 V and 18 V systems had tripped.
* The DC power supplies at both end stations and on the second floor had not tripped and had recovered to a state in which power was being supplied.
* The front-panel power buttons on the V2 I/O chassis were OFF after power was restored. This is the expected behavior following a power outage.

FEPCs

* k1ix1, k1iy0, k1asc0, and k1test0 had restarted but stopped during the boot process because k1boot was unavailable and the required file systems could not be mounted.
* All other FEPCs were powered off.

The following recovery work was performed on the real-time systems:

1. After waiting for the air-conditioning system and k1boot to recover, we switched off the 18 V and 24 V power supplies. On the rack side, we also switched off the circuit breakers, chassis power supplies, and power strips.

2. We placed the control panels in operation mode.

3. For both the 24 V and 18 V systems, we performed the following sequence:

   * Switched on the power supply
   * Measured the voltage at the circuit breaker
   * Switched on the circuit breaker
   * Measured the voltage at the power strip
   * Switched on the power strip
   * Switched on the chassis power supply

4. After powering on all equipment in the central area, we followed the same procedure for the second floor of the central area and the first and second floors of both the X- and Y-end stations. We also powered on the workstations.

Some equipment, including the OMC PZT drivers, POS-related equipment, and several other devices, remains powered off.

The DC power supplies on the second floor of the central area and on the first and second floors of both end stations had remained on. However, all real-time PCs and I/O chassis had their power switches set to OFF, so we switched them on.

5. We restored the real-time computers in the central area.

Recovered:

* Without Dolphin:
  k1mcf0, k1ix1, k1iy1, k1ex1, k1ey1, k1ex0, k1ey0, k1test0, k1iy0

* With Dolphin:
  k1lsc0, k1asc0, k1als0, k1ioo, k1ioo1, k1imc0

Although some systems show drift in IRIGB_TIME, we confirmed that they have otherwise recovered and are operating normally.

Remaining systems:

* Without Dolphin:
  k1px1 — The electronics power supply has not yet been checked.

* With Dolphin:
  k1pr2, k1pr0, k1prm, k1bs, k1sr2, k1sr3, k1srm, k1omc0, k1omc1

The cards on k1pr2 are not being recognized. Due to time constraints, we will investigate the issue in detail tomorrow.