> We concluded that the stuck of water happened again as klog#29363. 
​​As a reference, the stuck of the water was also reported in klog#29391.

For the temporal recovery, they changed the hoses for sending and returning to reverse the water flow for the beam dampers. The water flow came back. Although the cleaning inside the dampers is desired, it is not so easy because of their structure, except for maybe chemical treatment. I will search the water beam damper that is cheaper.

As Miyoki-san reported, we reversed the water flow by flipping the input and output tubes. then, the water seems to circulate.

in this state, we measured the temperature on the PMC REFL beam dump with a non-contact thermometer, the measured temperature wad 22 degrees. Before this restoration, the temperature was 25 degrees.  So maybe the water cooling system works well. 

Second we stopped the laser at once and then we restarted the fiber amp. This time, we didn't touch the seed laser. We confirmed that the temperature and the current values on the laser controller screen are their nominal values. After that we emitted the laser. We confirmed that the laser output on the FIB_OUT PD was 1W. Then, we increased the power gradually to 20W without PMC lock. That is, the PMC REFL dump received the most laser power. After 20 mins, we measured the temperature on the dump again. the temperature was 23-24 degrees. According to the previous log, if the cooling system does not work, the temperature got 40 degrees after 30 min when the laser output was 10W. Therefore the system seems fine.

Finally, we locked PMC and IMC.

The PMC_REFL camera was flickering more intensely than usual as shown in the attached movie (somehow image quality is quite worse than online viewer), so I checked the situation.

At first, I checked spectra of all PMC related DQ channels before and after the laser trouble. Figure 1 shows the spectra in some times. The situation in all times is PMC lock, IMC lock and IFO down. Blue and green curves represent the spectra in the last midnight and after the trouble. K1:PMC_MIXER_MON_OUT_DQ which is the error signal of the PMC control shows the large excess below a few tens of Herz. Time series plot on ndscope shown in Fig.2 also shows the large RMS on K1:PMC_MIXER_MON_OUT_DQ. (Two t-cursors show the time when spectra data of blue and green curves were taken, and crosshair shows the time when PMC lock was recovered after the trouble.

Next, I opened PMC overview in order to check other situation. Then I found soon that both K1:PSL-PMC_COMMON_GAIN and K1:PSL-PMC_FAST_GAIN show 22dB. These servo gains are usually set around 10dB in the case of 20W input to PMC. This fact means that open loop gain is ~24dB larger than the normal case. So I tried to decrease these gains. Then RMS on K1:PMC_MIXER_MON_OUT_DQ is getting smaller and noise excess below a few tens of Herz also disappears (see also the red curve in Fig.1). GigE@PMC_REFL also shows usual (no flickering) image.

These two gains is based on the REFL power by guardian. So I checked why such kind of large value was set. Gain was set in the main function of DOWN. According to the timing relation between the guardian request and laser power increasing (see also Fig.3), PMC was recovered with 1W at first. Then down was requested. At that time, servo gains were set for 1W input. After then laser power was increased to 20W. Finally, PMC was locked with 20W power and servo gains for 1W configuration.

I concluded that a trouble in gain settings seemed to be caused by a bad implementation of PMC guardian. We don't face this issue because laser power at fiber output is never changed in normal case. But if we change it for some reason, proper servo gain is not set until executing the main function of DOWN state. Only ways to set proper gain in current implementation are to load guardian after changing laser power or to wait lockloss after locking PMC with improper gain once. In order to improve this situation, gain setting code should be moved from the main function of DOWN to the 1st counter in run function of GRUB_RESONANCE. I might do it in this weekend if I'm willing to do it.

After these checks, I requested DOWN state to IO guardian and then requested PROVIDING_STABLE_LIGHT again in order to set more proper gains which is computed by guardian than ones set by my feeling (servo gains are now 9.6dB both).

The beam dampers in the PSL are maybe this type (https://www.ophiropt.com/ja/f/l250w-bb-50-water-cooled-thermopile-sensor).

Tips on circulating water. (https://www.youtube.com/watch?v=eGc7seR_QK4&t=4s)

I am now considering using this type with an additional SiC plate or so (https://www.kawaso-texcel.co.jp/product/brazing_heatsink_standard.html). The SiC or other plate should be polished?