Reproduced files are 14382.ffl, 14382.cache, 14383.ffl, and 14383.cache in Cache_GPS and CacheSecond_GPS. There is no missed GWF files in Minute trend. Old cache files are still kept as *.bak in the same directory. Difference between old and new files are shown in the attachment files. Now all GWF files since resuming O4c can be accessed via cache files at Kashiwa.
By the way, current cache generation process cannot serve correct cache when unordered data transfer is occurred due to a delay on the data transfer process and a manual recovery operation is required. So I started to test a new code to provide cache files including a function to recover automatically in the case of instability of the data transfer in the personal space (kmst2:/home/takahiro.yamamoto/work/makeCache). After testing, this new code will be merged to the product version of cache generation. This new function can automatically respond to instability on the data transfer and/or on network itself in the case that the resend process works well like this time, but manual operation is still required for problems such as long term disconnections e.g. klog#14712. This new code can probably mitigate also a duplication issue such as klog#34595.
Fig.1 and Fig.2 show the aftershocks of the Tohoku Earthquake (2011 March 11, M9.0) and the Kamchatka Earthquake (2025 July 30, M8.8) that was also reported by Washimi-kun. The data for the Tohoku (> M1) was got from Yahoo Earthquake Info. On the other hand, the data for the Kamchatka (>M2.5) was obtained from USGS as Washimi-kun.
The number of earthquakes with over M6 is more in the Tohoku Earthquake than in the Kamchatka Earthquake. So our situation is better.
Fig.3 shows the 5 years of aftershocks of the Tohoku Earthquake. Please note that earthquakes not only in the sea area, but also in the land area in the east part of Japan are included. If I select only the sea part, it might be a smaller number. Even 5 years have passed, M4-class earthquakes are more frequent than before the M9 earthquake.
As observed this morning, GRX seemed to be locked at TEM00 again. So no handover from GR to IR again. Different from the morning situation, IRX trans power reached ~ 20, while about 6 this morning.
PR3 alignment repeatability seemed to be bad?
Operation shift summary: Aug. 5th
Operators: Oshino, Washimi
Shift time: 2025-08-05 09:00-17:00 JST
Check Items:
VAC: [10:00] No issues were found.
CRY cooler: [10:00] No issues were found.
Temp: [10:00] No issues were found
Compressor: [10:00] No issues were found.
GAS: [10:00] No issues were found.
IFO state (JST):
10:00 Yokozawa-san is measuring the ASC TFs
13:18 PRFPMI lock has recovered by Yokozawa-san, (~4Mpc)
13:33 Lock loss
16:00 keeping "INCREASING LAS POWER" state at 40kpc
Because IFO was in out-of-observing-mode, reproduction of LL h(t) and omicron triggers are not essential. Actually what we will do is only providing zero-filled data even if we will reproduce them. Necessity of them depends on how different in convenience for end-users, open-data production and so on. (In principle, missing files are not a problem, but some softwares don't work well with file gaps.)
But the lock duration was not so long ~10 min.
So I will keep the LSC_LOCK state with INCREASE LAS POWER and checked the stability for a while.
And by the modification of the lsclib.py and ASC_LOCK.py guardian, we can lock with PRC1 Yaw ASC automatically.
As observed this morning, GRX seemed to be locked at TEM00 again. So no handover from GR to IR again. Different from the morning situation, IRX trans power reached ~ 20, while about 6 this morning.
PR3 alignment repeatability seemed to be bad?
lsclib.py
L169
for ax in ['DHARD', 'CHARD','DSOFT', 'CSOFT', 'MICH', 'INP1' ,'INP2', 'PRC2']:
->
for ax in ['DHARD', 'CHARD','DSOFT', 'CSOFT', 'MICH', 'INP1' ,'INP2','PRC1', 'PRC2']:
L199
for ax in [ 'MICH', 'INP1', 'INP2', 'PRC2']:
->
for ax in [ 'MICH', 'INP1', 'INP2', 'PRC1','PRC2']:
L229
for ax in ['MICH', 'PRC2']:
->
for ax in ['MICH', 'PRC1', 'PRC2']:
ASC_LOCK.py
L403 :
for DOF in ['MICH','PRC2','INP1','INP2']:
->
for DOF in ['MICH','PRC1','PRC2','INP1','INP2']:
L1574
ezca['ASC-PRC1_%s_TRAMP'%(dof)] = 0
L1660
ezca['ASC-PRC1_%s_GAIN'%(dof)] = 0
L1668
ezca['ASC-PRC1_%s_TRAMP'%(dof)] = 5
L1897
for DOF in ['MICH','PRC2','INP2']:
->
for DOF in ['MICH','PRC1','PRC2','INP2']:
Please ignore previous post...
The UGF seemed ~0.3 Hz
And I noticed the beam spot of ITMX and ETMX was bad, I performed the full initial alignment, then PRFPMI RF LOCKED recovered.
And the ouput matrix for the new ASC was backed
3. K1:ASC-OUTMTRX_Y_1_3: from 1 to 0
4. K1:ASC-OUTMTRX_Y_3_3: from 0 to 1.
As shown in Fig.1., after the several minutes, they backed and LSC_LOCKED and ASC_LOCKED guardian didn't change at this moment.
I changed again.
And I started the check of the new PRC1_Y control from around 9:30
[Shih-Hong Hsu, Chia-Jui Chou, Hirotaka Yuzurihara, Yi Yang]
In the study of scattered light noise, it is known that the scattered light noise is related to the seismic motion. In kagra, it is especially correlated with 0.3 Hz to 1 Hz (klog-33668). In this report, we check the coherence between the optical levers in PRCL (K1:VIS-PR{M, 2, 3}_TM_WIT_{L, P, Y}_DQ) and seismic motion near BS (K1:PEM-SEIS_BS_GND_{X, Y, Z}_OUT_DQ).
As a result, we found the length degree of freedom showed coherence larger than or close to 0.5 at 0.43 Hz for PRM, PR2, and PR3 with respect to x-direction seismic motion. The 0.43 Hz is chosen due to the frequency of potential scattering objects causing the scattered light noise (klog-33668).
Channels:
-
OPLEVs: K1:VIS-PR{M, 2, 3}_TM_WIT_{L, P, Y}_DQ
-
Seismic: K1:PEM-SEIS_BS_GND_{X, Y, Z}_OUT_DQ
Times:
-
t1: 2025-07-15 11:00:00 UTC (1436612418) (larger seismic motion)
-
t2: 2025-07-07 11:00:00 UTC (1435921218) (smaller seismic motion)
-
Duration: 3600 s
Coherence:
-
Gwpy: TimeSeries.coherence
-
fftlength: 64 s
-
Overlap: 32
-
Marker: 0.43 Hz
Coherence at 0.43 Hz larger than 0.5:
-
PRM_L-X: 0.579 (7-15)
-
PR2_L-X: 0.914 (7-15)
-
PR2_L-X: 0.893 (7-07)
-
PR3_L-X: 0.871 (7-15)
-
PR3_L-X: 0.721 (7-07)
-
PR2_P-X: 0.795 (7-15)
-
PR2_P-X: 0.689 (7-07)
-
PR3_P-X: 0.648 (7-15)
-
PR2_Y-X: 0.675 (7-15)
-
PR2_Y-X: 0.669 (7-07)
Fig. 1: Seismic motion: K1:PEM-SEIS_BS_GND_Z_BLRMS_300MHZ1000.mean
Fig. 2~4: Coherence between PRCL OPLEVs and seismic X/Y/Z-direction for 0.3 to 1 Hz
So, I'll immediately contact to Marco Meyer. Please wait.
Because re-send process at 3am everyday seems to be working well, missed data will be probably available around 3am tomorrow.
Now GWF files in attached text files which are not available on Kashiwa (They were provided after 3am today).
LL hoft also has many gap.
So many chopped files were provided by llhoftNoGap process as shown in the attached figure.
There is no report about SINET trouble for now. Investigation of the cause in KAGRA side (software, server hardware, network equipments, etc.) is also necessary.
And also especially for CAL/DET, some processes at Kashiwa with shorter cadence than 1-day (LL h(t), cache, segments, omicron triggers, etc.), must be rerun (some of them might be able to be skipped because now IFO is in OOOM).
So, I'll immediately contact to Marco Meyer. Please wait.
Because IFO was in out-of-observing-mode, reproduction of LL h(t) and omicron triggers are not essential. Actually what we will do is only providing zero-filled data even if we will reproduce them. Necessity of them depends on how different in convenience for end-users, open-data production and so on. (In principle, missing files are not a problem, but some softwares don't work well with file gaps.)
Operators: Oshino, Washimi
Shift time: 2025-08-04 09:00-17:00 JST
Check Items:
VAC: [10:00] No issues were found.
CRY cooler: [10:00] No issues were found.
Temp: [10:00] No issues were found
Compressor: [10:00] No issues were found.
GAS: [10:00] IY BF GAS output is about -11000 counts.
IFO state (JST):
9:00 IFO is STANDBY (Ushiba-san is inspecting the PR3)
10:00 IY BF GAS looks to be offloaded? (9:18 JST)
15:45 Ushiba-san gave up on recovering the IFO and went back to his home.
I restarted these processes and confirmed that it works fine.
Abstract:
Linear range of PRC2_Y seems very small and REFL_QPDA1_RF45_I signals seem no more sentive than REFL_QPDA2_RF45_I.
For stability tests, I changed PR3 ASC signals from REFL_QPDA1 to REFL_QPDA2, and then IFO could reach OBSERVATION state for a short time.
Since the large earthquakes occures, the stability should be checked later.
Detail:
Since PRC2 yaw seems a problematic DoF, I checked the signals of REFL QPDs while moving PR3.
Figure 1 shows the signals when PRFPMI was locked with ASC except for PRC2_Y with 1W.
At that time, PR3 was about 0.4 urad apart from the good value in +yaw direction (-43.2 of K1:VIS-PR3_TM_WIT_Y_DQ was good for IFO).
Since PRC2 flter sign is negative for closing loop, the error signals should be positive if the good values are -yaw direction, however the DC values of K1:ASC-PRC2_Y_IN1_DQ seems negative.
It implies that PRC2 might not measure PR3 motions well, so I moved PR3 intentionally and see the QPD signals.
Figure 2 and figure 3 show the QPD signals when PR3 moved in plus and minus yaw direction around good values, respectively.
According to the data shown in fig2 and fig3, REFL_QPDA1 signals seems less sensitive than REFL QPDA2 signals though REFL_QPDA1 signals are used for the current PRC2_Y signals.
Though I'm not so sure why REFL QPDA1 doesn't measure PR3 motion now, I anyway tried to use REFL_QPDA2 signals instead of REFL_QPDA1 signals instead.
Figure 4 shows the time-series data of POP90, PR3 OpLev, and QPD signals when PR3 yaw was controled with REFL_QPDA2_RF45 signals.
Snce IFO seems stable with PRC ASC with REFL QPDA2 RF45 signals, I requested OBSERVATION state to LSC_LOCK guardian and confirmed that IFO can reach OBSERVATION state.
Since the big earthquake happened, I cannot check the stability, so it is necessary to check it.
What I changed:
To test PRC ASC with REFL QPDA2 signals, I modified following.
1. K1:ASC-INMTRX_Y_3_18: from 1 to 0.
2. K1:ASC-INMTRX_Y_3_11: from 0 to 1
3. K1:ASC-OUTMTRX_Y_1_3: from 1 to 0
4. K1:ASC-OUTMTRX_Y_3_3: from 0 to 1.
5. Turn off INPUT of K1:ASC-PRC2_Y filter bank.
6. Setting gain filters (-3dB, -6dB, -12dB, and -24dB) at K1:ASC-PRC1_Y filter bank (fig5).
Note:
To test the PRC ASC with REFL QPDA2 RF45 signals, it can be engaged just turning off FM8 (null) at K1:ASC-PRC1_Y filter bank after reaching PRFPMI_RF_LOCKED state.
Since I didn't modified the guardian, resetting ASC signals and turn on/off ASC should be manualy done at this moment.
No check of UGF, so loop gain should be checked and adjusted.
And I noticed the beam spot of ITMX and ETMX was bad, I performed the full initial alignment, then PRFPMI RF LOCKED recovered.
And the ouput matrix for the new ASC was backed
3. K1:ASC-OUTMTRX_Y_1_3: from 1 to 0
4. K1:ASC-OUTMTRX_Y_3_3: from 0 to 1.
As shown in Fig.1., after the several minutes, they backed and LSC_LOCKED and ASC_LOCKED guardian didn't change at this moment.
I changed again.
And I started the check of the new PRC1_Y control from around 9:30
The UGF seemed ~0.3 Hz
Please ignore previous post...
lsclib.py
L169
for ax in ['DHARD', 'CHARD','DSOFT', 'CSOFT', 'MICH', 'INP1' ,'INP2', 'PRC2']:
->
for ax in ['DHARD', 'CHARD','DSOFT', 'CSOFT', 'MICH', 'INP1' ,'INP2','PRC1', 'PRC2']:
L199
for ax in [ 'MICH', 'INP1', 'INP2', 'PRC2']:
->
for ax in [ 'MICH', 'INP1', 'INP2', 'PRC1','PRC2']:
L229
for ax in ['MICH', 'PRC2']:
->
for ax in ['MICH', 'PRC1', 'PRC2']:
ASC_LOCK.py
L403 :
for DOF in ['MICH','PRC2','INP1','INP2']:
->
for DOF in ['MICH','PRC1','PRC2','INP1','INP2']:
L1574
ezca['ASC-PRC1_%s_TRAMP'%(dof)] = 0
L1660
ezca['ASC-PRC1_%s_GAIN'%(dof)] = 0
L1668
ezca['ASC-PRC1_%s_TRAMP'%(dof)] = 5
L1897
for DOF in ['MICH','PRC2','INP2']:
->
for DOF in ['MICH','PRC1','PRC2','INP2']:
[Shuta Tanaka, Ide, Obayashi (Aogaku), Yuzurihara]
We performed the lockloss investigation for the lockloss which happened between 2025/06/11~06/17 (first week). The previous lockloss investigation was posted in klog34687. During this period, there were 36 lockloss from the OBSERVATION state. There was no new phenomena. The plots are listed in DAC wiki.
ETMY MN oplev glitch
As reported in klog34259, in the begining after joining O4c, the ETMY MN oplev glitch oftern happened. In the previous report, we found the glitch behavior around 11 lockloss. We carefully checked the data to judge whether this glitch was the cause of the lockloss.
When there was a sudden fluctuation in K1:VIS-ETMY_MN_COILOUTF_V3_OUTPUT above 4000, we judged this glitch was the cause of the lockloss. This threshold was determined empirically. We checked the behavior of other channels, but did not find nothing that seemed to be related. As a result, 4 lockloss (i=1, 2, 4, 5)was caused by this glitch.
Details was sunnaruzed in JGWdoc (in Japanese).
Quick saturation of OMC DCPD
For 2 lockloss (i=16, 34), the quick saturation of OMC DCPD happened. This phenomena looks similar with the lockloss reported in klog34527. Compared with the usual saturation of OMC DCPD, the quickness (or the time scale) of the saturation is different.
Slowly saturation of OMC DCPD
For 22 lockloss, the slowly saturation of OMC DCPD can be seen. The oscillation just before the saturation was 20~40 Hz. Three of them have the ISC_WC_L label.
There was no lockloss with 18 Hz oscillation.
Earthquake
For 3 lockloss, the EQ label was asgned.
Excess of 1~10Hz seismic motion
For 4 lockloss (Three of them has also EQ label), 1Hz label was assigned. In addition, for 1 lockloss (i=27), we found the excess of the 1Hz seismic motion.
Alignment upward the AS port
There was no lockloss.
Lockloss by the human request
For 3 lockloss, the REQ label was asigned.
According to Fig1 in the original post, DC values of IM horizontal signals were jumped significantly while the other signals were not.
So, I checked the signals of each OSEM and found that only H3 OSEM signals were jumped (fig1).
Since it is unlikely due to PR3 real motions because it is difficult to move only one OSEM signal, there may be some issues with the H3 OSEM.
This DC value jump was recovered about 8.5 minutes later after the jump (fig2).
We continued to investigate this phenomena is hardware trouble or not.
Similar problem was (is still being?) repeated.
In the first two events, data transfer seemed to become timeout and failed files were sent the re-transfer process at 3am.
The third event occurred around 7am today and a failed file is still absence on Kashiwa cluster.
The failed file exists on Kamioka. So it will be probably re-send at next 3am.
/data/KAGRA/raw/full/14382/K-K1_C-1438263904-32.gwf
$ grep /data/KAGRA/raw/full/14382/K-K1_C-1438263[89] ~detchar/cache/Cache_GPS/14382.cache
K K1_C 1438263808 32 file://localhost/data/KAGRA/raw/full/14382/K-K1_C-1438263808-32.gwf
K K1_C 1438263840 32 file://localhost/data/KAGRA/raw/full/14382/K-K1_C-1438263840-32.gwf
K K1_C 1438263872 32 file://localhost/data/KAGRA/raw/full/14382/K-K1_C-1438263872-32.gwf
K K1_C 1438263936 32 file://localhost/data/KAGRA/raw/full/14382/K-K1_C-1438263936-32.gwf
K K1_C 1438263968 32 file://localhost/data/KAGRA/raw/full/14382/K-K1_C-1438263968-32.gwf
$ grep /data/KAGRA/raw/full/14382/K-K1_C-1438263[89] ~detchar/cache/Cache_GPS/14382.ffl
/data/KAGRA/raw/full/14382/K-K1_C-1438263808-32.gwf 1438263808 32 0 0
/data/KAGRA/raw/full/14382/K-K1_C-1438263840-32.gwf 1438263840 32 0 0
/data/KAGRA/raw/full/14382/K-K1_C-1438263872-32.gwf 1438263872 32 0 0
/data/KAGRA/raw/full/14382/K-K1_C-1438263936-32.gwf 1438263936 32 0 0
/data/KAGRA/raw/full/14382/K-K1_C-1438263968-32.gwf 1438263968 32 0 0
$ ls -l --full-time /data/KAGRA/raw/full/14382/K-K1_C-1438263[89]*
-rw-r--r-- 1 daq 482317613 2025-08-03 22:43:57.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438263808-32.gwf
-rw-r--r-- 1 daq 482233503 2025-08-03 22:44:27.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438263840-32.gwf
-rw-r--r-- 1 daq 481581101 2025-08-03 22:45:05.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438263872-32.gwf
-rw-r--r-- 1 daq 481812561 2025-08-04 03:00:08.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438263904-32.gwf
-rw-r--r-- 1 daq 481314142 2025-08-03 22:46:09.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438263936-32.gwf
-rw-r--r-- 1 daq 483906327 2025-08-03 22:46:36.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438263968-32.gwf
/data/KAGRA/raw/full/14382/K-K1_C-1438265376-32.gwf, /data/KAGRA/raw/full/14382/K-K1_C-1438265408-32.gwf
$ grep /data/KAGRA/raw/full/14382/K-K1_C-1438265[34] ~detchar/cache/Cache_GPS/14382.cache
K K1_C 1438265312 32 file://localhost/data/KAGRA/raw/full/14382/K-K1_C-1438265312-32.gwf
K K1_C 1438265344 32 file://localhost/data/KAGRA/raw/full/14382/K-K1_C-1438265344-32.gwf
K K1_C 1438265440 32 file://localhost/data/KAGRA/raw/full/14382/K-K1_C-1438265440-32.gwf
K K1_C 1438265472 32 file://localhost/data/KAGRA/raw/full/14382/K-K1_C-1438265472-32.gwf
$ grep /data/KAGRA/raw/full/14382/K-K1_C-1438265[34] ~detchar/cache/Cache_GPS/14382.ffl
/data/KAGRA/raw/full/14382/K-K1_C-1438265312-32.gwf 1438265312 32 0 0
/data/KAGRA/raw/full/14382/K-K1_C-1438265344-32.gwf 1438265344 32 0 0
/data/KAGRA/raw/full/14382/K-K1_C-1438265440-32.gwf 1438265440 32 0 0
/data/KAGRA/raw/full/14382/K-K1_C-1438265472-32.gwf 1438265472 32 0 0
$ ls -l --full-time /data/KAGRA/raw/full/14382/K-K1_C-1438265[34]*
-rw-r--r-- 1 daq 484744461 2025-08-03 23:08:59.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438265312-32.gwf
-rw-r--r-- 1 daq 484754345 2025-08-03 23:11:18.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438265344-32.gwf
-rw-r--r-- 1 daq 485229068 2025-08-04 03:00:32.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438265376-32.gwf
-rw-r--r-- 1 daq 484477094 2025-08-04 03:00:56.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438265408-32.gwf
-rw-r--r-- 1 daq 484117973 2025-08-03 23:11:41.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438265440-32.gwf
-rw-r--r-- 1 daq 483749699 2025-08-03 23:11:45.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438265472-32.gwf
/data/KAGRA/raw/full/14382/K-K1_C-1438294880-32.gwf
$ grep /data/KAGRA/raw/full/14382/K-K1_C-1438294[789] ~detchar/cache/Cache_GPS/14382.cache
K K1_C 1438294720 32 file://localhost/data/KAGRA/raw/full/14382/K-K1_C-1438294720-32.gwf
K K1_C 1438294752 32 file://localhost/data/KAGRA/raw/full/14382/K-K1_C-1438294752-32.gwf
K K1_C 1438294784 32 file://localhost/data/KAGRA/raw/full/14382/K-K1_C-1438294784-32.gwf
K K1_C 1438294816 32 file://localhost/data/KAGRA/raw/full/14382/K-K1_C-1438294816-32.gwf
K K1_C 1438294848 32 file://localhost/data/KAGRA/raw/full/14382/K-K1_C-1438294848-32.gwf
K K1_C 1438294912 32 file://localhost/data/KAGRA/raw/full/14382/K-K1_C-1438294912-32.gwf
K K1_C 1438294944 32 file://localhost/data/KAGRA/raw/full/14382/K-K1_C-1438294944-32.gwf
K K1_C 1438294976 32 file://localhost/data/KAGRA/raw/full/14382/K-K1_C-1438294976-32.gwf
$ grep /data/KAGRA/raw/full/14382/K-K1_C-1438294[789] ~detchar/cache/Cache_GPS/14382.ffl
/data/KAGRA/raw/full/14382/K-K1_C-1438294720-32.gwf 1438294720 32 0 0
/data/KAGRA/raw/full/14382/K-K1_C-1438294752-32.gwf 1438294752 32 0 0
/data/KAGRA/raw/full/14382/K-K1_C-1438294784-32.gwf 1438294784 32 0 0
/data/KAGRA/raw/full/14382/K-K1_C-1438294816-32.gwf 1438294816 32 0 0
/data/KAGRA/raw/full/14382/K-K1_C-1438294848-32.gwf 1438294848 32 0 0
/data/KAGRA/raw/full/14382/K-K1_C-1438294912-32.gwf 1438294912 32 0 0
/data/KAGRA/raw/full/14382/K-K1_C-1438294944-32.gwf 1438294944 32 0 0
/data/KAGRA/raw/full/14382/K-K1_C-1438294976-32.gwf 1438294976 32 0 0
$ ls -l --full-time /data/KAGRA/raw/full/14382/K-K1_C-1438294[789]*
-rw-r--r-- 1 daq 478908272 2025-08-04 07:19:10.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438294720-32.gwf
-rw-r--r-- 1 daq 480288904 2025-08-04 07:19:40.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438294752-32.gwf
-rw-r--r-- 1 daq 480812523 2025-08-04 07:20:13.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438294784-32.gwf
-rw-r--r-- 1 daq 480483696 2025-08-04 07:22:14.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438294816-32.gwf
-rw-r--r-- 1 daq 480454286 2025-08-04 07:21:17.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438294848-32.gwf
(/data/KAGRA/raw/full/14382/K-K1_C-1438294880-32.gwf)
-rw-r--r-- 1 daq 480316241 2025-08-04 07:23:27.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438294912-32.gwf
-rw-r--r-- 1 daq 480368106 2025-08-04 07:23:17.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438294944-32.gwf
-rw-r--r-- 1 daq 480390431 2025-08-04 07:23:38.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438294976-32.gwf
I accepted the following differences of SDFs (see related klog34731).
After full initial alignment (XARM, YARM, OMC, PRMI), IFO can be locked with ENGAGE_WFSDC, but IFO locked loss when PRC2_Y engaged, this is similar situation last week.
I tweaked the IMMT1 P from 29.5 to 35, 40. When I changed to 40, the IR tansmittance power decreased, but we need more investigation.
Long term check:
I checked several signals for a long period as shown in Fig.1.
POP_FORWARD_QPDA1_DC_PIT
POP_PDA1_DC
POP_PDA2_DC
REFL_PDA1_DC
had similar trend.
Coherence check
During the ENGAGE_WFSDC state, I checked the spectrum and coherence for the PRC2 and PR3 oplev.
No significant coherence between PRC2 and oplev, but several coherence between PRC2 P and PRC Y. (Fig.2. and FIg.3. )
I offloaded the BF GAS with the FR.
When I tried to automate a work in klog#34703, lack of GWF files in cache files were found.
Missing files I found are /data/KAGRA/raw/full/14382/K-K1_C-1438233056-32.gwf and /data/KAGRA/raw/full/14382/K-K1_C-1438245664-32.gwf. Although these two files surely exist on the Kashiwa cluster, they are not contained in the cache files. A relevant part of the cache file contents and the results of the ls command are shown below. Missing files have a larger delay between GPS time of the file head and file timestamp than other files. There may be a problem with cache file generation when the data transfer order is reversed for some reason.
/data/KAGRA/raw/full/14382/K-K1_C-1438233056-32.gwf
$ grep /data/KAGRA/raw/full/14382/K-K1_C-14382330 ~detchar/cache/Cache_GPS/14382.cache
/home/detchar/cache/Cache_GPS/14382.cache:K K1_C 1438233024 32 file://localhost/data/KAGRA/raw/full/14382/K-K1_C-1438233024-32.gwf
/home/detchar/cache/Cache_GPS/14382.cache:K K1_C 1438233088 32 file://localhost/data/KAGRA/raw/full/14382/K-K1_C-1438233088-32.gwf
$ grep /data/KAGRA/raw/full/14382/K-K1_C-14382330 ~detchar/cache/Cache_GPS/14382.ffl
/home/detchar/cache/Cache_GPS/14382.ffl:/data/KAGRA/raw/full/14382/K-K1_C-1438233024-32.gwf 1438233024 32 0 0
/home/detchar/cache/Cache_GPS/14382.ffl:/data/KAGRA/raw/full/14382/K-K1_C-1438233088-32.gwf 1438233088 32 0 0
$ ls -l --full-time /data/KAGRA/raw/full/14382/K-K1_C-14382330*
-rw-r--r-- 1 daq 471247162 2025-08-03 14:10:54.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438233024-32.gwf
-rw-r--r-- 1 daq 470978416 2025-08-03 14:12:17.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438233056-32.gwf
-rw-r--r-- 1 daq 471010349 2025-08-03 14:11:55.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438233088-32.gwf
/data/KAGRA/raw/full/14382/K-K1_C-1438245664-32.gwf
$ grep /data/KAGRA/raw/full/14382/K-K1_C-14382456 ~detchar/cache/Cache_GPS/14382.cache
K K1_C 1438245600 32 file://localhost/data/KAGRA/raw/full/14382/K-K1_C-1438245600-32.gwf
K K1_C 1438245632 32 file://localhost/data/KAGRA/raw/full/14382/K-K1_C-1438245632-32.gwf
K K1_C 1438245696 32 file://localhost/data/KAGRA/raw/full/14382/K-K1_C-1438245696-32.gwf
$ grep /data/KAGRA/raw/full/14382/K-K1_C-14382456 ~detchar/cache/Cache_GPS/14382.ffl
/data/KAGRA/raw/full/14382/K-K1_C-1438245600-32.gwf 1438245600 32 0 0
/data/KAGRA/raw/full/14382/K-K1_C-1438245632-32.gwf 1438245632 32 0 0
/data/KAGRA/raw/full/14382/K-K1_C-1438245696-32.gwf 1438245696 32 0 0
$ ls -l --full-time /data/KAGRA/raw/full/14382/K-K1_C-14382456*
-rw-r--r-- 1 daq 456124886 2025-08-03 17:40:27.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438245600-32.gwf
-rw-r--r-- 1 daq 456837281 2025-08-03 17:41:00.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438245632-32.gwf
-rw-r--r-- 1 daq 456100787 2025-08-03 17:42:18.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438245664-32.gwf
-rw-r--r-- 1 daq 456047603 2025-08-03 17:42:11.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438245696-32.gwf
Similar problem was (is still being?) repeated.
In the first two events, data transfer seemed to become timeout and failed files were sent the re-transfer process at 3am.
The third event occurred around 7am today and a failed file is still absence on Kashiwa cluster.
The failed file exists on Kamioka. So it will be probably re-send at next 3am.
/data/KAGRA/raw/full/14382/K-K1_C-1438263904-32.gwf
$ grep /data/KAGRA/raw/full/14382/K-K1_C-1438263[89] ~detchar/cache/Cache_GPS/14382.cache
K K1_C 1438263808 32 file://localhost/data/KAGRA/raw/full/14382/K-K1_C-1438263808-32.gwf
K K1_C 1438263840 32 file://localhost/data/KAGRA/raw/full/14382/K-K1_C-1438263840-32.gwf
K K1_C 1438263872 32 file://localhost/data/KAGRA/raw/full/14382/K-K1_C-1438263872-32.gwf
K K1_C 1438263936 32 file://localhost/data/KAGRA/raw/full/14382/K-K1_C-1438263936-32.gwf
K K1_C 1438263968 32 file://localhost/data/KAGRA/raw/full/14382/K-K1_C-1438263968-32.gwf
$ grep /data/KAGRA/raw/full/14382/K-K1_C-1438263[89] ~detchar/cache/Cache_GPS/14382.ffl
/data/KAGRA/raw/full/14382/K-K1_C-1438263808-32.gwf 1438263808 32 0 0
/data/KAGRA/raw/full/14382/K-K1_C-1438263840-32.gwf 1438263840 32 0 0
/data/KAGRA/raw/full/14382/K-K1_C-1438263872-32.gwf 1438263872 32 0 0
/data/KAGRA/raw/full/14382/K-K1_C-1438263936-32.gwf 1438263936 32 0 0
/data/KAGRA/raw/full/14382/K-K1_C-1438263968-32.gwf 1438263968 32 0 0
$ ls -l --full-time /data/KAGRA/raw/full/14382/K-K1_C-1438263[89]*
-rw-r--r-- 1 daq 482317613 2025-08-03 22:43:57.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438263808-32.gwf
-rw-r--r-- 1 daq 482233503 2025-08-03 22:44:27.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438263840-32.gwf
-rw-r--r-- 1 daq 481581101 2025-08-03 22:45:05.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438263872-32.gwf
-rw-r--r-- 1 daq 481812561 2025-08-04 03:00:08.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438263904-32.gwf
-rw-r--r-- 1 daq 481314142 2025-08-03 22:46:09.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438263936-32.gwf
-rw-r--r-- 1 daq 483906327 2025-08-03 22:46:36.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438263968-32.gwf
/data/KAGRA/raw/full/14382/K-K1_C-1438265376-32.gwf, /data/KAGRA/raw/full/14382/K-K1_C-1438265408-32.gwf
$ grep /data/KAGRA/raw/full/14382/K-K1_C-1438265[34] ~detchar/cache/Cache_GPS/14382.cache
K K1_C 1438265312 32 file://localhost/data/KAGRA/raw/full/14382/K-K1_C-1438265312-32.gwf
K K1_C 1438265344 32 file://localhost/data/KAGRA/raw/full/14382/K-K1_C-1438265344-32.gwf
K K1_C 1438265440 32 file://localhost/data/KAGRA/raw/full/14382/K-K1_C-1438265440-32.gwf
K K1_C 1438265472 32 file://localhost/data/KAGRA/raw/full/14382/K-K1_C-1438265472-32.gwf
$ grep /data/KAGRA/raw/full/14382/K-K1_C-1438265[34] ~detchar/cache/Cache_GPS/14382.ffl
/data/KAGRA/raw/full/14382/K-K1_C-1438265312-32.gwf 1438265312 32 0 0
/data/KAGRA/raw/full/14382/K-K1_C-1438265344-32.gwf 1438265344 32 0 0
/data/KAGRA/raw/full/14382/K-K1_C-1438265440-32.gwf 1438265440 32 0 0
/data/KAGRA/raw/full/14382/K-K1_C-1438265472-32.gwf 1438265472 32 0 0
$ ls -l --full-time /data/KAGRA/raw/full/14382/K-K1_C-1438265[34]*
-rw-r--r-- 1 daq 484744461 2025-08-03 23:08:59.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438265312-32.gwf
-rw-r--r-- 1 daq 484754345 2025-08-03 23:11:18.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438265344-32.gwf
-rw-r--r-- 1 daq 485229068 2025-08-04 03:00:32.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438265376-32.gwf
-rw-r--r-- 1 daq 484477094 2025-08-04 03:00:56.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438265408-32.gwf
-rw-r--r-- 1 daq 484117973 2025-08-03 23:11:41.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438265440-32.gwf
-rw-r--r-- 1 daq 483749699 2025-08-03 23:11:45.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438265472-32.gwf
/data/KAGRA/raw/full/14382/K-K1_C-1438294880-32.gwf
$ grep /data/KAGRA/raw/full/14382/K-K1_C-1438294[789] ~detchar/cache/Cache_GPS/14382.cache
K K1_C 1438294720 32 file://localhost/data/KAGRA/raw/full/14382/K-K1_C-1438294720-32.gwf
K K1_C 1438294752 32 file://localhost/data/KAGRA/raw/full/14382/K-K1_C-1438294752-32.gwf
K K1_C 1438294784 32 file://localhost/data/KAGRA/raw/full/14382/K-K1_C-1438294784-32.gwf
K K1_C 1438294816 32 file://localhost/data/KAGRA/raw/full/14382/K-K1_C-1438294816-32.gwf
K K1_C 1438294848 32 file://localhost/data/KAGRA/raw/full/14382/K-K1_C-1438294848-32.gwf
K K1_C 1438294912 32 file://localhost/data/KAGRA/raw/full/14382/K-K1_C-1438294912-32.gwf
K K1_C 1438294944 32 file://localhost/data/KAGRA/raw/full/14382/K-K1_C-1438294944-32.gwf
K K1_C 1438294976 32 file://localhost/data/KAGRA/raw/full/14382/K-K1_C-1438294976-32.gwf
$ grep /data/KAGRA/raw/full/14382/K-K1_C-1438294[789] ~detchar/cache/Cache_GPS/14382.ffl
/data/KAGRA/raw/full/14382/K-K1_C-1438294720-32.gwf 1438294720 32 0 0
/data/KAGRA/raw/full/14382/K-K1_C-1438294752-32.gwf 1438294752 32 0 0
/data/KAGRA/raw/full/14382/K-K1_C-1438294784-32.gwf 1438294784 32 0 0
/data/KAGRA/raw/full/14382/K-K1_C-1438294816-32.gwf 1438294816 32 0 0
/data/KAGRA/raw/full/14382/K-K1_C-1438294848-32.gwf 1438294848 32 0 0
/data/KAGRA/raw/full/14382/K-K1_C-1438294912-32.gwf 1438294912 32 0 0
/data/KAGRA/raw/full/14382/K-K1_C-1438294944-32.gwf 1438294944 32 0 0
/data/KAGRA/raw/full/14382/K-K1_C-1438294976-32.gwf 1438294976 32 0 0
$ ls -l --full-time /data/KAGRA/raw/full/14382/K-K1_C-1438294[789]*
-rw-r--r-- 1 daq 478908272 2025-08-04 07:19:10.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438294720-32.gwf
-rw-r--r-- 1 daq 480288904 2025-08-04 07:19:40.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438294752-32.gwf
-rw-r--r-- 1 daq 480812523 2025-08-04 07:20:13.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438294784-32.gwf
-rw-r--r-- 1 daq 480483696 2025-08-04 07:22:14.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438294816-32.gwf
-rw-r--r-- 1 daq 480454286 2025-08-04 07:21:17.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438294848-32.gwf
(/data/KAGRA/raw/full/14382/K-K1_C-1438294880-32.gwf)
-rw-r--r-- 1 daq 480316241 2025-08-04 07:23:27.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438294912-32.gwf
-rw-r--r-- 1 daq 480368106 2025-08-04 07:23:17.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438294944-32.gwf
-rw-r--r-- 1 daq 480390431 2025-08-04 07:23:38.000000000 +0900 /data/KAGRA/raw/full/14382/K-K1_C-1438294976-32.gwf
Operators: Hido, Tanaka
Shift time: 2025/8/3 9:00-17:00 JST
Check Items:
VAC: No issues were found.
CRY cooler: No issues were found.
Temp: IYA field temperature changed over 0.3 degrees from previous check at 10:00. At 16:00, no issues were found
Compressor: No issues were found.
GAS: IY BF GAS output was over 30000. we already contacted with the expert.
IFO state (JST):
9:00 LSC_LOCK tried to lock but failed because seimic motion was large due to aftershocks since last night.
Sometimes, IFO locked without ASC but the lock was lost by earthquakes
17:00 shift end. we left LSC_LOCK guardian in ENGAGE_WFSDC state