Reports 1-1 of 1 Clear search Modify search
DetChar (General)
hirotaka.yuzurihara - 16:58 Wednesday 15 January 2025 (32349) Print this report
Update script to generate Segment files for the past data

[Uchikata, Narikawa, YamaT, Yuzurihara]
I have two reports related to the segment files.

(1) Last year, we missed the automatic generation of the segment files due to the failure of the cache file generation (it's due to the expire of the GPS package). The period is 2024/06/28~07/18. We regenerated segment files at System-B (Kashiwa cluster) and transferred them to Kamioka.
The regenerated files were uploaded to DQSEGDB with the huge help of the administrator, Robert. The regular upload to DQSEGDB has resumed since 2025/01/09.

(2) During the manual regeneration, we used a different conda environment in the Kashiwa cluster from the usual environment in Kamioka. Actually, the package version of `python-logo-lw` is different between them, and it caused the different format segments. This time, we regenerated the segment files in the conda environment, including the consistent packages.
For the countermeasure next time, I'm updating the script (/users/DET/tools/Segments/Script/make24hourSegment_igwn.py) to add the option to read the data via k1nds2.
This script is different from the script running every 15 minutes. So it will not affect the regular generation of the segment files.
I'm testing the script for several dates to validate the script. It took 5 minutes ~ 30 minutes to finish reading the one-day data vis k1nds2. We guess the possible cause is the overload of the HDD in the Kashiwa cluster.

Comments to this report:
hirotaka.yuzurihara - 12:07 Tuesday 21 January 2025 (32408) Print this report

[YamaT, Yuzurihara]
I tried to validate the script. When we compare the Segment generated by the new script and the Segment of regular generation with 15 cadences, there was a slight difference due to missing frame data by rebooting the computer at Kamioka. This was happening only on the maintenance day (11/08). So, it will not affect providing the Segment for the data analysis purpose. 

Detail

The Segment generated by the new script is stored in /users/yuzu/work/20250115_segment . The differences between the generated Segment by the new script and the Segment of regular generation with 15 cadences are summarized below:

For the following Segments of 2024/11/08, the period of rebooting the computer was flagged in the Segment of regular generation with 15 cadences, not flagged in the Segment generated by the new script. We guess the cause is related to rebooting the computer.

  • K1-ETMX_OVERFLOW_OK
  • K1-ETMY_OVERFLOW_OK
  • K1-OMC_OVERFLOW_OK
  • K1-GRD_UNLOCKED

 

For the K1-DAQ_IPC_ERROR Segment of 2024/11/08, the period of 1415118400 ~ 1415118592 was flagged in the Segment of regular generation with 15 cadences, not flagged in the Segment generated by the new script. As I heard from YamaT-san, there was multiple rebooting of k1fr0 during the maintenance day. If there was no frame data in /frame0/, the script to make a cache file will use the frame file in /frame1/ . This time, it seems not to be happening. We will check it for future maintenance days. Anyway, this happens during maintenance day. It will not affect providing the Segment for the data analysis purpose. 
For following Segments of other dates, there was no difference.

 

For the following Segments, there was no difference for all the dates. 

  • K1-ETMX_OVERFLOW_VETO
  • K1-ETMY_OVERFLOW_VETO
  • K1-OMC_OVERFLOW_VETO
  • K1-GRD_PEM_EARTHQUAKE
  • K1-GRD_LOCKED
  • K1-GRD_SCIENCE_MODE

The script was uploaded in the git repo.

Search Help
×

Warning

×