Reports 1-1 of 1 Clear search Modify search
CAL (General)
hirotaka.yuzurihara - 9:02 Friday 28 November 2025 (35704) Print this report
TCam photo session 20251128

I took the TCam photos for commissioning at 08:35 ~ 08:40 this morning. The previous work is klog35660.

After taking the photos, the fitting process usually started to analyze the images. But, this time the process was not launched automatically. I'm checking the cause.

Comments to this report:
hirotaka.yuzurihara - 9:26 Friday 28 November 2025 (35706) Print this report

As I checked quickly, the calcenter01 seems to be rebooted on 2025-11-26 10:53:22 JST (I used `uptime -s`). I wonder the daemon process for the fitting was turned off at that time.

And also, I found the disc space (/dev/sdd1) of calceneter01 seems to be full. I'm not sure if this is a trigger of rebooting. Note that the images which consume a large ammount of disc space are stored in the different disc of /dev/sdc1.

dan.chen - 9:25 Wednesday 03 December 2025 (35746) Print this report

I think I have found a possible scenario for what happened.

As written in klog29655, we set up a backup system for the TCam data.
The crontab was configured to rsync /data2/ to /Tcam_BU_001 and /Tcam_BU_002 regularly.

However, at some point, the HDD for /Tcam_BU_001 might to have been disconnected.
Even after that, the crontab continued to run every (other) day, trying to write to /Tcam_BU_001.
Since the mount point did not exist anymore, the system automatically created a normal directory /Tcam_BU_001 on the main HDD (/dev/sdd1), and rsync started copying a large amount of TCam images into it.

As a result, /dev/sdd1 became full very quickly, which probably caused the unexpected behavior.

Currently, the HDD for /Tcam_BU_001 is not visible, but I can see a directory named /Tcam_BU_001 under the root filesystem on /dev/sdd1, which supports this scenario.

As an immediate countermeasure, I have disabled only the crontab for /Tcam_BU_001.
The crontab for /Tcam_BU_002 is still active, and it performs rsync from /data2 to /Tcam_BU_002 every two days at 2 AM.

Next, I will clean up the accidentally copied data from /dev/sdd1 after some confirmations.

dan.chen - 8:56 Thursday 04 December 2025 (35764) Print this report

I deleted Tcam data in /dev/sdd1 after confirming that the data is exactly same as the data in /Tcam_BU_002/.

This work made some space in /dev/sdd1, the Use% became 88% from 100%.

Later, we need to check the status of HDD which was used for /Tcam_BU_001.

Search Help
×

Warning

×