Reports 1-1 of 1 Clear search Modify search
DGS (General)
takahiro.yamamoto - 23:51 Saturday 12 July 2025 (34542) Print this report
EY TCam server seems to be unreachable
Dear tomorrow's shifters,

According to the beam spot archiver, auto-recovery for TCam viewer isn't working for EY.
EY TCam was alive at least until 11:30 JST.
But it cannot be seen since 11:40 JST.

If it's a problem on Vinagre on k1mon4, it should be recovered automatically.
So it seems to be a problem on PlanetaryImager on cam-eya.
It can be recovered by the procedure of Sec-1.1.5 of DGS manual (JGW-G2516704).
Please try to do it.

-----
We can see that the image layout and resolution has been changed between before 7/11 17:40 JST and after 7/11 17:50 JST.

In usual, camera viewer shows an image with 1/4 resolution with camera specification for reducing a network trafic and CPU load. It's important to avoid a hung up of camera server and/or applications. On the other hand, an image with 1/1 resolution is taken in the TCam session. We know image resolution sometimes is not reverted to 1/4 resolution. (I never reproduced this phenomena by manual request to guardian. So I wonder it comes form the compatibility of TCam guardian code and camera session script.) Anyway, this hang-up seemed to be caused by the large network trafic and heavy CPU load due to un-reverted image resolution after camera session.
Comments to this report:
takahiro.yamamoto - 14:19 Sunday 13 July 2025 (34546) Print this report
Hirose-san tried to recover EY TCam, but it didn't come back. So I checked the situation and found that not only the EY TCam server but also the rebooter were unreachable. A server for the laser control of EY Pcal and its rebooter which were connected a same network switch at the EYA booth are reachable. So the network around EYA has no problem. I'm not sure there is a way to recover a trouble on the rebooter itself remotely. If no way, we need to go to EYA for recovering them.

It is not so urgent unless we get into a situation that requires initial alignment. So we can probably wait until weekday. But if we will leave it until Friday, a time table of maintenance tasks will delay roughly 1 hour because TCam photo sessions must wait a recovering work and going to EYA will be probably done after the weekly calibration measurement. Hopefully I want to avoid an extension of whole maintenance time because completing all works by 22:00 is already tight schedule. I guess that the recovering work can be parallelly done with another commissioning activities. So This Tuesday or Thursday seems to be a good day for recovering TCam (though I'm not sure a detailed work plan on this Tuesday and Thursday yet).
satoru.ikeda - 17:56 Sunday 13 July 2025 (34549) Print this report

Since the rebooter is connected to the UPS, there is also a suspicion that something might be wrong with the UPS.

shinji.miyoki - 22:05 Sunday 13 July 2025 (34550) Print this report

>So This Tuesday or Thursday seems to be a good day for recovering TCam (though I'm not sure a detailed work plan on this Tuesday and Thursday yet).

I agree.

dan.chen - 6:21 Monday 14 July 2025 (34551) Print this report

I heard from Kimura-san that a battery alarm has been triggered on the UPS around the EYA chamber.
It might to be a UPS related to the EYA Tcam server.
It is unclear at this point whether it is related to the current trouble.
The replacement battery arrived last week and is ready to be installed.

dan.chen - 12:48 Tuesday 15 July 2025 (34562) Print this report

With Satoru Ikeda

Summary:
On-site investigation revealed that the connectivity issue of the EYA TCam server was caused by a faulty UPS. We removed the UPS and connected the rebooter directly to a 100 VAC outlet, which successfully restored the EYA TCam functionality.

Details:

  • We physically visited the EYA booth to inspect the UPS and server setup.

  • The UPS showed no visual indicators or alarm sounds.

  • We confirmed that the UPS input power was normal (100 VAC present).

  • We removed the UPS from the setup and attempted a reset with the connected load (rebooter) disconnected, but there was no reaction from the UPS—no LEDs, no sound.

  • Based on this, we judged that the UPS was faulty and did not proceed with battery replacement.

  • We brought the UPS back to Mozumi for further inspection.

  • The rebooter (to which the EY TCam server is connected) was plugged directly into a 100 VAC outlet.

  • After power-on, the TCam server booted normally. We verified that TCam image capture was functional.

Next Steps:

  • The UPS has alreadly be moved to Mozumi. We will check it later but, it is likely that it needs to be replaced.

dan.chen - 12:52 Tuesday 15 July 2025 (34563) Print this report

Time stamp:
0948: Entered the tunnel.
0951: Entered center front room
0951: Entered center experimental area
0955: Entered Yarm
1011: Entered Yend
1051: Out from Yend
1108: Entered center experimental area
1112: Entered parking area at center
1115: Went out tunnel and closed the shutter of the tunnel entrance.

Hiroshi Takaba - 10:22 Thursday 17 July 2025 (34577) Print this report
We removed the UPS battery and found it was a third-party product that had likely been replaced once.
The battery was physically broken, and only put out 3V of voltage.
We replaced with a genuine battery, an automatic check was performed and it worked normally.
However, the lifespan of the UPS itself is seven years in an environment of 25°C, and eight years had passed since it was manufactured in April 2017, so we decided to replace it with a new UPS.
Images attached to this comment
Search Help
×

Warning

×