[Ikeda, YamaT]

IO chassis for K1EY0 was replaced as V2 one in klog#36692.
IRIG-B synchronization issue appeared by using V2 IO chassis and disappeared by replacing a proper optical fiber for K1IX1, K1IY1, and K1EX0 in klog#36705.
Only for K1EY0, this issue still remains even though an optical fiber was already replaced.
So we tried to narrow down problematic points but we haven't identified a cause of this issue yet.
It doesn't seemed to be caused by a specific equipment and might be related to the environment around EY0 rack.

-----
During the IO chassis replacement, both the IO chassis and the real-time front-end were replaced with new hardware. The IO chassis was equipped with new V2 hardware. On the other hand, the real-time front-end utilized hardware that had been in use on the IXV until just prior to the replacement. Therefore, we do not believe that the real-time front-end or the PCIe IRIG-B interface board attached to it is the direct cause of this issue. And also, we guess that a cause of this issue is V2 IO chassis itself or a compatibility between it and other equipment.

We have been testing various combinations of V2 IO chassis and other equipments on the SK test bench in order to reproduce the issue, but so far we have been unable to reproduce it. On the other hand, we have obtained a specific IO chassis unit that has been confirmed not to exhibit this issue on the SK test bench, so we decided to bring it to EY0 for testing. As a result, even when using units that did not cause issues on the test bench, problems still occurred with the EY0. It appears that this is not an issue specific to the IO chassis alone, but rather likely a compatibility issue between the IO chassis and other devices within the EY0 environment. We also checked other components that are relatively easy to test, such as timing optical fibers, fanout ports, IRIG-B ports, IRIG-B breakout connectors, the timing slave board which was used in the V1 IO chassis, easing a current limitation of DC24V power supply and minor changes on BIOS settings related to PCIe slots, but none of them helped resolve the issue.

Untested component is around Fanout and IRIG-B chassis. These units are at the end of a four-unit daisy chain that includes o(10km)-long fiber connection from the Timing Master. Conducting fiber tests on a 10km scale in the SK server room is not practical, and we do not have enough spare units to replicate a four-unit daisy chain. Therefore, ongoing on-site troubleshooting at EY0 will likely be necessary going forward.