I tried using the a2A5328-4gmPRO camera while it was occupying k1cam1.
But high-resolution and/or high-bit-depth recording still failed.
So using dedicated camera server doesn't become a solution for this issue
And also, either an upgrade of the network capacity or an allocation of dedicated bandwidth seems to be necessary.
-----
Bottleneck investigation
To isolate a network capacity issue from a server capacity issue, I moved all acA640-120gm cameras to k1cam0 and k1cam2, then tested a2A5328-4gmPRO while k1cam1 exclusively occupied by it. But capturing high-resolution and/or high-bit-depth images was still failed. Increasing buffer size of NIC from 256kB to 4096kB (upper limit) by ethtool didn't change a situation. In this configuration, a25328-4gmPRO should be able to occupy 1Gbps band-width on the NIC of k1cam1. And also, this situation isn't probably improved by using 10GigE NIC for k1cam1 because NIC of a25328-4gmPRO is 1GbE.
These facts suggest that a current limitation is actual throughput of a network path which can be occupied by a2A5328-4gmPRO. (I haven't isolated just a bandwidth issue from buffer size issue of each network switch.) Current connection of the CAM network is shown in Fig.1. Though the core camera switch in the server room on which all traffic concentrates is 10GigE one, edge switches except at IOO and connections are 1GbE. (Because all cameras in the corner station were connected from IOO rack in the past, only the edge switch at IOO is 10GigE one.) acA640-120gm which is main cameras for KAGRA GigE system is used with 640x480 as resolution, 8bit depth (Mono8) and 25fps. So ~60Mbps per 1 camera is required and 1Gbps bandwidth is enough to stably use 12-13 cameras (corresponding to ~70% of full bandwidth) and managing 18 cameras with 2 camera servers and one 10GigE core switch is sufficient.
a2A5328-4gmPRO is designed as using almost full width of 1Gbps to send caputured images (It's natural behavior because if bandwidth is limited as slower than 1Gbps, a risk of a connection time-out increases for high-resolution cameras such as a2A5328-4gmPRO.) So sharing bandwidth of 1GbE with another devices is not proper configuration for a2A5328-4gmPRO. If network paths will be shared, the CAM network should be upgraded to 10GigE for at least the shared paths with a2A5328-4gmPRO. Another solution is connecting a2A5328-4gmPRO to the core switch in the server room directly. But it may be more tough work than preparing a dedicated TCam server physically near a2A5328-4gmPRO.
Simple camera server application for a2A5328-4gmPRO
To capture images easily for this investigation, I prepared a simple camera server application as /kagra/camera/test-a2A5328-4gmPRO/test_CameraServer.py. It runs on one of camera servers (currently k1cam1) and listens various requests from CDS-workstations via the client application as /kagra/camera/test-a2A5328-4gmPRO/test_CameraClient.py. Because this server application keeps a connection to camera devices (keeping camera.Open()) same as camlan- and pylon-camera-server, this server application must be stopped before another applications connect to same camera devices. This application is now managed by the systemd in the user slice. So it can be stopped by a following command on k1cam1.
> systemctl --user stop test_CameraServer.service
[Yuzu, YamaT]
a2A5328-4gmPRO camera was moved from a top plate of the fire alarm rack around OMC area to a top plate of k1boot in the server room (Fig.1).
It's now connected to the 10Gbps core switch directly. So a shortage of a bandwidth might be mitigated. (Even if so, we need to reconsider the system design of high resolution cameras. But we may be able to know one of requirements of that system from a change in the situation by difference in the connected switches.) Camera servers were already shut down for the planned power outage. So a test with the new configuration will be done after recovering from the power outage.
PoE doesn't seem to be enabled on the camera switch at the server room and I couldn't enable it because IP address of this switch is missing.
So I moved a2A5328-4gmPRO near the camera switch at the IOO0 rack on which PoE is enabled and a2A5328-4gmPRO came back online again (Fig.1).
But, I found that the switch at IOO0 rack and one at server room were only linked at 1G, contrary to what I had been told.
For this reason, I haven't yet done the test with the network which should have sufficient capability.
Anyway, both a2A5328-4gmPRO and acA4112-8gm are now online.
-----
In today's work, I understood the camera network configuration is completely chaos. IP address information on JGW wiki was not true for 3 of 6 network switches. I tried to do ping-search beyond L3 segment by using virtual NIC but 2 of 6 switches including the core switch at the server room are still missing. 2 of found 4 switches don't have a reachable route because the inconsistency between the physical and logical connections. A drastic reassessment is required.
As mentioned above, the link speed between the IOO0 and the server room is 1G though it should be 10G. According to my check, I couldn't find any reason why the link speed between IOO0 and the server room is limited as 1G on the switch at the IOO0 rack. So I'm doubting it comes from the settings on the core switch at the server room. But the core switch is still missing. So we need salvage it at first. Or using USB-RJ45 console cable if someone have...