Satoshi and Kiwamu,
Related logs: 17002, 17113
We searched for a cause for the glitches that have been seen recently in the PLLs for the green lasers. We found out two issues.
- The AA filter for ADC0 in the ALS0 rack has a busted channel.
- The output of the PFD (Phase Frequency Discriminator) unit picks up electrostatic discharge for some reason, making glitches.
The first one can be fixed by replacing the AA filter and seemed to be nothing to do with the glitch events. The second one turns out to be a hard one in that we don't understand how electrostatic discharge couples to the PFD. Currently, the PFD is taken out of the rack and brought to the electronic room in Mozumi for some tests.
Busted AA channel
We noticed that the noise level of the PFD output were different between the X and Y PLLs. The channels we were looking at were K1:ALS-X|Y_BEAT_ERR_OUT_DQ. The screenshot below shows the spectra of the two channels. The reference curves (purple and dark green) were the spectra taken when both PLLs were closed. The noise levels below several 100 Hz are completely different. We then disconnected the DB9 cables that carried the signals to the AA, confirming that the noise floor without signals were already different to begin with. The red and light green curves are the ones.
Now, leaving the DB9 cables disconnected, we checked out the raw noise spectra of a few selected channels. See the result below.
ADC0_CH25 has been used for monitoring the X PLL error signal, and CH29 is for the Y PLL error signal. It is evident that the noise floor in CH25 is much higher than the others. Powering off the AA filter for them, we confirmed that the noise floors of the three channels went down to the same level at around 5e-3 cnts/rtHz.
We plant to replace the AA filter unit by a spare if available.
Electrostatic discharge coupling to the PFD
Initially, we thought that the AA filters were a cause for the glitches. However, we could not conclude so.
In the course of performing several check-ups, we serendipitously figured that the action of me standing up from a chair in front of the rack actually made glitches. This was quite repeatable. Sometimes a single glitch event occurred and sometimes several glitches occurred in 10 seconds or so. Repeating the process, we concluded that it was the electrostatic discharge. It was not vibration or acoustic sound.
We took the unit out of the ALS1 rack and brought it back to Mozumi for testing. The symptom persists -- the PFD output changes as a Heaviside function as we discharge some electrostatics nearby. We used the ignition part of a lighter to produce electrostatic discharge in the electronic lab in Mozumi. We are going to make health check up of the PFD unit, anyway.
