2022/07/02 (Saturday)

Lab in-personnel: Giulia

Status	Run 76 in progress
Setup	Cables 2/3/1 → diverter/nothing/nothing → Side2-readout HV CONN 1/3/2 → HV 1(L)/2(R) → DCRC50(3)/DCRC55(1), DCRC33(2)→HV control, NaI trigger signal → DCRCX(4) Lead/poly shield fully assembled. Boron-filled socks covering NaI detectors. PuBe source behind lead wall. NaI trigger window -t10760 -T15760 (7.6-11.2 MeV)
0905	Check phonon channels. Relock DCRC3pA&B. DCRC3pB has SQUID locks back to where it was a couple weeks ago: offset 0.4 → baseline ~ -0.8. DCRC3pA required an offset change, 0.55→ 0.3.
0911	Start 0 V PuBe data series 65 min long, with 1000 initial randoms, 5 Hz in-run randoms, trigger on DCRC3pD threshold 10. Start series with the Start Button.
1020	MC = 29.1 mK
1030	MC/Coldplate = 29.0/152 mK, Still = 1.08 K / 0.095 mbar / 0 mW, G1/G2 = 131/125 mbar, 1K Pot = 60° / 1.55 K / 8.0±0.5 mbar, MB = 74.1%
1037	Filled LN2 cold trap dewar
1047	Dial HV up to -227.1 V using RootSequencer for prebias. Set it to rise about 5 V/s with 0.25 second steps.
1052	After 5 minutes dial HV down to -197.3 V
1055	Start -197.3 V PuBe data series 171 min long, with 1000 initial randoms, 5 Hz in-run randoms, trigger on DCRC4pA threshold 20. Start series with the Start Button.
1105	Check Vtest on the HV board: around -79 or -80 mV. Shoot, this is still too high - pulses look better today but leakage is still bad.
1110	Check again the purge gas flow at the Vent port on the LN2 Dewar. Didn't seem like there was any flow. Open valve more to restart flow
1110	Check Vtest again: -66 mV, better.
1115	Vtest = -44 mV, better
1118	Vtest = -5 mV, much better
1130	Vtest = -2 mV, better
1345	Something is wrong with the data. Dump sizes were around 7M till Dump 11, then rate fell off during Dump 12, then after that no traces, dumps only 394K. But randoms are on so there should always be something, DAQ problem presumably. Gonna stop the series.
1347	Try stopping the run, MIDAS is hung up trying to stop
1400	Getting an error trying to start a new run, DCRC_driver and Tower_fe crash. Gonna restart MIDAS
1405	Crap - restarting MIDAS didn't help, getting the same problem

Messages

14:04:56.410 2022/07/02 [triggerfe2,INFO] Client 'towerfe3_01' on buffer 'SYSTEM' removed by cm_watchdog because process pid 2035980 does not exist

14:04:56.408 2022/07/02 [triggerfe2,INFO] Client 'dcrc_driver01' on buffer 'SYSTEM' removed by cm_watchdog because process pid 2035954 does not exist

14:04:54.806 2022/07/02 [mhttpd,INFO] Run #47581 start aborted

14:04:54.804 2022/07/02 [mhttpd,ERROR] [midas.c:4444:cm_transition,ERROR] Could not start a run: cm_transition() status 503, message '(null)'

14:04:54.703 2022/07/02 [mhttpd,ERROR] [midas.c:5730:cm_shutdown,ERROR] Killing and Deleting client 'dcrc_driver01' pid 2035954

14:04:54.702 2022/07/02 [mhttpd,ERROR] [midas.c:5728:cm_shutdown,ERROR] Client 'dcrc_driver01' not responding to shutdown command

14:04:54.701 2022/07/02 [mhttpd,ERROR] [midas.c:5730:cm_shutdown,ERROR] Killing and Deleting client 'towerfe3_01' pid 2035980

14:04:54.699 2022/07/02 [mhttpd,ERROR] [midas.c:5728:cm_shutdown,ERROR] Client 'towerfe3_01' not responding to shutdown command

14:04:49.792 2022/07/02 [mhttpd,ERROR] [midas.c:3660:cm_transition_call,ERROR] cannot connect to client "dcrc_driver01" on host cdms1.spa.umn.edu, port 25390, status 503

14:04:49.789 2022/07/02 [mhttpd,ERROR] [midas.c:9321:rpc_client_connect,ERROR] timeout on receive remote computer info: 

14:04:49.787 2022/07/02 [mhttpd,ERROR] [midas.c:3660:cm_transition_call,ERROR] cannot connect to client "towerfe3_01" on host cdms1.spa.umn.edu, port 26873, status 503

14:04:49.785 2022/07/02 [mhttpd,ERROR] [midas.c:9321:rpc_client_connect,ERROR] timeout on receive remote computer info:

1420	Tried disabling all but DCRC1 and restarting DAQ programs - same behavior. Tried disabling all but DCRC3 - was able to start a run OK that way. hmm.
1425	With DCRC3,2,4 all enabled, was able to start DAQ programs and start a series with no problem.
1425	Try again with all DCRCs enabled, same problem. So maybe we need to powercycle DCRC1.
1435	Dial HV down to 0V. This was a little tricky. At first I tried this with all DCRCs enabled including DCRC1, but I guess running the RootSequencer produced the same problem - HV didn't change and DAQ programs stopped. I looked at the ODB and it looked like DCRC2 qI was set to >11, so the commands went to the ODB but didn't go to the DCRC. Then I got the programs started again with DCRC1 disabled and I think that suddenly reset DCRC2qI, so the bias suddenly went to 0V.

Next Day