owl shift, Sunday 8/29/99

John Shields (jshields@ksera)
Sun, 29 Aug 1999 06:56:07 -0500

Shift Summary for owl shift, 08/29/99

On Shift: J. Shields, J. Ladue

Hours of beam available at all intensities: 7.0
Hours of beam available at nominal intensity: 7.0
Hours of E832 data (special runs not included): 6.2
Hours of special runs (laser scans, muon runs...): 5 min

Accounting of lost time:

tape change, ramping of magnet for muon run, a bunch of kfrends.
Runs ended this shift:
Run 14370 - 19.7M events - BEAM_832
Run 14371 - BEAM_MUON

Tapes brought to FCC this shift: QK*345-347, QKB120
(including tapes for online split)

on-split: QKSM15,QKCC16,QKNL28,QKA20
Accesses during shift:


Overview of shift:
E832 datataking, investigated "DAQDEAD_SEC" errors, muon run

Description of Shift:

run spill time comment

1470 105 01:06 ktevana error, stream 5. Kfrended

113 01:17 ktevana error, stream 5. Kfrended

01:30 put in new on-split tape QKCC16

01:50 put in new on-split tape QKA020

154 02:11 ktevana error. Unknown crate. Kfrended.

03:00 Began getting "DAQDEAD_SEC out of limits"
errors. Looked at various histograms and see
only that an L3 process on ktev2 is using much
less time than normal. However, it IS
processing events.
The dead time is ~4-5 sec on average,
ranges from 1-12 sec in length, and occurs once
every 2 spills. Apparently a similar thing
occurred on the sunday 8/15/99 owl shift, but
the problem went away on its own that time.

03:30 the above errors continue. Shift crew calls
control room and Maxwell admits that he was
playing with some chromoticity stuff around 3 am,
when our errors showed up (*bad* Maxwell!
*BAD*!) He changes everything back, but it has
no effect. Shift crew throws up their collective
hands and pages Ed. Lucky Ed.

03:50 Ed notices that process "fsr" is running on ktev
and using ridiculous amounts of CPU. Apparently
fsr is some kind of de-fragmenter.
Final verdict is to ignore the problem and
hope it goes away when we initialize a new run.

232 03:54 ktevana errors

233 03:56 ktevana errors, stream 5-- missing FFF
word. Kfrended.

252 04:21 ktevana errors, stream 5-- missing FFFF word

260 04:30 ktevana errors, stream 5-- missing FFFF word

263 04:32 ktevana errors, stream 5-- missing FFFF word

266 04:45 ktevana errors, stream 5-- missing FFFF word

268 04:50 pipeline errors. INIT pipeline w. DBCs and

288 05:09 about 5 spills ago, DAQDEAD_SEC errors stop!
Looked at ktev machines and lo and behold, they
were no longer running "fsr".

05:16 tapes full. End of run

05:21 update ADGN constants

14371 05:40 take 5 min muon run

05:47 warning: 91% disk usage on ktev4A. Crew
responds by removing file

14372 06:01 began new E832 run

Follow up Notes on Previous Shift

did muon run. Also checked CSI chan 450 and found it to be stable.

Items to Follow up on Next Shift

page someone to replace Ar/Eth West