Job running on AS-ES-LAB-04-S1-its-c7-osg-20190309 === Contents of /cvmfs/oasis.opensciencegrid.org/gluex/update.details: === Update initiated: Fri May 24 06:05:22 CDT 2019 Update unix time: 1558695922 No proxy found for user ouser.gluex ========================================================================== Executing on node AS-ES-LAB-04-S1-its-c7-osg-20190309 JANA >>Initializing plugin "/group/halld/Software/builds/Linux_CentOS7-x86_64-gcc4.8.5-cntr/halld_recon/halld_recon-4.3.0/Linux_CentOS7-x86_64-gcc4.8.5-cntr/plugins/TPOL_tree.so" ... JANA >>Launching threads . JANA >>Opening source "root://nod28.phys.uconn.edu//Gluex/rawdata/PSskims/Run042122/hd_rawdata_042122_157.ps.evio" of type: EVIOpp - Reads EVIO formatted data from file or ET system JANA >>Created JCalibration object of type: JCalibrationCCDB JANA >>Generated via: JCalibration using CCDB for MySQL and SQLite databases JANA >>Run:42122 JANA >>URL: sqlite:////group/halld/www/halldweb/html/dist/ccdb.sqlite JANA >>context: variation=default JANA >>comment: JANA >>Creating DTranslationTable for run 42122 JANA >>Reading translation table from calib DB: Translation/DAQ2detector ... JANA >>23166 channels defined in translation table JANA >> JANA >> --- Configuration Parameters -- JANA >> EVENTS_TO_KEEP = 999999999 JANA >> EVENTS_TO_SKIP =  JANA >> EVIO:MAX_EVENT_RECYCLES = 10  JANA >> EVIO:MAX_OBJECT_RECYCLES = 10  JANA >> EVIO:MAX_PARSED_EVENTS = 10  JANA >> EVIO:NTHREADS = 1  JANA >> JANA:BATCH_MODE = 1  JANA >> PLUGINS = TPOL_tree JANA >> THREAD_TIMEOUT = 600  JANA >> THREAD_TIMEOUT_FIRST_EVENT = 3600  JANA >> ------------------------------- JANA >>In DPSCHit_factory, loading constants... JANA >>In DPSHit_factory, loading constants... JANA >>In DTAGHHit_factory, loading constants... JANA >>In DTAGMHit_factory_Calib, loading constants... JANA >>Creating DGeometry: JANA >> Run requested:42122 found:42122 JANA >> Run validity range: 42122-42122 JANA >> URL="ccdb://GEOMETRY/main_HDDS.xml" context="default" JANA >> Type="JGeometryXML" JANA >>Found 25 material maps in calib. DB JANA >>Read in 25 material maps for run 42122 containing 76153 grid points total No more events JANA >>Opening source "root://nod28.phys.uconn.edu//Gluex/rawdata/PSskims/Run042122/hd_rawdata_042122_158.ps.evio" of type: EVIOpp - Reads EVIO formatted data from file or ET system EVIO Processing rate = 116.409 Hz NDISPATCHER_STALLED = 3785828 (55.4%) NPARSER_STALLED = 3645902 (53.3%) NEVENTBUFF_STALLED = 6399 ( 0.1%) EVIO Statistics for root://nod28.phys.uconn.edu//Gluex/rawdata/PSskims/Run042122/hd_rawdata_042122_157.ps.evio : ------------------------ Nblocks: 8694 Nevents: 795723 Nerrors: 0 Nbad_blocks: 0 Nbad_events: 0 No more events JANA >>Opening source "root://nod28.phys.uconn.edu//Gluex/rawdata/PSskims/Run042122/hd_rawdata_042122_159.ps.evio" of type: EVIOpp - Reads EVIO formatted data from file or ET system EVIO Processing rate = 113.803 Hz NDISPATCHER_STALLED = 3853951 (55.2%) NPARSER_STALLED = 3704353 (53.0%) NEVENTBUFF_STALLED = 18965 ( 0.3%) EVIO Statistics for root://nod28.phys.uconn.edu//Gluex/rawdata/PSskims/Run042122/hd_rawdata_042122_158.ps.evio : ------------------------ Nblocks: 8678 Nevents: 794967 Nerrors: 0 Nbad_blocks: 0 Nbad_events: 0 No more events JANA >>Opening source "root://nod28.phys.uconn.edu//Gluex/rawdata/PSskims/Run042122/hd_rawdata_042122_165.ps.evio" of type: EVIOpp - Reads EVIO formatted data from file or ET system EVIO Processing rate = 115.301 Hz NDISPATCHER_STALLED = 3811158 (55.1%) NPARSER_STALLED = 3675346 (53.1%) NEVENTBUFF_STALLED = 6261 ( 0.1%) EVIO Statistics for root://nod28.phys.uconn.edu//Gluex/rawdata/PSskims/Run042122/hd_rawdata_042122_159.ps.evio : ------------------------ Nblocks: 8671 Nevents: 797721 Nerrors: 0 Nbad_blocks: 0 Nbad_events: 0 No more events JANA >>Opening source "root://nod28.phys.uconn.edu//Gluex/rawdata/PSskims/Run042122/hd_rawdata_042122_169.ps.evio" of type: EVIOpp - Reads EVIO formatted data from file or ET system EVIO Processing rate = 117.507 Hz NDISPATCHER_STALLED = 3770332 (55.6%) NPARSER_STALLED = 3636451 (53.7%) NEVENTBUFF_STALLED = 5991 ( 0.1%) EVIO Statistics for root://nod28.phys.uconn.edu//Gluex/rawdata/PSskims/Run042122/hd_rawdata_042122_165.ps.evio : ------------------------ Nblocks: 8671 Nevents: 796457 Nerrors: 0 Nbad_blocks: 0 Nbad_events: 0 No more events JANA >>Opening source "root://nod28.phys.uconn.edu//Gluex/rawdata/PSskims/Run042122/hd_rawdata_042122_174.ps.evio" of type: EVIOpp - Reads EVIO formatted data from file or ET system EVIO Processing rate = 114.121 Hz NDISPATCHER_STALLED = 3873029 (55.7%) NPARSER_STALLED = 3727562 (53.7%) NEVENTBUFF_STALLED = 5989 ( 0.1%) EVIO Statistics for root://nod28.phys.uconn.edu//Gluex/rawdata/PSskims/Run042122/hd_rawdata_042122_169.ps.evio : ------------------------ Nblocks: 8719 Nevents: 792822 Nerrors: 0 Nbad_blocks: 0 Nbad_events: 0 No more events JANA >>Opening source "root://nod28.phys.uconn.edu//Gluex/rawdata/PSskims/Run042122/hd_rawdata_042122_175.ps.evio" of type: EVIOpp - Reads EVIO formatted data from file or ET system EVIO Processing rate = 117.85 Hz NDISPATCHER_STALLED = 3780102 (56.0%) NPARSER_STALLED = 3635834 (53.9%) NEVENTBUFF_STALLED = 6356 ( 0.1%) EVIO Statistics for root://nod28.phys.uconn.edu//Gluex/rawdata/PSskims/Run042122/hd_rawdata_042122_174.ps.evio : ------------------------ Nblocks: 8690 Nevents: 794868 Nerrors: 0 Nbad_blocks: 0 Nbad_events: 0 No more events JANA >>Opening source "root://nod28.phys.uconn.edu//Gluex/rawdata/PSskims/Run042122/hd_rawdata_042122_180.ps.evio" of type: EVIOpp - Reads EVIO formatted data from file or ET system EVIO Processing rate = 117.78 Hz NDISPATCHER_STALLED = 3784508 (56.1%) NPARSER_STALLED = 3647179 (54.1%) NEVENTBUFF_STALLED = 5893 ( 0.1%) EVIO Statistics for root://nod28.phys.uconn.edu//Gluex/rawdata/PSskims/Run042122/hd_rawdata_042122_175.ps.evio : ------------------------ Nblocks: 8692 Nevents: 794598 Nerrors: 0 Nbad_blocks: 0 Nbad_events: 0 No more events JANA >>Opening source "root://nod28.phys.uconn.edu//Gluex/rawdata/PSskims/Run042122/hd_rawdata_042122_182.ps.evio" of type: EVIOpp - Reads EVIO formatted data from file or ET system EVIO Processing rate = 109.137 Hz NDISPATCHER_STALLED = 4054467 (55.8%) NPARSER_STALLED = 3889880 (53.5%) NEVENTBUFF_STALLED = 6179 ( 0.1%) EVIO Statistics for root://nod28.phys.uconn.edu//Gluex/rawdata/PSskims/Run042122/hd_rawdata_042122_180.ps.evio : ------------------------ Nblocks: 8697 Nevents: 793576 Nerrors: 0 Nbad_blocks: 0 Nbad_events: 0 No more events JANA >>Opening source "root://nod28.phys.uconn.edu//Gluex/rawdata/PSskims/Run042122/hd_rawdata_042122_184.ps.evio" of type: EVIOpp - Reads EVIO formatted data from file or ET system EVIO Processing rate = 114.803 Hz NDISPATCHER_STALLED = 3912709 (56.6%) NPARSER_STALLED = 3754041 (54.3%) NEVENTBUFF_STALLED = 6902 ( 0.1%) EVIO Statistics for root://nod28.phys.uconn.edu//Gluex/rawdata/PSskims/Run042122/hd_rawdata_042122_182.ps.evio : ------------------------ Nblocks: 8696 Nevents: 793808 Nerrors: 0 Nbad_blocks: 0 Nbad_events: 0 JANA >> JANA >>Telling all threads to quit ... JANA >>Merging thread 0 (0x2b292ea02700) ... Time in minutes for the plugin to complete: 928.264 EVIO Processing rate = 0 Hz NDISPATCHER_STALLED = 14 ( 0.0%) NPARSER_STALLED = 0 ( 0.0%) NEVENTBUFF_STALLED = 474616 (38.5%) EVIO Statistics for root://nod28.phys.uconn.edu//Gluex/rawdata/PSskims/Run042122/hd_rawdata_042122_184.ps.evio : ------------------------ Nblocks: 1 Nevents: 0 Nerrors: 0 Nbad_blocks: 0 Nbad_events: 0 JANA >>Merging event reader thread ... JANA >>Opening source "root://nod28.phys.uconn.edu//Gluex/rawdata/PSskims/Run042122/hd_rawdata_042122_157.ps.evio" of type: EVIOpp - Reads EVIO formatted data from file or ET system JANA >>Opening source "root://nod28.phys.uconn.edu//Gluex/rawdata/PSskims/Run042122/hd_rawdata_042122_158.ps.evio" of type: EVIOpp - Reads EVIO formatted data from file or ET system JANA >>Opening source "root://nod28.phys.uconn.edu//Gluex/rawdata/PSskims/Run042122/hd_rawdata_042122_159.ps.evio" of type: EVIOpp - Reads EVIO formatted data from file or ET system JANA >>Opening source "root://nod28.phys.uconn.edu//Gluex/rawdata/PSskims/Run042122/hd_rawdata_042122_165.ps.evio" of type: EVIOpp - Reads EVIO formatted data from file or ET system JANA >>Opening source "root://nod28.phys.uconn.edu//Gluex/rawdata/PSskims/Run042122/hd_rawdata_042122_169.ps.evio" of type: EVIOpp - Reads EVIO formatted data from file or ET system JANA >>Opening source "root://nod28.phys.uconn.edu//Gluex/rawdata/PSskims/Run042122/hd_rawdata_042122_174.ps.evio" of type: EVIOpp - Reads EVIO formatted data from file or ET system JANA >>Opening source "root://nod28.phys.uconn.edu//Gluex/rawdata/PSskims/Run042122/hd_rawdata_042122_175.ps.evio" of type: EVIOpp - Reads EVIO formatted data from file or ET system JANA >>Opening source "root://nod28.phys.uconn.edu//Gluex/rawdata/PSskims/Run042122/hd_rawdata_042122_180.ps.evio" of type: EVIOpp - Reads EVIO formatted data from file or ET system JANA >>Opening source "root://nod28.phys.uconn.edu//Gluex/rawdata/PSskims/Run042122/hd_rawdata_042122_182.ps.evio" of type: EVIOpp - Reads EVIO formatted data from file or ET system JANA >>Opening source "root://nod28.phys.uconn.edu//Gluex/rawdata/PSskims/Run042122/hd_rawdata_042122_184.ps.evio" of type: EVIOpp - Reads EVIO formatted data from file or ET system JANA >> 7154539 events processed (7154540 events read) Average rate: 114.3Hz JANA >>Closing shared object handle 0 ... Error: hd_ana exited with error code 70, giving up. Job finished with exit code 70