ATLAS Production System Twiki

Join this site and follow this blog to be up to date, or simply subscribe to our RSS feed!

Permanent Documentation Links
Blog Tags: prodsys1, prodsys2

Thursday, March 14, 2013

March 2013. Updated List of requirements (ProdSys SW development).

This is an updated list, the previous list can be found using link below

http://prodsys.blogspot.ch/2012/10/prodsys-splinter-meeting-october-2012.html


  1. AP transient datasets deletion
       Alexei before May 1st
         Mar 30.  version for testing is ready. Reported to MC Coordination
  2. 'clone' tasks use-cases
       Valeri, Dmitry (Wolfgang for validation and testing)
  3. fair share and priority policy
        Kaushik, Tadashi (Rod, Kaushik for testing)
  4. New G4 TRF integration (March-August)
       Sasha, Dmitry (Wolfgang, Jose for validation and testing)
   4.1 StoppedParticleG4_tf.py ☑ done March 27, 2013
   4.2 ISF Simulation: Sim_tf.py ☑ done April 11, 2013
   4.3 Simulation:
    4.3.1 AtlasG4_tf.py ☑ done April 11, 2013
    4.3.2 HITSMerge_tf.py ☑ done
    4.3.3 FilterHit_tf.py ☑ done
   4.4 Reconstruction:
    4.4.1 Reco_tf.py ☑ done
    4.4.2 Digi_tf.py ☑ done
    4.4.3 AODMerge_tf.py ☑ done
    4.4.4 ESDMerge_tf.py ☑ done
   4.5 Overlay:
    4.5.1. RAWOverlayFilter_trf.py ☑ done
    4.5.2. BSOverlayFilter_trf.py
5. log files archiving (TBD)
      Simone for technical specs
  6. For prodsysII: If a task has more than one output dataset, destination should be configurable per dataset. For example AOD datasets should go to DATADISK (default), RDO/ESD datasets should be replicated to group space. log files should go to DATADISK (default).  
    MORE INFO IS NEEDED. PLEASE DO NOT POST RANDOM REQUIREMENTS W/O DISCUSSING IT FIRST. 
  7. Implementation of the FTK emulation in the production system (March)
      High Priority for Trigger TDRs in April and September 2013.
      MC samples with FTK simulation are needed.
   7.1 Skim silicon data (1 RDO event -> 1 FTK input event) ☑ done
   7.2 FTK emulation for each FTK input event split into 64 tower regions ☑ done
     a) Emulate FTK response with TrigFTKSim_tf.py for each region split into four subregions
     b) Merge every four subregions into one tower region with TrigFTKMerge_tf.py
   7.3 Merge 64 regions into 1 FTK event with TrigFTKMerge_tf.py  ☑ done
   7.4 Combine same RDO and FTK events for reconstruction  ☑ done
  8. Integrate new FTK transformations (April)
      Sasha, Dmitry (Wolfgang for validation and testing)
   8.1 TrigFTKSM4_tf.py ☑ done
   8.2 TrigFTKMergeReco_tf.py ☑ done
  9. Provide RW needed for dynamic task brokerage (TBD)
      Sasha
10. Support for event counting in MC and GP (TBD)
      Sasha

8 comments:

  1. On request from Wolfgang, postExec argument for new G4 TRF is enclosed in quotes to escape brackets from shell.

    ReplyDelete
  2. On request from Wolfgang, support for single quotes is added to the Tag Definition.

    ReplyDelete
    Replies
    1. Support for single quotes is added to the job definition.

      Delete
  3. For item 10, I like to request the implementation of splitJobs option as default. This will allow us partial processing of AODs from reprocessed data. Thanks.

    ReplyDelete
    Replies
    1. P.S. We would like to launch a big production around ~20th of April, so would be very good to have the option ready by that time.

      Delete
  4. The Sim_tf.py integration progress is tracked in the
    https://its.cern.ch/jira/browse/PRODSYS-1

    ReplyDelete
  5. The TrigFTKSM4_tf.py integration progress is tracked in the ticket
    https://its.cern.ch/jira/browse/PRODSYS-3

    ReplyDelete
  6. Support for single quotes is added to the Task Definition. Panda monitor is updated.

    ReplyDelete