1. Overview
  2. Deployment
  3. Monitoring
  4. Maintaining
  5. Common Errors and Known Solutions
  6. Features
  7. Extinction Model and Burn-in
  8. Scientific Results
  9. Low-Latency Injections
  10. References
  11. Template Banks
  12. Pastro
  13. Offline Analysis
  14. Offline Filtering Rota
  15. Offline Rota
  16. Dropped data filtering
  17. Offline SNR Optimizer/Coinc generation
  18. MDC Documentation
  19. How to contribute to this documentation
  20. Diagrams

  1. Overview
  2. Deployment
  3. Monitoring
  4. Maintaining
  5. Common Errors and Known Solutions
  6. Features
  7. Extinction Model and Burn-in
  8. Scientific Results
  9. Low-Latency Injections
  10. References
  11. Template Banks
  12. Pastro
  13. Offline Analysis
  14. Offline Filtering Rota
  15. Offline Rota
  16. Dropped data filtering
  17. Offline SNR Optimizer/Coinc generation
  18. MDC Documentation
  19. How to contribute to this documentation
  20. Diagrams

Offline Rota

Table of Contents


Overview

Every two week during O4, we will create create an Open Box of the events we’ve seen so far. The Open Box consists of a rerank of the online data products (triggers and background). We use the branch offline_new-workflow, since it contains the code to find the required online data products, rerank them, and also implements the new extinction model. The ROTA starts on Friday, and involves performing reranks on Edward and Jacob from the start of ER15 till the midnight of the previous day (Thursday, EST), combining the two reranks, uploading the results to GraceDB, creating an open box presentation, and presenting it at the next CBC call.

Setting up the reranks

All-sky online rerank

  1. The reranks are set up in these directories:
    Edward: /home/gstlalcbc.offline/observing/4/a/runs/trigs.edward_yymmdd
    Jacob: /ligo/home/ligo.org/gstlalcbc/observing/4/a/offline/runs/trigs.jacob_yymmdd
    where yymmdd is the date on Friday.
    For GWTC runs, the directories can be set up as follows:
    Edward: /home/gstlalcbc.offline/observing/4/a/runs/gwtc4(5)_ranks/trigs.edward_yymmdd
    Jacob: /ligo/home/ligo.org/gstlalcbc/observing/4/a/offline/run/gwtc4(5)_ranks/trigs.jacob_yymmdd
    There is already a container set up with the correct branch in the relevant build dirs.

  2. You can download the required config from the O4a-offline repo (or copy it over from a previous rerank). In the config, you should change all the necessary parameters. If you’re copying it from a previous rerank, you’ll only have to change stop, rank-dir, and the summary section. You can create a Makefile by doing

    singularity exec -B /home <build-dir> gstlal_inspiral_workflow init -c config.yml -w rank

  3. Once that is done, you can set up the dag by doing

    singularity exec -B /home <build-dir> make all -j<N>

    Here, <N> should be maximum 5 for CIT, 10 for ICDS, and 32 for NEMO when this command is run on the head node where online dags are running. Note that you may need to do export OPENBLAS_NUM_THREADS=1 for the -j to work. You can submit the dag after the make command finishes (make sure you have a valid proxy), and create the summary page once the dag finishes.

IMBH offline rerank (for GWTC results)

  1. The rerank directory follows this naming convention: IMBH: /home/gstlalcbc.offline/observing/4/a/runs/gwtc4(5)_ranks/trigs.imbh_yymmdd

  2. You can download the following files from IMBH DCC entry (or copy it over from a previous rerank):

    • config_offline_rank.yml
    • reassigned_H1L1V1-O4_MANIFOLD_BANK-0-2000000000.xml
    • mass_model_combined_IMBHreassigned_allskyfull_BANK.h5
    • inspiral_dtdphi_pdf.h5 In the config, you should change all the necessary parameters. If you’re copying it from a previous rerank, you’ll only have to change data-dir, rank-dir, and the summary section (assuming there is no change in chunk-wise filtering). Also, move mass_model_combined_IMBHreassigned_allskyfull_BANK.h5 under mass_model/.

  3. You can create a Makefile by runing

    singularity exec -B /home <build-dir> gstlal_inspiral_workflow init -c config_offline_rank.yml -w rank

    In a generated Makefile, modify

    gstlal_inspiral_set_svdbin_option --config config_offline_rank.yml -w # ADDME

    to

    gstlal_inspiral_set_svdbin_option --config config_offline_rank.yml -w rank

  4. Generate a dag file

    singularity exec -B /home <build-dir> make rank

    Note that you might want to double request memory and disk in these sub files:

    • gstlal_inspiral_calc_likelihood.sub
    • gstlal_inspiral_calc_rank_pdfs.sub
    • gstlal_inspiral_plot_bin_background.sub

Combining the reranks

A bit of details about the new extinction model first. It performs two rounds of extinction, the first on every bin’s individual dist_stat_pdf. These get marginalized and saved to a file like rank/dist_stat_pdfs/13678/H1L1-GSTLAL_DIST_STAT_PDFS-1367800000-7356818.xml.gz. Then, the zerolag is added to this PDF, the second round of extinction with this zerolag is performed, and the result gets saved to rank/post_dist_stat_pdfs/13678/H1L1-GSTLAL_POST_DIST_STAT_PDFS-1367800000-7356818.xml.gz. When combining analyses, we don’t want to use the analysis-specific zerolag, we want to use the clustered-across-analyses zerolag. We also want to perform the second round of extinction not with the analysis-specific zerolag, but with the clustered-across-analyses zerolag. Consequently, we need to take the zerolag of the two analyses, cluster them, and then take the singly-extincted PDFs from both analyses, marginalize them and perform the second round of extinction with the clustered-across-analyses zerolag. After that, we can compute FARs for the new zerolag with the new doubly-extincted PDF. Fortunately, there’s a script which does all this for you.

All-sky open box

  1. In the IMBH rerank dir, 
    mkdir combined
cd combined
  2. Get the combine_analyses_far script from the the O4a-offline repo (or copy it over from a previous rerank). You’ll need to change the ANALYSIS_DATE and STOP parameters. Just run that script, and the combined summary page should be ready.

GWTC results (All-sky and IMBH combined)

  1. In the Edward rerank dir, 
    mkdir combined
cd combined
  2. Get the following files from the the O4a-offline repo (or copy it over from a previous rerank).
    • combine_analyses_with_trials_factor
    • far_simplify_and_cluster.sql You’ll need to change the ANALYSIS_DATE and STOP parameters. Just run that script, and the combined summary page should be ready.

Uploading the results to GraceDB

We intend to upload every week’s results to GraceDB with the offline tag. Since the results are cumulative, there will be a version number for the results, so people can look at the latest ones. This is still a work in progress.

Creating an Open Box presentation

Get the print_open_box_tables script from the O4a-offline repo, and run it like this:

./print_open_box_tables --start 1367800000 --stop 1375156818 --database H1L1-0000_1839_GSTLAL_TRIGGERS-1367800000-7356818.sqlite --far-threshold 3.8e-7 --previous-database ../../trigs.edward_230721/combined/H1L1-0000_1839_GSTLAL_TRIGGERS-1367800000-6147218.sqlite

This will print out a bunch of tables compatible with markdown. Create a git issue for the week’s offline ROTA in the the O4a-offline repo issues page, and copy and paste the tables in the issue description. This will be the open box presentation for that week.

Presenting at the CBC call

If there are significant changes or new events that are intersting and should be discussed, you can sign up to give a presentation at the CBC call.