Workflow tidy-ups: rho/tau rule script, TreeCorr CPU count, plain /nXXdataN paths - #368
Draft
cailmdaley wants to merge 4 commits into
Draft
cailmdaley wants to merge 4 commits into
cailmdaley wants to merge 4 commits into
Conversation
…rivate COSMO_INFERENCE
- workflow/scripts/run_rho_tau.py is a plain rule script: the IPython
autoreload branch, hard-coded interactive paths and progress prints go;
stream buffering and the output check come from cv_runner.
- workflow/README: launching with COSMO_INFERENCE=<tree> of one's own, with
the mask link the CosmoCov chain needs.
- Docstrings: calculate_rho_tau_stats; sacc_writers notes that the
integration-grid ξ± part feeds COSEBIs and pure-E/B and does not join
{version}.sacc.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
treecorr_config defaults num_threads to the process's CPU affinity. TreeCorr's own default is multiprocessing.cpu_count(), the node's count: a job holding 16 of a node's 24 CPUs ran 24 threads. A caller's num_threads still overrides it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ling common._plain resolves a path and spells /automnt/<disk>/... back as /<disk>/..., on every host; REPO_ROOT, COSMO_VAL and COSMO_INFERENCE go through it. Snakemake keys its persistence records and matches targets by path string, so a symlinked or /automnt-spelled root no longer declares paths a plainly named target cannot match, and a job step re-deriving the launch's paths on the node that owns the disk (which has no /automnt/<disk>) declares paths it can reach. The README names the disks a job sees under their plain spelling. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Small workflow tidy-ups. No change to any output.
What changes
workflow/scripts/run_rho_tau.pyis now a plain rule script. The IPython/autoreload branch, the hard-coded interactive paths and the progress prints are gone. Stream buffering and the output check come fromcv_runner(_unbuffer_streams,verify_outputs), like the other rule scripts.workflow/README.md: only its owner can write the sharedCOSMO_INFERENCEtree. Anyone else launches with a tree of their own that links the shareddata/mask/, the one input the CosmoCov rules take from it.os.sched_getaffinity), not the node's count, so a job holding 16 of 24 cores runs 16 threads. ξ± does not depend on the thread count (min_topis pinned).workflow/common.pyspells the checkout and output roots in the plain/nXXdataNform, so Snakemake targets match by path string and resolve on the node that owns the disk.calculate_rho_tau_stats.Verified
No test covers
run_rho_tau.py, so I ran it in the container against a synthetic catalogue, and it wrote all three declared outputs. Container suite: 307 passed, 1 xfailed. Host workflow tests: 10 passed, 1 skipped (the SLURM smoke test).— Claude (Opus) on behalf of Cail.
🤖 Generated with Claude Code