ngmix masked pixels: independent neighbour and defect handling (UberSeg, noise fill, interpolation, epoch cuts)

[cailmdaley]

This PR brings ShapePipe's approaches to masked pixels in ngmix into one design. It lands after the first Nibi production run, and changes no defaults on its own: defaults move only after the validation below (#813). It supersedes #915 and #916, and carries the UberSeg segmentation-map path split out of #897.

Design

Each pixel is asked two independent questions, and each has one key: is it a neighbour's pixel, and is it a defect? Defects are treated the same way under every blend handling. That way an A/B test on blend handling changes only the neighbour pixels. In earlier uberseg-vs-noisefill comparisons the defect treatment changed too, which confounded them.

Key	Values	Default
BLEND_HANDLING	none | uberseg	none
SEG_VIGNET_PATH	path; required iff uberseg	unset
DILATE_NEIGHBOUR	int ≥ 0	1
DEFECT_FILL	noise | interpolate	noise
EPOCH_CENTRAL_DEFECT_RADIUS	px, 0 disables	open (1)
EPOCH_INTERPOLATED_DEFECT_RADIUS	px	7
EPOCH_MASKED_FRACTION_CUT	(0, 1], counts all defects	1/3

• Neighbours. Under uberseg, neighbour pixels get zero weight and keep their raw light. This follows DES SV–Y6 practice: same-redshift neighbour light shears coherently under metacal (UberSeg neighbour masking (Erin Sheldon) — config-selectable, validated against noise-fill #776). none is develop's current behaviour. noisefill is renamed to none, because it never filled neighbours.
• Defects. Defects get zero weight and a fill. noise is the per-pixel RMS noise fill that develop uses today. interpolate (from ngmix: opt-in DEFECT_FILL = interpolate for narrow defects, with a 7 px veto #916) uses Clough–Tocher interpolation over runs of ≤ 3 px, averaged over quarter turns, and gives the quarter-turn orbit zero weight. This takes c₁ from −1.3e-3 to 2e-6 on the ngmix: opt-in DEFECT_FILL = interpolate for narrow defects, with a 7 px veto #916 grid.
• Epoch cuts. An epoch is dropped when a defect falls within a radius of the object. There is one radius for filled defects and one for interpolated ones. It is also dropped when its masked fraction exceeds EPOCH_MASKED_FRACTION_CUT, which now counts all defects, not only flags. Every cut is logged, so each run reports how many epochs it lost.
• Workflow. blend_handling and defect_fill go in config.yaml and are rendered into the ngmix template. Today neither reaches ngmix. uberseg adds a third vignetmaker run for segmentation stamps, plus the matching completeness entry. In catalogue mode it also adds tile_segmentation and seg_relabel.py. It is wired in both cfis and cfis_image_sims.

Not implemented; these remain A/B options in the decision record: filling the neighbour pixels themselves, MOF neighbour subtraction, and symmetrizing the neighbour weights (#815). The symmetrized defect fill is not used here because #915's own grid rejects it as a fill (m₁₁ −2.7% vs −0.64%); its code stays on feat/symmetrized-defect-fill for the symmetrization work in #815.

Validation before any default changes

Gate	Decides
#915/#916 test grids; one real tile per detection mode × blend handling	Merge; seg-map alignment in practice
Epoch-loss and defect-incidence census, 20–50 real tiles (from the cut log)	Veto radii, fraction cut (1/3 vs DES's 10%), noise vs interpolate
Blended-sims ladder, both arms	UberSeg Δm against the 0.005 target; σₑ / N_eff cost (#813, #814)
Data A/B on matched tiles incl. FLAGS ≤ 2, pure-EB / COSEBIs	Does UberSeg remove the FLAGS=2 B-modes (#814), and at what E-mode cost

Both A/B arms run on this code. The Nibi catalogue cannot serve as one of them, because this PR also changes which epochs are kept.

Open decisions

1. Central-defect veto on by default? A flagged column through the stamp centre gives c₁ = +0.18 in that epoch under today's fill, and CFHT's fixed orientation makes such biases coherent. Against: A/B validation of off-by-default shape-measurement options #813's contract, the pinned default in the decision record, and the fact that epoch loss hasn't been measured. The census resolves the radius either way.
2. Matching the seg map to Stephen Gwyn's objects: the footprint under each position (seg_relabel.py), or SExtractor ASSOC. The relabel's fallback count on one real tile settles it.
3. Should UberSeg's neighbour hole get defect-style treatment? It's a one-sided zero-weight hole, the same mechanism as the defect c₁. Neither epoch cut sees it. The A/B: 4-fold weight symmetrisation — shape impact and S/N cost #815 arm on the ladder settles it.
4. Where does UberSeg live: in ngmix (ngmix: config-selectable UberSeg neighbour masking (BLEND_HANDLING) #770), or in vignetmaker / its own module so other shape methods can reuse it (Masked pixels in ngmix: neighbour (UberSeg) and defect handling #602)?
5. Does the coadd seg map overlaid on epochs hold up without reprojection, at DILATE_NEIGHBOUR = 1? Check the neighbour flag and the registration warnings on the ladder.

State

• ngmix: noise-fill defects under both BLEND_HANDLING options, drop epochs with a defect near the object #915, then ngmix: opt-in DEFECT_FILL = interpolate for narrow defects, with a 7 px veto #916, onto develop
• UberSeg seg-map commit (from Use Stephen Gwyn's UNIONS tile catalogue as the tile galaxy sample #897's split)
• Workflow wiring
• Decision-record entries: defect_fill, blend_handling, central_defect_veto, epoch_masked_fraction_cut
• Validation gates above

The seg-map branch is built on #897, so #897's content (UNIONS catalogue detection, TILE_UNIQUE_ID, ngmix row chunks) rides along here until #897 merges into develop.

Closes #602.

🤖 Generated with Claude Code