Metacal flags are nonzero whenever a type has no measured shear, using ngmix's own bits

[cailmdaley]

Closes #853. Closes #889.

sp_validation keeps a galaxy only if NGMIX_MCAL_FLAGS == 0 and NGMIX_MCAL_TYPES_FAIL == 0, i.e. metacal succeeded. ShapePipe writes 0 in two cases where it did not:

• Objects ngmix never fit (no usable epoch, or the fit raised). make_cat fills their flag columns with 0 and their shapes with −10. In the 64-tile smk-g7 catalogue that is 18,983 of 1,851,100 objects (1.03%); only a separate -10 sentinel check keeps them out of the shear sample.
• A metacal type that reports flags == 0 with a non-finite or missing shear. It counts as a success.

Design. One function, get_type_flags in ngmix.py, decides whether a metacal type succeeded. NGMIX_FLAGS_<SHEAR>, NGMIX_MCAL_FLAGS (their OR) and NGMIX_MCAL_TYPES_FAIL (the count of nonzero types) all derive from it, including the values make_cat writes for never-fit objects, so the three columns cannot disagree. Every failure carries one of ngmix's own flag bits; ShapePipe adds none, so every nonzero value decodes with ngmix.flags.

Case	Flag written
Fitter reported failure	its own flags, unchanged
flags == 0 but g non-finite or missing	ngmix.flags.LM_FUNC_NOTFINITE (2**12)
No flags key, or metacal type absent	LM_FUNC_NOTFINITE
Never-fit object (every type treated as absent)	LM_FUNC_NOTFINITE in all five FLAGS_<SHEAR> and MCAL_FLAGS; MCAL_TYPES_FAIL = 5

A genuine LM fit can also set LM_FUNC_NOTFINITE; NGMIX_N_EPOCH == 0 identifies never-fit rows. The consumer cut is == 0, so it rejects both either way.

Also in this PR

• ngmix raises before fitting when centroid_source = "wcs" (the default) and the vignettes lack OFFSET (cut by an older vignetmaker). Otherwise every fit raises and the tile silently yields an empty catalogue.
• ngmix logs an error when a tile fits no object, or every fitted object is flagged. It logs rather than raises, so one empty edge tile cannot abort a campaign.
• ngmix_runner's early exits for tiles whose PSF or galaxy stamps are all empty now log the same error and write the empty catalogue the workflow's completeness check expects.
• astra.yaml: the catalogue_assembly.failure_sentinels rationale and option label describe the new flag values; the pinned option is unchanged.

Tests. Synthetic results for each failure path, plus two never-fit objects, go through ngmix's writer and make_cat. The test asserts that the galaxy cut keeps exactly the two clean objects, checks each object's exact MCAL_FLAGS, that MCAL_FLAGS and MCAL_TYPES_FAIL equal the OR and count of the per-type columns, and that no bit outside ngmix.flags appears. A parametrised test checks that NaN or ±inf in either shear component of any type yields LM_FUNC_NOTFINITE. An end-to-end shapepipe_run on an all-empty tile must write one empty catalogue and log the error once. Full suite: 689 passed, 1 skipped (plus the 2 known errors in tests/cluster/test_star_shear_response.py).

For reviewers

• Companion: Write metacal flag columns as int32; reduce_mem never narrows integers sp_validation#354 widens the flag columns to int32, so bit 12 and above survive into the comprehensive catalogue intact. The two can merge in either order: the == 0 cut rejects these objects even when the value wraps; only the decodable bit content needs Tutorial updates #354.
• Merge order with ngmix masked pixels: independent neighbour and defect handling (UberSeg, noise fill, interpolation, epoch cuts) #922 (masked pixels): a trial merge conflicts in ngmix.py (end of Ngmix.process) and ngmix_runner.py (import line), from this PR's run-health and empty-tile changes. Keeping both sides resolves both.

— Claude (Opus) on behalf of Cail

🤖 Generated with Claude Code

[cailmdaley]

see question on CosmoStat/sp_validation#354 (comment)

[cailmdaley]

@martinkilbinger in the end I mapped our failure cases onto the existing ngmix bits so the dtype doesn't need to be changed in sp_validation. however there were other problems with the bit-writing on the SP_validation side, so CosmoStat/sp_validation#354 still fixes these but is not high-priority.