diff --git a/docs/customize.rst b/docs/customize.rst index e5e5a24a..3f4a18e7 100644 --- a/docs/customize.rst +++ b/docs/customize.rst @@ -162,7 +162,7 @@ Fixing the case of a particular word ``capitalization_exceptions`` is the one pair-valued field — each entry maps a lowercase key to its exact-cased replacement (``"phd"`` → -``"PhD"``), so it isn't a fit for ``add()``/``remove()``. Change it with +``"Ph.D."``), so it isn't a fit for ``add()``/``remove()``. Change it with ``dataclasses.replace()`` instead, and pass the result to ``capitalized()``: @@ -170,17 +170,17 @@ maps a lowercase key to its exact-cased replacement (``"phd"`` → >>> import dataclasses >>> from nameparser import parse - >>> str(parse("jane smith dds").capitalized()) - 'Jane Smith Dds' + >>> str(parse("jane smith dphil").capitalized()) + 'Jane Smith Dphil' >>> default = Lexicon.default() >>> lex = dataclasses.replace( ... default, ... capitalization_exceptions=tuple(default.capitalization_exceptions) - ... + (("dds", "DDS"),)) - >>> str(parse("jane smith dds").capitalized(lex)) - 'Jane Smith DDS' + ... + (("dphil", "DPhil"),)) + >>> str(parse("jane smith dphil").capitalized(lex)) + 'Jane Smith DPhil' -Note the ``tuple(...) + ...``: assigning a bare ``(("dds", "DDS"),)`` +Note the ``tuple(...) + ...``: assigning a bare ``(("dphil", "DPhil"),)`` would *replace* the default exceptions rather than extend them, so ``"phd"`` and the rest would stop being fixed. diff --git a/docs/design/decisions.md b/docs/design/decisions.md index 8c8b0fd0..19489dbb 100644 --- a/docs/design/decisions.md +++ b/docs/design/decisions.md @@ -579,6 +579,10 @@ Excluded (SUFFIX_ACRONYMS — esq, removed 2026-09-08, #316/#489 bundle): - esq is OUT of SUFFIX_ACRONYMS and must not be put back by a sweep that finds "Esq." parsing and assumes the acronym set is what carries it. The SUFFIX_WORDS membership carries every single-token spelling — Esq, Esq., ESQ, esq — and stops being inert with the acronym entry gone. What the acronym entry uniquely covered is the multi-dot spelling, so "John Smith E.S.Q." reads family E.S.Q. where every release since 1.4.0 read suffix; that is the whole of the cost and it is a spelling nobody writes. The criterion is #suffix-acronym-collisions', asked of the WORD rather than of the frequency: does the entry describe the word or the machinery. Esquire is a contraction, not an initialism, so the initialism set was never its home; it arrived in the 2019-12-11 bulk Wikipedia post-nominal import (af5bdab, #93) and was never reviewed. This SUPERSEDES the dual-membership entry that stood here from the 2.2 cycle, which called the acronym membership load-bearing and the word membership inert and concluded that the singleton "is why the two sets cannot carry a disjointness assert". The assert now exists — `not (SUFFIX_ACRONYMS & SUFFIX_WORDS)` in suffixes.py's guard block — and it is what the removal buys: the two sets normalize differently (the word test strips edge periods, the acronym test strips all of them), so a word in both is matched by two rules and which one fired is unreadable from outside. AGENTS.md's esq gotcha, the two defending comment blocks in suffixes.py, the `# NOT asserted:` note and the `suffix_acronym_multidot_spelling` case row all retire with it; a row pinning `John Smith E.S.Q.` → family replaces the last of those. Measured over the corpus glob as it stood at the change (1123 distinct names, 1263 rows; the docs commit that follows adds this bundle's rules.md examples and makes it 1136 and 1284): exactly one moves, `John Smith E.S.Q.` itself, and it diffs at every baseline. Classified a behavior change, a 2.x parity break, on all four ledgers. +Excluded (SUFFIX_ACRONYMS — ph, removed 2026-09-13, #459/#521): + +- ph is OUT of SUFFIX_ACRONYMS and must not be put back by a sweep that finds "Ph. D." parsing and assumes the acronym set is what carries it. The split spelling is merged by group's own rule (the `PH`/`D` regexes of `_pipeline/_vocab.py`, v1's fix_phd; see #phd-merge, whose "merged back by vocabulary" means that regex stage and not a lexicon lookup), which never consults acronym membership, and the single-token spellings `phd`/`Ph.D.` are the `phd` entry's. The fragment arrived in the 2019-12-11 bulk Wikipedia post-nominal import (af5bdab, #93) and was never reviewed. It did two things. One was to let the merged `Ph. D.` token pass the acronym test on its first piece, which nothing read until #459's all-caps repair did — with `ph` in the set that repair read `john smith ph. d.` as `John Smith PH. D.` on the default path, not only under force, and that is what forced the removal. The other, found by the docs review of the fix-up (2026-09-22) and not at the removal, was to read a bare trailing `Ph.` — no `D.` behind it — as a credential: `John Smith Ph.` gave family Smith, suffix Ph. on every release from 1.4.0 through 2.3.0 (wheels measured), and out of the set it falls to the positional read, middle Smith, family Ph., suffix empty; `Smith, John Ph.` moves the same way, to middle Ph. The cost is that CLASS and not one spelling, measured 2026-09-22 on HEAD and on the five wheels with `Lexicon.default().add(suffix_acronyms={"ph"})` as the comparator: the acronym test strips periods, so the undotted `John Smith Ph`/`PH`/`ph` move identically; inside a credential run `Ph.` no longer joins it, so `John Smith Ph. MD` reads family Ph., suffix MD, and `John Smith MD Ph.` reads middle `Smith MD`, family Ph., the real credential leaving the suffix field with the fragment; `John Smith Ph. Jr.` keeps only the Jr.; and the comma form `Smith, Ph.` reads title Ph. where 2.2.0 and 2.3.0 read suffix. `John Smith P.H.` does not move, the by-shape class reading it either way. Every shape in the class is a bare `ph` with no `D.` behind it, which nobody writes — the same criterion as esq's, asked of the machinery rather than a surname: the entry described the merge, not a word. Classified a behavior change, a 2.x parity break, on all five ledgers (`change(suffix-acronym-collisions) ph leaves the acronym set`), with the cases.py row `removed_ph_fragment_leaves_a_bare_trailing_ph_a_name_word` written to carry the plain shape; the ledger rule is literal to that one name, so a future row on any other shape of the class arrives unexplained at every baseline and is classified by widening that rule, not filed as a regression. No corpus name had any shape of the class, which is why the removal measured as 0 role movers at review time (2026-09-11, 1143 names), the corpus-blindness AGENTS.md warns of. Under `capitalized()` the 13 `PH. D.` shapes the repair had moved under `force=True` go back to `Ph. D.`; `john smith phd` and `ph.d.` still give `Ph.D.`. + ### suffix-acronym-collisions — the trailing-position collision class, decided (2026-09-07, #342/#454) Closes #342 (a wordlist question) and #454 (a rules.md question) together, because they are the same question asked of two different words and answering one without the other would leave the criterion half-stated. No parser code moves in either. Two existing forks carry the whole thing and both were confirmed on the pre-bundle tree with a throwaway override before any wordlist was edited: `Parser(lexicon=Lexicon.default().remove(suffix_acronyms={"rai","cha"}).add(suffix_acronyms_ambiguous={"ba"})).parse(text)`. @@ -1325,6 +1329,10 @@ R3's earlier history is under `decisions.md#R2`, which this entry does not repea - 2026-08-29 — WHY THE BOUNDARY WENT UNNOTICED UNTIL #407, which is where a future reader should look for it. For an ALL-PARTICLE part the other three tag-driven views give the same answer through `replace()` and `revise()` alike: measured over `de la`, `van der`, `do`, `de` and `van de la`, all five agree on `family_particles=''`, on a `family_base` holding the whole part, and on initials from every word. They converge because an UNTAGGED part and a MARKED all-particle part reach the same place by different routes — untagged, no word is recognized as a particle; marked, none is ACTING as one — and all three views only ask which words are particles. Case repair is the one view that asks a second question, since it must also decide whether to lowercase, so it is where the two routes first come apart. The mirror case confirms the reading: on a MIXED part the convergence is the other way round — `de la vega` and `van der berg` diverge in all three views between `replace()` and `revise()` (`replace()` reports particles `''` and base `'de la vega'` where `revise()` reports `'de la'` and `'vega'`) and AGREE on case repair, R4's all-particle clause not reaching them. So before #407 the distinction was invisible on exactly the shape the clause is about, and visible only on shapes the clause does not govern. +- 2026-09-13 #459 (landed 2026-09-22, PR #521) — DECIDED: a credential acronym the exceptions map does not carry is an initialism, so a single-case word the parse put in the SUFFIX role from `suffix_acronyms` repairs to its all-caps spelling rather than a title-cased one (the clause in `_render._cap_word`, after the exceptions-map lookup and before the Mac/Mc rule — the order is load-bearing, since `mcse` is in the acronym set and matches the Mac/Mc shape, and reads `MCSE` only because the acronym clause is asked first; rules.md#R4 states the precedence and pins it with a row). The scope is narrow on purpose. The exceptions map is consulted first, so its five entries — the spellings `str.capitalize()` gets wrong, `md` → M.D. and `phd` → Ph.D. because they are punctuated, and `ii`/`iii`/`iv` → II/III/IV because a numeral is written all-caps and they are `suffix_words` the acronym clause would never see — keep theirs and the clause never touches them. The repair is gated on the SUFFIX role, so a word that is in the acronym vocabulary but parsed as an ordinary name word (`anh van do` → `Anh Van Do`) still repairs as that name word -- the gate is the whole reason the fix is safe on surnames that share a spelling with a credential. Because it is the ROLE and not a parse tag, the gate also reaches a suffix spliced in as raw text: `parse("john smith").replace(suffix="mba").capitalized()` gives `MBA` like the parsed name does, which is the contrast R4's Accepted paragraph draws against the all-particle clause. The same PR removed `ph` from `suffix_acronyms`; see `Excluded (SUFFIX_ACRONYMS — ph ...)`, beside the esq block, for why. The wider design the issue proposed (letter masks, `md` leaving the map, the given-role half of `QC MP`) stays on the rescoped #459; this clause is the narrow part #459 already accepts, not a re-litigation of those. +Reach, population first, measured 2026-09-22 on the PR branch merged with master at 23e52dc6, over the deduped corpus glob (1338 distinct names): 178 names carry acronym vocabulary in a suffix token and 30 of those are written in a single case, and 22 move on the default `capitalized()` path, 119 under `force=True`. Recompute by exec'ing the pre-change `_cap_word` -- `git show 23e52dc6:nameparser/_render.py`, the master this branch merged, which carries #397/#461's generation gate but not this clause -- into `nameparser._render`'s namespace and swapping it in for the changed one in a single process, then diffing `str(capitalized(parse(n), None, force=...))` over the glob. The digits have moved twice and neither move is the clause's: the PR's first head (2026-09-11, `ph` still in the set, 1143 names) measured 5 default and 71 forced; the commit that dropped `ph` took the 13 `PH. D.` forced movers back to `Ph. D.` and left 111 in the population, 5 default and 58 forced; and the merge brought in the corpus rows of PR #530 (#289/#516), PR #532 (#531) and PR #534 (#533), whose single-case trailing `MA` credentials (`DOE, JOHN MA`, `jane doe nee smith ma`, `JOHN SMITH, MA`, the CJK `田中 太郎, MA` family; their mixed-case twins hold still under R5) account for 15 of the 17 further default movers; the other two are `STEVEN HARDMAN, MD, DO, DDS` and `john smith mcse`, the row this PR's own rules.md amendment adds (`McSe` under the pre-change function, which is the precedence claim above measured). The ratio is the stable claim: every single-case name with an acronym suffix the map does not carry moves, and nothing else does. +Accepted costs, deferred to the rescoped #459 rather than relitigated here: the all-caps default reaches words conventionally written mixed-case -- `bsc`/`msc` read `BSC`/`MSC` under `force=True`, and `Dr. med. univ. Margit Popp, MSc` is a corpus name that reads `MSC` under `force=True` only, its mixed case holding it back on the default path under R5 -- which the letter-mask design #459 defers is meant to recover; `ii`/`iii`/`iv` are `suffix_words` rather than acronyms and need the exceptions map precisely because the clause would not see them there; and because the ambiguous five (`ba`, `do`, `ed`, `jd`, `ma`) are in `suffix_acronyms`, the clause moves WHICH parse triggers the repair rather than preventing it -- `john smith ed` → `John Smith ED`, `john smith ba` → `BA`, and `smith, ms.` → `MS.` on the default path. That is the same #342/#454-class cost #459 already accepts, and the alternative (reading classify's `vocab:suffix` tag) was measured and costs `jd` → `Jd`, because `jd` carries `vocab:suffix-ambiguous` and not `vocab:suffix`, so the role gate is the right instrument. + - 2026-09-20 #461/#397 — THE GROUNDING MOVED, AND THEN A DEFECT THE GROUNDING EXPOSED. R4's sentence rested on R3 by name ("the carve-out R3 states for initials"), and R3's carve-out is conditional now while R4's is not, so the cross-reference is CUT and replaced by this rule's own reason: a connective that initials because it joins nothing is still not written the way a name is written. Measured 2026-09-20, plain and forced, over `juan y`, `john and jane smith`, `duke of edinburgh`, `juan de y`, `juan y garcia`, `JUAN Y GARCIA`, `josep carod i rovira`, `JOSEP CAROD I ROVIRA` and — under `add(particles={"y"})` — `Anh y Van`: all nine are byte-identical to the same call at the parent, re-measured after the repair below and still byte-identical, and `tests/test_capitalization.py` is green. WHAT THE PLAIN CALL DOES, stated over a population rather than over nine names, because "capitalized() does not move" was drafted for this bullet and is false as written. Swept 2026-09-20 over the 1558 non-empty corpus-union-cases names under eight configurations against the parent commit, `capitalized()` moves on SIX distinct names and every one of them is a name whose ROLES moved under the join — it renders different fields rather than treating a word differently. The correct statement is therefore: where only `initials()` moves, `capitalized()` is byte-identical; where the roles move, it follows them. WHAT THE FORCED CALL DID, AND THE DEFECT THAT WAS FOUND WRITING THIS RECORD. The sweep above was run for the PLAIN claim and turned up a second answer: `capitalized(force=True)` moved on ten further names whose roles, initials and plain repair all stayed put, every one a lower-case `i` — `parse("Carod i")` forced gave "Carod i" where 1.4.0 and 2.3.0 gave "Carod I", and so did "John Quincy Smith i", "Josep Carod i", "Lluis Carod i", "Josep Lluis Carod i III", "Josep Lluis Carod i V", "Carod y de Rovira i", "Rovira, Josep Carod i Jr.", "Carod i Rovira" and "Josep i Rovira". Nine of those were a DEFECT and are REPAIRED here; the remaining two are the rule. diff --git a/docs/design/rules.md b/docs/design/rules.md index f171f78f..876c6ae4 100644 --- a/docs/design/rules.md +++ b/docs/design/rules.md @@ -2173,21 +2173,35 @@ R4. Rationale: case repair is a display concern, applied only on conventions rather than by the bearer's. A spelling written in a single case is repaired even where its bearer meant it, because nothing in the text marks it as a choice; where the text does - mark one, R5 defers to it. + mark one, R5 defers to it. A credential acronym the exceptions map + does not carry is an initialism, so a single-case word the parse + put in the suffix role from the acronym vocabulary repairs to its + all-caps spelling rather than a title-cased one, and that repair + outranks the Mac/Mc convention where a word fits both (MCSE, not + McSe); a word in that vocabulary that parsed as an ordinary name + word repairs as that name word, and a suffix word that is neither + an acronym nor an exceptions-map entry -- the generational `jr`, + `sr` -- keeps its title case. "juan mcdonald" → capitalized="Juan McDonald" "Juan McDonald" → capitalized_forced="Juan McDonald" "ANH DO" → capitalized="Anh Do" "anh van do" → capitalized="Anh Van Do" "john smith phd" → capitalized="John Smith Ph.D." + "john smith mba" → capitalized="John Smith MBA" + "john smith mcse" → capitalized="John Smith MCSE" + "john smith jr" → capitalized="John Smith Jr" · boundary "John Quincy Smith i" → capitalized_forced="John Quincy Smith I" "Carod i" → capitalized_forced="Carod I" "Smith, John, and" → capitalized_forced="John Smith and" "Doe, Jane, and Jr." → capitalized_forced="Jane Doe and Jr." "juan de la vega" → capitalized="Juan de la Vega" · boundary - Accepted: the clause reaches a part the parser read. A field - spliced in as raw text after the parse carries no reading of its - own, so a family set that way to "de la" stays lowercase where - those same two words parsed from a name are repaired to "De La". + Accepted: the all-particle clause reaches a part the parser read. + A field spliced in as raw text after the parse carries no reading + of its own, so a family set that way to "de la" stays lowercase + where those same two words parsed from a name are repaired to "De + La". The acronym repair is the contrast: it asks the role and the + vocabulary and not a reading, so a suffix spliced in as "mba" is + repaired to "MBA" exactly as a parsed one is. That is the boundary between splicing text into a field and revising a field through the parser — revise() classifies the value, so the repair follows it — rather than a gap between them. diff --git a/docs/release_log.rst b/docs/release_log.rst index feb769fb..433f873a 100644 --- a/docs/release_log.rst +++ b/docs/release_log.rst @@ -32,6 +32,10 @@ Release Log - **Add AmbiguityKind.CONJUNCTION_OR_INITIAL, reported when a one-letter connective in a name written wholly in one case is read as an initial:** ``parse("jose e maria santos").ambiguities`` and ``parse("JOSE E MARIA SANTOS").ambiguities`` both name it, and ``detail`` names the letter. That is the call the behavior change above had to make. A letter outside the marked set reports nothing, its reading not being in doubt, so ``JUAN GARCIA Y LOPEZ`` is silent; so is every mixed-case name, where the writing decided it. See the ``P3`` entry of ``docs/design/decisions.md`` (#383, #479) + - **Repair a credential acronym the case-repair exceptions map does not carry to all-caps instead of title-casing it.** ``HumanName("JOHN SMITH MBA").capitalize()`` gives ``John Smith MBA`` where every release since 1.4.0 gave ``John Smith Mba``; ``john smith jd`` gives ``John Smith JD``. The repair is keyed on the word having parsed in the suffix role from the acronym vocabulary, so a word that is an ordinary name merely sharing a spelling with an acronym is untouched, and the exceptions map still wins first -- ``john smith md`` gives ``John Smith M.D.`` and ``john smith phd`` gives ``John Smith Ph.D.`` as before, and the generational ``jr`` is unaffected (``john smith jr`` gives ``John Smith Jr``). The given-name half of a mixed run is unchanged, so ``QC MP`` gives ``Qc MP`` with the ``QC`` (given role) still title-cased and only the ``MP`` (suffix role) repaired. Twenty-two names move in the differential corpora on the default ``capitalize()`` path and 119 under ``force=True``, every one a single-case name with an acronym suffix the map does not carry; no role field moves. See the ``R4`` entry of ``docs/design/decisions.md`` (#459) + + - **Remove ph from the default post-nominal acronyms.** The fragment existed only so the merged ``Ph. D.`` token could pass the acronym test on its first piece, and the repair above would have read ``john smith ph. d.`` as ``John Smith PH. D.``; the parser merges the split spelling by its own rule, so ``HumanName("John Smith Ph. D.")`` still gives suffix ``Ph. D.``, ``john smith ph. d.`` capitalizes to ``John Smith Ph. D.``, and ``phd``/``Ph.D.`` are unchanged. The cost is a bare ``ph`` with no ``D.`` behind it, dotted or not, alone or inside a credential run: ``HumanName("John Smith Ph.")`` gives middle ``Smith``, last ``Ph.``, where every release since 1.4.0 gave suffix ``Ph.``, and ``John Smith MD Ph.`` gives middle ``Smith MD``, last ``Ph.``, the ``MD`` leaving the suffix with it. A caller who needs that back adds it: ``Lexicon.default().add(suffix_acronyms={"ph"})``. See the ``Excluded (SUFFIX_ACRONYMS -- ph)`` entry of ``docs/design/decisions.md`` (#459) + * 2.3.0 - September 12, 2026 nameparser 2.3 is parsing fixes and new honorific vocabulary; diff --git a/nameparser/_render.py b/nameparser/_render.py index 1ae68c43..66783384 100644 --- a/nameparser/_render.py +++ b/nameparser/_render.py @@ -323,6 +323,21 @@ def _cap_word(word: str, role: Role, tags: frozenset[str], exception = lex.capitalization_exceptions_map.get(key) if exception is not None: return exception + # A credential acronym the exceptions map doesn't carry (mba, jd, + # qc, mp, ...) is an initialism, not a word to title-case: a one- + # case name repairs to the acronym's caps instead of 'Mba' (#459). + # The exceptions map is consulted first and holds the entries that + # spell differently -- md -> M.D. and phd -> Ph.D. (the generational + # ii/iii/iv are suffix_words, not acronyms, and ride the map because + # str.capitalize() would give 'Ii'). The all-caps default is the + # right call for an initialism; its cost is that an acronym + # conventionally written mixed-case (bsc, msc) reads all-caps here + # (BSc -> BSC under force) rather than mixed, which the letter-mask + # design deferred to #459 is meant to recover. Gated on the SUFFIX + # role so a word that is a family name only happens to be in the + # vocabulary (anh van DO) still repairs as an ordinary name word. + if role is Role.SUFFIX and normalized.replace(".", "") in lex.suffix_acronyms: + return word.upper() if _MAC.match(word): return _MAC.sub( lambda m: m.group(1).capitalize() + m.group(2).capitalize(), diff --git a/nameparser/config/suffixes.py b/nameparser/config/suffixes.py index e8945ece..2afea5a3 100644 --- a/nameparser/config/suffixes.py +++ b/nameparser/config/suffixes.py @@ -828,7 +828,6 @@ 'pfmp', 'pg', 'pgmp', - 'ph', 'pharmd', 'phc', 'phd', diff --git a/tests/test_capitalization.py b/tests/test_capitalization.py index 3fc91b8e..59253f34 100644 --- a/tests/test_capitalization.py +++ b/tests/test_capitalization.py @@ -110,6 +110,38 @@ def test_capitalize_suffix_acronym_with_dots(self) -> None: hn.capitalize() self.assertEqual(hn.suffix, 'M.D.') + # A credential acronym the exceptions map doesn't carry is an + # initialism, so a one-case suffix repairs to all-caps instead of + # title-case (issue #459). + def test_capitalize_suffix_acronym_is_all_caps(self) -> None: + for src, expect in [ + ('JOHN SMITH MBA', 'John Smith MBA'), + ('john smith jd', 'John Smith JD'), + ('JOSE LUIS CPA', 'Jose Luis CPA'), + ('john smith pmp', 'John Smith PMP'), + ]: + hn = HumanName(src) + hn.capitalize() + self.m(str(hn), expect, hn) + + # The exceptions map's five keep their special casing; the new + # all-caps path must not shadow them (#459). + def test_capitalize_exceptions_still_win_over_acronyms(self) -> None: + for src, expect in [ + ('john smith md', 'John Smith M.D.'), + ('john smith phd', 'John Smith Ph.D.'), + ]: + hn = HumanName(src) + hn.capitalize() + self.m(str(hn), expect, hn) + + # A word in the acronym vocabulary that parses as a family name + # still repairs as an ordinary name word, not an acronym (#459). + def test_capitalize_family_name_in_acronym_vocab_stays_title_case(self) -> None: + hn = HumanName('anh van do') + hn.capitalize() + self.m(str(hn), 'Anh Van Do', hn) + # Leaving already-capitalized names alone def test_no_change_to_mixed_chase(self) -> None: hn = HumanName('Shirley Maclaine') diff --git a/tests/v2/cases.py b/tests/v2/cases.py index 36b5cd13..819b2c4a 100644 --- a/tests/v2/cases.py +++ b/tests/v2/cases.py @@ -401,6 +401,25 @@ def _check_cjk_shape_purity(self) -> None: "consideration reports: the parity note above is about " "the reading, not about whether a fork was called", shape=2), + Case("removed_ph_fragment_leaves_a_bare_trailing_ph_a_name_word", + "John Smith Ph.", + {"given": "John", "middle": "Smith", "family": "Ph."}, + classification="fix(#459)", + notes="the accepted cost of 'ph' leaving SUFFIX_ACRONYMS " + "(#459/#521, 2026-09-13), pinned so the reversal is " + "visible. The fragment let the merged 'Ph. D.' token pass " + "the acronym test on its first piece, and #459's all-caps " + "repair would have read 'john smith ph. d.' as " + "'John Smith PH. D.'; the merge itself keys on the PH/D " + "regexes of _pipeline/_vocab.py and never needed the " + "entry. What the entry uniquely carried was a bare " + "trailing 'Ph.' -- no 'D.' behind it -- as a credential, " + "a spelling nobody writes; out of the set it falls to the " + "positional read, family 'Ph.', where every release since " + "1.4.0 read suffix. Same criterion as the esq rows above, " + "asked of the machinery: the entry described the merge, " + "not a word. decisions.md, Excluded (SUFFIX_ACRONYMS -- ph)", + shape=1), Case("suffix_word_esq_still_reads_as_a_suffix", "John Smith Esq", {"given": "John", "family": "Smith", "suffix": "Esq"}, notes="the other half of the row above, and what the removal " diff --git a/tests/v2/test_ledger_guards.py b/tests/v2/test_ledger_guards.py index a68cde57..c52de55d 100644 --- a/tests/v2/test_ledger_guards.py +++ b/tests/v2/test_ledger_guards.py @@ -1204,6 +1204,11 @@ def test_case_shape_ids_exist_in_the_inventory() -> None: # post-nominal that is not title vocabulary. "fix(#316) a trailing Latin title on a native-script name is a title": ("王先生, V.", "田中さん, Dr.", "田中さん II"), + # The ph boundary: the merged split spelling in both positions, and + # the single-token spellings the `phd` entry carries. + "change(suffix-acronym-collisions) ph leaves the acronym set": + ("John Smith Ph. D.", "Smith, Ph. D.", "john smith phd", + "John Smith Ph.D."), # The esq boundary is every spelling SUFFIX_WORDS still carries, # in each of the three positions the corpora write it in. "change(suffix-acronym-collisions) esq leaves the acronym set": @@ -3170,6 +3175,12 @@ def _claim(rule: dict) -> _Claim: #: both is growth into names the rule genuinely describes. _CORPUS_CLAIMS: dict[str, dict[str, _Claim]] = { "expected_since_1.4.0.toml": { + # The ph removal (#459/#521): one literal name, the cases.py + # row written to carry the bare trailing `Ph.` shape, since no + # corpus name had it -- which is why the removal measured as + # zero role movers at review time. + "change(suffix-acronym-collisions) ph leaves the acronym set": + _Claim(1, ('family', 'middle', 'suffix'), '8a2e1dbb972d', None), # #436/#437's Latin alternation, first in every ledger. # Ten corpus names, `suffix` alone: the rule moves the # SEPARATOR and no role, so a widening that took a role would @@ -3887,6 +3898,12 @@ def _claim(rule: dict) -> _Claim: _Claim(1, ('maiden', 'suffix'), 'e20491ebfe62', None), }, "expected_since_2.0.0.toml": { + # The ph removal (#459/#521): one literal name, the cases.py + # row written to carry the bare trailing `Ph.` shape, since no + # corpus name had it -- which is why the removal measured as + # zero role movers at review time. + "change(suffix-acronym-collisions) ph leaves the acronym set": + _Claim(1, ('family', 'middle', 'suffix'), '8a2e1dbb972d', None), # #436/#437's Latin alternation, first in every ledger. # Ten corpus names, `suffix` alone: the rule moves the # SEPARATOR and no role, so a widening that took a role would @@ -4402,6 +4419,12 @@ def _claim(rule: dict) -> _Claim: # the 2.0.0 mapping above, the same regex classifying the same # names. "expected_since_2.2.0.toml": { + # The ph removal (#459/#521): one literal name, the cases.py + # row written to carry the bare trailing `Ph.` shape, since no + # corpus name had it -- which is why the removal measured as + # zero role movers at review time. + "change(suffix-acronym-collisions) ph leaves the acronym set": + _Claim(1, ('family', 'middle', 'suffix'), '8a2e1dbb972d', None), # #436/#437's Latin alternation, first in every ledger. # Ten corpus names, `suffix` alone: the rule moves the # SEPARATOR and no role, so a widening that took a role would @@ -4689,6 +4712,12 @@ def _claim(rule: dict) -> _Claim: '4de0e7570bd6', ('DEFAULT',)), }, "expected_since_2.1.0.toml": { + # The ph removal (#459/#521): one literal name, the cases.py + # row written to carry the bare trailing `Ph.` shape, since no + # corpus name had it -- which is why the removal measured as + # zero role movers at review time. + "change(suffix-acronym-collisions) ph leaves the acronym set": + _Claim(1, ('family', 'middle', 'suffix'), '8a2e1dbb972d', None), # #436/#437's Latin alternation, first in every ledger. # Ten corpus names, `suffix` alone: the rule moves the # SEPARATOR and no role, so a widening that took a role would @@ -5171,6 +5200,12 @@ def _claim(rule: dict) -> _Claim: '4de0e7570bd6', ('DEFAULT',)), }, "expected_since_2.3.0.toml": { + # The ph removal (#459/#521): one literal name, the cases.py + # row written to carry the bare trailing `Ph.` shape, since no + # corpus name had it -- which is why the removal measured as + # zero role movers at review time. + "change(suffix-acronym-collisions) ph leaves the acronym set": + _Claim(1, ('family', 'middle', 'suffix'), '8a2e1dbb972d', None), # #383/#479's three rules, the first this ledger carries. The # role rule is the 2.x shape of the 1.4.0 rule of the same # name -- two corpus names, the union of two disjoint role diff --git a/tools/differential/corpus_rules.jsonl b/tools/differential/corpus_rules.jsonl index 112d9847..4f6745ad 100644 --- a/tools/differential/corpus_rules.jsonl +++ b/tools/differential/corpus_rules.jsonl @@ -326,6 +326,9 @@ "de los Santos" "ibn Awf abdul Rahman" "john e smith" +"john smith jr" +"john smith mba" +"john smith mcse" "john smith phd" "john smith x.y.z." "john van der berg ma" diff --git a/tools/differential/corpus_shapes.jsonl b/tools/differential/corpus_shapes.jsonl index b74c1eb8..5053cae5 100644 --- a/tools/differential/corpus_shapes.jsonl +++ b/tools/differential/corpus_shapes.jsonl @@ -59,6 +59,7 @@ {"name": "John Smith J.u.n.i.o.r.", "shape": 1} {"name": "John Smith Jr.", "shape": 1} {"name": "John Smith Ma", "shape": 1} +{"name": "John Smith Ph.", "shape": 1} {"name": "John Smith Q.W.E.R.T.", "shape": 1} {"name": "John Smith R.A.I.", "shape": 1} {"name": "John Smith X.Y.Z.", "shape": 1} diff --git a/tools/differential/expected_since_1.4.0.toml b/tools/differential/expected_since_1.4.0.toml index f5900948..a84fc800 100644 --- a/tools/differential/expected_since_1.4.0.toml +++ b/tools/differential/expected_since_1.4.0.toml @@ -4361,3 +4361,32 @@ issue = "fix(#274/#397) a maiden clause inside a suffix-comma tail leaves the su # name in the corpora. name_regex = "^Smith, John, PhD née Puig Mr\\. - i Soler$" fields = ["maiden", "suffix"] + +[[change]] +issue = "change(suffix-acronym-collisions) ph leaves the acronym set" +# 'John Smith Ph.': 'ph' left SUFFIX_ACRONYMS with #459's all-caps +# repair (PR #521, 2026-09-13). The fragment let the merged 'Ph. D.' +# token pass the acronym test on its first piece -- the merge itself +# keys on the PH/D regexes of _pipeline/_vocab.py and keeps working +# without it -- and its only unique coverage was a bare trailing +# 'Ph.' with no 'D.' behind it, read as a credential by every release +# since 1.4.0. Out of the set that spelling falls to the positional +# read: given 'John', middle 'Smith', family 'Ph.', suffix ''. +# +# A BEHAVIOR CHANGE rather than a fix, which is what the `change` tag +# says: a deliberate 2.x parity break at every baseline, argued in +# decisions.md's `Excluded (SUFFIX_ACRONYMS — ph)` block beside esq's +# and carried by the 2.4.0 release-log bullet. No corpus name had the +# shape until the cases.py row was written to carry it, which is why +# the removal measured as 0 role movers at review time; 'John Smith +# Ph. D.', 'Smith, Ph. D.' and 'john smith phd' do not move. +# +# Literal to the one corpus name that carries the shape. The CLASS +# the removal moves is wider -- the undotted 'John Smith Ph'/'PH', +# 'Ph.' inside a credential run ('John Smith Ph. MD', 'MD Ph.', +# 'Ph. Jr.'), and the comma form 'Smith, Ph.' from 2.2.0 on -- every +# one a bare ph with no D. behind it (the decisions block lists them, +# measured 2026-09-22). A row carrying one of those arrives +# unexplained here by design: widen this rule to it, do not file it. +name_regex = "^John Smith Ph\\.$" +fields = ["family", "middle", "suffix"] diff --git a/tools/differential/expected_since_2.0.0.toml b/tools/differential/expected_since_2.0.0.toml index 4e884a2c..c14bf7ca 100644 --- a/tools/differential/expected_since_2.0.0.toml +++ b/tools/differential/expected_since_2.0.0.toml @@ -3252,3 +3252,32 @@ issue = "fix(#397) a link inside a maiden clause stays in the birth name" name_regex = "^(?:Doe, Jane nee Puig i Soler|Jane Doe nee Puig i Soler|Smith, John, PhD née Puig Mr\\. - i Soler)$" fields = ["family", "maiden", "middle", "suffix"] orders = ["DEFAULT"] + +[[change]] +issue = "change(suffix-acronym-collisions) ph leaves the acronym set" +# 'John Smith Ph.': 'ph' left SUFFIX_ACRONYMS with #459's all-caps +# repair (PR #521, 2026-09-13). The fragment let the merged 'Ph. D.' +# token pass the acronym test on its first piece -- the merge itself +# keys on the PH/D regexes of _pipeline/_vocab.py and keeps working +# without it -- and its only unique coverage was a bare trailing +# 'Ph.' with no 'D.' behind it, read as a credential by every release +# since 1.4.0. Out of the set that spelling falls to the positional +# read: given 'John', middle 'Smith', family 'Ph.', suffix ''. +# +# A BEHAVIOR CHANGE rather than a fix, which is what the `change` tag +# says: a deliberate 2.x parity break at every baseline, argued in +# decisions.md's `Excluded (SUFFIX_ACRONYMS — ph)` block beside esq's +# and carried by the 2.4.0 release-log bullet. No corpus name had the +# shape until the cases.py row was written to carry it, which is why +# the removal measured as 0 role movers at review time; 'John Smith +# Ph. D.', 'Smith, Ph. D.' and 'john smith phd' do not move. +# +# Literal to the one corpus name that carries the shape. The CLASS +# the removal moves is wider -- the undotted 'John Smith Ph'/'PH', +# 'Ph.' inside a credential run ('John Smith Ph. MD', 'MD Ph.', +# 'Ph. Jr.'), and the comma form 'Smith, Ph.' from 2.2.0 on -- every +# one a bare ph with no D. behind it (the decisions block lists them, +# measured 2026-09-22). A row carrying one of those arrives +# unexplained here by design: widen this rule to it, do not file it. +name_regex = "^John Smith Ph\\.$" +fields = ["family", "middle", "suffix"] diff --git a/tools/differential/expected_since_2.1.0.toml b/tools/differential/expected_since_2.1.0.toml index 7bd7fbe9..a8fd61dd 100644 --- a/tools/differential/expected_since_2.1.0.toml +++ b/tools/differential/expected_since_2.1.0.toml @@ -3163,3 +3163,32 @@ issue = "fix(#397) a link inside a maiden clause stays in the birth name" name_regex = "^(?:Doe, Jane nee Puig i Soler|Jane Doe nee Puig i Soler|Smith, John, PhD née Puig Mr\\. - i Soler)$" fields = ["family", "maiden", "middle", "suffix"] orders = ["DEFAULT"] + +[[change]] +issue = "change(suffix-acronym-collisions) ph leaves the acronym set" +# 'John Smith Ph.': 'ph' left SUFFIX_ACRONYMS with #459's all-caps +# repair (PR #521, 2026-09-13). The fragment let the merged 'Ph. D.' +# token pass the acronym test on its first piece -- the merge itself +# keys on the PH/D regexes of _pipeline/_vocab.py and keeps working +# without it -- and its only unique coverage was a bare trailing +# 'Ph.' with no 'D.' behind it, read as a credential by every release +# since 1.4.0. Out of the set that spelling falls to the positional +# read: given 'John', middle 'Smith', family 'Ph.', suffix ''. +# +# A BEHAVIOR CHANGE rather than a fix, which is what the `change` tag +# says: a deliberate 2.x parity break at every baseline, argued in +# decisions.md's `Excluded (SUFFIX_ACRONYMS — ph)` block beside esq's +# and carried by the 2.4.0 release-log bullet. No corpus name had the +# shape until the cases.py row was written to carry it, which is why +# the removal measured as 0 role movers at review time; 'John Smith +# Ph. D.', 'Smith, Ph. D.' and 'john smith phd' do not move. +# +# Literal to the one corpus name that carries the shape. The CLASS +# the removal moves is wider -- the undotted 'John Smith Ph'/'PH', +# 'Ph.' inside a credential run ('John Smith Ph. MD', 'MD Ph.', +# 'Ph. Jr.'), and the comma form 'Smith, Ph.' from 2.2.0 on -- every +# one a bare ph with no D. behind it (the decisions block lists them, +# measured 2026-09-22). A row carrying one of those arrives +# unexplained here by design: widen this rule to it, do not file it. +name_regex = "^John Smith Ph\\.$" +fields = ["family", "middle", "suffix"] diff --git a/tools/differential/expected_since_2.2.0.toml b/tools/differential/expected_since_2.2.0.toml index 68047f6e..f143faa1 100644 --- a/tools/differential/expected_since_2.2.0.toml +++ b/tools/differential/expected_since_2.2.0.toml @@ -1622,3 +1622,32 @@ issue = "fix(#397) a link inside a maiden clause stays in the birth name" name_regex = "^(?:Doe, Jane nee Puig i Soler|Jane Doe nee Puig i Soler|Smith, John, PhD née Puig Mr\\. - i Soler)$" fields = ["family", "maiden", "middle", "suffix"] orders = ["DEFAULT"] + +[[change]] +issue = "change(suffix-acronym-collisions) ph leaves the acronym set" +# 'John Smith Ph.': 'ph' left SUFFIX_ACRONYMS with #459's all-caps +# repair (PR #521, 2026-09-13). The fragment let the merged 'Ph. D.' +# token pass the acronym test on its first piece -- the merge itself +# keys on the PH/D regexes of _pipeline/_vocab.py and keeps working +# without it -- and its only unique coverage was a bare trailing +# 'Ph.' with no 'D.' behind it, read as a credential by every release +# since 1.4.0. Out of the set that spelling falls to the positional +# read: given 'John', middle 'Smith', family 'Ph.', suffix ''. +# +# A BEHAVIOR CHANGE rather than a fix, which is what the `change` tag +# says: a deliberate 2.x parity break at every baseline, argued in +# decisions.md's `Excluded (SUFFIX_ACRONYMS — ph)` block beside esq's +# and carried by the 2.4.0 release-log bullet. No corpus name had the +# shape until the cases.py row was written to carry it, which is why +# the removal measured as 0 role movers at review time; 'John Smith +# Ph. D.', 'Smith, Ph. D.' and 'john smith phd' do not move. +# +# Literal to the one corpus name that carries the shape. The CLASS +# the removal moves is wider -- the undotted 'John Smith Ph'/'PH', +# 'Ph.' inside a credential run ('John Smith Ph. MD', 'MD Ph.', +# 'Ph. Jr.'), and the comma form 'Smith, Ph.' from 2.2.0 on -- every +# one a bare ph with no D. behind it (the decisions block lists them, +# measured 2026-09-22). A row carrying one of those arrives +# unexplained here by design: widen this rule to it, do not file it. +name_regex = "^John Smith Ph\\.$" +fields = ["family", "middle", "suffix"] diff --git a/tools/differential/expected_since_2.3.0.toml b/tools/differential/expected_since_2.3.0.toml index 11b0725b..7415ed28 100644 --- a/tools/differential/expected_since_2.3.0.toml +++ b/tools/differential/expected_since_2.3.0.toml @@ -939,3 +939,32 @@ issue = "fix(#397) a link inside a maiden clause stays in the birth name" name_regex = "^(?:Doe, Jane nee Puig i Soler|Jane Doe nee Puig i Soler|Smith, John, PhD née Puig Mr\\. - i Soler)$" fields = ["family", "maiden", "middle", "suffix"] orders = ["DEFAULT"] + +[[change]] +issue = "change(suffix-acronym-collisions) ph leaves the acronym set" +# 'John Smith Ph.': 'ph' left SUFFIX_ACRONYMS with #459's all-caps +# repair (PR #521, 2026-09-13). The fragment let the merged 'Ph. D.' +# token pass the acronym test on its first piece -- the merge itself +# keys on the PH/D regexes of _pipeline/_vocab.py and keeps working +# without it -- and its only unique coverage was a bare trailing +# 'Ph.' with no 'D.' behind it, read as a credential by every release +# since 1.4.0. Out of the set that spelling falls to the positional +# read: given 'John', middle 'Smith', family 'Ph.', suffix ''. +# +# A BEHAVIOR CHANGE rather than a fix, which is what the `change` tag +# says: a deliberate 2.x parity break at every baseline, argued in +# decisions.md's `Excluded (SUFFIX_ACRONYMS — ph)` block beside esq's +# and carried by the 2.4.0 release-log bullet. No corpus name had the +# shape until the cases.py row was written to carry it, which is why +# the removal measured as 0 role movers at review time; 'John Smith +# Ph. D.', 'Smith, Ph. D.' and 'john smith phd' do not move. +# +# Literal to the one corpus name that carries the shape. The CLASS +# the removal moves is wider -- the undotted 'John Smith Ph'/'PH', +# 'Ph.' inside a credential run ('John Smith Ph. MD', 'MD Ph.', +# 'Ph. Jr.'), and the comma form 'Smith, Ph.' from 2.2.0 on -- every +# one a bare ph with no D. behind it (the decisions block lists them, +# measured 2026-09-22). A row carrying one of those arrives +# unexplained here by design: widen this rule to it, do not file it. +name_regex = "^John Smith Ph\\.$" +fields = ["family", "middle", "suffix"]