PEP 823: None-aware access operators

[cdce8p]

Basic requirements (all PEP Types)

• Read and followed PEP 1 & PEP 12
• File created from the latest PEP template
• PEP has next available number, & set in filename (pep-NNNN.rst), PR title (PEP 123: <Title of PEP>) and PEP header
• Title clearly, accurately and concisely describes the content in 79 characters or less
• Core dev/PEP editor listed as Author or Sponsor, and formally confirmed their approval
• Author, Status (Draft), Type and Created headers filled out correctly
• PEP-Delegate, Topic, Requires and Replaces headers completed if appropriate
• Required sections included
  • Abstract (first section)
  • Copyright (last section; exact wording from template required)
• Code is well-formatted (PEP 7/PEP 8) and is in code blocks, with the right lexer names if non-Python
• PEP builds with no warnings, pre-commit checks pass and content displays as intended in the rendered HTML
• Authors/sponsor added to .github/CODEOWNERS for the PEP

Standards Track requirements

• PEP topic discussed in a suitable venue with general agreement that a PEP is appropriate
• Suggested sections included (unless not applicable)
  • Motivation
  • Specification
  • Rationale
  • Backwards Compatibility
  • Security Implications
  • How to Teach This
  • Reference Implementation
  • Rejected Ideas
  • Open Issues
  • Acknowledgements
  • Footnotes
  • Change History
• Python-Version set to valid (pre-beta) future Python version, if relevant
• Any project stated in the PEP as supporting/endorsing/benefiting from the PEP formally confirmed such
• Right before or after initial merging, PEP discussion thread created and linked to in Discussions-To and Post-History

---

📚 Documentation preview 📚: https://pep-previews--4798.org.readthedocs.build/

[hugovk]

@gvanrossum Please can you confirm you're sponsoring this PEP?

[gvanrossum]

@gvanrossum Please can you confirm you're sponsoring this PEP?

Yes.

[gvanrossum]

Here are a ton of nits. I am strongly in favor of this proposal -- I just think the main exposition is a bit unwieldy. Thanks for writing this Marc!

Note that I didn't get to the Deferred/Rejected ideas yet. Some of the things I balk over in the main exposition (like using the walrus or match/case) might find a place in the latter.

[cdce8p]

Thanks for the detailed feedback! I'd like to address your general comments regarding the motivation section here directly instead of splitting it up for each comment.

While I was preparing to write this PEP, I reread a lot of the discussions from the last few years. One of the most common arguments against these operators was that they are basically just syntactic sugar and don't add anything new. Technically speaking this is correct. We both know that these would still be great additions and make certain coding patterns a lot easier to write. However, the question I asked myself was how can I best explain why all the other patterns are only suboptimal.

I ended up with the structure you read. It starts at "beginner" level with simple truthiness checks. I've actually seen code like this in production. Sometimes lazy code is enough even if it contains problems. It continues with the explicit is not None checks, before ultimately adding the assignment expressions as well. try and match were added as these were the other often suggested builtin alternatives.

By showcasing how each of these work, I can point out the subtle issues each option has and provide a justification why ?. and ?[ ] should be added.

In the end I also wanted to avoid falling into the same, IMO trap, PEP 505 did with just showing nice examples and basically saying the new operators are just better. https://peps.python.org/pep-0505/#examples That's also why I deliberately moved the common patterns towards the end of the motivation section.

--
I admit the example isn't ideal. It's adopted from a discussion post I saw. However, I wanted to add some "real" class and variable names for these examples. Of course, a?.b.c?[d].e would work as well, but that oftentimes seems contrived and far away. PEP 505 did that and I always found that somewhat unintuitive, even though it's strait forward. https://peps.python.org/pep-0505/#the-maybe-dot-and-maybe-subscript-operators

I'm happy to change it if we can come up with a better example.

[gvanrossum]

I'd like to address your general comments regarding the motivation section here directly instead of splitting it up for each comment.

You're right to push back -- am taking a pretty extreme POV here.

About syntactic sugar: half of Python is syntactic sugar. We don't need for, we can do everything with while. We don't need or / and, we can do that with if. Etc. (Brett Cannon has a blog series about how far you can take this.) So the argument against the PEP from "it's just syntactic sugar" isn't very strong.

My main feeling about the long exposition in the Motivation section is that many of the things you show are either just bad habits (no matter how often people write them -- this PEP isn't the place to educate them), or straw-men examples that look like they are only shown to shoot them down (the match/case version). If people have brought these up in the discussion and you want to capture that, maybe you can move them to the "Rejected ideas" section with subheadings like "Do nothing -- use match/case (or try/except, or whatever other suggestion was made)".

How do you feel about that option? If you don't like that either, I'll approve your current approach -- I don't want to die on this hill. :-)

[cdce8p]

How do you feel about that option? If you don't like that either, I'll approve your current approach -- I don't want to die on this hill. :-)

I'll think about this some more. You might not have gotten around to it yet but I do also have added a Common objections section with Just use ... subsections towards the end which similarly captures some of the ideas. Maybe I can come up with a good way to combine them.

[hugovk]

Just a note that this PEP (and #4799) is targeting Python 3.15 and the feature freeze is a couple of months away.

And these PRs will need to be merged, discussed/revised, submitted to the SC, approved, and implemented before then if they're to make 3.15, so there's not a huge amount of time.

[cdce8p]

Just a note that this PEP (and #4799) is targeting Python 3.15 and the feature freeze is a couple of months away.

Thanks for the reminder @hugovk. I was thinking about how to rewrite the motivation section based on the feedback but then life happened. I do plan to finish things up for the next round of reviews within the next few days, hopefully this weekend.

I'm aware that the time until the feature freeze is quite short. Thankfully the technical questions are AFAIK settled now after the intense prior discussions the last few months. Though of course there is always the possibility that something else comes up after these two have been posted.

[read-the-docs-community]

Documentation build overview

📚 pep-previews | 🛠️ Build #34666369 | 📁 Comparing 4386065 against latest (f866e77)

  🔍 Preview build  

55 files changed · + 1 added · ± 54 modified

+ Added

• pep-0823/index.html

± Modified

• index.html
• numerical/index.html
• pep-0000/index.html
• pep-0326/index.html
• pep-0416/index.html
• pep-0501/index.html
• pep-0532/index.html
• pep-0534/index.html
• pep-0585/index.html
• pep-0604/index.html
• and 44 more...

[cdce8p]

Sorry for the long delay here. I pushed it back, after it became clear that I won't had the time to finish it for the 3.15 release. Back at it again now.

Took some time to rewrite most of the motivation section including a different / simpler example. The whole section got a lot shorter as well. The detours regarding try-except and match are already covered in the later "Common objections" sections.

Also went through the remaining review comments and addressed most of them.

Will look at the feedback and update the draft for PEP 824 next.

[gtkacz]

Hey @cdce8p I'd love to help you out with getting this awesome PEP through the finish line, how could I help?

[cdce8p]

I'd love to help you out with getting this awesome PEP through the finish line, how could I help?

Just reading through it and leaving comments if you notice anything odd or unclear would certainly help. An outside perspective is always appreciated!

[gvanrossum]

Let's merge this one. (You need to create the discuss thread, add it to the PR, and let me know, then I'll merge the PR.)

[cdce8p]

Let's merge this one. (You need to create the discuss thread, add it to the PR, and let me know, then I'll merge the PR.)

If that's ok with you, I'd prefer to merge it first and then create a discuss thread. That way I can link to the PEP directly and don't need to reference the PR preview. AFAICT that's also the suggested order in PEP 1. I'll create a new PR to add the link afterwards.

In any case, I'd like to wait with merging until PEP 824 is also ready so that I can post the two threads at the same time.

--
Keeping track of the remaining items here and in #4799:

• Address review feedback for PEP 824
• (Optional) Add custom lexers for syntax highlighting. Would be great to have it that before posting the discuss threads as it improves the readability of the examples.
  • Infra: Add custom lexers for PEP 823 + 824 #5125
  • Update PRs to use new lexers
• Update Created in PEP headers
• Merge PEP 823
• Resolve conflict and merge PEP 824
• Fix cross-references in Deferred Ideas (needs to happen after both PRs are merged)
• Create discuss threads
• Update Discussion-To links and Post-History
• Update demo: https://pep823-and-pep824-demo.pages.dev/
  • Add discuss thread links
  • Rebase implementation

[warsaw]

If that's ok with you, I'd prefer to merge it first and then create a discuss thread. That way I can link to the PEP directly and don't need to reference the PR preview.

It's a glitch in the process but that's usually how it's done. It just means you'll need a quick follow up PR to update with the actual Discussions-To link.

[gvanrossum]

Okay!

[cdce8p]

While rebasing my implementation, I came across an under-specified edge case regarding assignments, in particular a?.b.c. Technically only the last attribute .c is in a Save context whereas a?.b defaults to Load. With the changes in python/cpython#157450 it would have been possible to parse a?.b.c. However, I do think it makes sense to raise a SyntaxError here as well, similar to just a?.b.

The rule I came up with, and coincidentally what I my implementation did prior to the rebase, is that the complete primary expression containing a None-aware access operator needs to be in a Load context. I pushed a small commit to mention that. 441563a

[gtkacz]

@cdce8p what about f(a?.b).c = x? Here the ?. is inside a call argument, so it's a separate expression and should clearly be fine since the target itself is f(...).c, but I think the phrase "complete primary expression containing a None-aware access operator" could be misread as forbidding this, no? If you agree then we maybe should document it

[cdce8p]

Good point @gtkacz! Pushed another commit to mention that: dd5e08a. I find that thinking about these as a self-contained expression inside larger ones makes the most sense to me.

Would you mind reading the whole Assignments section again and see if it works for you?

[gvanrossum]

In the end maybe we must specify this using the grammar.

I think we must also accept that (a?.b).c and a?.b.c are different things. (Hmm, IIRC we discussed this before, but I don't recall what we decided.

[cdce8p]

In the end maybe we must specify this using the grammar.

Up until last week, there were two different set of primary rules primary and t_primary. Those are now merged which would make it possible to do a?.b.c = 1. The validation if an expression type is allowed as assignment target happens via a grammar action.

I think we must also accept that (a?.b).c and a?.b.c are different things. (Hmm, IIRC we discussed this before, but I don't recall what we decided.

There is a whole section just for that case 😅 -> Parenthesized expressions - groupings

[gtkacz]

@cdce8p happy to help! I re read it and I think there's a couple things that still could use some clarification:

The first sentence says the "complete" primary expression has to be in a Load context, but in f(a?.b)[1] = 1 the complete primary expression is f(a?.b)[1], and I think that one is in Store? I assume what's meant is the complete None-aware primary expression, i.e. the chain the guard covers, up to whatever ends it, I think aomething like this could work better (paraphrasing)?

a None-aware access extends its guard to the end of the enclosing primary chain, and that whole chain must be in a Load context; while parentheses, argument lists and subscripts end the chain"

Another thing, I think (a?.b).c = 1 isn't covered anywhere and I can't work out from the text which way it's intended to go to, the groupings section says the group ends the chain, so a?.b is in Load and .c is just a normal attribute store, which by the rule above makes it legal, with an AttributeError at runtime if a is None, but then the last paragraph says "non-trivial" parenthesized expressions, which sounds like the group needs something else in it before that applies (also "non-trivial" isn't defined anywhere, and it sounds like it's the only thing that decides between a SyntaxError and working code, so I think properly defining it might be nice)

Keep up the awesome work! 🥳

[cdce8p]

It's always the finer details 😄 You're right "complete primary expression" was misleading, tbh the complete Assignments subsection was more complicated than it needed to be. The whole non-trivial groups thing was a leftover from the time when groups didn't short-circuit, so I completely removed it to avoid any confusion. (a?.b).c = 2 is valid as you said. It might raise an exception at runtime but that's a different story.

To clear up the handling of function arguments and subscripts, I added a small note to the Short-circuiting section. c7fff9c