PEP 824: None-coalescing operators

[cdce8p]

Basic requirements (all PEP Types)

• Read and followed PEP 1 & PEP 12
• File created from the latest PEP template
• PEP has next available number, & set in filename (pep-NNNN.rst), PR title (PEP 123: <Title of PEP>) and PEP header
• Title clearly, accurately and concisely describes the content in 79 characters or less
• Core dev/PEP editor listed as Author or Sponsor, and formally confirmed their approval
• Author, Status (Draft), Type and Created headers filled out correctly
• PEP-Delegate, Topic, Requires and Replaces headers completed if appropriate
• Required sections included
  • Abstract (first section)
  • Copyright (last section; exact wording from template required)
• Code is well-formatted (PEP 7/PEP 8) and is in code blocks, with the right lexer names if non-Python
• PEP builds with no warnings, pre-commit checks pass and content displays as intended in the rendered HTML
• Authors/sponsor added to .github/CODEOWNERS for the PEP

Standards Track requirements

• PEP topic discussed in a suitable venue with general agreement that a PEP is appropriate
• Suggested sections included (unless not applicable)
  • Motivation
  • Specification
  • Rationale
  • Backwards Compatibility
  • Security Implications
  • How to Teach This
  • Reference Implementation
  • Rejected Ideas
  • Open Issues
  • Acknowledgements
  • Footnotes
  • Change History
• Python-Version set to valid (pre-beta) future Python version, if relevant
• Any project stated in the PEP as supporting/endorsing/benefiting from the PEP formally confirmed such
• Right before or after initial merging, PEP discussion thread created and linked to in Discussions-To and Post-History

---

📚 Documentation preview 📚: https://pep-previews--4799.org.readthedocs.build/

[nschneid]

a few typos

[hugovk]

@gvanrossum And please can you confirm sponsorship of this as well as #4798?

[gvanrossum]

@gvanrossum And please can you confirm sponsorship of this as well as #4798?

Yes of course.

[gvanrossum]

Again, I love that these get a serious treatment and I hope we can get the PEP to make it into 3.15. Again I have some editorial suggestions (some of which are generic and could apply to 823 as well) and some grammar nits and typos.

[Hnasar]

Thanks for writing this up in such a clear way. Exciting to see this move forward!

[gvanrossum]

I do think that "coalescing" is a rather off-putting term, and we should come up with a better name for all proposed operations (?., ??, ??=).

[read-the-docs-community]

Documentation build overview

📚 pep-previews | 🛠️ Build #34590616 | 📁 Comparing 4cf9f87 against latest (c7fb679)

  🔍 Preview build  

57 files changed · + 1 added · ± 55 modified · - 1 deleted

+ Added

• pep-0824/index.html

± Modified

• index.html
• numerical/index.html
• pep-0000/index.html
• pep-0326/index.html
• pep-0416/index.html
• pep-0501/index.html
• pep-0532/index.html
• pep-0534/index.html
• pep-0585/index.html
• pep-0604/index.html
• and 45 more...

- Deleted

• pep-0835/index.html

[cdce8p]

Again, sorry for the long delay here.

Took some time to rewrite most of the motivation section. I dropped the whole argument that ?? is "safer" than or. It's a red herring and as @Hnasar pointed out, not always correct.

Also extended the "How to Teach This" section to point out

• That None isn't always the best option, especially if appropriate default values already exists.
• That ??= is different from binary assignment operators like += in that it can skip the right hand side entirely in some cases.
• Added a subsection on reading ?? and ??= out loud to address the question raised by @vpoulailleau.

Other changes

• Simplified the AST for ?? slightly. Instead of a separate CoalesceOp node, I suggest to add Coalesce to the boolean operators and reuse the BoolOp node instead. The implementation was already quite similar between these two.
• Added None- to the title and updated all naming to use None-coalescing throughout.
• Added small sections to "Common objections" regarding PEP 661 – Sentinel Values and PEP 671 – Syntax for late-bound function argument defaults.

[gtkacz]

Hey @cdce8p! I noticed two small things in the current PEP 824 draft that I would like to inquire about:

First, the assignment expression alternative in the "Motivation" section looks like it has its evaluation order backwards:

age = (val := user.get_age()) if val is not None else "unknown"

Because the condition of a conditional expression is evaluated first, this reads val before the assignment and raises NameError. See:

In [15]: age = (val := user.get_age()) if val is not None else "unknown"
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Cell In[15], line 1
----> 1 age = (val := user.get_age()) if val is not None else "unknown"

NameError: name 'val' is not defined

In [16]: age = val if (val := user.get_age()) is not None else "unknown"

In [17]: age
Out[17]: 20

I think the intended version is:

age = val if (val := user.get_age()) is not None else "unknown"

Second, do you think we could have the specification explicitly describe how ??= evaluates attribute and subscript targets? e.g.:

get_container()[get_key()] ??= make_default()

To me what should happen is get_container() and get_key() get evaluated exactly once, and make_default() only if the retrieved value is None. So an equivalent of this, written in current expanded code:

_container = get_container()
_key = get_key()
_current = _container[_key]

if _current is None:
    _container[_key] = make_default()

The reference implementation seems to agree with this, but I think documenting it would be useful, since the current if a is None: a = b expansion only explains simple names. It would also be worth having explicit tests for side-effectful attribute bases and subscript expressions to guarantee this behaviour

Thanks for your time and looking forward to this PEP helping my code less verbose ;)

[cdce8p]

Thanks for taking the time to read the draft and provide feedback @gtkacz! If you like a challenge, I've another open PR for the none-aware access operators, feel free to read #4798.

First, the assignment expression alternative in the "Motivation" section looks like it has its evaluation order backwards:
[...]

I think the intended version is:

age = val if (val := user.get_age()) is not None else "unknown"

Yes. This must have slipped through at some point.

Second, do you think we could have the specification explicitly describe how ??= evaluates attribute and subscript targets? e.g.:

get_container()[get_key()] ??= make_default()

Your intuition is correct here. Subexpressions on the left hand side are cached. This is actually similar to augmented assignments. My reference implementation for ??= reuses most of it. While reading your comment, I noticed that I myself wasn't quite sure about it. So that's certainly something which could be pointed out more explicitly in the PEP. I've added a small subsection with two more complex examples in my last commit.

[gtkacz]

@cdce8p more than happy to help! Your other PEP 823 will also be an incredible QOL improvement, and I'd love to help with it, but I'm not sure what would you'd want me to do? I'll move the convo there ;)

[gvanrossum]

Lots of nits, a few biggies (I don't like the "coalesce" name). Great PEP!

[gvanrossum]

Why the # ... ? If it's to fend off a merge conflict with the PR for PEP 823, I think there's a better way to do that (just fix the second PR after the first one has landed).

[cdce8p]

This was added in 68d6ded. I believe # ... is used in the CODEOWNERS file to show a missing PEP number. It's used a few times throughout the file.

Tbh I don't think it really matters. Resolving the inevitable merge conflict will get rid of it.

[gvanrossum]

Maybe change this to the day the PR lands?

[gvanrossum]

Another round. While at a C++ conference I finally found the time to look at these. :-)

[gvanrossum]

"You shouldn't be using None as default in the first place. Use an empty tuple instead."

The tuple doesn't work so well if the value is annotated as : list[int], for example.
You can circumvent the argument by using another mutable container type, e.g. dict or set.

And was that really the argument that got 505? That'd be quite silly.

PEP 671 is deader than 505, because Chris Angelico had an episode on the thread.

My secret idea is that after this PEP is accepted, we can propose using ??= to indicate deferred argument defaults. If you squint a little it's the perfect operator for that situation. It would mean that an explicitly passed None also coalesces to the default, but that's behavior I like. And if you don't want that you can do it using the hard way using a custom sentinel value; but my way covers most common use cases.