Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Doc/library/re.rst
Original file line number Diff line number Diff line change
Expand Up @@ -614,7 +614,7 @@ character ``'$'``.

Matches ``[0-9]`` if the :py:const:`~re.ASCII` flag is used.

__ https://www.unicode.org/versions/Unicode17.0.0/core-spec/chapter-4/#G124142
__ https://www.unicode.org/versions/Unicode18.0.0/core-spec/chapter-4/#G124142

For 8-bit (bytes) patterns:
Matches any decimal digit in the ASCII character set;
Expand Down
8 changes: 4 additions & 4 deletions Doc/library/stdtypes.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2157,7 +2157,7 @@ expression support in the :mod:`re` module).

The casefolding algorithm is `described in section 3.13.3 'Default Case
Folding' of the Unicode Standard
<https://www.unicode.org/versions/Unicode17.0.0/core-spec/chapter-3/#G53253>`__.
<https://www.unicode.org/versions/Unicode18.0.0/core-spec/chapter-3/#G53253>`__.

.. versionadded:: 3.3

Expand Down Expand Up @@ -2398,7 +2398,7 @@ expression support in the :mod:`re` module).
property being one of "Lm", "Lt", "Lu", "Ll", or "Lo". Note that this is different
from the `Alphabetic property defined in section 4.10 'Letters, Alphabetic, and
Ideographic' of the Unicode Standard
<https://www.unicode.org/versions/Unicode17.0.0/core-spec/chapter-4/#G91002>`__.
<https://www.unicode.org/versions/Unicode18.0.0/core-spec/chapter-4/#G91002>`__.
For example:

.. doctest::
Expand Down Expand Up @@ -2664,7 +2664,7 @@ expression support in the :mod:`re` module).

The lowercasing algorithm used is `described in section 3.13.2 'Default Case
Conversion' of the Unicode Standard
<https://www.unicode.org/versions/Unicode17.0.0/core-spec/chapter-3/#G34078>`__.
<https://www.unicode.org/versions/Unicode18.0.0/core-spec/chapter-3/#G34078>`__.


.. method:: str.lstrip(chars=None, /)
Expand Down Expand Up @@ -3176,7 +3176,7 @@ expression support in the :mod:`re` module).

The uppercasing algorithm used is `described in section 3.13.2 'Default Case
Conversion' of the Unicode Standard
<https://www.unicode.org/versions/Unicode17.0.0/core-spec/chapter-3/#G34078>`__.
<https://www.unicode.org/versions/Unicode18.0.0/core-spec/chapter-3/#G34078>`__.


.. method:: str.zfill(width, /)
Expand Down
10 changes: 5 additions & 5 deletions Doc/library/unicodedata.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@

This module provides access to the Unicode Character Database (UCD) which
defines character properties for all Unicode characters. The data contained in
this database is compiled from the `UCD version 17.0.0
<https://www.unicode.org/Public/17.0.0/ucd>`_.
this database is compiled from the `UCD version 18.0.0
<https://www.unicode.org/Public/18.0.0/ucd>`_.

The module uses the same names and symbols as defined by Unicode
Standard Annex #44, `"Unicode Character Database"
Expand Down Expand Up @@ -170,7 +170,7 @@ Standard Annex #44, `"Unicode Character Database"
.. function:: block(chr, /)

Returns the `block
<https://www.unicode.org/versions/Unicode17.0.0/core-spec/chapter-3/#G64189>`_
<https://www.unicode.org/versions/Unicode18.0.0/core-spec/chapter-3/#G64189>`_
assigned to the character *chr*. For example::

>>> unicodedata.block('S')
Expand Down Expand Up @@ -326,6 +326,6 @@ In addition, the module exposes the following constants:

.. rubric:: Footnotes

.. [#] https://www.unicode.org/Public/17.0.0/ucd/NameAliases.txt
.. [#] https://www.unicode.org/Public/18.0.0/ucd/NameAliases.txt

.. [#] https://www.unicode.org/Public/17.0.0/ucd/NamedSequences.txt
.. [#] https://www.unicode.org/Public/18.0.0/ucd/NamedSequences.txt
6 changes: 3 additions & 3 deletions Doc/reference/lexical_analysis.rst
Original file line number Diff line number Diff line change
Expand Up @@ -602,8 +602,8 @@ Unicode categories use the version of the Unicode Character Database as
included in the :mod:`unicodedata` module.

.. _UAX-31: https://www.unicode.org/reports/tr31/
.. _PropList.txt: https://www.unicode.org/Public/17.0.0/ucd/PropList.txt
.. _DerivedCoreProperties.txt: https://www.unicode.org/Public/17.0.0/ucd/DerivedCoreProperties.txt
.. _PropList.txt: https://www.unicode.org/Public/18.0.0/ucd/PropList.txt
.. _DerivedCoreProperties.txt: https://www.unicode.org/Public/18.0.0/ucd/DerivedCoreProperties.txt
.. _normalization form: https://www.unicode.org/reports/tr15/#Norm_Forms

.. seealso::
Expand Down Expand Up @@ -916,7 +916,7 @@ with the given *name*::
This sequence cannot appear in :ref:`bytes literals <bytes-literal>`.

.. versionchanged:: 3.3
Support for `name aliases <https://www.unicode.org/Public/17.0.0/ucd/NameAliases.txt>`__
Support for `name aliases <https://www.unicode.org/Public/18.0.0/ucd/NameAliases.txt>`__
has been added.

.. _string-escape-long-hex:
Expand Down
6 changes: 3 additions & 3 deletions Lib/re/_properties.py
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@
# CATEGORY_DIGIT matches Nd (but, like \d, is restricted to ASCII under the
# ASCII flag). The gc group memberships (L = Lu|Ll|Lt|Lm|Lo, N = Nd|Nl|No)
# are given by the Unicode Standard 4.5, Table 4-4 "General_Category Values"
# (https://www.unicode.org/versions/Unicode17.0.0/core-spec/chapter-4/#G124142)
# (https://www.unicode.org/versions/Unicode18.0.0/core-spec/chapter-4/#G124142)
# and listed in
# https://www.unicode.org/Public/UCD/latest/ucd/PropertyValueAliases.txt
# The compound categories Lu, N, Lm, Nl, No, Cf, Z, Zs, C and Cn are
Expand All @@ -127,7 +127,7 @@
# (private use) and the single code points Zl and Zp. Cc, Cs and Co are the
# control codes, surrogate and private-use areas, fixed by the Unicode
# Standard 23.1, 23.6 and 23.5:
# https://www.unicode.org/versions/Unicode17.0.0/core-spec/chapter-23/
# https://www.unicode.org/versions/Unicode18.0.0/core-spec/chapter-23/
# All five are listed in
# https://www.unicode.org/Public/UCD/latest/ucd/extracted/DerivedGeneralCategory.txt
_CC_RANGES = [(0x00, 0x1F), (0x7F, 0x9F)]
Expand Down Expand Up @@ -174,7 +174,7 @@ def _analytic_ranges():
# normalised.
# Noncharacter_Code_Point: U+FDD0..FDEF and the last two of every plane,
# permanently reserved (the Unicode Standard 23.7, "Noncharacters":
# https://www.unicode.org/versions/Unicode17.0.0/core-spec/chapter-23/).
# https://www.unicode.org/versions/Unicode18.0.0/core-spec/chapter-23/).
noncharacter = [(0xFDD0, 0xFDEF)]
noncharacter += [(plane | 0xFFFE, plane | 0xFFFF)
for plane in range(0, MAXUNICODE + 1, 0x10000)]
Expand Down
2 changes: 1 addition & 1 deletion Lib/stringprep.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
# when unicodedata is modified to ensure conformant behavior.
import unicodedata

assert unicodedata.unidata_version == '17.0.0'
assert unicodedata.unidata_version == '18.0.0'

from unicodedata import ucd_3_2_0 as unicodedata_320

Expand Down
36 changes: 30 additions & 6 deletions Lib/test/test_unicodedata.py
Original file line number Diff line number Diff line change
Expand Up @@ -55,9 +55,9 @@ def download_test_data_file(filename):
class UnicodeMethodsTest(unittest.TestCase):

# update this, if the database changes
expectedchecksum = ('47a99fa654ef1f50e89d2e9697b7b041fccb5a05'
expectedchecksum = ('2bb59305a65254fee9cd6ef981cc3cab7551c60e'
if quicktest else
'8b2615a9fc627676cbc0b6fac0191177df97ef5f')
'bb69894a259aee5971445eca8e617a1686a432e2')

def test_method_checksum(self):
h = hashlib.sha1()
Expand Down Expand Up @@ -345,6 +345,9 @@ def test_bidirectional(self):
# New in 17.0.0
self.assertEqual(self.db.bidirectional('\u088f'), '' if self.old else 'AL')
self.assertEqual(self.db.bidirectional('\U0001fbfa'), '' if self.old else 'ON')
# New in 18.0.0
self.assertEqual(self.db.bidirectional('\U00001b3a'), '' if self.old else 'L')
self.assertEqual(self.db.bidirectional('\U00001b3c'), '' if self.old else 'L')

self.assertRaises(TypeError, self.db.bidirectional)
self.assertRaises(TypeError, self.db.bidirectional, 'xx')
Expand Down Expand Up @@ -377,6 +380,8 @@ def test_decomposition(self):
self.assertEqual(self.db.decomposition('\U0001CCD6'), '' if self.old else '<font> 0041')
# New in 17.0.0
self.assertEqual(self.db.decomposition('\uA7F1'), '' if self.old else '<super> 0053')
# New in 18.0.0
self.assertEqual(self.db.decomposition('\u0558'), '' if self.old else '<super> 0567')

# Hangul characters
self.assertEqual(self.db.decomposition('\uAC00'), '1100 1161')
Expand Down Expand Up @@ -428,6 +433,8 @@ def test_combining(self):
self.assertEqual(self.db.combining('\u0897'), 0 if self.old else 230)
# New in 17.0.0
self.assertEqual(self.db.combining('\u1ACF'), 0 if self.old else 230)
# New in 18.0.0
self.assertEqual(self.db.combining('\U0001d127'), 0 if self.old else 220)

self.assertRaises(TypeError, self.db.combining)
self.assertRaises(TypeError, self.db.combining, 'xx')
Expand Down Expand Up @@ -696,11 +703,13 @@ def test_east_asian_width(self):
self.assertEqual(eaw('\U0001FAE9'), 'N' if self.old else 'W')
# New in 17.0.0
self.assertEqual(eaw('\U00016FF2'), 'N' if self.old else 'W')
# New in 18.0.0
self.assertEqual(eaw('\U0001F7DA'), 'N' if self.old else 'W')

def test_east_asian_width_unassigned(self):
eaw = self.db.east_asian_width
# unassigned
for char in '\u0530\u0ecf\u10c6\u20fc\uaaca\U000107bd\U000115f2':
for char in '\u0530\u0ecf\u10c6\u20fc\uaaca\U000115f2':
self.assertEqual(eaw(char), 'N')
self.assertIs(self.db.name(char, None), None)

Expand All @@ -721,9 +730,9 @@ class UnicodeFunctionsTest(unittest.TestCase, BaseUnicodeFunctionsTest):

# Update this if the database changes. Make sure to do a full rebuild
# (e.g. 'make distclean && make') to get the correct checksum.
expectedchecksum = ('00b13fa975a60b1d3f490f1fc8c126ab24990c75'
expectedchecksum = ('04a01eac45fd6ca3cfa5aafe90f278c1f7d62aa2'
if quicktest else
'ebfc9dd281c2226998fd435744dd2e9321899beb')
'f5354bfdd674d0229dbec9f8f038a86b31cf6b4b')

@requires_resource('network')
def test_all_names(self):
Expand Down Expand Up @@ -882,6 +891,9 @@ def test_grapheme_cluster_break(self):
# New in 17.0.0
self.assertEqual(gcb('\u1AEB'), 'Extend')
self.assertEqual(gcb('\U00011B67'), 'SpacingMark')
# New in 18.0.0
self.assertEqual(gcb('\U00011DF0'), 'Extend')
self.assertEqual(gcb('\U0001D25F'), 'Extend')

self.assertRaises(TypeError, gcb)
self.assertRaises(TypeError, gcb, b'x')
Expand Down Expand Up @@ -910,6 +922,8 @@ def test_indic_conjunct_break(self):
self.assertEqual(incb('\u1000'), 'Consonant')
self.assertEqual(incb('\U00011F33'), 'Consonant')
self.assertEqual(incb('\U0001E6F5'), 'Extend')
# New in 18.0.0
self.assertEqual(incb('\U00011A3A'), 'Linker')

self.assertRaises(TypeError, incb)
self.assertRaises(TypeError, incb, b'x')
Expand All @@ -930,6 +944,8 @@ def test_extended_pictographic(self):
# New in 17.0.0
self.assertIs(ext_pict('\u2388'), False)
self.assertIs(ext_pict('\U0001FA6D'), False)
# New in 18.0.0
self.assertIs(ext_pict('\U0001F6D9'), True)

self.assertRaises(TypeError, ext_pict)
self.assertRaises(TypeError, ext_pict, b'x')
Expand Down Expand Up @@ -1087,6 +1103,14 @@ def test_block(self):
# New in 17.0.0
self.assertEqual(self.db.block('\u1AEB'), 'Combining Diacritical Marks Extended')
self.assertEqual(self.db.block('\U00011B67'), 'Sharada Supplement')
# New in 18.0.0
self.assertEqual(self.db.block('\U00011DF2'), 'Bengali Supplement')
self.assertEqual(self.db.block('\U000125F2'), 'Archaic Cuneiform Numerals')
self.assertEqual(self.db.block('\U00018EF2'), 'Jurchen')
self.assertEqual(self.db.block('\U000191A2'), 'Jurchen Radicals')
self.assertEqual(self.db.block('\U0001D252'), 'Musical Symbols Supplement')
self.assertEqual(self.db.block('\U0001DB52'), 'Miscellaneous Symbols and Arrows Extended')
self.assertEqual(self.db.block('\U0003DB52'), 'Seal')
# Unassigned
self.assertEqual(self.db.block('\U00100000'), 'Supplementary Private Use Area-B')
self.assertEqual(self.db.block('\U0010FFFF'), 'Supplementary Private Use Area-B')
Expand All @@ -1104,7 +1128,7 @@ class Unicode_3_2_0_FunctionsTest(unittest.TestCase, BaseUnicodeFunctionsTest):
old = True
expectedchecksum = ('883824cb6c0ccf994e4451ebf281e2d6d479af47'
if quicktest else
'68cd01e2c680b851c1fcab012efb5635b2229c2b')
'677542db0fed40ddbd491b95622cb649a4b7fca0')


class UnicodeMiscTest(unittest.TestCase):
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Update Unicode to 18.0.0
Loading