Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-128629: Add Py_PACK_VERSION and Py_PACK_FULL_VERSION #128630

Merged
merged 2 commits into from
Jan 9, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
102 changes: 76 additions & 26 deletions Doc/c-api/apiabiversion.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,13 @@
API and ABI Versioning
***********************


Build-time version constants
----------------------------

CPython exposes its version number in the following macros.
Note that these correspond to the version code is **built** with,
not necessarily the version used at **run time**.
Note that these correspond to the version code is **built** with.
See :c:var:`Py_Version` for the version used at **run time**.

See :ref:`stable` for a discussion of API and ABI stability across versions.

Expand Down Expand Up @@ -37,37 +41,83 @@ See :ref:`stable` for a discussion of API and ABI stability across versions.
.. c:macro:: PY_VERSION_HEX
The Python version number encoded in a single integer.
See :c:func:`Py_PACK_FULL_VERSION` for the encoding details.

The underlying version information can be found by treating it as a 32 bit
number in the following manner:

+-------+-------------------------+-------------------------+--------------------------+
| Bytes | Bits (big endian order) | Meaning | Value for ``3.4.1a2`` |
+=======+=========================+=========================+==========================+
| 1 | 1-8 | ``PY_MAJOR_VERSION`` | ``0x03`` |
+-------+-------------------------+-------------------------+--------------------------+
| 2 | 9-16 | ``PY_MINOR_VERSION`` | ``0x04`` |
+-------+-------------------------+-------------------------+--------------------------+
| 3 | 17-24 | ``PY_MICRO_VERSION`` | ``0x01`` |
+-------+-------------------------+-------------------------+--------------------------+
| 4 | 25-28 | ``PY_RELEASE_LEVEL`` | ``0xA`` |
+ +-------------------------+-------------------------+--------------------------+
| | 29-32 | ``PY_RELEASE_SERIAL`` | ``0x2`` |
+-------+-------------------------+-------------------------+--------------------------+
Use this for numeric comparisons, for example,
``#if PY_VERSION_HEX >= ...``.

Thus ``3.4.1a2`` is hexversion ``0x030401a2`` and ``3.10.0`` is
hexversion ``0x030a00f0``.

Use this for numeric comparisons, e.g. ``#if PY_VERSION_HEX >= ...``.

This version is also available via the symbol :c:var:`Py_Version`.
Run-time version
----------------

.. c:var:: const unsigned long Py_Version
The Python runtime version number encoded in a single constant integer, with
the same format as the :c:macro:`PY_VERSION_HEX` macro.
The Python runtime version number encoded in a single constant integer.
See :c:func:`Py_PACK_FULL_VERSION` for the encoding details.
This contains the Python version used at run time.

Use this for numeric comparisons, for example, ``if (Py_Version >= ...)``.

.. versionadded:: 3.11

All the given macros are defined in :source:`Include/patchlevel.h`.

Bit-packing macros
------------------

.. c:function:: uint32_t Py_PACK_FULL_VERSION(int major, int minor, int micro, int release_level, int release_serial)
Return the given version, encoded as a single 32-bit integer with
the following structure:
+------------------+-------+----------------+-----------+--------------------------+
| | No. | | | Example values |
| | of | | +-------------+------------+
| Argument | bits | Bit mask | Bit shift | ``3.4.1a2`` | ``3.10.0`` |
+==================+=======+================+===========+=============+============+
| *major* | 8 | ``0xFF000000`` | 24 | ``0x03`` | ``0x03`` |
+------------------+-------+----------------+-----------+-------------+------------+
| *minor* | 8 | ``0x00FF0000`` | 16 | ``0x04`` | ``0x0A`` |
+------------------+-------+----------------+-----------+-------------+------------+
| *micro* | 8 | ``0x0000FF00`` | 8 | ``0x01`` | ``0x00`` |
+------------------+-------+----------------+-----------+-------------+------------+
| *release_level* | 4 | ``0x000000F0`` | 4 | ``0xA`` | ``0xF`` |
+------------------+-------+----------------+-----------+-------------+------------+
| *release_serial* | 4 | ``0x0000000F`` | 0 | ``0x2`` | ``0x0`` |
+------------------+-------+----------------+-----------+-------------+------------+
For example:
+-------------+------------------------------------+-----------------+
| Version | ``Py_PACK_FULL_VERSION`` arguments | Encoded version |
+=============+====================================+=================+
| ``3.4.1a2`` | ``(3, 4, 1, 0xA, 2)`` | ``0x030401a2`` |
+-------------+------------------------------------+-----------------+
| ``3.10.0`` | ``(3, 10, 0, 0xF, 0)`` | ``0x030a00f0`` |
+-------------+------------------------------------+-----------------+
Out-of range bits in the arguments are ignored.
That is, the macro can be defined as:
.. code-block:: c
#ifndef Py_PACK_FULL_VERSION
#define Py_PACK_FULL_VERSION(X, Y, Z, LEVEL, SERIAL) ( \
(((X) & 0xff) << 24) | \
(((Y) & 0xff) << 16) | \
(((Z) & 0xff) << 8) | \
(((LEVEL) & 0xf) << 4) | \
(((SERIAL) & 0xf) << 0))
#endif
``Py_PACK_FULL_VERSION`` is primarily a macro, intended for use in
``#if`` directives, but it is also available as an exported function.

.. versionadded:: 3.14

.. c:function:: uint32_t Py_PACK_VERSION(int major, int minor)
Equivalent to ``Py_PACK_FULL_VERSION(major, minor, 0, 0, 0)``.
The result does not correspond to any Python release, but is useful
in numeric comparisons.
.. versionadded:: 3.14
2 changes: 2 additions & 0 deletions Doc/data/stable_abi.dat

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 4 additions & 0 deletions Doc/whatsnew/3.14.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1243,6 +1243,10 @@ New features
file.
(Contributed by Victor Stinner in :gh:`127350`.)

* Add macros :c:func:`Py_PACK_VERSION` and :c:func:`Py_PACK_FULL_VERSION` for
bit-packing Python version numbers.
(Contributed by Petr Viktorin in :gh:`128629`.)


Porting to Python 3.14
----------------------
Expand Down
26 changes: 20 additions & 6 deletions Include/patchlevel.h
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@

#ifndef _Py_PATCHLEVEL_H
#define _Py_PATCHLEVEL_H
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just Py_PATCHLEVEL_H, as done in other header files?

Copy link
Member Author

@encukou encukou Jan 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because I don't want to add this define to public API. It's a private detail, users don't need this; we should be able to, for example, merge this code into another header and remove patchlevel.h.

/* Python version identification scheme.

When the major or minor version changes, the VERSION variable in
Expand Down Expand Up @@ -26,10 +27,23 @@
#define PY_VERSION "3.14.0a3+"
/*--end constants--*/


#define _Py_PACK_FULL_VERSION(X, Y, Z, LEVEL, SERIAL) ( \
(((X) & 0xff) << 24) | \
(((Y) & 0xff) << 16) | \
(((Z) & 0xff) << 8) | \
(((LEVEL) & 0xf) << 4) | \
(((SERIAL) & 0xf) << 0))

/* Version as a single 4-byte hex number, e.g. 0x010502B2 == 1.5.2b2.
Use this for numeric comparisons, e.g. #if PY_VERSION_HEX >= ... */
#define PY_VERSION_HEX ((PY_MAJOR_VERSION << 24) | \
(PY_MINOR_VERSION << 16) | \
(PY_MICRO_VERSION << 8) | \
(PY_RELEASE_LEVEL << 4) | \
(PY_RELEASE_SERIAL << 0))
#define PY_VERSION_HEX _Py_PACK_FULL_VERSION( \
PY_MAJOR_VERSION, \
PY_MINOR_VERSION, \
PY_MICRO_VERSION, \
PY_RELEASE_LEVEL, \
PY_RELEASE_SERIAL)

// Public Py_PACK_VERSION is declared in pymacro.h; it needs <inttypes.h>.

#endif //_Py_PATCHLEVEL_H
9 changes: 9 additions & 0 deletions Include/pymacro.h
Original file line number Diff line number Diff line change
Expand Up @@ -190,4 +190,13 @@
// "comparison of unsigned expression in '< 0' is always false".
#define _Py_IS_TYPE_SIGNED(type) ((type)(-1) <= 0)

#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x030E0000 // 3.14
// Version helpers. These are primarily macros, but have exported equivalents.
PyAPI_FUNC(uint32_t) Py_PACK_FULL_VERSION(int x, int y, int z, int level, int serial);
PyAPI_FUNC(uint32_t) Py_PACK_VERSION(int x, int y);
#define Py_PACK_FULL_VERSION _Py_PACK_FULL_VERSION
#define Py_PACK_VERSION(X, Y) Py_PACK_FULL_VERSION(X, Y, 0, 0, 0)
#endif // Py_LIMITED_API < 3.14


#endif /* Py_PYMACRO_H */
43 changes: 43 additions & 0 deletions Lib/test/test_capi/test_misc.py
Original file line number Diff line number Diff line change
Expand Up @@ -3335,6 +3335,49 @@ def run(self):
self.assertEqual(len(set(py_thread_ids)), len(py_thread_ids),
py_thread_ids)

class TestVersions(unittest.TestCase):
full_cases = (
(3, 4, 1, 0xA, 2, 0x030401a2),
(3, 10, 0, 0xF, 0, 0x030a00f0),
(0x103, 0x10B, 0xFF00, -1, 0xF0, 0x030b00f0), # test masking
)
xy_cases = (
(3, 4, 0x03040000),
(3, 10, 0x030a0000),
(0x103, 0x10B, 0x030b0000), # test masking
)

def test_pack_full_version(self):
for *args, expected in self.full_cases:
with self.subTest(hexversion=hex(expected)):
result = _testlimitedcapi.pack_full_version(*args)
self.assertEqual(result, expected)

def test_pack_version(self):
for *args, expected in self.xy_cases:
with self.subTest(hexversion=hex(expected)):
result = _testlimitedcapi.pack_version(*args)
self.assertEqual(result, expected)

def test_pack_full_version_ctypes(self):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the purpose of testing the API through ctypes? It should be the same as testing _testlimitedcapi, no?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It tests the exported function.
(Unlike most similar API, these are macros even in limited API. This tests the case of not using the C headers at all.)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The _testcapi module tests Py_INCREF() macro and function this way:

#define TEST_REFCOUNT() \
    do { \
        PyObject *obj = PyList_New(0); \
        if (obj == NULL) { \
            return NULL; \
        } \
        assert(Py_REFCNT(obj) == 1); \
        \
        /* test Py_NewRef() */ \
        PyObject *ref = Py_NewRef(obj); \
        assert(ref == obj); \
        assert(Py_REFCNT(obj) == 2); \
        Py_DECREF(ref); \
        \
        /* test Py_XNewRef() */ \
        PyObject *xref = Py_XNewRef(obj); \
        assert(xref == obj); \
        assert(Py_REFCNT(obj) == 2); \
        Py_DECREF(xref); \
        \
        assert(Py_XNewRef(NULL) == NULL); \
        \
        Py_DECREF(obj); \
        Py_RETURN_NONE; \
    } while (0)


// Test Py_NewRef() and Py_XNewRef() macros
static PyObject*
test_refcount_macros(PyObject *self, PyObject *Py_UNUSED(ignored))
{
    TEST_REFCOUNT();
}

#undef Py_NewRef
#undef Py_XNewRef

// Test Py_NewRef() and Py_XNewRef() functions, after undefining macros.
static PyObject*
test_refcount_funcs(PyObject *self, PyObject *Py_UNUSED(ignored))
{
    TEST_REFCOUNT();
}

Maybe you can do something similar to avoid ctypes.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The C tests do something similar, with #undef Py_PACK_FULL_VERSION.
(But if I had misspelled that #undef, the test would quietly be ineffective.)

Also, I don't particularly want to avoid ctypes :)

ctypes = import_helper.import_module('ctypes')
ctypes_func = ctypes.pythonapi.Py_PACK_FULL_VERSION
ctypes_func.restype = ctypes.c_uint32
ctypes_func.argtypes = [ctypes.c_int] * 5
for *args, expected in self.full_cases:
with self.subTest(hexversion=hex(expected)):
result = ctypes_func(*args)
self.assertEqual(result, expected)

def test_pack_version_ctypes(self):
ctypes = import_helper.import_module('ctypes')
ctypes_func = ctypes.pythonapi.Py_PACK_VERSION
ctypes_func.restype = ctypes.c_uint32
ctypes_func.argtypes = [ctypes.c_int] * 2
for *args, expected in self.xy_cases:
with self.subTest(hexversion=hex(expected)):
result = ctypes_func(*args)
self.assertEqual(result, expected)

if __name__ == "__main__":
unittest.main()
2 changes: 2 additions & 0 deletions Lib/test/test_stable_abi_ctypes.py

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Add macros :c:func:`Py_PACK_VERSION` and :c:func:`Py_PACK_FULL_VERSION` for
bit-packing Python version numbers.
4 changes: 4 additions & 0 deletions Misc/stable_abi.toml
Original file line number Diff line number Diff line change
Expand Up @@ -2540,3 +2540,7 @@
added = '3.14'
[function.PyType_Freeze]
added = '3.14'
[function.Py_PACK_FULL_VERSION]
added = '3.14'
[function.Py_PACK_VERSION]
added = '3.14'
2 changes: 1 addition & 1 deletion Modules/Setup.stdlib.in
Original file line number Diff line number Diff line change
Expand Up @@ -163,7 +163,7 @@
@MODULE__TESTBUFFER_TRUE@_testbuffer _testbuffer.c
@MODULE__TESTINTERNALCAPI_TRUE@_testinternalcapi _testinternalcapi.c _testinternalcapi/test_lock.c _testinternalcapi/pytime.c _testinternalcapi/set.c _testinternalcapi/test_critical_sections.c
@MODULE__TESTCAPI_TRUE@_testcapi _testcapimodule.c _testcapi/vectorcall.c _testcapi/heaptype.c _testcapi/abstract.c _testcapi/unicode.c _testcapi/dict.c _testcapi/set.c _testcapi/list.c _testcapi/tuple.c _testcapi/getargs.c _testcapi/datetime.c _testcapi/docstring.c _testcapi/mem.c _testcapi/watchers.c _testcapi/long.c _testcapi/float.c _testcapi/complex.c _testcapi/numbers.c _testcapi/structmember.c _testcapi/exceptions.c _testcapi/code.c _testcapi/buffer.c _testcapi/pyatomic.c _testcapi/run.c _testcapi/file.c _testcapi/codec.c _testcapi/immortal.c _testcapi/gc.c _testcapi/hash.c _testcapi/time.c _testcapi/bytes.c _testcapi/object.c _testcapi/monitoring.c _testcapi/config.c
@MODULE__TESTLIMITEDCAPI_TRUE@_testlimitedcapi _testlimitedcapi.c _testlimitedcapi/abstract.c _testlimitedcapi/bytearray.c _testlimitedcapi/bytes.c _testlimitedcapi/codec.c _testlimitedcapi/complex.c _testlimitedcapi/dict.c _testlimitedcapi/eval.c _testlimitedcapi/float.c _testlimitedcapi/heaptype_relative.c _testlimitedcapi/list.c _testlimitedcapi/long.c _testlimitedcapi/object.c _testlimitedcapi/pyos.c _testlimitedcapi/set.c _testlimitedcapi/sys.c _testlimitedcapi/tuple.c _testlimitedcapi/unicode.c _testlimitedcapi/vectorcall_limited.c
@MODULE__TESTLIMITEDCAPI_TRUE@_testlimitedcapi _testlimitedcapi.c _testlimitedcapi/abstract.c _testlimitedcapi/bytearray.c _testlimitedcapi/bytes.c _testlimitedcapi/codec.c _testlimitedcapi/complex.c _testlimitedcapi/dict.c _testlimitedcapi/eval.c _testlimitedcapi/float.c _testlimitedcapi/heaptype_relative.c _testlimitedcapi/list.c _testlimitedcapi/long.c _testlimitedcapi/object.c _testlimitedcapi/pyos.c _testlimitedcapi/set.c _testlimitedcapi/sys.c _testlimitedcapi/tuple.c _testlimitedcapi/unicode.c _testlimitedcapi/vectorcall_limited.c _testlimitedcapi/version.c
@MODULE__TESTCLINIC_TRUE@_testclinic _testclinic.c
@MODULE__TESTCLINIC_LIMITED_TRUE@_testclinic_limited _testclinic_limited.c

Expand Down
3 changes: 3 additions & 0 deletions Modules/_testlimitedcapi.c
Original file line number Diff line number Diff line change
Expand Up @@ -83,5 +83,8 @@ PyInit__testlimitedcapi(void)
if (_PyTestLimitedCAPI_Init_VectorcallLimited(mod) < 0) {
return NULL;
}
if (_PyTestLimitedCAPI_Init_Version(mod) < 0) {
return NULL;
}
return mod;
}
93 changes: 93 additions & 0 deletions Modules/_testlimitedcapi/clinic/version.c.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Modules/_testlimitedcapi/parts.h
Original file line number Diff line number Diff line change
Expand Up @@ -40,5 +40,6 @@ int _PyTestLimitedCAPI_Init_Sys(PyObject *module);
int _PyTestLimitedCAPI_Init_Tuple(PyObject *module);
int _PyTestLimitedCAPI_Init_Unicode(PyObject *module);
int _PyTestLimitedCAPI_Init_VectorcallLimited(PyObject *module);
int _PyTestLimitedCAPI_Init_Version(PyObject *module);

#endif // Py_TESTLIMITEDCAPI_PARTS_H
Loading
Loading