incompatible behaviour between 3.12.0rc1 and previous versions in ctypes #107940
Description
Bug report
Testing out 3.12.0rc1 on NumPy, we came across numpy/numpy#24399. I have distilled this down to a reproducer that fails on 3.12.0rc1, but passes on previous versions.
Checklist
- I am confident this is a bug in CPython, not a bug in a third-party project
- I have searched the CPython issue tracker, and am confident this bug has not been reported before
A clear and concise description of the bug
from ctypes import Structure, pointer, c_int16, cast, POINTER
class Struct(Structure):
_fields_ = [('a', c_int16)]
Struct3 = 3 * Struct
c_array = (2 * Struct3)(
Struct3(Struct(a=1), Struct(a=2), Struct(a=3)),
Struct3(Struct(a=4), Struct(a=5), Struct(a=6))
)
assert c_array[0][0].a == 1
p_obj = cast(pointer(c_array), POINTER(pointer(c_array)._type_))
obj = p_obj.contents
# Works on 3.12, obj is 2d
assert obj[0][0].a == 1
p_obj = cast(pointer(c_array[0]), POINTER(pointer(c_array)._type_))
obj = p_obj.contents
# Fails on 3.12.0rc1, but works on 3.11 and earlier: obj is not 2d
assert obj[0][0].a == 1Your environment
- CPython versions tested on: 3.12rc1
- Operating system and architecture: ubuntu 22.04
Edit: 3.12rc1 -> 3.12.0rc1
Activity
[-]incompatible behaviour between 3.12rc1 and previous versions in ctypes[/-][+]incompatible behaviour between 3.12.0rc1 and previous versions in ctypes[/+]mattip commentedon Aug 14, 2023
I see #107131 touched
Pointer_get_contentsto solve #46376. Perhaps that PR did not takecastinto account when checking the equivalency of pointers?Only reuse the pointer object when it matches the `_type_` of the con…
Only reuse the pointer object when it matches the `_type_` of the con…
[3.11] Revert "gh-46376: Return existing pointer when possible in cty…
vstinner commentedon Aug 30, 2023
In the 3.12 branch, I confirm that the commit 54aaaad introduced the regression: issue #46376 of PR #107487 (backport of PR #107131).
vstinner commentedon Aug 30, 2023
It seems like the PCP project is affected by this regression, see the Fedora issue: https://bugzilla.redhat.com/show_bug.cgi?id=2231751
vstinner commentedon Aug 30, 2023
I proposed a revert until the regression is understood and fixed: #108688
Only reuse the pointer object when it matches the `_type_` of the con…
ambv commentedon Sep 4, 2023
Revert for
mainmerged. Revert for3.12up: #108864ambv commentedon Sep 4, 2023
3.12 revert landed as well.
ambv commentedon Sep 4, 2023
3.11 was reverted before 3.11.5 in GH-108412.
ambv commentedon Sep 4, 2023
We'll close this and continue search for a good solution in the original issue.
vstinner commentedon Sep 4, 2023
Thanks. IMO it's a reasonable practical solution to this complicated problem.
If someone comes up with a new fix, please ping me, I will try to review it! But I'm not available to investigate the issue and attempt writing a better ctypes fix.