Repository navigation
test_sampling_with_collapsed_export fails when the profiler records an unreadable frame #160334
Description
Bug report
Bug description:
test_sampling_with_collapsed_export failed in the ASan job of my PR gh-160296 (the change there is in a multiprocessing test and is unrelated):
FAIL: test_sampling_with_collapsed_export (test.test_profiling.test_sampling_profiler.test_integration.TestSampleProfilerIntegration.test_sampling_with_collapsed_export)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/runner/work/cpython/cpython/Lib/test/test_profiling/test_sampling_profiler/test_integration.py", line 512, in test_sampling_with_collapsed_export
self.assertIn(":", part)
~~~~~~~~~~~~~^^^^^^^^^^^
AssertionError: ':' not found in '<unreadable frame>'
https://github.com/python/cpython/actions/runs/39832455069/job/120012213367
Since gh-154194, _remote_debugging returns a degraded frame (file "~", no location, function "<unreadable frame>") when it cannot read a code object, instead of failing the sample. CollapsedStackCollector turns the None location into line 0, and export() writes frames with file "~" and line 0 as just the function name, so the collapsed line contains <unreadable frame> with no colon. The test expects every part to look like file:func:line, so it fails whenever a sample hits an unreadable frame. I have only seen it on the ASan build so far.
gh-154195 updated test_binary_format.py for the new frame, but not this test.
Reproducer, without a target process:
import os
import tempfile
from profiling.sampling.stack_collector import CollapsedStackCollector
collector = CollapsedStackCollector(1000)
# The first (innermost) frame is what _remote_debugging returns
# when it cannot read a code object (gh-154194).
frames = [
("~", None, "<unreadable frame>", None),
("/tmp/script.py", (10, 10, 0, 1), "main", None),
]
collector.process_frames(frames, thread_id=1)
fd, path = tempfile.mkstemp(suffix=".txt")
os.close(fd)
collector.export(path)
with open(path) as f:
line = f.read().strip()
os.unlink(path)
print(line)
stack_trace = line.rsplit(" ", 1)[0]
for part in stack_trace.split(";"):
print(repr(part), ":" in part)Output:
Collapsed stack output written to /tmp/tmp....txt
tid:1;script.py:main:10;<unreadable frame> 1
'tid:1' True
'script.py:main:10' True
'<unreadable frame>' False
<GC> frames (on by default in sample()) take the same "~" branch in export(), so they would fail this assertion too. The test could accept these special <...> frames instead of requiring a colon. I can open a PR if that approach is fine.
The same test and the gh-154194 change are also in 3.15.
CPython versions tested on:
CPython main branch
Operating systems tested on:
Linux, macOS
Activity
lysnikolaou commentedon Oct 23, 2026
cc @pablogsal @ivonastojanovic
pythongh-160864: Retry CI after known sampling flake
pythongh-160334: Add NEWS entry
Merge branch 'main' into pythongh-160334-collapsed-unreadable-frame
vstinner commentedon Nov 9, 2026
I failed to reproduce the issue locally.
./configure --with-pydebug CC=clang LD=clang --with-address-sanitizer./python -m test test_profiling.test_sampling_profiler -m test_sampling_with_collapsed_export -j20 -F -W(my laptop has 12 CPUs)I ran the command for 2 minutes (127 runs).
devdavidejesus commentedon Nov 9, 2026
Thanks for trying. I don't expect the full test to fail reliably:
_remote_debuggingonly returns this frame when reading a frame's code object fails with a non-fatal error (parse_code_object()inModules/_remote_debugging/code_objects.c), so it depends on such a read failing during sampling. I have only seen it once, in the ASan job linked above.The failing check itself is deterministic: the snippet in the description feeds that frame to
CollapsedStackCollectordirectly, without a target process, and the resulting line fails the":" in partassertion.vstinner commentedon Nov 9, 2026
Only once means that the test is very unlikely to fail, so it's hard to reproduce. It doesn't matter, I enabled auto-merge on the PR (the fix LGTM) ;-)
devdavidejesus commentedon Nov 9, 2026
Thanks! Fair point, once is not much, but a red ASan job on an unrelated PR is apparently enough to make me paranoid about flaky tests these days 😅 And thanks @raman118 for the fix.
gh-160334: Accept special frames in collapsed profiling test (#161515)
[3.15] gh-160334: Accept special frames in collapsed profiling test (G…