Skip to content

Make the stack-size estimate fast on large scripts (same output) - #21

Open
tmwatchanan wants to merge 1 commit into
openmultiplayer:masterfrom
tmwatchanan:perf/stack-estimate-memo
Open

tmwatchanan wants to merge 1 commit into
openmultiplayer:masterfrom
tmwatchanan:perf/stack-estimate-memo

Conversation

@tmwatchanan

@tmwatchanan tmwatchanan commented Nov 1, 2026 •

Copy link
Copy Markdown

What this PR does / why we need it:

The stack-size estimate pawncc prints after compiling (estimated max. usage=...) can take minutes on large scripts. On a 9,000-function script it took about 15 minutes, while the rest of the compile takes about 8 s.

max_stacksize() re-walks every chain of callers from every function and remembers nothing, so the work grows exponentially. This PR remembers each function's result and reuses it wherever it can't depend on the path. Inside recursion loops (found with Tarjan's SCCs), the walk is still path by path, so the output is unchanged: the same numbers, the same -R report, the same .amx.

It also fixes a heap overflow in the old code. A self-recursive function can appear twice on a path, which overflows symstack/rsymstack.

Results

Program Before After With -e (estimate skipped)
Real 9,000-function script 813-1,063 s 7.8-9.2 s 7.4-8.8 s
40 layers × 4 functions (4⁴⁰ paths) over 600 s 0.06 s 0.03 s
Chain of 5,000 functions 23.9 s 0.53 s 0.36 s

Review notes

All changes are in sc1.c:

  • stack_scc(): new; finds the recursion loops.
  • max_stacksize_recurse(): same loop over callers in the same order, now with the memo.
  • recursion_chain(): new; rebuilds the -R text.

Testing:

  • Fuzzing: 8,400 compiles of random programs (mostly recursive) under 4 flag sets, comparing stdout, stderr and .amx. Zero differences from the original.
  • Real script: identical output on the 9,000-function script.
  • Test suite: 95 pass. The 3 failures also fail on master.
  • Windows: the MSVC build is clean.

New tests in tests/stack_estimate_*, with a stdout_pattern option in run_tests.py:

  • the exact -R+ text;
  • the overflow, which crashes master;
  • a 4⁴⁰-path program, which finishes only with the memo.

Benchmarks, fuzzer and full report: branch perf/stack-estimate-bench.

Which issue(s) this PR fixes:

No existing issue. It fixes very slow compiles of large scripts (the stack estimate runs for minutes) and a heap overflow in the same code.

What kind of pull this is:

  • A Bug Fix
  • A New Feature
  • Some repository meta (documentation, etc)
  • Other

Additional Documentation:

max_stacksize() walked every simple path of the caller graph from every
function, with no reuse: exponential in shared callees and O(n^3) on a
chain. On a 9,010-function script it made 10.7 billion calls
(800-1,050 s of CPU) for a compile that takes 8 s without it.

The result of the walk above a function depends only on the functions
on the path that are in its own strongly connected component (only those
can be met above it). When the walk enters another component, none are
on the path, so that result is computed once and reused. Components come
from Tarjan's algorithm; inside a recursive component the walk is
path-by-path as before, so every printed number and every -R recursion
line stays the same. The chain of the -R report is rebuilt by replaying
the last path of the old walk. The same script now takes 0.2 s in the
estimate, with identical output and .amx.

This also fixes a heap overflow: a self-recursive function can be on the
path twice, so the path could exceed the numfunctions+1 entries of
symstack/rsymstack (free(): invalid pointer on random programs).

run_tests.py: output_check tests can match stdout (stdout_pattern).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant