Repository navigation
Make the stack-size estimate fast on large scripts (same output) - #21
Open
tmwatchanan wants to merge 1 commit into
Open
tmwatchanan wants to merge 1 commit into
tmwatchanan wants to merge 1 commit into
Conversation
max_stacksize() walked every simple path of the caller graph from every function, with no reuse: exponential in shared callees and O(n^3) on a chain. On a 9,010-function script it made 10.7 billion calls (800-1,050 s of CPU) for a compile that takes 8 s without it. The result of the walk above a function depends only on the functions on the path that are in its own strongly connected component (only those can be met above it). When the walk enters another component, none are on the path, so that result is computed once and reused. Components come from Tarjan's algorithm; inside a recursive component the walk is path-by-path as before, so every printed number and every -R recursion line stays the same. The chain of the -R report is rebuilt by replaying the last path of the old walk. The same script now takes 0.2 s in the estimate, with identical output and .amx. This also fixes a heap overflow: a self-recursive function can be on the path twice, so the path could exceed the numfunctions+1 entries of symstack/rsymstack (free(): invalid pointer on random programs). run_tests.py: output_check tests can match stdout (stdout_pattern).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
The stack-size estimate pawncc prints after compiling (
estimated max. usage=...) can take minutes on large scripts. On a 9,000-function script it took about 15 minutes, while the rest of the compile takes about 8 s.max_stacksize()re-walks every chain of callers from every function and remembers nothing, so the work grows exponentially. This PR remembers each function's result and reuses it wherever it can't depend on the path. Inside recursion loops (found with Tarjan's SCCs), the walk is still path by path, so the output is unchanged: the same numbers, the same-Rreport, the same.amx.It also fixes a heap overflow in the old code. A self-recursive function can appear twice on a path, which overflows
symstack/rsymstack.Results
-e(estimate skipped)Review notes
All changes are in
sc1.c:stack_scc(): new; finds the recursion loops.max_stacksize_recurse(): same loop over callers in the same order, now with the memo.recursion_chain(): new; rebuilds the-Rtext.Testing:
.amx. Zero differences from the original.New tests in
tests/stack_estimate_*, with astdout_patternoption inrun_tests.py:-R+text;Benchmarks, fuzzer and full report: branch
perf/stack-estimate-bench.Which issue(s) this PR fixes:
No existing issue. It fixes very slow compiles of large scripts (the stack estimate runs for minutes) and a heap overflow in the same code.
What kind of pull this is:
Additional Documentation: