chromium/src
0
Fork 0
Code Activity
Files
00217a17f90588f6fe05061d3d435ac29b8c94b5
src/build/docs/debugging_slow_builds.md
Bruce Dawson d76e68103d Update "Debugging Slow Builds" docs 
This updates the debugging slow builds docs to give more details about
what NINJA_SUMMARIZE_BUILD=1 does, and how to interpret the weighted
time numbers.

It also adds a mention of tools/buildstate.py, and updates the
instructions for finding build information on the bots.

Bug: None
Change-Id: I3eb6900012a577569c0e123e176221bf8889b768
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4766017
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Auto-Submit: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Andrew Grieve <agrieve@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1181532}
2023-08-09 16:48:03 +00:00

2.4 KiB
Raw Blame History

Debugging Slow Builds

Did you know that Ninja writes a log to disk after each build?

To see what kinds of files took the longest for your previous build:

cd out/Default
# Lives in depot_tools:
post_build_ninja_summary.py

Because the build is highly parallelized the elapsed time values are usually not meaningful so the weighted time numbers are calculated to approximate the impact of build steps on wall-clock time.

You can also set NINJA_SUMMARIZE_BUILD=1 to have this command run after each autoninja invocation. Setting this environment variable also runs ninja with -d stats which causes it to print out internal information such as StartEdge times, which measures the times to create processes, and it modifies the NINJA_STATUS environment variable to add information such as how many processes are running at any given time - both are useful for detecting slow process creation. You can get this last benefit on its own by setting NINJA_STATUS=[%r processes, %f/%t @ %o/s : %es ] (trailing space is intentional).

To generate a Chrome trace of your most recent build:

git clone https://github.com/nico/ninjatracing
ninjatracing/ninjatracing out/Default/.ninja_log > trace.json
# Then open in https://ui.perfetto.dev/

If your build is stuck on a long-running build step you can see what it is by running tools/buildstate.py.

Slow Bot Builds

Our bots run ninjatracing and post_build_ninja_summary.py as well.

Find the trace at: postprocess for reclient > gsutil upload ninja_log > ninja_log:

  • ".ninja_log in table format (full)" is for post_build_ninja_summary.py.
  • "trace viewer (sort_by_end)" is for ninjatracing.

Advanced(ish) Tips

  • Use gn gen --tracelog trace.json to create a trace for gn gen.
  • Many Android templates make use of md5_check.py to optimize incremental builds.
    • Set PRINT_BUILD_EXPLANATIONS=1 to have these commands log which inputs changed.
  • If you suspect files are being rebuilt unnecessarily during incremental builds:
    • Use ninja -n -d explain to figure out why ninja thinks a target is dirty.
    • Ensure actions are taking advantage of ninja's restat=1 feature by not updating timestamps on outputs when their contents do not change.