Force jumbo builds to fail.
We no longer have any builders supporting the jumbo build config, and it's time to now update all of the build files to remove the jumbo templates. Before we do that, though, it's probably good to figure out how to handle existing uses of the jumbo builds. This CL asserts that |use_jumbo_build| is set to false. Once we've run with this for a while (to ensure that people have adjusted their configs), we can probably update any CI build configs to remove the use of it and the other jumbo-related GN args and then remove the actual declare_args() of the three jumbo args, turning uses of them into warnings. Bug: 994387 Change-Id: Ibe395e51c837692ff9a7b6d01141c32ddab5459b Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1874583 Commit-Queue: Dirk Pranke <dpranke@chromium.org> Reviewed-by: Bruce Dawson <brucedawson@chromium.org> Reviewed-by: Aaron Gable <agable@chromium.org> Reviewed-by: Nico Weber <thakis@chromium.org> Cr-Commit-Position: refs/heads/master@{#710481}
This commit is contained in:
Dirk Pranke
@ -6,42 +6,18 @@ import("//build/config/nacl/config.gni") # To see if jumbo should be turned off
|
||||
import("//build/toolchain/goma.gni")
|
||||
|
||||
declare_args() {
|
||||
# If true, use a jumbo build (files compiled together) to speed up
|
||||
# compilation.
|
||||
use_jumbo_build = false
|
||||
|
||||
# A list of build targets to exclude from jumbo builds, for optimal
|
||||
# round trip time when frequently changing a set of cpp files. The
|
||||
# targets can be just the short name (in which case it matches any
|
||||
# target with that name), a directory prefixed with the root
|
||||
# specifier //, or a full build target label.
|
||||
#
|
||||
# Example:
|
||||
# These would all exclude the "browser" target in a file
|
||||
# content/browser/BUILD.gn, and potentially more.
|
||||
#
|
||||
# jumbo_build_excluded = [ "browser" ]
|
||||
# jumbo_build_excluded = [ "//content/browser" ]
|
||||
# jumbo_build_excluded = [ "//content/browser:browser" ]
|
||||
jumbo_build_excluded = []
|
||||
|
||||
# How many files to group on average. Smaller numbers give more
|
||||
# parallellism, higher numbers give less total CPU usage. Higher
|
||||
# numbers also give longer single-file recompilation times.
|
||||
#
|
||||
# Recommendations:
|
||||
# Higher numbers than 100 does not reduce wall clock compile times
|
||||
# even for 4 cores or less so no reason to go higher than 100.
|
||||
# Going from 50 to 100 with a 4 core CPU saves about 3% CPU time and
|
||||
# 3% wall clock time in a tree with blink, v8 and content
|
||||
# jumbofied. At the same time it increases the compile time for the
|
||||
# largest jumbo chunks by 10-20% and reduces the chance to use all
|
||||
# available CPU cores. So set the default to 50 to balance between
|
||||
# high and low-core build performance. -1 means do the default which
|
||||
# varies depending on whether goma is enabled.
|
||||
jumbo_file_merge_limit = -1
|
||||
}
|
||||
|
||||
# TODO(crbug.com/994387): Remove all uses of the jumbo_* templates from the
|
||||
# BUILD.gn files, and then remove this whole file.
|
||||
assert(!use_jumbo_build,
|
||||
"The jumbo build is no longer supported. Please remove any uses of " +
|
||||
"'use_jumbo_build', 'jumbo_build_excluded', or " +
|
||||
"'jumbo_file_merge_limit' from your args.gn file and re-run GN.")
|
||||
|
||||
# Normal builds benefit from lots of jumbification
|
||||
jumbo_file_merge_default = 50
|
||||
|
||||
|
||||
@ -58,8 +58,7 @@ ninja clang-apply-replacements
|
||||
## Running clang-tidy
|
||||
|
||||
Running clang-tidy is (hopefully) simple.
|
||||
1. Build chrome normally.\* Note that [Jumbo builds](jumbo.md) are not
|
||||
supported.
|
||||
1. Build chrome normally.
|
||||
```
|
||||
ninja -C out/Release chrome
|
||||
```
|
||||
@ -99,9 +98,9 @@ Copy-Paste Friendly (though you'll still need to stub in the variables):
|
||||
chrome/browser
|
||||
```
|
||||
|
||||
\*It's not clear which, if any, `gn` flags outside of `use_jumbo_build` may
|
||||
cause issues for `clang-tidy`. I've had no problems building a component release
|
||||
build, both with and without goma. if you run into issues, let us know!
|
||||
\*It's not clear which, if any, `gn` flags may cause issues for
|
||||
`clang-tidy`. I've had no problems building a component release build,
|
||||
both with and without goma. if you run into issues, let us know!
|
||||
|
||||
## Questions
|
||||
|
||||
|
||||
@ -15,8 +15,6 @@ with a traditional find-and-replace regexp:
|
||||
|
||||
## Caveats
|
||||
|
||||
* Clang tools do not work with jumbo builds.
|
||||
|
||||
* Invocations of a clang tool runs on on only one build config at a time. For
|
||||
example, running the tool across a `target_os="win"` build won't update code
|
||||
that is guarded by `OS_POSIX`. Performing a global refactoring will often
|
||||
|
||||
118
docs/jumbo.md
118
docs/jumbo.md
@ -1,116 +1,12 @@
|
||||
# Jumbo / Unity builds
|
||||
|
||||
To improve compilation times it is possible to use "unity builds",
|
||||
called Jumbo builds, in Chromium. The idea is to merge many
|
||||
translation units ("source files") and compile them together. Since a
|
||||
large portion of Chromium's code is in shared header files that
|
||||
dramatically reduces the total amount of work needed.
|
||||
[Jumbo / Unity builds are no longer supported in Chromium](crbug.com/994387).
|
||||
|
||||
## Build instructions
|
||||
They were a mechanism for speeding up local builds by combining multiple
|
||||
files into a single compilation unit.
|
||||
|
||||
If jumbo isn't already enabled, you enable it in `gn` by setting
|
||||
`use_jumbo_build = true` then compile as normal.
|
||||
We are still in the process of cleaning up the build files to remove
|
||||
references to the jumbo templates.
|
||||
|
||||
## Implementation
|
||||
|
||||
Jumbo is currently implemented as a combined `gn` template and a
|
||||
python script. Eventually it may become a native `gn` feature. By
|
||||
(indirectly) using the template `internal_jumbo_target`, each target
|
||||
will split into one action to "merge" the files and one action to
|
||||
compile the merged files and any files left outside the merge.
|
||||
|
||||
Template file: `//build/config/jumbo.gni`
|
||||
Merge script: `//build/config/merge_for_jumbo.py`
|
||||
|
||||
### Merge
|
||||
|
||||
The "merge" is currently done by creating wrapper files that `#include` the
|
||||
source files.
|
||||
|
||||
## Jumbo Pros and Cons
|
||||
|
||||
### Pros
|
||||
|
||||
* Everything compiles significantly faster. When fully enabled
|
||||
everywhere this can save hours for a full build (binaries and tests)
|
||||
on a moderate computer. Linking is faster because there is less
|
||||
redundant data (debug information, inline functions) to merge.
|
||||
* Certain code bugs can be statically detected by the compiler when it
|
||||
sees more/all the relevant source code.
|
||||
|
||||
### Cons
|
||||
|
||||
* By merging many files, symbols that have internal linkage in
|
||||
different `cc` files can collide and cause compilation errors.
|
||||
* The smallest possible compilation unit grows which can add
|
||||
10-20 seconds to some single file recompilations (though link
|
||||
times often shrink).
|
||||
|
||||
### Mixed blessing
|
||||
* Slightly different compiler warnings will be active.
|
||||
|
||||
## Tuning
|
||||
|
||||
By default on average `50`, or `8` when using goma, files are merged at a
|
||||
time. The more files that are are merged, the less total CPU time is
|
||||
needed, but parallelism is reduced. This number can be changed by
|
||||
setting `jumbo_file_merge_limit`.
|
||||
|
||||
## Naming
|
||||
|
||||
The term jumbo is used to avoid the confusion resulting from talking
|
||||
about unity builds since unity is also the name of a graphical
|
||||
environment, a 3D engine, a webaudio filter and part of the QUIC
|
||||
congestion control code. Jumbo has been used as name for a unity build
|
||||
system in another browser engine.
|
||||
|
||||
## Want to make your favourite piece of code jumbo?
|
||||
|
||||
1. Add `import("//build/config/jumbo.gni")` to `BUILD.gn`.
|
||||
2. Change your target, for instance `static_library`, to
|
||||
`jumbo_static_library`. So far `source_set`, `component`,
|
||||
`static_library` are supported.
|
||||
3. Recompile and test.
|
||||
|
||||
### Example
|
||||
Change from:
|
||||
|
||||
source_set("foothing") {
|
||||
sources = [
|
||||
"foothing.cc"
|
||||
"fooutil.cc"
|
||||
"fooutil.h"
|
||||
]
|
||||
}
|
||||
to:
|
||||
|
||||
import("//build/config/jumbo.gni") # ADDED LINE
|
||||
jumbo_source_set("foothing") { # CHANGED LINE
|
||||
sources = [
|
||||
"foothing.cc"
|
||||
"fooutil.cc"
|
||||
"fooutil.h"
|
||||
]
|
||||
}
|
||||
|
||||
|
||||
If you see some compilation errors about colliding symbols, resolve
|
||||
those by renaming symbols or removing duplicate code. If it's
|
||||
impractical to change the code, add a `jumbo_excluded_sources`
|
||||
variable to your target in `BUILD.gn`:
|
||||
|
||||
`jumbo_excluded_sources = [ "problematic_file.cc" ]`
|
||||
|
||||
## More information and pictures
|
||||
There are more information and pictures in a
|
||||
[Google Document](https://docs.google.com/document/d/19jGsZxh7DX8jkAKbL1nYBa5rcByUL2EeidnYsoXfsYQ)
|
||||
|
||||
## Mailing List
|
||||
Public discussions happen on the generic blink-dev and chromium-dev
|
||||
mailing lists.
|
||||
|
||||
https://groups.google.com/a/chromium.org/group/chromium-dev/topics
|
||||
|
||||
## Bugs / feature requests
|
||||
Related bugs use the label `jumbo` in the bug database.
|
||||
See [the open bugs](http://code.google.com/p/chromium/issues/list?q=label:jumbo).
|
||||
**[TODO(crbug.com/994387)](crbug.com/994387)**: Remove this page itself
|
||||
when all references to jumbo have been removed from the build files.
|
||||
|
||||
@ -153,15 +153,6 @@ use_goma=true
|
||||
goma_dir=/path/to/goma-client
|
||||
```
|
||||
|
||||
#### Jumbo/Unity builds
|
||||
|
||||
Jumbo builds merge many translation units ("source files") and compile them
|
||||
together. Since a large portion of Chromium's code is in shared header files,
|
||||
this dramatically reduces the total amount of work needed. Check out the
|
||||
[Jumbo / Unity builds](jumbo.md) for more information.
|
||||
|
||||
Enable jumbo builds by setting the GN arg `use_jumbo_build=true`.
|
||||
|
||||
#### Disable NaCl
|
||||
|
||||
By default, the build includes support for
|
||||
|
||||
@ -140,15 +140,6 @@ in your args.gn to disable debug symbols altogether. This makes both full
|
||||
rebuilds and linking faster (at the cost of not getting symbolized backtraces
|
||||
in gdb).
|
||||
|
||||
#### Jumbo/Unity builds
|
||||
|
||||
Jumbo builds merge many translation units ("source files") and compile them
|
||||
together. Since a large portion of Chromium's code is in shared header files,
|
||||
this dramatically reduces the total amount of work needed. Check out the
|
||||
[Jumbo / Unity builds](jumbo.md) for more information.
|
||||
|
||||
Enable jumbo builds by setting the GN arg `use_jumbo_build=true`.
|
||||
|
||||
#### CCache
|
||||
|
||||
You might also want to [install ccache](ccache_mac.md) to speed up the build.
|
||||
|
||||
@ -239,7 +239,6 @@ in the editor that appears when you create your output directory
|
||||
(`gn args out/Default`) or on the gn gen command line
|
||||
(`gn gen out/Default --args="is_component_build = true is_debug = true"`).
|
||||
Some helpful settings to consider using include:
|
||||
* `use_jumbo_build = true` - [Jumbo/unity](jumbo.md) builds.
|
||||
* `is_component_build = true` - this uses more, smaller DLLs, and incremental
|
||||
linking.
|
||||
* `enable_nacl = false` - this disables Native Client which is usually not
|
||||
@ -310,7 +309,7 @@ $ autoninja -C out\Default base
|
||||
Longest build steps:
|
||||
...
|
||||
1.2 weighted s to build base.dll, base.dll.lib, base.dll.pdb (1.2 s CPU time)
|
||||
8.5 weighted s to build obj/base/base/base_jumbo_38.obj (30.1 s CPU time)
|
||||
8.5 weighted s to build obj/base/base/values.obj (30.1 s CPU time)
|
||||
Time by build-step type:
|
||||
...
|
||||
1.2 s weighted time to generate 1 PEFile (linking) files (1.2 s CPU time)
|
||||
|
||||
Reference in New Issue
Block a user