This change replaces a usage of os.path.relpath with os.path.abspath
since finding a relative path between different drives on Windows is
not specified.
On Windows, os.path.relpath can throw a ValueError if the current
directory is not on the same drive as the parameter directory.
(see: https://docs.python.org/3/library/os.path.html#os.path.relpath).
This is normally fine since gn_helpers is usually run from the same
drive as the output directory. However, gn_helper_unittests calls
CreateBuildCommand with a temporary directory that is created on the C:
drive, which causes it to throw a ValueError if the current directory
is on a different drive (which is often, if the checkout itself is on a
different drive).
Change-Id: I70d77a6a9596195aef5b981ff56f15b3958dcccc
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5667023
Commit-Queue: Andrew Grieve <agrieve@chromium.org>
Reviewed-by: Andrew Grieve <agrieve@chromium.org>
Commit-Queue: Michael Tang <tangm@microsoft.com>
Cr-Commit-Position: refs/heads/main@{#1321137}
The downstream iOS bots import a gn file from $root_build_dir/args.gn
that contains nested scopes. Support to parse the file has been added
to //build/gn_helpers.py with https://crrev.com/c/2513217.
The next step on the bot is pretty printing the data parsed, which
fails as nested scopes are converted to nested dictionaries. This
CL adds support to pretty print those.
Bug: 1144711
Change-Id: I3130d493c6385f2016d91245e8a5f12e710495cd
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2517679
Commit-Queue: Sylvain Defresne <sdefresne@chromium.org>
Auto-Submit: Sylvain Defresne <sdefresne@chromium.org>
Reviewed-by: Andrew Grieve <agrieve@chromium.org>
Cr-Commit-Position: refs/heads/master@{#823627}
The downstream iOS bots import a gn file from $root_build_dir/args.gn
that would not be correctly parsed by //build/gn_helpers.py as:
1. define some values of type scope (`{ a = "value"}`),
2. it uses comments in many places (before all variables that are
defined, above individual items in lists, ...).
To ensure that the file $root_build_dir/args.gn could be parsed, fix
//build/gn_helpers.py to support the scope values and to consider
comments as valid in any location where whitespace would have been.
Bug: 1144711
Change-Id: I0c48167f1bc818dc9f6253287c2c967d13756523
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2513217
Commit-Queue: Sylvain Defresne <sdefresne@chromium.org>
Commit-Queue: Andrew Grieve <agrieve@chromium.org>
Reviewed-by: Andrew Grieve <agrieve@chromium.org>
Auto-Submit: Sylvain Defresne <sdefresne@chromium.org>
Cr-Commit-Position: refs/heads/master@{#823230}
This reverts commit 67fc091c1c.
Reason for reland: Writing both build_vars.json & build_vars.txt
Original change's description:
> Revert "build_vars.txt: Changed to build_vars.json"
>
> This reverts commit 6e002e1f5d.
>
> Reason for revert: Official build failure crbug.com/1114609
>
> Original change's description:
> > build_vars.txt: Changed to build_vars.json
> >
> > And moved helper function into gn_helpers.py
> >
> > It was only ever in .properties format so that it
> > could be easily parsed by bash, but we no longer need
> > that.
> >
> > Also fixes default_android_sdk_version to be a string.
> > -next builds need it to be a string.
> >
> > Bug: None
> > Change-Id: I99541f18510de93dae4c0d68b734638564f74ee2
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2340673
> > Commit-Queue: Andrew Grieve <agrieve@chromium.org>
> > Reviewed-by: Mohamed Heikal <mheikal@chromium.org>
> > Reviewed-by: Dirk Pranke <dpranke@google.com>
> > Cr-Commit-Position: refs/heads/master@{#796020}
>
> TBR=dpranke@google.com,agrieve@chromium.org,mheikal@chromium.org
>
> # Not skipping CQ checks because original CL landed > 1 day ago.
>
> Bug: None
> Change-Id: Iad8597f6af00c534245fd1a40d1809d952758f47
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2344626
> Reviewed-by: Ben Mason <benmason@chromium.org>
> Commit-Queue: Ben Mason <benmason@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#796387}
Bug: 1114750
Change-Id: I1019351a294e54701c093911f67a1fb6b7bd34d0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2346965
Reviewed-by: Ben Mason <benmason@chromium.org>
Commit-Queue: Andrew Grieve <agrieve@chromium.org>
Cr-Commit-Position: refs/heads/master@{#796456}
And moved helper function into gn_helpers.py
It was only ever in .properties format so that it
could be easily parsed by bash, but we no longer need
that.
Also fixes default_android_sdk_version to be a string.
-next builds need it to be a string.
Bug: None
Change-Id: I99541f18510de93dae4c0d68b734638564f74ee2
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2340673
Commit-Queue: Andrew Grieve <agrieve@chromium.org>
Reviewed-by: Mohamed Heikal <mheikal@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@google.com>
Cr-Commit-Position: refs/heads/master@{#796020}
ToGNString() in gn_helpers.py prints (nested) lists as a one-liners.
This can create every long lines, which are difficult to diff across
versions. This CL reimpements ToGNString(), and adds a "pretty" option
to print list each leaf elements on a separate line, with indent.
Details:
* Keep pretty print off by default, and preserve existing output.
* Reason: ToGNString() outputs may be committed into code base. We
want each caller to make its own decisions regarding pretty print.
* Allow new line to appear in string (GN expresses this as "$0x0A").
* Fix Unicode representation (after UTF-8 encoding need "$0xXX").
* Add stricter checks for GN identifiers for {'identifier': 'value'}.
* gn_helpers_unittest.py:
* Add many more tests, for outputs without and with pretty print.
* Make script compatible with Python 2 and Python 3.
Bug: 1074446
Change-Id: I7a5e0174bdebf8bfb03cdcf42aed7c26ae56f120
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2231505
Commit-Queue: Samuel Huang <huangs@chromium.org>
Reviewed-by: Andrew Grieve <agrieve@chromium.org>
Cr-Commit-Position: refs/heads/master@{#775776}
The following Python functions are directly involved in extracting .info
files from .aar:
* gn_helpers.py: ToGNString().
* aar.py: _IsManifestEmpty().
* aar.py: _CreateInfo().
We wish to port these files to Google3 to enable .info file extraction
(while keeping the Chromium versions around for verification). However,
these functions produce many Python lint warnings in Google3. This CL
fixes these warnings in Chromium first, and improve comments.
Change-Id: I2a826f70ac1d15f2f358195327fd0dd3f12db5fc
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2213206
Reviewed-by: Andrew Grieve <agrieve@chromium.org>
Commit-Queue: Samuel Huang <huangs@chromium.org>
Cr-Commit-Position: refs/heads/master@{#771582}
The libassistant build calls gn_helpers to read the out dir's args.gn
file (while it's being invoked by ninja, mind you!) and passes its
contents through gn_helpers, applies a filtering, then creates a sub
build dir (within the original build dir) using its modified set of GN
args.
This is problematic since crbug.com/937821 is trying to transition
CrOS bots to using import() lines in their args file, which breaks
libassistant's build.
This fixes that by adding import support to gn_helpers.
Bug: 937821
Change-Id: I790bc27368611f31b63a0135c0e5919f35b21e6e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2107709
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Commit-Position: refs/heads/master@{#752503}
The changes are to allow build files to be generated using Python 3 on Windows.
The scripts still work with Python 2.
There are no intended behaviour changes.
Bug: 941669
Change-Id: I52c912c624660c7a46c05f8f36871744abd7b8d4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1521487
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Raul Tambre <raul@tambre.ee>
Cr-Commit-Position: refs/heads/master@{#641870}
I'd like to add comments to the GN args of some bots. This will let us
do that without mb or simplechrome SDK crashing.
Bug: 901471
Change-Id: I7f49434e02591639a02ef9cfd0f026680f6b9de1
Reviewed-on: https://chromium-review.googlesource.com/c/1318188
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Commit-Position: refs/heads/master@{#605544}
This CL switches MB from passing the gn arguments on the
command line via --args to actually writing them to an
args.gn file. This makes some of the escaping slightly cleaner
and also will enable us to embed import() lines in a follow-on
CL, so that we can use //build/args templates.
R=brettw@chromium.org
BUG=602528
Review URL: https://codereview.chromium.org/1880903002
Cr-Commit-Position: refs/heads/master@{#387205}
For some reason, cros_chrome_sdk_unittests passes values as a unicode
object, instead of a str.
BUG=561142
Review URL: https://codereview.chromium.org/1851723002
Cr-Commit-Position: refs/heads/master@{#384839}
The Android build code doesn't properly unescape the GN lists, use the official Python GN parser instead of ast for the zip script.
Document what should be done for build_utils, but don't fix it due to cast shell build errors as explained in the comment I added.
Update mojo zip script to use the gn helpers.
BUG=573132, 571022
Review URL: https://codereview.chromium.org/1618243004
Cr-Commit-Position: refs/heads/master@{#372461}
Some scripts take serialized GN values including strings and lists. These are serialized and escaped according to GN rules. In particular, the string escaping is not possible to duplicate with Python's "ast" module. This simple parser will convert values properly.
Add unit tests for new code, as well as existing code in gn_helpers.
Fix omissions in existing ToGNString code.
BUG=573132,571022
Review URL: https://codereview.chromium.org/1553993002
Cr-Commit-Position: refs/heads/master@{#371000}
Adds support for a new value type, a dictionary. To script it looks the same as a scope, but the existing scope didn't have the right memory management semantics for this.
Adds a new input conversion mode to interpret the result of a script as a GN block (basically an eval) and get the result in a dictionary. This allows returning named sets of things from a script. Updates the accessor and defined stuff to work with this new type.
Adds some features to the input file manager so that when we do dynamic reading from a script we keep the text of the read around which means we can give proper errors that refer to the script output, and don't have to do the weird recursive set origin thing to clear out the pointers that became invalid when the InputFile went out of scope.
This allows us to give better error messages when parsing input. The error handling in the input conversion code is changed accordingly.
Checks for unused variables both in code that instantiates a template, and in the template code itself.
Adds a shared python script for outputting values from Python to GN.
Adds a python script to read .gypi files and return the value as a GN scope.
BUG=
R=dpranke@chromium.org, scottmg@chromium.org
Review URL: https://codereview.chromium.org/223783005
git-svn-id: svn://svn.chromium.org/chrome/trunk/src@262045 0039d316-1c4b-4281-b951-d872f2087c98
