chromium/src
0
Fork 0
Code Activity

Update docs to require Python 3.8.

Browse Source 
We had not updated the docs in a while so there were references
to the Python3 migration and Python3.6 still present in some
places.

This CL updates the docs to refer to the current Python version
(3.8) and removes documentation for the Python3 migration, which
is no longer necessary.

Change-Id: Id59d10d90dbd8ceff0ec8203be5e4dfbdc58c4b3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4824013
Reviewed-by: Andrew Grieve <agrieve@chromium.org>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Commit-Queue: Dirk Pranke <dpranke@google.com>
Reviewed-by: Peter Wen <wnwen@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1190394}
This commit is contained in:
Dirk Pranke
2023-08-30 23:19:36 +00:00
committed by Chromium LUCI CQ
parent 6dd802e769
commit 8766b622c7
3 changed files with 8 additions and 143 deletions

7
docs/linux/build_instructions.md

@ -15,9 +15,10 @@ Are you a Google employee? See
* A 64-bit Intel machine with at least 8GB of RAM. More than 16GB is highly
recommended.
* At least 100GB of free disk space.
* You must have Git and Python v3.6+ installed already (and `python3` must point
to a Python v3.6+ binary).
* You must have Git and Python v3.8+ installed already (and `python3` must point
to a Python v3.8+ binary). Depot_tools bundles an appropriate version
of Python in `$depot_tools/python-bin`, if you don't have an appropriate
version already on your system.
Most development is done on Ubuntu (Chromium's build infrastructure currently
runs 22.04, Jammy Jellyfish). There are some instructions for other distros

133
docs/python3_migration.md

@ -1,133 +0,0 @@
# Migrating Chromium to Python3
This page describes the current status and how to migrate code.
[crbug.com/941669](https://crbug.com/941669) tracks the overall migration.
*It is now safe to write new code using Python3 as long as it isn't called
from other Python2 files. See the
[Python style guide](../styleguide/python/python.md) for the latest on this.*
## Status
As of the time of writing (2021-07-19), we're in the following state:
* depot_tools is fully Python3-compatible.
* [gclient hooks](#gclient-hooks) are being migrated to use Python3
([crbug.com/1208028](https://crbug.com/1208028)).
* GN is fully Python3-compatible, meaning that all the scripts invoked
through exec_script() are using Python3.
* The [build](#gn_ninja-actions) (scripts invoked by Ninja) uses Python3.
* We are updating the various test harnesses and frameworks to use
Python3, but most still use Python2. It is possible to use
Python3 for tests if you're ready to do so
([crbug.com/1275016](https://crbug.com/1275016)).
* [PRESUBMIT checks](#presubmit-checks) are being migrated to use Python3
([crbug.com/1207012](https://crbug.com/1207012)).
* Python3-compatible pylint checks are available
([crbug.com/1207012](https://crbug.com/1207012)).
## Migrating Python code from 2 to 3
This document will not attempt to replicate the general information in
the [Python.org Porting HOWTO](https://docs.python.org/3/howto/pyporting.html)
or the many, many other guides on the Internet.
However, here are a few notes that may be helpful in a Chromium context:
* Most of our Python code is pretty straightforward and moves easily
from Python2 to Python3, so don't stress out about this!
* When migrating code, please make the Right Changes, rather than the
minimum needed to get things to work. For example, make sure your code
is clear about whether something should be a byte string or a unicode
string, rather than trying to handle both "to be compatible".
(If you really do need the code to handle both, though, that can be okay.)
* Do not assume you can use [vpython] regardless of context! vpython has
performance issues in some situations and so we don't want to use it yet for
things invoked by gclient, PRESUBMITs, or Ninja. However, all tests are run
under vpython and so you can assume it there.
* Some people find the `2to3` tool to be useful to partially or
completely automate the migration of existing files, and the
`six` module to shim compatibility across the two. The `six` module
is available in vpython and in `//third_party/six/src/six.py`.
* shebang lines mostly don't matter. A "shebang line" is the line at the
beginning of many unix scripts, like `#!/usr/bin/env python`. They are
not that important for us because most of our python invocations come
from tools like Ninja or Swarming and invoke python directly. So, while
you should keep them accurate where they are useful, we can't rely
on them to tell which code has been migrated and which hasn't.
* The major gotchas for us tend to have to do with processing output
from a subprocess (e.g., `subprocess.check_output()`). By default
output is returned as a binary string, so get in the habit of calling
`.check_output().decode('utf-8')` instead. This is compatible across
2 and 3. 'utf-8' is the default in Python3 (ASCII was the default in
Python2), but being explicitly is probably a good idea until we have
migrated everything.
* Be aware that `filter` and `map` return generators in Python3, which
are one-shot objects. If you reference them inside another loop, e.g.,
foo = [ ... ]
bar = filter(some_function, [ ...])
for x in foo:
for y in bar:
do_something(x, y)
this won't work right, because on the second and subsequent iterations,
bar will be an empty list.
Best practice is to use a list comprehension instead of `map` or `filter`,
but you can also explicitly cast the results of map or filter to a list
if the list comprehension is too awkward.
* Some modules (like `urllib2`) were renamed and/or moved around in Python 3.
A Google search will usually quickly tell you the new location.
* Watch out for places where you reference `basestring` or `unicode` directly.
`six` provides some compatibility types to help here.
## Testing your migrations
Generally speaking, test your changes the same way we do everything else:
make a change locally, and rely on the CQ and CI bots to catch problems.
However, here are some specific guidelines for the different contexts
where we use Python:
### gclient hooks
To switch a gclient hook from Python2 to Python3, simply change `python`
to `python3` in the DEPS file, and make sure things still run :).
### GN/Ninja actions
All targets in the build use Python3 now, and anything declared via an
`action()` rule in GN will use Python3.
### Tests
Test targets that run by invoking python scripts (like telemetry_unittests
or blink_web_tests) should eventually migrate to using the [script_test]
GN templates. Once you do that, they will use Python3 by default.
Some tests still need to be migrated to `script_test()`
([crbug.com/1208648](https://crbug.com/1208648)). The process for
doing that is not yet well documented, so ask on python@chromium.org (or
ask dpranke@ directly).
There is no general mechanism for migrating tests that are C++ executables
that launch python via subprocesses, so you're on your own for dealing with
that.
### Presubmit checks
Presubmit checks are run using Python 3 by default now.
[script_test]: https://source.chromium.org/?q=script_test%20file:testing%2Ftest.gni&ss=chromium
[vpython]: https://chromium.googlesource.com/infra/infra/+/refs/heads/main/doc/users/vpython.md

11
styleguide/python/python.md

@ -3,13 +3,10 @@
_For other languages, please see the [Chromium style
guides](https://chromium.googlesource.com/chromium/src/+/main/styleguide/styleguide.md)._
As of 2021-05-12, Chromium is transitioning from Python 2 to Python 3 (follow
[crbug.com/941669](https://crbug.com/941669) for updates). See
[//docs/python3_migration.md](../../docs/python3_migration.md) for more on
how to migrate code.
For new (Python 3) code, you can assume Python 3.8 (and that's what the bots
will use), but we are increasingly seeing people running 3.9 locally as well.
We currently require Python 3.8 (as in, that's what the bots use, so don't
assume or require anything older *or* newer), but most newer versions of
Python3 will work fine for most things. There is an appropriate version
of Python3 in `$depot_tools/python-bin`, if you don't have one already.
We (often) use a tool called [vpython] to manage Python packages; vpython
is a wrapper around virtualenv. However, it is not safe to use vpython