0
Files
src/build/compute_build_timestamp.py
Bruce Dawson 02f0edc3fb Don't quantize build timestamps on official builds
Official builds - the PDBs and the PE files - often get published to a
symbol server. For PE files the path name in the symbol server is:

    "%s\%08X%s\%s" % (peName, timeStamp, imageSize, peName)

Since the peName for a particular DLL/EXE never changes and since the
size often doesn't change this means that the timeStamp is the only
differentiator between nearby builds. With the strategy of setting the
build timestamp to 5 am of the last commit time it is easy to get
a build that overwrites the previous build on the symbol server. This
happened when build 75.0.3770.143 overwrote all of the PE files from
build 75.0.3770.142, which complicated the investigation of (restricted
view, sorry) crbug.com/964273. This probably happened many other times.
The PDB files were never overwritten which is why this was not noticed
earlier.

When the 5 am quantization was added we were using the current time for
the build timestamps. Now that we are using the last commit time it is
less important to quantize to 5 am so this change removes that
quantization, for official builds only.

An increased number of days where we do multiple builds of one channel
means that this issue is hit more frequently than when the
quantization was initially added.

Bug: 993509
Change-Id: Ibfac95569b713ede056d3ff070db0c05b4a38c77
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1754527
Reviewed-by: Nico Weber <thakis@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Cr-Commit-Position: refs/heads/master@{#687349}
2019-08-15 18:23:05 +00:00

128 lines
4.8 KiB
Python
Executable File

#!/usr/bin/env python
# Copyright 2018 The Chromium Authors. All rights reserved.
# Use of this source code is governed by a BSD-style license that can be
# found in the LICENSE file.
"""Returns a timestamp that approximates the build date.
build_type impacts the timestamp generated, both relative to the date of the
last recent commit:
- default: the build date is set to the most recent first Sunday of a month at
5:00am. The reason is that it is a time where invalidating the build cache
shouldn't have major reprecussions (due to lower load).
- official: the build date is set to the current date at 5:00am, or the day
before if the current time is before 5:00am.
Either way, it is guaranteed to be in the past and always in UTC.
"""
# The requirements for the timestamp:
# (1) for the purposes of continuous integration, longer duration
# between cache invalidation is better, but >=1mo is preferable.
# (2) for security purposes, timebombs would ideally be as close to
# the actual time of the build as possible. It must be in the past.
# (3) HSTS certificate pinning is valid for 70 days. To make CI builds enforce
# HTST pinning, <=1mo is preferable.
#
# On Windows, the timestamp is also written in the PE/COFF file header of
# executables of dlls. That timestamp and the executable's file size are
# the only two pieces of information that identify a given executable on
# the symbol server, so rarely changing timestamps can cause conflicts there
# as well. We only upload symbols for official builds to the symbol server.
from __future__ import print_function
import argparse
import calendar
import datetime
import doctest
import os
import sys
THIS_DIR = os.path.abspath(os.path.dirname(__file__))
def GetFirstSundayOfMonth(year, month):
"""Returns the first sunday of the given month of the given year.
>>> GetFirstSundayOfMonth(2016, 2)
7
>>> GetFirstSundayOfMonth(2016, 3)
6
>>> GetFirstSundayOfMonth(2000, 1)
2
"""
weeks = calendar.Calendar().monthdays2calendar(year, month)
# Return the first day in the first week that is a Sunday.
return [date_day[0] for date_day in weeks[0] if date_day[1] == 6][0]
def GetUnofficialBuildDate(build_date):
"""Gets the approximate build date given the specific build type.
>>> GetUnofficialBuildDate(datetime.datetime(2016, 2, 6, 1, 2, 3))
datetime.datetime(2016, 1, 3, 5, 0)
>>> GetUnofficialBuildDate(datetime.datetime(2016, 2, 7, 5))
datetime.datetime(2016, 2, 7, 5, 0)
>>> GetUnofficialBuildDate(datetime.datetime(2016, 2, 8, 5))
datetime.datetime(2016, 2, 7, 5, 0)
"""
if build_date.hour < 5:
# The time is locked at 5:00 am in UTC to cause the build cache
# invalidation to not happen exactly at midnight. Use the same calculation
# as the day before.
# See //base/build_time.cc.
build_date = build_date - datetime.timedelta(days=1)
build_date = datetime.datetime(build_date.year, build_date.month,
build_date.day, 5, 0, 0)
day = build_date.day
month = build_date.month
year = build_date.year
first_sunday = GetFirstSundayOfMonth(year, month)
# If our build is after the first Sunday, we've already refreshed our build
# cache on a quiet day, so just use that day.
# Otherwise, take the first Sunday of the previous month.
if day >= first_sunday:
day = first_sunday
else:
month -= 1
if month == 0:
month = 12
year -= 1
day = GetFirstSundayOfMonth(year, month)
return datetime.datetime(
year, month, day, build_date.hour, build_date.minute, build_date.second)
def main():
if doctest.testmod()[0]:
return 1
argument_parser = argparse.ArgumentParser()
argument_parser.add_argument(
'build_type', help='The type of build', choices=('official', 'default'))
args = argument_parser.parse_args()
# The mtime of the revision in build/util/LASTCHANGE is stored in a file
# next to it. Read it, to get a deterministic time close to "now".
# That date is then modified as described at the top of the file so that
# it changes less frequently than with every commit.
# This intentionally always uses build/util/LASTCHANGE's commit time even if
# use_dummy_lastchange is set.
lastchange_file = os.path.join(THIS_DIR, 'util', 'LASTCHANGE.committime')
last_commit_timestamp = int(open(lastchange_file).read())
build_date = datetime.datetime.utcfromtimestamp(last_commit_timestamp)
# For official builds we want full fidelity time stamps because official
# builds are typically added to symbol servers and Windows symbol servers
# use the link timestamp as the prime differentiator, but for unofficial
# builds we do lots of quantization to avoid churn.
if args.build_type != 'official':
build_date = GetUnofficialBuildDate(build_date)
print(int(calendar.timegm(build_date.utctimetuple())))
return 0
if __name__ == '__main__':
sys.exit(main())