Skip to content

Navigation Menu

Sign in

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit 11bc9c2

Browse filesBrowse files
busunkim96gcf-owl-bot[bot]
busunkim96
and
gcf-owl-bot[bot]
authored
chore: fix release build (#659)
* chore: fix release build Fix release build by migrating to secret manager secrets and use templated kokoro configs for docs/ and release/ * fix: fix config names * chore: add populate secrets script * docs: fix license * chore: preserve original year * chore: revert years in manually committed files * 🦉 Updates from OwlBot See https://github.com/googleapis/repo-automation-bots/blob/master/packages/owl-bot/README.md * chore: update lockfile * chore: add .kokoro/docker directory Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
1 parent 306eeba commit 11bc9c2
Copy full SHA for 11bc9c2

15 files changed

+139
-163
lines changed

‎.github/.OwlBot.lock.yaml

Copy file name to clipboard
+9
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
# Copyright 2021 Google LLC
2+
#
3+
# Use of this source code is governed by a BSD-style
4+
# license that can be found in the LICENSE file or at
5+
# https://developers.google.com/open-source/licenses/bsd
6+
7+
docker:
8+
image: gcr.io/cloud-devrel-public-resources/owlbot-python:latest
9+
digest: sha256:82d046d572a11b84d64d4b9af93ad55a1b6a4854917504ba7557acdfce907dde

‎.github/.OwlBot.yaml

Copy file name to clipboard
+8
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
# Copyright 2021 Google LLC
2+
#
3+
# Use of this source code is governed by a BSD-style
4+
# license that can be found in the LICENSE file or at
5+
# https://developers.google.com/open-source/licenses/bsd
6+
7+
docker:
8+
image: gcr.io/repo-automation-bots/owlbot-python:latest

‎.kokoro/build.sh

Copy file name to clipboardExpand all lines: .kokoro/build.sh
+3-11
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,9 @@
11
#!/bin/bash
22
# Copyright 2020 Google LLC
33
#
4-
# Licensed under the Apache License, Version 2.0 (the "License");
5-
# you may not use this file except in compliance with the License.
6-
# You may obtain a copy of the License at
7-
#
8-
# https://www.apache.org/licenses/LICENSE-2.0
9-
#
10-
# Unless required by applicable law or agreed to in writing, software
11-
# distributed under the License is distributed on an "AS IS" BASIS,
12-
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13-
# See the License for the specific language governing permissions and
14-
# limitations under the License.
4+
# Use of this source code is governed by a BSD-style
5+
# license that can be found in the LICENSE file or at
6+
# https://developers.google.com/open-source/licenses/bsd
157

168
set -eo pipefail
179

‎.kokoro/docker/docs/Dockerfile

Copy file name to clipboard
+5-44
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,8 @@
11
# Copyright 2020 Google LLC
22
#
3-
# Licensed under the Apache License, Version 2.0 (the "License");
4-
# you may not use this file except in compliance with the License.
5-
# You may obtain a copy of the License at
6-
#
7-
# http://www.apache.org/licenses/LICENSE-2.0
8-
#
9-
# Unless required by applicable law or agreed to in writing, software
10-
# distributed under the License is distributed on an "AS IS" BASIS,
11-
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12-
# See the License for the specific language governing permissions and
13-
# limitations under the License.
3+
# Use of this source code is governed by a BSD-style
4+
# license that can be found in the LICENSE file or at
5+
# https://developers.google.com/open-source/licenses/bsd
146

157
from ubuntu:20.04
168

@@ -40,6 +32,7 @@ RUN apt-get update \
4032
libssl-dev \
4133
libsqlite3-dev \
4234
portaudio19-dev \
35+
python3-distutils \
4336
redis-server \
4437
software-properties-common \
4538
ssh \
@@ -59,40 +52,8 @@ RUN apt-get update \
5952
&& rm -rf /var/lib/apt/lists/* \
6053
&& rm -f /var/cache/apt/archives/*.deb
6154

62-
63-
COPY fetch_gpg_keys.sh /tmp
64-
# Install the desired versions of Python.
65-
RUN set -ex \
66-
&& export GNUPGHOME="$(mktemp -d)" \
67-
&& echo "disable-ipv6" >> "${GNUPGHOME}/dirmngr.conf" \
68-
&& /tmp/fetch_gpg_keys.sh \
69-
&& for PYTHON_VERSION in 3.7.8 3.8.5; do \
70-
wget --no-check-certificate -O python-${PYTHON_VERSION}.tar.xz "https://www.python.org/ftp/python/${PYTHON_VERSION%%[a-z]*}/Python-$PYTHON_VERSION.tar.xz" \
71-
&& wget --no-check-certificate -O python-${PYTHON_VERSION}.tar.xz.asc "https://www.python.org/ftp/python/${PYTHON_VERSION%%[a-z]*}/Python-$PYTHON_VERSION.tar.xz.asc" \
72-
&& gpg --batch --verify python-${PYTHON_VERSION}.tar.xz.asc python-${PYTHON_VERSION}.tar.xz \
73-
&& rm -r python-${PYTHON_VERSION}.tar.xz.asc \
74-
&& mkdir -p /usr/src/python-${PYTHON_VERSION} \
75-
&& tar -xJC /usr/src/python-${PYTHON_VERSION} --strip-components=1 -f python-${PYTHON_VERSION}.tar.xz \
76-
&& rm python-${PYTHON_VERSION}.tar.xz \
77-
&& cd /usr/src/python-${PYTHON_VERSION} \
78-
&& ./configure \
79-
--enable-shared \
80-
# This works only on Python 2.7 and throws a warning on every other
81-
# version, but seems otherwise harmless.
82-
--enable-unicode=ucs4 \
83-
--with-system-ffi \
84-
--without-ensurepip \
85-
&& make -j$(nproc) \
86-
&& make install \
87-
&& ldconfig \
88-
; done \
89-
&& rm -rf "${GNUPGHOME}" \
90-
&& rm -rf /usr/src/python* \
91-
&& rm -rf ~/.cache/
92-
9355
RUN wget -O /tmp/get-pip.py 'https://bootstrap.pypa.io/get-pip.py' \
94-
&& python3.7 /tmp/get-pip.py \
9556
&& python3.8 /tmp/get-pip.py \
9657
&& rm /tmp/get-pip.py
9758

98-
CMD ["python3.7"]
59+
CMD ["python3.8"]

‎.kokoro/docker/docs/fetch_gpg_keys.sh

Copy file name to clipboardExpand all lines: .kokoro/docker/docs/fetch_gpg_keys.sh
+5-13
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,13 @@
11
#!/bin/bash
22
# Copyright 2020 Google LLC
33
#
4-
# Licensed under the Apache License, Version 2.0 (the "License");
5-
# you may not use this file except in compliance with the License.
6-
# You may obtain a copy of the License at
7-
#
8-
# http://www.apache.org/licenses/LICENSE-2.0
9-
#
10-
# Unless required by applicable law or agreed to in writing, software
11-
# distributed under the License is distributed on an "AS IS" BASIS,
12-
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13-
# See the License for the specific language governing permissions and
14-
# limitations under the License.
4+
# Use of this source code is governed by a BSD-style
5+
# license that can be found in the LICENSE file or at
6+
# https://developers.google.com/open-source/licenses/bsd
157

168
# A script to fetch gpg keys with retry.
179
# Avoid jinja parsing the file.
18-
#
10+
#
1911

2012
function retry {
2113
if [[ "${#}" -le 1 ]]; then
@@ -42,4 +34,4 @@ retry 3 gpg --keyserver ha.pool.sks-keyservers.net --recv-keys \
4234
retry 3 gpg --keyserver ha.pool.sks-keyservers.net --recv-keys \
4335
E3FF2839C048B25C084DEBE9B26995E310250568
4436

45-
#
37+
#

‎.kokoro/docs/common.cfg

Copy file name to clipboardExpand all lines: .kokoro/docs/common.cfg
+1-1
Original file line numberDiff line numberDiff line change
@@ -62,4 +62,4 @@ before_action {
6262
keyname: "docuploader_service_account"
6363
}
6464
}
65-
}
65+
}

‎.kokoro/populate-secrets.sh

Copy file name to clipboard
+35
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
#!/bin/bash
2+
# Copyright 2020 Google LLC
3+
#
4+
# Use of this source code is governed by a BSD-style
5+
# license that can be found in the LICENSE file or at
6+
# https://developers.google.com/open-source/licenses/bsd
7+
8+
set -eo pipefail
9+
10+
function now { date +"%Y-%m-%d %H:%M:%S" | tr -d '\n' ;}
11+
function msg { println "$*" >&2 ;}
12+
function println { printf '%s\n' "$(now) $*" ;}
13+
14+
15+
# Populates requested secrets set in SECRET_MANAGER_KEYS from service account:
16+
# kokoro-trampoline@cloud-devrel-kokoro-resources.iam.gserviceaccount.com
17+
SECRET_LOCATION="${KOKORO_GFILE_DIR}/secret_manager"
18+
msg "Creating folder on disk for secrets: ${SECRET_LOCATION}"
19+
mkdir -p ${SECRET_LOCATION}
20+
for key in $(echo ${SECRET_MANAGER_KEYS} | sed "s/,/ /g")
21+
do
22+
msg "Retrieving secret ${key}"
23+
docker run --entrypoint=gcloud \
24+
--volume=${KOKORO_GFILE_DIR}:${KOKORO_GFILE_DIR} \
25+
gcr.io/google.com/cloudsdktool/cloud-sdk \
26+
secrets versions access latest \
27+
--project cloud-devrel-kokoro-resources \
28+
--secret ${key} > \
29+
"${SECRET_LOCATION}/${key}"
30+
if [[ $? == 0 ]]; then
31+
msg "Secret written to ${SECRET_LOCATION}/${key}"
32+
else
33+
msg "Error retrieving secret ${key}"
34+
fi
35+
done

‎.kokoro/publish-docs.sh

Copy file name to clipboardExpand all lines: .kokoro/publish-docs.sh
+3-11
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,9 @@
11
#!/bin/bash
22
# Copyright 2020 Google LLC
33
#
4-
# Licensed under the Apache License, Version 2.0 (the "License");
5-
# you may not use this file except in compliance with the License.
6-
# You may obtain a copy of the License at
7-
#
8-
# https://www.apache.org/licenses/LICENSE-2.0
9-
#
10-
# Unless required by applicable law or agreed to in writing, software
11-
# distributed under the License is distributed on an "AS IS" BASIS,
12-
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13-
# See the License for the specific language governing permissions and
14-
# limitations under the License.
4+
# Use of this source code is governed by a BSD-style
5+
# license that can be found in the LICENSE file or at
6+
# https://developers.google.com/open-source/licenses/bsd
157

168
set -eo pipefail
179

‎.kokoro/release.sh

Copy file name to clipboard
+5-15
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,9 @@
11
#!/bin/bash
22
# Copyright 2020 Google LLC
33
#
4-
# Licensed under the Apache License, Version 2.0 (the "License");
5-
# you may not use this file except in compliance with the License.
6-
# You may obtain a copy of the License at
7-
#
8-
# https://www.apache.org/licenses/LICENSE-2.0
9-
#
10-
# Unless required by applicable law or agreed to in writing, software
11-
# distributed under the License is distributed on an "AS IS" BASIS,
12-
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13-
# See the License for the specific language governing permissions and
14-
# limitations under the License.
15-
16-
#!/bin/bash
4+
# Use of this source code is governed by a BSD-style
5+
# license that can be found in the LICENSE file or at
6+
# https://developers.google.com/open-source/licenses/bsd
177

188
set -eo pipefail
199

@@ -28,7 +18,7 @@ python3 -m pip install --upgrade twine wheel setuptools
2818
export PYTHONUNBUFFERED=1
2919

3020
# Move into the package, build the distribution and upload.
31-
TWINE_PASSWORD=$(cat "${KOKORO_KEYSTORE_DIR}/73713_google_cloud_pypi_password")
21+
TWINE_PASSWORD=$(cat "${KOKORO_GFILE_DIR}/secret_manager/google-cloud-pypi-token")
3222
cd github/python-spanner-django
3323
python3 setup.py sdist bdist_wheel
34-
twine upload --username gcloudpypi --password "${TWINE_PASSWORD}" dist/*
24+
twine upload --username __token__ --password "${TWINE_PASSWORD}" dist/*

‎.kokoro/release/common.cfg

Copy file name to clipboardExpand all lines: .kokoro/release/common.cfg
+4-38
Original file line numberDiff line numberDiff line change
@@ -23,42 +23,8 @@ env_vars: {
2323
value: "github/python-spanner-django/.kokoro/release.sh"
2424
}
2525

26-
# Fetch the token needed for reporting release status to GitHub
27-
before_action {
28-
fetch_keystore {
29-
keystore_resource {
30-
keystore_config_id: 73713
31-
keyname: "yoshi-automation-github-key"
32-
}
33-
}
34-
}
35-
36-
# Fetch PyPI password
37-
before_action {
38-
fetch_keystore {
39-
keystore_resource {
40-
keystore_config_id: 73713
41-
keyname: "google_cloud_pypi_password"
42-
}
43-
}
44-
}
45-
46-
# Fetch magictoken to use with Magic Github Proxy
47-
before_action {
48-
fetch_keystore {
49-
keystore_resource {
50-
keystore_config_id: 73713
51-
keyname: "releasetool-magictoken"
52-
}
53-
}
54-
}
55-
56-
# Fetch api key to use with Magic Github Proxy
57-
before_action {
58-
fetch_keystore {
59-
keystore_resource {
60-
keystore_config_id: 73713
61-
keyname: "magic-github-proxy-api-key"
62-
}
63-
}
26+
# Tokens needed to report release status back to GitHub
27+
env_vars: {
28+
key: "SECRET_MANAGER_KEYS"
29+
value: "releasetool-publish-reporter-app,releasetool-publish-reporter-googleapis-installation,releasetool-publish-reporter-pem,google-cloud-pypi-token"
6430
}

‎.kokoro/release/release.cfg

Copy file name to clipboard
+1-1
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
# Format: //devtools/kokoro/config/proto/build.proto
1+
# Format: //devtools/kokoro/config/proto/build.proto

‎.kokoro/trampoline.sh

Copy file name to clipboard
+14-17
Original file line numberDiff line numberDiff line change
@@ -1,23 +1,20 @@
11
#!/bin/bash
2-
# Copyright 2020 Google LLC.
2+
# Copyright 2017 Google LLC
33
#
4-
# Licensed under the Apache License, Version 2.0 (the "License");
5-
# you may not use this file except in compliance with the License.
6-
# You may obtain a copy of the License at
7-
#
8-
# http://www.apache.org/licenses/LICENSE-2.0
9-
#
10-
# Unless required by applicable law or agreed to in writing, software
11-
# distributed under the License is distributed on an "AS IS" BASIS,
12-
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13-
# See the License for the specific language governing permissions and
14-
# limitations under the License.
4+
# Use of this source code is governed by a BSD-style
5+
# license that can be found in the LICENSE file or at
6+
# https://developers.google.com/open-source/licenses/bsd
157

168
set -eo pipefail
179

18-
python3 "${KOKORO_GFILE_DIR}/trampoline_v1.py" || ret_code=$?
19-
20-
chmod +x ${KOKORO_GFILE_DIR}/trampoline_cleanup.sh
21-
${KOKORO_GFILE_DIR}/trampoline_cleanup.sh || true
10+
# Always run the cleanup script, regardless of the success of bouncing into
11+
# the container.
12+
function cleanup() {
13+
chmod +x ${KOKORO_GFILE_DIR}/trampoline_cleanup.sh
14+
${KOKORO_GFILE_DIR}/trampoline_cleanup.sh
15+
echo "cleanup";
16+
}
17+
trap cleanup EXIT
2218

23-
exit ${ret_code}
19+
$(dirname $0)/populate-secrets.sh # Secret Manager secrets.
20+
python3 "${KOKORO_GFILE_DIR}/trampoline_v1.py"

‎.kokoro/trampoline_v2.sh

Copy file name to clipboardExpand all lines: .kokoro/trampoline_v2.sh
+3-11
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,9 @@
11
#!/usr/bin/env bash
22
# Copyright 2020 Google LLC
33
#
4-
# Licensed under the Apache License, Version 2.0 (the "License");
5-
# you may not use this file except in compliance with the License.
6-
# You may obtain a copy of the License at
7-
#
8-
# http://www.apache.org/licenses/LICENSE-2.0
9-
#
10-
# Unless required by applicable law or agreed to in writing, software
11-
# distributed under the License is distributed on an "AS IS" BASIS,
12-
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13-
# See the License for the specific language governing permissions and
14-
# limitations under the License.
4+
# Use of this source code is governed by a BSD-style
5+
# license that can be found in the LICENSE file or at
6+
# https://developers.google.com/open-source/licenses/bsd
157

168
# trampoline_v2.sh
179
#

‎.trampolinerc

Copy file name to clipboardExpand all lines: .trampolinerc
+1-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Copyright 2021 Google LLC
1+
# Copyright 2020 Google LLC
22
#
33
# Licensed under the Apache License, Version 2.0 (the "License");
44
# you may not use this file except in compliance with the License.

‎owlbot.py

Copy file name to clipboard
+42
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# Copyright 2021 Google LLC
2+
#
3+
# Use of this source code is governed by a BSD-style
4+
# license that can be found in the LICENSE file or at
5+
# https://developers.google.com/open-source/licenses/bsd
6+
7+
"""This script is used to synthesize generated parts of this library."""
8+
import re
9+
10+
import synthtool as s
11+
import synthtool.gcp as gcp
12+
13+
# ----------------------------------------------------------------------------
14+
# Add templated files
15+
# ----------------------------------------------------------------------------
16+
templated_files = gcp.CommonTemplates().py_library(microgenerator=True)
17+
18+
# Just move templates for building docs and releases
19+
# Presubmit and continuous are configured differently
20+
s.move(templated_files / ".trampolinerc")
21+
s.move(templated_files / ".kokoro" / "docker")
22+
s.move(templated_files / ".kokoro" / "docs")
23+
s.move(templated_files / ".kokoro" / "release.sh")
24+
s.move(templated_files / ".kokoro" / "trampoline_v2.sh")
25+
s.move(templated_files / ".kokoro" / "trampoline.sh")
26+
s.move(templated_files / ".kokoro" / "populate-secrets.sh")
27+
s.move(templated_files / ".kokoro" / "release")
28+
29+
# Replace the Apache Licenses in the `.kokoro` directory
30+
# with the BSD license expected in this repository
31+
s.replace(
32+
".kokoro/**/*",
33+
"# Copyright.*(\d{4}).*# limitations under the License\.",
34+
"""# Copyright \g<1> Google LLC
35+
#
36+
# Use of this source code is governed by a BSD-style
37+
# license that can be found in the LICENSE file or at
38+
# https://developers.google.com/open-source/licenses/bsd""",
39+
flags=re.DOTALL
40+
)
41+
42+
s.shell.run(["nox", "-s", "blacken"], hide_output=False)

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.