Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Restoring missing requirements.txt files #144

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Nov 21, 2015
Merged

Restoring missing requirements.txt files #144

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion 2 appengine/bigquery/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1 @@
google-api-python-client
google-api-python-client==1.4.2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we think it's better to pin versions? I feel like given we have CI tests its better to let them auto upgrade and only pin if they break.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, now I understand that's part of the point of this patch...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But I guess I'm still confused, why not keep the dependencies unpinned?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By pinning the requirements, we can be certain that our users get the versions that have been tested and vetted by us, and aren't surprised by breaking changes in new versions.

The scripts to check and update are for us, so that we can be made aware of new releases and get them tested and released to users.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

alright fine with me...its an age-old debate in other contexts too (static vs dynamic linkins is similar), advantages and disadvantages each way but if you want to pin, fine by me

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, in this case I just want to have the confidence that what we're giving to our users works.

2 changes: 1 addition & 1 deletion 2 appengine/mailgun/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1 @@
httplib2
httplib2==0.9.2
8 changes: 1 addition & 7 deletions 8 appengine/ndb/transactions/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1 @@
# This requirements file lists all third-party dependencies for this project.
#
# Run 'pip install -r requirements.txt -t lib/' to install these dependencies
# in `lib/` subdirectory.
#
# Note: The `lib` directory is added to `sys.path` by `appengine_config.py`.
Flask==0.10
Flask==0.10.1
2 changes: 1 addition & 1 deletion 2 appengine/storage/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1 @@
google-api-python-client
google-api-python-client==1.4.2
4 changes: 2 additions & 2 deletions 4 bigquery/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@ This section contains samples for [Google BigQuery](https://cloud.google.com/big
1. Your environment must be setup with [authentication
information](https://developers.google.com/identity/protocols/application-default-credentials#howtheywork). If you're running in your local development environment and you have the [Google Cloud SDK](https://cloud.google.com/sdk/) installed, you can do this easily by running:

$ gcloud auth login
$ gcloud init

2. Install dependencies from the top-level [`requirements.txt`](../requirements.txt):
2. Install dependencies in `requirements.txt`:

$ pip install -r requirements.txt

Expand Down
1 change: 1 addition & 0 deletions 1 bigquery/api/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
google-api-python-client==1.4.2
4 changes: 2 additions & 2 deletions 4 blog/introduction_to_data_models_in_cloud_datastore/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,12 @@ using [Google Cloud Datastore](https://cloud.google.com/datastore).

2. [Enable the Datastore API](https://console.developers.google.com/project/_/apiui/apiview/datastore/overview).

3. Install the [Google Cloud SDK](https://cloud.google.com/sdk) and be sure to run ``gcloud auth``.
3. Install the [Google Cloud SDK](https://cloud.google.com/sdk) and be sure to run ``gcloud init``.


## Running the samples

Install dependencies from the top-level [`requirements.txt`](../../requirements.txt):
Install dependencies from `requirements.txt`:

pip install -r requirements.txt

Expand Down
1 change: 1 addition & 0 deletions 1 blog/introduction_to_data_models_in_cloud_datastore/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
gcloud==0.8.0
4 changes: 2 additions & 2 deletions 4 cloud_logging/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@ This section contains samples for [Google Cloud Logging](https://cloud.google.co
1. Your environment must be setup with [authentication
information](https://developers.google.com/identity/protocols/application-default-credentials#howtheywork). If you're running in your local development environment and you have the [Google Cloud SDK](https://cloud.google.com/sdk/) installed, you can do this easily by running:

$ gcloud auth login
$ gcloud init

2. Install dependencies from the top-level [`requirements.txt`](../requirements.txt):
2. Install dependencies from `requirements.txt`:

$ pip install -r requirements.txt

Expand Down
1 change: 1 addition & 0 deletions 1 cloud_logging/api/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
google-api-python-client==1.4.2
2 changes: 1 addition & 1 deletion 2 compute/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ locally and set the `GOOGLE_APPLICATION_CREDENTIALS` environment variable.

$ export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service_account.json

2. Install dependencies from the top-level [`requirements.txt`](../requirements.txt):
2. Install dependencies from `requirements.txt`

$ pip install -r requirements.txt

Expand Down
1 change: 1 addition & 0 deletions 1 compute/api/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
google-api-python-client==1.4.2
2 changes: 1 addition & 1 deletion 2 monitoring/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ locally and set the `GOOGLE_APPLICATION_CREDENTIALS` environment variable.

$ export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service_account.json

2. Install dependencies from the top-level [`requirements.txt`](../requirements.txt):
2. Install dependencies from `requirements.txt`:

$ pip install -r requirements.txt

Expand Down
1 change: 1 addition & 0 deletions 1 monitoring/api/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
google-api-python-client==1.4.2
16 changes: 8 additions & 8 deletions 16 requirements-dev.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
beautifulsoup4==4.4.0
coverage==3.7.1
beautifulsoup4==4.4.1
coverage==4.0.2
Flask==0.10.1
funcsigs==0.4
itsdangerous==0.24
Expand All @@ -8,11 +8,11 @@ MarkupSafe==0.23
mock==1.3.0
nose==1.3.7
nose-exclude==0.4.1
nosegae==0.5.7
pbr==1.8.0
nosegae==0.5.8
pbr==1.8.1
PyYAML==3.11
waitress==0.8.10
WebOb==1.4.1
WebTest==2.0.18
Werkzeug==0.10.4
nose-timer
WebOb==1.5.1
WebTest==2.0.20
Werkzeug==0.11.2
nose-timer==0.5.0
16 changes: 3 additions & 13 deletions 16 requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,4 @@
gcloud==0.7.1
gcloud==0.8.0
google-api-python-client==1.4.2
google-apitools==0.4.11
httplib2==0.9.1
oauth2client==1.5.1
protobuf==3.0.0a1
protorpc==0.11.1
pyasn1==0.1.8
pyasn1-modules==0.0.7
pycrypto==2.6.1
rsa==3.2
simplejson==3.8.0
six==1.9.0
uritemplate==0.6
oauth2client==1.5.2
requests[security]==2.8.1
53 changes: 53 additions & 0 deletions 53 scripts/check_requirements.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
#!/usr/bin/env python

# Copyright (C) 2013 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""
Checks dependencies in requirements.txt to ensure they are the latest version.
"""

import argparse
import sys

from update_requirements import get_package_info, read_requirements


def check_req(req):
info = get_package_info(req.project_name)
newest_version = info['version']
current_spec = req.specs[0] if req.specs else ('==', 'unspecified')
if current_spec[1] != newest_version:
return req, newest_version


def main(req_file):
reqs = read_requirements(req_file)
outdated_reqs = filter(None, [check_req(req) for req in reqs])

if outdated_reqs:
for req in outdated_reqs:
print("{} is out of date, latest version is {}".format(*req))
sys.exit(1)


if __name__ == '__main__':
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument(
'requirements_file',
help='Path the the requirements.txt file to check.')

args = parser.parse_args()

main(args.requirements_file)
74 changes: 74 additions & 0 deletions 74 scripts/update_requirements.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
#!/usr/bin/env python

# Copyright (C) 2013 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""
Updates dependencies in requirements.txt to the latest version.
"""

import argparse

from pip.req.req_file import parse_requirements
from pkg_resources import Requirement
import requests


def get_package_info(package):
url = 'https://pypi.python.org/pypi/{}/json'.format(package)
r = requests.get(url)
r.raise_for_status()
return r.json()['info']


def read_requirements(req_file):
return [x.req for x in parse_requirements(req_file, session={})]


def update_req(req):
info = get_package_info(req.project_name)
newest_version = info['version']
current_spec = req.specs[0] if req.specs else ('==', 'unspecified')
new_spec = ('==', newest_version)
if current_spec != new_spec:
newreq = Requirement(req.unsafe_name, [new_spec], req.extras)
print('Updated {} from {} -> {}'.format(
req.project_name,
current_spec[1],
newest_version))
return newreq
return req


def write_requirements(reqs, req_file):
with open(req_file, 'w') as f:
for req in reqs:
f.write('{}\n'.format(req))


def main(req_file):
reqs = read_requirements(req_file)
reqs = [update_req(req) for req in reqs]
write_requirements(reqs, req_file)


if __name__ == '__main__':
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument(
'requirements_file',
help='Path the the requirements.txt file to update.')

args = parser.parse_args()

main(args.requirements_file)
4 changes: 2 additions & 2 deletions 4 storage/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@ This section contains samples for [Google Cloud Storage](https://cloud.google.co
1. Your environment must be setup with [authentication
information](https://developers.google.com/identity/protocols/application-default-credentials#howtheywork). If you're running in your local development environment and you have the [Google Cloud SDK](https://cloud.google.com/sdk/) installed, you can do this easily by running:

$ gcloud auth login
$ gcloud init

2. Install dependencies from the top-level [`requirements.txt`](../requirements.txt):
2. Install dependencies from `requirements.txt`:

$ pip install -r requirements.txt

Expand Down
2 changes: 1 addition & 1 deletion 2 storage/api/compose_objects.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@

def main(bucket, destination, sources):
# Get the application default credentials. When running locally, these are
# available after running `gcloud auth login`. When running on compute
# available after running `gcloud init`. When running on compute
# engine, these are available from the environment.
credentials = GoogleCredentials.get_application_default()

Expand Down
2 changes: 1 addition & 1 deletion 2 storage/api/list_objects.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@
def main(bucket):
# [START list_bucket]
# Get the application default credentials. When running locally, these are
# available after running `gcloud auth login`. When running on compute
# available after running `gcloud init`. When running on compute
# engine, these are available from the environment.
credentials = GoogleCredentials.get_application_default()

Expand Down
1 change: 1 addition & 0 deletions 1 storage/api/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
google-api-python-client==1.4.2
3 changes: 1 addition & 2 deletions 3 storage/transfer_service/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,7 @@ These samples are used on the following documentation pages:
storage-transfer-<accountId>@partnercontent.gserviceaccount.com
1. Set up gcloud for application default credentials.
1. `gcloud components update`
1. `gcloud auth login`
1. `gcloud config set project PROJECT_ID`
1. `gcloud init`
1. Install [Google API Client Library for Python](https://developers.google.com/api-client-library/python/start/installation).

## Transfer from Amazon S3 to Google Cloud Storage
Expand Down
1 change: 1 addition & 0 deletions 1 storage/transfer_service/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
google-api-python-client==1.4.2
16 changes: 15 additions & 1 deletion 16 tox.ini
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[tox]
skipsdist = True
envlist = pep8, gae, py27, py34
envlist = pep8, reqcheck, gae, py27, py34

[testenv]
passenv = PYTHONPATH GOOGLE_* GCLOUD_* TEST_* TRAVIS*
Expand All @@ -15,6 +15,20 @@ commonargs =
--cover-branches
--cover-inclusive

[testenv:reqcheck]
deps =
requests[security]
commands =
bash -c "find . -name requirements.txt |\
xargs -n 1 ./scripts/check_requirements.py"

[testenv:requpdate]
deps =
requests[security]
commands =
bash -c "find . -name requirements.txt |\
xargs -n 1 ./scripts/update_requirements.py"

[testenv:gae]
deps =
{[testenv]deps}
Expand Down
Morty Proxy This is a proxified and sanitized view of the page, visit original site.