Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit c61f9d1

Browse filesBrowse files
author
Jon Wayne Parrott
committed
Adding Natural Language API samples.
Change-Id: I68a1b5a11c2b3703963466b195be37a2c796bf79
1 parent 1294827 commit c61f9d1
Copy full SHA for c61f9d1
Expand file treeCollapse file tree

20 files changed

+2177
-1
lines changed

‎.coveragerc

Copy file name to clipboardExpand all lines: .coveragerc
+1Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ include =
99
dns/*
1010
datastore/*
1111
error_reporting/*
12+
language/*
1213
managed_vms/*
1314
monitoring/*
1415
speech/*

‎conftest.py

Copy file name to clipboardExpand all lines: conftest.py
+23-1Lines changed: 23 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,9 +15,10 @@
1515
import os
1616

1717
import pytest
18+
import requests
1819

1920

20-
class Namespace:
21+
class Namespace(object):
2122
def __init__(self, **kwargs):
2223
self.__dict__.update(kwargs)
2324

@@ -48,3 +49,24 @@ def resource(request):
4849
testing resource"""
4950
local_path = os.path.dirname(request.module.__file__)
5051
return lambda *args: get_resource_path(args, local_path)
52+
53+
54+
def fetch_gcs_resource(resource, tmpdir, _chunk_size=1024):
55+
resp = requests.get(resource, stream=True)
56+
dest_file = str(tmpdir.join(os.path.basename(resource)))
57+
with open(dest_file, 'wb') as f:
58+
for chunk in resp.iter_content(_chunk_size):
59+
f.write(chunk)
60+
61+
return dest_file
62+
63+
64+
@pytest.fixture(scope='module')
65+
def remote_resource(cloud_config):
66+
"""Provides a function that downloads the given resource from Cloud
67+
Storage, returning the path to the downloaded resource."""
68+
remote_uri = 'http://storage.googleapis.com/{}/'.format(
69+
cloud_config.storage_bucket)
70+
71+
return lambda path, tmpdir: fetch_gcs_resource(
72+
remote_uri + path.strip('/'), tmpdir)

‎language/README.md

Copy file name to clipboard
+17Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
# Google Cloud Natural Language API examples
2+
3+
This directory contains Python examples that use the
4+
[Google Cloud Natural Language API](https://cloud.google.com/natural-language/).
5+
6+
- [api](api) has a simple command line tool that shows off the API's features.
7+
8+
- [movie_nl](movie_nl) combines sentiment and entity analysis to come up with
9+
actors/directors who are the most and least popular in the imdb movie reviews.
10+
11+
- [ocr_nl](ocr_nl) uses the [Cloud Vision API](https://cloud.google.com/vision/)
12+
to extract text from images, then uses the NL API to extract entity information
13+
from those texts, and stores the extracted information in a database in support
14+
of further analysis and correlation.
15+
16+
- [syntax_triples](syntax_triples) uses syntax analysis to find
17+
subject-verb-object triples in a given piece of text.

‎language/api/README.md

Copy file name to clipboard
+87Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
2+
# Google Cloud Natural Language API Sample
3+
4+
This Python sample demonstrates the use of the [Google Cloud Natural Language API][NL-Docs]
5+
for sentiment, entity, and syntax analysis.
6+
7+
[NL-Docs]: https://cloud.google.com/natural-language/docs/
8+
9+
## Setup
10+
11+
Please follow the [Set Up Your Project](https://cloud.google.com/natural-language/docs/getting-started#set_up_your_project)
12+
steps in the Quickstart doc to create a project and enable the
13+
Cloud Natural Language API. Following those steps, make sure that you
14+
[Set Up a Service Account](https://cloud.google.com/natural-language/docs/common/auth#set_up_a_service_account),
15+
and export the following environment variable:
16+
17+
```
18+
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/your-project-credentials.json
19+
```
20+
21+
## Run the sample
22+
23+
Install [pip](https://pip.pypa.io/en/stable/installing) if not already installed.
24+
25+
To run the example, install the necessary libraries using pip:
26+
27+
```sh
28+
$ pip install -r requirements.txt
29+
```
30+
31+
Then, run the script:
32+
33+
```sh
34+
$ python analyze.py <command> <text-string>
35+
```
36+
37+
where `<command>` is one of: `entities`, `sentiment`, or `syntax`.
38+
39+
The script will write to STDOUT the json returned from the API for the requested feature.
40+
41+
For example, if you run:
42+
43+
```sh
44+
$ python analyze.py entities "Tom Sawyer is a book written by a guy known as Mark Twain."
45+
```
46+
47+
You will see something like the following returned:
48+
49+
```
50+
{
51+
"entities": [
52+
{
53+
"salience": 0.49785897,
54+
"mentions": [
55+
{
56+
"text": {
57+
"content": "Tom Sawyer",
58+
"beginOffset": 0
59+
}
60+
}
61+
],
62+
"type": "PERSON",
63+
"name": "Tom Sawyer",
64+
"metadata": {
65+
"wikipedia_url": "http://en.wikipedia.org/wiki/The_Adventures_of_Tom_Sawyer"
66+
}
67+
},
68+
{
69+
"salience": 0.12209519,
70+
"mentions": [
71+
{
72+
"text": {
73+
"content": "Mark Twain",
74+
"beginOffset": 47
75+
}
76+
}
77+
],
78+
"type": "PERSON",
79+
"name": "Mark Twain",
80+
"metadata": {
81+
"wikipedia_url": "http://en.wikipedia.org/wiki/Mark_Twain"
82+
}
83+
}
84+
],
85+
"language": "en"
86+
}
87+
```

‎language/api/analyze.py

Copy file name to clipboard
+115Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
#!/usr/bin/env python
2+
3+
# Copyright 2016 Google, Inc
4+
#
5+
# Licensed under the Apache License, Version 2.0 (the "License");
6+
# you may not use this file except in compliance with the License.
7+
# You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing, software
12+
# distributed under the License is distributed on an "AS IS" BASIS,
13+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
# See the License for the specific language governing permissions and
15+
# limitations under the License.
16+
17+
"""Analyzes text using the Google Cloud Natural Language API."""
18+
19+
import argparse
20+
import json
21+
import sys
22+
23+
from googleapiclient import discovery
24+
import httplib2
25+
from oauth2client.client import GoogleCredentials
26+
27+
28+
def get_service():
29+
credentials = GoogleCredentials.get_application_default()
30+
scoped_credentials = credentials.create_scoped(
31+
['https://www.googleapis.com/auth/cloud-platform'])
32+
http = httplib2.Http()
33+
scoped_credentials.authorize(http)
34+
return discovery.build('language', 'v1beta1', http=http)
35+
36+
37+
def get_native_encoding_type():
38+
"""Returns the encoding type that matches Python's native strings."""
39+
if sys.maxunicode == 65535:
40+
return 'UTF16'
41+
else:
42+
return 'UTF32'
43+
44+
45+
def analyze_entities(text, encoding='UTF32'):
46+
body = {
47+
'document': {
48+
'type': 'PLAIN_TEXT',
49+
'content': text,
50+
},
51+
'encodingType': encoding,
52+
}
53+
54+
service = get_service()
55+
56+
request = service.documents().analyzeEntities(body=body)
57+
response = request.execute()
58+
59+
return response
60+
61+
62+
def analyze_sentiment(text):
63+
body = {
64+
'document': {
65+
'type': 'PLAIN_TEXT',
66+
'content': text,
67+
}
68+
}
69+
70+
service = get_service()
71+
72+
request = service.documents().analyzeSentiment(body=body)
73+
response = request.execute()
74+
75+
return response
76+
77+
78+
def analyze_syntax(text, encoding='UTF32'):
79+
body = {
80+
'document': {
81+
'type': 'PLAIN_TEXT',
82+
'content': text,
83+
},
84+
'features': {
85+
'extract_syntax': True,
86+
},
87+
'encodingType': encoding,
88+
}
89+
90+
service = get_service()
91+
92+
request = service.documents().annotateText(body=body)
93+
response = request.execute()
94+
95+
return response
96+
97+
98+
if __name__ == '__main__':
99+
parser = argparse.ArgumentParser(
100+
description=__doc__,
101+
formatter_class=argparse.RawDescriptionHelpFormatter)
102+
parser.add_argument('command', choices=[
103+
'entities', 'sentiment', 'syntax'])
104+
parser.add_argument('text')
105+
106+
args = parser.parse_args()
107+
108+
if args.command == 'entities':
109+
result = analyze_entities(args.text, get_native_encoding_type())
110+
elif args.command == 'sentiment':
111+
result = analyze_sentiment(args.text)
112+
elif args.command == 'syntax':
113+
result = analyze_syntax(args.text, get_native_encoding_type())
114+
115+
print(json.dumps(result, indent=2))

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.