Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit 5c63e3c

Browse filesBrowse files
author
chenyumic
authored
Added samples for Storage, OCR, and Slack tutorial (GoogleCloudPlatform#1585)
* Added samples for Storage, OCR, and Slack tutorial * Completed requested changes + Lint fixes (F-strings removed) * Minor fix * Updated the tests * Minor fixes * Use newer version of flask (1.0.2 instead of 0.12.2) * Minor fix
1 parent c5635d1 commit 5c63e3c
Copy full SHA for 5c63e3c
Expand file treeCollapse file tree

18 files changed

+552
-4
lines changed

‎.gitignore

Copy file name to clipboardExpand all lines: .gitignore
+2-1Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,4 +21,5 @@ junit.xml
2121
credentials.dat
2222
.nox
2323
.vscode/
24-
*sponge_log.xml
24+
*sponge_log.xml
25+
.DS_store

‎functions/gcs/README.md

Copy file name to clipboard
+11Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
<img src="https://avatars2.githubusercontent.com/u/2810941?v=3&s=96" alt="Google Cloud Platform logo" title="Google Cloud Platform" align="right" height="96" width="96"/>
2+
3+
# Google Cloud Functions - Cloud Storage sample
4+
5+
See:
6+
7+
* [Cloud Functions Cloud Storage tutorial][tutorial]
8+
* [Cloud Functions Cloud Storage source code][code]
9+
10+
[tutorial]: https://cloud.google.com/functions/docs/tutorials/storage
11+
[code]: main.py

‎functions/gcs/main.py

Copy file name to clipboard
+34Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
# Copyright 2018, Google, LLC.
2+
# Licensed under the Apache License, Version 2.0 (the "License");
3+
# you may not use this file except in compliance with the License.
4+
# You may obtain a copy of the License at
5+
#
6+
# http://www.apache.org/licenses/LICENSE-2.0
7+
#
8+
# Unless required by applicable law or agreed to in writing, software
9+
# distributed under the License is distributed on an "AS IS" BASIS,
10+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
11+
# See the License for the specific language governing permissions and
12+
# limitations under the License.
13+
14+
15+
# [START functions_helloworld_storage_generic]
16+
def hello_gcs_generic(data, context):
17+
"""Background Cloud Function to be triggered by Cloud Storage.
18+
This generic function logs relevant data when a file is changed.
19+
20+
Args:
21+
data (dict): The Cloud Functions event payload.
22+
context (google.cloud.functions.Context): Metadata of triggering event.
23+
Returns:
24+
None; the output is written to Stackdriver Logging
25+
"""
26+
27+
print('Event ID: {}'.format(context.event_id))
28+
print('Event type: {}'.format(context.event_type))
29+
print('Bucket: {}'.format(data['bucket']))
30+
print('File: {}'.format(data['name']))
31+
print('Metageneration: {}'.format(data['metageneration']))
32+
print('Created: {}'.format(data['timeCreated']))
33+
print('Updated: {}'.format(data['updated']))
34+
# [END functions_helloworld_storage_generic]

‎functions/gcs/main_test.py

Copy file name to clipboard
+37Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
# Copyright 2018, Google, LLC.
2+
# Licensed under the Apache License, Version 2.0 (the "License");
3+
# you may not use this file except in compliance with the License.
4+
# You may obtain a copy of the License at
5+
#
6+
# http://www.apache.org/licenses/LICENSE-2.0
7+
#
8+
# Unless required by applicable law or agreed to in writing, software
9+
# distributed under the License is distributed on an "AS IS" BASIS,
10+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
11+
# See the License for the specific language governing permissions and
12+
# limitations under the License.
13+
14+
import mock
15+
16+
import main
17+
18+
19+
class TestGCFPyGCSSample(object):
20+
def test_hello_gcs_generic(self, capsys):
21+
event = {
22+
'bucket': 'some-bucket',
23+
'name': 'some-filename',
24+
'metageneration': 'some-metageneration',
25+
'timeCreated': '0',
26+
'updated': '0'
27+
}
28+
context = mock.MagicMock()
29+
context.event_id = 'some-id'
30+
context.event_type = 'gcs-event'
31+
32+
main.hello_gcs_generic(event, context)
33+
34+
out, _ = capsys.readouterr()
35+
36+
assert 'some-bucket' in out
37+
assert 'some-id' in out

‎functions/helloworld/sample_pubsub_test_system.py

Copy file name to clipboardExpand all lines: functions/helloworld/sample_pubsub_test_system.py
-2Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,5 @@ def test_print_name(publisher_client):
5858
start_time
5959
], stdout=subprocess.PIPE)
6060
logs = str(log_process.communicate()[0])
61-
print logs
62-
print start_time
6361
assert 'Hello, {}!'.format(name) in logs
6462
# [END functions_pubsub_system_test]

‎functions/ocr/README.md

Copy file name to clipboard
+11Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
<img src="https://avatars2.githubusercontent.com/u/2810941?v=3&s=96" alt="Google Cloud Platform logo" title="Google Cloud Platform" align="right" height="96" width="96"/>
2+
3+
# Google Cloud Functions - OCR (Optical Character Recognition) sample
4+
5+
See:
6+
7+
* [Cloud Functions OCR tutorial][tutorial]
8+
* [Cloud Functions OCR sample source code][code]
9+
10+
[tutorial]: https://cloud.google.com/functions/docs/tutorials/ocr
11+
[code]: app/main.py

‎functions/ocr/app/config.json

Copy file name to clipboard
+7Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
{
2+
"RESULT_TOPIC": "[YOUR_RESULT_TOPIC_NAME]",
3+
"RESULT_BUCKET": "[YOUR_TEXT_BUCKET_NAME]",
4+
"TRANSLATE_TOPIC": "[YOUR_TRANSLATE_TOPIC_NAME]",
5+
"TRANSLATE": true,
6+
"TO_LANG": ["en", "fr", "es", "ja", "ru"]
7+
}

‎functions/ocr/app/main.py

Copy file name to clipboard
+161Lines changed: 161 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,161 @@
1+
# Copyright 2018, Google, LLC.
2+
# Licensed under the Apache License, Version 2.0 (the "License");
3+
# you may not use this file except in compliance with the License.
4+
# You may obtain a copy of the License at
5+
#
6+
# http://www.apache.org/licenses/LICENSE-2.0
7+
#
8+
# Unless required by applicable law or agreed to in writing, software
9+
# distributed under the License is distributed on an "AS IS" BASIS,
10+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
11+
# See the License for the specific language governing permissions and
12+
# limitations under the License.
13+
14+
# [START functions_ocr_setup]
15+
import base64
16+
import json
17+
import os
18+
19+
from google.cloud import pubsub_v1
20+
from google.cloud import storage
21+
from google.cloud import translate
22+
from google.cloud import vision
23+
24+
vision_client = vision.ImageAnnotatorClient()
25+
translate_client = translate.Client()
26+
publisher = pubsub_v1.PublisherClient()
27+
storage_client = storage.Client()
28+
29+
project_id = os.environ['GCLOUD_PROJECT']
30+
31+
with open('config.json') as f:
32+
data = f.read()
33+
config = json.loads(data)
34+
# [END functions_ocr_setup]
35+
36+
37+
# [START functions_ocr_detect]
38+
def detect_text(bucket, filename):
39+
print('Looking for text in image {}'.format(filename))
40+
41+
futures = []
42+
43+
text_detection_response = vision_client.text_detection({
44+
'source': {'image_uri': 'gs://{}/{}'.format(bucket, filename)}
45+
})
46+
annotations = text_detection_response.text_annotations
47+
if len(annotations) > 0:
48+
text = annotations[0].description
49+
else:
50+
text = ''
51+
print('Extracted text {} from image ({} chars).'.format(text, len(text)))
52+
53+
detect_language_response = translate_client.detect_language(text)
54+
src_lang = detect_language_response['language']
55+
print('Detected language {} for text {}.'.format(src_lang, text))
56+
57+
# Submit a message to the bus for each target language
58+
for target_lang in config.get('TO_LANG', []):
59+
topic_name = config['TRANSLATE_TOPIC']
60+
if src_lang == target_lang:
61+
topic_name = config['RESULT_TOPIC']
62+
message = {
63+
'text': text,
64+
'filename': filename,
65+
'target_lang': target_lang,
66+
'src_lang': src_lang
67+
}
68+
message_data = json.dumps(message).encode('utf-8')
69+
topic_path = publisher.topic_path(project_id, topic_name)
70+
future = publisher.publish(topic_path, data=message_data)
71+
futures.append(future)
72+
for future in futures:
73+
future.result()
74+
# [END functions_ocr_detect]
75+
76+
77+
# [START message_validatation_helper]
78+
def validate_message(message, param):
79+
var = message.get(param)
80+
if not var:
81+
raise ValueError('{} is not provided. Make sure you have \
82+
property {} in the request'.format(param, param))
83+
return var
84+
# [END message_validatation_helper]
85+
86+
87+
# [START functions_ocr_process]
88+
def process_image(file, context):
89+
"""Cloud Function triggered by Cloud Storage when a file is changed.
90+
Args:
91+
file (dict): Metadata of the changed file, provided by the triggering
92+
Cloud Storage event.
93+
context (google.cloud.functions.Context): Metadata of triggering event.
94+
Returns:
95+
None; the output is written to stdout and Stackdriver Logging
96+
"""
97+
bucket = validate_message(file, 'bucket')
98+
name = validate_message(file, 'name')
99+
100+
detect_text(bucket, name)
101+
102+
print('File {} processed.'.format(file['name']))
103+
# [END functions_ocr_process]
104+
105+
106+
# [START functions_ocr_translate]
107+
def translate_text(event, context):
108+
if event.get('data'):
109+
message_data = base64.b64decode(event['data']).decode('utf-8')
110+
message = json.loads(message_data)
111+
else:
112+
raise ValueError('Data sector is missing in the Pub/Sub message.')
113+
114+
text = validate_message(message, 'text')
115+
filename = validate_message(message, 'filename')
116+
target_lang = validate_message(message, 'target_lang')
117+
src_lang = validate_message(message, 'src_lang')
118+
119+
print('Translating text into {}.'.format(target_lang))
120+
translated_text = translate_client.translate(text,
121+
target_language=target_lang,
122+
source_language=src_lang)
123+
topic_name = config['RESULT_TOPIC']
124+
message = {
125+
'text': translated_text['translatedText'],
126+
'filename': filename,
127+
'lang': target_lang,
128+
}
129+
message_data = json.dumps(message).encode('utf-8')
130+
topic_path = publisher.topic_path(project_id, topic_name)
131+
future = publisher.publish(topic_path, data=message_data)
132+
future.result()
133+
# [END functions_ocr_translate]
134+
135+
136+
# [START functions_ocr_save]
137+
def save_result(event, context):
138+
if event.get('data'):
139+
message_data = base64.b64decode(event['data']).decode('utf-8')
140+
message = json.loads(message_data)
141+
else:
142+
raise ValueError('Data sector is missing in the Pub/Sub message.')
143+
144+
text = validate_message(message, 'text')
145+
filename = validate_message(message, 'filename')
146+
lang = validate_message(message, 'lang')
147+
148+
print('Received request to save file {}.'.format(filename))
149+
150+
bucket_name = config['RESULT_BUCKET']
151+
result_filename = '{}_{}.txt'.format(filename, lang)
152+
bucket = storage_client.get_bucket(bucket_name)
153+
blob = bucket.blob(result_filename)
154+
155+
print('Saving result to {} in bucket {}.'.format(result_filename,
156+
bucket_name))
157+
158+
blob.upload_from_string(text)
159+
160+
print('File saved.')
161+
# [END functions_ocr_save]

‎functions/ocr/app/main_test.py

Copy file name to clipboard
+89Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
# Copyright 2018, Google, LLC.
2+
# Licensed under the Apache License, Version 2.0 (the "License");
3+
# you may not use this file except in compliance with the License.
4+
# You may obtain a copy of the License at
5+
#
6+
# http://www.apache.org/licenses/LICENSE-2.0
7+
#
8+
# Unless required by applicable law or agreed to in writing, software
9+
# distributed under the License is distributed on an "AS IS" BASIS,
10+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
11+
# See the License for the specific language governing permissions and
12+
# limitations under the License.
13+
14+
import base64
15+
import concurrent.futures
16+
import json
17+
18+
import mock
19+
20+
import main
21+
22+
23+
class TestGCFPyOCRSample():
24+
@mock.patch.object(main, 'publisher')
25+
@mock.patch.object(main, 'translate_client')
26+
@mock.patch.object(main, 'vision_client')
27+
def test_detect_text(self, mock_vision_client, mock_translate_client,
28+
mock_publisher):
29+
mock_annotation = mock.MagicMock()
30+
mock_annotation.description = 'sample text'
31+
mock_annotations = mock.MagicMock()
32+
mock_annotations.text_annotations = [mock_annotation]
33+
mock_vision_client.text_detection.return_value = mock_annotations
34+
35+
mock_translate_client.detect_language.return_value = {'language': 'en'}
36+
37+
mock_future = concurrent.futures.Future()
38+
mock_future.set_result(True)
39+
mock_publisher.publish.return_value = mock_future
40+
41+
main.detect_text('sample-bucket', 'sample-file')
42+
43+
@mock.patch.object(main, 'detect_text')
44+
def test_process_image(self, m):
45+
m.return_value = None
46+
event = {
47+
'bucket': 'sample-bucket',
48+
'name': 'sample-file'
49+
}
50+
context = {}
51+
main.process_image(event, context)
52+
53+
@mock.patch.object(main, 'publisher')
54+
@mock.patch.object(main, 'translate_client')
55+
def test_translate_text(self, mock_translate_client, mock_publisher):
56+
mock_translate_client.translate.return_value = {'translatedText': ''}
57+
58+
mock_future = concurrent.futures.Future()
59+
mock_future.set_result(True)
60+
mock_publisher.publish.return_value = mock_future
61+
62+
data = base64.b64encode(json.dumps({
63+
'text': 'menu',
64+
'filename': 'sample-file',
65+
'target_lang': 'es',
66+
'src_lang': 'en'
67+
}).encode('utf-8'))
68+
event = {
69+
'data': data
70+
}
71+
context = {}
72+
main.translate_text(event, context)
73+
74+
@mock.patch.object(main, 'storage_client')
75+
def test_save_result(self, m):
76+
bucket = m.bucket.return_value
77+
file = bucket.file.return_value
78+
file.save.return_value = None
79+
80+
data = base64.b64encode(json.dumps({
81+
'text': 'menu',
82+
'filename': 'sample-file',
83+
'lang': 'fr',
84+
}).encode('utf-8'))
85+
event = {
86+
'data': data
87+
}
88+
context = {}
89+
main.save_result(event, context)

‎functions/ocr/app/requirements.txt

Copy file name to clipboard
+4Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
google-cloud-pubsub==0.35.4
2+
google-cloud-storage==1.10.0
3+
google-cloud-translate==1.3.1
4+
google-cloud-vision==0.32.0

‎functions/ocr/images/menu.jpg

Copy file name to clipboard
208 KB
Loading

‎functions/ocr/images/sign.png

Copy file name to clipboard
39.9 KB
Loading

‎functions/slack/README.md

Copy file name to clipboard
+11Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
<img src="https://avatars2.githubusercontent.com/u/2810941?v=3&s=96" alt="Google Cloud Platform logo" title="Google Cloud Platform" align="right" height="96" width="96"/>
2+
3+
# Google Cloud Functions - Slack Slash Command sample
4+
5+
See:
6+
7+
* [Cloud Functions Slack tutorial][tutorial]
8+
* [Cloud Functions Slack sample source code][code]
9+
10+
[tutorial]: https://cloud.google.com/functions/docs/tutorials/slack
11+
[code]: main.py

‎functions/slack/config.json

Copy file name to clipboard
+4Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
{
2+
"SLACK_TOKEN": "YOUR_SLACK_TOKEN",
3+
"KG_API_KEY": "YOUR_KG_API_KEY"
4+
}

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.