Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit ce25e94

Browse filesBrowse files
arithmetic1728arithmetic1728
authored
feat!: migrate to use microgen (#71)
* feat!: migrate to use microgen * update * update * update * update
1 parent f6eb330 commit ce25e94
Copy full SHA for ce25e94

File tree

Expand file treeCollapse file tree

196 files changed

+48837
-47388
lines changed
Filter options

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Dismiss banner
Expand file treeCollapse file tree

196 files changed

+48837
-47388
lines changed

‎packages/google-cloud-dataproc/.coveragerc

Copy file name to clipboardExpand all lines: packages/google-cloud-dataproc/.coveragerc
+6-7Lines changed: 6 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -21,15 +21,14 @@ branch = True
2121
[report]
2222
fail_under = 100
2323
show_missing = True
24+
omit = google/cloud/dataproc/__init__.py
2425
exclude_lines =
2526
# Re-enable the standard pragma
2627
pragma: NO COVER
2728
# Ignore debug-only repr
2829
def __repr__
29-
# Ignore abstract methods
30-
raise NotImplementedError
31-
omit =
32-
*/gapic/*.py
33-
*/proto/*.py
34-
*/core/*.py
35-
*/site-packages/*.py
30+
# Ignore pkg_resources exceptions.
31+
# This is added at the module level as a safeguard for if someone
32+
# generates the code and tries to run it without pip installing. This
33+
# makes it virtually impossible to test properly.
34+
except pkg_resources.DistributionNotFound

‎packages/google-cloud-dataproc/README.rst

Copy file name to clipboardExpand all lines: packages/google-cloud-dataproc/README.rst
+6-4Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
Python Client for Google Cloud Dataproc API
22
===========================================
33

4-
|ga| |pypi| |versions|
4+
|ga| |pypi| |versions|
55

66
`Google Cloud Dataproc API`_: Manages Hadoop-based clusters and jobs on Google Cloud Platform.
77

@@ -49,11 +49,13 @@ dependencies.
4949

5050
Supported Python Versions
5151
^^^^^^^^^^^^^^^^^^^^^^^^^
52-
Python >= 3.5
52+
Python >= 3.6
5353

5454
Deprecated Python Versions
5555
^^^^^^^^^^^^^^^^^^^^^^^^^^
56-
Python == 2.7. Python 2.7 support will be removed on January 1, 2020.
56+
Python == 2.7.
57+
58+
The last version of this library compatible with Python 2.7 is google-cloud-dataproc==1.1.0.
5759

5860

5961
Mac/Linux
@@ -107,4 +109,4 @@ Next Steps
107109
- Read the `Client Library Documentation`_ for Google Cloud Dataproc API
108110
API to see other available methods on the client.
109111
- Read the `Product documentation`_ to learn more about the product and see
110-
How-to Guides.
112+
How-to Guides.

‎packages/google-cloud-dataproc/UPGRADING.md

Copy file name to clipboard
+168Lines changed: 168 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,168 @@
1+
# 2.0.0 Migration Guide
2+
3+
The 2.0 release of the `google-cloud-dataproc` client is a significant upgrade based on a [next-gen code generator](https://github.com/googleapis/gapic-generator-python), and includes substantial interface changes. Existing code written for earlier versions of this library will likely require updates to use this version. This document describes the changes that have been made, and what you need to do to update your usage.
4+
5+
If you experience issues or have questions, please file an [issue](https://github.com/googleapis/python-dataproc/issues).
6+
7+
## Supported Python Versions
8+
9+
> **WARNING**: Breaking change
10+
11+
The 2.0.0 release requires Python 3.6+.
12+
13+
14+
## Method Calls
15+
16+
> **WARNING**: Breaking change
17+
18+
Methods expect request objects. We provide a script that will convert most common use cases.
19+
20+
* Install the library
21+
22+
```py
23+
python3 -m pip install google-cloud-dataproc
24+
```
25+
26+
* The script `fixup_dataproc_v1_keywords.py` is shipped with the library. It expects an input directory (with the code to convert) and an empty destination directory.
27+
28+
```sh
29+
$ fixup_dataproc_v1_keywords.py --input-directory .samples/ --output-directory samples/
30+
```
31+
32+
**Before:**
33+
```py
34+
from google.cloud import dataproc
35+
36+
client = dataproc.ClusterControllerClient()
37+
38+
clusters = client.list_clusters(project_id="project_id", region="region")
39+
```
40+
41+
42+
**After:**
43+
```py
44+
from google.cloud import dataproc
45+
46+
client = dataproc.ClusterControllerClient()
47+
48+
clusters = client.list_clusters(request={
49+
'project_id' : "project_id", 'region' : "region"
50+
})
51+
```
52+
53+
### More Details
54+
55+
In `google-cloud-dataproc<2.0.0`, parameters required by the API were positional parameters and optional parameters were keyword parameters.
56+
57+
**Before:**
58+
```py
59+
def get_cluster(
60+
self,
61+
project_id,
62+
region,
63+
cluster_name,
64+
retry=google.api_core.gapic_v1.method.DEFAULT,
65+
timeout=google.api_core.gapic_v1.method.DEFAULT,
66+
metadata=None,
67+
):
68+
```
69+
70+
In the 2.0.0 release, all methods have a single positional parameter `request`. Method docstrings indicate whether a parameter is required or optional.
71+
72+
Some methods have additional keyword only parameters. The available parameters depend on the [`google.api.method_signature` annotation](https://github.com/googleapis/googleapis/blob/master/google/cloud/dataproc/v1/clusters.proto#L88) specified by the API producer.
73+
74+
75+
**After:**
76+
```py
77+
def get_cluster(
78+
self,
79+
request: clusters.GetClusterRequest = None,
80+
*,
81+
project_id: str = None,
82+
region: str = None,
83+
cluster_name: str = None,
84+
retry: retries.Retry = gapic_v1.method.DEFAULT,
85+
timeout: float = None,
86+
metadata: Sequence[Tuple[str, str]] = (),
87+
) -> clusters.Cluster:
88+
```
89+
90+
> **NOTE:** The `request` parameter and flattened keyword parameters for the API are mutually exclusive.
91+
> Passing both will result in an error.
92+
93+
94+
Both of these calls are valid:
95+
96+
```py
97+
response = client.get_cluster(
98+
request={
99+
"project_id": project_id,
100+
"region": region,
101+
"cluster_name": cluster_name
102+
}
103+
)
104+
```
105+
106+
```py
107+
response = client.get_cluster(
108+
project_id=project_id,
109+
region=region,
110+
cluster_name=cluster_name
111+
)
112+
```
113+
114+
This call is invalid because it mixes `request` with a keyword argument `cluster_name`. Executing this code
115+
will result in an error.
116+
117+
```py
118+
response = client.get_cluster(
119+
request={
120+
"project_id": project_id,
121+
"region": region
122+
},
123+
cluster_name=cluster_name
124+
)
125+
```
126+
127+
128+
129+
## Enums and Types
130+
131+
132+
> **WARNING**: Breaking change
133+
134+
The submodules `enums` and `types` have been removed.
135+
136+
**Before:**
137+
```py
138+
139+
from google.cloud import dataproc
140+
141+
status = dataproc.enums.ClusterStatus.State.CREATING
142+
cluster = dataproc.types.Cluster(cluster_name="name")
143+
```
144+
145+
146+
**After:**
147+
```py
148+
from google.cloud import dataproc
149+
150+
status = dataproc.ClusterStatus.State.CREATING
151+
cluster = dataproc.Cluster(cluster_name="name")
152+
```
153+
154+
## Path Helper Methods
155+
The following path helper methods have been removed. Please construct the paths manually.
156+
157+
```py
158+
project = 'my-project'
159+
location = 'project-location'
160+
region = 'project-region'
161+
workflow_template = 'template'
162+
autoscaling_policy = 'policy'
163+
164+
location_path = f'projects/{project}/locations/{location}'
165+
region_path = f'projects/{project}/regions/{region}'
166+
workflow_template_path = f'projects/{project}/regions/{region}/workflowTemplates/{workflow_template}'
167+
autoscaling_policy_path = f'projects/{project}/locations/{location}/autoscalingPolicies/{autoscaling_policy}'
168+
```

‎packages/google-cloud-dataproc/docs/UPGRADING.md

Copy file name to clipboard
+1Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
../UPGRADING.md

‎packages/google-cloud-dataproc/docs/dataproc_v1/services.rst

Copy file name to clipboard
+15Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
Services for Google Cloud Dataproc v1 API
2+
=========================================
3+
4+
.. automodule:: google.cloud.dataproc_v1.services.autoscaling_policy_service
5+
:members:
6+
:inherited-members:
7+
.. automodule:: google.cloud.dataproc_v1.services.cluster_controller
8+
:members:
9+
:inherited-members:
10+
.. automodule:: google.cloud.dataproc_v1.services.job_controller
11+
:members:
12+
:inherited-members:
13+
.. automodule:: google.cloud.dataproc_v1.services.workflow_template_service
14+
:members:
15+
:inherited-members:

‎packages/google-cloud-dataproc/docs/dataproc_v1/types.rst

Copy file name to clipboard
+5Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
Types for Google Cloud Dataproc v1 API
2+
======================================
3+
4+
.. automodule:: google.cloud.dataproc_v1.types
5+
:members:

‎packages/google-cloud-dataproc/docs/dataproc_v1beta2/services.rst

Copy file name to clipboard
+15Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
Services for Google Cloud Dataproc v1beta2 API
2+
==============================================
3+
4+
.. automodule:: google.cloud.dataproc_v1beta2.services.autoscaling_policy_service
5+
:members:
6+
:inherited-members:
7+
.. automodule:: google.cloud.dataproc_v1beta2.services.cluster_controller
8+
:members:
9+
:inherited-members:
10+
.. automodule:: google.cloud.dataproc_v1beta2.services.job_controller
11+
:members:
12+
:inherited-members:
13+
.. automodule:: google.cloud.dataproc_v1beta2.services.workflow_template_service
14+
:members:
15+
:inherited-members:

‎packages/google-cloud-dataproc/docs/dataproc_v1beta2/types.rst

Copy file name to clipboard
+5Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
Types for Google Cloud Dataproc v1beta2 API
2+
===========================================
3+
4+
.. automodule:: google.cloud.dataproc_v1beta2.types
5+
:members:

‎packages/google-cloud-dataproc/docs/gapic/v1/api.rst

Copy file name to clipboardExpand all lines: packages/google-cloud-dataproc/docs/gapic/v1/api.rst
-6Lines changed: 0 additions & 6 deletions
This file was deleted.

‎packages/google-cloud-dataproc/docs/gapic/v1/types.rst

Copy file name to clipboardExpand all lines: packages/google-cloud-dataproc/docs/gapic/v1/types.rst
-5Lines changed: 0 additions & 5 deletions
This file was deleted.

‎packages/google-cloud-dataproc/docs/gapic/v1beta2/api.rst

Copy file name to clipboardExpand all lines: packages/google-cloud-dataproc/docs/gapic/v1beta2/api.rst
-6Lines changed: 0 additions & 6 deletions
This file was deleted.

‎packages/google-cloud-dataproc/docs/gapic/v1beta2/types.rst

Copy file name to clipboardExpand all lines: packages/google-cloud-dataproc/docs/gapic/v1beta2/types.rst
-5Lines changed: 0 additions & 5 deletions
This file was deleted.

‎packages/google-cloud-dataproc/docs/index.rst

Copy file name to clipboardExpand all lines: packages/google-cloud-dataproc/docs/index.rst
+14-5Lines changed: 14 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -7,11 +7,20 @@ API Reference
77
.. toctree::
88
:maxdepth: 2
99

10-
gapic/v1/api
11-
gapic/v1/types
12-
gapic/v1beta2/api
13-
gapic/v1beta2/types
10+
dataproc_v1/services
11+
dataproc_v1/types
12+
dataproc_v1beta2/services
13+
dataproc_v1beta2/types
1414

15+
Migration Guide
16+
---------------
17+
18+
See the guide below for instructions on migrating to the 2.x release of this library.
19+
20+
.. toctree::
21+
:maxdepth: 2
22+
23+
UPGRADING
1524

1625
Changelog
1726
---------
@@ -21,4 +30,4 @@ For a list of all ``google-cloud-dataproc`` releases:
2130
.. toctree::
2231
:maxdepth: 2
2332

24-
changelog
33+
changelog

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.