Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings
This repository was archived by the owner on Mar 6, 2026. It is now read-only.

Commit fa76e31

Browse filesBrowse files
google-labs-jules[bot]chalmerloweLinchin
authored
feat: Add ExternalRuntimeOptions to BigQuery routine (#2311)
* feat: Add ExternalRuntimeOptions to BigQuery routine This change introduces the `ExternalRuntimeOptions` class to the `google.cloud.bigquery.routine` module, allowing users to configure runtime options for external routines. Key changes: - Created the `ExternalRuntimeOptions` class with setters and getters for `container_memory`, `container_cpu`, `runtime_connection`, `max_batching_rows`, and `runtime_version`. - Updated the `Routine` class to include an `external_runtime_options` property that accepts an `ExternalRuntimeOptions` object. - Added comprehensive unit tests for the new class and its integration with the `Routine` class, including tests for both valid and invalid input values. * Update google/cloud/bigquery/routine/routine.py * feat: Add ExternalRuntimeOptions to BigQuery routine This change introduces the `ExternalRuntimeOptions` class to the `google.cloud.bigquery.routine` module, allowing users to configure runtime options for external routines. Key changes: - Created the `ExternalRuntimeOptions` class with setters and getters for `container_memory`, `container_cpu`, `runtime_connection`, `max_batching_rows`, and `runtime_version`. - Updated the `Routine` class to include an `external_runtime_options` property that accepts an `ExternalRuntimeOptions` object. - Added comprehensive unit tests for the new class and its integration with the `Routine` class, including tests for both valid and invalid input values. * feat: Add ExternalRuntimeOptions to BigQuery routine This change introduces the `ExternalRuntimeOptions` class to the `google.cloud.bigquery.routine` module, allowing users to configure runtime options for external routines. Key changes: - Created the `ExternalRuntimeOptions` class with setters and getters for `container_memory`, `container_cpu`, `runtime_connection`, `max_batching_rows`, and `runtime_version`. - Updated the `Routine` class to include an `external_runtime_options` property that accepts an `ExternalRuntimeOptions` object. - Added comprehensive unit tests for the new class and its integration with the `Routine` class, including tests for both valid and invalid input values. - Added additional tests to improve code coverage based on feedback. * feat: Add ExternalRuntimeOptions to BigQuery routine This change introduces the `ExternalRuntimeOptions` class to the `google.cloud.bigquery.routine` module, allowing users to configure runtime options for external routines. Key changes: - Created the `ExternalRuntimeOptions` class with setters and getters for `container_memory`, `container_cpu`, `runtime_connection`, `max_batching_rows`, and `runtime_version`. - Updated the `Routine` class to include an `external_runtime_options` property that accepts an `ExternalRuntimeOptions` object. - Added comprehensive unit tests for the new class and its integration with the `Routine` class, including tests for both valid and invalid input values. - Added additional tests to improve code coverage based on feedback. - Addressed PyType errors by using helper functions for type conversion. * Update tests/unit/routine/test_external_runtime_options.py * feat: Add ExternalRuntimeOptions to BigQuery routine This change introduces the `ExternalRuntimeOptions` class to the `google.cloud.bigquery.routine` module, allowing users to configure runtime options for external routines. Key changes: - Created the `ExternalRuntimeOptions` class with setters and getters for `container_memory`, `container_cpu`, `runtime_connection`, `max_batching_rows`, and `runtime_version`. - Updated the `Routine` class to include an `external_runtime_options` property that accepts an `ExternalRuntimeOptions` object. - Added comprehensive unit tests for the new class and its integration with the `Routine` class, including tests for both valid and invalid input values. - Added additional tests to improve code coverage based on feedback. - Addressed PyType errors by using helper functions for type conversion. - Addressed formatting nits from code review. --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: Chalmer Lowe <chalmerlowe@google.com> Co-authored-by: Lingqing Gan <lingqing.gan@gmail.com>
1 parent 7fbd8c2 commit fa76e31
Copy full SHA for fa76e31

5 files changed

+421-1Lines changed: 421 additions & 1 deletion

File tree

Expand file treeCollapse file tree
Open diff view settings
Filter options
Expand file treeCollapse file tree
Open diff view settings
Collapse file

‎google/cloud/bigquery/__init__.py‎

Copy file name to clipboardExpand all lines: google/cloud/bigquery/__init__.py
+2Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -98,6 +98,7 @@
9898
from google.cloud.bigquery.routine import RoutineReference
9999
from google.cloud.bigquery.routine import RoutineType
100100
from google.cloud.bigquery.routine import RemoteFunctionOptions
101+
from google.cloud.bigquery.routine import ExternalRuntimeOptions
101102
from google.cloud.bigquery.schema import PolicyTagList
102103
from google.cloud.bigquery.schema import SchemaField
103104
from google.cloud.bigquery.schema import FieldElementType
@@ -181,6 +182,7 @@
181182
"RoutineArgument",
182183
"RoutineReference",
183184
"RemoteFunctionOptions",
185+
"ExternalRuntimeOptions",
184186
# Shared helpers
185187
"SchemaField",
186188
"FieldElementType",
Collapse file

‎google/cloud/bigquery/routine/__init__.py‎

Copy file name to clipboardExpand all lines: google/cloud/bigquery/routine/__init__.py
+2Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@
2121
from google.cloud.bigquery.routine.routine import RoutineReference
2222
from google.cloud.bigquery.routine.routine import RoutineType
2323
from google.cloud.bigquery.routine.routine import RemoteFunctionOptions
24+
from google.cloud.bigquery.routine.routine import ExternalRuntimeOptions
2425

2526

2627
__all__ = (
@@ -30,4 +31,5 @@
3031
"RoutineReference",
3132
"RoutineType",
3233
"RemoteFunctionOptions",
34+
"ExternalRuntimeOptions",
3335
)
Collapse file

‎google/cloud/bigquery/routine/routine.py‎

Copy file name to clipboardExpand all lines: google/cloud/bigquery/routine/routine.py
+184-1Lines changed: 184 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@
1515
# limitations under the License.
1616

1717
"""Define resources for the BigQuery Routines API."""
18-
18+
import typing
1919
from typing import Any, Dict, Optional, Union
2020

2121
import google.cloud._helpers # type: ignore
@@ -69,6 +69,7 @@ class Routine(object):
6969
"determinism_level": "determinismLevel",
7070
"remote_function_options": "remoteFunctionOptions",
7171
"data_governance_type": "dataGovernanceType",
72+
"external_runtime_options": "externalRuntimeOptions",
7273
}
7374

7475
def __init__(self, routine_ref, **kwargs) -> None:
@@ -349,6 +350,37 @@ def data_governance_type(self, value):
349350
)
350351
self._properties[self._PROPERTY_TO_API_FIELD["data_governance_type"]] = value
351352

353+
@property
354+
def external_runtime_options(self):
355+
"""Optional[google.cloud.bigquery.routine.ExternalRuntimeOptions]:
356+
Configures the external runtime options for a routine.
357+
358+
Raises:
359+
ValueError:
360+
If the value is not
361+
:class:`~google.cloud.bigquery.routine.ExternalRuntimeOptions` or
362+
:data:`None`.
363+
"""
364+
prop = self._properties.get(
365+
self._PROPERTY_TO_API_FIELD["external_runtime_options"]
366+
)
367+
if prop is not None:
368+
return ExternalRuntimeOptions.from_api_repr(prop)
369+
370+
@external_runtime_options.setter
371+
def external_runtime_options(self, value):
372+
api_repr = value
373+
if isinstance(value, ExternalRuntimeOptions):
374+
api_repr = value.to_api_repr()
375+
elif value is not None:
376+
raise ValueError(
377+
"value must be google.cloud.bigquery.routine.ExternalRuntimeOptions "
378+
"or None"
379+
)
380+
self._properties[
381+
self._PROPERTY_TO_API_FIELD["external_runtime_options"]
382+
] = api_repr
383+
352384
@classmethod
353385
def from_api_repr(cls, resource: dict) -> "Routine":
354386
"""Factory: construct a routine given its API representation.
@@ -736,3 +768,154 @@ def __repr__(self):
736768
for property_name in sorted(self._PROPERTY_TO_API_FIELD)
737769
]
738770
return "RemoteFunctionOptions({})".format(", ".join(all_properties))
771+
772+
773+
class ExternalRuntimeOptions(object):
774+
"""Options for the runtime of the external system.
775+
776+
Args:
777+
container_memory (str):
778+
Optional. Amount of memory provisioned for a Python UDF container
779+
instance. Format: {number}{unit} where unit is one of "M", "G", "Mi"
780+
and "Gi" (e.g. 1G, 512Mi). If not specified, the default value is
781+
512Mi. For more information, see `Configure container limits for
782+
Python UDFs <https://cloud.google.com/bigquery/docs/user-defined-functions-python#configure-container-limits>`_
783+
container_cpu (int):
784+
Optional. Amount of CPU provisioned for a Python UDF container
785+
instance. For more information, see `Configure container limits
786+
for Python UDFs <https://cloud.google.com/bigquery/docs/user-defined-functions-python#configure-container-limits>`_
787+
runtime_connection (str):
788+
Optional. Fully qualified name of the connection whose service account
789+
will be used to execute the code in the container. Format:
790+
"projects/{projectId}/locations/{locationId}/connections/{connectionId}"
791+
max_batching_rows (int):
792+
Optional. Maximum number of rows in each batch sent to the external
793+
runtime. If absent or if 0, BigQuery dynamically decides the number of
794+
rows in a batch.
795+
runtime_version (str):
796+
Optional. Language runtime version. Example: python-3.11.
797+
"""
798+
799+
_PROPERTY_TO_API_FIELD = {
800+
"container_memory": "containerMemory",
801+
"container_cpu": "containerCpu",
802+
"runtime_connection": "runtimeConnection",
803+
"max_batching_rows": "maxBatchingRows",
804+
"runtime_version": "runtimeVersion",
805+
}
806+
807+
def __init__(
808+
self,
809+
container_memory: Optional[str] = None,
810+
container_cpu: Optional[int] = None,
811+
runtime_connection: Optional[str] = None,
812+
max_batching_rows: Optional[int] = None,
813+
runtime_version: Optional[str] = None,
814+
_properties: Optional[Dict] = None,
815+
) -> None:
816+
if _properties is None:
817+
_properties = {}
818+
self._properties = _properties
819+
820+
if container_memory is not None:
821+
self.container_memory = container_memory
822+
if container_cpu is not None:
823+
self.container_cpu = container_cpu
824+
if runtime_connection is not None:
825+
self.runtime_connection = runtime_connection
826+
if max_batching_rows is not None:
827+
self.max_batching_rows = max_batching_rows
828+
if runtime_version is not None:
829+
self.runtime_version = runtime_version
830+
831+
@property
832+
def container_memory(self) -> Optional[str]:
833+
"""Optional. Amount of memory provisioned for a Python UDF container instance."""
834+
return _helpers._str_or_none(self._properties.get("containerMemory"))
835+
836+
@container_memory.setter
837+
def container_memory(self, value: Optional[str]):
838+
if value is not None and not isinstance(value, str):
839+
raise ValueError("container_memory must be a string or None.")
840+
self._properties["containerMemory"] = value
841+
842+
@property
843+
def container_cpu(self) -> Optional[int]:
844+
"""Optional. Amount of CPU provisioned for a Python UDF container instance."""
845+
return _helpers._int_or_none(self._properties.get("containerCpu"))
846+
847+
@container_cpu.setter
848+
def container_cpu(self, value: Optional[int]):
849+
if value is not None and not isinstance(value, int):
850+
raise ValueError("container_cpu must be an integer or None.")
851+
self._properties["containerCpu"] = value
852+
853+
@property
854+
def runtime_connection(self) -> Optional[str]:
855+
"""Optional. Fully qualified name of the connection."""
856+
return _helpers._str_or_none(self._properties.get("runtimeConnection"))
857+
858+
@runtime_connection.setter
859+
def runtime_connection(self, value: Optional[str]):
860+
if value is not None and not isinstance(value, str):
861+
raise ValueError("runtime_connection must be a string or None.")
862+
self._properties["runtimeConnection"] = value
863+
864+
@property
865+
def max_batching_rows(self) -> Optional[int]:
866+
"""Optional. Maximum number of rows in each batch sent to the external runtime."""
867+
return typing.cast(
868+
int, _helpers._int_or_none(self._properties.get("maxBatchingRows"))
869+
)
870+
871+
@max_batching_rows.setter
872+
def max_batching_rows(self, value: Optional[int]):
873+
if value is not None and not isinstance(value, int):
874+
raise ValueError("max_batching_rows must be an integer or None.")
875+
self._properties["maxBatchingRows"] = _helpers._str_or_none(value)
876+
877+
@property
878+
def runtime_version(self) -> Optional[str]:
879+
"""Optional. Language runtime version."""
880+
return _helpers._str_or_none(self._properties.get("runtimeVersion"))
881+
882+
@runtime_version.setter
883+
def runtime_version(self, value: Optional[str]):
884+
if value is not None and not isinstance(value, str):
885+
raise ValueError("runtime_version must be a string or None.")
886+
self._properties["runtimeVersion"] = value
887+
888+
@classmethod
889+
def from_api_repr(cls, resource: dict) -> "ExternalRuntimeOptions":
890+
"""Factory: construct external runtime options given its API representation.
891+
Args:
892+
resource (Dict[str, object]): Resource, as returned from the API.
893+
Returns:
894+
google.cloud.bigquery.routine.ExternalRuntimeOptions:
895+
Python object, as parsed from ``resource``.
896+
"""
897+
ref = cls()
898+
ref._properties = resource
899+
return ref
900+
901+
def to_api_repr(self) -> dict:
902+
"""Construct the API resource representation of this ExternalRuntimeOptions.
903+
Returns:
904+
Dict[str, object]: External runtime options represented as an API resource.
905+
"""
906+
return self._properties
907+
908+
def __eq__(self, other):
909+
if not isinstance(other, ExternalRuntimeOptions):
910+
return NotImplemented
911+
return self._properties == other._properties
912+
913+
def __ne__(self, other):
914+
return not self == other
915+
916+
def __repr__(self):
917+
all_properties = [
918+
"{}={}".format(property_name, repr(getattr(self, property_name)))
919+
for property_name in sorted(self._PROPERTY_TO_API_FIELD)
920+
]
921+
return "ExternalRuntimeOptions({})".format(", ".join(all_properties))

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.