Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

fix: make read_gbq_function work for multi-param functions #947

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Sep 6, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions 8 bigframes/functions/remote_function.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@

from __future__ import annotations

import inspect
import logging
from typing import cast, Optional, TYPE_CHECKING
import warnings
Expand Down Expand Up @@ -149,6 +150,13 @@ def func(*ignored_args, **ignored_kwargs):
expr = node(*ignored_args, **ignored_kwargs) # type: ignore
return ibis_client.execute(expr)

func.__signature__ = inspect.signature(func).replace( # type: ignore
parameters=[
inspect.Parameter(name, inspect.Parameter.POSITIONAL_OR_KEYWORD)
for name in ibis_signature.parameter_names
]
)

# TODO: Move ibis logic to compiler step

func.__name__ = routine_ref.routine_id
Expand Down
27 changes: 22 additions & 5 deletions 27 bigframes/session/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -1239,12 +1239,22 @@ def read_gbq_function(

**Examples:**

Use the ``cw_lower_case_ascii_only`` function from Community UDFs.
(https://github.com/GoogleCloudPlatform/bigquery-utils/blob/master/udfs/community/cw_lower_case_ascii_only.sqlx)

>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None

Use the [cw_lower_case_ascii_only](https://github.com/GoogleCloudPlatform/bigquery-utils/blob/master/udfs/community/README.md#cw_lower_case_ascii_onlystr-string)
function from Community UDFs.

>>> func = bpd.read_gbq_function("bqutil.fn.cw_lower_case_ascii_only")

You can run it on scalar input. Usually you would do so to verify that
it works as expected before applying to all values in a Series.

>>> func('AURÉLIE')
'aurÉlie'

You can apply it to a BigQuery DataFrame Series.

>>> df = bpd.DataFrame({'id': [1, 2, 3], 'name': ['AURÉLIE', 'CÉLESTINE', 'DAPHNÉ']})
>>> df
id name
Expand All @@ -1254,7 +1264,6 @@ def read_gbq_function(
<BLANKLINE>
[3 rows x 2 columns]

>>> func = bpd.read_gbq_function("bqutil.fn.cw_lower_case_ascii_only")
>>> df1 = df.assign(new_name=df['name'].apply(func))
>>> df1
id name new_name
Expand All @@ -1264,9 +1273,17 @@ def read_gbq_function(
<BLANKLINE>
[3 rows x 3 columns]

You can even use a function with multiple inputs. For example, let's use
[cw_instr4](https://github.com/GoogleCloudPlatform/bigquery-utils/blob/master/udfs/community/README.md#cw_instr4source-string-search-string-position-int64-ocurrence-int64)
from Community UDFs.

>>> func = bpd.read_gbq_function("bqutil.fn.cw_instr4")
>>> func('TestStr123456Str', 'Str', 1, 2)
14

Args:
function_name (str):
the function's name in BigQuery in the format
The function's name in BigQuery in the format
`project_id.dataset_id.function_name`, or
`dataset_id.function_name` to load from the default project, or
`function_name` to load from the default project and the dataset
Expand Down
9 changes: 8 additions & 1 deletion 9 tests/system/small/test_remote_function.py
Original file line number Diff line number Diff line change
Expand Up @@ -671,12 +671,19 @@ def square1(x):


@pytest.mark.flaky(retries=2, delay=120)
def test_read_gbq_function_runs_existing_udf(session, bigquery_client, dataset_id):
def test_read_gbq_function_runs_existing_udf(session):
func = session.read_gbq_function("bqutil.fn.cw_lower_case_ascii_only")
got = func("AURÉLIE")
assert got == "aurÉlie"


@pytest.mark.flaky(retries=2, delay=120)
def test_read_gbq_function_runs_existing_udf_4_params(session):
func = session.read_gbq_function("bqutil.fn.cw_instr4")
got = func("TestStr123456Str", "Str", 1, 2)
assert got == 14


@pytest.mark.flaky(retries=2, delay=120)
def test_read_gbq_function_reads_udfs(session, bigquery_client, dataset_id):
dataset_ref = bigquery.DatasetReference.from_string(dataset_id)
Expand Down
Morty Proxy This is a proxified and sanitized view of the page, visit original site.