Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

refactor: Extract data loading logic into class #913

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Aug 28, 2024
Merged

Conversation

TrevorBergeron
Copy link
Contributor

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes #<issue_number_goes_here> 🦕

@TrevorBergeron TrevorBergeron requested review from a team as code owners August 22, 2024 17:25
@TrevorBergeron TrevorBergeron requested a review from arwas11 August 22, 2024 17:25
@product-auto-label product-auto-label bot added size: xl Pull request size is extra large. api: bigquery Issues related to the googleapis/python-bigquery-dataframes API. labels Aug 22, 2024
@arwas11 arwas11 removed their request for review August 22, 2024 17:28
Copy link
Contributor

@GarrettWu GarrettWu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What a large PR. Do we have a bug number or any other contexts? What are we trying to solve here?

@TrevorBergeron
Copy link
Contributor Author

What a large PR. Do we have a bug number or any other contexts? What are we trying to solve here?

I do not have a bug number. Here is a previous commit in this effort: 92fdb93.

The goal has been to make Session easier to understand by splitting it's state and functionality into distinct components. Mostly just moving code to new files right now. The hope is by creating some smaller components, it will be easier to optimize each component. Additionally, I do want to try making session thread-safe at some point, and pushing session state into individual objects will help with setting up necessary locks.

scan_index_uniqueness: bool,
metrics: Optional[bigframes.session.metrics.ExecutionMetrics] = None,
):
self.bqclient = bqclient
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some members are "private" while others are public. Are they really different or they can be consistent?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

made them all private

bqclient: bigquery.Client,
storage_manager: bigframes.session.temp_storage.TemporaryGbqStorageManager,
default_index_type: bigframes.enums.DefaultIndexKind,
scan_index_uniqueness: bool,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add docs for the args? At least scan_index_uniqueness and metrics aren't clear at first glance.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added

array_value: bigframes.core.ArrayValue,
col_id_overrides: Mapping[str, str],
uri: str,
format: Literal["JSON", "CSV", "PARQUET"],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Python usually takes lower cases

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed to lower case

col_id_overrides: Mapping[str, str],
uri: str,
format: Literal["JSON", "CSV", "PARQUET"],
export_options: Dict[str, Union[bool, str]],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any chance to make these consistent? Mapping and Dict. Prefer Mapping as a more general type.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed to use mapping for both

@TrevorBergeron TrevorBergeron merged commit 5f9a801 into main Aug 28, 2024
21 of 23 checks passed
@TrevorBergeron TrevorBergeron deleted the session_loader branch August 28, 2024 23:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery-dataframes API. size: xl Pull request size is extra large.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants
Morty Proxy This is a proxified and sanitized view of the page, visit original site.