Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

feat (Gen AI): Add new sample for Evaluation - Pairwise summarization metric #12608

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 11 commits into from
Oct 3, 2024
Merged

feat (Gen AI): Add new sample for Evaluation - Pairwise summarization metric #12608

merged 11 commits into from
Oct 3, 2024

Conversation

Valeriy-Burlaka
Copy link
Member

@Valeriy-Burlaka Valeriy-Burlaka commented Sep 19, 2024 •
edited
Loading

Description

Adds a Python sample for the "Evaluate an output: pairwise summarization quality section of the model-reference/evaluation Vertex AI page.

Checklist

  • I have followed Sample Guidelines from AUTHORING_GUIDE.MD
  • README is updated to include all relevant information
  • Tests pass: nox -s py-3.8 and nox -s py-3.11
  • Lint pass: nox -s lint
  • These samples need a new API enabled in testing projects to pass (let us know which ones)
  • These samples need a new/updated env vars in testing projects set to pass (let us know which ones)
  • This sample adds a new sample directory, and I updated the CODEOWNERS file with the codeowners for this sample
  • This sample adds a new Product API, and I updated the Blunderbuss issue/PR auto-assigner with the codeowners for this sample
  • Please merge this PR for me once it is approved

@Valeriy-Burlaka Valeriy-Burlaka requested review from a team as code owners September 19, 2024 13:15
@google-cla Google CLA
Copy link

google-cla bot commented Sep 19, 2024

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@snippet-bot Snippet Bot
Copy link

snippet-bot bot commented Sep 19, 2024 •
edited
Loading

Here is the summary of changes.

You are about to add 1 region tag.

This comment is generated by snippet-bot.
If you find problems with this result, please file an issue at:
https://github.com/googleapis/repo-automation-bots/issues.
To update this comment, add snippet-bot:force-run label or use the checkbox below:

  • Refresh this comment

@product-auto-label product-auto-label bot added the samples Issues that are directly related to samples. label Sep 19, 2024
@Valeriy-Burlaka Valeriy-Burlaka marked this pull request as draft September 19, 2024 13:17
@Valeriy-Burlaka Valeriy-Burlaka force-pushed the vburlaka-genai-samples-evaluate-pairwise-sum-quality branch from d481281 to 624b33f Compare September 24, 2024 15:48
@Valeriy-Burlaka Valeriy-Burlaka marked this pull request as ready for review September 24, 2024 15:48
@Valeriy-Burlaka Valeriy-Burlaka requested a review from a team as a code owner September 24, 2024 15:48
@Valeriy-Burlaka

This comment was marked as outdated.

Sign in to view
@Valeriy-Burlaka

This comment was marked as outdated.

Sign in to view
@msampathkumar msampathkumar self-assigned this Oct 1, 2024
msampathkumar
msampathkumar reviewed Oct 3, 2024
View reviewed changes
generative_ai/evaluation/pairwise_summarization_quality.py Outdated Show resolved Hide resolved
msampathkumar
msampathkumar approved these changes Oct 3, 2024
View reviewed changes
Copy link
Member

@msampathkumar msampathkumar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Overall looks good to me.

@msampathkumar
Copy link
Member

Please also check and resolve the conflicts.

@msampathkumar msampathkumar added the waiting-response Waiting for the author's response. label Oct 3, 2024
@Valeriy-Burlaka Valeriy-Burlaka removed the waiting-response Waiting for the author's response. label Oct 3, 2024
msampathkumar
msampathkumar reviewed Oct 3, 2024
View reviewed changes
generative_ai/evaluation/pairwise_summarization_quality.py Outdated Show resolved Hide resolved
msampathkumar
msampathkumar reviewed Oct 3, 2024
View reviewed changes
generative_ai/evaluation/pairwise_summarization_quality.py Outdated Show resolved Hide resolved
msampathkumar
msampathkumar reviewed Oct 3, 2024
View reviewed changes
generative_ai/evaluation/pairwise_summarization_quality.py Outdated Show resolved Hide resolved
Valeriy-Burlaka
Valeriy-Burlaka commented Oct 3, 2024
View reviewed changes
generative_ai/evaluation/pairwise_summarization_quality.py Outdated Show resolved Hide resolved
@msampathkumar msampathkumar changed the title feat: (Gen AI): Add sample for Gen AI Evaluation: Pairwise summarization quality feat (Gen AI): Add new sample for Evaluation - Pairwise summarization metric Oct 3, 2024
@Valeriy-Burlaka Valeriy-Burlaka merged commit 5aab168 into GoogleCloudPlatform:main Oct 3, 2024
11 checks passed
riathakkar pushed a commit to riathakkar/python-docs-samples that referenced this pull request Oct 8, 2024
@Valeriy-Burlaka @riathakkar
feat (Gen AI): Add new sample for Evaluation - Pairwise summarization…
e867643 
… metric (GoogleCloudPlatform#12608)

* Add sample for Gen AI Evaluation: Pairwise summarization quality

* refresh CI builds

* refresh CI builds

* Update comments to follow the runnable-snippets-plan standard

* move pairwise_evaluation sample and test to a dedicated directory

* refresh CI checks

* specify exact model version

* update region tag names to follow standard schema

* add example response
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@msampathkumar msampathkumar msampathkumar approved these changes

@m-strzelczyk m-strzelczyk Awaiting requested review from m-strzelczyk

Assignees

@msampathkumar msampathkumar

@dandhlee dandhlee

Labels
samples Issues that are directly related to samples.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@Valeriy-Burlaka @msampathkumar @dandhlee
Morty Proxy This is a proxified and sanitized view of the page, visit original site.