Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

feat: add MemorystoreDocumentSaver class #18

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Feb 14, 2024

Conversation

craig08
Copy link
Contributor

@craig08 craig08 commented Feb 13, 2024

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes #14 🦕

@product-auto-label product-auto-label bot added the api: redis Issues related to the googleapis/langchain-google-memorystore-redis-python API. label Feb 13, 2024
@craig08 craig08 force-pushed the doc-saver branch 2 times, most recently from 3dfaa0b to b6c03e3 Compare February 13, 2024 17:54
src/langchain_google_memorystore_redis/__init__.py Outdated Show resolved Hide resolved
src/langchain_google_memorystore_redis/doc_saver.py Outdated Show resolved Hide resolved
src/langchain_google_memorystore_redis/doc_saver.py Outdated Show resolved Hide resolved
src/langchain_google_memorystore_redis/doc_saver.py Outdated Show resolved Hide resolved
@craig08 craig08 self-assigned this Feb 13, 2024
@craig08 craig08 force-pushed the doc-saver branch 2 times, most recently from 3a89de6 to e1dfe23 Compare February 14, 2024 00:16
@craig08 craig08 marked this pull request as ready for review February 14, 2024 00:17
@craig08 craig08 requested a review from a team as a code owner February 14, 2024 00:17
@craig08 craig08 requested review from PingXie and removed request for a team February 14, 2024 00:18
src/langchain_google_memorystore_redis/vector_store.py Outdated Show resolved Hide resolved
src/langchain_google_memorystore_redis/doc_saver.py Outdated Show resolved Hide resolved
tests/test_doc_saver.py Show resolved Hide resolved
@craig08 craig08 assigned PingXie and unassigned craig08 Feb 14, 2024
Copy link
Contributor

@PingXie PingXie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@craig08 craig08 merged commit 5bd6b26 into googleapis:staging Feb 14, 2024
@craig08 craig08 deleted the doc-saver branch February 14, 2024 21:31
craig08 added a commit that referenced this pull request Feb 16, 2024
* feat(vector_store): Implement RedisVectorStore class

This commit introduces the RedisVectorStore class, extending VectorStore
to enable vector storage and retrieval using Memorystore Redis.

* feat: add integration test for MemorystoreChatMessageHistory (#13)

* fix: fix linter and encoding of MemorystoreChatMessageHistory class. (#20)

* fix: fix linter and encoding of MemorystoreChatMessageHistory class.

* fix: use relative path for __init__ imports

* feat(vector store): Add usage demo and enhance Redis integration (#17)

* feat(vector store): Add usage demo and enhance Redis integration

- Add Jupyter notebook in `docs/` for using langchain vector store with Memorystore Redis.
- Update `vector_store.py` for improved performance and reliability with Redis.
- Modify `__init__.py` to reflect package structure changes and new functionalities.
- Include `requirements.txt` and `setup.py` for easy installation and dependency management.
- Add `state_of_the_union.txt` in `docs/` as a sample dataset for the notebook demo.

* incorporate review feedback

* added missing license header

* excluded the test file from license check

* reformatted source files

* incorporated review feedback

* fixed lint errors

* fixed more lint errors

* removed key_prefix argument from vectorstore

* incorporated review feedback

* fixed formatting errors

* fixed a bad merge

* suppress mypy errors for setuptools

* trying mypy.ini

* remove setup.py and requirements

* add numpy dependency

* Update integration.cloudbuild.yaml

* feat: add MemorystoreDocumentSaver class (#18)

* feat: add MemorystoreDocumentSaver class

* fix: use variable in the worker pool of the Cloud Build integration

* fix: update doc saver for json encoding

* fix: align key_prefix behavior

* fix: add back key_prefix if keys_or_ids is given

* fix: fix indent of the comments

* fix: get encoder directly from the Redis client

* fix: add pipeline support for Redis client

* fix: reorder prefix and content_field in constructor

* fix: insert prefix to all given document IDs

* fix: fix tests prefix and chat encoding

* feat: add MemorystoreDocumentLoader class (#24)

* refactor(vector-store): enhance validation and initialization processes (#22)

* feat(vector store): Add usage demo and enhance Redis integration

- Add Jupyter notebook in `docs/` for using langchain vector store with Memorystore Redis.
- Update `vector_store.py` for improved performance and reliability with Redis.
- Modify `__init__.py` to reflect package structure changes and new functionalities.
- Include `requirements.txt` and `setup.py` for easy installation and dependency management.
- Add `state_of_the_union.txt` in `docs/` as a sample dataset for the notebook demo.

* incorporate review feedback

* added missing license header

* excluded the test file from license check

* reformatted source files

* incorporated review feedback

* fixed lint errors

* fixed more lint errors

* removed key_prefix argument from vectorstore

* incorporated review feedback

* fixed formatting errors

* fixed a bad merge

* suppress mypy errors for setuptools

* trying mypy.ini

* remove setup.py and requirements

* add numpy dependency

* refactor(vector-store): enhance validation and initialization processes

- Add validation for unsupported data types and negative vector sizes in VectorIndexConfig.
- Refactor field_name parameters in HNSWConfig and FLATConfig to support type hinting with optional strings.
- Enhance RedisVectorStore initialization by enforcing type checks for client, index_name, and embedding_service.
- Introduce an optional 'ids' parameter in methods to allow explicit document identifiers, improving document management.
- Adjust method documentation and error messages for clarity and consistency.

* Incorporated review feedback

* fixed formatting

* incorporated review feedback

* formatting

* fixed bugs

* fix formatting

* added vector store unittests

* actualy added vector store tests

* fix formatting

* fix: fix styles and formats

---------

Co-authored-by: Craig Chi <craigchi@google.com>

* fix: rename doc_ to document_ for consistency

* fix: add delete API for document saver and raise error if key exists

* fix: batch delete keys in document saver if specified

---------

Co-authored-by: Ping Xie <pingxie@google.com>
Co-authored-by: Ping Xie <pingxie@outlook.com>
Co-authored-by: Averi Kitsch <akitsch@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: redis Issues related to the googleapis/langchain-google-memorystore-redis-python API.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants
Morty Proxy This is a proxified and sanitized view of the page, visit original site.