Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

[Feature Request] Expose the index LSN (x-pinecone-max-indexed-lsn) for robust freshness verification #470

Copy link
Copy link
@Amayas29

Description

@Amayas29
Issue body actions

What motivated you to submit this feature request?
We need a reliable way to verify whether recent write operations are visible to queries. The use of vector count comparison fails when insert/delete operations offset each other (e.g., inserting 5 and deleting 5 documents leaves the count unchanged). An LSN in index statistics would provide more robust verification of freshness.

Describe the solution you'd like
Expose the index lsn (x-pinecone-max-indexed-lsn) in describe_index_stats()

Describe alternatives you've considered
We currently use vector count comparison to detect ingestion freshness

write operations (upsert/delete) code:

# Store the count before write operation
initial_count = (
  index.describe_index_stats()
  .get("namespaces", {})
  .get(namespace, {})
  .get("vector_count", None)
)

index.upsert(...)  

Search code:

count = (
  index.describe_index_stats()
  .get("namespaces", {})
  .get(namespace, {})
  .get("vector_count", None)
)

if count == initial_count:
  LOGGER.warning(
      "Recently ingested documents may not yet be visible to queries "
      "due to Pinecone's indexing delay."
  )
  # Potentially followed by a sleep to await visibility.
Reactions are currently unavailable

Metadata

Metadata

Assignees

Labels

enhancementNew feature or requestNew feature or requeststatus:backlogThis issue has been added to our backlogThis issue has been added to our backlogwontfixThis will not be worked onThis will not be worked on

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

    Morty Proxy This is a proxified and sanitized view of the page, visit original site.