Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Commit f3534cc

Browse filesBrowse files
authored
docs: present documentarray modification with context manager (#406)
* docs: present documentarray modification with context manager * docs: fix writting
1 parent 2a06be1 commit f3534cc
Copy full SHA for f3534cc

1 file changed

+40-17Lines changed: 40 additions & 17 deletions

File tree

Expand file treeCollapse file tree
Open diff view settings
Filter options
Expand file treeCollapse file tree
Open diff view settings
Collapse file

‎docs/advanced/document-store/index.md‎

Copy file name to clipboardExpand all lines: docs/advanced/document-store/index.md
+40-17Lines changed: 40 additions & 17 deletions
  • Display the source diff
  • Display the rich diff
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,8 @@ extend
1313
benchmark
1414
```
1515

16-
Documents inside a DocumentArray can live in a [document store](https://en.wikipedia.org/wiki/Document-oriented_database) instead of in memory, e.g. in SQLite, Redis. Comparing to the in-memory storage, the benefit of using an external store is often about longer persistence and faster retrieval.
16+
Documents inside a DocumentArray can live in a [document store](https://en.wikipedia.org/wiki/Document-oriented_database) instead of in memory, e.g. in SQLite, Redis.
17+
Comparing to the in-memory storage, the benefit of using an external store is often about longer persistence and faster retrieval.
1718

1819
The look-and-feel of a DocumentArray with external store is **almost the same** as a regular in-memory DocumentArray. This allows users to easily switch between backends under the same DocArray idiom.
1920

@@ -24,24 +25,44 @@ from docarray import DocumentArray, Document
2425

2526
da = DocumentArray(storage='sqlite', config={'connection': 'example.db'})
2627

27-
da.append(Document())
28+
with da:
29+
da.append(Document())
2830
da.summary()
2931
```
3032

3133
```text
32-
Documents Summary
33-
34-
Length 1
35-
Homogenous Documents True
36-
Common Attributes ('id',)
37-
38-
Attributes Summary
39-
40-
Attribute Data type #Unique values Has empty value
41-
──────────────────────────────────────────────────────────
42-
id ('str',) 1 False
43-
44-
```
34+
╭──────── Documents Summary ─────────╮
35+
│ │
36+
│ Length 1 │
37+
│ Homogenous Documents True │
38+
│ Common Attributes ('id',) │
39+
│ Multimodal dataclass False │
40+
│ │
41+
╰────────────────────────────────────╯
42+
╭───────────────────── Attributes Summary ─────────────────────╮
43+
│ │
44+
│ Attribute Data type #Unique values Has empty value │
45+
│ ────────────────────────────────────────────────────────── │
46+
│ id ('str',) 1 False │
47+
│ │
48+
╰──────────────────────────────────────────────────────────────╯
49+
╭──────────────────────── DocumentArraySqlite Config ────────────────────────╮
50+
│ │
51+
│ connection example.db │
52+
│ table_name DocumentArraySqlite97c8c833586444a89272ff0ff4287edb │
53+
│ serialize_config {} │
54+
│ conn_config {} │
55+
│ journal_mode DELETE │
56+
│ synchronous OFF │
57+
│ │
58+
╰────────────────────────────────────────────────────────────────────────────╯
59+
```
60+
Note that `da` was modified inside a `with` statement. This context manager ensures that the the `DocumentArray` indices,
61+
which allow users to access the `DocumentArray` by position (allowing statements such as `da[1]`),
62+
are properly mapped and saved to the storage backend.
63+
This is the recommended default usage to modify a DocumentArray that lives on a document store to avoid
64+
unexpected behaviors that can yield to, for example, inaccessible elements by position.
65+
4566

4667
Creating, retrieving, updating, deleting Documents are identical to the regular {ref}`DocumentArray<documentarray>`. All DocumentArray methods such as `.summary()`, `.embed()`, `.plot_embeddings()` should work out of the box.
4768

@@ -136,7 +157,8 @@ The output is:
136157
0
137158
```
138159

139-
Looks like `db` is not really up-to-date with `da`. This is true and false. True in the sense that `1` is not `0`, number speaks by itself. False in the sense that, the Document is already written to the storage backend. You just can't see it.
160+
Looks like `db` is not really up-to-date with `da`. This is true and false. True in the sense that `1` is not `0`, number speaks by itself.
161+
False in the sense that, the Document is already written to the storage backend. You just can't see it.
140162

141163
To prove it does persist, run the following code snippet multiple times and you will see the length is increasing one at a time:
142164

@@ -148,7 +170,8 @@ da.append(Document(text='hello'))
148170
print(len(da))
149171
```
150172

151-
Simply put, the reason of this behavior is that certain meta information **not synced immediately** to the backend on *every* operation; it would be very costly to do so. As a consequence, your multiple references to the same backend would look different if they are written in one code block as the example above.
173+
Simply put, the reason of this behavior is that certain meta information **not synced immediately** to the backend on *every* operation; it would be very costly to do so.
174+
As a consequence, your multiple references to the same backend would look different if they are written in one code block as the example above.
152175

153176
To solve this problem, simply use `with` statement and use DocumentArray as a context manager. The last example can be refactored into the following:
154177

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.