Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

gh-69093: Support basic incremental I/O to blobs in sqlite3 #30680

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 110 commits into from
Apr 15, 2022
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
110 commits
Select commit Hold shift + click to select a range
4fa0e91
support BLOB incremental I/O in sqlite module
palaviv Feb 24, 2017
3fe9108
Note that blob size cannot be changed using the blob object
palaviv Feb 25, 2017
865c1c8
Use assertRaises in tests
palaviv Feb 25, 2017
788fd54
Fix doc error
palaviv Feb 25, 2017
a1361e5
blob support sequence protocol
palaviv Mar 4, 2017
5aedfba
Calculate blob length once at creation
palaviv Mar 4, 2017
db6ef32
Add initial Doc for sequence protocol
palaviv Mar 4, 2017
219f4cb
Don't support blob operation
palaviv Apr 18, 2018
6dafe0e
move news entry to blurb
palaviv Apr 18, 2018
ffac901
Add blob to PCBuild
palaviv Apr 18, 2018
3475fa1
Update version
palaviv May 8, 2019
f6015ff
Fix memory leak
palaviv May 8, 2019
01e526c
Update version
palaviv Jul 29, 2020
354bebf
Fix CR comments in documentation and testing
palaviv Aug 3, 2020
9709456
Fix CR comments in code
palaviv Aug 3, 2020
e6e5099
Make readonly and dbname keyword arguements only
palaviv Aug 3, 2020
e9a8080
Fix more CR comments
palaviv Aug 3, 2020
d4fb1b5
Add more indicative error on write bigger then blob length
palaviv Aug 3, 2020
a7288f9
Merge branch 'main' into sqlite-blob
Sep 10, 2021
525a9c3
Adapt sqlite3.Connection.open_blob() to AC
Sep 10, 2021
2f65cc8
Adapt sqlite.Blob to AC
Sep 10, 2021
0dd047f
PEP 7 and normalised naming
Sep 10, 2021
d34a77b
Use Py_NewRef and Py_IsNone
Sep 10, 2021
9d55705
Harden blob_open()
Sep 10, 2021
d47ea17
Move blob init to blob_open()
Sep 10, 2021
e07a116
initialise rc in blob_ass_subscript()
Sep 10, 2021
982c812
Improve blob.seek() parameter naming
Sep 10, 2021
e92495b
Adapt to heap types and allow calling close multiple times
Sep 10, 2021
8f2ce8a
Consolidate tests
Sep 10, 2021
0a87520
Naming: read_length => length
Sep 10, 2021
32132cf
Wrap error handling in support function
Sep 10, 2021
287803e
Add blob seek position markers as constants
Sep 10, 2021
0fc5a39
Adjust SQLITE_ABORT error message
Sep 10, 2021
cd0bde1
Remove unneeded sqlite3_blob_close() and fix GC tracking
Sep 10, 2021
cf7e15e
Use close_blob() in pysqlite_close_all_blobs()
Sep 10, 2021
7e77217
Refactor write/read
Sep 10, 2021
c57e45a
Sipmlify __exit__
Sep 10, 2021
dd76f72
Use new slice API
Sep 10, 2021
237684e
Refactor very large functions
Sep 11, 2021
58905e8
Simplify subscript slice
Sep 11, 2021
9d69fca
Consolidate more tests
Sep 11, 2021
285bb3d
Use supplied offset in inner_read()
Sep 11, 2021
2f3051a
Simplify test
Sep 11, 2021
fd7c311
Simplify assign subscript slice
Sep 11, 2021
0644e87
Early error checking, and use PyBytes_AS_STRING() when possible
Sep 11, 2021
ced431a
Expand test suite
Sep 11, 2021
2dae1b9
Remove unneeded parts of sequence protocol
Sep 11, 2021
e8fa47e
Normalise error messages
Sep 11, 2021
6441668
Improve error message/type for to large subscript assignment
Sep 11, 2021
03d2152
Normalise naming: write_inner => inner_write
Sep 11, 2021
9360e62
Adjust comment
Sep 11, 2021
8eb16f7
Move blob_seterror() closer to where it's first used
Sep 11, 2021
411d07e
Fetch state from connection in check_blob()
Sep 11, 2021
8149e36
Remove unused declaration
Sep 11, 2021
56b4caa
Doc adjustments
Sep 11, 2021
afdeb2e
Add What's New and adjust NEWS
Sep 11, 2021
b12219f
Use sqlite3_blob_bytes() iso. storing length on blob object
Sep 12, 2021
32a21d8
Merge branch 'main' into sqlite-blob
Sep 12, 2021
36b0ca1
Also remove length from blob_open()
Sep 12, 2021
3d91705
Add get subscript index helper
Sep 12, 2021
ceee315
Expand test suite
Sep 12, 2021
44f4cd3
Merge branch 'main' into sqlite-blob
Sep 21, 2021
220b576
Merge branch 'main' into sqlite-blob
Nov 17, 2021
d9b5cdf
Format docs with linewidth 80
Nov 17, 2021
aaa2721
Improve tests
Nov 17, 2021
18e5118
Merge branch 'main' into sqlite-blob
Jan 2, 2022
8f685ba
Add clinic_state() stub. (Currently not used)
Jan 2, 2022
96df661
Visit/clear blob type during module traverse/clear
Jan 2, 2022
97d12a8
Harden some tests, simplify others
Jan 2, 2022
2e63b3e
Simplify example
Jan 3, 2022
be27747
Update docs
Jan 3, 2022
5add365
Simplify tests
Jan 3, 2022
bacf087
Safe close
Jan 3, 2022
5ff202d
Condense a comment
Jan 3, 2022
1fa5901
Make sure we catch int overflow for all types of writes
Jan 3, 2022
16f4d0d
Merge branch 'main' into sqlite-blob-all
Jan 18, 2022
7aec288
Fix typo in test name
Jan 18, 2022
aaaf7ab
Improve test
Jan 18, 2022
f9e65c0
Improve tests
Jan 18, 2022
6a5c864
Always check length in inner write
Jan 18, 2022
ea1045c
Fix missing array sentinel
Jan 18, 2022
2f848d9
Match apsw's API open_blob => blobopen
Jan 19, 2022
8d906bc
Fix overflow test on win x64
Jan 19, 2022
90b75d1
Remove mapping protocol support
Jan 3, 2022
be34fe6
Remove context manager support
Jan 3, 2022
b8ab73b
Update Modules/_sqlite/blob.c
Jan 19, 2022
1d05c70
Don't use module; it's a C++ keyword
Jan 19, 2022
b48742a
Sync with main
Apr 11, 2022
47f7dec
Doc adjustments
Apr 11, 2022
703d4fd
Normalise test names
Apr 11, 2022
a167eb3
Doc: add a sembreak
Apr 11, 2022
b38f0df
Try to align docstrings and docs
Apr 11, 2022
1b5f953
More docstring adjustments
Apr 11, 2022
95b386f
Test adjustments
Apr 11, 2022
26c3623
Simplify read-at-offset test
Apr 11, 2022
fac770a
Simplify tests further
Apr 11, 2022
290bb18
Simplify tests further
Apr 11, 2022
8708b34
Doc: mapping protocol is removed in this PR
Apr 11, 2022
e15c087
Sync with main bco. gh-79097
Apr 12, 2022
a9c8928
Address review: reword docs
Apr 12, 2022
5c1ddfc
Address review: link with file-like object
Apr 12, 2022
e49b35f
Address review: spell out the subtests of test_blob_sequence_not_supp…
Apr 12, 2022
9e7ae6b
Use SEEK_* macros iso. magical numbers
Apr 12, 2022
453f522
Address review: singular => plural
Apr 12, 2022
154edca
Don't use magical numbers in exception messages
Apr 12, 2022
f784a81
IT'S ESS CUE EL LITE
Apr 12, 2022
0bc67ed
🤦
Apr 13, 2022
8d834bf
Address Alex' review
Apr 14, 2022
2590112
Address Berker's comments from gh-271: remove unneeded class prefix
Apr 14, 2022
69a5a1e
Address Berker's comments from gh-271: move __len__ doc to class desc…
Apr 14, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Naming: read_length => length
  • Loading branch information
Erlend E. Aasland committed Sep 10, 2021
commit 0a87520c85592cfea15bb8b75fa6d2a27794f8f7
24 changes: 12 additions & 12 deletions 24 Modules/_sqlite/blob.c
Original file line number Diff line number Diff line change
Expand Up @@ -118,17 +118,17 @@ blob_length(pysqlite_Blob *self)
};

static PyObject *
inner_read(pysqlite_Blob *self, int read_length, int offset)
inner_read(pysqlite_Blob *self, int length, int offset)
{
PyObject *buffer = PyBytes_FromStringAndSize(NULL, read_length);
PyObject *buffer = PyBytes_FromStringAndSize(NULL, length);
if (buffer == NULL) {
return NULL;
}
char *raw_buffer = PyBytes_AS_STRING(buffer);

int rc;
Py_BEGIN_ALLOW_THREADS
rc = sqlite3_blob_read(self->blob, raw_buffer, read_length, self->offset);
rc = sqlite3_blob_read(self->blob, raw_buffer, length, self->offset);
Py_END_ALLOW_THREADS

if (rc != SQLITE_OK) {
Expand All @@ -151,34 +151,34 @@ inner_read(pysqlite_Blob *self, int read_length, int offset)
/*[clinic input]
_sqlite3.Blob.read as blob_read

read_length: int = -1
length: int = -1
/

Read data from blob.
[clinic start generated code]*/

static PyObject *
blob_read_impl(pysqlite_Blob *self, int read_length)
/*[clinic end generated code: output=9c4881a77860b216 input=753a766082129348]*/
blob_read_impl(pysqlite_Blob *self, int length)
/*[clinic end generated code: output=1fc99b2541360dde input=b4b443e99af5548f]*/
{
if (!check_blob(self)) {
return NULL;
}

if (read_length < 0) {
if (length < 0) {
/* same as file read. */
read_length = self->length;
length = self->length;
}

/* making sure we don't read more then blob size */
if (read_length > self->length - self->offset) {
read_length = self->length - self->offset;
if (length > self->length - self->offset) {
length = self->length - self->offset;
}

PyObject *buffer = inner_read(self, read_length, self->offset);
PyObject *buffer = inner_read(self, length, self->offset);
if (buffer != NULL) {
/* update offset on sucess. */
self->offset += read_length;
self->offset += length;
}

return buffer;
Expand Down
14 changes: 7 additions & 7 deletions 14 Modules/_sqlite/clinic/blob.c.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Morty Proxy This is a proxified and sanitized view of the page, visit original site.