Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit 42fa4b6

Browse filesBrowse files
adunstanadunstan
committed
Assorted minor cleanups in the test_json_parser module
Per gripes from Michael Paquier Discussion: https://postgr.es/m/ZhTQ6_w1vwOhqTQI@paquier.xyz Along the way, also clean up a handful of typos in 3311ea8 and ea7b4e9, found by Alexander Lakhin, and a couple of stylistic snafus noted by Daniel Westermann and Daniel Gustafsson.
1 parent b8a7bfa commit 42fa4b6
Copy full SHA for 42fa4b6

File tree

Expand file treeCollapse file tree

6 files changed

+43
-29
lines changed
Filter options
Expand file treeCollapse file tree

6 files changed

+43
-29
lines changed

‎src/backend/backup/basebackup_incremental.c

Copy file name to clipboardExpand all lines: src/backend/backup/basebackup_incremental.c
+3-3Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@
3434
#define BLOCKS_PER_READ 512
3535

3636
/*
37-
* we expect the find the last lines of the manifest, including the checksum,
37+
* We expect to find the last lines of the manifest, including the checksum,
3838
* in the last MIN_CHUNK bytes of the manifest. We trigger an incremental
3939
* parse step if we are about to overflow MAX_CHUNK bytes.
4040
*/
@@ -88,8 +88,8 @@ struct IncrementalBackupInfo
8888
* Files extracted from the backup manifest.
8989
*
9090
* We don't really need this information, because we use WAL summaries to
91-
* figure what's changed. It would be unsafe to just rely on the list of
92-
* files that existed before, because it's possible for a file to be
91+
* figure out what's changed. It would be unsafe to just rely on the list
92+
* of files that existed before, because it's possible for a file to be
9393
* removed and a new one created with the same name and different
9494
* contents. In such cases, the whole file must still be sent. We can tell
9595
* from the WAL summaries whether that happened, but not from the file

‎src/common/jsonapi.c

Copy file name to clipboardExpand all lines: src/common/jsonapi.c
+3-3Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -356,7 +356,7 @@ makeJsonLexContextCstringLen(JsonLexContext *lex, char *json,
356356
* need explicit stacks for predictions, field names and null indicators, but
357357
* we don't need the input, that will be handed in bit by bit to the
358358
* parse routine. We also need an accumulator for partial tokens in case
359-
* the boundary between chunks happns to fall in the middle of a token.
359+
* the boundary between chunks happens to fall in the middle of a token.
360360
*/
361361
#define JS_STACK_CHUNK_SIZE 64
362362
#define JS_MAX_PROD_LEN 10 /* more than we need */
@@ -1414,9 +1414,9 @@ json_lex(JsonLexContext *lex)
14141414
}
14151415

14161416
/*
1417-
* Add any remaining alpha_numeric chars. This takes care of the
1417+
* Add any remaining alphanumeric chars. This takes care of the
14181418
* {null, false, true} literals as well as any trailing
1419-
* alpha-numeric junk on non-string tokens.
1419+
* alphanumeric junk on non-string tokens.
14201420
*/
14211421
for (int i = added; i < lex->input_length; i++)
14221422
{

‎src/common/parse_manifest.c

Copy file name to clipboardExpand all lines: src/common/parse_manifest.c
+1-1Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -806,7 +806,7 @@ json_manifest_finalize_wal_range(JsonManifestParseState *parse)
806806
* the rest of the file.
807807
*
808808
* For an incremental parse, this will just be called on the last chunk of the
809-
* manifest, and the cryptohash context paswed in. For a non-incremental
809+
* manifest, and the cryptohash context passed in. For a non-incremental
810810
* parse incr_ctx will be NULL.
811811
*/
812812
static void

‎src/test/modules/test_json_parser/README

Copy file name to clipboardExpand all lines: src/test/modules/test_json_parser/README
+9-10Lines changed: 9 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -4,22 +4,21 @@ Module `test_json_parser`
44
This module contains two programs for testing the json parsers.
55

66
- `test_json_parser_incremental` is for testing the incremental parser, It
7-
reads in a file and pases it in very small chunks (60 bytes at a time) to
8-
the incremental parser. It's not meant to be a speed test but to test the
9-
accuracy of the incremental parser. It takes one argument: the name of the
10-
input file.
7+
reads in a file and passes it in very small chunks (default is 60 bytes at a
8+
time) to the incremental parser. It's not meant to be a speed test but to
9+
test the accuracy of the incremental parser. There are two option arguments,
10+
"-c nn" specifies an alternative chunk size, and "-s" specifies using
11+
semantic routines. The semantic routines re-output the json, although not in
12+
a very pretty form. The required non-option argument is the input file name.
1113
- `test_json_parser_perf` is for speed testing both the standard
1214
recursive descent parser and the non-recursive incremental
1315
parser. If given the `-i` flag it uses the non-recursive parser,
14-
otherwise the stardard parser. The remaining flags are the number of
16+
otherwise the standard parser. The remaining flags are the number of
1517
parsing iterations and the file containing the input. Even when
1618
using the non-recursive parser, the input is passed to the parser in a
1719
single chunk. The results are thus comparable to those of the
1820
standard parser.
1921

20-
The easiest way to use these is to run `make check` and `make speed-check`
21-
22-
The sample input file is a small extract from a list of `delicious`
22+
The sample input file is a small, sanitized extract from a list of `delicious`
2323
bookmarks taken some years ago, all wrapped in a single json
24-
array. 10,000 iterations of parsing this file gives a reasonable
25-
benchmark, and that is what the `speed-check` target does.
24+
array.

‎src/test/modules/test_json_parser/test_json_parser_incremental.c

Copy file name to clipboardExpand all lines: src/test/modules/test_json_parser/test_json_parser_incremental.c
+21-7Lines changed: 21 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -3,16 +3,17 @@
33
* test_json_parser_incremental.c
44
* Test program for incremental JSON parser
55
*
6-
* Copyright (c) 2023, PostgreSQL Global Development Group
6+
* Copyright (c) 2024, PostgreSQL Global Development Group
77
*
88
* IDENTIFICATION
99
* src/test/modules/test_json_parser/test_json_parser_incremental.c
1010
*
11-
* This progam tests incremental parsing of json. The input is fed into
11+
* This program tests incremental parsing of json. The input is fed into
1212
* the parser in very small chunks. In practice you would normally use
1313
* much larger chunks, but doing this makes it more likely that the
14-
* full range of incement handling, especially in the lexer, is exercised.
15-
* If the "-c SIZE" option is provided, that chunk size is used instead.
14+
* full range of increment handling, especially in the lexer, is exercised.
15+
* If the "-c SIZE" option is provided, that chunk size is used instead
16+
* of the default of 60.
1617
*
1718
* The argument specifies the file containing the JSON input.
1819
*
@@ -31,6 +32,9 @@
3132
#include "mb/pg_wchar.h"
3233
#include "pg_getopt.h"
3334

35+
#define BUFSIZE 6000
36+
#define DEFAULT_CHUNK_SIZE 60
37+
3438
typedef struct DoState
3539
{
3640
JsonLexContext *lex;
@@ -67,14 +71,13 @@ JsonSemAction sem = {
6771
int
6872
main(int argc, char **argv)
6973
{
70-
/* max delicious line length is less than this */
71-
char buff[6001];
74+
char buff[BUFSIZE];
7275
FILE *json_file;
7376
JsonParseErrorType result;
7477
JsonLexContext lex;
7578
StringInfoData json;
7679
int n_read;
77-
size_t chunk_size = 60;
80+
size_t chunk_size = DEFAULT_CHUNK_SIZE;
7881
struct stat statbuf;
7982
off_t bytes_left;
8083
JsonSemAction *testsem = &nullSemAction;
@@ -88,6 +91,11 @@ main(int argc, char **argv)
8891
{
8992
case 'c': /* chunksize */
9093
sscanf(optarg, "%zu", &chunk_size);
94+
if (chunk_size > BUFSIZE)
95+
{
96+
fprintf(stderr, "chunk size cannot exceed %d\n", BUFSIZE);
97+
exit(1);
98+
}
9199
break;
92100
case 's': /* do semantic processing */
93101
testsem = &sem;
@@ -121,6 +129,12 @@ main(int argc, char **argv)
121129
{
122130
n_read = fread(buff, 1, chunk_size, json_file);
123131
appendBinaryStringInfo(&json, buff, n_read);
132+
133+
/*
134+
* Append some trailing junk to the buffer passed to the parser. This
135+
* helps us ensure that the parser does the right thing even if the
136+
* chunk isn't terminated with a '\0'.
137+
*/
124138
appendStringInfoString(&json, "1+23 trailing junk");
125139
bytes_left -= n_read;
126140
if (bytes_left > 0)

‎src/test/modules/test_json_parser/test_json_parser_perf.c

Copy file name to clipboardExpand all lines: src/test/modules/test_json_parser/test_json_parser_perf.c
+6-5Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,14 @@
11
/*-------------------------------------------------------------------------
22
*
33
* test_json_parser_perf.c
4-
* Performancet est program for both flavors of the JSON parser
4+
* Performance test program for both flavors of the JSON parser
55
*
6-
* Copyright (c) 2023, PostgreSQL Global Development Group
6+
* Copyright (c) 2024, PostgreSQL Global Development Group
77
*
88
* IDENTIFICATION
99
* src/test/modules/test_json_parser/test_json_parser_perf.c
1010
*
11-
* This progam tests either the standard (recursive descent) JSON parser
11+
* This program tests either the standard (recursive descent) JSON parser
1212
* or the incremental (table driven) parser, but without breaking the input
1313
* into chunks in the latter case. Thus it can be used to compare the pure
1414
* parsing speed of the two parsers. If the "-i" option is used, then the
@@ -28,11 +28,12 @@
2828
#include <stdio.h>
2929
#include <string.h>
3030

31+
#define BUFSIZE 6000
32+
3133
int
3234
main(int argc, char **argv)
3335
{
34-
/* max delicious line length is less than this */
35-
char buff[6001];
36+
char buff[BUFSIZE];
3637
FILE *json_file;
3738
JsonParseErrorType result;
3839
JsonLexContext *lex;

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.