Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit 5fd9dfa

Browse filesBrowse files
bmomjianbmomjian
committed
Move pg_stat_statements query jumbling to core.
Add compute_query_id GUC to control whether a query identifier should be computed by the core (off by default). It's thefore now possible to disable core queryid computation and use pg_stat_statements with a different algorithm to compute the query identifier by using a third-party module. To ensure that a single source of query identifier can be used and is well defined, modules that calculate a query identifier should throw an error if compute_query_id specified to compute a query id and if a query idenfitier was already calculated. Discussion: https://postgr.es/m/20210407125726.tkvjdbw76hxnpwfi@nol Author: Julien Rouhaud Reviewed-by: Alvaro Herrera, Nitin Jadhav, Zhihong Yu
1 parent a282ee6 commit 5fd9dfa
Copy full SHA for 5fd9dfa

File tree

13 files changed

+995
-785
lines changed
Filter options

13 files changed

+995
-785
lines changed

‎contrib/pg_stat_statements/pg_stat_statements.c

Copy file name to clipboardExpand all lines: contrib/pg_stat_statements/pg_stat_statements.c
+25-780Lines changed: 25 additions & 780 deletions
Large diffs are not rendered by default.

‎contrib/pg_stat_statements/pg_stat_statements.conf

Copy file name to clipboard
+1Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1 +1,2 @@
11
shared_preload_libraries = 'pg_stat_statements'
2+
compute_query_id = on

‎doc/src/sgml/config.sgml

Copy file name to clipboardExpand all lines: doc/src/sgml/config.sgml
+25Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7622,6 +7622,31 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
76227622
<title>Statistics Monitoring</title>
76237623
<variablelist>
76247624

7625+
<varlistentry id="guc-compute-query-id" xreflabel="compute_query_id">
7626+
<term><varname>compute_query_id</varname> (<type>boolean</type>)
7627+
<indexterm>
7628+
<primary><varname>compute_query_id</varname> configuration parameter</primary>
7629+
</indexterm>
7630+
</term>
7631+
<listitem>
7632+
<para>
7633+
Enables in-core computation of a query identifier. The <xref
7634+
linkend="pgstatstatements"/> extension requires a query identifier
7635+
to be computed. Note that an external module can alternatively
7636+
be used if the in-core query identifier computation method
7637+
isn't acceptable. In this case, in-core computation should
7638+
remain disabled. The default is <literal>off</literal>.
7639+
</para>
7640+
<note>
7641+
<para>
7642+
To ensure that a only one query identifier is calculated and
7643+
displayed, extensions that calculate query identifiers should
7644+
throw an error if a query identifier has already been computed.
7645+
</para>
7646+
</note>
7647+
</listitem>
7648+
</varlistentry>
7649+
76257650
<varlistentry>
76267651
<term><varname>log_statement_stats</varname> (<type>boolean</type>)
76277652
<indexterm>

‎doc/src/sgml/pgstatstatements.sgml

Copy file name to clipboardExpand all lines: doc/src/sgml/pgstatstatements.sgml
+19-1Lines changed: 19 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,14 @@
2020
This means that a server restart is needed to add or remove the module.
2121
</para>
2222

23+
<para>
24+
The module will not track statistics unless query
25+
identifiers are calculated. This can be done by enabling <xref
26+
linkend="guc-compute-query-id"/> or using a third-party module that
27+
computes its own query identifiers. Note that all statistics tracked
28+
by this module must be reset if the query identifier method is changed.
29+
</para>
30+
2331
<para>
2432
When <filename>pg_stat_statements</filename> is loaded, it tracks
2533
statistics across all databases of the server. To access and manipulate
@@ -84,7 +92,7 @@
8492
<structfield>queryid</structfield> <type>bigint</type>
8593
</para>
8694
<para>
87-
Internal hash code, computed from the statement's parse tree
95+
Hash code to identify identical normalized queries.
8896
</para></entry>
8997
</row>
9098

@@ -386,6 +394,16 @@
386394
are compared strictly on the basis of their textual query strings, however.
387395
</para>
388396

397+
<note>
398+
<para>
399+
The following details about constant replacement and
400+
<structfield>queryid</structfield> only applies when <xref
401+
linkend="guc-compute-query-id"/> is enabled. If you use an external
402+
module instead to compute <structfield>queryid</structfield>, you
403+
should refer to its documentation for details.
404+
</para>
405+
</note>
406+
389407
<para>
390408
When a constant's value has been ignored for purposes of matching the query
391409
to other queries, the constant is replaced by a parameter symbol, such

‎src/backend/parser/analyze.c

Copy file name to clipboardExpand all lines: src/backend/parser/analyze.c
+12-2Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,8 @@
4646
#include "parser/parsetree.h"
4747
#include "rewrite/rewriteManip.h"
4848
#include "utils/builtins.h"
49+
#include "utils/guc.h"
50+
#include "utils/queryjumble.h"
4951
#include "utils/rel.h"
5052

5153

@@ -107,6 +109,7 @@ parse_analyze(RawStmt *parseTree, const char *sourceText,
107109
{
108110
ParseState *pstate = make_parsestate(NULL);
109111
Query *query;
112+
JumbleState *jstate = NULL;
110113

111114
Assert(sourceText != NULL); /* required as of 8.4 */
112115

@@ -119,8 +122,11 @@ parse_analyze(RawStmt *parseTree, const char *sourceText,
119122

120123
query = transformTopLevelStmt(pstate, parseTree);
121124

125+
if (compute_query_id)
126+
jstate = JumbleQuery(query, sourceText);
127+
122128
if (post_parse_analyze_hook)
123-
(*post_parse_analyze_hook) (pstate, query);
129+
(*post_parse_analyze_hook) (pstate, query, jstate);
124130

125131
free_parsestate(pstate);
126132

@@ -140,6 +146,7 @@ parse_analyze_varparams(RawStmt *parseTree, const char *sourceText,
140146
{
141147
ParseState *pstate = make_parsestate(NULL);
142148
Query *query;
149+
JumbleState *jstate = NULL;
143150

144151
Assert(sourceText != NULL); /* required as of 8.4 */
145152

@@ -152,8 +159,11 @@ parse_analyze_varparams(RawStmt *parseTree, const char *sourceText,
152159
/* make sure all is well with parameter types */
153160
check_variable_parameters(pstate, query);
154161

162+
if (compute_query_id)
163+
jstate = JumbleQuery(query, sourceText);
164+
155165
if (post_parse_analyze_hook)
156-
(*post_parse_analyze_hook) (pstate, query);
166+
(*post_parse_analyze_hook) (pstate, query, jstate);
157167

158168
free_parsestate(pstate);
159169

‎src/backend/tcop/postgres.c

Copy file name to clipboardExpand all lines: src/backend/tcop/postgres.c
+5-1Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -668,6 +668,7 @@ pg_analyze_and_rewrite_params(RawStmt *parsetree,
668668
ParseState *pstate;
669669
Query *query;
670670
List *querytree_list;
671+
JumbleState *jstate = NULL;
671672

672673
Assert(query_string != NULL); /* required as of 8.4 */
673674

@@ -686,8 +687,11 @@ pg_analyze_and_rewrite_params(RawStmt *parsetree,
686687

687688
query = transformTopLevelStmt(pstate, parsetree);
688689

690+
if (compute_query_id)
691+
jstate = JumbleQuery(query, query_string);
692+
689693
if (post_parse_analyze_hook)
690-
(*post_parse_analyze_hook) (pstate, query);
694+
(*post_parse_analyze_hook) (pstate, query, jstate);
691695

692696
free_parsestate(pstate);
693697

‎src/backend/utils/misc/Makefile

Copy file name to clipboardExpand all lines: src/backend/utils/misc/Makefile
+1Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ OBJS = \
2222
pg_rusage.o \
2323
ps_status.o \
2424
queryenvironment.o \
25+
queryjumble.o \
2526
rls.o \
2627
sampling.o \
2728
superuser.o \

‎src/backend/utils/misc/guc.c

Copy file name to clipboardExpand all lines: src/backend/utils/misc/guc.c
+10Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -534,6 +534,7 @@ extern const struct config_enum_entry dynamic_shared_memory_options[];
534534
/*
535535
* GUC option variables that are exported from this module
536536
*/
537+
bool compute_query_id = false;
537538
bool log_duration = false;
538539
bool Debug_print_plan = false;
539540
bool Debug_print_parse = false;
@@ -1458,6 +1459,15 @@ static struct config_bool ConfigureNamesBool[] =
14581459
true,
14591460
NULL, NULL, NULL
14601461
},
1462+
{
1463+
{"compute_query_id", PGC_SUSET, STATS_MONITORING,
1464+
gettext_noop("Compute query identifiers."),
1465+
NULL
1466+
},
1467+
&compute_query_id,
1468+
false,
1469+
NULL, NULL, NULL
1470+
},
14611471
{
14621472
{"log_parser_stats", PGC_SUSET, STATS_MONITORING,
14631473
gettext_noop("Writes parser performance statistics to the server log."),

‎src/backend/utils/misc/postgresql.conf.sample

Copy file name to clipboardExpand all lines: src/backend/utils/misc/postgresql.conf.sample
+1Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -596,6 +596,7 @@
596596

597597
# - Monitoring -
598598

599+
#compute_query_id = off
599600
#log_parser_stats = off
600601
#log_planner_stats = off
601602
#log_executor_stats = off

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.