Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit 6b4d23f

Browse filesBrowse files
mhagandermhagander
committed
Track identical top vs nested queries independently in pg_stat_statements
Changing pg_stat_statements.track between 'all' and 'top' would control if pg_stat_statements tracked just top level statements or also statements inside functions, but when tracking all it would not differentiate between the two. Being table to differentiate this is useful both to track where the actual query is coming from, and to see if there are differences in executions between the two. To do this, add a boolean to the hash key indicating if the statement was top level or not. Experience from the pg_stat_kcache module shows that in at least some "reasonable worloads" only <5% of the queries show up both top level and nested. Based on this, admittedly small, dataset, this patch does not try to de-duplicate those query *texts*, and will just store one copy for the top level and one for the nested. Author: Julien Rohaud Reviewed-By: Magnus Hagander, Masahiro Ikeda Discussion: https://postgr.es/m/20201202040516.GA43757@nol
1 parent 2e0e066 commit 6b4d23f
Copy full SHA for 6b4d23f

File tree

Expand file treeCollapse file tree

7 files changed

+173
-9
lines changed
Filter options
Expand file treeCollapse file tree

7 files changed

+173
-9
lines changed

‎contrib/pg_stat_statements/Makefile

Copy file name to clipboardExpand all lines: contrib/pg_stat_statements/Makefile
+2-1Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,8 @@ OBJS = \
66
pg_stat_statements.o
77

88
EXTENSION = pg_stat_statements
9-
DATA = pg_stat_statements--1.4.sql pg_stat_statements--1.8--1.9.sql \
9+
DATA = pg_stat_statements--1.4.sql \
10+
pg_stat_statements--1.9--1.10.sql pg_stat_statements--1.8--1.9.sql \
1011
pg_stat_statements--1.7--1.8.sql pg_stat_statements--1.6--1.7.sql \
1112
pg_stat_statements--1.5--1.6.sql pg_stat_statements--1.4--1.5.sql \
1213
pg_stat_statements--1.3--1.4.sql pg_stat_statements--1.2--1.3.sql \

‎contrib/pg_stat_statements/expected/pg_stat_statements.out

Copy file name to clipboardExpand all lines: contrib/pg_stat_statements/expected/pg_stat_statements.out
+40Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -901,4 +901,44 @@ SELECT dealloc FROM pg_stat_statements_info;
901901
0
902902
(1 row)
903903

904+
--
905+
-- top level handling
906+
--
907+
SET pg_stat_statements.track = 'top';
908+
DELETE FROM test;
909+
DO $$
910+
BEGIN
911+
DELETE FROM test;
912+
END;
913+
$$ LANGUAGE plpgsql;
914+
SELECT query, toplevel, plans, calls FROM pg_stat_statements WHERE query LIKE '%DELETE%' ORDER BY query COLLATE "C", toplevel;
915+
query | toplevel | plans | calls
916+
-----------------------+----------+-------+-------
917+
DELETE FROM test | t | 1 | 1
918+
DO $$ +| t | 0 | 1
919+
BEGIN +| | |
920+
DELETE FROM test;+| | |
921+
END; +| | |
922+
$$ LANGUAGE plpgsql | | |
923+
(2 rows)
924+
925+
SET pg_stat_statements.track = 'all';
926+
DELETE FROM test;
927+
DO $$
928+
BEGIN
929+
DELETE FROM test;
930+
END;
931+
$$ LANGUAGE plpgsql;
932+
SELECT query, toplevel, plans, calls FROM pg_stat_statements WHERE query LIKE '%DELETE%' ORDER BY query COLLATE "C", toplevel;
933+
query | toplevel | plans | calls
934+
-----------------------+----------+-------+-------
935+
DELETE FROM test | f | 1 | 1
936+
DELETE FROM test | t | 2 | 2
937+
DO $$ +| t | 0 | 2
938+
BEGIN +| | |
939+
DELETE FROM test;+| | |
940+
END; +| | |
941+
$$ LANGUAGE plpgsql | | |
942+
(3 rows)
943+
904944
DROP EXTENSION pg_stat_statements;

‎contrib/pg_stat_statements/pg_stat_statements--1.9--1.10.sql

Copy file name to clipboard
+57Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
/* contrib/pg_stat_statements/pg_stat_statements--1.9--1.10.sql */
2+
3+
-- complain if script is sourced in psql, rather than via ALTER EXTENSION
4+
\echo Use "ALTER EXTENSION pg_stat_statements UPDATE TO '1.10'" to load this file. \quit
5+
6+
/* First we have to remove them from the extension */
7+
ALTER EXTENSION pg_stat_statements DROP VIEW pg_stat_statements;
8+
ALTER EXTENSION pg_stat_statements DROP FUNCTION pg_stat_statements(boolean);
9+
10+
/* Then we can drop them */
11+
DROP VIEW pg_stat_statements;
12+
DROP FUNCTION pg_stat_statements(boolean);
13+
14+
/* Now redefine */
15+
CREATE FUNCTION pg_stat_statements(IN showtext boolean,
16+
OUT userid oid,
17+
OUT dbid oid,
18+
OUT toplevel bool,
19+
OUT queryid bigint,
20+
OUT query text,
21+
OUT plans int8,
22+
OUT total_plan_time float8,
23+
OUT min_plan_time float8,
24+
OUT max_plan_time float8,
25+
OUT mean_plan_time float8,
26+
OUT stddev_plan_time float8,
27+
OUT calls int8,
28+
OUT total_exec_time float8,
29+
OUT min_exec_time float8,
30+
OUT max_exec_time float8,
31+
OUT mean_exec_time float8,
32+
OUT stddev_exec_time float8,
33+
OUT rows int8,
34+
OUT shared_blks_hit int8,
35+
OUT shared_blks_read int8,
36+
OUT shared_blks_dirtied int8,
37+
OUT shared_blks_written int8,
38+
OUT local_blks_hit int8,
39+
OUT local_blks_read int8,
40+
OUT local_blks_dirtied int8,
41+
OUT local_blks_written int8,
42+
OUT temp_blks_read int8,
43+
OUT temp_blks_written int8,
44+
OUT blk_read_time float8,
45+
OUT blk_write_time float8,
46+
OUT wal_records int8,
47+
OUT wal_fpi int8,
48+
OUT wal_bytes numeric
49+
)
50+
RETURNS SETOF record
51+
AS 'MODULE_PATHNAME', 'pg_stat_statements_1_10'
52+
LANGUAGE C STRICT VOLATILE PARALLEL SAFE;
53+
54+
CREATE VIEW pg_stat_statements AS
55+
SELECT * FROM pg_stat_statements(true);
56+
57+
GRANT SELECT ON pg_stat_statements TO PUBLIC;

‎contrib/pg_stat_statements/pg_stat_statements.c

Copy file name to clipboardExpand all lines: contrib/pg_stat_statements/pg_stat_statements.c
+43-7Lines changed: 43 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -87,7 +87,7 @@ PG_MODULE_MAGIC;
8787
#define PGSS_TEXT_FILE PG_STAT_TMP_DIR "/pgss_query_texts.stat"
8888

8989
/* Magic number identifying the stats file format */
90-
static const uint32 PGSS_FILE_HEADER = 0x20201218;
90+
static const uint32 PGSS_FILE_HEADER = 0x20201227;
9191

9292
/* PostgreSQL major version number, changes in which invalidate all entries */
9393
static const uint32 PGSS_PG_MAJOR_VERSION = PG_VERSION_NUM / 100;
@@ -119,7 +119,8 @@ typedef enum pgssVersion
119119
PGSS_V1_1,
120120
PGSS_V1_2,
121121
PGSS_V1_3,
122-
PGSS_V1_8
122+
PGSS_V1_8,
123+
PGSS_V1_10
123124
} pgssVersion;
124125

125126
typedef enum pgssStoreKind
@@ -141,16 +142,17 @@ typedef enum pgssStoreKind
141142
* Hashtable key that defines the identity of a hashtable entry. We separate
142143
* queries by user and by database even if they are otherwise identical.
143144
*
144-
* Right now, this structure contains no padding. If you add any, make sure
145-
* to teach pgss_store() to zero the padding bytes. Otherwise, things will
146-
* break, because pgss_hash is created using HASH_BLOBS, and thus tag_hash
147-
* is used to hash this.
145+
* If you add a new key to this struct, make sure to teach pgss_store() to
146+
* zero the padding bytes. Otherwise, things will break, because pgss_hash is
147+
* created using HASH_BLOBS, and thus tag_hash is used to hash this.
148+
148149
*/
149150
typedef struct pgssHashKey
150151
{
151152
Oid userid; /* user OID */
152153
Oid dbid; /* database OID */
153154
uint64 queryid; /* query identifier */
155+
bool toplevel; /* query executed at top level */
154156
} pgssHashKey;
155157

156158
/*
@@ -297,6 +299,7 @@ PG_FUNCTION_INFO_V1(pg_stat_statements_reset_1_7);
297299
PG_FUNCTION_INFO_V1(pg_stat_statements_1_2);
298300
PG_FUNCTION_INFO_V1(pg_stat_statements_1_3);
299301
PG_FUNCTION_INFO_V1(pg_stat_statements_1_8);
302+
PG_FUNCTION_INFO_V1(pg_stat_statements_1_10);
300303
PG_FUNCTION_INFO_V1(pg_stat_statements);
301304
PG_FUNCTION_INFO_V1(pg_stat_statements_info);
302305

@@ -1224,9 +1227,14 @@ pgss_store(const char *query, uint64 queryId,
12241227
query = CleanQuerytext(query, &query_location, &query_len);
12251228

12261229
/* Set up key for hashtable search */
1230+
1231+
/* memset() is required when pgssHashKey is without padding only */
1232+
memset(&key, 0, sizeof(pgssHashKey));
1233+
12271234
key.userid = GetUserId();
12281235
key.dbid = MyDatabaseId;
12291236
key.queryid = queryId;
1237+
key.toplevel = (exec_nested_level == 0);
12301238

12311239
/* Lookup the hash table entry with shared lock. */
12321240
LWLockAcquire(pgss->lock, LW_SHARED);
@@ -1406,7 +1414,8 @@ pg_stat_statements_reset(PG_FUNCTION_ARGS)
14061414
#define PG_STAT_STATEMENTS_COLS_V1_2 19
14071415
#define PG_STAT_STATEMENTS_COLS_V1_3 23
14081416
#define PG_STAT_STATEMENTS_COLS_V1_8 32
1409-
#define PG_STAT_STATEMENTS_COLS 32 /* maximum of above */
1417+
#define PG_STAT_STATEMENTS_COLS_V1_10 33
1418+
#define PG_STAT_STATEMENTS_COLS 33 /* maximum of above */
14101419

14111420
/*
14121421
* Retrieve statement statistics.
@@ -1418,6 +1427,16 @@ pg_stat_statements_reset(PG_FUNCTION_ARGS)
14181427
* expected API version is identified by embedding it in the C name of the
14191428
* function. Unfortunately we weren't bright enough to do that for 1.1.
14201429
*/
1430+
Datum
1431+
pg_stat_statements_1_10(PG_FUNCTION_ARGS)
1432+
{
1433+
bool showtext = PG_GETARG_BOOL(0);
1434+
1435+
pg_stat_statements_internal(fcinfo, PGSS_V1_10, showtext);
1436+
1437+
return (Datum) 0;
1438+
}
1439+
14211440
Datum
14221441
pg_stat_statements_1_8(PG_FUNCTION_ARGS)
14231442
{
@@ -1537,6 +1556,10 @@ pg_stat_statements_internal(FunctionCallInfo fcinfo,
15371556
if (api_version != PGSS_V1_8)
15381557
elog(ERROR, "incorrect number of output arguments");
15391558
break;
1559+
case PG_STAT_STATEMENTS_COLS_V1_10:
1560+
if (api_version != PGSS_V1_10)
1561+
elog(ERROR, "incorrect number of output arguments");
1562+
break;
15401563
default:
15411564
elog(ERROR, "incorrect number of output arguments");
15421565
}
@@ -1628,6 +1651,8 @@ pg_stat_statements_internal(FunctionCallInfo fcinfo,
16281651

16291652
values[i++] = ObjectIdGetDatum(entry->key.userid);
16301653
values[i++] = ObjectIdGetDatum(entry->key.dbid);
1654+
if (api_version >= PGSS_V1_10)
1655+
values[i++] = BoolGetDatum(entry->key.toplevel);
16311656

16321657
if (is_allowed_role || entry->key.userid == userid)
16331658
{
@@ -1765,6 +1790,7 @@ pg_stat_statements_internal(FunctionCallInfo fcinfo,
17651790
api_version == PGSS_V1_2 ? PG_STAT_STATEMENTS_COLS_V1_2 :
17661791
api_version == PGSS_V1_3 ? PG_STAT_STATEMENTS_COLS_V1_3 :
17671792
api_version == PGSS_V1_8 ? PG_STAT_STATEMENTS_COLS_V1_8 :
1793+
api_version == PGSS_V1_10 ? PG_STAT_STATEMENTS_COLS_V1_10 :
17681794
-1 /* fail if you forget to update this assert */ ));
17691795

17701796
tuplestore_putvalues(tupstore, tupdesc, values, nulls);
@@ -2437,10 +2463,20 @@ entry_reset(Oid userid, Oid dbid, uint64 queryid)
24372463
if (userid != 0 && dbid != 0 && queryid != UINT64CONST(0))
24382464
{
24392465
/* If all the parameters are available, use the fast path. */
2466+
memset(&key, 0, sizeof(pgssHashKey));
24402467
key.userid = userid;
24412468
key.dbid = dbid;
24422469
key.queryid = queryid;
24432470

2471+
/* Remove the key if it exists, starting with the top-level entry */
2472+
key.toplevel = false;
2473+
entry = (pgssEntry *) hash_search(pgss_hash, &key, HASH_REMOVE, NULL);
2474+
if (entry) /* found */
2475+
num_remove++;
2476+
2477+
/* Also remove entries for top level statements */
2478+
key.toplevel = true;
2479+
24442480
/* Remove the key if exists */
24452481
entry = (pgssEntry *) hash_search(pgss_hash, &key, HASH_REMOVE, NULL);
24462482
if (entry) /* found */

‎contrib/pg_stat_statements/pg_stat_statements.control

Copy file name to clipboard
+1-1Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
# pg_stat_statements extension
22
comment = 'track planning and execution statistics of all SQL statements executed'
3-
default_version = '1.9'
3+
default_version = '1.10'
44
module_pathname = '$libdir/pg_stat_statements'
55
relocatable = true

‎contrib/pg_stat_statements/sql/pg_stat_statements.sql

Copy file name to clipboardExpand all lines: contrib/pg_stat_statements/sql/pg_stat_statements.sql
+21Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -364,4 +364,25 @@ SELECT query, plans, calls, rows FROM pg_stat_statements ORDER BY query COLLATE
364364
SELECT pg_stat_statements_reset();
365365
SELECT dealloc FROM pg_stat_statements_info;
366366

367+
--
368+
-- top level handling
369+
--
370+
SET pg_stat_statements.track = 'top';
371+
DELETE FROM test;
372+
DO $$
373+
BEGIN
374+
DELETE FROM test;
375+
END;
376+
$$ LANGUAGE plpgsql;
377+
SELECT query, toplevel, plans, calls FROM pg_stat_statements WHERE query LIKE '%DELETE%' ORDER BY query COLLATE "C", toplevel;
378+
379+
SET pg_stat_statements.track = 'all';
380+
DELETE FROM test;
381+
DO $$
382+
BEGIN
383+
DELETE FROM test;
384+
END;
385+
$$ LANGUAGE plpgsql;
386+
SELECT query, toplevel, plans, calls FROM pg_stat_statements WHERE query LIKE '%DELETE%' ORDER BY query COLLATE "C", toplevel;
387+
367388
DROP EXTENSION pg_stat_statements;

‎doc/src/sgml/pgstatstatements.sgml

Copy file name to clipboardExpand all lines: doc/src/sgml/pgstatstatements.sgml
+9Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -87,6 +87,15 @@
8787
</para></entry>
8888
</row>
8989

90+
<row>
91+
<entry role="catalog_table_entry"><para role="column_definition">
92+
<structfield>toplevel</structfield> <type>bool</type>
93+
</para>
94+
<para>
95+
True if the query was executed as a top level statement
96+
</para></entry>
97+
</row>
98+
9099
<row>
91100
<entry role="catalog_table_entry"><para role="column_definition">
92101
<structfield>queryid</structfield> <type>bigint</type>

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.