Skip to content

Navigation Menu

Sign in

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit 28e626b

Browse filesBrowse files
anarazelanarazel
committed
pgstat: Infrastructure for more detailed IO statistics
This commit adds the infrastructure for more detailed IO statistics. The calls to actually count IOs, a system view to access the new statistics, documentation and tests will be added in subsequent commits, to make review easier. While we already had some IO statistics, e.g. in pg_stat_bgwriter and pg_stat_database, they did not provide sufficient detail to understand what the main sources of IO are, or whether configuration changes could avoid IO. E.g., pg_stat_bgwriter.buffers_backend does contain the number of buffers written out by a backend, but as that includes extending relations (always done by backends) and writes triggered by the use of buffer access strategies, it cannot easily be used to tune background writer or checkpointer. Similarly, pg_stat_database.blks_read cannot easily be used to tune shared_buffers / compute a cache hit ratio, as the use of buffer access strategies will often prevent a large fraction of the read blocks to end up in shared_buffers. The new IO statistics count IO operations (evict, extend, fsync, read, reuse, and write), and are aggregated for each combination of backend type (backend, autovacuum worker, bgwriter, etc), target object of the IO (relations, temp relations) and context of the IO (normal, vacuum, bulkread, bulkwrite). What is tracked in this series of patches, is sufficient to perform the aforementioned analyses. Further details, e.g. tracking the number of buffer hits, would make that even easier, but was left out for now, to keep the scope of the already large patchset manageable. Bumps PGSTAT_FILE_FORMAT_ID. Author: Melanie Plageman <melanieplageman@gmail.com> Reviewed-by: Andres Freund <andres@anarazel.de> Reviewed-by: Justin Pryzby <pryzby@telsasoft.com> Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com> Discussion: https://postgr.es/m/20200124195226.lth52iydq2n2uilq@alap3.anarazel.de
1 parent 49c2c5f commit 28e626b
Copy full SHA for 28e626b

15 files changed

+569
-8
lines changed

‎doc/src/sgml/monitoring.sgml

Copy file name to clipboardExpand all lines: doc/src/sgml/monitoring.sgml
+2Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5444,6 +5444,8 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i
54445444
the <structname>pg_stat_bgwriter</structname>
54455445
view, <literal>archiver</literal> to reset all the counters shown in
54465446
the <structname>pg_stat_archiver</structname> view,
5447+
<literal>io</literal> to reset all the counters shown in the
5448+
<structname>pg_stat_io</structname> view,
54475449
<literal>wal</literal> to reset all the counters shown in the
54485450
<structname>pg_stat_wal</structname> view or
54495451
<literal>recovery_prefetch</literal> to reset all the counters shown

‎src/backend/utils/activity/Makefile

Copy file name to clipboardExpand all lines: src/backend/utils/activity/Makefile
+1Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ OBJS = \
2222
pgstat_checkpointer.o \
2323
pgstat_database.o \
2424
pgstat_function.o \
25+
pgstat_io.o \
2526
pgstat_relation.o \
2627
pgstat_replslot.o \
2728
pgstat_shmem.o \

‎src/backend/utils/activity/meson.build

Copy file name to clipboardExpand all lines: src/backend/utils/activity/meson.build
+1Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ backend_sources += files(
99
'pgstat_checkpointer.c',
1010
'pgstat_database.c',
1111
'pgstat_function.c',
12+
'pgstat_io.c',
1213
'pgstat_relation.c',
1314
'pgstat_replslot.c',
1415
'pgstat_shmem.c',

‎src/backend/utils/activity/pgstat.c

Copy file name to clipboardExpand all lines: src/backend/utils/activity/pgstat.c
+26Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -72,6 +72,7 @@
7272
* - pgstat_checkpointer.c
7373
* - pgstat_database.c
7474
* - pgstat_function.c
75+
* - pgstat_io.c
7576
* - pgstat_relation.c
7677
* - pgstat_replslot.c
7778
* - pgstat_slru.c
@@ -359,6 +360,15 @@ static const PgStat_KindInfo pgstat_kind_infos[PGSTAT_NUM_KINDS] = {
359360
.snapshot_cb = pgstat_checkpointer_snapshot_cb,
360361
},
361362

363+
[PGSTAT_KIND_IO] = {
364+
.name = "io",
365+
366+
.fixed_amount = true,
367+
368+
.reset_all_cb = pgstat_io_reset_all_cb,
369+
.snapshot_cb = pgstat_io_snapshot_cb,
370+
},
371+
362372
[PGSTAT_KIND_SLRU] = {
363373
.name = "slru",
364374

@@ -582,6 +592,7 @@ pgstat_report_stat(bool force)
582592

583593
/* Don't expend a clock check if nothing to do */
584594
if (dlist_is_empty(&pgStatPending) &&
595+
!have_iostats &&
585596
!have_slrustats &&
586597
!pgstat_have_pending_wal())
587598
{
@@ -628,6 +639,9 @@ pgstat_report_stat(bool force)
628639
/* flush database / relation / function / ... stats */
629640
partial_flush |= pgstat_flush_pending_entries(nowait);
630641

642+
/* flush IO stats */
643+
partial_flush |= pgstat_flush_io(nowait);
644+
631645
/* flush wal stats */
632646
partial_flush |= pgstat_flush_wal(nowait);
633647

@@ -1322,6 +1336,12 @@ pgstat_write_statsfile(void)
13221336
pgstat_build_snapshot_fixed(PGSTAT_KIND_CHECKPOINTER);
13231337
write_chunk_s(fpout, &pgStatLocal.snapshot.checkpointer);
13241338

1339+
/*
1340+
* Write IO stats struct
1341+
*/
1342+
pgstat_build_snapshot_fixed(PGSTAT_KIND_IO);
1343+
write_chunk_s(fpout, &pgStatLocal.snapshot.io);
1344+
13251345
/*
13261346
* Write SLRU stats struct
13271347
*/
@@ -1496,6 +1516,12 @@ pgstat_read_statsfile(void)
14961516
if (!read_chunk_s(fpin, &shmem->checkpointer.stats))
14971517
goto error;
14981518

1519+
/*
1520+
* Read IO stats struct
1521+
*/
1522+
if (!read_chunk_s(fpin, &shmem->io.stats))
1523+
goto error;
1524+
14991525
/*
15001526
* Read SLRU stats struct
15011527
*/

‎src/backend/utils/activity/pgstat_bgwriter.c

Copy file name to clipboardExpand all lines: src/backend/utils/activity/pgstat_bgwriter.c
+6-1Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ PgStat_BgWriterStats PendingBgWriterStats = {0};
2424

2525

2626
/*
27-
* Report bgwriter statistics
27+
* Report bgwriter and IO statistics
2828
*/
2929
void
3030
pgstat_report_bgwriter(void)
@@ -56,6 +56,11 @@ pgstat_report_bgwriter(void)
5656
* Clear out the statistics buffer, so it can be re-used.
5757
*/
5858
MemSet(&PendingBgWriterStats, 0, sizeof(PendingBgWriterStats));
59+
60+
/*
61+
* Report IO statistics
62+
*/
63+
pgstat_flush_io(false);
5964
}
6065

6166
/*

‎src/backend/utils/activity/pgstat_checkpointer.c

Copy file name to clipboardExpand all lines: src/backend/utils/activity/pgstat_checkpointer.c
+6-1Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ PgStat_CheckpointerStats PendingCheckpointerStats = {0};
2424

2525

2626
/*
27-
* Report checkpointer statistics
27+
* Report checkpointer and IO statistics
2828
*/
2929
void
3030
pgstat_report_checkpointer(void)
@@ -62,6 +62,11 @@ pgstat_report_checkpointer(void)
6262
* Clear out the statistics buffer, so it can be re-used.
6363
*/
6464
MemSet(&PendingCheckpointerStats, 0, sizeof(PendingCheckpointerStats));
65+
66+
/*
67+
* Report IO statistics
68+
*/
69+
pgstat_flush_io(false);
6570
}
6671

6772
/*

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.