Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit 64f34eb

Browse filesBrowse files
nathan-bossartnathan-bossart
committed
Use CREATE DATABASE ... STRATEGY = FILE_COPY in pg_upgrade.
While this strategy is ordinarily quite costly because it requires performing two checkpoints, testing shows that it tends to be a faster choice than WAL_LOG during pg_upgrade, presumably because fsync is turned off. Furthermore, we can skip the checkpoints altogether because the problems they are intended to prevent don't apply to pg_upgrade. Instead, we just need to CHECKPOINT once in the new cluster after making any changes to template0 and before restoring the rest of the databases. This ensures that said template0 changes are written out to disk prior to creating the databases via FILE_COPY. Co-authored-by: Matthias van de Meent Reviewed-by: Ranier Vilela, Dilip Kumar, Robert Haas, Michael Paquier Discussion: https://postgr.es/m/Zl9ta3FtgdjizkJ5%40nathan
1 parent 4b4b931 commit 64f34eb
Copy full SHA for 64f34eb

File tree

Expand file treeCollapse file tree

3 files changed

+34
-4
lines changed
Filter options
Expand file treeCollapse file tree

3 files changed

+34
-4
lines changed

‎src/backend/commands/dbcommands.c

Copy file name to clipboardExpand all lines: src/backend/commands/dbcommands.c
+15-3Lines changed: 15 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -563,9 +563,14 @@ CreateDatabaseUsingFileCopy(Oid src_dboid, Oid dst_dboid, Oid src_tsid,
563563
* happened while we're copying files, a file might be deleted just when
564564
* we're about to copy it, causing the lstat() call in copydir() to fail
565565
* with ENOENT.
566+
*
567+
* In binary upgrade mode, we can skip this checkpoint because pg_upgrade
568+
* is careful to ensure that template0 is fully written to disk prior to
569+
* any CREATE DATABASE commands.
566570
*/
567-
RequestCheckpoint(CHECKPOINT_IMMEDIATE | CHECKPOINT_FORCE |
568-
CHECKPOINT_WAIT | CHECKPOINT_FLUSH_ALL);
571+
if (!IsBinaryUpgrade)
572+
RequestCheckpoint(CHECKPOINT_IMMEDIATE | CHECKPOINT_FORCE |
573+
CHECKPOINT_WAIT | CHECKPOINT_FLUSH_ALL);
569574

570575
/*
571576
* Iterate through all tablespaces of the template database, and copy each
@@ -657,10 +662,17 @@ CreateDatabaseUsingFileCopy(Oid src_dboid, Oid dst_dboid, Oid src_tsid,
657662
* seem to be much we can do about that except document it as a
658663
* limitation.
659664
*
665+
* In binary upgrade mode, we can skip this checkpoint because neither of
666+
* these problems applies: we don't ever replay the WAL generated during
667+
* pg_upgrade, and we don't support taking base backups during pg_upgrade
668+
* (not to mention that we don't concurrently modify template0, either).
669+
*
660670
* See CreateDatabaseUsingWalLog() for a less cheesy CREATE DATABASE
661671
* strategy that avoids these problems.
662672
*/
663-
RequestCheckpoint(CHECKPOINT_IMMEDIATE | CHECKPOINT_FORCE | CHECKPOINT_WAIT);
673+
if (!IsBinaryUpgrade)
674+
RequestCheckpoint(CHECKPOINT_IMMEDIATE | CHECKPOINT_FORCE |
675+
CHECKPOINT_WAIT);
664676
}
665677

666678
/*

‎src/bin/pg_dump/pg_dump.c

Copy file name to clipboardExpand all lines: src/bin/pg_dump/pg_dump.c
+7-1Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3144,10 +3144,16 @@ dumpDatabase(Archive *fout)
31443144
* since those can't be altered later. Other DB properties are left to
31453145
* the DATABASE PROPERTIES entry, so that they can be applied after
31463146
* reconnecting to the target DB.
3147+
*
3148+
* For binary upgrade, we use the FILE_COPY strategy because testing has
3149+
* shown it to be faster. When the server is in binary upgrade mode, it
3150+
* will also skip the checkpoints this strategy ordinarily performs.
31473151
*/
31483152
if (dopt->binary_upgrade)
31493153
{
3150-
appendPQExpBuffer(creaQry, "CREATE DATABASE %s WITH TEMPLATE = template0 OID = %u",
3154+
appendPQExpBuffer(creaQry,
3155+
"CREATE DATABASE %s WITH TEMPLATE = template0 "
3156+
"OID = %u STRATEGY = FILE_COPY",
31513157
qdatname, dbCatId.oid);
31523158
}
31533159
else

‎src/bin/pg_upgrade/pg_upgrade.c

Copy file name to clipboardExpand all lines: src/bin/pg_upgrade/pg_upgrade.c
+12Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -534,9 +534,21 @@ static void
534534
create_new_objects(void)
535535
{
536536
int dbnum;
537+
PGconn *conn_new_template1;
537538

538539
prep_status_progress("Restoring database schemas in the new cluster");
539540

541+
/*
542+
* Ensure that any changes to template0 are fully written out to disk
543+
* prior to restoring the databases. This is necessary because we use the
544+
* FILE_COPY strategy to create the databases (which testing has shown to
545+
* be faster), and when the server is in binary upgrade mode, it skips the
546+
* checkpoints this strategy ordinarily performs.
547+
*/
548+
conn_new_template1 = connectToServer(&new_cluster, "template1");
549+
PQclear(executeQueryOrDie(conn_new_template1, "CHECKPOINT"));
550+
PQfinish(conn_new_template1);
551+
540552
/*
541553
* We cannot process the template1 database concurrently with others,
542554
* because when it's transiently dropped, connection attempts would fail.

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.