Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit 360392f

Browse filesBrowse files
author
Amit Kapila
committed
Avoid unconditionally filling in missing values with NULL in pgoutput.
52e4f0c introduced a bug in pgoutput in which missing values in tuples were incorrectly filled in with NULL. The problem was the use of CreateTupleDescCopy where CreateTupleDescCopyConstr was required, as the former drops the constraints in the tuple description (specifically, the default value constraint) on the floor. The bug could result in incorrectness when a table replicated via `REPLICA IDENTITY FULL` underwent a schema change that added a column with a default value. The problem is that in such cases updates fill NULL values in old tuples for missing columns for default values. Then on the subscriber, we failed to find a matching tuple and missed updating the required row. Author: Nikhil Benesch Reviewed-by: Hou Zhijie, Amit Kapila Backpatch-through: 15 Discussion: http://postgr.es/m/CAPWqQZTEpZQamYsGMn6ZDRvVywwpVPiKH6OY4KSgA+NmeqFNzA@mail.gmail.com
1 parent dc9f8a7 commit 360392f
Copy full SHA for 360392f

File tree

Expand file treeCollapse file tree

2 files changed

+55
-2
lines changed
Filter options
Expand file treeCollapse file tree

2 files changed

+55
-2
lines changed

‎src/backend/replication/pgoutput/pgoutput.c

Copy file name to clipboardExpand all lines: src/backend/replication/pgoutput/pgoutput.c
+2-2Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1136,8 +1136,8 @@ init_tuple_slot(PGOutputData *data, Relation relation,
11361136
* Create tuple table slots. Create a copy of the TupleDesc as it needs to
11371137
* live as long as the cache remains.
11381138
*/
1139-
oldtupdesc = CreateTupleDescCopy(RelationGetDescr(relation));
1140-
newtupdesc = CreateTupleDescCopy(RelationGetDescr(relation));
1139+
oldtupdesc = CreateTupleDescCopyConstr(RelationGetDescr(relation));
1140+
newtupdesc = CreateTupleDescCopyConstr(RelationGetDescr(relation));
11411141

11421142
entry->old_slot = MakeSingleTupleTableSlot(oldtupdesc, &TTSOpsHeapTuple);
11431143
entry->new_slot = MakeSingleTupleTableSlot(newtupdesc, &TTSOpsHeapTuple);

‎src/test/subscription/t/100_bugs.pl

Copy file name to clipboardExpand all lines: src/test/subscription/t/100_bugs.pl
+53Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -438,4 +438,57 @@
438438
$node_publisher->stop('fast');
439439
$node_subscriber->stop('fast');
440440

441+
# The bug was that pgoutput was incorrectly replacing missing attributes in
442+
# tuples with NULL. This could result in incorrect replication with
443+
# `REPLICA IDENTITY FULL`.
444+
445+
$node_publisher->rotate_logfile();
446+
$node_publisher->start();
447+
448+
$node_subscriber->rotate_logfile();
449+
$node_subscriber->start();
450+
451+
# Set up a table with schema `(a int, b bool)` where the `b` attribute is
452+
# missing for one row due to the `ALTER TABLE ... ADD COLUMN ... DEFAULT`
453+
# fast path.
454+
$node_publisher->safe_psql(
455+
'postgres', qq(
456+
CREATE TABLE tab_default (a int);
457+
ALTER TABLE tab_default REPLICA IDENTITY FULL;
458+
INSERT INTO tab_default VALUES (1);
459+
ALTER TABLE tab_default ADD COLUMN b bool DEFAULT false NOT NULL;
460+
INSERT INTO tab_default VALUES (2, true);
461+
CREATE PUBLICATION pub1 FOR TABLE tab_default;
462+
));
463+
464+
# Replicate to the subscriber.
465+
$node_subscriber->safe_psql(
466+
'postgres', qq(
467+
CREATE TABLE tab_default (a int, b bool);
468+
CREATE SUBSCRIPTION sub1 CONNECTION '$publisher_connstr' PUBLICATION pub1;
469+
));
470+
471+
$node_subscriber->wait_for_subscription_sync($node_publisher, 'sub1');
472+
$result = $node_subscriber->safe_psql('postgres',
473+
"SELECT a, b FROM tab_default");
474+
is($result, qq(1|f
475+
2|t), 'check snapshot on subscriber');
476+
477+
# Update all rows in the table and ensure the rows with the missing `b`
478+
# attribute replicate correctly.
479+
$node_publisher->safe_psql('postgres',
480+
"UPDATE tab_default SET a = a + 1");
481+
$node_publisher->wait_for_catchup('sub1');
482+
483+
# When the bug is present, the `1|f` row will not be updated to `2|f` because
484+
# the publisher incorrectly fills in `NULL` for `b` and publishes an update
485+
# for `1|NULL`, which doesn't exist in the subscriber.
486+
$result = $node_subscriber->safe_psql('postgres',
487+
"SELECT a, b FROM tab_default");
488+
is($result, qq(2|f
489+
3|t), 'check replicated update on subscriber');
490+
491+
$node_publisher->stop('fast');
492+
$node_subscriber->stop('fast');
493+
441494
done_testing();

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.